Archive for the 'Web Development' Category

Problem Solved

Thursday, August 4th, 2011

For the past couple of weeks, Evan and I have been trying to figure out why StreetsblogSF sometimes takes a long time to load.  I wrote up a post about our work here.  I’m pleased to announce that we have found the solution to this.

First we activated the slow query log on our Mysql database.  It showed us the queries that were taking a long time to run.  Here was the worst of them:

SELECT SQL_CALC_FOUND_ROWS  wp_posts.* FROM wp_posts  LEFT JOIN wp_ec3_schedule ec3_sch ON ec3_sch.post_id=id  WHERE 1=1  AND wp_posts.post_type = ‘post’ AND (wp_posts.post_status
= ‘publish’) AND ec3_sch.post_id IS NULL  GROUP BY wp_posts.ID ORDER BY wp_posts.post_date DESC LIMIT 3030, 10;

This took 12 seconds to run on average.  When it was being run, any other database query would be stalled.


Unsolvable Problems

Tuesday, July 26th, 2011

Occasionally I come across problems which can’t seem to be fixed.  Often there’s a way to workaround the problem so that it doesn’t have to be solved.  Sometimes, though, I just have to leave it unsolved and put up with the imperfection.  Here is a great example.

StreetsblogSF runs using the same software (WordPress/lighttpd) and on the same hardware as Streetsblog NYC, DC, and LA.  Unlike the others, however, it occasionally takes a long time, up to a minute, to load in the browser.  Go to it now and see if this happens for you.  There doesn’t seem to be any pattern to when these long loads occur.  Most of the time the site loads in just a few seconds.

I’m not sure why StreetsblogSF behaves like this.  It is setup and configured the same as the other sites, has the same webserver settings, installed plugins etc.  I checked the server logs and there are no reports of anything unusual.  I watched the site load in firebug but got no additional information. (more…)

How People Consume our Content

Friday, June 17th, 2011

How do people consume the content we create on our blogs?  That is the question I’d like to examine in this post.

In the charts below, I compare the various ways that people consume our content for each of our blogs during the past month. People can come directly to our sites on their desktop browsers.  They can come on their mobile devices, by hitting our mobile themes.  They can subscribe to our RSS feeds using Google Reader or a similar RSS reader.  And they can also subscribe to our email feeds which delivers a daily digest email to their inboxes.


Universal Subtitles comes to Streetfilms

Wednesday, June 8th, 2011

Over the last years, Streetfilms has become a resource for people all over the world who are interested in improving the street life of their cities. People have also started to translate the films and create new videos with subtitles for their native audience. Here’s an example of a Streetfilms video translated for Streets of Iran. This is all very exciting for the Streetfilms team!

Today, in an effort to facilitate this process, we’ve released an update to which incorporates Universal Subtitles into the site. Universal Subtitles is a system in which any film can have subtitles added to it and displayed in any language. People can play the film with subtitles by choosing any of the available translations, or even with English subtitles which is helpful for the hearing impaired.

The real genius of Universal Subtitles is that translations are crowd-sourced, that is, anyone can add a translation. Once it is added, it is available immediately on the site for anyone to use. Hopefully, over time, Streetfilms will grow a large repository of subtitles for many of our films. (more…)

The Growth of Mobile

Friday, May 13th, 2011

Earlier this year I acknowledged the growing trend of mobile browsers and soon after rolled out an improved mobile theme to GothamSchools.  This week I deployed the same mobile platform to and the other city Streetsblogs.  See the new theme below:

As with GothamSchools, this new theme has many advanced features such as reformatting images on the fly to fit a particular display and integrated Share buttons. (more…)

Sharing Usage Analysis

Tuesday, April 26th, 2011

A few weeks ago I upgraded our sharing infrastructure on our various Streetsblogs to use the ShareThis service.  Previously we had a custom set of icons that were not integrated into a sharing service.

I made this upgrade partly to make it more obvious to people how to share posts and in so doing promote sharing and expand the reach of our content.  The old buttons were in the sidebar and the new ones are larger and directly below the post.  I decided to have just the twitter, facebook, and email buttons due to data from our ShareThis widget on GothamSchools which shows that these are the most popular channels.

Another benefit of going with a service like ShareThis is that it provides analytics that tell us which posts are the most shared, which channels are most popular, etc.  See below: (more…)

Disqus Usage Analysis

Tuesday, April 19th, 2011

A couple of weeks ago we deployed the Disqus commenting system across our blogs.  Read more about that here.  Now I want to analyze how the communities have engaged the various new features offered by Disqus.

We are currently running on free Disqus accounts.  If we were to upgrade to a Plus account, we would get access to some built-in analytics.  In their absence, I did this analysis by hand, by grepping through the source of Disqus moderation screens.

First, I looked at how users were authenticating when they place a comment.

blog anonymous disqus google twitter facebook yahoo 88% 8.5% 0 3% 0.5% 0 60% 33% 2% 3% 0.5% 1.5% 60% 26% 5% 4% 2% 3% 52.5% 21% 6% 14.5% 5.5% 0.5% 61% 37% 0% 2% 0% 0% 42% 38% 12% 0% 6% 2% 55% 25% 10% 6% 3% 1%

The anonymous comment rate is the percentage of comments that are made anonymously as opposed to by people who authenticate with Google, Twitter, Facebook, Yahoo, or Disqus.  This indicates something about how much people are willing to expose of themselves.

It makes sense that people at are the ones who are most willing to authenticate and reveal their true identities.  This blog is intended to be read by the most dedicated members of the Livable Streets initiative.  GothamSchools, on the other extreme, is a public journalism site that doesn’t have a strong bias so naturally has the most commenters who do not wish to reveal their identities.

I then looked at how much people were using the “Likes” button and how often people are writing a reply to a comment.

blog Likes / 100 comments Replies / 100 comments 39 45 34 39 66 29 55 34 84 58 72 34 48 30

It’s good to see here that the new Disqus features are being well used.

The Great Disqus Deployment

Friday, April 1st, 2011

Several weeks ago I deployed the Disqus commenting system on  The editors there are happy with this new system.  So this week I deployed Disqus on StreetsblogNYC, StreetsblogLA, StreetsblogDC, StreetsblogSF, and Streetfilms.

Technically, it was a pretty straightforward process. I had to strip out our old commenting system on these sites, which integrates with, and then install and style the Disqus system.  This will allow us to shut down for good, something we’ve been wanting to do for some time.

I did run into some problems exporting our existing comments to Disqus.  For some reason, the built-in exporter kept hitting errors and failing.  The Disqus engineers were responsive to my complaints and said that they’re building a command-line tool for large exports to be in the next release. (more…)

Measuring Community Activity on a Blog

Thursday, March 17th, 2011

I’ve been thinking about how to measure community activity on our blogs.  This would be useful so that we can judge our effectiveness over time at enabling conversation and participation among the communities we are trying to serve.

People interact with our blogs at varying depths.  It would be interesting to measure statistics at each level.  First, people simply navigate to a single blog posts, then they might browse other blog posts, then they might “like” a comment or leave a comment, or several comments, and finally they may share the blog post on a social network via one of our sharing widgets.

Looking at the statistics for each of these levels would show us how shallow or deep typical blog visitors go in interacting with the blog.

For example, on GothamSchools in Feb 2011 we can gather these statistics:

127533 visits to the blog
77795 visits viewed more than one page
1449 comments were left by
390 commenters
117 shares to social networks using the sharethis widget

You can see how activity dropped off at deeper levels of participation. (more…)

Developing WordPress Mobile Themes

Wednesday, March 16th, 2011

Here at OpenPlans, we decided we wanted to upgrade our mobile presence for a few of our blogs.  The number of smartphones that were shipped in the fourth quarter of 2010 exceeded the number of personal computers shipped, so we see our mobile presence becoming increasingly important.

Choosing a Mobile Plugin

The GothamSchools mobile theme shown in the Firefox browser.

The first step was to survey the various mobile plugins for WordPress.  I narrowed down the choices to WPTouch and WP Mobile Pack.  I ended up choosing WPTouch because it is compatible with the Disqus commenting system.  We recently launched Disqus on GothamSchools and plan on doing so soon on our other blogs.

The advantages WPTouch offers over our previous mobile theme, are:

  • it has a menu that gives access to all parts of the site and search
  • it has share buttons that will increase the viral nature of blog posts
  • it reformats images, videos etc. on the fly to be compatible with the specific mobile device
  • it has wizzy AJAX features, like auto-loading of more posts in the post listing
  • it is under active development and has a large community so is a good bet for future releases and ongoing support
  • it is compatible with wp-super-cache

I decided to buy the WPTouch Pro version of WPTouch instead of using the free version.  The Pro version is quite a bit more sophisticated than the free version and promises future upgrades and support.

Customizing the Mobile Theme

I then set out to customize the default WPTouch theme for  I created a child theme of the default WPTouch theme in which I added some styles and slightly changed the markup in the index and archive templates.  Many other settings were easily customized using the WPTouch admin panel.  Because I made only a few changes to the template code, the new Gotham mobile theme will hopefully remain compatible with future upgrades to the WPTouch Pro plugin and its default theme. (more…)