Community Live: Hack The Government Day

by Wendy M. Grossman

More than 100 people showed up for Saturday's Rewired State, held at the Guardian's Kings Cross offices (and to meetings in Brighton and Manchester), to prove that they could do a better job than the government of making government information usable by the public.

Organisers James Darling, Emma Mulqueeny, and Richard Pope are still filling in the list of the day's projects. But after the dozens of two-minute presentations at day's end even Brian Hoadley, who leads Directgov's Innovations site, admitted that the "Hack the government" coders won hands down.

The goal was simple: showing what could be done if the government were providing structured data and open APIs instead of PDFs, Excel files, and the kinds of centuries-old lists that civil servants are used to. Screen scraping for the public benefit has been a British tradition ever since Stefan Magdalinski co-founded the original UpMyStreet in 1998. Since then, he and various groupings of volunteers have gone on to create sites such as Fax Your MP and TheyWorkForYou, and set up My Society to do more of the same. Other related sites include Farmsubsidy.org, which tracks the distribution of agricultural subsidies in the EU, and UNDemocracy, which turns the United Nations' debates into a searchable, linkable archive.

Community Live - Hack The Government Day

Slideshow, 12 images

There's an irony here. As anyone who's come up against MPs in a trivia quiz knows, politicians are experts on using TheyWorkForYou.com. Yet that expertise has not translated into better public information sites. Given the shortness of the time available – developers were limited to what they could do on Saturday – some of the projects weren't finished, and a few came over all shy about performing in front of a roomful of strangers. At least half a dozen, however, showed what the government could do if it operated under the same time constraints and had only pizza and beer to fuel its efforts.

The two most notable crowd-pleasers won prizes for their creators. These were Job Centre Pro Plus, the brainchild of Harry Metcalfe, Sam Smith, and Glyn Wintle (who won a Lego Mindstorms NXT), and Companies Open House (which won its five inventors an Arduino microcontroller and a bag of bits and pieces for hardware hacking).

Most people, Metcalfe explained, just want to know what jobs are available near them; they don't want to jump through all the hoops on the Job Centre Web site. Enter your postcode into Job Centre Pro Plus, and it will search for jobs in the field of your choice within a distance you specify. Its API will plot the available jobs on Google Maps.

The biggest problem with the official Companies House Web site, on the other hand, is that it doesn't give companies permanent URLs; in addition, the search service is closed between midnight and 7am every day. Companies Open House aims to change all that; a search on the site embeds the Companies House information in a permanent page. The site as demonstrated noted whether the company was an official government supplier and whether it had hired lobbyists within the last six months.

Job Centre Pro Plus was one of four projects Hoadley hoped to give space to on the Directgov Innovations site. The other three were the revamped sports ground locator Active Places, Directgov's schools site, and property information sites. Launched in 2005, Sport England's Active Places was built with £5 million in lottery money; it's intended to help people find sports facilities near them. In a written answer in February 2008, the DCMS showed that the site had 218,378 visitors in 2007.

Active Places Reloaded scraped the data, geo-coded it, and created a Web service to mash with Google Maps. The result: easily readable maps showing, for example, the sports facilities open to the public around Kings Cross. Combined with I Am Near, you can see, for example, the five public swimming pools near your current location.

Behind the Directgov schools site, there's a 4Mb CSV file of the schools in the UK. The team built a bulk data loader in Python and with very little code, created an API for schools. Using that, you can now find schools in an area based on their phone codes, postcode, government classification, or even how many have male heads. Time spent: three hours.

What was bugging Tom Lea and Luke Redpath, however, was the difficulty of finding an NHS dentist or doctor. The NHS Web site makes you click through to each dentist's listing to see if he's accepting patients. The revised "Find Me a Dentist" site lets you put in your post code and gives you back the list of all dentists in your area accepting new NHS patients.

Other projects included a tag cloud for MPs to compare how parties and politicians vote using data scraped from TheyWorkForYou.com; the design keeps each tag in the same place so that you can see it expand or shrink over a specified time line. Andrew Walkingshaw's council tax tracker compares council tax rates with the number of voters living there. A second voting tracker graphically displayed voting patterns around the country. Another project used Greasemonkey and JavaScript to automatically turn the tables of data available from the National Statistics Office into graphs.

Improved property search added statistics on local crime and jobs that display as bar charts alongside the property search results. EpicBin issues a reminder the night before garbage and recycling collections, along with a reminder of what goes in which bin. A project comparing crime and punishment standards in 1707 and 2007 found, ironically, that it was easier to parse the 18th century data than the current set.

The Guardian's involvement wasn't casual. The Guardian's technology director, Mike Bracken, closed up proceedings by explaining that the paper is investing in becoming "the world's leading liberal voice". As part of that, "We're putting our money into data." The paper can't buy a lot of data – much of it is under government restriction; hence the paper's "Free Our Data" campaign – but it wants to help make projects like those at Rewired State real and usable. Bracken picked the MP tag cloud and council tax projects to pursue.

"Government really does have to be changed from the outside as well as from the inside," he concluded, before asking everyone to come back and do it again next year.