Tag Archives: Open Data

#ODCC – Open data and the ‘new digital fields of exchange’

Leave a reply

Today marked the first Open Data Cities Conference which kicked off in Brighton, set up by former head of digital development at the Telegraph Greg Hadfield.

The conference said it would “focus on how publicly-funded organisations can engage with citizens to build more creative, prosperous and accountable communities”.

Among those citizens are of course the journalists working to encourage the opening up of data held by such organisations, wishing to use it to inform their audience about the local area and/or their interests.

“Connected localism” and adopting a “principle of openness”

An interesting phrase used at the conference was “connected localism”. The man behind it, Jonathan Carr-West of the Local Government Information Unit, spoke to the conference about the importance of creating a cultural mindset around openness, as opposed to just focusing on whether or not data is useful. And once this mindset has been established, “connected localism” can thrive.

We’re going to hear a lot today about data and what we use it for and how we make it useful. That’s really important and I don’t want to move away from that too far, but I would suggest … usefulness is not the whole story.

We don’t always know what’s useful … We need to adopt … a principle of openness. Whether you’re a small organisation, a council, a government.

He added the “assumption” needs to be that information is made open and data is shared.

Don’t over-think whether it’s going to be useful or not.

And this “principle of openness” is “what creates a field of exchange within which connected localism can occur”.

If we have openness as the way of doing things, if it is culturally embedded in our practice, that would begin to enable that connected localism.

We’ll talk a lot about open cities, but we should remember in this sense it’s not just making the city open, it’s that open data is effectively a new city.

It enables us to perform radical transformations to public services, to how we live … that we need if we’re to meet the profound challenges our society faces.

He cited Mumsnet as an example of “connected localism”, and one of the “new digital fields of exchange where people can connect”, and share/discuss/solve common interests.

Encouraging responses to information requests

Tom Steinberg of MySociety offered some tips for conference delegates on how to encourage more open data and the release of information, such as that asked for in freedom of information requests:

1. Don’t expect to win an economic argument about open data with people who do not have some other reason to think it’s a good idea. It is really hard with open data as it is a new issue so literature is new.

2. You should show them tools that will improve their lives based on open data. If you’re persuading a councillor use something like TheyWorkForYou and show them how they can get sent email alerts when an issue is mentioned in parliament. 10 per cent of everyone working in parliament uses it each week.

3. Don’t shout too loudly about how it [open data] will hold everyone to account and expose wrongdoing. If people are overworked, having their lives made harder is not a thing that will make them your friend.

4. Make mock-ups. For lots of kinds of open data there aren’t good examples as government hasn’t released the data. But use the amazing power of Photoshop to say ‘here’s a page where people could go to, for example, if they wanted to complain that their bin had not been collected’. This is a way of connecting the abstruse nature of data to a concrete thing.

He suggested that bodies such as councils should consider having a person specially dedicated to looking out for, and filtering, requests, and possibly add a button to their websites asking exactly what data people want.

How the BBC is opening up its archives

An interesting example of how one organisation is opening up its archived data is the BBC, as speaker Bill Thompson, who is head of partnership development in archive development at the broadcaster, explained.

The situation, as he posed it, is about turning the BBC “into a data repository with an API” and making this data “available for public service use, for people who can find a value in it”.

One project called BBC Redux provides a store of digital recordings which, when combined with the BBC’s Snippets project, enables users to search programmes, such as news bulletins, from the last five years, for the mention of a given keyword using subtitle data.

For more from the conference follow #ODCC on Twitter.

What’s happening to mark open data day

Leave a reply

The use of open data in our newsrooms has been growing in the past few years and many people believe that the future of data journalism relies on the collaboration between developers, designers and journalists to create better ways of extracting information from open datasets.

Tomorrow (3 December) is International Open Data Day and there is a series of worldwide events set up to gather coders, programmers and journalists around “live hacking” challenges.

International Open Data Hackathon

Where? The Barbican in London and around the world

When? Saturday, 3 December from 11am

Better tools. More Data. Bigger Fun. That’s how the 2011 Open Data Day Hackathon describes this year’s global event, taking place in more than 32 countries this weekend.

For journalists, it’s an occasion to give hacking a go and meet people from the world of data.

The past year has seen open data continue to gain traction around the world with new open data catalogues launched in Europe, North America and Africa and more data available from organisations such as the World Bank.

Open Data Day is a gathering of citizens in cities around the world to write applications, liberate data, create visualisations and publish analyses using open public data. Its aim is to show support for and encourage the adoption of open data policies by the world’s local, regional and national governments.

Join the Open Knowledge Foundation and CKAN at the Barbican tomorrow (Saturday, 3 December) as they assemble a “crack-team” of coders to break data out of its internet prisons and load it into the Data Hub.

For details about the event, see this blog post, and sign up on the event’s meetup page or by filling out the event’s Google form.

Participants will be on IRC and will also be using the hashtags #seizedata and #odhdLDN on Twitter. All journalists, data scrapers, coders and #opendata enthusiasts can join.

David Eaves, the organiser of this year’s Open Data Hackathon believes this event is a great opportunity to teach journalists, as well as the general public, how to tackle data on a day-to-day basis:

Its a Maker Faire-like opportunity for people to celebrate open data by creating visualisations, writing up analyses, building apps or doing what ever they want with data.

What I do want is for people to have fun, to learn, and to engage those who are still wrestling with the opportunities around open data … And we’ve got better tools. With a number of governments using Socrata there are more API’s out there for us to leverage. ScraperWiki has gotten better and new tools like Buzzdata, the Data Hub and Google’s Fusion Tables are emerging every day.

Who’s it for? Everyone. David Eaves says:

If you have an idea for using open data, want to find an interesting project to contribute towards, or simply want to see what’s happening, then definitely come along.

You can also check out the HackFest 2011 topic page on BuzzData.

London “Random Hacks of Kindness” event

Where? @Forward in London, and around the world

When? 3-4 December 2011, from 9am Saturday until 6pm Sunday

Starting on the same day as the Open Data Hackathon, the Random Hacks of Kindness’ Codesprint will gather thousands of experts in 25 countries to develop open tech solutions over two days of hacking challenges.

The unprecedented gatherings in collaboration with Google, Microsoft, Yahoo!, NASA, HP and the World Bank will bring together some of the world’’ most innovative social enterprises and volunteer technologists.

London’s event promises to be exciting as over 100 tech heads will gather to tackle one issue: financial exclusion and illiteracy. It will be the first ever hack day addressing this theme.

Financial and enterprise education group MyBnk will head a panel of CEOs and IT specialists from LSE, Morgan Stanley, Fair Finance, Three Hands, Toynbee Hall and the Forward Foundation to make major advances in helping young people master money management.

Mike Mompi, head of strategy and innovation at My BNK and the organiser of London RHoK event says:

The main objectives of the weekend are problem solving, capacity building, partnerships, and impact

A £500 cash prize will be given at the end of Sunday for the winning solution (among other prizes) and several media organisations, including The Huffington Post, will be joining in.

People from RHoK have hosted three global events to date, in 31 cities around the globe with over 3,000 participants. Past events resulted in apps and alert systems to warn people of bushfires in Australia and recipients of food stamps to sources of fresh produce in Philadelphia.

The RHoK community is open for anyone to join.

If you want to get an idea of what’s in store for this weekend, check out last year’s hackathon videos.

You will be able to follow the event on Twitter @RHoKLondon and the hashtag #rhokLDN. It is still possible to sign up for this weekend’s free event via this link.

Visual.ly illustrates the evolution of open data

Leave a reply
A recently launched tool to share data visualisations Visual.ly has created and shared a history of the open data movement.
Visual.ly allows news sites and blogs to embed the uploaded visualisations – in the true spirit of the open data movement.
The visualisation has a timeline on the evolution of APIs and the release of public data, including facts and figures on Data.gov.uk, a site where journalists can access and work with public data which launched in public beta in January last year.

David Higgerson: Journalists must keep pushing for open data

Leave a reply

David Higgerson, head of multimedia for Trinity Mirror Regionals, has published the address he made about data journalism at the FutureEverything conference in Manchester last week, making some interesting points.

Higgerson says that for journalists the biggest challenge is going to keep “pushing” for data to become available.

Councils have to issue details of all spending over £500 – but some councils have decided to publish all spending because it’s cheaper to do so. As journalists, we should push for that to happen everywhere.

FOI is key here. The more we ask for something under FOI because it isn’t freely available, the greater the chance its release will become routine, rather than requested. That’s the challenge for today’s data journalists: Not creating stunning visualisations, but helping to decide what is released, rather than just passively accepting what’s released.

Read his post in full here…

Journalism.co.uk is running a one-day digital journalism conference looking at data in the news industry next week at Thomson Reuters. news:rewired – noise to signal will take place on Friday 27 May. You can find out more information and buy tickets by following this link.

NPR: Finding stories in a ‘sea of government data’

Leave a reply

At the end of last week, NPR’s On The Media show spoke to Texas Tribune reporter Matt Stiles and Duke University computational journalism professor Sarah Cohen about how to find good stories in a “sea of government data”.

Listen to the full interview below:

Journalism.co.uk will be looking at open government data and the skills needed to find stories in datasets at its upcoming news:rewired conference. See the full agenda at this link.

#ijf11: The key term in open data? It’s ‘re-use’, says Jonathan Gray

2 Replies

If there were one key word in open data it would be “re-use”, according to Open Knowledge Foundation community coordinator Jonathan Gray.

Speaking on an open data panel at the International Journalism Festival, Gray said the freedom to re-use open government data is what makes it distinctive from the government information that has been available online for years but locked up under an all rights reserved license or a confusing mixture of different terms and conditions.

Properly open data, Gray said, is “free for anyone to re-use or redistribute for any purpose”.

The important thing about open data is moving from a situation of legal uncertainly to legal clarity.

And he sketched out in his presentation what the word “open” should mean in this context:

Open = use, re-use, redistribution, commerical re-use, derivative works.

The Open Knowledge Foundation promotes open data but most importantly, Gray said, was finding beneficial ways to apply that data.

Perhaps the signal example from the foundation itself is Where Does My Money Go, which analyses data about UK public spending.

Open Knowledge Foundation projects like Where Does My Money Go are about “giving people literacy with public information”, Gray said.

Nothing will replace years of working with this information day in and day out, and harnessing external expertise is essential. But the key is allowing a lot more people to understand complex information quickly.

Along with its visualisation and analysis projects, the foundation has established opendefinition.org, which provides criteria for openness in relation to data, content, and software services, and opendatasearch.org, which is aggregating open data sets from around the world. See a full list of OKF projects at this link.

“Tools so good that they are invisible”

This is what the open data movement needs, Gray said, “tools that are so good that they are invisible”.

Before the panel he suggested the example of some of the Google tools that millions use every day, simple effective open tools that we turn to without thinking, that are “so good we don’t even know that they are there”.

Along with Guardian data editor Simon Rogers, Gray was leaving Perugia for Rome, to take part in a meeting with senior Italian politicians about taking the open data movement forward in Italy. And he had been in France the week before talking to people about an upcoming open data portal in France – “there is a lot of top level enthusiasm for it there”.

In an introduction to the session, Ernesto Belisario president of the Italian Association for Open Government, revealed enthusiasm for open data is not restricted to larger, more developed countries.

Georgia has established its own open data portal, opendata.ge, and according to Belisario, took out an advert to promote the country’s increasing transparency ranking.

Some are expensive – the US, which began open government data publishing with data.gov, spend £34 million a year maintaining the various open data sites.

Others are cheap by comparison, with the UK’s opendata.gov.uk reportedly costing £250,000 to set up.

Some countries will pioneer with open data, some will bitterly resist. But with groups like the Open Knowledge Foundation busy flying representatives around the world to discuss it, that movement “from legal uncertainty to legal clarity” seems likely to move from strength to strength.

See Gray’s full presentation at this link.

See more from #ijf11 on the Journalism.co.uk Editor’s Blog.

Currybet: What open government data giveth, closed state data taketh away

Leave a reply

Government information architect Martin Belam has an interesting post about some of the limitations of the recent government data release, particularly the difficulty of – and cost associated with – cross-referencing the data with Companies House records.

Using the Guardian’s data explorer tool, you can get a comprehensive list of suppliers. Wouldn’t it be wonderful if you could instantly cross-reference that with the records at Companies House?

I’d love to be able to get an instant snapshot of how many of these companies are large, medium or small enterprises. Over time you could use that to measure whether the intention to open up Government service tendering to wider competition was on track or not.

Full post at this link…

Poligraft: the transparency tool set to make investigative journalism easier

Leave a reply

The Sunlight Foundation has launched a new tool – Poligraft – to encourage greater transparency of public figures and assist journalists in providing the extra details behind stories.

By scanning news articles, press releases or blog posts, which can be submitted to the program by inserting the URL or pasting the entire article, the technology can then pick out people or organisations and identify the financial or political links between them.

Discussing the impact of this technology, Megan Taylor writes on PoynterOnline that it is a simple yet powerful tool for the news industry.

Anyone can use this, but it could be especially powerful in the hands of hands of journalists, bloggers, and others reporting or analyzing the news. It would take hours to look these things up by hand, and many people don’t know how to find or use the information.

Journalists could paste in their copy to do a quick check for connections they might have missed. Bloggers could run Poligraft on a series of political stories to reveal the web of contributions leading to a bill. All this information is public record, but it’s never easy to dig through. What is possible when investigative journalism is made just a little bit easier?

See a video below from the Sunshine Foundation posted on Youtube explaining how the technology works:

Hatip: Editorsweblog

The difficult reality of open council data and journalism

Leave a reply

Andy Mabbett has an interesting post on his pigsonthewing blog about the difficulties surrounding open data for councils and subsequent media interpretations and reports.

Giving an example of grant funding and council spending, where conditions may involve money being spent on certain forms of advertising, he adds that often this is misinterpreted by some members of the press, unaware of the attached conditions or other related spending on important projects involved. As a result, the headlines focus on what appears to be unusual spending.

As a supporter of the principles of open public data, he says a solution needs to be found.

What can council’s do to prevent this scenario? Annotate every spend item in their published data? Surely impractical. List such items separately? I don’t know (and don’t get me wrong, I’m an open-data advocate; and this is a relatively minor matter, which shouldn’t stop such data from being published), but do I hope somebody has an answer.

See his full post here…

Should newspapers publish full interview transcripts online?

12 Replies

Washington Post economic and domestic policy blogger Ezra Klein has called for newspapers to make full interview transcripts available online, where there are not the traditional space restrictions of a print edition.

Klein cites last week’s New York Times article on Paul Volcker, which is “clearly and proudly set around a wide-ranging, on-the-record interview with Volcker himself”:

But that interview, aside from a few isolated quotes, is nowhere to be found. This is a baffling waste of good information. Reporters are endlessly interviewing newsmakers and then using, at most, a handful of lines out of thousands of words. The paper, of course, may not have room for thousands of words of interview transcripts, but the web certainly does.

Klein’s comments echo those of Wikileaks founder Julian Assange, who criticised the media on Friday for not making use of the huge amount of space available online to make primary source material more readily available.

The main issue for Klein, like Assange, seems to be one of transparency, especially for the interviewee:

It’s safer to have your full comments, and the questions that led to them, out in the open, rather than just the lines the author thought interesting enough to include in the article.

“And for the institution itself,” writes Klein, “it’s a no-brainer. You get a lot more inward links if you provide enough transcript that every niche media site can find something to point their readers toward.”

But news organisations considering such a move would have to weigh any potential increase in traffic – and any respect garnered by increased openness – with what is surely, for most, an unwelcome level of transparency. To say nothing of having to transcribe the hours and hours of interviews conducted by a newspaper such as the New York Times.

It is an interesting question for online journalism nonetheless. With programmes like the Open Government Data Initiative tipping more and more raw materials into the internet, will news organisations benefit overall from taking the same open approach?

Read Ezra Klein’s post here.