Who? Ian Douglas

Where? Ian is the Telegraph’s head of digital production. He writes about technology, science and beekeeping.

Twitter? @IanDouglas

Just as we like to supply you with fresh and innovative tips every day, we are recommending journalists to follow online too. Recommended journalists can be from any sector of the industry: please send suggestions (you can nominate yourself) to sarah.booker at journalism.co.uk; or to @journalismnews.

• #followjourn: @emmabarnett – Emma Barnett/journalist
• #followjourn: @jackschofield – Jack Schofield/journalist
• #followjourn: @alex_willis – Alexandra Willis/journalist
• #followjourn: @johnplunkett149 – John Plunkett/Journalist
• #followjourn: @Nath_Rowden – Nathan Rowden/journalist

Andy Dickinson from the Department of Journalism at UCLAN sums up today’s announcement in this tweet: ‘Telegraph to drip-publish MP expenses online’.

[Update #1: Editor of Telegraph.co.uk, Marcus Warren, responded like this: 'Drip-publish? The whole cabinet at once....that's a minor flood, I think']

Yes, let the data war commence. The Guardian yesterday released its ‘major crowdsourcing tool’ as reported by Journalism.co.uk at this link. As described by one of its developers, Simon Willison, on his own blog, the Guardian is ‘crowdsourcing the analysis of the 700,000+ scanned [official] MP expenses documents’. It’s the Guardian’s ‘first live Django-powered application’. It’s also the first time the news site has hosted something on Amazon EC2, he says. Within 90 minutes of launch, 1700 users had ‘audited’ its data, reported the editor of Guardian.co.uk, Janine Gibson.

The Telegraph was keeping mum, save a few teasing tweets from Telegraph.co.uk editor Marcus Warren. A version of its ‘uncensored’ data was coming, but they would not say what and how much.

Now we know a bit more. As well as printing its data in a print supplement with Saturday’s newspaper they will gradually release the information online. As yet, copies of claim forms have been published using Issuu software, underneath each cabinet member’s name. See David Miliband’s 2005-6 expenses here, for example. From the Telegraph’s announcement:

• “Complete records of expense claims made by every Cabinet minister have been published by The Telegraph for the first time.”
• “In the coming weeks the expense claims of every MP, searchable by name and constituency, will be published on this website.”
• “There will be weekly releases region by region and a full schedule will be published on Tuesday.”
• “Tomorrow [Saturday], the Daily Telegraph will publish a comprehensive 68-page supplement setting out a summary of the claims of every sitting MP.”

Details of what’s included but not included in the official data at this link.  “Sensitive information, such as precise home addresses, phone numbers and bank account details, has been removed from the files by the Telegraph’s expenses investigation team,” the Telegraph reports.

So who is winning in the data wars? Here’s what Paul Bradshaw had to say earlier this morning:

“We may see more stories, we may see interesting mashups, and this will give The Guardian an edge over the newspaper that bought the unredacted data – The Telegraph. When – or if – they release their data online, you can only hope the two sets of data will be easy to merge.”

Update #2: Finally, Martin Belam’s post on open and closed journalism (published Thursday 18th) ended like this:

“I think the Telegraph’s bunkered attitude to their scoop, and their insistence that they alone determined what was ‘in the public interest’ from the documents is a marked contrast to the approach taken by The Guardian. The Telegraph are physically publishing a selection of their data on Saturday, but there is, as yet, no sign of it being made online in machine readable format.

“Both are news organisations passionately committed to what they do, and both have a strategy that they believe will deliver their digital future. As I say, I have a massive admiration for the scoop that The Telegraph pulled off, and I’m a strong believer in media plurality. As we endlessly debate ‘the future of news™’ I think both approaches have a role to play in our media landscape. I don’t expect this to be the last time we end up debating the pros and cons of the ‘closed’ and ‘open’ approaches to data driven journalism.”

It has provoked an interesting comment from Ian Douglas, the Telegraph’s head of digital production.

“I think you’re missing the fundamental difference in source material. No publisher would have released the completely unredacted scans for crowdsourced investigation, there was far too much on there that could never be considered as being in the public interest and could be damaging to private individuals (contact details of people who work for the MPs, for example, or suppliers). The Guardian, good as their project is, is working solely with government-approved information.”

“Perhaps you’ll change your mind when you see the cabinet expenses in full on the Telegraph website today [Friday], and other resources to come.”

Related Journalism.co.uk links:

• ‘Has the Telegraph failed by keeping expenses process and data to itself?’ [15/06/09]
• ‘Telegraph ‘didn’t tell any lies but was selective in its facts’, says Lib Dem Voice site editor’ [11/06/09]
• ‘Telegraph to publish ‘unredacted’ information… in print‘ [18/06/09]
• ‘Guardian launches ‘major crowd-sourcing experiment’ with MPs’ expenses application’ [18/06/09]

• MPs’ expenses data will be officially released Thursday but how much will be edited out?
• Telegraph.co.uk: Guide to the full MP expenses database
• Telegraph to publish ‘unredacted’ expenses information… in print
• Malcolm Coles: MPs’ expenses – the best of the web
• Heather Brooke and Telegraph named in PSA Awards

Appropriately enough a Twitter update from @BenLaMothe alerted Journalism.co.uk to an innovative new use of Twitter on Telegraph.co.uk’s sport pages.

After displaying Twitterfall, which can be set up to aggregate tweets containing multiple terms, on its big news screens, a stream of relevant Twitter updates are displayed in a widget on the right-hand side of the site’s live Premiership football match report pages.

Developed by a team of students, using Twitterfall could provide a neat way of following the conversations around certain players, transfer gossip or matches as they’re played.

Ian Douglas, head of digital production at Telegraph.co.uk, explained to Journalism.co.uk that list of club names and key player names are currently being tracked, but if new trends or keywords emerge they can be quickly added.

Certain tweaks to avoid irrelevant updates have been made – #chelsea is being used as opposed to Chelsea to avoid tweets about nights on the Kings Road, for example.

The Telegraph wanted to trial Twitterfall on pages that have ‘a lot of activity and a lot of people talking’, said Douglas, but is being considered for other areas of the site and potentially topic pages. The appropriateness of the widget to a given page, because it updates so rapidly, must be taken into consideration, he added.

The title is happy to look outside of its own development team to third parties when necessary, said Douglas, with other recent collaborations including this interactive guide to new Formula One cars.

• Twitterfall makes it onto Telegraph newsroom screens
• TEAMtalk goes all a Twitter for football finale
• Tip of the day from Journalism.co.uk – tools for aggregating Twitter updates
• Twitter users play with Telegraph’s Twitterfall #Budget feature
• More Telegraph and Twitterfall – spreads to topic pages

Interesting to see that Malcolm Coles’ recent blog post, which highlighted newspapers’ terms and conditions forbidding linking from their websites, has resulted in some action – outlined in the comments below the post.

Ian Douglas, head of digital production at Telegraph.co.uk says:

“Thanks for pointing this out. As you saw, we deleted it as soon as we heard about it. I’m afraid it say more about the relevance of Ts and Cs than the various papers’ attitude to linking.”

David Black, group director of digital publishing at Trinity Mirror, writes:

“Malcolm, Thanks for spotting this – it’s being fixed for Mirror.co.uk.”

Malcolm Coles has now followed up with another post – what other sites have these kinds of T&Cs banning links to their site? Full post at this link.

• Malcolm Coles: ‘The eight stages of linking-out denial’
• New appointment: Ayers is made regional web publishing director at Trinity Mirror Regionals
• Media release: Chris Ellis appointed MD of digital for Trinity Mirror nationals
• Producer responds to Guardian TV review: ‘If an opera is reviewed, you get someone who knows about opera’
• NUJ Release: NUJ condemns more job cuts at Trinity Mirror regionals

Earlier this week we highlighted ACAP’s response to some of the criticism that it felt had been leveled at it.

The Telegraph’s Ian Douglas (himself a critic of ACAP at launch) here responds to the response (phew, that’s a lot of responding!!).

• ACAP answers its critics
• Early problems with ACAP
• Eric Schmidt – Google resistance to ACAP based on technology
• paidContentUK: Microsoft saying no to ACAP
• Guardian gives comedians right of reply: There isn’t a ‘new offensiveness’ say Herring and Burns

ACAP was designed to be a system that allows content publishers to embed into their websites information that details access and use policies in a language that search engines can understand.

Over on Currybet.net Martin Belam has outlined some of the major flaws, as he sees them, of ACAP – which launched in New York last week.

Here’s a brief outline, but you have to go to his blog to get the necessary full picture:

It isn’t user centred

“On the ACAP site I didn’t see anything that explained to me why this would currently be a good thing for end users.

“It seems like a weak electronic online DRM – with the vague promise that in the future more ‘stuff’ will be published, precisely because you can do less with it.”

It isn’t technically sound

“I’ve no doubt that there has been technical input into the specification.

“It certainly doesn’t seem, though, to have been open to the round-robin peer review that the wider Internet community would expect if you were introducing a major new protocol you effectively intended to replace robots.txt”

The ACAP website tools don’t work

“I was unaware that there was a ‘known bug in Mozilla Firefox’ that prevented it saving a text file as a text file.

“I was going to make a cheap shot at the way that was phrased, as it clearly should have been ‘there is a known bug in our script which affects Mozilla Firefox’.

I thought though that I ought to check it in Internet Explorer first – and found that the ACAP tool didn’t work in that browser either.”

Update:

Ian Douglas, on the Telegraph, seems to have similar feelings about ACAP being too publisher-centric:

“Throughout Acap’s documents I found no examples of clear benefits for readers of the websites or increased flexibility of uses for the content or help with making web searches more relevant.

The new protocol focuses entirely on the desires of publishers, and only those publishers who fear what web users will do with the content if they don’t retain control over it at every point.”

• ACAP answers its critics
• Eric Schmidt – Google resistance to ACAP based on technology
• Google European Public Policy Blog: On working with newspapers
• UK national newspapers neglecting sitemaps for better search indexing
• Google introduces option for five-a-day free click limit