Browse > Home /

Signals intelligence journalism: using public information websites to source stories

Useful information is more widely and easily available than ever and the increasing amount of online data released by the government and others can help improve the originality of journalists’ work.

Look to VentnorBlog – the hyperlocal online effort based in the Isle of Wight which Journalism.co.uk commended during the Vestas protest coverage – for some inspiration.

[For those unfamiliar with the story, locals had been protesting against the closure of the wind turbine factory in front of national, local and hyperlocal media. Despite a long and well-publicised campaign in August 2009, Danish company Vestas has now pulled out of manufacturing on the Isle of Wight but protests and attacks by critics in the press continue. A national day of action to support redundant Vestas workers has been planned for Thursday, September 17.]

Last week, using the Area Ship Traffic Website, AIS, VB was able to report where two barges held by an agent – NEG  Micron Rotors – who used to own the Vestas’ factory were due to head. They would be used to move the blades from the factory, which are so huge that they can only travel away on the water on special vessels.

The correspondent who tipped off VentnorBlog knew that the wind turbine blades can only be transferred from the riverside to barge when it is high tide and across a public footpath so, using the information on the AIS site, concluded that the barges would be moved in a specific time slot.

As a result Vestas protesters asked supporters to join them at the Marine Gate on the River Medina. Of course VentnorBlog got down there to take some pictures.

Now let’s take that one step further: how can journalists tap into this kind of publicly available data to scoop stories?

Tony Hirst, Open University academic, Isle of Wight resident and prolific data masher, shared some thoughts with Journalism.co.uk. He said that we should look to signals intelligence for further inspiration: the interception and analysis of ‘signals’ emitted by whoever you are surveying. As military historians would be the first to tell you, they can be a very rich source of intelligence about others’ actions and intentions, he explained.

“A major component of SIGINT is COMINT, or Communications Intelligence, which focuses on the communications between parties of interest. Even if communications are encrypted, Traffic Analysis, or the study of who’s talking to whom, how frequently, at what time of day, or  – historically – in advance of what sort of action, can be used to learn about the intentions of others.”

And this is relevant to journalists, he added:

“For starters, data is information, or raw intelligence. The job of the analyst, or the data journalist, is to identify signals in that information in order to identify something of meaning – ‘intelligence’ about intentions, or ‘evidence’ for a particular storyline.

The VentnorBlog story, he said, describes how a ‘sharp-eyed follower of movements at the plant’ knew where two barges were headed and at what time – valuable journalistic information:

“Amid the mess of Solent shipping information was a meaningful signal relating to the Vestas story – the movement of the barge that takes wind turbine blades from the Vestas factory on the Isle of Wight to the mainland.”

Do you have suggestions for sources of ‘signals intelligence’ journalism? Or examples of where it has been done well?

Tags: , , , , , , , , , , , , , , , , , , , ,

Similar posts:

Academics threaten Observer boycott: the letters in full

August 21st, 2009 | 1 Comment | Posted by in Job losses, Jobs, Newspapers

As reported on the main site, a number of high profile figures in business and academia have already, or are threatening to, cancel their subscriptions to the Observer, after the paper – the threatened closure of which has been widely reported – cut the weekly column by management expert Simon Caulkin. Below:

(1) Original letter to editors of the Guardian and Observer protesting the decision from over 60 signatories, never published.

(2) Follow-up letter from a key figure in the campaign, Philip Whiteley on behalf of over 80 signatories, questioning the lack of response, never published.

(3) Email reply from Observer editor, John Mullholland.

Hat-tip: Private Eye Issue 1243, August 21 – September 3, page 7, for a story that alerted us to the protest.

Letters in full:

(1) Original letter to editors of the Guardian and Observer

15 June 2009

The Editor
The Observer

Dear Sir

We are astonished and appalled by your decision to drop the Simon Caulkin column just at the point when the ideas he has covered over the years have become more relevant than ever.

We are living through one of the biggest crises of governance in history. September 2008 saw not just the end of Lehman Brothers but the end of 30 years’ dominance of neo-liberalism as the guiding ideology in running major private and public sector institutions. The notion that ‘maximising shareholder value’ can be considered in isolation from society was exposed as a pretence – bad for business as well as for society. The mechanistic strictures of the dominant management orthodoxy, with its dehumanising notion of people as a ‘resource’, its target culture and its opaque lexicon of competences, outputs and so on, have wrought terrible damage in social care, the NHS and education, as well as in the private sector.

Over the past 16 years, one journalist alone has been consistent in exposing the shallowness and limitations of these approaches. Simon Caulkin has set out a coherent alternative, rather than merely channelling protest. The unifying theme of the thinkers that he has championed – W Edwards Deming, Jeffrey Pfeffer, John Seddon, Gary Hamel and others – has been that organisations and economies are best managed by understanding the inter-dependence of different stakeholders.

Your decision, therefore, is ill-judged and ill-timed. A wiser choice would be to elevate Simon’s column to the main section of the paper. There is huge potential in the ideas he has promoted to assist ideological renewal of political parties, as well as to help governance generally.

We hope that you will see this as not just a letter of protest, but as sincere advice to recommend urgently that you reconsider your decision, and retain a vital element of your paper that could continue make a major contribution to policy debate.

Yours sincerely

Ricardo Semler, entrepreneur and author
Andrew Campbell, Director, Ashridge Business School
Philip Whiteley, chair Human Capital Forum
Dennis Tourish, Professor of Leadership and Management, Aberdeen Business School, Robert Gordon University
Susan White, Professor of Social Work, Department of Applied Social Science, Lancaster University
Petra Wilton, Director of Policy and Research, Chartered Management Institute
Joe Lamb, Emeritus Professor St Andrews University
Professor Jonathan Michie, President, Kellogg College, University of Oxford
Susan Scott-Parker OBE, chief executive of the Employers’ Forum on Disability
Professor Chris Brady, Dean, BPP Business School
H. Thomas Johnson, Professor of Business Administration Portland State University, USA
Mark Goyder, Director Tomorrow’s Company
Alistair Mant, Chairman, Socio-technical Strategy Group, Adjunct Professor, Swinburne University of Technology (Melbourne)
Ismail Erturk, Senior Lecturer in Banking, The University of Manchester
Su Maddock, Director Whitehall Innovation Hub
Dave Wastell, Professor of Information Systems, Nottingham University Business School
Gary Kirwan, Senior Employment Relations Adviser, Royal College of Nursing
Howard Clark, The Systems Thinking Review
Jim Standen, Director, Lignum Quality Services
Professor Bob Galliers, Provost and Vice President for Academic Affairs, Bentley University, Massachusetts, USA
Nigel Nicholson, Professor of Organisational Behaviour, University of London
Clive Bone, Chairman, Institute of Value Management
GD Cox
Keith Reader
Professor Anthony Hopwood, Said Business School
Alison Widdup, Managing Director, Better for Everyone
Fred John, Estates Officer, NHS.
Roy Madron, political scientist, UK/Brazil
Dr Richard Howells, Director, Centre for Cultural, Media and Creative Industries Research School of Arts and Humanities King’s College London
Margaret McCartney (Dr) GP and writer
Max Mckeown, Strategist and Leadership Innovation Expert
Sally Garratt, Director Garratt Learning Systems
Bob Garratt, Visiting Professor Cass Business School, London
Andrew Sturdy, Professor of Organisational Behaviour and Associate Dean, Warwick Business School, University of Warwick
Dr Martin Parker Professor of Culture and Organization, Director of Research and Deputy Head of School Editor-in-Chief of ‘Organization’ University of Leicester School of Management Leicester
Dr Gordon Pearson, Keele University
Jan Gillett, Chairman PMI
Dr. Mihaela Kelemen, Professor of Management Studies
Ian Christie, Associate, Green Alliance, Visiting professor, Centre for Environmental Strategy, University of Surrey
John Carlisle, Visiting Professor Sheffield Hallam University, Founder, Cooperation Works Ltd and the Intlizyo AIDS Trust, South Africa
Morice Mendoza, editor and writer
Dr Olivier Sykes, Department of Civic Design, University of Liverpool
Ron Glatter, Emeritus Professor of Educational Administration and Management, The Open University
Bob Bischhof, Chairman – Vitalize Health Products, Non Executive Director – Henderson Eurotrust Plc, Member of Board – German British Chamber of Industry and Commerce
Dr Paul Hodgkin, Chief Executive, Patient Opinion
Alastair Mitchell-Baker, Director Tricordant Ltd
Adam Hogg, Managing Director, (Retired) Conquest Inns
Simon Hollington, Director, Leading Edge Personal Development Ltd
Dr Philip McGovern, Programme Leader – Technology Management Programmes ITT
Neela Bettridge, Founding Partner, Article 13
John Orsmond, Chairman Data Vantage Group
Peter Medway
Paul H Ray, sociologist, USA
Tim Pidsley, director Tricordant, New Zealand
Dr Timothy Wadsworth, NHS
Dr Bruce Tofield, University of East Anglia
Warwick Mansell, freelance journalist and author Education by Numbers: the Tyranny of Testing
Professor Tom Keenoy, The University of Leicester School of Management
Bill Cooke, Professor of Management and Society, Lancaster University Management School
Dr Leslie Budd AcSS MCIT MCILT, Reader in Social Enterprise, Open University
Ken Starkey, Professor of Management and Organisational Learning, Nottingham University Business School

(2) Follow-up letter from a key figure in the campaign, Philip Whiteley, on behalf of 80 signatories

29 June 09

Dear Mr Rusbridger, Mr Mulholland

We write to register a double protest over the unjustified decision to drop the Simon Caulkin column, and your refusal to acknowledge the wave of anger that this decision has provoked.

Some 60 distinguished figures, including some of the most influential people in the world of business and management education, jointly signed a letter condemning your decision. You did not publish this, nor even give any of us the courtesy of an acknowledgment. In addition to this jointly signed correspondence, we know that over 200 people have individually registered their protest. The only letter to appear was mildly expressed. In short, you have seriously misled your readers over both the nature and extent of the protest, and of the support that Simon commands.

The Guardian/Observer has a strong tradition of respecting and upholding the principle of freedom of speech and dissent, so we find it shocking to be denied a space for an entirely legitimate argument, made by some of your (previously) most loyal and long-standing subscribers.

Doubtless you have made this move on business grounds; but you appear to have made no calculation of the business consequences of this decision. The supporters of this campaign are not just any readers, but long-standing subscribers who have passed on the habit of reading the Guardian/Observer to friends, colleagues, children and (given the number of professors and authors co-signing) to students and readers also, but who are now reconsidering their loyalty.

Questions of governance and management do not constitute a side issue to those of economics and politics: quite the reverse. It is the culture of management that has led to chronic waste in the public sector and the banking crisis in the private sector. Simon Caulkin possesses a deep understanding of the underlying causal factors of these crises.

Since we began this campaign, the extent of the protest has grown, as can be seen by the extended list of signatories to this letter.

If there is a necessity to drop pages, we urge you to move Simon’s weekly contribution to the main section of the paper.

Yours

Philip Whiteley
On behalf of over 80 signatories (see list below)

Cc
Will Hutton
Polly Toynbee
Dan Roberts
Liz Forgan

Signed by:
Ricardo Semler, entrepreneur and author
Andrew Campbell, Director, Ashridge Business School
Philip Whiteley, chair Human Capital Forum
Dennis Tourish, Professor of Leadership and Management, Aberdeen Business School, Robert Gordon University
Susan White, Professor of Social Work, Department of Applied Social Science, Lancaster University
Su Maddock, Director Whitehall Innovation Hub
Petra Wilton, Director of Policy and Research, Chartered Management Institute
Joe Lamb, Emeritus Professor St Andrews University
Professor Jonathan Michie, President, Kellogg College, University of Oxford
Susan Scott-Parker OBE, chief executive of the Employers’ Forum on Disability
Professor Chris Brady, Dean, BPP Business School
H. Thomas Johnson, Professor of Business Administration Portland State University, USA
Professor Christopher Grey, Head of Industrial Relations and Organizational Behaviour Group, Warwick Business School
Mark Goyder, Director Tomorrow’s Company
Alistair Mant, Chairman, Socio-technical Strategy Group, Adjunct Professor, Swinburne University of Technology (Melbourne)
Hilary Wainwright, Co-editor Red Pepper magazine, Fellow Centre for Participation Studies, Bradford University
Ismail Erturk, Senior Lecturer in Banking, The University of Manchester
Charlie Hedges, Chartered Geologist
Dave Wastell, Professor of Information Systems, Nottingham University Business School
Professor Martin Parker, University of Leicester
Gary Kirwan, Senior Employment Relations Adviser, Royal College of Nursing
Howard Clark, The Systems Thinking Review
Jim Standen, Director, Lignum Quality Services
Professor Bob Galliers, Provost and Vice President for Academic Affairs, Bentley University, Massachusetts, USA
David Davies, Director Didero Ltd
Nigel Nicholson, Professor of Organisational Behaviour, University of London
Clive Bone, Chairman, Institute of Value Management
GD Cox
Professor Anthony Hopwood, Said Business School
Alison Widdup, Managing Director, Better for Everyone
Fred John, Estates Officer, NHS.
Roy Madron, political scientist, UK/Brazil
Dr Richard Howells, Director, Centre for Cultural, Media and Creative Industries Research School of Arts and Humanities King’s College London
Max Mckeown, Strategist and Leadership Innovation Expert
Sally Garratt, Director Garratt Learning Systems
Bob Garratt, Visiting Professor Cass Business School, London
Andrew Sturdy, Professor of Organisational Behaviour and Associate Dean, Warwick Business School, University of Warwick
Dr Martin Parker Professor of Culture and Organization, Director of Research and Deputy Head of School Editor-in-Chief of ‘Organization’ University of Leicester School of Management Leicester
Dr Gordon Pearson, Keele University
Jan Gillett, Chairman PMI
Dr. Mihaela Kelemen, Professor of Management Studies
Ian Christie, Associate, Green Alliance, Visiting professor, Centre for Environmental Strategy, University of Surrey
John Carlisle, Visiting Professor Sheffield Hallam University, Founder, Cooperation Works Ltd and the Intlizyo AIDS Trust, South Africa
Morice Mendoza, editor and writer
Dr Olivier Sykes, Department of Civic Design, University of Liverpool
Ron Glatter, Emeritus Professor of Educational Administration and Management, The Open University
Bob Bischhof, Chairman – Vitalize Health Products, Non Executive Director – Henderson Eurotrust Plc, Member of Board – German British Chamber of Industry and Commerce
Dr Paul Hodgkin, Chief Executive, Patient Opinion
Alastair Mitchell-Baker, Director Tricordant Ltd
Adam Hogg, Managing Director, (Retired) Conquest Inns
Simon Hollington, Director, Leading Edge Personal Development Ltd
Dr Philip McGovern, Programme Leader, Technology Management Programmes, Institute of Technology, Tallaght, Dublin, Republic of Ireland
Neela Bettridge, Founding Partner, Article 13
John Orsmond, Chairman Data Vantage Group
Peter Medway
Paul H Ray, sociologist, USA
Tim Pidsley, director Tricordant, New Zealand
Dr Timothy Wadsworth, NHS
Dr Bruce Tofield, University of East Anglia
Professor Tom Keenoy, The University of Leicester School of Management
Bill Cooke, Professor of Management and Society, Lancaster University Management School
Dr Leslie Budd AcSS MCIT MCILT, Reader in Social Enterprise, Open University
Ken Starkey, Professor of Management and Organisational Learning, Nottingham University Business School
Kieran Doyle, General Manager Production at Sulzer Pumps UK Ltd
Dr Luke Mitcheson, Consultant Clinical Psychologist
Paul Buxton, Policy Officer, Crawley Borough Council
Roger Evans
Martin Meteyard (former Chair, Cafedirect plc)
Christopher Bird Owner, IT U Consulting Group
Laurence Barrett Associate Management Consultant
Paul Hodgkin Chief Executive at Patient Opinion
Bob Birtwell Tutor at University of Surrey
Andrew Campbell Director at Ashridge
Kathy Sheehy Williams Programme Manager at WEA
Rob Worth
Natascha Wolf, self-employed writer
Paul Summers, Corporate Programme Manager, Portsmouth City Council
David Kauders, Partner, Kauders Portfolio Management
Dave Kerr, Business Improvement Manager, Atkins
Paul Barratt, PMBprod
Kate Gott, PhD Student, Brunel Business School
Kevin Cryan, Analyst at DHL
Donal Carroll Associate at Open University Business School & Director at Critical Difference
Tim Casserley, Discovery Alliance & Edge Equilibrium & Author
Emma Langman, Head of Business Improvement at E Squared Thinking Ltd and Visiting Fellow in Systems at University of Bristol

(3) Reply from Observer editor, John Mullholland (by email)

1 July 2009 [by email]

Dear Phil Whiteley

Thank you for your letter and I must apologise for the delay in responding.

Simon Caulkin is a tremendous writer and his column has added enormously to our understanding of British business and management. For these reasons, the decision to lose the column was not taken lightly. It followed much discussion and only after exploring many different options did we reluctantly conclude that we had to take this course of action.

As you will doubtlessly appreciate, this was just one of a host of difficult decisions we have had to make in order to reduce costs across the newspapers at Guardian News and Media.

Newspapers and media groups are experiencing the most difficult trading conditions imaginable. Not only are we suffering, like everyone else, from the catastrophic fallout from the credit crunch in terms of severely reduced advertising revenues but, additionally, our industry is under structural assault from digital media which is causing enormous disruption to our business models.

In these circumstances, we are having to make extremely difficult decisions many of which have caused real anguish as we seek to cut costs. I do hope that Simon can continue to have a relationship with the paper and that we can continue to publish his writing from time to time. Should the economic climate change, then perhaps we can revisit the issue.

Thank you for taking the trouble to write and I completely understand your sense of loss but hope you can appreciate the dilemmas we are facing.

Yours sincerely
John Mulholland
Editor
The Observer

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Similar posts:

#DataJourn part 2: Q&A with ‘data juggler’ Tony Hirst

As explained in part one of today’s #datajourn conversation, Tony Hirst is the ‘data juggler’ (as titled by Guardian tech editor Charles Arthur) behind some of the most interesting uses of the Guardian’s Open Platform (unless swear words are your thing – in which case check out Tom Hume’s work)

Journalism.co.uk sent OU academic, mashup artist and Isle of Wight resident, Tony Hirst, some questions over. Here are his very comprehensive answers.

What’s your primary interest in – and motivation for – playing with the Guardian’s Open Platform?
TH: Open Platform is a combination of two things – the Guardian API, and the Guardian Data store. My interest in the API is twofold: first, at the technical level, does it play nicely with ‘mashup tools’ such as yahoo pipes, Google spreadsheet’s =importXML formula, and so on; secondly, what sort of content does it expose that might support a ‘news and learning’ mashup site where we can automatically pull in related open educational resources around a news story to help people learn more about the issues involved with that story?

One of the things I’ve been idling about lately is what a ‘university API’ might look at, so the architecture of the Guardian API, and in particular the way the URIs that call on the API, are structured is of interest in that regard (along with other APIs, such as the New York Times’ APIs, the BBC programmes’ API, and so on).

The data blog resources – which are currently being posted on Google spreadsheets – are a handy source of data in a convenient form that I can use to try out various ‘mashup recipes’. I’m not so interested in the data as is, more in the ways in which it can be combined with other data sets (for example, in Dabble DB) and or displayed using third party visualisation tools. What inspires me is trying to find ‘mashup patterns’ that other people can use with other data sets. I’ve written several blog posts showing how to pull data from Google spreadsheets in IBM’s Many Eyes Wikified visualisation tool: it’d be great if other people realised they could use a similar approach to visualise sets of data I haven’t looked at.

Playing with the actual data also turns up practical ‘issues’ about how easy it is to create mashups with public data. For example, one silly niggle I had with the MPs’ expenses data was that pound signs appeared in many of the data cells, which meant that Many Eyes Wikified, for example, couldn’t read the amounts as numbers, and so couldn’t chart them. (In fact, I don’t think it likes pound signs at all because of the character encoding!) Which meant I had to clean the data, which introduced another step in the chain where errors could be introduced, and which also raised the barrier to entry for people wanting to use the data directly from the data store spreadsheet. If I can help find some of the obstacles to effective data reuse, then maybe I can help people publish their data in way that makes it easier for other people to reuse (including myself!).

Do you feel content with the way journalists present data in news stories, or could we learn from developers and designers?
TH: There’s a problem here in that journalists have to present stories that are: a) subject to space and layout considerations beyond their control; and b) suited to their audience. Just publishing tabulated data is good in the sense that it provides the reader with evidence for claims made in a story (as well as potentially allowing other people to interrogate the data and maybe look for other interpretations of it), but I suspect is meaningless, or at least of no real interest, to most people. For large data sets, you wouldn’t want to publish them within a story anyway.

An important thing to remember about data is that it can be used to tell stories, and that it may hide a great many patterns. Some of these patterns are self-evident if the data is visualised appropriately. ‘Geo-data’ is a fine example of this. It’s natural home is on a map (as long as the geo-coding works properly, that is (i.e. the mapping from location names, for example, to latitude/longitude co-ordinates than can be plotted on a map).

Finding ways of visualising and interacting data is getting easier all the time. I try to find mashup patterns that don’t require much, if any, writing of computer programme code, and so in theory should be accessible to many non-developers. But it’s a confidence thing: and at the moment, I suspect that it is the developers who are more likely to feel confident taking data from one source, putting it into an application, and then providing the user with a simple user interface that they can ‘just use’.

You mentioned about ‘lowering barriers to entry’ – what do you mean by that, and how is it useful?

TH: Do you write SQL code to query databases? Do you write PHP code parse RSS feeds and filter out items of interest? Are you happy writing Javascript to parse a JSON feed, or would rather use XMLHTTPRequest and a server side proxy to pull in an XML feed into a web page and get around the domain security model?

Probably none of the above.

On the other hand, could you copy and paste a URL to a data set into a ‘fetch’ block in a Yahoo pipe, identify which data element related to a place name so that you could geocode the data, and then take the URL of the data coming out from the pipe and paste it into the Google maps search box to get a map based view of your data? Possibly…

Or how about taking a spreadsheet URL, pasting it into Many Eyes Wikified, choosing the chart type you wanted based on icons depicting those chart types, and then selecting the data elements you wanted to plot on each axis from a drop down menu? Probably…

What kind of recognition/reward would you like for helping a journalist produce a news story?
TH: A mention for my employer, The Open University, and a link to my personal blog, OUseful.info. If I’d written a ‘How To’ explanation describing how a mashup or visualisation was put together, a link to that would be nice too. And if I ever met the journalist concerned, a coffee would be appreciated! I also find it valuable knowing what sorts of things journalists would like to be able to do with the technology that they can’t work out how to do. This can feed into our course development process, identifying the skills requirements that are out there, and then potentially servicing those needs through our course provision. There’s also the potential for us to offer consultancy services to journalists too, producing tools and visualisations as part of a commercial agreement.

One of the things my department is looking at at the moment is a revamped website. it’s a possibility that I’ll start posting stories there about any news related mashups I put together, and if that is the case, then links to that content would be appropriate. This isn’t too unlike the relationship we have with the BBC, where we co-produce televlsion and radio programmes and get links back to supporting content on OU websites from BBC website, as well as programme credits. For example, I help pull together the website around the BBC World Service programme Digital Planet, which we co-produce every so often. which gets a link from the World Service website (as well as the programme’s Facebook group!), and the OU gets a mention in the closing credits. The rationale behind this approach is getting traffic to OU sites, of course, where we can then start to try to persuade people to sign up for related courses!

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Similar posts:

© Mousetrap Media Ltd. Theme: modified version of Statement