Columbia University’s study offers tool to measure the impact of stories such as the Panama Papers that exposed corruption. Photo by Shutterstock

Now and then, a headline with jaw-dropping revelations grabs the globe’s attention. The Watergate Scandal, the News of the World phone hacking revelations and the Panama Papers are just some examples of stories that challenged the status quo. Behind each of these historic reports are months of painstaking investigative journalism. Yet, the practice is becoming a dying art in today’s clickbait economy as news outlets struggle with dwindling resources. 

Therefore, demonstrating the power and importance of investigative journalism by measuring its impact is especially important today. Media organisations should have the tools to do this – whether they want to determine if there is an audience for an in-depth report before beginning months of investigation or show donors that a particular investigative reporting project has had an effect, such as a policy change. 

Since techniques for measuring impact are constantly evolving, a group of us worked together to develop a taxonomy – a tool to bring together what we know and propose a systematic way of measuring impact.

Gauging Impact

Example of taxonomy measurements.

In our recent paper, Understanding Journalism Impact: A Multi-dimensional Taxonomy for Professional, Organisation, and Societal Change, we include dozens of different indicators so that organisations trying to gauge impact can choose specific measures and have the guidance and tools they need.

We built on the influential three-stage framework findings of the authors of The Journalism of Outrage: Investigative Reporting and Agenda Building in America, which suggest that the three phases of impact are individual, deliberative and substantive and unfold over time. We also leaned heavily on  Phil Napoli’s comprehensive 2014 look at measuring impact. Our taxonomy includes the impact that collaborative investigative journalism projects can have on the individual and reporting can have on networks of journalists and society.

We gathered internal measures such as the effect on the beliefs and attitudes of journalists who do collaborative investigative reporting, as well as how this collaboration can affect the policies and behaviours of media organisations and the institutions and culture of the journalism community. Our checklist of impacts includes the effects that reporting such as the Panama Papers can have on the beliefs and attitudes of citizens, the behaviours and policies of governments and organisations and the institutions and cultures of society.

Targeted assessments

With dozens of metrics and a table that follows the three-part structure mentioned above, we aim to allow organisations trying to measure impact to choose which questions interest them. For example, an organisation that wants to look at how laws have changed as a result of investigative reporting would use one part of our taxonomy, while an editor wondering how the reporters involved were affected (for example, by learning new skills) might look at another function.

Though doing a cost-benefit analysis of investigative journalism is pretty much impossible, James D. Hamilton was able to make estimates for a few case studies described in his path-breaking book Democracy’s Detectives. Building on his approaches, we included measures that could be translated into an economic benefit, such as a journalist who gets a new job after being involved in a major investigative journalism project.

Depending on what one is looking for, there are, of course, many ways to try to gauge impact. In some cases, surveys may be helpful. In other cases, it may be more revealing to scrape data from the web, conduct focus groups or analyse which laws have been passed or policies have been put in place after a large investigative reporting project is published. 

As part of our research, we looked at previous surveys of reporters carried out by the International Consortium of Investigative Journalists and built a new survey with questions related to our taxonomy. The next phase of our research will be to work with different news organisations to help them analyse the effects of their investigative reporting and see whether our taxonomy and list of survey questions are useful in the real world. 

Ultimately, our aim was to create a comprehensive and flexible taxonomy. No measurement tool will ever perfectly reflect what every entity hopes to achieve. Still, we are encouraged by advancements in efforts to measure media impact and hope that our taxonomy will assist in expanding the future of research in the field.

About the authors of the study:

Anya Schiffrin is the Director of Technology, Media, and Communications specialisation at the School of International and Public Affairs (SIPA), Columbia University. She wrote the paper with SIPA colleague, adjunct associate professor André Corrêa d’Almeida, and Adelina Yankova, Columbia University PhD student, Lindsay Green-Barber, the co-founder of Impact Architects, and Dylan W. Groves, assistant professor Lafayette College.

Contact the authors or Impact Architects to learn more about the new taxonomy.

Opinions expressed on this website are those of the authors alone and do not necessarily reflect or represent the views, policies or positions of the EJO or the organisations with which they are affiliated.

If you liked this story, you may also be interested in: Audit of British Tory MP Demonstrates the Power of Investigative Journalism

Follow us on Facebook and Twitter.