© Pixabay/Sandid – CC0

Journalists and computer scientists are working to develop automated fact-checking (AFC) to enable large-scale responses to online misinformation, according to a new Factsheet by the Reuters Institute for the Study of Journalism (RISJ). The study identifies a number of new automated tools, but finds their application remains limited by the complexity and nuances of most misinformation online.

“Understanding the Promise and Limits of Automated Fact-Checking”, by Lucas Graves, senior research fellow at RISJ, highlights the work of professional fact-checkers, platform companies, and artificial intelligence researchers.

Graves concludes that despite new initiatives aimed at developing automated verification tools, they are still incapable of replicating the sensitive contextual judgements context required in fact-checking.

Some AFC tools are able to verify a range of simple factual claims, where clear data is available, but only under human supervision – and it is likely that such supervision will be required for the foreseeable future.

Practitioners and researchers agree that the most effective AFC are those that assist human fact-checkers to identify factual claims and to deliver their conclusions as effectively as possible.

Graves finds that non-profit, independent fact-checking organisations, such as Full Fact, a UK-based charity and Chequeado, a fact-checking non-profit based in Brazil, have generally pioneered the implementation of automated verification tools, with little help from traditional media organisations.

While many of the tools have been built at low cost, non-profit organisations are likely to need financial and logistical support from larger organisations in the future, if they are to realise their full potential.

Automated fact-checking hubs

Duke Reporters Lab at Duke University Tech & Check Alerts, in beta testing since early 2018, automatically generates a daily email newsletter neatly listing 15 promising political claims harvested from transcripts of CNN programming. The programme uses the ClaimBuster API but identifies the speaker and strips out statements by journalists; modules are being developed to pull claims from the Congressional Record, the California legislature, and the Facebook feeds of candidates in contested congressional races.  The email goes out daily at 10 a.m. EST to PolitiFact, FactCheck.org, the Washington Post, and the Associated Press.

Another new project, FactStream, offers live, ‘second- screen’ fact-checking of major political events via a mobile app. Fact-checkers use the platform to respond to dubious claims in real time, issuing either a capsule summary from a previous fact-check, or, for new claims, a ‘quick take’ adding context. Its first public test came during the 2018 State of the Union address, fed by alerts from PolitiFact, FactCheck.org, and the Washington Post; reportedly more than 3,000 people used the app at some point during the speech.

Full Fact: The UK-based fact-checking charity Full Fact plans to release two AFC tools in 2018, developed with more than £500,000 in foundation support and meant for adoption by fact-checkers around the world. Both tools rely on a monitoring infrastructure that constantly scans major newspapers as well as broadcast news and parliamentary sources, using subtitle feeds as well as text to speech conversion.

Designed to help the fact-checkers respond quickly to misinformation, Live uses human-tailored searches to scan these sources in real time for statements relating to claims which Full Fact has already investigated. An exact match automatically pulls up the relevant conclusion; in other cases, Live tries to display contextual information such as official statistics.

Trends is based on the same engine but takes a longer view, offering claim-by-claim reports on the career of individual falsehoods. The purpose is to show fact-checkers whether their efforts have made a difference and help them to target their interventions more strategically.

The full report “Understanding the Promise and Limits of Automated Fact-Checking” is published by the Reuters Institute for the Study of Journalism and can be found here.

You may also be interested in Federico Guerrini’s “From Traditional To Online Fact Checking” and “Seeking Accuracy, Objectivity and Scope: Fact-Checking in Ukraine and US”.

Sign up for the EJO’s regular monthly newsletter.