Thursday , November 23 2017
Home / T. Cowen: Marginal Revolution / Checking how fact-checkers check?

Checking how fact-checkers check?

Summary:
That title made me think of the woodchuck…anyway, here is the abstract: Fact-checking has gained prominence as a reformist movement to revitalize truth-seeking ideals in journalism. While fact-checkers are often assumed to code facts accurately, no studies have formally assessed fact-checkers’ performance. I evaluate the performance of two major online fact-checkers, Politfact at Tampa Bay Times and Fact Checker at Washington Post, comparing their interrater reliability using a method that is regularly utilized across the social sciences. I show that fact-checkers rarely fact-check the same statement, and when they do, there is little agreement in their ratings. Approximately, 1 in 10 statements is fact-checked by both fact-checking outlets, and among claims that both outlets check, their

Topics:
Tyler Cowen considers the following as important: , ,

This could be interesting, too:

Tyler Cowen writes There is no great lung stagnation

Tyler Cowen writes What should I ask Ross Douthat?

Tyler Cowen writes The show so far, a continuing series

Alex Tabarrok writes Thrasher and Gaus on the Calculus of Consent

That title made me think of the woodchuck…anyway, here is the abstract:

Fact-checking has gained prominence as a reformist movement to revitalize truth-seeking ideals in journalism. While fact-checkers are often assumed to code facts accurately, no studies have formally assessed fact-checkers’ performance. I evaluate the performance of two major online fact-checkers, Politfact at Tampa Bay Times and Fact Checker at Washington Post, comparing their interrater reliability using a method that is regularly utilized across the social sciences. I show that fact-checkers rarely fact-check the same statement, and when they do, there is little agreement in their ratings. Approximately, 1 in 10 statements is fact-checked by both fact-checking outlets, and among claims that both outlets check, their factual ratings have a Cohen’s κ of 0.52, an agreement rate much lower than what is acceptable for social scientific coding. The results suggest that difficulties in fact-checking elites’ statements may limit the ability of journalistic fact-checking to hold politicians accountable.

That paper (pdf) is by Chloe Lim, political science at Stanford.  For the pointer I thank Andrew Hall, some interesting political science papers on his home page.  Here is his very interesting book manuscript on how the devaluing of political offices drives polarization, worthy of a top publisher…

The post Checking how fact-checkers check? appeared first on Marginal REVOLUTION.

Tyler Cowen
Tyler Cowen is an American economist, academic, and writer. He occupies the Holbert C. Harris Chair of economics as a professor at George Mason University and is co-author, with Alex Tabarrok, of the popular economics blog Marginal Revolution. Cowen and Tabarrok have also ventured into online education by starting Marginal Revolution University. He currently writes the "Economic Scene" column for the New York Times, and he also writes for such publications as The New Republic, the Wall Street Journal, Forbes, Newsweek, and the Wilson Quarterly.

Leave a Reply

Your email address will not be published. Required fields are marked *