Wednesday , August 5 2020
Home / Tim Harford: Undercover Economist / Book of the Week 24 – Dark Data by David Hand

Book of the Week 24 – Dark Data by David Hand

Summary:
Book of the Week 24 – Dark Data by David Hand What is Dark Data? Consider the example of the 1986 Challenger shuttle disaster. Should the shuttle launch, despite fears that low temperatures might weaken the “o-ring” seals? A graph of seven previous occasions when the o-rings had been stressed showed no relationship between temperature and the degree of damage. Alas, what was missing was the data from all the launches where there had been no damage at all: in each case, air temperatures were higher. On viewing all the data, a clear relationship was visible. “Dark data are data you do not have,” writes Hand. “They might be data you thought you had, or hoped to have, or wished you had. But they are data you don’t have. You might be aware that you do not have them, or

Topics:
Tim Harford considers the following as important:

This could be interesting, too:

Tim Harford writes Book of the Week 28 – The Monopolists by Mary Pilon

Tim Harford writes Book of the Week 27 – The Shock of the Old by David Edgerton

Tim Harford writes Announcing “How To Make The World Add Up”

Tim Harford writes Announcing “How To Make The World Add Up”

Book of the Week 24 – Dark Data by David Hand

Book of the Week 24 – Dark Data by David Hand

What is Dark Data?

Consider the example of the 1986 Challenger shuttle disaster. Should the shuttle launch, despite fears that low temperatures might weaken the “o-ring” seals? A graph of seven previous occasions when the o-rings had been stressed showed no relationship between temperature and the degree of damage.

Alas, what was missing was the data from all the launches where there had been no damage at all: in each case, air temperatures were higher. On viewing all the data, a clear relationship was visible.

“Dark data are data you do not have,” writes Hand. “They might be data you thought you had, or hoped to have, or wished you had. But they are data you don’t have. You might be aware that you do not have them, or you might be unaware.”

This book, then has some commonalities with Caroline Criado Perez’s Invisible Women, a book about systematic failure to collect data about women or issues that might be of particular relevance to women. Hand’s book is a shade more technical, but that is not the only difference: as the Challenger example makes clear, there are many different kinds of missing data, and many reasons why we might fail to have them. (The most mundane of all: time series data when some of the data points lie in the future…)

I enjoyed this book a lot; it is well-written and accesible, although mostly aimed at practioners.

UK: Blackwells Amazon

US: Powell’s Amazon

My NEW book The Next Fifty Things That Made the Modern Economy is NOW OUT. Details, and to order on Hive, Blackwells, Amazon or Watersones. Bill Bryson comments, “Endlessly insightful and full of surprises — exactly what you would expect from Tim Harford.”

Tim Harford
Tim is an economist, journalist and broadcaster. He is author of “Messy” and the million-selling “The Undercover Economist”, a senior columnist at the Financial Times, and the presenter of Radio 4’s “More or Less” and the iTunes-topping series “Fifty Things That Made the Modern Economy”. Tim has spoken at TED, PopTech and the Sydney Opera House and is a visiting fellow of Nuffield College, Oxford.

Leave a Reply

Your email address will not be published. Required fields are marked *