Wednesday , August 15 2018
Home / Econbrowser - James Hamilton / Predictions With and Without Confidence Intervals: Puerto Rico Post-Maria

Predictions With and Without Confidence Intervals: Puerto Rico Post-Maria

Summary:
Compare this assessment (5/31/2018): Thus, the data suggests that the hurricane accelerated the deaths of ill and dying people, rather than killing them outright. I would expect the excess deaths at a year horizon (through, say, Oct. 1, 2018) to total perhaps 200-400. Still a notable number, but certainly not 4,600. With a contemporaneous prediction: From the survey data, we estimated a mortality rate of 14.3 deaths (95% confidence interval [CI], 9.8 to 18.9) per 1000 persons from September 20 through December 31, 2017. This rate yielded a total of 4645 excess deaths during this period (95% CI, 793 to 8498), equivalent to a 62% increase in the mortality rate as compared with the same period in 2016. Subsequent to this report, the government released additional mortality data, as

Topics:
Menzie Chinn considers the following as important:

This could be interesting, too:

Tyler Cowen writes Monday assorted links

Tyler Cowen writes The decline of private ownership in America

Tyler Cowen writes Sunday assorted links

Tyler Cowen writes 10 Things I Learned: Tyler Cowen

Compare this assessment (5/31/2018):

Thus, the data suggests that the hurricane accelerated the deaths of ill and dying people, rather than killing them outright. I would expect the excess deaths at a year horizon (through, say, Oct. 1, 2018) to total perhaps 200-400. Still a notable number, but certainly not 4,600.

With a contemporaneous prediction:

From the survey data, we estimated a mortality rate of 14.3 deaths (95% confidence interval [CI], 9.8 to 18.9) per 1000 persons from September 20 through December 31, 2017. This rate yielded a total of 4645 excess deaths during this period (95% CI, 793 to 8498), equivalent to a 62% increase in the mortality rate as compared with the same period in 2016.

Subsequent to this report, the government released additional mortality data, as discussed here. A plot of estimates from that post is reproduced below:

Predictions With and Without Confidence Intervals: Puerto Rico Post-Maria
Figure 1: Estimates from Santos-Lozada and Jeffrey Howard (Nov. 2017) for September and October (calculated as difference of midpoint estimates), and Nashant Kishore et al. (May 2018) for December 2017 (blue triangles), and Roberto Rivera and Wolfgang Rolke (Feb. 2018) (red square), and calculated excess deaths using average deaths for 2015 and 2016 compared to 2017 and 2018 using administrative data released 6/1 (black square), and Santos-Lozada estimate based on administrative data released 6/1 (large dark blue triangle), end-of-month figures, all on log scale. + indicate upper and lower bounds for 95% confidence intervals. Orange + denotes Steven Kopits 5/31 estimate for range of excess deaths through September 2018; Orange triangle is Steven Kopits estimate for year-end as of June 4. Cumulative figure for Santos-Lozada and Howard October figure author’s calculations based on reported monthly figures.

From NYT today:

Puerto Rico is now estimating that Hurricane Maria killed more than 1,400 people, far more than the official death toll of 64, in a report to Congress seeking billions to help the island recover from the devastating storm.

The government, relying on updated statistics it first reported in June, said there were 1,427 more deaths from September to December 2017 than the average for the same time period over the previous four years.

These new numbers are apparently (my inference) prompted by this JAMA letter by Santos-Lozada and Howard:

…based on death records following Hurricane Maria, the estimated hurricane-related mortality burden of 1139 excess deaths through December 2017 was higher than the official death toll of 64. The estimate is conservative, because the expected number of deaths used the upper 95% CI and did not consider the population denominators, which were decreasing. The strength of the present approach is that it is based on death counts from vital statistics records and is consistent with previous estimates1 and methods.4,5 The primary limitation of the study is that the specific cause of each individual death is not known; thus only an aggregate number of deaths in excess of historical variation can be estimated. Another recent study2 suggested that there were 4645 excess deaths (95% CI, 783-8498), but it was based on a survey that underestimated prehurricane mortality, overestimated posthurricane mortality, and had a large CI, indicating a high level of uncertainty. Future studies would benefit from careful analysis of deaths from vital records rather than surveys. [emphasis added — MDC]

From the letter:
Predictions With and Without Confidence Intervals: Puerto Rico Post-Maria

The Puerto Rico government estimates apparently use the mean of past years rather than upper bound for the confidence interval to do the comparison. It is important to note that since the estimate Santos-Lozada and Howard estimate does not take into account population drop over the recent years (which apparently has been substantial), the average expected mortality rates are probably overstated, implying the gap between observed and counterfactual is under-stated.

While the numbers of fatalities for September-December are likely fairly firm, I don’t really know that for certain. That’s because there is a large backlog of bodies at the coroners office, predating the hurricane, prompting a request for additional mortuary officers. Interestingly, the request for additional mortuary officers came only on July 23, 2018. So, asserting that the given numbers are firm is foolhardy. (Listen also to this item).

So, there still seems to be substantial opacity regarding the gathering and reporting of mortality statistics. In these situations, it only seems right and proper to include confidence intervals. (I would also not be surprised to see the government’s estimate of excess mortality rise over time, given the incentives that governments face.)

Menzie Chinn
He is Professor of Public Affairs and Economics at the University of Wisconsin, Madison

Leave a Reply

Your email address will not be published. Required fields are marked *