Europe's reckless precaution

When following the precautionary principle is letting COVID-19 slaughter thousands across the EU

Much of the content of this post was summarized this morning by Prof. Alex Tabarrok at Marginal Revolution. However, I think this such important material that it bears repeating and elaborating on more here as well.

Nearly everything the EU has done to provide their citizenry with the AstraZeneca vaccine has been bundled. First, the European Commission spent months going back and forth with AstraZeneca about the price point of the vaccines ($5-$40 / dose). As a result the EU signed a contract with AstraZeneca three months later than the UK. That three month delay delayed the planning and ramp up of production of the vaccine within the EU. Note that the Moderna’s vaccine costs $35-40 per dose and Pfizer’s vaccine costs $20 per dose, and that the expected economic return for each dose of a COVID-19 vaccine has been estimated by economists to be around $1000 - $3000. Then, as one might have predicted, there were production glitches in AstraZeneca’s factory in Seneffe, Belgium — glitches that likely could have been ironed out earlier with an earlier contract. There’s also the case of an AstraZeneca factory in Leiden, Netherlands run by Halix B.V., which has still not gotten regulatory approval from the EMA and is not expected to get approval until March 25th at the earliest.

EU countries rush to ban AstraZeneca out of precaution

Fast-forward to early last week, when the Norwegian Medicines Agency announced they were looking into a few cases of blood clots and reduced platelet counts. On March 10 Austria banned (euphemism “suspended”) the vaccine, and nine other countries followed suit (Denmark, Norway, Iceland, Austria, Estonia, Lithuania, Luxembourg, Italy, and Latvia).

Belgium was a notable exception. Belgian’s health minster rightly pointed out “We take a bigger risk if we don't give the vaccine". Unfortunately though, Belgium’s largest vaccination center was closed Tuesday morning and anectodotal reports from BBC News suggested that people who once wanted the AstraZeneca vaccine were now reluctant to get it.

The ban in Germany sparked blowback from both the public and legislature in the Bundestag due to the lack of transparency into the decision making process. In a subsequent German report, released late in the day on Tuesday, the German government explained that 7 Germans, aged 20 - 50 years old, had cerebral venous sinus thrombosis (CVST) associated with platelet deficiency 4 - 16 days after receiving the AstraZeneca vaccine. Three out of those 7 died. Those 7 events out of 1.6 million vaccinations in Germany were judged to be statistically significant compared to the background rate for the same age group.

On Tuesday the EMA’s safety committee reviewed the data and concluded “the vaccine is not associated with an increase in the overall risk of blood clots (thromboembolic events) in those who receive it.” They noted that the total number of blood clots among those who had gotten the vaccine was actually lower than what would be expected from a similar number of people in the general population.

However, they also concluded that if you look at two particular types of blood clotting in isolation among certain age groups, then statistically significant effects can be found. In particular they looked at “disseminated intravascular coagulation” (DIC), which is jargon for many small clots at once, and CVST, which was mentioned above. They note “based on pre-COVID figures it was calculated that less than 1 reported case of DIC might have been expected by 16 March among people under 50 within 14 days of receiving the vaccine, whereas 5 cases had been reported” and “on average 1.35 cases of CVST might have been expected among this age group whereas by the same cut-off date there had been 12.”

There are two very important points to take into account here:

The first point, which is noted by the EMA, is that COVID-19 positive individuals are at higher risk for blood clotting (about 2% who receive treatment for their illness get blood clots). So, with the COVID-19 pandemic ravaging Europe, background levels are expected to be higher. Unfortunately this was not taken into account in the statistical analysis. A physicist would have estimated the size of this effect with a Fermi estimate. [Physicists find Fermi estimation aesthetically pleasing but it seems epidemiologists and medical people view it to be “crude” and aesthetically unappealing, even though Fermi estimates are typically accurate to within in a factor of 10, and such an estimate may be better than no estimate at all.]

The second point, pointed out early by Rob Wiblin on Twitter, is that this analysis wreaks of p-hacking, also called “the multiple comparisons problem” or “data dredging”. The basic idea of p-hacking is the more possible side effects you look for, the more likely you are to find something that appears statistically significant just by chance. There are simple ways to correct for this, such as using the Bonfferroni correction, and also not-so-simple techniques such Empirical Bayes and other false discovery rate methods. It appears neither the EMA or the Paul-Ehrlich-Institut in Germany used any of these well-known techniques in their analysis. I find this very disappointing.

Consider the following other points, which bolster the case that p-hacking is the issue here:

• Phil Bryan, vaccines safety lead for the UK’s Medicines and Healthcare Products Regulatory Agency, said, “More than 11 million doses of the AstraZeneca vaccine have now been administered across the UK. Reports of blood clots received so far are not greater than the number that would have occurred naturally in the vaccinated population.”

• An AstraZenenca spokesperson said : “An analysis of our safety data of more than 10 million records has shown no evidence of an increased risk of pulmonary embolism or deep vein thrombosis in any defined age group, gender, batch or in any particular country with Covid-19 Vaccine AstraZeneca. In fact, the observed number of these types of events is significantly lower in those vaccinated than what would be expected among the general population.”

The devastation of the precautionary principle

The irony of all this is that if any individual wants to avoid getting blood clots, a vaccine is the best way to do that. Without mass vaccination, it is estimated that 60-75% of people will eventually get COVID-19, no matter what we do. The risks of blood clotting from COVID-19 are significant. Rassen et al. analyzed health claims data for 70,288 patients diagnosed with COVID-19 in the US. Their findings, reported in the peer-reviewed journal CMAJ, found that about 2% developed a blood clot. Of course, this is among people who received some form of treatment which required billing their insurance.

People are testing positive for COVID-19 at a rate of 1,686/1,000,000 in Germany and 5,223/1,000,000 in France every two week period (I got these numbers by Googling the # of daily COVID-19 cases and population of each country). At least in Ohio/Florida in August about 21% of people who tested positive for COVID-19 required hospitalization. So, if we assume 20% of people who test positive for COVID-19 require some form of visit to a clinic requiring an insurance claim, and 2% of those people get blood clots, the risk of getting blood clots from COVID-19 in a 2 week period is ~20/1,000,000 in France and ~7/1,000,000 in Germany. This is obviously a crude estimation, but the estimated risks are higher than the reported rates of blood clotting per 2 week periods after AstraZeneca vaccination, which is roughly 2.6/1,000,000.¹ To put these numbers in perspective, that the average American will die in a car crash in any given 2 week period is about 4.5 / 1,000,000.

But the above analysis is a bit silly, because blood clots from COVID-19 should be the least of your worries. You might also die. Stanley Pignal, a reporter at The Economist, reports that a French analysis found that for every 100,000 people you vaccinate today rather than tomorrow you save 15 lives. He then notes that Germany has 1.7 million AstraZeneca doses which are not being distributed, and if they are held up from being distributed for a week, then an estimated 1,785 people will die. I would love to see how this estimate was done. Incidentally, this estimate on the cost of delay (1,050 deaths per million doses per week) is close to the result of a very crude estimate I did a few weeks ago for the delay in the J&J vaccine.

The above estimate on the COVID-19 slaughter enabled by bans on the AstraZeneca vaccine is likely an underestimate, because it doesn’t account for the increased vaccine hesitancy such bans cause. This post is already getting long though, and that is a subject best left for a future post. If you want you can read here about how many doctors are very concerned that many in Europe will not get the AstraZeneca vaccine now.

This trolley problem meme sums everything up well:

Here are some frequencies, in summary:

3.3 / 1,000,000 - fraction of people who got some form of blood clotting after getting the Pfizer vaccine in the UK between December 9th, 2020 and February 28th, 2021.
4.0 / 1,000,000 - fraction of people who got some form of blood clotting after getting the AstraZeneca vaccine in the UK between January 4th 2021 and February 28th, 2021.
2.0 / 1,000,000 - fraction of people who got some form of blood clotting after getting the AstraZeneca vaccine in the EU.
550 / 1,000,000 - approximate risk of blood clotting from hormonal birth control pills.
20,000 /1,000,000 - fraction of people who billed their insurance for COVID-19 treatment that developed blood clots (see discussion above).
4.3 / 1,000,000 - fraction of people in Germany who got CVST within 16 days of getting the vaccine.
0.2 / 1,000,000 - average fraction of people in the US who get CVST within a 16 day interval.
48 / 1,000,000 - average number of people dying from COVID-19 in Germany currently per 16 day period (assuming 500 deaths per day).
65 / 1,000,000 - average number of people dying from COVID-19 across EU currently per 14 day period (see here).
2 / 1,000,000 - chance of being struct by lightning each year.
0.3 / 1,000,000 - fraction of people who got DIC in the UK & EEA after getting the AstraZeneca vaccine.
~3.0 / 1,000,000 - fraction of people who got DIC in Germany after getting the AstraZeneca vaccine.
100,000 / 1,000,000 (10%) - approximate risk of dying from COVID-19 (conditional on getting it) for those aged 70-79.
18,000 / 1,000,000 ( 1.8%) - approximate risk of dying from COVID-19 (conditional on having a positive test) averaged across age groups. Note: I calculated this in a naive way, by dividing the number of deaths reported in the US by # of cases reported. The case fatality rate varies widely between countries (between 0.25 - 8% according to one paper) and generally rates calculated in this way probably overestimate the true mortality risk from COVID-19 (by a factor of 2-10x?) due to asymptomatic cases that are never reported.

addendum - there’s a good NY Times article from March 19th on the sway of the deadly precautionary principle in Europe.

1. boring math details: I got this rate using the stats from this tweet (see the tweet’s thread for linked references and total numbers of vaccines given out in same period). I took the reporting window and subtracted 14 to compensate for the lack of a full 14 day follow-up at the end of window. Then, I divided the remaining number of days by 14. The result was that the window covered 3 2 week periods in full. I took the rate of clotting cases reported in that period (4/1,000,000 vaccinations given in the period) and divided it by 3 to get 1.3 / 1,000,000. The result of this calculation is an underestimate however, because it assumes all the people got the vaccine at the very beginning of the reporting window, which obviously isn’t true. (Most likely half way through the reporting window only ~50% of those people had actually gotten the vaccine). I didn’t bother to calculate the correction factor (although I think it could be done, assuming a linear uptake of vaccines), and instead multiplied by 2x to compensate for it. (Note the lowest the correction factor can be is 1x and the highest it can be is 3x, so 2x is reasonable).