FDA devastation during the pandemic - a review
We need a congressional commission now!
I do not think we have had a proper reckoning with what happened during the pandemic. It seems people just want to forget the entire thing.
2,977 people died on September 11th, 2001. The September 11th attacks led to the creation of the 9/11 Commission about a year later. That commission held high-profile public hearings and produced an extensive report that became widely read, selling over 800,000 copies.
The pandemic directly caused at least 1,130,000 deaths. That is the same as 379 9/11s. So where is the reckoning? Why no commission?
There was a major push for a COVID-19 commission, but ultimately it stalled on Capitol Hill. In early 2021 four foundations (Schmidt Futures, the Skoll Foundation, Stand Together, and The Rockefeller Foundation) created the COVID Commission Planning Group, led by former 9/11 commission director Philip Zelikow. Sadly, their website is now offline.
On March 15, 2022 the Senate’s health committee voted 20-to-2 in favor of the PREVENT Pandemics Act. Part of the act prescribed the establishment of “an independent task force to conduct a comprehensive review of the COVID-19 response”. The term “task force” was seen as less divisive-sounding than “commission”.1 Terminology aside, the act called for an independent committee with full subpoena powers, very similar to the 9/11 Commission.
Many in the Senate advocated for the bill, but it received no support from The White House. Many parts of the PREVENT Act were later adopted into the Consolidated Appropriations Act which was signed into law by Joe Biden on December 29, 2022. Sadly, though, the independent task force measure was dropped.
If we’re not going to have a comprehensive COVID-19 commission, I’d like to suggest something more targeted -- a commission to investigate the FDA’s response in particular.
I would hope that such a commission would be easier for Congress to pass. The FDA is pretty non-partisan by design. That should allay fears that the commission might become politicized.
Let’s review some of the major issues at the FDA during the pandemic:
The rapid test shortage
“We spent our own million dollars developing this thing, at their encouragement, and then they just treat you like a criminal.” - Chris Patterson, CEO of WHPM, Nov 2021.
The episode with rapid tests is a complex boondoggle. What follows is my attempt at an overview:
The first rapid test was developed in March 2020 by E25Bio. The inventors immediately contacted the FDA. They had a factory ready to go that could make 100,000 per week. However, the FDA didn’t put out guidance on rapid tests until July 2020. The guidance asked companies to validate their tests against PCR tests and also required the test’s sensitivity to be 90% as good as gold-standard 40x RT-PCR (later lowered to 80% in November 2021).
When the FDA finally approved the first rapid test in November 2020, it was only available by prescription. Many articles speculate that FDA officials were worried people were not smart enough to understand the risk of false negatives with such tests. This sounds right, but it’s also worth pointing out that PCR tests, which have very low negative rates, also required a doctor’s prescription initially as well.
We can look at timelines for two different tests (picked somewhat randomly) to get a sense of the delay here - Roche’s rapid test was authorized on Feb 26th, 2021 in Germany, on March 1st, 2021 in the UK, but not until December 24th, 2021 in the US. ACON’s test was authorized in Germany on April 8th, 2021, in the EU on May 14th, 2021, and in the US on October 4th, 2021. In general, dozens of rapid tests were approved in the EU in spring 2021 that were not approved in the US until winter 2021. That delay may not seem like a big deal looking back now, but at the time it was critical. When the delta variant hit in July 2021 other countries were awash in cheap rapid tests, but there were barely any available in the US. The first tests approved in the US were also about three times as expensive as tests in other countries ($15 vs ~$5).
So what happened?
It appears FDA officials fundamentally did not understand that it’s OK if rapid tests are not as sensitive as PCR tests. The chief benefit of rapid tests is that they can be administered easily at home and give immediate results. PCR tests, by contrast, require driving somewhere and then waiting 12 - 24 hours for a result. Rapid tests are actually quite sensitive during days ~2-4 of illness when patients are most infectious. The high sensitivity of PCR tests can actually be a problem, since people are flagged as “positive” far after they’ve stopped being infectious.
It appears much of went wrong is due to completely unnecessary delays by FDA officials. At least one scientist quit in protest. As Pro Publica reported:
“The FDA reviewer who quit this May described what the delays looked like from the inside. With a background in virology, he could evaluate the hundreds of pages in an application within a few days. But then, something strange happened: The applications would go nowhere for months as higher-up officials seemed paralyzed by indecision.”
I suspect that one of the culprits here is Jeffrey Shuren, who serves as the Director of the FDA’s device center. Dr. Shuren is the same character that appears responsible for screwing over MELA Sciences in 2010 (something I wrote about previously).
What should change? We need a commission to fully answer that question.
However, many changes have already been proposed. In a December 2021 Twitter thread, Michael Mina identified three changes that could speed up test approvals. First, he suggests acting immediately after EUAs happen in peer nations (reciprocity!). Second, he argues that new rapid tests should be compared to a gold standard rapid test like BinaxNOW, instead of requiring comparison with PCR. That would speed up trials. Third, he suggests the FDA should allow EUA trials to recruit people who have already tested positive. This would obviously speed things along as well.
Now, to the FDA’s credit, they did ask Booz Allen Hamilton to come in and do a postmortem on what happened with tests. Their lengthy report has a lot of granular recommendations.
Most likely larger systemic changes are necessary however, like having the emergency use pathway administered by a different agency.
Pfizer should have been authorized earlier
Maxwell Tabarrok has a fantastic article entitled “How Many People Are In The Invisible Graveyard?”. The article looks at how many people died because of delays in authorizing the Pfizer vaccine. The Pfizer vaccine is of particular interest because it had the largest trials that reported out their results first because of larger statistical power. Pfizer also had much larger manufacturing capacity than Moderna.
I think there is a strong case that the Pfizer vaccine should have been approved after their Phase II data came out on June 29th, 2020. (See footnote for details2)
According to Tabarrok’s analysis, that would have saved about 160,000 lives.
You might think that the delay was OK because Pfizer needed time to manufacture the doses. In actuality it appears Pfizer had millions of doses already manufactured in August. Furthermore, as both Tabarrok and economist Tyler Cowen have argued, if the FDA had made clear that an earlier approval was possible, then that would have led to vaccine production ramping up faster (elasticity of supply).
Even if one thinks that waiting for the Phase III trial was warranted, then in an ideal world the vaccine would have been authorized shortly after the Phase III trial’s read-out date, which was going to be in mid-October. According to Tabarrok, that would have saved 60,000 lives.
What actually happened is that on October 6th the FDA updated their guidance to require four additional weeks of follow-up in the Phase III trial, effectively pushing out any chance of an EUA until after the election.
The FDA did not approve Pfizer’s vaccine until December 11th, 2020, three weeks after Pfizer submitted their EUA application. Even if we just look at that three week delay, that is at least 10,000 lives lost.
All of Tabarrok’s estimates only consider first-order effects. They do not consider the second-order effects of slowed transmission and fewer variants of concern. A full investigation is needed to more accurately determine the full number of bodies in the invisible graveyard.
The AstraZeneca vaccine was never authorized
This, in my view, was the second or third most devastating thing the FDA did. This travesty was also wildly underreported. Part of the reason is there isn’t much to report on. All of the key decisions took place behind closed doors. There is zero transparency into the FDA’s decision making here.
Let’s briefly review what happened:
July 30, 2020: It is reported that the Trump administration was exploring a pathway by which the FDA could issue an authorization for the AstraZeneca vaccine in October. This causes pushback from Democrats.
October 9th, 2020: Out of the blue Nancy Pelosi tells reporters that the FDA should not approve the AstraZeneca vaccine on the basis of their first Phase III trial alone, which was conducted on patients from the UK, Brazil, and South Africa.
December 8th, 2020: Phase III efficacy data is published in The Lancet. (90% effectiveness)
December 30th, 2020: AstraZeneca’s vaccine is approved in the UK.
In an ideal world, AstraZeneca’s vaccine would have been approved in the US in December 2020. The vaccine was by far the cheapest ($4 a shot) and did not require ultra-cold storage like the mRNA vaccines. In late January 2021 we learned that a facility in Baltimore had millions of doses stockpiled and was manufacturing millions more. In March the New York Times estimated there were 28 million doses there.
I imagine some people may be tempted to say that everything worked out OK because AstraZeneca was later shown to be a “sup-par” vaccine and was paused in the EU due to concerns about clotting. Well, as analyses have shown, that pause was a massive mistake, costing many lives. The clotting risk is about equivalent across all the vaccines, but AstraZeneca appears to have been targeted by EU countries for political reasons. All of the vaccines have similar efficacy at preventing hospitalization and death, with some evidence showing AstraZeneca is actually superior to Pfizer in that regard.
Anyway, this brings us to our next topic:
We never donated our extra AstraZeneca vaccines
By April 2021 the US really had no need for the 28+ million AstraZeneca doses that were languishing in a Baltimore warehouse.3 Vaccine supply had caught up with demand. At that point we should have immediately donated all of them to India. A big wave was hitting India, with 300,000 cases and 2,500 reported deaths each day (the actual number was probably several times higher). Even a very conservative estimate shows that donating those doses could have saved ~5,000 lives.4
While this failure lies directly with The White House, I also blame this failure on the FDA because I suspect if AstraZeneca had been authorized then any extra doses would have been donated right away.
Addendum (added 1-29-24): someone pointed out that it later came out that Emergent, the company manufacturing AstraZeneca doses in Baltimore, had serious issues in their manufacturing plant. It’s quite possible the FDA knew about at this at the time and that is why the doses were not donated. This also may tie into why AstraZeneca was not authorized, but I still think there is no good excuse for that.
The FDA’s crackdown on laboratory developed tests
"Dr. Stephen Hahn, 60, the commissioner of the Food and Drug Administration, enforced regulations that paradoxically made it tougher for hospitals, private clinics and companies to deploy diagnostic tests in an emergency," - The New York Times on March 28, 2020.
In January 2020, many scientists wanted to start testing for COVID-19 using PCR. Many started to do so without government approval. The FDA then began to issue cease-and-desist letters. Prior to the pandemic, laboratory developed tests (LDTs) were never FDA regulated. The FDA had for a long time claimed they have the right to regulate such tests but had always chosen not to (a view that many legal scholars completely reject). That situation suddenly changed when the pandemic started and the FDA began to require approval. As Alex Tabarrok explained, the FDA “reversed the logic of emergency.”
The CDC then tried to develop a standard test and failed. Former CDC Director Tom Frieden has said that the situation with PCR tests was “a perfect storm of three separate failures,” noting the crackdown on private labs, the botched test at the CDC, and overstrict FDA rules. The delay caused by these failings is widely acknowledged to have left the US “blind” during the start of the pandemic. Many lives were undoubtedly lost as a result.
Distilleries could not produce hand sanitizer due to FDA regulations
We now know that COVID-19 isn’t transmitted much on surfaces, but it very well could have been different. We really didn’t know at the time. To their credit, very early in the pandemic (around February 2020) the FDA loosened their rules regarding hand sanitizer production. However, they still required the addition of a denaturant to the alcohol. This was a big problem for distilleries, who wanted to help meet the growing demand for hand sanitizer. Introducing a denaturant to their lines would have rendered them useless for future alcohol production barring extremely costly cleaning procedures. Reason Magazine had an article about this in April 2020.
The FDA made it hard to import masks from other countries
Unlike N95 masks for construction workers, masks for healthcare workers have to be manufactured on a production line certified by the FDA. To their credit, on March 19th, 2020 the FDA loosened their regulations to allow industrial N95 masks to be sold to healthcare workers. Unfortunately, the FDA made it nearly impossible to import masks from other countries. Chinese manufacturers had much larger capacity and could also manufacture N95s at three times lower cost. This led to “grey markets” popping up. I believe this situation was eventually improved after a few months, but a severe shortage of masks (especially N95s) persisted for quite some time.
The FDA barred their inspectors from traveling
“Grocery store workers are working, meat packers are working, hell bars and restaurants are open in many parts of the country but FDA inspectors aren’t inspecting. It boggles the mind.” - Alex Tabarok, Marginal Revolution.
The New York Times wrote about this in March 2021. FDA officials prevented many of their inspectors from traveling during the pandemic, citing guidelines from the CDC which discouraged travel. As a result the roll-out of several important anti-cancer drugs was delayed months because inspectors could not inspect their manufacturing facilities. At least 52 drug applications were delayed.
The FDA never endorsed challenge trials
"Right now, it gives people some ethical heartburn and scientifically, it's complicated.” - FDA CBER Director Peter Marks on challenge trials, July 2020.
In April 2020 35 federal lawmakers called on the FDA to consider human challenge trials. Over 1,500 volunteers had signed up with 1DaySooner at that time. According to a report, an FDA spokesman “expressed skepticism” and said that the idea “flies in the face of most federal regulations on protection of human research subjects.”
The UK approved human challenge trials in February 2021. The UK’s trial was obviously way too late, but it was a valuable proof-of-concept. 18 people were deliberately infected, 16 of whom developed symptoms. None had any long-term consequences to their health. Part of the trick was infecting people with the lowest possible dose (essentially variolation, something Robin Hanson argued for in February 2020). In many viral illnesses, symptom severity appears to be correlated with initial viral load, and COVID-19 seems to be the same way.
However, I put this last because I’m not sure how much we can blame the FDA for this failing. Bioethicists came out in droves to publish papers denouncing challenge trials. It’s also very unlikely that local IRBs would stomach challenge trials. So FDA officials may have looked at all that and decided it wasn’t going to happen regardless.
Conclusion
I hope it is clear why we need a fully-empowered independent commission to investigate the FDA’s failings and make recommendations to congress.
The good news is that all of the problems outlined here appear avoidable in future pandemics. We just need the political will to legislate the necessary changes.
A overarching theme here is that the original intent of the Emergency Use Authorization legislation was not followed by the FDA. Instead, the FDA tried to follow their normal procedures but on a compressed timeline. Read Nikki Teran’s excellent article “Taking Emergency Use Authorization Seriously” for more on that. New legislation is clearly needed.
For further reference see my vaccine approval timeline.
Thank you to Ben Ballweg for doing a very thorough proofreading of this post.
According to the New York Times, “The measure avoids the use of the word ‘commission,’ which acquired negative connotations in Washington after a bitter partisan debate doomed an effort to create a commission to investigate the Jan. 6 Capitol attack. Instead, the panel is called a ‘task force’..”
The Phase II data showed a strong immunogencity response, even in older folks. According to virologists, this means that it’s very likely for the vaccines to have high efficacy. I have been told that historically 85% of vaccines that pass Phase II end up passing Phase III. On this prior, the vaccine should have been made available to high-risk individuals after Phase II. Post-hoc data confirms this - 50 vaccines have been approved after Phase III and only three topped during or after Phase III. That is a 94% success rate so far after passing Phase II.
Except if we wanted to encourage heterologous boosting. (Allowing people to get one dose of mRNA vaccine and one dose of an adenovirus vaccine like AstraZeneca).
Fermi estimate details:
Assumptions based on April 2021 data:
The true number of COVID-19 deaths was about 5x what was reported, so 15,000 per day.
91.4% of Indians were completely unvaccinated (1,188 M people).
From these two assumptions, the chance of dying each day for the average person was 1 in 79200. Let’s assume this held for a month.
We assume the AstraZeneca vaccine has 95% effectiveness at preventing death.
Vaccines were being injected at a rate of about 2 M per day. However, injection rate capacity had likely not maxed out. We assume a delivery of 28 M AstraZeneca doses to India would increase the number of vaccines injected by 1 M per day for 28 days.
Being very conservative, let’s just integrate the number of dose-days for those 28 days. We get 406 M dose days. Multiplying that by 0.95 (effectiveness) and then by (1/79200) we get 4,869 lives saved.
Excellent and powerful analysis! Clearly, the FDA approach and consequences need to be thoroughly reviewed and investigated and lessons learned for future pandemic & similar emergency decision making and response. The whole COVID matter is a different story, of which the FDA issues are a 'tip -of-the-iceberg' indicator. The clear absence of a 9/11 like commission with unlimited subpoena power was too dangerous for too many. With literally trillions sloshing around controlled by a certain 'family', this was never going to happen.
Separately, the questions of right and wrong, ethics and morals are perennial challenges. This is a matter for all of humanity to deal with. It's been around forever. Minor improvement keep occurring with occasional back sliding.
In case, I applaud you highly logical, persistent, and persuasive approach.
Very convincing.