The announcement came on November 18, 2020. Having only just begun studying the effects of its novel mRNA vaccine eight months earlier, Pfizer—with great fanfare—announced that its drug was “95% effective against COVID-19″.
In support of its claim, Pfizer submitted an article to the New England Journal of Medicine (NEJM) which summarized the data collected during Phase 2/3 of the clinical trial. NEJM published the article on December 10, 2020. It was on the basis of conclusions drawn from the safety and efficacy data that the Food and Drug Administration (FDA) authorized Pfizer’s COVID-19 vaccine for emergency use. (See here for the NEJM article. All quotes below are taken from this article, unless otherwise indicated.)
Yet a close reading of the article reveals that Pfizer’s “95%” claim is deceptive.
Pfizer pulls their awesome statistic from a magic hat, as it were, by performing two tricks: (1) defining and redefining a case of COVID-19 such that the meaning of the term deviates from common usage and is thus conducive to misrepresentation of the trial data; and (2) reporting only a measure of relative risk reduction, which is not very meaningful unless the absolute risk reduction is also reported—something Pfizer failed to do in its press release and the NEJM article.
(Note: Less discernible tricks might still be buried in the full text or hyperlinked supporting documents. The two tricks I focus on are easy to see. I’m grabbing low-hanging fruit here.)
What’s in a Case?
Until the time of Pfizer’s announcement, and before its experimental vaccine became widely available to the public, a COVID-19 case was determined and reported on the basis of a positive PCR test, despite the test’s widely known tendency to produce false positives. (Of course, this tendency could have been corrected by setting the tests to a lower cycle threshold and testing multiple times over a five-day period [the average incubation period], without duplication of reports. But instead of making this correction in a genuine effort to understand the true prevalence and lethality of the virus, the public heath establishment inexplicably chose not to address the issue, thereby allowing the artificial inflation of case and death counts.) Right or wrong, the PCR test—and that alone—was the decisive diagnostic factor. This was common knowledge.
(Note: This was not always the case, especially in the spring of 2020, when tests were scarce. In response to this scarcity, the Centers for Disease Control and Prevention [CDC] instructed hospitals and health clinics to report probable or “suspected” cases of COVID-19 when no test was available—yet another reason to look askance at official COVID statistics—not to mention the fact that the CDC made no distinction between people who died “with” the virus and those who died “from” it.)
For its clinical trial, however, and more particularly its first primary efficacy calculation, Pfizer adopted a different and rather curious definition of a COVID-19 case. As defined by Pfizer, a “confirmed COVID-19” case occurs when there is at least one of a number of specified symptoms, in addition to a positive PCR test:
“Confirmed Covid-19 was defined according to the Food and Drug Administration criteria as the presence of at least one of the following symptoms: fever, new or increased cough, new or increased shortness of breath, chills, new or increased muscle pain, new loss of taste or smell, sore throat, diarrhea, or vomiting, combined with a respiratory specimen obtained during the symptomatic period or within 4 days before or after it that was [sic] positive for SARS-CoV-2 by nucleic acid amplification–based testing, either at the central laboratory or at a local testing facility (using a protocol-defined acceptable test).”
Thus, what would have counted as a “case” in the real world (a positive PCR test without symptoms) was, by design, excluded from the numerator in Pfizer’s efficacy calculation.
What’s the meaning of this?
First, it implies that Pfizer is unconcerned about asymptomatic infection. This is puzzling because asymptomatic infections represented a large fraction of cases then being reported. Recall the sky-high case numbers that scrolled across our TV screens on a daily basis throughout the second half of 2020—numbers that helped fan the flames of hysteria and were cited repeatedly as justification for expediting the clinical trials. Many people who returned to work or who went to the hospital for an unrelated reason found out they were “infected” by taking the PCR test. All these asymptomatic people were then classified as COVID-19 cases and reported as such, with reports of a single case frequently being duplicated as people were tested and retested multiple times.
Recall that it was precisely this ginned up fear of asymptomatic transmission that prompted calls for universal mask mandates, requiring people who were not even sick or experiencing symptoms to cover their faces. These people could be carriers of disease and infect others, we were told—despite the fact that Dr. Fauci (before he contradicted himself) stated that “an epidemic is not driven by asymptomatic carriers”. It seems odd, then, that Pfizer would adopt a definition of a case that excludes asymptomatic infections and thus a significant number of actual cases.
Second, this narrow definition of a case imposes certain limits on what Pfizer can say about the efficacy of its drug. So, what are the implications of using this definition?
It is hard to overstate the importance of this: Pfizer, by using this definition, is not in a position to claim that the drug being tested prevents infection. In this case, “vaccine efficacy” cannot refer to the drug’s ability to stop infection per se. It can only refer to the drug’s ability to stop a subset of infections: infections accompanied by at least one of a number of specified symptoms that occur within a very limited time range. In the NEJM article, Pfizer acknowledges this limitation in its discussion—an admission which comes at the end of the paper: “These data do not address whether vaccination prevents asymptomatic infection; a serologic end point that can detect a history of infection regardless of whether symptoms were present (SARS-CoV-2 N-binding antibody) will be reported later.”
There you have it—straight from the horse’s mouth.
But this was not how Pfizer’s experimental drug was sold to the public. The selling point was the unqualified conclusion trumpeted in the abstract of Pfizer’s NEJM article: A “two-dose regimen of BNT162b2 [the experimental vaccine] conferred 95% protection against Covid-19 in persons 16 years of age or older.” This blatant misrepresentation of the data—this awesome statistic—became the focal point of a fraudulent global marketing campaign. Pfizer’s failure to be forthright in discussing the limitations of its trial data, specifically its definition of a case of “confirmed COVID-19” and how it affects the evaluation of efficacy, amounts to intellectual dishonesty. Pfizer thus promoted its drug by encouraging the formation of a false impression in the minds of unsuspecting people around the world.
Many questions remain unaddressed in the public discourse surrounding Pfizer’s trial results. For instance, how many asymptomatic infections got swept under the rug when Pfizer did its primary efficacy calculation? How would these infections, had they been counted as cases, have affected the efficacy assessment? It is interesting to note that two of Pfizer’s secondary outcome measures (measures 19 and 20) do count asymptomatic infections as cases, particularly those that occurred after the second dose. (A description of all primary and secondary outcome measures can be viewed at the National Institute of Health’s clinical trials site.) But there is no mention of these measures in Pfizer’s press release or the NEJM article. Why did Pfizer include asymptomatic infections as bona fide cases for secondary measures, while excluding these same cases from its primary measure? Pfizer offers no explanation. One would think that such data on infections would have some bearing on the assessment of the drug’s ability to stop infections.
Nevertheless, Pfizer conjured up its awesome statistic in accordance with its first primary endpoint. Pfizer describes this endpoint in the NEJM article: “The first primary end point was the efficacy of BNT162b2 [the vaccine being tested] against confirmed Covid-19 with onset at least 7 days after the second dose in participants who had been without serologic or virologic evidence of SARS-CoV-2 infection up to 7 days after the second dose”.
Notice that the endpoint excludes a subset of confirmed COVID-19 cases. Not included in the case count are symptomatic infections recorded outside the reporting period: the period between the first dose and up to seven days after the second dose (i.e., twenty-eight days after the initial dose, with twenty-one days separating the first and second doses). This omission of a number of confirmed cases from the endpoint further limits what Pfizer can say about efficacy.
To its credit, Pfizer does mention in its press release and the NEJM article that the awesome statistic refers to the drug’s performance twenty-eight days after the first dose. Pfizer thus acknowledges one of the trial limitations imposed by the definition of its terms. (Of course, such acknowledgement did not even materialize as fine print in Pfizer’s ubiquitous advertising campaign, not to mention the strange and total absence of any mention of possible side effects in all those gimmicky commercials, posters, and billboards.) However, there is another layer of deception here. Besides the omission of an untold number of asymptomatic and symptomatic infections mentioned above, it is likely that additional symptomatic infections were excluded from the efficacy calculation. These other infections occurred within the reporting period, arising twenty-eight days after the first dose (i.e., seven days after the second dose).
How did these cases go unrecognized?
Recall that Pfizer used the FDA’s criteria to define “confirmed COVID-19” for its first primary endpoint. The FDA’s criteria included the following symptoms: (1) fever, (2) new or increased cough, (3) new or increased shortness of breath, (4) chills, (5) new or increased muscle pain, (6) new loss of taste or smell, (7) sore throat, (8) diarrhea, (9) vomiting. However, in regard to certain secondary outcome measures not reported in the NEJM article (for instance, see measures 14 to 17), Pfizer changed its definition of confirmed COVID-19. It did so by dropping the FDA criteria in favor of “CDC-defined symptoms.” CDC-defined symptoms included (1) fever or chills, (2) shortness of breath, (3) cough, (4) muscle or body aches, (5) new loss of taste or smell, (6) sore throat, (7) diarrhea, (8) nausea or vomiting, (9) fatigue, (10) headache, (11) congestion or runny nose. (For CDC-defined symptoms, see here.)
Now compare these two lists. Included on the CDC’s list, but missing from the FDA’s, are “fatigue,” “headache,” and “congestion or runny nose.” The conclusion we draw from this discrepancy is that trial participants who became infected with SARS-CoV-2 and experienced fatigue, a headache, or a runny nose—or any combination of the three—without also reporting any of the symptoms on the FDA list, were not counted as “confirmed COVID-19” cases relative to the first primary endpoint. Consequently, these symptomatic infections entered the efficacy calculation as successes (i.e., false negatives) rather than as true failures. This has the foreseeable effect of inflating the denominator while shrinking the numerator, thus making the drug look more effective than it really is.
Revision of Pfizer’s Efficacy Claim
It seems strange that Pfizer would effectively change horses in midstream by redefining a key concept for a number of secondary measures. Moreover, it remains unclear why Pfizer would refer to the more restrictive FDA list of symptoms rather than the more inclusive CDC list in counting cases for its efficacy assessment. Why would Pfizer change the definition of confirmed COVID-19 for any of these measures? Pfizer offers no explanation.
Pfizer’s silence on this matter forces one to speculate, but here I speculate on solid ground. It seems very likely that Pfizer uses multiple definitions of confirmed COVID-19 in order to hedge against an unfavorable result. If one definition does not facilitate the desired outcome, then perhaps another one will. With multiple definitions in play, Pfizer now has reporting options. Such a maneuver has to strike the critical, honest thinker as conceptual sleight of hand and a mark of bad faith.
With all this in mind, a revision of Pfizer’s efficacy claim is in order.
Recall the initial claim: A “two-dose regimen of BNT162b2 conferred 95% protection against Covid-19 in persons 16 years of age or older.” Or, as effectively communicated to the public: Pfizer’s drug is “95% effective against COVID-19.”
Now here is the claim revised in light of the trial’s limitations: Two doses of Pfizer’s mRNA vaccine reduced by ninety-five percent the probability of becoming infected with the original SARS-CoV-2 and experiencing a number of FDA-specified symptoms, beginning twenty-eight days after the first dose and up to a median of two months after the second dose, in persons sixteen years of age or older and who had not already been infected with SARS-CoV-2 (prior to the trial) and “who had been without serologic or virologic evidence of SARS-CoV-2 infection up to 7 days after the second dose.”
There is a world of difference between these two claims. The former amounts to deceptive sloganeering that glosses over the trial’s shortcomings. As such, it misrepresents the data, obscuring more than it illuminates. The latter is wordier and less punchy, but it comes closer to truthfully representing the data. It cuts Pfizer’s awesome statistic down to size by putting it in the context of the study design and its limitations.
I should point out that Pfizer’s study design excluded people who had previously been infected and recovered from COVID-19. In other words, Pfizer did not study the effects of its drug on people who had already acquired some degree of natural immunity to SARS-CoV-2. Therefore, the happy conclusions Pfizer draws from the trial data (however questionable they may be) certainly do not apply to a large fraction of the population. It is reckless and unethical to push these shots on people for whom the drug is completely untested—or, for that matter, anyone else who prefers not to be an unpaid lab rat for Big Pharma.
Discussion: The Semi-Attached Figure
The deception described above is an example of what statisticians call the “semi-attached figure.” As the author of How to Lie with Statistics puts it, “If you can’t prove what you want to prove, demonstrate something else and pretend that they are the same thing. In the daze that follows the collision of statistics with the human mind, hardly anybody will notice the difference” (see p.74).
Having come this far in exploring Pfizer’s sleight of hand, you can count yourself among the minority of people who notice the difference.
What I find fascinating and at the same time stupefying is that when the real-world data on “breakthrough cases” crashed the party and made the deception impossible to maintain, the head of the CDC and the CEO of Pfizer both admitted that the mRNA vaccines can no longer stop infection, particularly infection by variants. Of course, anyone who read the NEJM article would have known already that Pfizer, in fact, failed to demonstrate that the mRNA shots stop infection in the first place. And yet, despite reality reasserting itself, we continue to be pressured, if not required by our employers, to get vaccinated and boosted, as if the CEO of Pfizer never admitted that the shots have failed to work as advertised.
Note also that the average follow-up period in Pfizer’s trial was two months. Pfizer therefore cannot say anything conclusive about the effects of its mRNA vaccine two months after the second dose. No one, in fact, is in a position to know the long-term effects of this novel technology. The experiment is ongoing. As Pfizer acknowledges in its paper, the trial does not end with the reporting of these results. Furthermore, the experiment is now uncontrolled. Amazingly, Pfizer removed the blinds from the control group and offered the mRNA shots to participants who had initially received the placebo: “Participants who originally received placebo will be offered the opportunity to receive BNT162b2 at defined points as part of the study”. Elsewhere in the discussion, we find this statement: “[M]ost participants who initially received placebo have now been immunized with BNT162b2, ending the placebo-controlled period of the trial [emphasis added].”
What does this mean exactly? It means that, by eliminating the control group, Pfizer has destroyed the scientific basis of its trial. Yet public health officials and regulators ignore this plainly stated fact. In their strange eagerness to promote Pfizer’s experimental product, these officials continue to harangue the public about the necessity of getting these shots, as if the vanishing, short-term benefit of an increasingly ineffective drug, formulated to protect against a variant no longer circulating, somehow outweighs the long-term risks of receiving multiple injections in a vast, ongoing, and uncontrolled experiment. What could go wrong?
Risky Business
We now come to another layer of deception in the reporting of trial results. This deception comes in the form of Pfizer producing a measure of relative risk reduction, without mentioning absolute risk reduction. Here, it is instructive to make explicit what remains implicit in the data, since Pfizer is less than forthcoming about what it is actually reporting.
To be clear, the 95% figure is a measure of relative risk reduction (RRR). It means someone who takes Pfizer’s drug is 95% less likely to get “confirmed COVID-19” than someone who does not take the drug.
That sounds impressive.
Of course, it sounds less impressive when we consider that Pfizer’s definition of “confirmed COVID-19” excludes an unknown number of asymptomatic infections and symptomatic cases, as noted above. The claim is even less impressive when we calculate absolute risk reduction.
Absolute risk reduction (ARR), also known as risk difference, tells us the proportion of trial participants who avoided a bad outcome as a result of having been exposed to the treatment rather than the placebo. We find ARR by subtracting the observed risk in the experimental (treatment) group from the observed risk in the control (placebo) group. The result is the risk difference.
Since Pfizer does not report ARR in the NEJM article or its press release, it is up to us to figure it out—which we can do. All the information we need to calculate ARR is available in the article. But before we do the calculation, we should understand why knowing ARR is important. Some background is in order.
For one thing, by failing to report ARR, Pfizer stands in violation of the Consolidated Standards of Reporting Trials (CONSORT). CONSORT is the industry benchmark and checklist of best practices dedicated to ensuring transparency and replicability of clinical trial results. CONSORT item 17b states: “For binary outcomes, presentation of both absolute and relative effect sizes is recommended”
The CONSORT Group elsewhere explains the importance of ARR:
“When the primary outcome is binary, both the relative effect… and the absolute effect (risk difference) should be reported (with confidence intervals), as neither the relative measure nor the absolute measure alone gives a complete picture of the effect and its implications. Different audiences may prefer either relative or absolute risk, but both doctors and lay people tend to overestimate the effect when it is presented in terms of relative risk. The size of the risk difference is less generalizable to other populations than the relative risk since it depends on the baseline risk in the unexposed group, which tends to vary across populations. For diseases where the outcome is common, a relative risk near unity might indicate clinically important differences in public health terms. In contrast, a large relative risk when the outcome is rare may not be so important for public health (although it may be important to an individual in a high-risk category)” (emphasis added).
Reinforcing this CONSORT recommendation are the authors of “Common Pitfalls in Statistical Analysis: Absolute Risk Reduction, Relative Risk Reduction, and Number Needed to Treat”—a paper which appeared in Perspectives in Clinical Research in early 2016. In this paper, the authors stress the importance of ARR: “Physicians tend to over-estimate the efficacy of an intervention when results are expressed as relative measures rather than as absolute measures. ARR (expressed along with baseline risk) is probably a more useful tool than RRR to express efficacy of an intervention. Thus, reporting of absolute measures is a must.”
The authors go on to explain the dangers of viewing RRR in isolation: “The problem with using RRR is that we cannot assess the actual effect size if the event rate in the control group is not known. A particular RRR may thus imply very different ARRs, depending on the baseline risk [risk in the control group]. For instance, a 50% RRR may represent an ARR of 40% (if the absolute risk comes down from 80% to 40%), a major effect, or of only 1% (if the absolute risk comes down from 2% to 1%), probably an inconsequential effect size” (emphasis added).
In this example, we see the inadequacy of RRR in the absence of ARR. We see how an impressive-sounding RRR can be misleading in terms of effect size. We need to know ARR in order to put RRR in meaningful context. Otherwise, RRR can be worse than useless; it can be deceptive.
The question now arises: Do we see similar circumstances in Pfizer’s trial results? What is the magnitude of ARR implied by the heralded 95% RRR?
Let’s do the math.
DIY Efficacy Calculation
Since we are using numbers provided by Pfizer, let’s pretend for a moment that nothing sketchy is going on here that would complicate our calculation or interpretation of results. Let’s assume we are dealing with valid data collected in a straightforward manner to assess vaccine efficacy. We are not going to worry about the issues noted above, particularly Pfizer’s odd definition and redefinition of confirmed COVID-19. Let’s just see if Pfizer’s awesome statistic can withstand scrutiny on Pfizer’s own terms.
As noted above, we find ARR (absolute risk reduction or risk difference) by subtracting the risk in the experimental group from the risk in the control group. Risk is represented as the frequency of “confirmed COVID-19.”
With regard to the first primary endpoint, Pfizer reports eight (8) cases of confirmed COVID-19 out of 18,198 participants in the experimental group. The risk in the experimental group is 0.04% (8 ÷ 18,198 = 0.00043 = 0.04%).
In the control group, there are one hundred and sixty-two (162) cases out of 18,325 participants. Risk in the control group is 0.88% (162 ÷ 18,325 = .00884 = 0.88%).
Here, ARR is 0.84% (.0088 – .0004 = .0084 = 0.84%).
Dividing ARR by the baseline risk (risk in the control group) gives us RRR. Hence, RRR is 95% (.0084 ÷ .0088 = .9545 = 95%).
Note: Pfizer uses a slightly different method to arrive at RRR. For RRR, Pfizer subtracts from one (1) the ratio of confirmed cases (c1) per 1000 person years of follow up (y1) in the experimental group to the corresponding infection rate in the control group (c2/y2). Multiplying the result by a hundred gives us the percentage. Here’s the equation: RRR = 100 * (1 – [(c1/y1) ÷ (c2/y2)]). Despite the difference in methods, we get the same result (95% RRR). (See here for standard methods of calculation).
Thus, with absolute risk falling from 0.88% to .04%, we see that the 95% RRR in Pfizer’s trial represents an ARR of 0.84%.
Discussion
If an ARR of one percent is “probably an inconsequential effect size,” as the authors quoted above suggest, then what are we to make of an ARR that is less than one percent (in this case, 0.84%)? It is indeed hard to reconcile this small effect size with the global campaign to mandate the mRNA shots for everyone, regardless of individual risk, especially when the long-term risks of these injections remain unknown.
Notice also that confirmed COVID-19 was a rare occurrence in both the experimental group and control group. According to Pfizer, less than one percent of participants in each group came down with confirmed COVID-19. Severe cases were extremely rare. All together there were ten severe cases out of 43,355 participants (.0002%), with nine in the control group and one in the treatment group.
Recall the statement made by the CONSORT Group regarding rare outcomes in clinical trials: “…a large relative risk when the outcome is rare may not be so important for public health (although it may be important to an individual in a high-risk category).”
Again, it is hard to reconcile the study results with the global campaign to vaccinate everyone on the planet. Here we have a large measure of relative risk: 95% RRR. However, in light of ARR (0.84%), the importance of RRR for public health is questionable, especially with regard to healthy adults and children—not to mention six-month-old babies—all of whom are virtually at no risk of a severe outcome. What remains unquestionable is that RRR in isolation—even a large one—is inadequate for assessing efficacy. As such, Pfizer’s awesome statistic hardly justifies coercive mandates or undue pressure on people to take these experimental shots. (Of course, we are leaving aside concerns about individual rights and constitutional limits on governmental authority).
It is hard to imagine an innocent explanation for Pfizer’s failure to provide a calculation of absolute risk reduction. Pfizer not only fails to do basic math in plain sight for all to see, but also violates industry standards designed to promote transparency. Now that we know the importance of both RRR and ARR in discussions of clinical trial results, we see that Pfizer’s failure to mention ARR, while trumpeting a large RRR, amounts to scientific fraud, a lie by omission.
This is intellectual dishonesty at its worst.
Conclusion
Pfizer has a lot of explaining to do—as do Pfizer’s partners at the FDA. At this point, the FDA has to be seen as a captured agency. (If there is a term that Americans need to incorporate into their vocabulary, it is regulatory capture [see here].) Hundreds of millions of people rolled up their sleeves on the basis of Pfizer’s awesome statistic. They trusted government officials and Pfizer to tell the truth about the mRNA shots. They were led to believe that they could protect themselves and stop the spread of COVID-19 by getting the jab. That’s what we were told.
Deborah Birx, the former response coordinator for the White House Coronavirus Task Force, recently told Fox News that she “knew” the COVID vaccines would not “protect against infection,” that she and other public health officials “overplayed” the vaccines. I wonder how she knew all this. Did she read Pfizer’s NEJM article? Regardless, if she did know these things, why didn’t she speak out about what she knew when vaccine mandates were being imposed on millions of Americans?
Now more and more people are getting boosters mere months after receiving the initial two doses. Many of these people still believe they are following “The Science.” At this point, they are just following “The Politics.”And, most likely, they have not read Pfizer’s NEJM article. Otherwise, they would know that Pfizer’s awesome statistic is a semi-attached figure and the culmination of a taxpayer funded magic show: a carefully crafted illusion involving conceptual sleight of hand and lies by omission.
Here, as usual, the devil lurks in the details. The more one digs into the data, the harder it is to escape the conclusion that a con game is underway, and we are the marks. The unexplained failure to report absolute risk reduction is itself a strong indication of scientific fraud, as the omission of ARR violates industry standards and best practices. It’s not as if Pfizer was unaware of its industry’s own standards. There is also the no small matter of Pfizer eliminating the control group from its trial. This move torpedoes the scientific basis for any evaluation of the long-term effects of taking Pfizer’s drug. All these shortcomings serve to help Pfizer and its partners overstate the efficacy of the mRNA shots, while maintaining a facade of scientific respectability.
I invite everyone to look into the data for themselves and come to their own conclusions. And, above all, question authority—always question authority—especially an authority that insists on not being questioned.