About confidence intervals for the Biontech/Pfizer Covid-19 vaccine candidate

Probably, many of you have read the positive news from the Biontech/Pfizer press release[1] from November 9th:

“Vaccine candidate was found to be more than 90% effective in preventing COVID-19 in participants without evidence of prior SARS-CoV-2 infection in the first interim efficacy analysis”

Not being a biostatistician, I was curious how vaccine efficacy is exactly measured. Also, how does the confidence interval look like? Helpfully, Biontech and Pfizer also published a detailed study plan here[2].

The sample vaccine efficacy can be defined as

[VE = 1-IRR = 1 – frac{c_v/n_v}{c_p/n_p}]

where $n_v$ and $n_p$ are the number of subjects that got a Covid-19 vaccine and a placebo, respectively, while $c_v$ and $c_p$ are the respective number of subjects that fell ill to the Covid-19 disease. IRR stands for incidence rate ratio and measures the ratio of the share of vaccinated subjects that got Covid-19 to the share in the placebo group.

The press release stated that so far 38955 subjects got the two doses of the vaccine or placebo of which 94 subjects fell ill with Covid-19. Furthermore, the study plan stated that the same number of subjects was assigned to the treatment and control group and let’s assume that also in the 38955 subjects analysed so far the ratio is almost equal. An efficacy of at least 90% then implies that from the 94 subjects with Covid-19 at most 8 could have been vaccinated.

The following code computes the IRR and vaccine efficacy assuming that indeed 8 vaccinated subjects got Covid-19.

n = 38955 # number of subjects
nv = round(n/2) # vaccinated
np = n-nv # got placebo
# number of covid cases
cv = 8
cp = 94-cv
# percentage of subjects in control group
# who got Covid-19
## [1] 0.44
# percentage of vaccinated subjects
# who got Covid-19
## [1] 0.04
# incidence rate ratio in % in sample
IRR = (cv/nv)/(cp/np)
## [1] 9.3
# vaccine efficacy in % in sample
VE = 1-IRR
## [1] 90.7

Assume for the moment this data came from a finished experiment. We could then compute an approximative 95% confidence interval for the vaccine efficacy e.g. using the following formula described in Hightower et. al. 1988[3]

arv = cv/nv
arp = cp/np
# CI for IRR
ci.lower = exp(log(IRR) - 1.96 * sqrt((1-arv)/cv + (1-arp)/cp)) 
ci.upper = exp(log(IRR) + 1.96 * sqrt((1-arv)/cv + (1-arp)/cp)) 
IRR.ci = c(ci.lower, ci.upper)
## [1]  4.5 19.2
VE.ci = rev(1-IRR.ci)
## [1] 80.8 95.5

This means we would be 95% confident that the vaccine reduces the risk of getting Covid-19 between 80.8% and 95.5%. As far as I understood, e.g. the function ciBinomial in the package gsDesign[4] allows a more precise computation of the confidence interval:

IRR.ci = ciBinomial(cv,cp,nv,np,scale = "RR")
VE.ci = rev(1-IRR.ci)
##   upper lower
## 1  81.1  95.4

Given that it is only stated that the vaccine is more than 90% effective, the number of Covoid-19 cases may also have been lower than 8 subjects.

The next clean threshold for a press statement would probably be at least 95% effectiveness, which would be exceeded if only 4 vaccinated subjects had Covid-19. So it also seems well reasonable that only 5 vaccinated subjects had Covid-19. This would yield the following vaccine efficacy and confidence interval:

cv = 5; cp = 94-cv
VE = 1-(cv/nv)/(cp/np)
## [1] 94.4
IRR.ci = ciBinomial(cv,cp,nv,np,scale = "RR")
VE.ci = rev(1-IRR.ci)
##   upper lower
## 1  86.6  97.7

Looks even better.

However, those confidence intervals assume a finished, non-adaptive experiment. Yet, the interim evaluations are triggered when the number of Covid-19 cases among the subjects exceeds certain thresholds. The press release states:

“After discussion with the FDA, the companies recently elected to drop the 32-case interim analysis and conduct the first interim analysis at a minimum of 62 cases. Upon the conclusion of those discussions, the evaluable case count reached 94 and the DMC performed its first analysis on all cases.”

“The trial is continuing to enroll and is expected to continue through the final analysis when a total of 164 confirmed COVID-19 cases have accrued.”

I am no expert, but possible the calculation of the confidence interval is not valid for such adaptive rules where the evaluation is triggered by the number of disease cases.

Indeed, Biontech and Pfizer state that they will the assess the precision of the estimated vaccine efficacy using a Bayesian framework with a particular prior distribution described on p. 102-103[5] of their study plan. Alas, I know very little of Bayesian analysis so I abstain from computing the posterior distribution given the data at hand.

But even absent the full-fledged Bayesian analysis, the numbers really look like very good news.


  1. ^ press release (investors.biontech.de)
  2. ^ here (pfe-pfizercom-d8-prod.s3.amazonaws.com)
  3. ^ Hightower et. al. 1988 (apps.who.int)
  4. ^ gsDesign (cran.r-project.org)
  5. ^ p. 102-103 (pfe-pfizercom-d8-prod.s3.amazonaws.com)

Source URL: Read More
The public content above was dynamically discovered – by graded relevancy to this site’s keyword domain name. Such discovery was by systematic attempts to filter for “Creative Commons“ re-use licensing and/or by Press Release distributions. “Source URL” states the content’s owner and/or publisher. When possible, this site references the content above to generate its value-add, the dynamic sentimental analysis below, which allows us to research global sentiments across a multitude of topics related to this site’s specific keyword domain name. Additionally, when possible, this site references the content above to provide on-demand (multilingual) translations and/or to power its “Read Article to Me” feature, which reads the content aloud to visitors. Where applicable, this site also auto-generates a “References” section, which appends the content above by listing all mentioned links. Views expressed in the content above are solely those of the author(s). We do not endorse, offer to sell, promote, recommend, or, otherwise, make any statement about the content above. We reference the content above for your “reading” entertainment purposes only. Review “DMCA & Terms”, at the bottom of this site, for terms of your access and use as well as for applicable DMCA take-down request.

Acquire this Domain
You can acquire this site’s domain name! We have nurtured its online marketing value by systematically curating this site by the domain’s relevant keywords. Explore our content network – you can advertise on each or rent vs. buy the domain. Buy@TLDtraders.com | Skype: TLDtraders | +1 (475) BUY-NAME (289 – 6263). Thousands search by this site’s exact keyword domain name! Most are sent here because search engines often love the keyword. This domain can be your 24/7 lead generator! If you own it, you could capture a large amount of online traffic for your niche. Stop wasting money on ads. Instead, buy this domain to gain a long-term marketing asset. If you can’t afford to buy then you can rent the domain.

About Us
We are Internet Investors, Developers, and Franchisers – operating a content network of several thousand sites while federating 100+ eCommerce and SaaS startups. With our proprietary “inverted incubation” model, we leverage a portfolio of $100M in valued domains to impact online trends, traffic, and transactions. We use robotic process automation, machine learning, and other proprietary approaches to power our content network. Contact us to learn how we can help you with your online marketing and/or site maintenance.