Another Vote of ‘Confidence’ – a Summary of the Moderna COVID-19 vaccine results
Written by PHASTAR on . Posted in Blog
Another Monday, another COVID-19 vaccine trial delivering some positive news. This week it is Moderna who have announced the results[1] from their first interim analysis, and the numbers are just as impressive as those released by Pfizer and BioNTech last week and discussed here[2]. We also have some actual numbers to take a look at in the Moderna press release, so let’s dive right in…
We are told that the first interim analysis was based on 95 COVID-19 cases, which is very similar to the 94 cases that were analysed by Pfizer and BioNTech. We are told that there were 90 cases of COVID-19 in the placebo group and just 5 cases in the mRNA-1273 vaccine group. Using the formula for vaccine efficacy, this gives us an estimated point estimate of vaccine efficacy of 94.4% (compared with over 90% reported by Pfizer and BioNTech). This is an impressive figure, but just how impressive is it?
Also, it is worth noting that per the study protocol[3], a COVID-19 case was defined as having at least two of the following symptoms: fever (≥38 °C), chills, myalgia, headache, sore throat, new olfactory and taste disorder(s), OR experienced at least one of the following respiratory signs/symptoms: cough, shortness of breath or difficulty breathing, OR clinical or radiological evidence of pneumonia, AND must have at least one nasopharyngeal swab, nasal swab or saliva sample (or respiratory, if hospitalized) positive for SARS-CoV-2 by RT-PCR.
According to the study protocol[4], for the primary efficacy objective, the null hypothesis of the Moderna study was that the vaccine efficacy (VE) of mRNA-1273 to prevent first occurrence of COVID-19 is ≤ 30% (ie, H0 efficacy: VE ≤ 0.3). What that means is that the study would be considered to meet the primary efficacy objective if the lower bound of the corresponding confidence interval (CI) of VE ruled out 30%, either at one of the interim analyses or at the primary analysis. We are told that this Moderna study, known as the COVE study, enrolled more than 30,000 participants. We can use this to calculate an estimated confidence interval for the 94.4% VE figure. According to the protocol, the one-sided alpha for rejecting the null hypothesis of VE ≤ 0.3 was 0.0002 at the first interim analysis, planned to take place at 53 COVID-19 cases. For the second interim analysis, planned to take place at 106 COVID-19 cases, the one-sided alpha was 0.0073. Let’s take this second value and consider a 98.54% confidence interval for our point estimate of 94.4% assuming a 1:1 randomisation of 30,000 participants. Using the score confidence interval for the relative risk (“riskscoreci” command in the “PropCIs” package in R) we get a CI of approximately 84% to 98%. This lower bound of 84% is way above the targeted 30% and provides overwhelming evidence of efficacy.
One question that has been asked of these new findings is whether the Moderna vaccine is better than the Pfizer and BioNtech vaccine. Well, obviously 94% VE is greater than 90% VE. But remember that these are point estimates from two independent clinical trials. Caution should be exercised when interpreting findings across independent trials given differences in study population, endpoints, regions, etc. Only a head-to-head clinical trial comparing vaccines can fully assess vaccine efficacy. Furthermore, point estimates from samples are subject to uncertainty and we see that the confidence interval of 84% to 98% contains the assumed 90% point estimate from the Pfizer trial. That is even if the 90% reported by Pfizer and BioNTech is a point estimate. This value could have been the lower bound on a confidence interval around a higher point estimate! As more and more data comes in from both trials, these specific point estimates are likely to change, but what does seem certain right now is that we now have two candidate vaccines that both look like they work and they both work more effectively than we hoped for, when viewed against a placebo control.
Furthermore, today’s press release includes more details on COVID-19 severity and reports that 11 severe cases of COVID-19 were seen in this interim analysis. All 11 cases occurred in the placebo group, and none in the mRNA-1273 vaccine group. So perhaps we will see that vaccination not only affects whether a person contracts COVID-19, but also virus severity for those who are infected. What still appears to be missing from these interim results, is whether the observed vaccine efficacy levels are consistent across age groups. This press release does state that “Preliminary analysis suggests a broadly consistent safety and efficacy profile across all evaluated subgroups.”, but we are not given any further details. This issue remains vitally important due to the differing ways in which COVID-19 affects individuals in different age categories.
We are also given more information on the safety data and the derivation of the primary outcome in today’s announcement. We are told that in the primary analysis of VE of COVID-19, cases were counted if they occurred two weeks after the second dose of vaccine. So any COVID-19 cases contracted after just one dose of the vaccine were not included in the analyses. This is interesting as it would be useful to know if just one dose of the vaccine offers any protection, especially if further scrutiny of the data shows that trial adherence levels were lower than hoped. With regards to safety, we are told that a review of safety data by the Data Safety Monitoring Board (DSMB) did not report any significant safety concerns and that the vaccine was generally well tolerated. Most adverse events were mild or moderate in severity. In clinical trials we typically grade adverse events, with grade 3 referring to adverse events that are severe, but not necessarily life threatening. We are not told how many adverse events were classified as grade 3, but we are told that those grade 3 events “greater than or equal to 2% in frequency after the first dose included injection site pain (2.7%), and after the second dose included fatigue (9.7%), myalgia (8.9%), arthralgia (5.2%), headache (4.5%), pain (4.1%) and erythema/redness at the injection site (2.0%). These solicited adverse events were generally short-lived”. Ultimately, we will need to see the full results from the trial to fully understand the overall risk/benefit profile of this vaccine.
So, we’re now two for two and eagerly awaiting the results from the Oxford AstraZeneca vaccine trial.