The WHO-commissioned meta-study on the effectiveness of facemasks, published in the medical journal The Lancet in June 2020, has been instrumental in shifting global facemask policies during the Covid-19 pandemic. However, the meta-study, which claimed a risk reduction of 80% with facemasks, is seriously flawed on several levels and should be retracted.
A. General flaws
- Of the 29 studies analyzed by the Lancet meta-study, seven studies are unpublished and non-peer-reviewed observational studies that should not be used to guide clinical practice according to the medRxiv disclaimer (references 3, 4, 31, 36, 37, 40 and 70; see table above).
- Of the 29 studies considered by the meta-study, only four are about the SARS-CoV-2 virus; the other 25 studies are about the SARS-1 virus or the MERS virus, both of which have very different transmission characteristics: they were transmitted almost exclusively by severely ill hospitalized patients and not by community transmission.
- Of the four studies relating to the SARS-CoV-2 virus, two were misinterpreted by the Lancet meta-study authors (refs. 44 and 70), one is inconclusive (ref. 37), and one is about N95 (FFP2) respirators and not about medical masks or cloth masks (see detailed analysis below).
- The Lancet meta-study is used to guide global facemask policy for the general population. However, of the 29 studies considered by the meta-study, only three are classified as relating to a non-health-care (i.e. community) setting. Of these three studies, one is misclassified (ref. 50, relating to masks in a hospital environment), one showed no benefit of facemasks (ref. 69), and one is a poorly designed retrospective study about SARS-1 in Beijing based on telephone interviews (ref. 74). None of these studies refer to SARS-CoV-2.
- The authors of the Lancet meta-study acknowledge that the certainty of the evidence regarding facemasks is “low” as all of the studies are observational and none is a randomized controlled trial (RCT). The WHO itself admitted that its updated facemask policy guidelines were based not on new evidence but on “political lobbying”.
In view of these shortcomings, University of Toronto epidemiology professor Peter Jueni called the WHO study “methodologically flawed” and “essentially useless”.
B. Study misinterpretations
As mentioned above, several studies have been misinterpreted by the authors of the meta-study. All of the misinterpretations resulted in falsely claiming or exaggerating a benefit of facemasks. In the following, only the four studies relating to SARS-CoV-2 are reviewed. (HCW: health care worker)
- Heinzerling et al. (ref. 44): The meta-study claims that 0 of 31 HCW wearing a facemask and 3 of 6 HCW not wearing a facemask got infected. This is not correct: the study showed that 0 of 3 HCW wearing a facemask and 3 of 34 HCW not wearing a facemask got infected. This result was not statistically significant (p=0.73). Moreover, of the 3 HCW who got infected, one reported wearing a facemask “most of the time”, but the meta-study classified this HCW as “not wearing a facemask”.
- Wang et al. (ref. 41): This study, which did show a benefit of facemasks, was about N95 (FFP2) respirators in a health-care setting, not about medical masks or cloth masks.
- Wang et al. (ref. 70): The meta-study claims that 1 of 1286 HCW wearing a facemask and 119 of 4036 wearing “no facemask” got infected. This is not correct: according to the study, 78.3% (94/120) of infected HCW were in fact wearing a surgical mask, and only 20.8% (25/120) did not wear any mask when exposed to the source of infection. The “1 of 1286 HCW” mentioned in the meta-study refers to HCW wearing an N95 (FFP2) respirator, not a medical or cloth mask.
- Burke et al. (ref. 37): This study had no results relating to the use of facemasks.
Non-Covid studies were also misinterpreted or misrepresented by the the Lancet meta-study authors. For instance, in the case of the non-Covid study with the allegedly biggest impact of masks (Kim et al. (ref. 49) about N95 respirators in a hospital with MERS patients), the meta-study authors incorrectly mixed serological and PCR results, again exaggerating the benefit of (N95) masks. The actual results of the Kim et al. study were not statistically significant (p=0.159).
Update: A US analyst has reviewed all 29 studies and found numerous additional mistakes.
C. Studies relating to “social distancing”
In an additional section, the WHO meta-study evaluated studies on the benefit of “social distancing” measures. However, several independent experts have shown that this section is seriously flawed, too, as the authors again misinterpreted several studies and made several statistical errors. As with the studies on facemasks, all of these mistakes resulted in falsely claiming or exaggerating a benefit of “social distancing” measures.
As shown in this analysis, the WHO-commissioned meta-study on the effectiveness of facemasks and social distancing, published in The Lancet, is seriously flawed and should be retracted. Health authorities may want to reconsider their Covid-19 policy guidelines.
See also: Are face masks effective? The evidence.
Original study: Chu et al., Physical distancing, face masks, and eye protection to prevent person-to-person transmission of SARS-CoV-2 and COVID-19: a systematic review and meta-analysis, The Lancet, June 2020, Volume 395, Issue 10242, P1973-1987.