Difficult editorial decisions

Published:October 27, 2020DOI:https://doi.org/10.1016/j.cmi.2020.10.013
      In the present issue of CMI we publish a systematic review and meta-analysis on hydroxychloroquine with and without azithromycin for treating COVID-19 [
      • Fiolet T.
      • Guihur A.
      • Rebeaud M.E.
      • Mulot M.
      • Peiffer-Smadja N.
      • Mahamat-Saleh Y.
      Effect of hydroxychloroquine with or without azithromycin on the mortality of coronavirus disease 2019 (COVID-19) patients: a systematic review and meta-analysis.
      ]; two accompanying commentaries [
      • Raoult D.
      Rational for meta-analysis and randomized treatment: the COVID-19 example.
      ,
      • Paul M.
      Has the door closed on hydroxychloroquine for SARS-COV-2?.
      ]; and letters to the editor addressing the systematic review [
      • Kow C.S.
      • Hasan S.S.
      Re: effect of hydroxychloroquine with or without azithromycin on the mortality of COVID-19 patients’ by Fiolet et al.
      ,
      • Million M.
      • Roussel Y.
      • Lagier J.C.
      • Raoult D.
      Re: 'effect of hydroxychloroquine with or without azithromycin on the mortality of COVID-19 patients’ by Fiolet et al.
      ,
      • Lacout A.
      • Marcy P.Y.
      • Perronne C.
      Re: 'Effect of hydroxychloroquine with or without azithromycin on the mortality of COVID-19 patients' by Fiolet et al.
      ], including the authors' response [
      • Fiolet T.
      • Guihur A.
      • Rebeaud M.E.
      • Mulot M.
      • Peiffer-Smadja N.
      • Mahamat-Saleh Y.
      ‘Effect of hydroxychloroquine with or without azithromycin on the mortality of COVID-19 patients’ - Author's reply.
      ]. Almost all involved difficult editorial decisions, although of different kinds. This is a partial explanation on how we made these decisions.
      When the systematic review [
      • Fiolet T.
      • Guihur A.
      • Rebeaud M.E.
      • Mulot M.
      • Peiffer-Smadja N.
      • Mahamat-Saleh Y.
      Effect of hydroxychloroquine with or without azithromycin on the mortality of coronavirus disease 2019 (COVID-19) patients: a systematic review and meta-analysis.
      ] was submitted there were already more than 15 systematic reviews published on the topic. In an Editorial Note [
      • Leibovici L.
      • Paul M.
      The CMI welcomes systematic reviews.
      ] we detailed our policy on good reasons to repeat a systematic review: “… if a meaningful body of original studies was published since the last review; if the previous review suffered from glaring methodological defects or did not include studies that should have been included; or if the study did not address a primary outcome that matters to patients”. The authors of the present review made a good case showing that new studies were published; the methods of their review were sound; and the chosen outcomes were those that matter to patients. The higher mortality that might be associated with the administration of hydroxychloroquine and azithromycin rang an alarm bell, and it might have influenced our decision.
      The second difficulty in assessing the systematic review was its inclusion of observational studies. While the methodology of systematic reviews of randomized controlled trials is uniform to a high degree, this is not the case for systematic reviews of observational studies [
      • Mueller M.
      • D’Addario M.
      • Egger M.
      • Cevallos M.
      • Dekkers O.
      • Mugglin C.
      • et al.
      Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations.
      ]. Observational studies looking at an intervention suffer from a huge source of bias: patients were prescribed the interventional drug for a reason, and other patients were not given the drug for a reason. The two groups are different: it is likely that the outcomes will differ between groups. There are techniques to adjust for these differences, but we can never be sure how successful these techniques were. Another important source of bias in COVID-19 observational studies is immortal time bias [
      • Wolkewitz M.
      • Lambert J.
      • von Cube M.
      • Bugiera Lars
      • Grodd Marlon
      Derek Hazard, et al. Statistical analysis of clinical COVID-19 data: a concise overview of lessons learned, common errors and how to avoid them.
      ]. We thought the systematic review adequately addressed these problems in the original studies; but actually we can never be sure.
      The systematic review was rigorously peer-reviewed by two reviewers, and accepted for publication after a second revision. No less important were the letters to the editor [
      • Kow C.S.
      • Hasan S.S.
      Re: effect of hydroxychloroquine with or without azithromycin on the mortality of COVID-19 patients’ by Fiolet et al.
      ,
      • Million M.
      • Roussel Y.
      • Lagier J.C.
      • Raoult D.
      Re: 'effect of hydroxychloroquine with or without azithromycin on the mortality of COVID-19 patients’ by Fiolet et al.
      ,
      • Lacout A.
      • Marcy P.Y.
      • Perronne C.
      Re: 'Effect of hydroxychloroquine with or without azithromycin on the mortality of COVID-19 patients' by Fiolet et al.
      ]: their authors pointed to possible problems in the original manuscript, while the authors had the opportunity to respond. Letters to the editor are an important source of criticism. In the peer-review process we hear the opinion of two or three reviewers. The article was read by thousands: we will gain insight even if just a few of them will share their criticism with us.
      I thought the systematic review should be accompanied by a Commentary, to put its finding into perspective. Professor Mical Paul, an editor with CMI, had an idea: she would ask Professor Didier Raoult to write the commentary with her. It was an intriguing idea. They are both distinguished clinicians. Professor Paul is an advocate of evidence-based medicine; the more influential among her publications are systematic reviews and randomized controlled trials. Professor Raoult (whose work does not need an introduction) believes in what he can prove in the laboratory. His esteem for randomized controlled trials and systematic reviews is not that high. The notion that they could reach a synthesis of their views and offer a joint perspective for the clinician was appealing. Alas it did not happen: they could not agree on a common message. They each wrote their own piece. Both Commentaries were peer-reviewed by more than two peer-reviewers. The reviewers had an easy task with Professor Paul's piece [
      • Paul M.
      Has the door closed on hydroxychloroquine for SARS-COV-2?.
      ]: it is a thoughtful Commentary using the paradigm of evidence-based medicine and her experience as a clinician. It was easy to decide on accepting it for publication.
      Professor Raoult's Commentary [
      • Raoult D.
      Rational for meta-analysis and randomized treatment: the COVID-19 example.
      ] was more difficult for the peer-reviewers and for me as an editor. It proposes a different paradigm from the paramount one of randomized controlled trials and systematic reviews. It would have been easy to reject it as being outside the common paradigm. For opinion articles editors have latitude in their decisions, much more than for original studies. Here is where my bias kicked in: I appreciate Professor Raoult not only for his research, but also for being a dedicated and brilliant clinician. I wanted to read his opinion even if it is not the one I hold.
      I hope the readers will be able to reach their own synthesis.

      References

        • Fiolet T.
        • Guihur A.
        • Rebeaud M.E.
        • Mulot M.
        • Peiffer-Smadja N.
        • Mahamat-Saleh Y.
        Effect of hydroxychloroquine with or without azithromycin on the mortality of coronavirus disease 2019 (COVID-19) patients: a systematic review and meta-analysis.
        Clin Microbiol Infect. 2020; (Epub ahead of print)https://doi.org/10.1016/j.cmi.2020.08.022
        • Raoult D.
        Rational for meta-analysis and randomized treatment: the COVID-19 example.
        Clin Microbiol Infect. 2020; 27: 6-8
        • Paul M.
        Has the door closed on hydroxychloroquine for SARS-COV-2?.
        Clin Microbiol Infect. 2020; 27: 3-5
        • Kow C.S.
        • Hasan S.S.
        Re: effect of hydroxychloroquine with or without azithromycin on the mortality of COVID-19 patients’ by Fiolet et al.
        Clin Microbiol Infect. 2020; (Epub ahead of print)https://doi.org/10.1016/j.cmi.2020.09.047
        • Million M.
        • Roussel Y.
        • Lagier J.C.
        • Raoult D.
        Re: 'effect of hydroxychloroquine with or without azithromycin on the mortality of COVID-19 patients’ by Fiolet et al.
        Clin Microbiol Infect. 2020; (Epub ahead of print)https://doi.org/10.1016/j.cmi.2020.09.026
        • Lacout A.
        • Marcy P.Y.
        • Perronne C.
        Re: 'Effect of hydroxychloroquine with or without azithromycin on the mortality of COVID-19 patients' by Fiolet et al.
        Clin Microbiol Infect. 2020; (Epub ahead of print)https://doi.org/10.1016/j.cmi.2020.09.027
        • Fiolet T.
        • Guihur A.
        • Rebeaud M.E.
        • Mulot M.
        • Peiffer-Smadja N.
        • Mahamat-Saleh Y.
        ‘Effect of hydroxychloroquine with or without azithromycin on the mortality of COVID-19 patients’ - Author's reply.
        Clin Microbiol Infect. 2020; (S1198-743X(20)30613-3. Epub ahead of print)https://doi.org/10.1016/j.cmi.2020.10.002
        • Leibovici L.
        • Paul M.
        The CMI welcomes systematic reviews.
        Clin Microbiol Infect. 2016; 22: 397
        • Mueller M.
        • D’Addario M.
        • Egger M.
        • Cevallos M.
        • Dekkers O.
        • Mugglin C.
        • et al.
        Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations.
        BMC Med Res Methodol. 2018; 18: 44
        • Wolkewitz M.
        • Lambert J.
        • von Cube M.
        • Bugiera Lars
        • Grodd Marlon
        Derek Hazard, et al. Statistical analysis of clinical COVID-19 data: a concise overview of lessons learned, common errors and how to avoid them.
        Clin Epidemiol. 2020; 12: 925-928

      Linked Article