Advertisement

Towards precision dosing of vancomycin in critically ill patients: an evaluation of the predictive performance of pharmacometric models in ICU patients

  • C.B. Cunio
    Affiliations
    Department of Clinical Pharmacology & Toxicology, St Vincent's Hospital, Sydney, Australia

    School of Medical Sciences, University of New South Wales, Sydney, Australia
    Search for articles by this author
  • D.W. Uster
    Affiliations
    Department of Clinical Pharmacy, Institute of Pharmacy, University of Hamburg, Hamburg, Germany
    Search for articles by this author
  • J.E. Carland
    Affiliations
    Department of Clinical Pharmacology & Toxicology, St Vincent's Hospital, Sydney, Australia

    St Vincent's Clinical School, Univeristy of New South Wales, Sydney, Australia

    Centre of Applied Medical Research, St Vincent's Hospital, Sydney, Australia
    Search for articles by this author
  • H. Buscher
    Affiliations
    St Vincent's Clinical School, Univeristy of New South Wales, Sydney, Australia

    Centre of Applied Medical Research, St Vincent's Hospital, Sydney, Australia

    Department of Intensive Care Medicine, St Vincent's Hospital, Sydney, Australia
    Search for articles by this author
  • Z. Liu
    Affiliations
    Stats Central, University of New South Wales, Sydney, Australia
    Search for articles by this author
  • J. Brett
    Affiliations
    Department of Clinical Pharmacology & Toxicology, St Vincent's Hospital, Sydney, Australia

    St Vincent's Clinical School, Univeristy of New South Wales, Sydney, Australia
    Search for articles by this author
  • M. Stefani
    Affiliations
    Department of Clinical Pharmacology & Toxicology, St Vincent's Hospital, Sydney, Australia

    St Vincent's Clinical School, Univeristy of New South Wales, Sydney, Australia
    Search for articles by this author
  • G.R.D. Jones
    Affiliations
    St Vincent's Clinical School, Univeristy of New South Wales, Sydney, Australia

    SydPath, St Vincent's Hospital, Sydney, Australia
    Search for articles by this author
  • R.O. Day
    Affiliations
    Department of Clinical Pharmacology & Toxicology, St Vincent's Hospital, Sydney, Australia

    School of Medical Sciences, University of New South Wales, Sydney, Australia

    St Vincent's Clinical School, Univeristy of New South Wales, Sydney, Australia

    Centre of Applied Medical Research, St Vincent's Hospital, Sydney, Australia
    Search for articles by this author
  • S.G. Wicha
    Affiliations
    Department of Clinical Pharmacy, Institute of Pharmacy, University of Hamburg, Hamburg, Germany
    Search for articles by this author
  • S.L. Stocker
    Correspondence
    Corresponding author. Sophie Stocker, Department of Clinical Pharmacology & Toxicology, St Vincent's Hospital, Sydney, Australia.
    Affiliations
    Department of Clinical Pharmacology & Toxicology, St Vincent's Hospital, Sydney, Australia

    St Vincent's Clinical School, Univeristy of New South Wales, Sydney, Australia

    Centre of Applied Medical Research, St Vincent's Hospital, Sydney, Australia
    Search for articles by this author

      Abstract

      Objectives

      Vancomycin dose recommendations depend on population pharmacokinetic models. These models have not been adequately assessed in critically ill patients, who exhibit large pharmacokinetic variability. This study evaluated model predictive performance in intensive care unit (ICU) patients and identified factors influencing model performance.

      Methods

      Retrospective data from ICU adult patients administered vancomycin were used to evaluate model performance to predict serum concentrations a priori (no observed concentrations included) or with Bayesian forecasting (using concentration data). Predictive performance was determined using relative bias (rBias, bias) and relative root mean squared error (rRMSE, precision). Models were considered clinically acceptable if rBias was between ±20% and 95% confidence intervals included zero. Models were compared with rRMSE; no threshold was used. The influence of clinical factors on model performance was assessed with multiple linear regression.

      Results

      Data from 82 patients were used to evaluate 12 vancomycin models. The Goti model was the only clinically acceptable model with both a priori (rBias 3.4%) and Bayesian forecasting (rBias 1.5%) approaches. Bayesian forecasting was superior to a priori prediction, improving with the use of more recent concentrations. Four models were clinically acceptable with Bayesian forecasting. Renal replacement therapy status (p < 0.001) and sex (p = 0.007) significantly influenced the performance of the Goti model.

      Conclusions

      The Goti, Llopis and Roberts models are clinically appropriate to inform vancomycin dosing in critically ill patients. Implementing the Goti model in dose prediction software could streamline dosing across both ICU and non-ICU patients, considering it is also the most accurate model in non-ICU patients.

      Keywords

      Introduction

      Vancomycin is a widely used glycopeptide antibiotic [
      • Rybak M.
      The pharmacokinetic and pharmacodynamic properties of vancomycin.
      ]. It is first-line treatment for serious methicillin-resistant Staphylococcus aureus infections [
      • Turner R.B.
      • Kojiro K.
      • Shephard E.A.
      • Won R.
      • Chang E.
      • Chan D.
      • et al.
      Review and validation of bayesian dose-optimizing software and equations for calculation of the vancomycin area under the curve in critically ill patients.
      ]. Drug exposure, described by area under the concentration–time curve over 24 hours to minimum inhibitory concentration ratio, is the best predictor of vancomycin efficacy [
      • Rybak M.
      • Le J.
      • Lodise T.
      • Levine D.
      • Bradley J.
      • Liu C.
      • et al.
      Therapeutic monitoring of vancomycin: a revised consensus guideline and review of the American society of health-system pharmacists, the infectious diseases society of America, the pediatric infectious diseases society and the society of infectious diseases pharmacists.
      ]. Vancomycin has a narrow therapeutic range (400–700 mg∙h/L) and exhibits significant inter- and intraindividual pharmacokinetic variability [
      • Rybak M.
      • Le J.
      • Lodise T.
      • Levine D.
      • Bradley J.
      • Liu C.
      • et al.
      Therapeutic monitoring of vancomycin: a revised consensus guideline and review of the American society of health-system pharmacists, the infectious diseases society of America, the pediatric infectious diseases society and the society of infectious diseases pharmacists.
      ], which is often more marked in critically ill patients, such as those in an intensive care unit (ICU), compared to heterogeneous populations [
      • Roberts J.A.
      • Taccone F.S.
      • Udy A.A.
      • Vincent J.L.
      • Jacobs F.
      • Lipman J.
      Vancomycin dosing in critically ill patients: robust methods for improved continuous-infusion regimens.
      ]. These perturbations in vancomycin pharmacokinetics are primarily due to varying volume of distribution and clearance [
      • Roberts J.A.
      • Taccone F.S.
      • Udy A.A.
      • Vincent J.L.
      • Jacobs F.
      • Lipman J.
      Vancomycin dosing in critically ill patients: robust methods for improved continuous-infusion regimens.
      ,
      • Revilla N.
      • Martín-Suárez A.
      • Pérez M.P.
      • González F.M.
      • Fernández de Gatta M.D.M.
      Vancomycin dosing assessment in intensive care unit patients based on a population pharmacokinetic/pharmacodynamic simulation.
      ], and the interference of medical interventions such as renal replacement therapy (RRT) and extracorporeal membrane oxygenation (ECMO) [
      • Jamal J.P.J.A.
      • Economou A.C.
      • Lipman A.J.
      • Roberts A.J.
      Improving antibiotic dosing in special situations in the ICU: burns, renal replacement therapy and extracorporeal membrane oxygenation.
      ]. Therefore, therapeutic drug monitoring is recommended to optimize patient outcomes and minimize toxicities, including nephrotoxicity [
      • Rybak M.
      • Le J.
      • Lodise T.
      • Levine D.
      • Bradley J.
      • Liu C.
      • et al.
      Therapeutic monitoring of vancomycin: a revised consensus guideline and review of the American society of health-system pharmacists, the infectious diseases society of America, the pediatric infectious diseases society and the society of infectious diseases pharmacists.
      ]. User-friendly dose prediction software that incorporates Bayesian forecasting can estimate vancomycin exposure using a single serum vancomycin concentration obtained at any time during the dosing interval [
      • Neely M.N.
      • Kato L.
      • Youn G.
      • Kraler L.
      • Bayard D.
      • van Guilder M.
      • et al.
      Prospective trial on the use of trough concentration versus area under the curve to determine therapeutic vancomycin dosing.
      ]. Furthermore, the accuracy of each subsequent concentration improves, in contrast to linear modelling. The accuracy of dose-prediction software is dependent on the population pharmacokinetic (popPK) model used [
      • Turner R.B.
      • Kojiro K.
      • Shephard E.A.
      • Won R.
      • Chang E.
      • Chan D.
      • et al.
      Review and validation of bayesian dose-optimizing software and equations for calculation of the vancomycin area under the curve in critically ill patients.
      ]. However, there is limited evidence to guide model selection for vancomycin in critically ill patients.
      Several vancomycin popPK models have been developed, both in heterogeneous and ICU-only patient populations. An evaluation of the predictive performance of 31 vancomycin models [
      • Broeker A.
      • Nardecchia M.
      • Klinker K.P.
      • Derendorf H.
      • Day R.O.
      • Marriott D.J.
      • et al.
      Towards precision dosing of vancomycin: a systematic evaluation of pharmacometric models for Bayesian forecasting.
      ], in a heterogeneous cohort with predominantly non-ICU patients, found that the Goti [
      • Goti V.
      • Chaturvedula A.
      • Fossler M.J.
      • Mok S.
      • Jacob J.T.
      Hospitalized patients with and without hemodialysis have markedly different vancomycin pharmacokinetics: a population pharmacokinetic model-based analysis.
      ] and Medellin-Garibay [
      • Medellín-Garibay S.E.
      • Ortiz-Martín B.
      • Rueda-Naharro A.
      • García B.
      • Romano-Moreno S.
      • Barcia E.
      Pharmacokinetics of vancomycin and dosing recommendations for trauma patients.
      ] models were most accurate. However, the validity of extrapolating these findings into critically ill patients remains unknown, particularly as variability in vancomycin pharmacokinetics is often greater in critically ill compared to heterogenous patients [
      • Roberts J.A.
      • Taccone F.S.
      • Udy A.A.
      • Vincent J.L.
      • Jacobs F.
      • Lipman J.
      Vancomycin dosing in critically ill patients: robust methods for improved continuous-infusion regimens.
      ,
      • Revilla N.
      • Martín-Suárez A.
      • Pérez M.P.
      • González F.M.
      • Fernández de Gatta M.D.M.
      Vancomycin dosing assessment in intensive care unit patients based on a population pharmacokinetic/pharmacodynamic simulation.
      ,
      • Medellín-Garibay S.E.
      • Ortiz-Martín B.
      • Rueda-Naharro A.
      • García B.
      • Romano-Moreno S.
      • Barcia E.
      Pharmacokinetics of vancomycin and dosing recommendations for trauma patients.
      ,
      • Donadello K.
      • Roberts J.A.
      • Cristallini S.
      • Beumier M.
      • Shekar K.
      • Jacobs F.
      • et al.
      Vancomycin population pharmacokinetics during extracorporeal membrane oxygenation therapy: a matched cohort study.
      ,
      • Llopis-Salvia P.
      • Jiménez-Torres N.V.
      Population pharmacokinetic parameters of vancomycin in critically ill patients.
      ,
      • Medellín-Garibay S.E.
      • Romano-Moreno S.
      • Tejedor-Prado P.
      • Rubio-Álvaro N.
      • Rueda-Naharro A.
      • Blasco-Navalpotro M.A.
      • et al.
      Influence of mechanical ventilation on the pharmacokinetics of vancomycin administered by continuous infusion in critically ill patients.
      ].
      Therefore, this study aimed to identify vancomycin popPK models with clinically acceptable predictive performance specifically in critically ill adult patients; and to identify the influence of clinical parameters, such as the number of vancomycin concentrations included for Bayesian forecasting, on predictive performance.

      Methods

       Population pharmacokinetic model selection

      Published vancomycin popPK models in critically ill adult patients were identified with a literature search. Models requiring covariate data that were unavailable in our data set were excluded. Additionally, some models not specifically developed in critically ill populations were included. These were: one model [
      • Buelga D.S.
      • del Mar Fernandez de Gatta M.
      • Herrera E.V.
      • Dominguez-Gil A.
      • García M.J.
      Population pharmacokinetic analysis of vancomycin in patients with hematological malignancies.
      ] available in a commercial dose prediction software used internationally to guide vancomycin dosing; two models reported to be least biased and imprecise in a predominantly non-ICU population [
      • Broeker A.
      • Nardecchia M.
      • Klinker K.P.
      • Derendorf H.
      • Day R.O.
      • Marriott D.J.
      • et al.
      Towards precision dosing of vancomycin: a systematic evaluation of pharmacometric models for Bayesian forecasting.
      ]; and a new unique pooled popPK model developed from ICU patients [
      • Colin P.J.
      • Allegaert K.
      • Thomson A.H.
      • Touw D.J.
      • Dolton M.
      • de Hoog M.
      • et al.
      Vancomycin pharmacokinetics throughout life: results from a pooled population analysis and evaluation of current dosing recommendations.
      ] that has not been externally validated. This comprehensive model selection allowed comparison of not only models developed in ICU populations, but also those currently guiding vancomycin dosing and models previously identified as accurate, in order to simplify and streamline clinical implementation of models that perform acceptably in ICU patients. The selected models were encoded in NONMEM® 7.4.3 (Icon Development Solutions, Hanover, MD, USA).

       Validation data set

      Data were obtained retrospectively for patients admitted to the St Vincent's Hospital (SVH) ICU from July 2018 to August 2019 (Supplementary Text S1). Subjects were ≥18 years of age, received ≥48 hours of intravenous vancomycin therapy, and had at least two vancomycin and creatinine concentrations measured while in the ICU. Vancomycin doses were selected at the clinicians' discretion. Vancomycin concentrations were determined by routine immunoassays (EMIT® 2000, Siemens Healthineers, Erlangen, Germany). The lower limit of quantification was 2 mg/L, below which concentrations were excluded from the analysis. Further clinical parameter definitions are presented in Supplementary Text S2 and Supplementary Table S1.
      This study was approved by the SVH Human Research Ethics Committee (2019/ETH09850).

       Predictive performance

      To evaluate model performance, vancomycin concentrations were predicted and compared to the observed values. Serum concentrations were predicted using either patient covariates only (a priori), or both patient covariates and observed concentration data (Bayesian forecasting). The predictive performance was quantified by calculating relative bias (rBias, bias (Eq. (1)) and relative root mean squared error (rRMSE, precision (Eq. (2)) [
      • Sheiner L.B.
      • Beal S.
      • Rosenberg B.
      • Marathe V.V.
      Forecasting individual pharmacokinetics.
      ]:
      rBias=1N1ipredictediobservediobservedi×100
      (1)


      rRMSE=1N1i(predictediobservedi)2(observedi)2×100
      (2)


      where N is the number of vancomycin concentrations (Supplementary Fig. S1).
      A model was unbiased if the rBias 95% confidence interval included zero. Unbiased models were clinically acceptable if rBias was between −20% and 20% [
      • Sheiner L.B.
      • Beal S.
      • Rosenberg B.
      • Marathe V.V.
      Forecasting individual pharmacokinetics.
      ,
      • Sheiner L.B.
      • Beal S.L.
      Some suggestions for measuring predictive performance.
      ]. Models were also compared using rRMSE; no clinically acceptable threshold was used due to a lack of sufficient literature. Models with a lower rRMSE were more precise and therefore more clinically appropriate. The fit of the data to the models when using Bayesian forecasting was inspected visually with goodness-of-fit plots (predicted vs. observed concentration) and prediction-corrected visual predictive checks [
      • Bergstrand M.
      • Hooker A.C.
      • Wallin J.E.
      • Karlsson M.O.
      Prediction-corrected visual predictive checks for diagnosing nonlinear mixed-effects models.
      ], using PsN 4.7.0 and the ‘vpc’ package for R 1.1.0 (https://www.r-project.org/).

       Detailed Bayesian forecasting analysis

      To assess the optimal number of vancomycin concentrations required for Bayesian forecasting, courses of therapy in which concentrations were obtained within at least four distinct dosing intervals were identified. These dosing intervals were not necessarily consecutive; samples were obtained at any time throughout a dosing interval. The Bayesian prediction of concentrations in the fourth dosing interval was tested. This was computed with varying combinations of observed concentrations from the three preceding dosing intervals [
      • Broeker A.
      • Nardecchia M.
      • Klinker K.P.
      • Derendorf H.
      • Day R.O.
      • Marriott D.J.
      • et al.
      Towards precision dosing of vancomycin: a systematic evaluation of pharmacometric models for Bayesian forecasting.
      ]: (i) first only; (ii) most recent (third) only; (iii) second and third; and (iv) first, second and third.

       Clinical parameter analysis

      For the best model, the effects of several clinical parameters (Supplementary Text S3) on bias and precision were assessed using multiple linear regression. Only data associated with the first concentration in the fourth dosing interval were included.

      Results

       Demographics

      Of the 466 patients who received intravenous vancomycin, 82 (91 courses of therapy) were admitted to the ICU during their treatment and had data included in the analysis (Supplementary Fig. S2). Seven patients received two courses of vancomycin, and one patient received three. Each course of therapy was analysed separately. Patient demographics are provided in Supplementary Tables S2 and S3. The median (range) therapy duration was 6 (2–42) days. Vancomycin infusions were both intermittent (n = 1273) and continuous (n = 100), some with preceding loading doses. A total of 746 concentrations were available, of which 648 (87%) were obtained while patients were in the ICU. Two concentrations were below the lower limit of quantification. The median (range) number of concentrations per course of therapy was 6 (1–37). The median (range) concentration was 19.2 (2.5–69.1) mg/L.

       Model selection

      Of the 12 vancomycin popPK models developed in critically ill populations [
      • Roberts J.A.
      • Taccone F.S.
      • Udy A.A.
      • Vincent J.L.
      • Jacobs F.
      • Lipman J.
      Vancomycin dosing in critically ill patients: robust methods for improved continuous-infusion regimens.
      ,
      • Revilla N.
      • Martín-Suárez A.
      • Pérez M.P.
      • González F.M.
      • Fernández de Gatta M.D.M.
      Vancomycin dosing assessment in intensive care unit patients based on a population pharmacokinetic/pharmacodynamic simulation.
      ,
      • Donadello K.
      • Roberts J.A.
      • Cristallini S.
      • Beumier M.
      • Shekar K.
      • Jacobs F.
      • et al.
      Vancomycin population pharmacokinetics during extracorporeal membrane oxygenation therapy: a matched cohort study.
      ,
      • Llopis-Salvia P.
      • Jiménez-Torres N.V.
      Population pharmacokinetic parameters of vancomycin in critically ill patients.
      ,
      • Medellín-Garibay S.E.
      • Romano-Moreno S.
      • Tejedor-Prado P.
      • Rubio-Álvaro N.
      • Rueda-Naharro A.
      • Blasco-Navalpotro M.A.
      • et al.
      Influence of mechanical ventilation on the pharmacokinetics of vancomycin administered by continuous infusion in critically ill patients.
      ,
      • Moore J.
      • Healy J.R.
      • Thoma B.
      • Peahota M.
      • Ahamadi M.
      • Schmidt L.
      • et al.
      A population pharmacokinetic model for vancomycin in adult patients receiving extracorporeal membrane oxygenation therapy.
      ,
      • Mulla H.
      • Pooboni S.
      Population pharmacokinetics of vancomycin in patients receiving extracorporeal membrane oxygenation.
      ,
      • Udy A.A.
      • Covajes C.
      • Taccone F.S.
      • Jacobs F.
      • Vincent J.L.
      • Lipman J.
      • et al.
      Can population pharmacokinetic modelling guide vancomycin dosing during continuous renal replacement therapy in critically ill patients?.
      ,
      • Li X.
      • Sun S.
      • Ling X.
      • Chen K.
      • Wang Q.
      • Zhao Z.
      Plasma and cerebrospinal fluid population pharmacokinetics of vancomycin in postoperative neurosurgical patients after combined intravenous and intraventricular administration.
      ,
      • Li X.
      • Wu Y.
      • Sun S.
      • Mei S.
      • Wang J.
      • Wang Q.
      • et al.
      Population Pharmacokinetics of vancomycin in postoperative neurosurgical patients.
      ,
      • Li X.
      • Wu Y.
      • Sun S.
      • Zhao Z.
      • Wang Q.
      Population pharmacokinetics of vancomycin in postoperative neurosurgical patients and the application in dosing recommendation.
      ,
      • Mangin O.
      • Urien S.
      • Mainardi J.L.
      • Fagon J.Y.
      • Faisy C.
      Vancomycin pharmacokinetic and pharmacodynamic models for critically ill patients with post-sternotomy mediastinitis.
      ] four were excluded [
      • Li X.
      • Sun S.
      • Ling X.
      • Chen K.
      • Wang Q.
      • Zhao Z.
      Plasma and cerebrospinal fluid population pharmacokinetics of vancomycin in postoperative neurosurgical patients after combined intravenous and intraventricular administration.
      ,
      • Li X.
      • Wu Y.
      • Sun S.
      • Mei S.
      • Wang J.
      • Wang Q.
      • et al.
      Population Pharmacokinetics of vancomycin in postoperative neurosurgical patients.
      ,
      • Li X.
      • Wu Y.
      • Sun S.
      • Zhao Z.
      • Wang Q.
      Population pharmacokinetics of vancomycin in postoperative neurosurgical patients and the application in dosing recommendation.
      ,
      • Mangin O.
      • Urien S.
      • Mainardi J.L.
      • Fagon J.Y.
      • Faisy C.
      Vancomycin pharmacokinetic and pharmacodynamic models for critically ill patients with post-sternotomy mediastinitis.
      ] because they included covariates for which data were unavailable in the validation data set. An additional four vancomycin popPK models developed from heterogeneous patient cohorts [
      • Goti V.
      • Chaturvedula A.
      • Fossler M.J.
      • Mok S.
      • Jacob J.T.
      Hospitalized patients with and without hemodialysis have markedly different vancomycin pharmacokinetics: a population pharmacokinetic model-based analysis.
      ,
      • Medellín-Garibay S.E.
      • Ortiz-Martín B.
      • Rueda-Naharro A.
      • García B.
      • Romano-Moreno S.
      • Barcia E.
      Pharmacokinetics of vancomycin and dosing recommendations for trauma patients.
      ,
      • Buelga D.S.
      • del Mar Fernandez de Gatta M.
      • Herrera E.V.
      • Dominguez-Gil A.
      • García M.J.
      Population pharmacokinetic analysis of vancomycin in patients with hematological malignancies.
      ,
      • Colin P.J.
      • Allegaert K.
      • Thomson A.H.
      • Touw D.J.
      • Dolton M.
      • de Hoog M.
      • et al.
      Vancomycin pharmacokinetics throughout life: results from a pooled population analysis and evaluation of current dosing recommendations.
      ] (i.e. including non-ICU patients) were analysed. Therefore, 12 models were evaluated (Supplementary Table S4).

       Overall model evaluation

      Using the a priori approach, the Moore, Goti and Udy models demonstrated clinically acceptable bias (Fig. 1(a), Supplementary Table S5). However, the precision of all models was >40% (Fig. 1(b)).
      Fig. 1
      Fig. 1Overall model rBias (a) and rRMSE (b) for a priori predicted concentrations and the overall model rBias (c) and rRMSE (d) for Bayesian predicted concentrations. A priori predictions were made using patient covariates only. Bayesian predictions were made using both patient covariates and all available individual ICU observed concentrations (n = 648). Black bars represent models developed in ICU populations; white bars, models developed in general patient cohorts. Models were clinically acceptable (∗) if their rBias () was between −20% and 20% and a 95% confidence interval included zero. ICU, intensive care unit; rBias, relative bias (bias); rRMSE, relative root mean squared error (precision).
      With Bayesian forecasting, the precision of models to predict concentrations improved compared to the a priori approach (Fig. 1(b), (d)). Five models were clinically acceptable (Fig. 1(c), Supplementary Table S5). Goodness-of-fit plots for these five models using Bayesian forecasting (Fig. 2, Supplementary Fig. S3) demonstrated minimal spread and clinically acceptable fit of the model-predicted concentrations to the observed data.
      Fig. 2
      Fig. 2Goodness-of-fit plots for the five clinically acceptable models, depicting the Bayesian model-predicted concentrations against all the available corresponding ICU observed concentrations (n = 648). The line of identity, where observed concentrations equals predicted concentrations, represents a perfect model whereby all points would fall on the line. Models with more spread around the line of identity do not fit the patient data as well. The goodness-of-fit plots for the remaining seven models are presented in .
      The Goti model was the only clinically acceptable model using both the a priori and Bayesian forecasting approach. The visual predictive checks (Fig. 3, Supplementary Fig. S4) also suggested that the Goti model best fit the observed data using Bayesian forecasting. In contrast, the vancomycin concentration predictions by the Buelga, Colin, Llopis-Salvia and Roberts models underestimated observed values.
      Fig. 3
      Fig. 3Prediction-corrected visual predictive checks [
      • Bergstrand M.
      • Hooker A.C.
      • Wallin J.E.
      • Karlsson M.O.
      Prediction-corrected visual predictive checks for diagnosing nonlinear mixed-effects models.
      ] of the five clinically acceptable models. The solid black line is the median of the prediction-corrected observations; dashed lines represent the 5th and 95th percentiles for the median; and shading represents 95% confidence intervals of the 5th and 95th percentiles (light blue) and the median (dark blue) of the simulation (n = 1000). The shaded confidence intervals represent the performance of each model. These visual predictive checks show that the Buelga and Colin models underestimate the data for higher concentrations, as the 95th percentile of the median of the observed data always lies above the confidence interval of predictions for the 95th percentile. The Roberts model was variable in its prediction of concentrations. The Goti and Llopis models generally fit the observed data well, with the Goti model performing the best. presents the visual predictive checks for the other seven models evaluated.

       Detailed Bayesian forecasting analysis

      Fifty-one patients (Supplementary Table S6) each received one course of vancomycin therapy with concentration data for at least four dosing intervals. The concentrations were obtained both throughout and across different dosing intervals (Supplementary Fig. S5). The Roberts and Llopis-Salvia models were clinically acceptable under all conditions (Fig. 4). The Goti model was clinically acceptable under all conditions except when only the oldest (first) available concentration was used. Conversely, the Colin model was clinically acceptable except when the most recent (third) concentration alone was used. The Buelga model was biased for all scenarios.
      Fig. 4
      Fig. 4Comparisons of bias and precision for vancomycin concentrations in the fourth dosing interval when calculated using different combinations of concentrations in the preceding dosing intervals: (1) first only; (3) most recent (third) only; (2 & 3) second and third; and (1, 2 & 3) first, second and third. Black bars indicate scenarios where the bias was clinically acceptable; white bars, scenarios where the bias was not clinically acceptable (not between -20% and 20% and/or a 95% confidence interval did not pass through zero). rBias, relative bias (bias); rRMSE, relative root mean squared error (precision).
      All models demonstrated similar precision, which improved when using any combination of concentrations other than the oldest concentration alone to make Bayesian predictions. In terms of both bias and precision, using all available concentrations was not always better than using only the most recent concentration.

       Clinical parameter analysis

      The Goti model was used to assess the influence of clinical parameters on model predictive performance, as it was clinically acceptable, and is reported to be the most accurate in non-ICU patients [
      • Broeker A.
      • Nardecchia M.
      • Klinker K.P.
      • Derendorf H.
      • Day R.O.
      • Marriott D.J.
      • et al.
      Towards precision dosing of vancomycin: a systematic evaluation of pharmacometric models for Bayesian forecasting.
      ]. In multiple linear regression analysis, RRT status and sex remained significant for bias, and Sequential Organ Failure Assessment (SOFA) score for precision (Table 1, Supplementary Table S7). Overall, the Goti model–predicted concentrations were 16% less biased for female rather than male subjects, and 22% more biased for patients receiving RRT compared to those not receiving RRT. An increase in SOFA score was associated with a decrease in precision.
      Table 1Multiple linear regression analysis of the influence of clinical parameters on predictive performance (bias and precision) of the Goti model
      Clinical parameterCoefficient estimate95% CIp
      LowerUpper
      Bias
       Sex−15.604−26.770−4.439<0.001
       RRT status22.30712.51132.1030.007
      Precision
       SOFA score0.1170.0770.1570.005
      Shown are significant predictors of bias and precision as suggested by multiple linear regression. Significance level of α = 0.05 was used. Male sex was the reference sex, and presence of RRT (i.e. ‘yes’ status) was the reference status. Overall, the Goti model was 16% less biased for female than male sex. Vancomycin concentrations predicted in patients receiving RRT were associated with a mean bias value of 22% above those who did not receive this medical intervention. SOFA score data are presented after square root transformation; an increase in SOFA score is associated with a decrease in precision. CI, confidence interval; RRT, renal replacement therapy; SOFA, Sequential Organ Failure Assessment.

      Discussion

      This study informs healthcare institutions on the most appropriate model to use in dosing software to calculate the optimal vancomycin dose in critically ill patients. The Goti, Roberts, Llopis-Salvia and Colin models all demonstrated clinically acceptable bias in both the overall model evaluation and detailed Bayesian forecasting analyses, with more accurate results when using Bayesian forecasting over an a priori approach.
      The Goti model was clinically acceptable in ICU patients, even with a limited number of concentrations. This is consistent with a study conducted in a primarily non-ICU patient population [
      • Broeker A.
      • Nardecchia M.
      • Klinker K.P.
      • Derendorf H.
      • Day R.O.
      • Marriott D.J.
      • et al.
      Towards precision dosing of vancomycin: a systematic evaluation of pharmacometric models for Bayesian forecasting.
      ]. Interestingly, the Goti model was more accurate than models developed specifically in ICU populations, despite being developed with a predominantly non-ICU heterogeneous cohort. This may reflect the large comprehensive data set from which it was built, including patients undergoing RRT and ECMO [
      • Goti V.
      • Chaturvedula A.
      • Fossler M.J.
      • Mok S.
      • Jacob J.T.
      Hospitalized patients with and without hemodialysis have markedly different vancomycin pharmacokinetics: a population pharmacokinetic model-based analysis.
      ]. Further, it is the only model which includes RRT status as a covariate. This is important as RRT can significantly impact vancomycin pharmacokinetics [
      • Roberts D.M.
      • Liu X.
      • Roberts J.A.
      • Nair P.
      • Cole L.
      • Roberts M.S.
      • et al.
      A multicenter study on the effect of continuous hemodiafiltration intensity on antibiotic pharmacokinetics.
      ] and is commonly required in critically ill patients. Therefore, the Goti model demonstrates clinical utility: it accurately predicts vancomycin drug exposure in both critically ill and non-ICU patients, using only the most recent concentration.
      While the Colin model was clinically acceptable in the overall evaluation, the model was biased when only the most recent vancomycin concentration was used in the detailed Bayesian forecasting analysis. This may limit the clinical utility of the model, as it may provide less adequate dosing decisions when only one, recent, concentration is available.
      Although the Roberts and Llopis-Salvia models were clinically acceptable, they did not perform as well as the Goti model in the visual predictive checks. Further, they reportedly have poorer predictive performance in non–critically ill patient populations than the Goti model [
      • Broeker A.
      • Nardecchia M.
      • Klinker K.P.
      • Derendorf H.
      • Day R.O.
      • Marriott D.J.
      • et al.
      Towards precision dosing of vancomycin: a systematic evaluation of pharmacometric models for Bayesian forecasting.
      ]. This limits their generalizability, and thus their clinical utility to guide vancomycin dosing decisions in heterogeneous adult patient populations.
      Of note, Guo et al. [
      • Guo T.
      • van Hest R.M.
      • Roggeveen L.F.
      • Fleuren L.M.
      • Thoral P.J.
      • Bosman R.J.
      • et al.
      External evaluation of population pharmacokinetic models of vancomycin in large cohorts of intensive care unit patients.
      ] suggested that the Roberts model was clinically acceptable in ICU patients. However, they only examined the predictive performance of a limited number of models (n = 6) using an a priori approach. This does not reflect clinical practice where Bayesian forecasting methodology would be used [
      • Rybak M.
      • Le J.
      • Lodise T.
      • Levine D.
      • Bradley J.
      • Liu C.
      • et al.
      Therapeutic monitoring of vancomycin: a revised consensus guideline and review of the American society of health-system pharmacists, the infectious diseases society of America, the pediatric infectious diseases society and the society of infectious diseases pharmacists.
      ]. Further, they used a less stringent threshold for clinical acceptability than the current study, using ±20% bias, without consideration of the 95% confidence interval. We reinterpreted their findings using the thresholds from the present study, which indicated that the Roberts model was biased in one of their two sample populations. Turner et al. [
      • Turner R.B.
      • Kojiro K.
      • Shephard E.A.
      • Won R.
      • Chang E.
      • Chan D.
      • et al.
      Review and validation of bayesian dose-optimizing software and equations for calculation of the vancomycin area under the curve in critically ill patients.
      ] analysed the use of Bayesian forecasting in critically ill patients, but they compared the performance of various dosing software without specifically identifying the most accurate model(s). The strengths of our study in addition to ter Heine et al. [
      • ter Heine R.
      • Keizer R.J.
      • van Steeg K.
      • Smolders E.J.
      • van Luin M.
      • Derijks H.J.
      • et al.
      Prospective validation of a model-informed precision dosing tool for vancomycin in intensive care patients.
      ] are the broader testing of models and the inclusion of RRT and ECMO patients. By covering model use in two modalities which impact vancomycin pharmacokinetics and are common to critically ill patients, we increase the clinically utility of our findings.
      When all available concentrations obtained in the ICU were used (up to 37 per course of therapy), the Buelga model displayed the best predictive performance. However, it was not clinically acceptable when only a maximum of three concentrations were used. Thus, the model may require at least four concentrations to adequately estimate vancomycin exposure. Consensus guidelines recommend attainment of therapeutic targets within 24 to 48 hours of therapy initiation [
      • Rybak M.
      • Le J.
      • Lodise T.
      • Levine D.
      • Bradley J.
      • Liu C.
      • et al.
      Therapeutic monitoring of vancomycin: a revised consensus guideline and review of the American society of health-system pharmacists, the infectious diseases society of America, the pediatric infectious diseases society and the society of infectious diseases pharmacists.
      ]. For an average twice-daily dosing regimen, with concentrations obtained after each dose, the Buelga model would require at least 48 hours before its performance becomes clinically acceptable. Therefore, the Buelga model was not considered suitable in our cohort.
      Consistent with a previous study [
      • Broeker A.
      • Nardecchia M.
      • Klinker K.P.
      • Derendorf H.
      • Day R.O.
      • Marriott D.J.
      • et al.
      Towards precision dosing of vancomycin: a systematic evaluation of pharmacometric models for Bayesian forecasting.
      ], the predictive performance of the models examined improved with Bayesian forecasting compared to an a priori approach. It did not necessarily improve when including more than one concentration; however, better performance was achieved with more recent concentrations. This suggests that some models can accurately predict optimal dosing in a critically ill population using a single recent vancomycin concentration. These subgroup analysis findings should be interpreted with caution considering the small sample size. Further investigation into any difference associated with the number of concentrations included may be warranted.
      This study identified that the predictive performance of the Goti model in critically ill patients depended on RRT status, sex and SOFA score. Therefore, despite including haemodialysis status as a covariate, the Goti model did not account for all of the variability in vancomycin pharmacokinetics contributed by RRT. This is not surprising, as RRT is only incorporated as a categorical covariate. In reality, RRT is a continuous and multidimensional covariate due to its numerous modalities and equipment options, which may have differing effects on vancomycin disposition [
      • Roberts D.M.
      • Liu X.
      • Roberts J.A.
      • Nair P.
      • Cole L.
      • Roberts M.S.
      • et al.
      A multicenter study on the effect of continuous hemodiafiltration intensity on antibiotic pharmacokinetics.
      ]. The impact of sex on bias is difficult to interpret and may simply be an artefact of the male-dominated patient population. The SOFA score analysis could reflect that a model is more imprecise with increasing severity of illness, likely associated with more unstable patients. Given the moderate sample size, the clinical parameter analysis should be interpreted carefully. Prospective studies analysing RRT status and SOFA score in critically ill patients receiving vancomycin could help determine whether their influence is clinically significant.
      A strength of the study was the large number of concentrations collected per patient, covering several dosing intervals, thereby ensuring robustness of the results. There was no clinically significant difference in predictive performance between continuous and intermittent infusions (Supplementary Tables S8 and S9); however, considering the small sample size of continuous infusions this finding should be interpreted cautiously. A limitation was that the clinical acceptability threshold for models is not commonly specified for vancomycin. Ideally, the threshold for acceptability would be the maximum bias and precision, whereby the difference between the model-predicted and observed concentrations does not alter the dose recommendations. Further studies are required to quantify this threshold and thereby inform model selection in Bayesian forecasting software.
      This study identified the Goti, Llopis and Roberts models as clinically appropriate to inform loading and maintenance vancomycin dosing decisions in critically ill patients when using Bayesian forecasting. Model performance improved with the use of more recent concentration data, even if only a single concentration was used. RRT status, sex and SOFA score influenced the performance of the Goti model. Use of the Goti model in dose prediction software could simplify and streamline clinical practice, considering that it is appropriate for use in both non-ICU [
      • Broeker A.
      • Nardecchia M.
      • Klinker K.P.
      • Derendorf H.
      • Day R.O.
      • Marriott D.J.
      • et al.
      Towards precision dosing of vancomycin: a systematic evaluation of pharmacometric models for Bayesian forecasting.
      ] and ICU patients.

      Transparency declaration

      All authors report no conflicts of interest relevant to this article.

      Acknowledgement

      The authors would like to thank Suhel Al-Soufi for collating the APACHE (Acute Physiology and Chronic Health Evaluation) scores.

      Appendix A. Supplementary data

      The following is the Supplementary data to this article:

      References

        • Rybak M.
        The pharmacokinetic and pharmacodynamic properties of vancomycin.
        Clin Infect Dis. 2006; 42: S35-S39
        • Turner R.B.
        • Kojiro K.
        • Shephard E.A.
        • Won R.
        • Chang E.
        • Chan D.
        • et al.
        Review and validation of bayesian dose-optimizing software and equations for calculation of the vancomycin area under the curve in critically ill patients.
        Pharmacotherapy. 2018; 38: 1174-1183
        • Rybak M.
        • Le J.
        • Lodise T.
        • Levine D.
        • Bradley J.
        • Liu C.
        • et al.
        Therapeutic monitoring of vancomycin: a revised consensus guideline and review of the American society of health-system pharmacists, the infectious diseases society of America, the pediatric infectious diseases society and the society of infectious diseases pharmacists.
        Am J Health Syst Pharm. 2020; 77: 835-864
        • Roberts J.A.
        • Taccone F.S.
        • Udy A.A.
        • Vincent J.L.
        • Jacobs F.
        • Lipman J.
        Vancomycin dosing in critically ill patients: robust methods for improved continuous-infusion regimens.
        Antimicrob Agents Chemother. 2011; 55: 2704-2709
        • Revilla N.
        • Martín-Suárez A.
        • Pérez M.P.
        • González F.M.
        • Fernández de Gatta M.D.M.
        Vancomycin dosing assessment in intensive care unit patients based on a population pharmacokinetic/pharmacodynamic simulation.
        Br J Clin Pharmacol. 2010; 70: 201-212
        • Jamal J.P.J.A.
        • Economou A.C.
        • Lipman A.J.
        • Roberts A.J.
        Improving antibiotic dosing in special situations in the ICU: burns, renal replacement therapy and extracorporeal membrane oxygenation.
        Curr Opin Crit Care. 2012; 18: 460-471
        • Neely M.N.
        • Kato L.
        • Youn G.
        • Kraler L.
        • Bayard D.
        • van Guilder M.
        • et al.
        Prospective trial on the use of trough concentration versus area under the curve to determine therapeutic vancomycin dosing.
        Antimicrob Agents Chemother. 2018; 62 (e02042–17)
        • Broeker A.
        • Nardecchia M.
        • Klinker K.P.
        • Derendorf H.
        • Day R.O.
        • Marriott D.J.
        • et al.
        Towards precision dosing of vancomycin: a systematic evaluation of pharmacometric models for Bayesian forecasting.
        Clini Microbiol Infect. 2019; 25: 1286e1-1286e7
        • Goti V.
        • Chaturvedula A.
        • Fossler M.J.
        • Mok S.
        • Jacob J.T.
        Hospitalized patients with and without hemodialysis have markedly different vancomycin pharmacokinetics: a population pharmacokinetic model-based analysis.
        Ther Drug Monit. 2018; 40: 212-221
        • Medellín-Garibay S.E.
        • Ortiz-Martín B.
        • Rueda-Naharro A.
        • García B.
        • Romano-Moreno S.
        • Barcia E.
        Pharmacokinetics of vancomycin and dosing recommendations for trauma patients.
        J Antimicrob Chemother. 2016; 71: 471-479
        • Donadello K.
        • Roberts J.A.
        • Cristallini S.
        • Beumier M.
        • Shekar K.
        • Jacobs F.
        • et al.
        Vancomycin population pharmacokinetics during extracorporeal membrane oxygenation therapy: a matched cohort study.
        Crit Care. 2014; 18: 632
        • Llopis-Salvia P.
        • Jiménez-Torres N.V.
        Population pharmacokinetic parameters of vancomycin in critically ill patients.
        J Clin Pharm Ther. 2006; 31: 447-454
        • Medellín-Garibay S.E.
        • Romano-Moreno S.
        • Tejedor-Prado P.
        • Rubio-Álvaro N.
        • Rueda-Naharro A.
        • Blasco-Navalpotro M.A.
        • et al.
        Influence of mechanical ventilation on the pharmacokinetics of vancomycin administered by continuous infusion in critically ill patients.
        Antimicrob Agents Chemother. 2017; 61 (e01249–17)
        • Buelga D.S.
        • del Mar Fernandez de Gatta M.
        • Herrera E.V.
        • Dominguez-Gil A.
        • García M.J.
        Population pharmacokinetic analysis of vancomycin in patients with hematological malignancies.
        Antimicrob Agents Chemother. 2005; 49: 4934-4941
        • Colin P.J.
        • Allegaert K.
        • Thomson A.H.
        • Touw D.J.
        • Dolton M.
        • de Hoog M.
        • et al.
        Vancomycin pharmacokinetics throughout life: results from a pooled population analysis and evaluation of current dosing recommendations.
        Clin Pharmacokinet. 2019; 58: 767-780
        • Sheiner L.B.
        • Beal S.
        • Rosenberg B.
        • Marathe V.V.
        Forecasting individual pharmacokinetics.
        Clin Pharmacol Ther. 1979; 26: 294-305
        • Sheiner L.B.
        • Beal S.L.
        Some suggestions for measuring predictive performance.
        J Pharmacokinet Biopharm. 1981; 9: 503-512
        • Bergstrand M.
        • Hooker A.C.
        • Wallin J.E.
        • Karlsson M.O.
        Prediction-corrected visual predictive checks for diagnosing nonlinear mixed-effects models.
        AAPS J. 2011; 13: 143-151
        • Moore J.
        • Healy J.R.
        • Thoma B.
        • Peahota M.
        • Ahamadi M.
        • Schmidt L.
        • et al.
        A population pharmacokinetic model for vancomycin in adult patients receiving extracorporeal membrane oxygenation therapy.
        CPT Pharmacometrics Syst Pharmacol. 2016; 5: 495-502
        • Mulla H.
        • Pooboni S.
        Population pharmacokinetics of vancomycin in patients receiving extracorporeal membrane oxygenation.
        Br J Clin Pharmacol. 2005; 60: 265-275
        • Udy A.A.
        • Covajes C.
        • Taccone F.S.
        • Jacobs F.
        • Vincent J.L.
        • Lipman J.
        • et al.
        Can population pharmacokinetic modelling guide vancomycin dosing during continuous renal replacement therapy in critically ill patients?.
        Int J Antimicrob Agents. 2013; 41: 564-568
        • Li X.
        • Sun S.
        • Ling X.
        • Chen K.
        • Wang Q.
        • Zhao Z.
        Plasma and cerebrospinal fluid population pharmacokinetics of vancomycin in postoperative neurosurgical patients after combined intravenous and intraventricular administration.
        Eur J Clin Pharmacol. 2017; 73: 1599-1607
        • Li X.
        • Wu Y.
        • Sun S.
        • Mei S.
        • Wang J.
        • Wang Q.
        • et al.
        Population Pharmacokinetics of vancomycin in postoperative neurosurgical patients.
        J Pharm Sci. 2015; 104: 3960-3967
        • Li X.
        • Wu Y.
        • Sun S.
        • Zhao Z.
        • Wang Q.
        Population pharmacokinetics of vancomycin in postoperative neurosurgical patients and the application in dosing recommendation.
        J Pharm Sci. 2016; 105: 3425-3431
        • Mangin O.
        • Urien S.
        • Mainardi J.L.
        • Fagon J.Y.
        • Faisy C.
        Vancomycin pharmacokinetic and pharmacodynamic models for critically ill patients with post-sternotomy mediastinitis.
        Clin Pharmacokinet. 2014; 53: 849-861
        • Roberts D.M.
        • Liu X.
        • Roberts J.A.
        • Nair P.
        • Cole L.
        • Roberts M.S.
        • et al.
        A multicenter study on the effect of continuous hemodiafiltration intensity on antibiotic pharmacokinetics.
        Crit Care. 2015; 19: 84
        • Guo T.
        • van Hest R.M.
        • Roggeveen L.F.
        • Fleuren L.M.
        • Thoral P.J.
        • Bosman R.J.
        • et al.
        External evaluation of population pharmacokinetic models of vancomycin in large cohorts of intensive care unit patients.
        Antimicrob Agents Chemother. 2019; 63 (e02543–18)
        • ter Heine R.
        • Keizer R.J.
        • van Steeg K.
        • Smolders E.J.
        • van Luin M.
        • Derijks H.J.
        • et al.
        Prospective validation of a model-informed precision dosing tool for vancomycin in intensive care patients.
        Br J Clin Pharmacol. 2020;