Biostatistics Center site logo

Methodological Research

Many new and complex issues are confronted in the conduct of Biostatistics Center studies creating a need for novel methodologies. The Center researchers advance the mission of the collaborative research by developing and implementing innovative practical methods for the design, execution, data monitoring, analyses and reporting of the clinical studies.

Current research foci include the design and analyses of studies with patient-focused outcome measures that integrate efficacy and safety outcomes, personalizing treatment, cost-effectiveness analyses, response-adaptive randomization, and pragmatic evaluation of diagnostic technologies based on diagnostic yield.

The Innovations in Design, Education and Analysis (IDEA) Committee

The Innovations in Design, Education and Analysis (IDEA) Committee serves as a catalyst for methodological research at the Biostatistics Center and provides a resource for innovation for the Center researchers.

The IDEA Committee Mission:

  • Foster biostatistical science by developing and implementing innovative approaches for the design, monitoring, analysis, and reporting of clinical research studies

  • Provide expert input on the design and analysis of Biostatistics Center studies

  • Serve as a think-tank for designing new and complex Biostatistics Center studies

Examples of Recent Methodological Innovations

Frequency of Evidence-Based Screening for Retinopathy in Type I Diabetes

Patients with type I diabetes are at high risk for retinopathy. Screening is conducted to ensure timely treatment. The American Diabetes Association recommends annual examinations but this is based on dated epidemiological studies. The optimal screening frequency is unknown. More frequent screening visits imply: (i) more rapid disease detection and more effective resulting treatment, and (ii) higher costs and increased patient burden. Data from the Center’s DCC and EDIC studies provided the opportunity to re-evaluate the optimal screening frequency and develop cost-effective screening strategies.

Professors Bebu and Lachin developed statistical methods to identify optimal screening schedules. The methodologies were applied resulting in a proposed screening schedule will result in: (i) 31% shorter delay in disease detection, (ii) 58% fewer examinations, and (iii) 43.4% (~$1 billion) in cost savings, over a 20-year period. A web application implementing these methods has been accesses more than 9,500 times at the Biostatistics Center in the last year.

Bebu, I., Lachin, J.M. Optimal Screening Schedules For Disease Progression, With Application to Diabetic Retinopathy, Biostatistics, 19, 1-13, 2018.

Nathan, D., Bebu, I., Hainsworth, D., Klein, R., Tamborlane, W., Lorenzi, G., Gubitosi-Klug, R., Lachin, J.M. Evidence-based screening frequency for retinopathy in type 1 diabetes, New England Journal of  Medicine, 376, 1507-1516, 2017.

Master Protocol for Multiple Infection Diagnostics (MASTERMIND)

New diagnostics are urgently needed to address emerging antimicrobial resistance. Professor Evans was the senior author on the Antibacterial Resistance Leadership Group’s (ALRGs) MASTERMIND design for advancement of infectious diseases diagnostics. The goal of MASTERMIND is to generate the data necessary to evaluate diagnostic tests by promoting research that might not have otherwise been feasible with conventional study designs. MASTERMIND uses a single subject’s sample(s) to evaluate multiple diagnostic tests at the same time, providing efficiencies of specimen collection and characterization. MASTERMIND also offers central trial organization, standardization of methods and definitions, and common comparators. MASTERMIND is currently being applied in a study to evaluate the performance of nucleic acid amplification tests for the detection of Neisseria gonorrhoeae and Chlamydia trachomatis in extragenital sites. 

Patel R, Tsalik EL, Petzold E, Fowler Jr. VG, Klausner JD, Evans SR. MASTERMIND – Bringing Microbial Diagnostics to the Clinic. Clin Infect Dis. 2017 Feb 1;64(3):355-360. doi: 10.1093/cid/ciw788. Epub 2016 Dec 7. PMCID: PMC5894935.

Response-Adaptive Randomization

The Biostatistics Center has long-standing expertise in randomization methodology including novel approaches such as frequentist and Bayesian response adaptive randomization. Dr. John M. Lachin is co-author of “Randomization in Clinical Trial – Theory and Practice” a seminal textbook of standard and covariate-adaptive response-adaptive randomization. Recent additions to the Biostatistics Center, Dr. Yunyun Jiang and Dr. Diane Uschner, have devoted their research careers to this important topic, evaluating the performance of Bayesian response adaptive randomization in confirmatory phase III clinical trials, developing software for the assessment and implementation of response adaptive randomization, and developing methodology for robust inference following response adaptive randomization. The Biostatistics Center has strong relationships with other experts in the field of response adaptive randomization including Dr. Feifang Hu, Professor of Statistics at the George Washington University and Dr. William F. Rosenberger, Professor of Statistics at George Mason University and former faculty member at the Biostatistics Center.

Rosenberger WF, Lachin JM. Randomization in Clinical Trials - Theory and Practice. Wiley Series in probability and statistics, 2nd edition, 2016.

Jiang Y, Zhao W, Durkalski-Mauldin V. Impact of adaption algorithm, timing, and stopping boundaries on the performance of Bayesian response adaptive randomization in confirmative trials with a binary endpoint. Contemporary clinical trials. 62:114-120. 2017.

Jiang Y, Zhao W, Durkalski-Mauldin V. Time-trend impact in two-arm clinical trials with a binary outcome and Bayesian response adaptive randomization. (submitted to Journal of Biopharmaceutical Statistics)

Rosenberger WF, Uschner D, Wang Y. The 15th Armitage Lecture - Randomization: The forgotten component of the randomized clinical trial. Statistics in Medicine: 1-12, 2018. DOI: 10.1002/sim.7901

Uschner D, Schindler D, Hilgers RD, Heussen N. randomizeR: An R Package for the Assessment and Implementation of Randomization in Clinical Trials. Journal of Statistical Software. 85(8):1-22, 2018. DOI: 10.18637/jss.v085.i08.

Uschner D, Hilgers RD, Heussen N. The impact of selection bias in randomized multi-arm parallel group clinical trials. PLOS ONE, 13(1):1-18, 2018. DOI: 10.1371/journal.pone.0192065

Hilgers RD, Uschner D, Rosenberger WF, Heussen N. ERDO - a framework to select an appropriate randomization procedure for clinical trials. BMC Med Res Methodol. 17(1):159. 2017

Fitzner C, Uschner D, Hilgers RD. Is covariate-balancing randomization beneficial? Evaluation based on the STOP IgAN trial. (will be submitted to BMC)

Missing Data Methodologies

Last observation carried forward is common statistical approach to the analysis of longitudinal repeated measures data where observations may be missing. Several evaluations have demonstrated the faults of this approach. Dr. Lachin used a repeated measures model to further illustrate the limitations of the last observation carried forward approach and concludes that it should not be employed in any analyses.

Lachin JM. Fallacies of last observation carried forward analyses. Clinical Trials, 2016, 13(2), 161-68 doi: 10.1177/1740774515602688. PMC4785044.

Pragmatic Benefit:risk Assessment of Interventions

Randomized trials are the gold standard for evaluating intervention effects. However studies often fail to provide the necessary evidence to inform medical decision-making. The important implications of these deficiencies are largely absent from discourse in medical research communities. Professor Evans developed pragmatic benefit:risk methodologies to inform patient management.

Typical analyses of clinical trials involve intervention comparisons for each efficacy and safety outcome. Outcome-specific effects are estimated and potentially combined in benefit:risk analyses believing that this informs the totality of effects on patients. However such approaches do not incorporate associations between outcomes, are confounded by competing risks, and since efficacy and safety analyses are often conducted on different analysis populations, the population to which these analyses apply, is unclear.

Professor Evans proposed the desirability of outcome ranking (DOOR) and partial credit methodologies as a remedy to these issues. The methods can incorporate patient values and estimate personalized effects. The methods were used to compare ceftazidime-avibactam vs. colistin for the treatment of infections due to carbapenem-resistant Enterobacteriaceae (CRE), a pathogen characterized as having an urgent hazard level by the Centers for Disease Control (CDC) and as a Priority 1 (Critical) pathogen by the World Health Organization (WHO).

Evans SR, Rubin D, Follmann D, Pennello G, Huskins WC, Powers JH, Schoenfeld D,  Chuang-Stein C, Cosgrove SE, Fowler Jr. VG, Lautenbach E, Chambers HF. Desirability of Outcome Ranking (DOOR) and Response Adjusted for Duration of Antibiotic Risk (RADAR). Clin Infect Dis. 2015 Sep 1;61(5):800-6. doi: 10.1093/cid/civ495. Epub 2015 Jun 25. PMCID: PMC4542892.

Evans SR, Follmann D. Fundamentals and Innovation in Antibiotic Trials. Statistics in Biopharmaceutical Research. 2015;7:4:331-336. DOI:10.1080/19466315.2015.1094406. NIHMSID: NMH748791. PMCID: PMC4831648

Evans SR, Follmann D. Using Outcomes to Analyze Patients Rather than Patients to Analyze Outcomes: A Step toward Pragmatism in Benefit:risk Evaluation. Stat Biopharm Res. 2016;8(4):386-393. doi: 10.1080/19466315.2016.1207561. Epub 2016 Dec 6. PMCID: PMC5394932.

Van Duin D, Lok J, Earley M, Cober E, Richter S, Perez F, Salata R, Kalayjian R, Watkins R, Doi Y, Kaye K, Fowler V, Paterson D, Bonomo R, Evans SR. Colistin vs. Ceftazidime-avibactam in the Treatment of Infections due to Carbapenem-Resistant Enterobacteriaceae, Clin Infect Dis. 2018 Jan 6;66(2):163-171. doi: 10.1093/cid/cix783. PMCID: PMC5850032.

Pragmatic Benefit:risk Assessment of Diagnostics

Standard evaluation of diagnostics consists of estimating sensitivity, specificity, and positive/negative predictive values and likelihood ratios. However these measures have limited utility for guiding clinical decision-making. Diagnostic utility depends on prevalence and the relative importance of potential errors (false positive vs. false negative). Professor Evans and colleagues proposed benefit-risk evaluation of diagnostics: a framework (BED-FRAME) for pragmatic diagnostic evaluation. They defined weighted accuracy and diagnostic yield measures to communicate the expected clinical impact of diagnostic application and the tradeoffs of diagnostic alternatives. The methods are being used to design a study evaluating the utility of a host response-based diagnostic test categorizing acute respiratory tract illness into bacterial, viral, or neither etiology in a regulatory setting.

Evans SR, Pennello G, Pantoja-Galicia N, Jiang H, Hujer AM, Hujer KM, Manca C, Hill C, Jacobs MR, Chen L, Patel R, Kreiswirth BN, Bonomo RA. Benefit:risk evaluation for diagnostics: a framework (BED-FRAME). Clin Infect Dis. 2016; 63(6):812–7. PMCID: PMC4996133.

Liu Y, Tsalik EL, Jiang Y, Ko ER, Woods CW, Henao R, Evans SR. Average Weighted Accuracy (AWA): Pragmatic Analysis for a RADICAL Study. Clin Infect Dis.; Forthcoming

Pennello G, Pantoja-Galicia N, Evans SR. Benefit-Risk Comparisons of Diagnostic Tests Based on Diagnostic Yield and Expected Utility. J of Biopharm Stats. 2016; 26(6):1083-1097. PMCID: PMC5471848.

Prediction for Interim Monitoring of Clinical Trials

Professor Evans and colleagues introduced use of prediction for data monitoring of clinical trials and as a valuable tool for data safety and monitoring boards (DSMBs). The methods are the foundation for the commercial software EAST PREDICT.

Evans SR, Li L. A comparison of goodness of fit tests for the logistic GEE model. Stat Med. 2005 Apr 30;24(8):1245-61. PubMed PMID: 15580592.

Li L, Evans SR, Uno H, Wei LJ. Predicted Interval Plots: A Graphical Tool for Data Monitoring in Clinical Trials. Statistics in Biopharmaceutical Research, 1:4:348-355, 2009. PMID: 21423789; PMCID: PMC3059316

SMART COMPASS

Patient management is not based on a single decision. Rather, it is dynamic: based on a sequence of decisions, with therapeutic adjustments made over time. Adjustments are personalized: tailored to individual patients as new information becomes available. However, strategies allowing for such adjustments are infrequently studied. Traditional antibiotic trials are often nonpragmatic, comparing drugs for definitive therapy when drug susceptibilities are known. COMparing Personalized Antibiotic StrategieS (COMPASS) is a trial design that compares strategies consistent with clinical practice. Strategies are decision rules that guide empiric and definitive therapy decisions. Sequential, multiple-assignment, randomized (SMART) COMPASS allows evaluation when there are multiple, definitive therapy options. SMART COMPASS is pragmatic, mirroring clinical, antibiotic-treatment decision-making and addressing the most relevant issue for treating patients: identification of the patient-management strategy that optimizes the ultimate patient outcomes. SMART COMPASS is valuable in the setting of antibiotic resistance, when therapeutic adjustments may be necessary due to resistance.

Evans SR, Follmann D, Liu Y, Holland T, Doernberg SB, Rouphael N, Hamasaki T, Jiang Y, Lok JL, Tran TTT, Harris AD, Fowler, Jr. VG, Boucher H, Kreiswirth BN, Bonomo RA, van Duin D, Paterson DL, Chambers H. Sequential Multiple Assignment Randomized Trials for COMparing Personalized Antibiotic StrategieS (SMART-COMPASS). Clin Infect Dis. 2019;68(11):1961–7. PMID: 30351426 PMC Journal - In Process

Books

 

Sozu T, Hamasaki T, Sugimoto T, Evans SR. Sample size determination in clinical trials with multiple endpoints. ISBN 978-3-319-22005-5. Springer. 2015.

Hamasaki T, Asakura K, Evans SR, Ochiai T. Group-Sequential Clinical Trials with Multiple Co-Objectives. •ISBN 978-4-431-55898-9. Springer. 2016.