Statistics and Its Interface

Volume 5 (2012)

Number 4

Statistical performance of group sequential methods for observational post-licensure medical product safety surveillance: A simulation study

Pages: 381 – 390



Andrea Cook (Biostatistics Unit, Group Health Research Institute, Seattle, Wash., U.S.A.)

Lisa Jackson (Biostatistics Unit, Group Health Research Institute, Seattle, Wash., U.S.A.)

Jennifer Nelson (Biostatistics Unit, Group Health Research Institute, Seattle, Wash., U.S.A.)

Shanshan Zhao (Department of Biostatistics, University of Washington, Seattle, Wash., U.S.A.)


In order to improve post-licensure drug and vaccine safety surveillance, new systems are being developed that prospectively monitor observational health care data from large health plans. Continuous sequential testing has been proposed in this setting to facilitate rapid detection, but group sequential methods commonly used in randomized clinical trials (RCTs) have received less consideration. We propose a group sequential approach tailored for safety and to account for complications like confounding that arise in this non-randomized setting and thus have not been previously examined in RCTs. For comparability with prior continuous monitoring applications, we use a likelihood ratio statistic and historical controls. We compute sequential boundaries using Monte Carlo simulation and show how they can accommodate unequal between-test sample sizes and changes in confounder distributions among accruing subjects over time. We evaluate via simulation the performance of this approach across sequential designs suited for safety and not previously addressed by simulation studies evaluating RCT boundaries. Such designs include much higher frequency testing and designs that employ early conservatism followed by frequent testing. Contrary to prior RCT simulations, we found major differences in the average time-to-surveillanceend and overall power. We apply this methodology to safety data on a new pediatric combination vaccine.


medical product safety, observational study, post-licensure surveillance, sequential testing

Full Text (PDF format)