Statistics and Its Interface

Volume 8 (2015)

Number 4

A penalized likelihood approach for robust estimation of isoform expression

Pages: 437 – 445



Hui Jiang (Department of Biostatistics, Center for Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, Mich., U.S.A.)

Julia Salzman (Department of Biochemistry, Stanford Cancer Institute, Stanford University, Stanford, California, U.S.A.)


Ultra high-throughput sequencing of transcriptomes (RNA-Seq) has enabled the accurate estimation of gene expression at individual isoform level. However, systematic biases introduced during the sequencing and mapping processes as well as incompleteness of the transcript annotation databases may cause the estimates of isoform abundances to be unreliable, and in some cases, highly inaccurate. This paper introduces a penalized likelihood approach to detect and correct for such biases in a robust manner. Our model extends those previously proposed by introducing bias parameters for reads. An L1 penalty is used for the selection of non-zero bias parameters. We introduce an efficient algorithm for model fitting and analyze the statistical properties of the proposed model. Our experimental studies on both simulated and real datasets suggest that the model has the potential to improve isoform-specific gene expression estimates and identify incompletely annotated gene models.


penalized likelihood, robust estimation, RNA-Seq, isoform expression

Published 19 October 2015