JavaScript isn't enabled in your browser, so this file can't be opened. Enable and reload.

1 of 17

A General Form of Covariate Adjustment in Randomized Clinical Trials

Ting Ye – Department of Biostatistics, University of Washington

Joint work with

Marlena Bannick, Department of Biostatistics, University of Washington

Jun Shao, Department of Statistics, University of Wisconsin-Madison

Yu Du, Global Statistical Sciences, Eli Lilly and Company

Jingyi Liu, Global Statistical Sciences, Eli Lilly and Company

Yanyao Yi, Global Statistical Sciences, Eli Lilly and Company

2 of 17

Why adjusting for covariates?

Design Stage

covariate-adaptive randomization

balance across baseline covariates to gain credibility and efficiency

“Balance of treatment groups with respect to one or more specific prognostic covariates can enhance the credibility of the results of the trial” – EMA (2015) Guideline

Analysis Stage

model-assisted analysis

more efficient use of data under minimal assumption required by unadjusted analysis

“Incorporating prognostic baseline covariates in the design and analysis of clinical trial data can result in a more efficient use of data to demonstrate and quantify the effects of treatment. Moreover, this can be done with minimal impact on bias or the Type I error rate.” – FDA (2023) Guidance

3 of 17

Covariate Adjustment for Unconditional Treatment Effect

4 of 17

Covariate Adjustment for Unconditional Treatment Effect

	Working Model		Representative Estimator
Unadjusted			ANOVA

5 of 17

Covariate Adjustment for Unconditional Treatment Effect

	Working Model		Representative Estimator
Unadjusted			ANOVA
Linear adjustment

* Ye, T., Shao, J., Yi, Y., and Zhao, Q. (2023). Toward better practice of covariate adjustment in analyzing randomized clinical trials. JASA.

6 of 17

Covariate Adjustment for Unconditional Treatment Effect

	Working Model		Representative Estimator
Unadjusted			ANOVA
Linear adjustment
G-computation			G-computation using logistic model (FDA guidance)

* Guo, K. and Basse, G. (2021). The generalized Oaxaca-blinder estimator. JASA.

* Bannick, M., Shao, J., Liu, J., Du, Y., Yi, Y., and Ye, T. (2023+) A general form of covariate adjustment in randomized clinical trials. arXiv:2306.10213

7 of 17

Covariate Adjustment for Unconditional Treatment Effect

	Working Model		Representative Estimator
Unadjusted			ANOVA
Linear adjustment
G-computation			G-computation using logistic model (FDA guidance)
AIPW (doubly robust)

* Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., and Newey, W. (2017) Double/debiased/neyman machine learning of treatment effects. American Economic Review.

* Bannick, M., Shao, J., Liu, J., Du, Y., Yi, Y., and Ye, T. (2023+) A general form of covariate adjustment in randomized clinical trials. arXiv:2306.10213

8 of 17

Covariate Adjustment for Unconditional Treatment Effect

	Working Model		Representative Estimator
Unadjusted			ANOVA
Linear adjustment
G-computation			G-computation using logistic model (FDA guidance)
AIPW (doubly robust)

the general form of covariate adjustment in randomized clinical trials

9 of 17

Covariate Adjustment for Unconditional Treatment Effect

	Working Model		Representative Estimator
Unadjusted			ANOVA
Linear adjustment
G-computation			G-computation using logistic model (FDA guidance)
AIPW (doubly robust)

10 of 17

Covariate Adjustment for Unconditional Treatment Effect

	Working Model		Representative Estimator
Unadjusted			ANOVA
Linear adjustment
G-computation			G-computation using logistic model (FDA guidance)
AIPW (doubly robust)

11 of 17

Covariate Adjustment for Unconditional Treatment Effect

	Working Model		Representative Estimator
Unadjusted			ANOVA
Linear adjustment
G-computation			G-computation using logistic model (FDA guidance)
AIPW (doubly robust)

12 of 17

Covariate Adjustment for Unconditional Treatment Effect

	Working Model		Representative Estimator
Unadjusted			ANOVA
Linear adjustment
G-computation			G-computation using logistic model (FDA guidance)
AIPW (doubly robust)

All methods discussed and beyond are available in the R package

Robust Inference for Covariate Adjustment in Randomization Clinical Trials [RobinCAR]

https://github.com/tye27/RobinCar

13 of 17

Simulation (G-computation can be biased)


Method	Bias	SD
G-computation using Poisson regression	-0.005	0.249
G-computation using NB with unknow dispersion parameter	0.159	0.263

Poisson regression uses “log” link, a canonical link, hence, G-computation is unbiased

Negative Binomial regression with unknown dispersion parameter (estimated by maximum likelihood) is NOT a GLM with canonical link, hence, G-computation is biased.

14 of 17

Simulation (AIPW is more general than G-computation)


Method	Bias	SD
G-computation using Poisson regression	-0.005	0.249
G-computation using NB with unknow dispersion parameter	0.159	0.263
AIPW using Poisson regression	-0.005	0.249
AIPW using NB with unknow dispersion parameter	-0.005	0.253

AIPW is always unbiased, regardless of using GLM with canonical link or not

Poisson regression uses “log” link, a canonical link, hence, G-computation is equal to AIPW.

15 of 17

Simulation (CASE 1)

						Correct			Naïve^‡
Working model	Method		Bias	SD		SE	CP		SE	CP
Simple Randomization
(unadjusted)	Sample Mean		0.03	3.13		3.12	94.80
GLM-logistic	AIPW		0.00	2.74		2.76	95.16
GLM-logistic	JC		0.01	2.67		2.67	94.98
Random Forest	AIPW-CF		0.02	2.73		2.74	95.06
Random Forest	JC		0.02	2.71		2.72	95.18
Stratified Permuted Block Randomization (block size of 6)
(unadjusted)	Sample Mean		0.02	2.80		2.81	95.08		3.12	97.36
GLM-logistic	AIPW		0.00	2.75		2.75	95.20		2.76	95.24
GLM-logistic	JC		0.01	2.67		2.67	95.00
Random Forest	AIPW-CF		0.02	2.71		2.72	95.04		2.74	95.68
Random Forest	JC		0.03	2.70		2.71	95.00

All estimators have negligible biases compared to their standard deviations.

Correct SEs derived under corresponding randomization scheme are very close to SD, as a result, coverage probabilities are close to 95%.

Covariate adjustment achieves >20% variance reduction under simple randomization.

Joint calibration always shows smaller SD.

16 of 17

Simulation (CASE 2)

						Correct			Naïve^‡
Working model	Method		Bias	SD		SE	CP		SE	CP
Simple Randomization
(unadjusted)	Sample Mean		0.08	3.05		3.06	95.32
GLM-logistic	AIPW		0.10	2.98		2.99	95.30
GLM-logistic	JC		0.05	2.73		2.73	95.00
Random Forest	AIPW-CF		0.03	2.73		2.74	95.30
Random Forest	JC		0.03	2.72		2.71	94.98
Stratified Permuted Block Randomization (block size of 6)
(unadjusted)	Sample Mean		0.05	2.74		2.75	95.28		3.06	97.20
GLM-logistic	AIPW		0.05	2.75		2.75	95.22		2.99	96.90
GLM-logistic	JC		0.05	2.72		2.73	95.20
Random Forest	AIPW-CF		0.05	2.71		2.72	95.44		2.74	95.68
Random Forest	JC		0.04	2.70		2.71	95.32

All estimators have negligible biases compared to their standard deviations. Correct SE is very close to SD and corresponding CP is close to 95%.

When working model is poorly specified, AIPW may not improve efficiency (vs. sample mean),

Joint calibration restores the efficiency gain (guaranteed).

AIPW using random forest with cross-fitting shows consistent variance reduction.

Joint calibration performs almost identically under SR and CAR. (Universal Applicability) [AIPW with GLM does not]

Using SE derived under SR may lose efficiency

17 of 17

All methods are available in the one-stop user-friendly R package

RobinCAR

https://github.com/tye27/RobinCar

Summary

Unadjusted, linearly adjusted and G-computation estimators are all special cases of AIPW estimator
One asymptotic theory of AIPW estimator fits all.
Joint calibration strategy achieves guaranteed efficiency gain and universal applicability simultaneously.