##TODO
Example analysis
First, let us load necessary packages.
library(boot) # non-parametric bootstrap in MAIC and ML G-computation
library(copula) # simulating BC covariates from Gaussian copula
library(rstanarm) # fit outcome regression, draw outcomes in Bayesian G-computation
library(outstandR)
library(simcovariates)
Data
Next, we load the data to use in the analysis. The data comes from a simulation study in Remiro‐Azócar A, Heath A, Baio G (2020). We consider binary outcomes using the log-odds ratio as the measure of effect. The binary outcome may be response to treatment or the occurrence of an adverse event. For trials AC and BC, outcome for subject is simulated from a Bernoulli distribution with probabilities of success generated from logistic regression.
For the BC trial, the individual-level covariates and outcomes are aggregated to obtain summaries. The continuous covariates are summarized as means and standard deviations, which would be available to the analyst in the published study in a table of baseline characteristics in the RCT publication. The binary outcomes are summarized in an overall event table. Typically, the published study only provides aggregate information to the analyst.
Use the in-built gen_data()
function.
library(dplyr)
#>
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#>
#> filter, lag
#> The following objects are masked from 'package:base':
#>
#> intersect, setdiff, setequal, union
library(MASS)
#>
#> Attaching package: 'MASS'
#> The following object is masked from 'package:dplyr':
#>
#> select
N <- 200
allocation <- 2/3 # active treatment vs. placebo allocation ratio (2:1)
b_trt <- log(0.17) # conditional effect of active treatment vs. common comparator
b_X <- -log(0.5) # conditional effect of each prognostic variable
b_EM <- -log(0.67) # conditional interaction effect of each effect modifier
meanX_AC <- c(0.45, 0.45) # mean of normally-distributed covariate in AC trial
meanX_BC <- c(0.6, 0.6) # mean of each normally-distributed covariate in BC
meanX_EM_AC <- c(0.45, 0.45) # mean of normally-distributed EM covariate in AC trial
meanX_EM_BC <- c(0.6, 0.6) # mean of each normally-distributed EM covariate in BC
sdX <- c(0.4, 0.4) # standard deviation of each covariate (same for AC and BC)
sdX_EM <- c(0.4, 0.4) # standard deviation of each EM covariate
corX <- 0.2 # covariate correlation coefficient
b_0 <- -0.6 # baseline intercept coefficient ##TODO: fixed value
AC.IPD <- gen_data(N, b_trt, b_X, b_EM, b_0,
meanX_AC, sdX,
meanX_EM_AC, sdX_EM,
corX, allocation,
family = gaussian(link = "identity"))
Similarly, for the aggregate data but with the additional summarise step.
BC.IPD <- gen_data(N, b_trt, b_X, b_EM, b_0,
meanX_BC, sdX,
meanX_EM_BC, sdX_EM,
corX, allocation,
family = gaussian(link = "identity"))
cov.X <- BC.IPD %>%
summarise(across(starts_with("X"),
list(mean = mean, sd = sd),
.names = "{fn}.{col}"))
out.B <- dplyr::filter(BC.IPD, trt == 1) %>%
summarise(y.B.sum = sum(y),
y.B.bar = mean(y),
y.B.sd = sd(y),
N.B = n())
out.C <- dplyr::filter(BC.IPD, trt == 0) %>%
summarise(y.C.sum = sum(y),
y.C.bar = mean(y),
y.C.sd = sd(y),
N.C = n())
BC.ALD <- cbind.data.frame(cov.X, out.B, out.C)
This general format of data sets consist of the following.
AC.IPD
: Individual patient data
-
X*
: patient measurements -
trt
: treatment ID (integer) -
y
: continuous outcome measure
BC.ALD
: Aggregate-level data
-
mean.X*
: mean patient measurement -
sd.X*
: standard deviation of patient measurement -
mean.y
: -
sd.y
:
Note that the wildcard *
here is usually an integer from
1 or the trial identifier B, C.
Our data look like the following.
head(AC.IPD)
#> X1 X2 X3 X4 trt y
#> 1 0.42066874 1.0957407 0.37118099 1.3291540 1 0.8949999
#> 2 0.51227329 0.9079984 0.30560144 -0.1842358 1 -1.7550940
#> 3 1.00347612 0.8136847 1.25054503 1.1986315 1 -0.6521005
#> 4 0.05641685 0.5318140 0.54799796 0.6694090 1 -0.6260983
#> 5 0.42997845 0.5274304 0.09140568 0.1222184 1 -1.2882834
#> 6 0.06916082 -0.3125090 0.99150666 -0.1102693 1 -3.0170689
There are 4 correlated continuous covariates generated per subject, simulated from a multivariate normal distribution.
BC.ALD
#> mean.X1 sd.X1 mean.X2 sd.X2 mean.X3 sd.X3 mean.X4 sd.X4
#> 1 0.6081015 0.3865186 0.6462269 0.412924 0.5903258 0.3842429 0.635176 0.3831984
#> y.B.sum y.B.bar y.B.sd N.B y.C.sum y.C.bar y.C.sd N.C
#> 1 -135.0634 -1.015514 1.159787 133 12.38076 0.1847875 1.019683 67
In this case, we have 4 covariate mean and standard deviation values; and the event total, average and sample size for each treatment B, and C.
Model fitting in R
The outstandR package has been written to be easy to
use and essential consists of a single function,
outstandR()
. This can be used to run all of the different
types of model, which we will call strategies. The first two
arguments of outstandR()
are the individual and
aggregate-level data, respectively.
A strategy
argument of outstandR
takes
functions called strategy_*()
, where the wildcard
*
is replaced by the name of the particular method
required, e.g. strategy_maic()
for MAIC. Each specific
example is provided below.
MAIC
Using the individual level data for AC firstly we perform
non-parametric bootstrap of the maic.boot
function with
R = 1000
replicates. This function fits treatment
coefficient for the marginal effect for A vs C. The
returned value is an object of class boot
from the
{boot}
package. We then calculate the bootstrap mean and
variance in the wrapper function maic_boot_stats
.
The formula used in this model is
which corresponds to the following R
formula
object passed as an argument to the strategy
function.
lin_form <- as.formula("y ~ X3 + X4 + trt*X1 + trt*X2")
outstandR_maic <-
outstandR(AC.IPD, BC.ALD,
strategy = strategy_maic(formula = lin_form,
family = gaussian(link = "identity")))
The returned object is of class outstandR
.
outstandR_maic
#> Object of class 'outstandR'
#> Scale: risk_difference
#> Common treatment: C
#> Individual patient data study: AC
#> Aggregate level data study: BC
#> Confidence interval level: 0.95
#>
#> # A tibble: 3 × 5
#> Treatments Estimate Std.Error lower.0.95 upper.0.95
#> <chr> <dbl> <dbl> <dbl> <dbl>
#> 1 AB -0.339 0.0533 -0.791 0.113
#> 2 AC -1.54 0.0276 -1.87 -1.21
#> 3 BC -1.20 0.0256 -1.51 -0.887
We see that this is a list object with 3 parts, each containing
statistics between each pair of treatments. These are the mean
contrasts, variances and confidence intervals (CI), respectively. The
default CI is for 95% but can be altered in outstandR
with
the CI
argument.
STC
STC is the conventional outcome regression method. It involves fitting a regression model of outcome on treatment and covariates to the IPD. IPD effect modifiers are centred at the mean BC values.
where is the intercept, are the covariate coefficients, and are the effect modifier coefficients, are the indicator variables of effect alternative treatment. is the link function e.g. .
As already mentioned, running the STC analysis is almost identical to
the previous analysis but we now use the strategy_stc()
strategy function instead and a formula with centered covariates.
However, outstandR()
knows how to handle this so we can
simply pass the same (uncentred) formula as before.
outstandR_stc <-
outstandR(AC.IPD, BC.ALD,
strategy = strategy_stc(formula = lin_form,
family = gaussian(link = "identity")))
outstandR_stc
#> Object of class 'outstandR'
#> Scale: risk_difference
#> Common treatment: C
#> Individual patient data study: AC
#> Aggregate level data study: BC
#> Confidence interval level: 0.95
#>
#> # A tibble: 3 × 5
#> Treatments Estimate Std.Error lower.0.95 upper.0.95
#> <chr> <dbl> <dbl> <dbl> <dbl>
#> 1 AB 0.714 0.902 -1.15 2.58
#> 2 AC -0.486 0.877 -2.32 1.35
#> 3 BC -1.20 0.0256 -1.51 -0.887
For the last two approaches, we perform G-computation firstly with a frequentist MLE approach and then a Bayesian approach.
Parametric G-computation with maximum-likelihood estimation
G-computation marginalizes the conditional estimates by separating the regression modelling from the estimation of the marginal treatment effect for A versus C. First, a regression model of the observed outcome on the covariates and treatment is fitted to the AC IPD:
In the context of G-computation, this regression model is often called the “Q-model.” Having fitted the Q-model, the regression coefficients are treated as nuisance parameters. The parameters are applied to the simulated covariates to predict hypothetical outcomes for each subject under both possible treatments. Namely, a pair of predicted outcomes, also called potential outcomes, under A and under C, is generated for each subject.
By plugging treatment C into the regression fit for every simulated observation, we predict the marginal outcome mean in the hypothetical scenario in which all units are under treatment C:
To estimate the marginal or population-average treatment effect for A versus C in the linear predictor scale, one back-transforms to this scale the average predictions, taken over all subjects on the natural outcome scale, and calculates the difference between the average linear predictions:
outstandR_gcomp_ml <-
outstandR(AC.IPD, BC.ALD,
strategy = strategy_gcomp_ml(formula = lin_form,
family = gaussian(link = "identity")))
outstandR_gcomp_ml
#> Object of class 'outstandR'
#> Scale: risk_difference
#> Common treatment: C
#> Individual patient data study: AC
#> Aggregate level data study: BC
#> Confidence interval level: 0.95
#>
#> # A tibble: 3 × 5
#> Treatments Estimate Std.Error lower.0.95 upper.0.95
#> <chr> <dbl> <dbl> <dbl> <dbl>
#> 1 AB -0.348 0.0596 -0.826 0.130
#> 2 AC -1.55 0.0339 -1.91 -1.19
#> 3 BC -1.20 0.0256 -1.51 -0.887
Bayesian G-computation with MCMC
The difference between Bayesian G-computation and its maximum-likelihood counterpart is in the estimated distribution of the predicted outcomes. The Bayesian approach also marginalizes, integrates or standardizes over the joint posterior distribution of the conditional nuisance parameters of the outcome regression, as well as the joint covariate distribution.
Draw a vector of size of predicted outcomes under each set intervention from its posterior predictive distribution under the specific treatment. This is defined as where is the posterior distribution of the outcome regression coefficients , which encode the predictor-outcome relationships observed in the AC trial IPD. This is given by:
In practice, the integrals above can be approximated numerically, using full Bayesian estimation via Markov chain Monte Carlo (MCMC) sampling.
The average, variance and interval estimates of the marginal treatment effect can be derived empirically from draws of the posterior density.
We can draw a vector of size of predicted outcomes under each set intervention from its posterior predictive distribution under the specific treatment.
outstandR_gcomp_stan <-
outstandR(AC.IPD, BC.ALD,
strategy = strategy_gcomp_stan(formula = lin_form,
family = gaussian(link = "identity")))
#>
#> SAMPLING FOR MODEL 'continuous' NOW (CHAIN 1).
#> Chain 1:
#> Chain 1: Gradient evaluation took 7.5e-05 seconds
#> Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 0.75 seconds.
#> Chain 1: Adjust your expectations accordingly!
#> Chain 1:
#> Chain 1:
#> Chain 1: Iteration: 1 / 2000 [ 0%] (Warmup)
#> Chain 1: Iteration: 200 / 2000 [ 10%] (Warmup)
#> Chain 1: Iteration: 400 / 2000 [ 20%] (Warmup)
#> Chain 1: Iteration: 600 / 2000 [ 30%] (Warmup)
#> Chain 1: Iteration: 800 / 2000 [ 40%] (Warmup)
#> Chain 1: Iteration: 1000 / 2000 [ 50%] (Warmup)
#> Chain 1: Iteration: 1001 / 2000 [ 50%] (Sampling)
#> Chain 1: Iteration: 1200 / 2000 [ 60%] (Sampling)
#> Chain 1: Iteration: 1400 / 2000 [ 70%] (Sampling)
#> Chain 1: Iteration: 1600 / 2000 [ 80%] (Sampling)
#> Chain 1: Iteration: 1800 / 2000 [ 90%] (Sampling)
#> Chain 1: Iteration: 2000 / 2000 [100%] (Sampling)
#> Chain 1:
#> Chain 1: Elapsed Time: 0.1 seconds (Warm-up)
#> Chain 1: 0.104 seconds (Sampling)
#> Chain 1: 0.204 seconds (Total)
#> Chain 1:
#>
#> SAMPLING FOR MODEL 'continuous' NOW (CHAIN 2).
#> Chain 2:
#> Chain 2: Gradient evaluation took 9e-06 seconds
#> Chain 2: 1000 transitions using 10 leapfrog steps per transition would take 0.09 seconds.
#> Chain 2: Adjust your expectations accordingly!
#> Chain 2:
#> Chain 2:
#> Chain 2: Iteration: 1 / 2000 [ 0%] (Warmup)
#> Chain 2: Iteration: 200 / 2000 [ 10%] (Warmup)
#> Chain 2: Iteration: 400 / 2000 [ 20%] (Warmup)
#> Chain 2: Iteration: 600 / 2000 [ 30%] (Warmup)
#> Chain 2: Iteration: 800 / 2000 [ 40%] (Warmup)
#> Chain 2: Iteration: 1000 / 2000 [ 50%] (Warmup)
#> Chain 2: Iteration: 1001 / 2000 [ 50%] (Sampling)
#> Chain 2: Iteration: 1200 / 2000 [ 60%] (Sampling)
#> Chain 2: Iteration: 1400 / 2000 [ 70%] (Sampling)
#> Chain 2: Iteration: 1600 / 2000 [ 80%] (Sampling)
#> Chain 2: Iteration: 1800 / 2000 [ 90%] (Sampling)
#> Chain 2: Iteration: 2000 / 2000 [100%] (Sampling)
#> Chain 2:
#> Chain 2: Elapsed Time: 0.093 seconds (Warm-up)
#> Chain 2: 0.115 seconds (Sampling)
#> Chain 2: 0.208 seconds (Total)
#> Chain 2:
#>
#> SAMPLING FOR MODEL 'continuous' NOW (CHAIN 3).
#> Chain 3:
#> Chain 3: Gradient evaluation took 1.2e-05 seconds
#> Chain 3: 1000 transitions using 10 leapfrog steps per transition would take 0.12 seconds.
#> Chain 3: Adjust your expectations accordingly!
#> Chain 3:
#> Chain 3:
#> Chain 3: Iteration: 1 / 2000 [ 0%] (Warmup)
#> Chain 3: Iteration: 200 / 2000 [ 10%] (Warmup)
#> Chain 3: Iteration: 400 / 2000 [ 20%] (Warmup)
#> Chain 3: Iteration: 600 / 2000 [ 30%] (Warmup)
#> Chain 3: Iteration: 800 / 2000 [ 40%] (Warmup)
#> Chain 3: Iteration: 1000 / 2000 [ 50%] (Warmup)
#> Chain 3: Iteration: 1001 / 2000 [ 50%] (Sampling)
#> Chain 3: Iteration: 1200 / 2000 [ 60%] (Sampling)
#> Chain 3: Iteration: 1400 / 2000 [ 70%] (Sampling)
#> Chain 3: Iteration: 1600 / 2000 [ 80%] (Sampling)
#> Chain 3: Iteration: 1800 / 2000 [ 90%] (Sampling)
#> Chain 3: Iteration: 2000 / 2000 [100%] (Sampling)
#> Chain 3:
#> Chain 3: Elapsed Time: 0.091 seconds (Warm-up)
#> Chain 3: 0.104 seconds (Sampling)
#> Chain 3: 0.195 seconds (Total)
#> Chain 3:
#>
#> SAMPLING FOR MODEL 'continuous' NOW (CHAIN 4).
#> Chain 4:
#> Chain 4: Gradient evaluation took 1e-05 seconds
#> Chain 4: 1000 transitions using 10 leapfrog steps per transition would take 0.1 seconds.
#> Chain 4: Adjust your expectations accordingly!
#> Chain 4:
#> Chain 4:
#> Chain 4: Iteration: 1 / 2000 [ 0%] (Warmup)
#> Chain 4: Iteration: 200 / 2000 [ 10%] (Warmup)
#> Chain 4: Iteration: 400 / 2000 [ 20%] (Warmup)
#> Chain 4: Iteration: 600 / 2000 [ 30%] (Warmup)
#> Chain 4: Iteration: 800 / 2000 [ 40%] (Warmup)
#> Chain 4: Iteration: 1000 / 2000 [ 50%] (Warmup)
#> Chain 4: Iteration: 1001 / 2000 [ 50%] (Sampling)
#> Chain 4: Iteration: 1200 / 2000 [ 60%] (Sampling)
#> Chain 4: Iteration: 1400 / 2000 [ 70%] (Sampling)
#> Chain 4: Iteration: 1600 / 2000 [ 80%] (Sampling)
#> Chain 4: Iteration: 1800 / 2000 [ 90%] (Sampling)
#> Chain 4: Iteration: 2000 / 2000 [100%] (Sampling)
#> Chain 4:
#> Chain 4: Elapsed Time: 0.087 seconds (Warm-up)
#> Chain 4: 0.113 seconds (Sampling)
#> Chain 4: 0.2 seconds (Total)
#> Chain 4:
outstandR_gcomp_stan
#> Object of class 'outstandR'
#> Scale: risk_difference
#> Common treatment: C
#> Individual patient data study: AC
#> Aggregate level data study: BC
#> Confidence interval level: 0.95
#>
#> # A tibble: 3 × 5
#> Treatments Estimate Std.Error lower.0.95 upper.0.95
#> <chr> <dbl> <dbl> <dbl> <dbl>
#> 1 AB -0.335 0.0560 -0.799 0.129
#> 2 AC -1.54 0.0304 -1.88 -1.19
#> 3 BC -1.20 0.0256 -1.51 -0.887
Multiple imputation marginalisation
ref
outstandR_mim <-
outstandR(AC.IPD, BC.ALD,
strategy = strategy_mim(formula = lin_form,
family = gaussian(link = "identity")))
#>
#> SAMPLING FOR MODEL 'continuous' NOW (CHAIN 1).
#> Chain 1:
#> Chain 1: Gradient evaluation took 1.5e-05 seconds
#> Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 0.15 seconds.
#> Chain 1: Adjust your expectations accordingly!
#> Chain 1:
#> Chain 1:
#> Chain 1: Iteration: 1 / 4000 [ 0%] (Warmup)
#> Chain 1: Iteration: 400 / 4000 [ 10%] (Warmup)
#> Chain 1: Iteration: 800 / 4000 [ 20%] (Warmup)
#> Chain 1: Iteration: 1001 / 4000 [ 25%] (Sampling)
#> Chain 1: Iteration: 1400 / 4000 [ 35%] (Sampling)
#> Chain 1: Iteration: 1800 / 4000 [ 45%] (Sampling)
#> Chain 1: Iteration: 2200 / 4000 [ 55%] (Sampling)
#> Chain 1: Iteration: 2600 / 4000 [ 65%] (Sampling)
#> Chain 1: Iteration: 3000 / 4000 [ 75%] (Sampling)
#> Chain 1: Iteration: 3400 / 4000 [ 85%] (Sampling)
#> Chain 1: Iteration: 3800 / 4000 [ 95%] (Sampling)
#> Chain 1: Iteration: 4000 / 4000 [100%] (Sampling)
#> Chain 1:
#> Chain 1: Elapsed Time: 0.098 seconds (Warm-up)
#> Chain 1: 0.276 seconds (Sampling)
#> Chain 1: 0.374 seconds (Total)
#> Chain 1:
#>
#> SAMPLING FOR MODEL 'continuous' NOW (CHAIN 2).
#> Chain 2:
#> Chain 2: Gradient evaluation took 9e-06 seconds
#> Chain 2: 1000 transitions using 10 leapfrog steps per transition would take 0.09 seconds.
#> Chain 2: Adjust your expectations accordingly!
#> Chain 2:
#> Chain 2:
#> Chain 2: Iteration: 1 / 4000 [ 0%] (Warmup)
#> Chain 2: Iteration: 400 / 4000 [ 10%] (Warmup)
#> Chain 2: Iteration: 800 / 4000 [ 20%] (Warmup)
#> Chain 2: Iteration: 1001 / 4000 [ 25%] (Sampling)
#> Chain 2: Iteration: 1400 / 4000 [ 35%] (Sampling)
#> Chain 2: Iteration: 1800 / 4000 [ 45%] (Sampling)
#> Chain 2: Iteration: 2200 / 4000 [ 55%] (Sampling)
#> Chain 2: Iteration: 2600 / 4000 [ 65%] (Sampling)
#> Chain 2: Iteration: 3000 / 4000 [ 75%] (Sampling)
#> Chain 2: Iteration: 3400 / 4000 [ 85%] (Sampling)
#> Chain 2: Iteration: 3800 / 4000 [ 95%] (Sampling)
#> Chain 2: Iteration: 4000 / 4000 [100%] (Sampling)
#> Chain 2:
#> Chain 2: Elapsed Time: 0.088 seconds (Warm-up)
#> Chain 2: 0.302 seconds (Sampling)
#> Chain 2: 0.39 seconds (Total)
#> Chain 2:
outstandR_mim
#> Object of class 'outstandR'
#> Scale: risk_difference
#> Common treatment: C
#> Individual patient data study: AC
#> Aggregate level data study: BC
#> Confidence interval level: 0.95
#>
#> # A tibble: 3 × 5
#> Treatments Estimate Std.Error lower.0.95 upper.0.95
#> <chr> <dbl> <dbl> <dbl> <dbl>
#> 1 AB -0.339 0.0544 -0.796 0.118
#> 2 AC -1.54 0.0287 -1.87 -1.21
#> 3 BC -1.20 0.0256 -1.51 -0.887
Model comparison
Combine all outputs for relative effects table of all contrasts and methods.
knitr::kable(
data.frame(
`MAIC` = unlist(outstandR_maic$contrasts),
`STC` = unlist(outstandR_stc$contrasts),
`Gcomp ML` = unlist(outstandR_gcomp_ml$contrasts),
`Gcomp Bayes` = unlist(outstandR_gcomp_stan$contrasts),
`MIM` = unlist(outstandR_mim$contrasts))
)
MAIC | STC | Gcomp.ML | Gcomp.Bayes | MIM | |
---|---|---|---|---|---|
AB | -0.3390522 | 0.7139631 | -0.3479049 | -0.3351921 | -0.3389365 |
AC | -1.5393536 | -0.4863384 | -1.5482063 | -1.5354936 | -1.5392380 |
BC | -1.2003014 | -1.2003014 | -1.2003014 | -1.2003014 | -1.2003014 |