R/psfmi_stab.R
psfmi_stab.Rdpsfmi_stab Stability analysis of predictors and prediction models selected with
the psfmi_lr, psfmi_coxr or psfmi_mm functions of the psfmi package.
psfmi_stab(
pobj,
boot_method = NULL,
nboot = 20,
p.crit = 0.05,
start_model = TRUE,
direction = NULL
)An object of class pmods (pooled models), produced by a previous call to
psfmi_lr, psfmi_coxr or psfmi_mm.
A single string to define the bootstrap method. Use "single" after a call to
psfmi_lr and psfmi_coxr and "cluster" after a call to psfmi_mm.
A numerical scalar. Number of bootstrap samples to evaluate the stability. Default is 20.
A numerical scalar. Used as P-value selection criterium during bootstrap model selection.
If TRUE the bootstrap evaluation takes place from the start model of object pobj, if FALSE the final model is used for the evaluation.
The direction of predictor selection, "BW" for backward selection and "FW" for forward selection. #'
A psfmi_stab object from which the following objects can be extracted: bootstrap
inclusion (selection) frequency of each predictor bif, total number each predictor is
included in the bootstrap samples as bif_total, percentage a predictor is selected
in each bootstrap sample as bif_perc and number of times a prediction model is selected in
the bootstrap samples as model_stab.
The function evaluates predictor selection frequency in stratified or cluster bootstrap samples.
The stratification factor is the variable that separates the imputed datasets. The same bootstrap cases
are drawn in each bootstrap sample. It uses as input an object of class pmods as a result of a
previous call to the psfmi_lr, psfmi_coxr or psfmi_mm functions.
In combination with the psfmi_mm function a cluster bootstrap method is used where bootstrapping
is used on the level of the clusters only (and not also within the clusters).
https://mwheymans.github.io/psfmi/articles/psfmi_StabilityAnalysis.html
Heymans MW, van Buuren S. et al. Variable selection under multiple imputation using the bootstrap in a prognostic study. BMC Med Res Methodol. 2007;13:7-33.
Eekhout I, van de Wiel MA, Heymans MW. Methods for significance testing of categorical covariates in logistic regression models after multiple imputation: power and applicability analysis. BMC Med Res Methodol. 2017;17(1):129.
Sauerbrei W, Schumacher M. A bootstrap resampling procedure for model building: application to the Cox regression model. Stat Med. 1992;11:2093–109.
Royston P, Sauerbrei W (2008) Multivariable model-building – a pragmatic approach to regression analysis based on fractional polynomials for modelling continuous variables. (2008). Chapter 8, Model Stability. Wiley, Chichester
Heinze G, Wallisch C, Dunkler D. Variable selection - A review and recommendations for the practicing statistician. Biom J. 2018;60(3):431-449.
http://missingdatasolutions.rbind.io/
pool_lr <- psfmi_coxr(formula = Surv(Time, Status) ~ Pain + factor(Satisfaction) +
rcs(Tampascale,3) + Radiation + Radiation*factor(Satisfaction) + Age + Duration +
Previous + Radiation*rcs(Tampascale, 3), data=lbpmicox, p.crit = 0.157, direction="FW",
nimp=5, impvar="Impnr", keep.predictors = NULL, method="D1")
#> Entered at Step 1 is - Pain
#> Entered at Step 2 is - Duration
#> Entered at Step 3 is - rcs(Tampascale,3)
#>
#> Selection correctly terminated,
#> No new variables entered the model
pool_lr$RR_Model
#> NULL
pool_lr$multiparm
#> $`Step 0 - selected - Pain`
#> p-value D1
#> Pain 0.002972173
#> Radiation 0.181042842
#> Age 0.351345242
#> Duration 0.044028841
#> Previous 0.714681296
#> factor(Satisfaction) 0.708284253
#> rcs(Tampascale,3) 0.032223680
#> factor(Satisfaction)*Radiation 0.712860014
#> rcs(Tampascale,3)*Radiation 0.452891382
#>
#> $`Step 1 - selected - Duration`
#> p-value D1
#> Radiation 0.31938607
#> Age 0.22333023
#> Duration 0.04853205
#> Previous 0.54425893
#> factor(Satisfaction) 0.76439993
#> rcs(Tampascale,3) 0.07594972
#> factor(Satisfaction)*Radiation 0.66328781
#> rcs(Tampascale,3)*Radiation 0.44642240
#>
#> $`Step 2 - selected - rcs(Tampascale,3)`
#> p-value D1
#> Radiation 0.32679294
#> Age 0.40528789
#> Previous 0.44216497
#> factor(Satisfaction) 0.81050733
#> rcs(Tampascale,3) 0.06066281
#> factor(Satisfaction)*Radiation 0.70440131
#> rcs(Tampascale,3)*Radiation 0.35955920
#>
if (FALSE) {
stab_res <- psfmi_stab(pool_lr, direction="FW", start_model = TRUE,
boot_method = "single", nboot=20, p.crit=0.05)
stab_res$bif
stab_res$bif_perc
stab_res$model_stab
}