Bootstrapped RMSE

Generate n bootstrap replicates to compute n root mean squared errors.

bootstrap_rmse(
  model,
  data,
  n = 50,
  frac = 1,
  show_progress = TRUE,
  rmse_model = NULL
)

Arguments

model

a model object

data

data used to fit model object

n

number of bootstrap replicates (defaults to 50)

frac

fraction used in training set if cross-validation is applied (defaults to 1)

show_progress

show progress bar (defaults to TRUE)

rmse_model

numeric RMSE to show as vertical dashed line in autoplot() (defaults to NULL)

Value

A list with components

rmse_bs

numerical vector with n root mean squared errors

rmse_mod

root mean squared error for fitted (i.e. original) model

Details

To test the predictive ability of the fitted model it might be helpful to determine the variation in the computed RMSE. The variation is calculated by computing the root mean squared errors from n generated bootstrap replicates. More precisely, for each iteration a sample with replacement is taken from the data set and the model is refitted using this sample. Then, the root mean squared error is calculated.

Author

Martin Haringa

Examples

if (FALSE) {
mod1 <- glm(nclaims ~ age_policyholder, data = MTPL,
    offset = log(exposure), family = poisson())

# Use all records in MTPL
x <- bootstrap_rmse(mod1, MTPL, n = 80, show_progress = FALSE)
print(x)
autoplot(x)

# Use 80% of records to test whether predictive ability depends on which 80%
# is used. This might for example be useful in case portfolio contains large
# claim sizes
x_frac <- bootstrap_rmse(mod1, MTPL, n = 50, frac = .8,
 show_progress = FALSE)
autoplot(x_frac) # Variation is quite small for Poisson GLM
}