Calculate interval forecast coverage

This is a wrapper for the function conformalForecast::coverage. Calculates the mean coverage and the ifinn matrix for prediction intervals on validation set. If window is not NULL, a matrix of the rolling means of interval forecast coverage is also returned.

Usage

avgCoverage(object, level = 95, window = NULL, na.rm = FALSE)

Arguments

object

An object of class bb_cvforecast or cb_cvforecast.

level

Target confidence level for prediction intervals.

window

If not NULL, the rolling mean matrix for coverage is also returned.

na.rm

A logical indicating whether NA values should be stripped before the rolling mean computation proceeds.

Value

A list of class coverage with the following components:

mean

Mean coverage across the validation set.

ifinn

A indicator matrix as a multivariate time series, where the \(h\)th column holds the coverage for forecast horizon \(h\). The time index corresponds to the period for which the forecast is produced.

rollmean

If window is not NULL, a matrix of the rolling means of interval forecast coverage will be returned.

Examples

# \donttest{
library(dplyr)
library(tibble)
library(tidyr)
library(tsibble)

# Simulate data
n = 1055
set.seed(123)
sim_data <- tibble(x_lag_000 = runif(n)) |>
  mutate(
    # Add x_lags
    x_lag = lag_matrix(x_lag_000, 5)) |>
  unpack(x_lag, names_sep = "_") |>
  mutate(
    # Response variable
    y = (0.9*x_lag_000 + 0.6*x_lag_001 + 0.45*x_lag_003)^3 + rnorm(n, sd = 0.1),
    # Add an index to the data set
    inddd = seq(1, n)) |>
  drop_na() |>
  select(inddd, y, starts_with("x_lag")) |>
  # Make the data set a `tsibble`
  as_tsibble(index = inddd)

# Training set
sim_train <- sim_data[1:1000, ]
# Test set
sim_test <- sim_data[1001:1050, ]

# Index variables
index.vars <- colnames(sim_data)[3:8]

# Model fitting
pprModel <- model_ppr(data = sim_train,
                      yvar = "y",
                      index.vars = index.vars)
                      
# Conformal bootstrap prediction intervals (2-steps-ahead interval forecasts)
set.seed(12345)
pprModel_cb <- cb_cvforecast(object = pprModel,
                             data = sim_data,
                             yvar = "y",
                             predictor.vars = index.vars,
                             h = 2,
                             ncal = 30,
                             num.futures = 100,
                             window = 1000)
                             
# Mean coverage of generated 95% conformal bootstrap prediction intervals
cov_data <- avgCoverage(object = pprModel_cb)
cov_data$mean
#>       h=1       h=2 
#> 0.9473684 0.8947368 
# }