Getting started
Introduction
This vignette briefly outlines the functionality of
EpiSoon. To get started load the required packages.
- Load the package (
bstsfor models,ggplot2for plotting, andcowplotfor theming)
Forecast Rts, score and plot
- We use an example dataframe built into the package but this could be replaced with your own data.
EpiSoon::example_obs_rts
#> rt date
#> 1 2.490547 2020-03-01
#> 2 2.442588 2020-03-02
#> 3 2.402473 2020-03-03
#> 4 2.335572 2020-03-04
#> 5 2.266551 2020-03-05
#> 6 2.192293 2020-03-06
#> 7 2.146429 2020-03-07
#> 8 2.104371 2020-03-08
#> 9 2.059281 2020-03-09
#> 10 2.027134 2020-03-10
#> 11 2.014678 2020-03-11
#> 12 1.998946 2020-03-12
#> 13 1.968350 2020-03-13
#> 14 1.947376 2020-03-14
#> 15 1.906984 2020-03-15
#> 16 1.812842 2020-03-16
#> 17 1.718532 2020-03-17
#> 18 1.665646 2020-03-18
#> 19 1.639927 2020-03-19
#> 20 1.633795 2020-03-20
#> 21 1.682025 2020-03-21
#> 22 1.561653 2020-03-22- Fit a
bstsmodel and produce a Rt forecast. Any appropriately wrapped model can be used (seebsts_modelandfable_modelfor an examples).
rt_forecast <- forecast_rt(EpiSoon::example_obs_rts[1:10, ],
model = function(...) {
EpiSoon::bsts_model(model = function(ss, y) {
bsts::AddAutoAr(ss, y = y, lags = 10)
}, ...)
},
horizon = 21, samples = 10
)
rt_forecast
#> # A tibble: 210 × 4
#> sample date rt horizon
#> <int> <date> <dbl> <int>
#> 1 1 2020-03-11 2.01 1
#> 2 2 2020-03-11 2.03 1
#> 3 3 2020-03-11 2.02 1
#> 4 4 2020-03-11 2.02 1
#> 5 5 2020-03-11 1.93 1
#> 6 6 2020-03-11 1.92 1
#> 7 7 2020-03-11 2.01 1
#> 8 8 2020-03-11 1.97 1
#> 9 9 2020-03-11 1.92 1
#> 10 10 2020-03-11 1.98 1
#> # ℹ 200 more rows- Score the forecast
rt_scores <- score_forecast(rt_forecast, EpiSoon::example_obs_rts)
rt_scores
#> date horizon bias dss crps overprediction
#> <Date> <int> <num> <num> <num> <num>
#> 1: 2020-03-11 1 -0.4 -5.752953 0.01448112 0.0000000000
#> 2: 2020-03-12 2 0.0 -5.643835 0.02156277 0.0000000000
#> 3: 2020-03-13 3 -0.4 -4.595235 0.03580466 0.0000000000
#> 4: 2020-03-14 4 -0.4 -4.232007 0.04132858 0.0000000000
#> 5: 2020-03-15 5 -0.2 -3.826053 0.05843569 0.0000000000
#> 6: 2020-03-16 6 -0.2 -3.636286 0.05277040 0.0000000000
#> 7: 2020-03-17 7 0.2 -3.654930 0.05203256 0.0005016528
#> 8: 2020-03-18 8 0.2 -3.320587 0.05520598 0.0054659463
#> 9: 2020-03-19 9 0.0 -3.210000 0.06402717 0.0000000000
#> 10: 2020-03-20 10 0.0 -3.114569 0.07082416 0.0000000000
#> 11: 2020-03-21 11 -0.6 -2.486308 0.10059077 0.0000000000
#> 12: 2020-03-22 12 -0.2 -2.998514 0.05218940 0.0000000000
#> underprediction dispersion log_score mad ae_median se_mean
#> <num> <num> <num> <num> <num> <num>
#> 1: 0.0007849459 0.01369618 -2.0764194 0.03462397 0.01792677 0.0010662912
#> 2: 0.0000000000 0.02156277 -1.5914546 0.07344587 0.01092331 0.0005099318
#> 3: 0.0140832594 0.02172140 -1.1974079 0.11244717 0.05223657 0.0029086969
#> 4: 0.0147909713 0.02653761 -1.0342433 0.13374138 0.07584682 0.0041245428
#> 5: 0.0201723564 0.03826333 -0.6468182 0.17201561 0.11646927 0.0054443543
#> 6: 0.0064585855 0.04631182 -0.5531616 0.18204276 0.05788597 0.0001761350
#> 7: 0.0000000000 0.05153091 -0.5403748 0.19153397 0.04985039 0.0011622751
#> 8: 0.0000000000 0.04974003 -0.4776594 0.23907252 0.04689545 0.0015068377
#> 9: 0.0000000000 0.06402717 -0.3772549 0.21539400 0.01348289 0.0002625244
#> 10: 0.0000000000 0.07082416 -0.3192667 0.23435383 0.04743857 0.0022844761
#> 11: 0.0340666694 0.06652411 -0.2574754 0.18835055 0.16682118 0.0273092181
#> 12: 0.0002512048 0.05193819 -0.5365079 0.19367768 0.03692569 0.0052337365- Summarise the forecast scores
summarise_scores(rt_scores)
#> # A tibble: 10 × 8
#> score bottom lower median mean upper top sd
#> <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 ae_median 1.16e-2 3.22e-2 4.86e-2 5.77e-2 0.0624 0.153 0.0449
#> 2 bias -5.45e-1 -4 e-1 -2 e-1 -1.67e-1 0 0.2 0.253
#> 3 crps 1.64e-2 3.99e-2 5.25e-2 5.16e-2 0.0598 0.0924 0.0226
#> 4 dispersion 1.59e-2 2.53e-2 4.80e-2 4.36e-2 0.0550 0.0696 0.0192
#> 5 dss -5.72e+0 -4.32e+0 -3.65e+0 -3.87e+0 -3.19 -2.63 1.02
#> 6 log_score -1.94e+0 -1.08e+0 -5.47e-1 -8.01e-1 -0.453 -0.274 0.564
#> 7 mad 4.53e-2 1.28e-1 1.85e-1 1.64e-1 0.199 0.238 0.0635
#> 8 overprediction 0 0 0 4.97e-4 0 0.00410 0.00157
#> 9 se_mean 2.00e-4 9.27e-4 1.90e-3 4.33e-3 0.00440 0.0213 0.00747
#> 10 underprediction 0 0 5.18e-4 7.55e-3 0.0143 0.0302 0.0110- Summarise the forecast
summarised_rt_forecast <- summarise_forecast(rt_forecast)
summarised_rt_forecast
#> # A tibble: 21 × 9
#> date horizon median mean sd bottom lower upper top
#> <date> <int> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 2020-03-11 1 2.00 1.98 0.0433 1.92 1.98 2.03 2.03
#> 2 2020-03-12 2 1.99 1.98 0.0576 1.90 1.96 2.05 2.05
#> 3 2020-03-13 3 1.92 1.91 0.0845 1.80 1.80 1.92 2.04
#> 4 2020-03-14 4 1.87 1.88 0.102 1.73 1.85 1.99 2.03
#> 5 2020-03-15 5 1.79 1.83 0.130 1.66 1.66 1.81 2.00
#> 6 2020-03-16 6 1.75 1.80 0.171 1.55 1.55 1.78 2.08
#> 7 2020-03-17 7 1.77 1.75 0.166 1.49 1.64 1.88 1.98
#> 8 2020-03-18 8 1.71 1.70 0.196 1.35 1.53 1.77 1.97
#> 9 2020-03-19 9 1.63 1.62 0.211 1.23 1.48 1.72 1.88
#> 10 2020-03-20 10 1.59 1.59 0.216 1.21 1.46 1.76 1.84
#> # ℹ 11 more rows- Plot the forecast against observed data
Forecast cases, score and plot
- Forecasting cases requires the observed cases on which the observed Rt estimates were based
EpiSoon::example_obs_cases
#> # A tibble: 63 × 2
#> cases date
#> <dbl> <date>
#> 1 1 2020-01-20
#> 2 0 2020-01-21
#> 3 1 2020-01-22
#> 4 0 2020-01-23
#> 5 0 2020-01-24
#> 6 0 2020-01-25
#> 7 1 2020-01-26
#> 8 0 2020-01-27
#> 9 0 2020-01-28
#> 10 0 2020-01-29
#> # ℹ 53 more rows- It also requires an assumption to be made about the serial interval (defined using probability distribution).
EpiSoon::example_serial_interval
#> 1 2 3 4 5 6 7 8 9 10 11 12 14
#> 0.00 0.03 0.25 0.17 0.09 0.15 0.13 0.05 0.05 0.03 0.02 0.01 0.01 0.01- Forecast cases (using the case data on which the observed Rt estimates were based)
case_forecast <- forecast_cases(EpiSoon::example_obs_cases, rt_forecast,
serial_interval = EpiSoon::example_serial_interval
)
case_forecast
#> sample date cases horizon
#> <num> <Date> <int> <int>
#> 1: 1 2020-03-11 188 1
#> 2: 1 2020-03-12 188 2
#> 3: 1 2020-03-13 261 3
#> 4: 1 2020-03-14 263 4
#> 5: 1 2020-03-15 327 5
#> ---
#> 206: 10 2020-03-27 1103 17
#> 207: 10 2020-03-28 1232 18
#> 208: 10 2020-03-29 1262 19
#> 209: 10 2020-03-30 1414 20
#> 210: 10 2020-03-31 1475 21- Score the cases forecast
case_scores <- score_case_forecast(case_forecast, EpiSoon::example_obs_cases)
#> Warning: Predictions appear to be integer-valued.
#> ! The log score uses kernel density estimation, which may not be appropriate
#> for integer-valued forecasts.
#> ℹ See the scoringRules package for alternatives for discrete probability
#> distributions.
case_scores
#> date horizon bias dss crps overprediction underprediction
#> <Date> <int> <num> <num> <num> <num> <num>
#> 1: 2020-03-11 1 0.2 5.001724 3.47 0.8 0
#> 2: 2020-03-12 2 0.3 4.100388 1.95 0.2 0
#> 3: 2020-03-13 3 0.8 7.909026 18.07 12.4 0
#> 4: 2020-03-14 4 0.6 6.027251 7.39 4.6 0
#> 5: 2020-03-15 5 0.4 7.890742 20.28 13.2 0
#> 6: 2020-03-16 6 1.0 10.239892 47.85 36.0 0
#> 7: 2020-03-17 7 1.0 10.980322 64.72 47.2 0
#> 8: 2020-03-18 8 0.8 10.688826 70.49 48.4 0
#> 9: 2020-03-19 9 0.4 9.896952 46.02 4.0 0
#> 10: 2020-03-20 10 0.8 10.196352 53.21 15.2 0
#> 11: 2020-03-21 11 0.0 10.354958 59.01 0.0 0
#> 12: 2020-03-22 12 0.8 12.565235 171.19 119.0 0
#> dispersion log_score mad ae_median se_mean
#> <num> <num> <num> <num> <num>
#> 1: 2.67 3.572189 11.8608 5.0 12.25
#> 2: 1.75 3.119982 8.1543 1.5 5.29
#> 3: 5.67 4.715069 22.2390 26.0 723.61
#> 4: 2.79 3.863162 11.1195 13.0 151.29
#> 5: 7.08 4.975225 21.4977 41.5 973.44
#> 6: 11.85 5.686328 50.4084 62.0 5343.61
#> 7: 17.52 5.945639 62.2692 83.0 9409.00
#> 8: 22.09 6.027108 106.0059 94.0 12432.25
#> 9: 42.02 6.030329 137.1405 83.5 6037.29
#> 10: 38.01 6.108769 135.6579 79.5 8817.21
#> 11: 59.01 6.478646 238.6986 29.0 96.04
#> 12: 52.19 6.958192 177.9120 281.5 72792.04- Summarise the cases scores
summarise_scores(case_scores)
#> # A tibble: 10 × 8
#> score bottom lower median mean upper top sd
#> <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 ae_median 2.46 22.8 51.8 66.6 83.1 230. 75.2
#> 2 bias 0.055 0.375 0.7 0.592 0.8 1 0.326
#> 3 crps 2.37 15.4 46.9 47.0 60.4 143. 46.2
#> 4 dispersion 2.00 4.95 14.7 21.9 39.0 57.1 20.7
#> 5 dss 4.35 7.42 10.0 8.82 10.4 12.1 2.63
#> 6 log_score 3.24 4.50 5.82 5.29 6.05 6.83 1.23
#> 7 mad 8.97 19.1 56.3 81.9 136. 222. 76.4
#> 8 overprediction 0.0550 3.20 12.8 25.1 38.8 99.6 34.5
#> 9 se_mean 7.20 137. 3159. 9733. 8965. 56193. 20336.
#> 10 underprediction 0 0 0 0 0 0 0- Summarise the cases forecast
summarised_case_forecast <- summarise_case_forecast(case_forecast)
summarised_case_forecast
#> # A tibble: 21 × 9
#> date horizon median mean sd bottom lower upper top
#> <date> <int> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 2020-03-11 1 172 170. 12.3 145 161 175 188
#> 2 2020-03-12 2 196. 196. 7.80 186 186 196 211
#> 3 2020-03-13 3 234 235. 19.9 199 219 249 261
#> 4 2020-03-14 4 264 263. 12.2 246 256 269 283
#> 5 2020-03-15 5 314. 304. 30.7 260 298 327 351
#> 6 2020-03-16 6 328 339. 47.6 279 279 336 432
#> 7 2020-03-17 7 379 393 60.1 314 314 389 491
#> 8 2020-03-18 8 437 454. 85.1 319 319 442 585
#> 9 2020-03-19 9 483 477. 116. 290 393 578 622
#> 10 2020-03-20 10 534. 548. 128. 319 469 642 750
#> # ℹ 11 more rows- Plot the forecast against observed case data
Use iterative fitting to explore a forecast
- To explore the quality of a models forecast it can help to
iteratively forecast from each available data point. This is supported
in
EpiSoonusing the following:
it_rt_forecast <- iterative_rt_forecast(EpiSoon::example_obs_rts,
model = function(...) {
EpiSoon::bsts_model(model = function(ss, y) {
bsts::AddAutoAr(ss, y = y, lags = 10)
}, ...)
},
horizon = 7, samples = 10, min_points = 4
)
it_rt_forecast
#> # A tibble: 1,260 × 5
#> forecast_date sample date rt horizon
#> <chr> <int> <date> <dbl> <int>
#> 1 2020-03-05 1 2020-03-06 2.16 1
#> 2 2020-03-05 2 2020-03-06 2.18 1
#> 3 2020-03-05 3 2020-03-06 2.25 1
#> 4 2020-03-05 4 2020-03-06 2.21 1
#> 5 2020-03-05 5 2020-03-06 2.21 1
#> 6 2020-03-05 6 2020-03-06 2.20 1
#> 7 2020-03-05 7 2020-03-06 2.23 1
#> 8 2020-03-05 8 2020-03-06 2.24 1
#> 9 2020-03-05 9 2020-03-06 2.16 1
#> 10 2020-03-05 10 2020-03-06 2.15 1
#> # ℹ 1,250 more rows- We can then iteratively forecast cases using the following:
it_cases_forecast <- iterative_case_forecast(
it_fit_samples = it_rt_forecast,
cases = EpiSoon::example_obs_cases,
serial_interval = EpiSoon::example_serial_interval
)
it_cases_forecast
#> forecast_date sample date cases horizon
#> <char> <num> <Date> <int> <int>
#> 1: 2020-03-05 1 2020-03-06 82 1
#> 2: 2020-03-05 1 2020-03-07 106 2
#> 3: 2020-03-05 1 2020-03-08 95 3
#> 4: 2020-03-05 1 2020-03-09 119 4
#> 5: 2020-03-05 1 2020-03-10 115 5
#> ---
#> 1256: 2020-03-22 10 2020-03-25 773 3
#> 1257: 2020-03-22 10 2020-03-26 811 4
#> 1258: 2020-03-22 10 2020-03-27 787 5
#> 1259: 2020-03-22 10 2020-03-28 872 6
#> 1260: 2020-03-22 10 2020-03-29 917 7- All functionality shown above is also supported for iterative forecasting.
Evaluate a model
In real world use we are likely to want to evaluate a model by
iteratively forecasting Rts and cases, summarising these forecasts,
scoring them and then returning them in a sensible format. These steps
are all contained in the evaluate_model function.
model_eval <- evaluate_model(EpiSoon::example_obs_rts,
EpiSoon::example_obs_cases,
model = function(...) {
EpiSoon::bsts_model(model = function(ss, y) {
bsts::AddAutoAr(ss, y = y, lags = 10)
}, ...)
},
horizon = 21, samples = 10,
serial_interval = EpiSoon::example_serial_interval
)
#> Warning: Predictions appear to be integer-valued.
#> ! The log score uses kernel density estimation, which may not be appropriate
#> for integer-valued forecasts.
#> ℹ See the scoringRules package for alternatives for discrete probability
#> distributions.
#> Predictions appear to be integer-valued.
#> ! The log score uses kernel density estimation, which may not be appropriate
#> for integer-valued forecasts.
#> ℹ See the scoringRules package for alternatives for discrete probability
#> distributions.
#> Predictions appear to be integer-valued.
#> ! The log score uses kernel density estimation, which may not be appropriate
#> for integer-valued forecasts.
#> ℹ See the scoringRules package for alternatives for discrete probability
#> distributions.
#> Predictions appear to be integer-valued.
#> ! The log score uses kernel density estimation, which may not be appropriate
#> for integer-valued forecasts.
#> ℹ See the scoringRules package for alternatives for discrete probability
#> distributions.
#> Predictions appear to be integer-valued.
#> ! The log score uses kernel density estimation, which may not be appropriate
#> for integer-valued forecasts.
#> ℹ See the scoringRules package for alternatives for discrete probability
#> distributions.
#> Predictions appear to be integer-valued.
#> ! The log score uses kernel density estimation, which may not be appropriate
#> for integer-valued forecasts.
#> ℹ See the scoringRules package for alternatives for discrete probability
#> distributions.
#> Predictions appear to be integer-valued.
#> ! The log score uses kernel density estimation, which may not be appropriate
#> for integer-valued forecasts.
#> ℹ See the scoringRules package for alternatives for discrete probability
#> distributions.
#> Predictions appear to be integer-valued.
#> ! The log score uses kernel density estimation, which may not be appropriate
#> for integer-valued forecasts.
#> ℹ See the scoringRules package for alternatives for discrete probability
#> distributions.
#> Predictions appear to be integer-valued.
#> ! The log score uses kernel density estimation, which may not be appropriate
#> for integer-valued forecasts.
#> ℹ See the scoringRules package for alternatives for discrete probability
#> distributions.
#> Predictions appear to be integer-valued.
#> ! The log score uses kernel density estimation, which may not be appropriate
#> for integer-valued forecasts.
#> ℹ See the scoringRules package for alternatives for discrete probability
#> distributions.
#> Predictions appear to be integer-valued.
#> ! The log score uses kernel density estimation, which may not be appropriate
#> for integer-valued forecasts.
#> ℹ See the scoringRules package for alternatives for discrete probability
#> distributions.
#> Predictions appear to be integer-valued.
#> ! The log score uses kernel density estimation, which may not be appropriate
#> for integer-valued forecasts.
#> ℹ See the scoringRules package for alternatives for discrete probability
#> distributions.
#> Predictions appear to be integer-valued.
#> ! The log score uses kernel density estimation, which may not be appropriate
#> for integer-valued forecasts.
#> ℹ See the scoringRules package for alternatives for discrete probability
#> distributions.
#> Predictions appear to be integer-valued.
#> ! The log score uses kernel density estimation, which may not be appropriate
#> for integer-valued forecasts.
#> ℹ See the scoringRules package for alternatives for discrete probability
#> distributions.
#> Predictions appear to be integer-valued.
#> ! The log score uses kernel density estimation, which may not be appropriate
#> for integer-valued forecasts.
#> ℹ See the scoringRules package for alternatives for discrete probability
#> distributions.
#> Predictions appear to be integer-valued.
#> ! The log score uses kernel density estimation, which may not be appropriate
#> for integer-valued forecasts.
#> ℹ See the scoringRules package for alternatives for discrete probability
#> distributions.
#> Predictions appear to be integer-valued.
#> ! The log score uses kernel density estimation, which may not be appropriate
#> for integer-valued forecasts.
#> ℹ See the scoringRules package for alternatives for discrete probability
#> distributions.
#> Predictions appear to be integer-valued.
#> ! The log score uses kernel density estimation, which may not be appropriate
#> for integer-valued forecasts.
#> ℹ See the scoringRules package for alternatives for discrete probability
#> distributions.
model_eval
#> $forecast_rts
#> # A tibble: 399 × 10
#> forecast_date date horizon median mean sd bottom lower upper top
#> <chr> <date> <int> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 2020-03-04 2020-03-05 1 2.22 2.24 0.241 1.85 2.13 2.45 2.61
#> 2 2020-03-04 2020-03-06 2 2.16 2.14 0.321 1.69 1.96 2.43 2.64
#> 3 2020-03-04 2020-03-07 3 2.06 2.03 0.512 1.29 1.83 2.40 3.02
#> 4 2020-03-04 2020-03-08 4 1.83 1.89 0.612 1.06 1.37 2.15 3.08
#> 5 2020-03-04 2020-03-09 5 1.68 1.77 0.664 0.892 1.34 2.09 3.01
#> 6 2020-03-04 2020-03-10 6 1.67 1.68 0.782 0.731 1.08 2.09 3.09
#> 7 2020-03-04 2020-03-11 7 1.39 1.49 0.769 0.646 0.646 1.63 2.74
#> 8 2020-03-04 2020-03-12 8 1.29 1.40 0.825 0.472 0.472 1.51 2.84
#> 9 2020-03-04 2020-03-13 9 1.25 1.25 0.802 0.317 0.317 1.47 2.40
#> 10 2020-03-04 2020-03-14 10 1.19 1.17 0.746 0.260 0.515 1.70 2.38
#> # ℹ 389 more rows
#>
#> $rt_scores
#> forecast_date date horizon bias dss crps
#> <char> <Date> <int> <num> <num> <num>
#> 1: 2020-03-04 2020-03-05 1 0.0 -2.9372262 0.06759288
#> 2: 2020-03-04 2020-03-06 2 0.0 -2.3539689 0.09955286
#> 3: 2020-03-04 2020-03-07 3 0.0 -1.3847999 0.12868520
#> 4: 2020-03-04 2020-03-08 4 -0.2 -0.9461037 0.18111714
#> 5: 2020-03-04 2020-03-09 5 -0.4 -0.7085712 0.23085618
#> ---
#> 167: 2020-03-19 2020-03-21 2 -1.0 7.4313717 0.09737761
#> 168: 2020-03-19 2020-03-22 3 -0.6 -5.8652057 0.01825756
#> 169: 2020-03-20 2020-03-21 1 -0.6 -4.5023594 0.04039481
#> 170: 2020-03-20 2020-03-22 2 -0.4 -5.4202159 0.01686785
#> 171: 2020-03-21 2020-03-22 1 1.0 -3.2491872 0.05420992
#> overprediction underprediction dispersion log_score mad
#> <num> <num> <num> <num> <num>
#> 1: 0.00000000 -1.387779e-17 0.06759288 -0.27146482 0.26320379
#> 2: 0.00000000 0.000000e+00 0.09955286 0.09567151 0.39618179
#> 3: 0.00000000 0.000000e+00 0.12868520 0.42541376 0.43825696
#> 4: 0.00000000 1.216084e-02 0.16895630 0.67067789 0.57457087
#> 5: 0.00000000 4.406386e-02 0.18679233 0.83984366 0.55690197
#> ---
#> 167: 0.00000000 8.723458e-02 0.01014302 4.63199250 0.03661510
#> 168: 0.00000000 8.233575e-03 0.01002398 -1.80313140 0.03890749
#> 169: 0.00000000 2.587416e-02 0.01452065 -1.12309022 0.07861717
#> 170: 0.00000000 3.332371e-03 0.01353548 -1.71631872 0.05714595
#> 171: 0.04371258 0.000000e+00 0.01049734 -0.72477498 0.04078146
#> ae_median se_mean
#> <num> <num>
#> 1: 0.04605271 0.0005484063
#> 2: 0.03374633 0.0024870824
#> 3: 0.08204136 0.0136303920
#> 4: 0.27472614 0.0480782416
#> 5: 0.37794999 0.0857160883
#> ---
#> 167: 0.11211035 0.0131034752
#> 168: 0.02776417 0.0004761249
#> 169: 0.04757344 0.0036156210
#> 170: 0.01855471 0.0001272526
#> 171: 0.07563612 0.0063175595
#>
#> $forecast_cases
#> # A tibble: 171 × 10
#> forecast_date date horizon median mean sd bottom lower upper top
#> <chr> <date> <int> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 2020-03-04 2020-03-05 1 69.5 71.4 14.2 51 58 79 95
#> 2 2020-03-04 2020-03-06 2 89 85.7 16.9 59 82 102 111
#> 3 2020-03-04 2020-03-07 3 91.5 94.2 33.8 49 69 113 147
#> 4 2020-03-04 2020-03-08 4 104 108. 45.7 47 86 146 190
#> 5 2020-03-04 2020-03-09 5 122. 127. 68.7 49 49 148 257
#> 6 2020-03-04 2020-03-10 6 136. 150. 101. 35 92 211 365
#> 7 2020-03-04 2020-03-11 7 139 158. 122. 38 38 189 396
#> 8 2020-03-04 2020-03-12 8 152. 186. 179. 25 25 201 594
#> 9 2020-03-04 2020-03-13 9 180 205. 204. 17 17 262 604
#> 10 2020-03-04 2020-03-14 10 164. 216. 223. 9 9 252 622
#> # ℹ 161 more rows
#>
#> $case_scores
#> sample forecast_date date horizon bias dss crps
#> <char> <char> <Date> <int> <num> <num> <num>
#> 1: 1 2020-03-04 2020-03-05 1 0.3 5.588466 4.74
#> 2: 1 2020-03-04 2020-03-06 2 0.5 6.177548 8.63
#> 3: 1 2020-03-04 2020-03-07 3 0.0 6.975718 9.28
#> 4: 1 2020-03-04 2020-03-08 4 0.0 7.559519 12.79
#> 5: 1 2020-03-04 2020-03-09 5 0.0 8.384188 21.15
#> ---
#> 167: 1 2020-03-19 2020-03-21 2 -1.0 98.034876 112.51
#> 168: 1 2020-03-19 2020-03-22 3 1.0 24.102698 140.85
#> 169: 1 2020-03-20 2020-03-21 1 -1.0 29.554353 99.19
#> 170: 1 2020-03-20 2020-03-22 2 1.0 32.332309 146.30
#> 171: 1 2020-03-21 2020-03-22 1 1.0 54.493075 184.63
#> overprediction underprediction dispersion log_score mad ae_median
#> <num> <num> <num> <num> <num> <num>
#> 1: 6.000000e-01 0.0 4.14 3.815681 15.5673 6.5
#> 2: 4.400000e+00 0.0 4.23 4.294925 15.5673 16.0
#> 3: 0.000000e+00 0.0 9.28 4.622172 32.6172 3.5
#> 4: 0.000000e+00 0.0 12.79 4.985931 52.6323 2.0
#> 5: 3.552714e-15 0.0 21.15 5.398460 81.5430 6.5
#> ---
#> 167: 0.000000e+00 108.9 3.61 79.753110 10.3782 116.0
#> 168: 1.265000e+02 0.0 14.35 13.390032 41.5128 156.0
#> 169: 0.000000e+00 92.5 6.69 13.991874 31.8759 114.5
#> 170: 1.391000e+02 0.0 7.20 21.440220 29.6520 158.0
#> 171: 1.737000e+02 0.0 10.93 38.001637 42.2541 196.5
#> se_mean
#> <num>
#> 1: 70.56
#> 2: 161.29
#> 3: 38.44
#> 4: 39.69
#> 5: 123.21
#> ---
#> 167: 14256.36
#> 168: 26634.24
#> 169: 12588.84
#> 170: 26994.49
#> 171: 40401.00- All functionality outlined above can be applied to this output but a
special plotting function (
plot_forecast_evaluation) is also provided. First evaluate the Rt forecast against observed values.
- Then evaluate forecast cases against observed values.
plot_forecast_evaluation(model_eval$forecast_cases,
EpiSoon::example_obs_cases,
horizon_to_plot = 7
)
Wrapper functions
EpiSoon provides several wrapper functions
(compare_models and compare_timeseries). These
both wrap evaluate_model and can be used to rapidly explore
several forecasting models (compare_models) against
multiple time series (compare_timeseries). All lower level
summary and plotting functions can be then used with the output of these
wrappers to explore the results. See the function documentation for
further details.
Supporting generic modelling packages
EpiSoon supports the use of generic forecasting models
if they are used in a wrapper that accepts a standardised set of inputs
and outputs its forecast in the form the package expects. Examples of
model wrappers are those for the bsts and
fable packages (bsts_model and
fable_model). See the examples and documentation for
fable_model for further details.