NEWS


scoringutils 1.2.2.9000

This update represents a major rewrite of the package and introduces breaking changes. If you want to keep using the older version, you can download it using remotes::install_github("epiforecasts/[email protected]").

The update aims to make the package more modular and customisable and overall cleaner and easier to work with. In particular, we aimed to make the suggested workflows for evaluating forecasts more explicit and easier to follow (see visualisation below). To do that, we clarified input formats and made them consistent across all functions. We refactord many functions to S3-methods and introduced forecast objects with separate classes for different types of forecasts. A new set of as_forecast_<type>() functions was introduced to validate the data and convert inputs into a forecast object (a data.table with a forecast class and an additional class corresponding to the forecast type (see below)). Another major update is the possibility for users to pass in their own scoring functions into score(). We updated and improved all function documentation and added new vignettes to guide users through the package. Internally, we refactored the code, improved input checks, updated notifications (which now use the cli package) and increased test coverage.

The most comprehensive documentation for the new package after the rewrite is the revised version of our original scoringutils paper.

Package updates

score()

Creating a forecast object

Updated workflows

Input formats

(Re-)Validating forecast objects

Pairwise comparisons and relative skill

Computing coverage values

Obtaining and plotting forecast counts

Renamed functions

Deleted functions

Function changes

Internal package updates

Documentation and testing

scoringutils 1.2.2 (2023-11-29)

Package updates

Bug fixes

scoringutils 1.2.1

Package updates

scoringutils 1.2.0

This major release contains a range of new features and bug fixes that have been introduced in minor releases since 1.1.0. The most important changes are:

Thanks to @nikosbosse, @seabbs, and @sbfnk for code and review contributions. Thanks to @bisaloo for the suggestion to use a linting GitHub Action that only triggers on changes, and @adrian-lison for the suggestion to add a warning to interval_score() if the interval range is between 0 and 1.

Package updates

example_quantile |> 
  set_forecast_unit(c("model", "location", "forecast_date", "horizon", "target_type")) |> 
  check_forecasts() |> 
  score()

Documentation for the transform_forecasts() has also been extended. This functions allows the user to easily add transformations of forecasts, as suggested in the paper "Scoring epidemiological forecasts on transformed scales". In an epidemiological context, for example, it may make sense to apply the natural logarithm first before scoring forecasts, in order to obtain scores that reflect how well models are able to predict exponential growth rates, rather than absolute values. Users can now do something like the following to score a transformed version of the data in addition to the original one:

data <- example_quantile[true_value > 0, ]
data |>
  transform_forecasts(fun = log_shift, offset = 1) |> 
  score() |> 
  summarise_scores(by = c("model", "scale"))

Here we use the log_shift() function to apply a logarithmic transformation to the forecasts. This function was introduced in scoringutils 1.1.2 as a helper function that acts just like log(), but has an additional argument offset that can add a number to every prediction and observed value before applying the log transformation.

Feature updates

scoringutils 1.1.7

Release by @seabbs in #305. Reviewed by @nikosbosse and @sbfnk.

Breaking changes

Package updates

Bug fixes

scoringutils 1.1.6

Feature updates

scoringutils 1.1.5

Feature updates

scoringutils 1.1.4

Package updates

scoringutils 1.1.3

Package updates

Package updates

scoringutils 1.1.2

Feature updates

scoringutils 1.1.1

scoringutils 1.1.0 (2023-01-30)

A minor update to the package with some bug fixes and minor changes.

Feature updates

Package updates

Bug fixes

scoringutils 1.0.0 (2022-05-13)

Major update to the package and most package functions with lots of breaking changes.

Feature updates

New functions and function changes

Bug fixes

Package data updated

Other breaking changes

scoringutils 0.1.8

Feature updates

scoringutils 0.1.7.2 (2021-07-21)

Package updates

scoringutils 0.1.7 (2021-07-14)

Feature updates

Package updates

scoringutils 0.1.5

Package updates

scoringutils 0.1.4 (2020-11-17)

Feature updates

package updates

scoringutils 0.1.3

(Potentially) Breaking changes

Feature updates

scoringutils 0.1.2

(Potentially) Breaking changes

Feature updates

Package updates

scoringutils 0.1.1

Feature updates

Package updates

scoringutils 0.1

Feature updates

Package updates