Package: scoringutils 1.2.2.9000

Nikos Bosse

scoringutils: Utilities for Scoring and Assessing Predictions

`scoringutils` facilitates the evaluation of forecasts in a convenient framework based on `data.table`. It allows user to to check their forecasts and diagnose issues, to visualise forecasts and missing data, to transform data before scoring, to handle missing forecasts, to aggregate scores, and to visualise the results of the evaluation. The package mostly focuses on the evaluation of probabilistic forecasts and allows evaluating several different forecast types and input formats. Find more information about the package in the Vignettes as well as in the accompanying paper (<doi:10.48550/arXiv.2205.07090>).

Authors:Nikos Bosse [aut, cre], Sam Abbott [aut], Hugo Gruson [aut], Johannes Bracher [ctb], Toshiaki Asakura [ctb], James Mba Azam [ctb], Sebastian Funk [aut], Michael Chirico [ctb]

scoringutils_1.2.2.9000.tar.gz
scoringutils_1.2.2.9000.zip(r-4.5)scoringutils_1.2.2.9000.zip(r-4.4)scoringutils_1.2.2.9000.zip(r-4.3)
scoringutils_1.2.2.9000.tgz(r-4.4-any)scoringutils_1.2.2.9000.tgz(r-4.3-any)
scoringutils_1.2.2.9000.tar.gz(r-4.5-noble)scoringutils_1.2.2.9000.tar.gz(r-4.4-noble)
scoringutils_1.2.2.9000.tgz(r-4.4-emscripten)scoringutils_1.2.2.9000.tgz(r-4.3-emscripten)
scoringutils.pdf |scoringutils.html✨
scoringutils/json (API)
NEWS

# Install 'scoringutils' in R:
install.packages('scoringutils', repos = c('https://epiforecasts.r-universe.dev', 'https://cloud.r-project.org'))

Peer review:

Bug tracker:https://github.com/epiforecasts/scoringutils/issues

Datasets:

On CRAN:

forecast-evaluationforecasting

67 exports 47 stars 4.23 score 40 dependencies 5 dependents 701 downloads

Last updated 2 days agofrom:e262a63297

Exports:add_relative_skillae_median_quantileae_median_sampleas_forecast_binaryas_forecast_pointas_forecast_quantileas_forecast_sampleassert_forecastbias_quantilebias_samplebrier_scorecrps_samplecustomise_metriccustomize_metricdispersion_quantiledispersion_sampledss_sampleget_correlationsget_coverageget_duplicate_forecastsget_forecast_countsget_forecast_typeget_forecast_unitget_metricsget_pairwise_comparisonsget_pitinterval_coverageinterval_coverage_deviationis_forecastis_forecast_binaryis_forecast_pointis_forecast_quantileis_forecast_samplelog_shiftlogs_binarylogs_samplemad_samplemetrics_binarymetrics_pointmetrics_quantilemetrics_samplenew_forecastoverprediction_quantileoverprediction_samplepit_sampleplot_correlationsplot_forecast_countsplot_heatmapplot_interval_coverageplot_pairwise_comparisonsplot_pitplot_quantile_coverageplot_wisquantile_scoresample_to_quantilescorese_mean_sampleselect_metricsset_forecast_unitsummarise_scoressummarize_scorestheme_scoringutilstransform_forecastsunderprediction_quantileunderprediction_samplevalidate_forecastwis

Dependencies:backportscheckmateclicolorspacedata.tableevaluatefansifarverggplot2gluegtablehighrisobandknitrlabelinglatticelifecyclemagrittrMASSMatrixMetricsmgcvmunsellnlmepillarpkgconfigR6RColorBrewerRcppRcppArmadillorlangscalesscoringRulestibbleutf8vctrsviridisLitewithrxfunyaml

Deprecated functions

Rendered fromDeprecated-functions.Rmdusingknitr::rmarkdownon Jul 25 2024.

Last update: 2024-04-04
Started: 2024-04-04

Deprecated Visualisations

Rendered fromDeprecated-visualisations.Rmdusingknitr::rmarkdownon Jul 25 2024.

Last update: 2024-07-22
Started: 2024-03-18

Scoring rules in scoringutils

Nikos Bosse

Rendered fromscoring-rules.Rmdusingknitr::rmarkdownon Jul 25 2024.

Last update: 2024-07-22
Started: 2024-04-08

Readme and manuals

Help Manual

Help pageTopics
Add relative skill scores based on pairwise comparisonsadd_relative_skill
Absolute error of the median (quantile-based version)ae_median_quantile
Absolute error of the median (sample-based version)ae_median_sample
Create a 'forecast' objectas_forecast as_forecast_binary as_forecast_point as_forecast_quantile as_forecast_sample
Common functionality for as_forecast_<type> functionsas_forecast_generic
Assert Inputs Have Matching Dimensionsassert_dims_ok_point
Assert that input is a forecast object and passes validationsassert_forecast assert_forecast.default assert_forecast.forecast_binary assert_forecast.forecast_point assert_forecast.forecast_quantile assert_forecast.forecast_sample
Validation common to all forecast typesassert_forecast_generic
Assert that forecast type is as expectedassert_forecast_type
Assert that inputs are correct for binary forecastassert_input_binary
Assert that inputs are correct for interval-based forecastassert_input_interval
Assert that inputs are correct for point forecastassert_input_point
Assert that inputs are correct for quantile-based forecastassert_input_quantile
Assert that inputs are correct for sample-based forecastassert_input_sample
Determines bias of quantile forecastsbias_quantile
Determine bias of forecastsbias_sample
Check column names are present in a data.framecheck_columns_present
Check Inputs Have Matching Dimensionscheck_dims_ok_point
Check that there are no duplicate forecastscheck_duplicates
Check that inputs are correct for binary forecastcheck_input_binary
Check that inputs are correct for interval-based forecastcheck_input_interval
Check that inputs are correct for point forecastcheck_input_point
Check that inputs are correct for quantile-based forecastcheck_input_quantile
Check that inputs are correct for sample-based forecastcheck_input_sample
Check that all forecasts have the same number of quantiles or samplescheck_number_per_forecast
Check whether an input is an atomic vector of mode 'numeric'check_numeric_vector
Helper function to convert assert statements into checkscheck_try
(Continuous) ranked probability scorecrps_sample dispersion_sample overprediction_sample underprediction_sample
Customises a metric function with additional arguments.customise_metric customize_metric
Dawid-Sebastiani scoredss_sample
Assure that data has a 'model' columnensure_model_column
Binary forecast example dataexample_binary
Point forecast example dataexample_point
Quantile example dataexample_quantile
Continuous forecast example dataexample_sample_continuous
Discrete forecast example dataexample_sample_discrete
Calculate correlation between metricsget_correlations
Get quantile and interval coverage values for quantile-based forecastsget_coverage
Find duplicate forecastsget_duplicate_forecasts
Count number of available forecastsget_forecast_counts
Infer forecast type from dataget_forecast_type
Get unit of a single forecastget_forecast_unit
Get names of the metrics that were used for scoringget_metrics
Obtain pairwise comparisons between modelsget_pairwise_comparisons
Probability integral transformation (data.frame version)get_pit
Get type of a vector or matrix of observed values or predictionsget_type
Interval coverage (for quantile-based forecasts)interval_coverage
Interval coverage deviation (for quantile-based forecasts)interval_coverage_deviation
Interval scoreinterval_score
Test whether an object is a forecast objectis_forecast is_forecast_binary is_forecast_point is_forecast_quantile is_forecast_sample
Log transformation with an additive shiftlog_shift
Logarithmic score (sample-based version)logs_sample
Determine dispersion of a probabilistic forecastmad_sample
Default metrics and scoring rules for binary forecastsmetrics_binary
Default metrics and scoring rules for point forecastsmetrics_point
Default metrics and scoring rules for quantile-based forecastsmetrics_quantile
Default metrics and scoring rules sample-based forecastsmetrics_sample
Probability integral transformation (sample-based version)pit_sample
Plot correlation between metricsplot_correlations
Visualise the number of available forecastsplot_forecast_counts
Create a heatmap of a scoring metricplot_heatmap
Plot interval coverageplot_interval_coverage
Plot heatmap of pairwise comparisonsplot_pairwise_comparisons
PIT histogramplot_pit
Plot quantile coverageplot_quantile_coverage
Plot contributions to the weighted interval scoreplot_wis
Print information about a forecast objectprint.forecast
Quantile scorequantile_score
Run a function safelyrun_safely
Change data from a sample based format to a quantile formatsample_to_quantile
Evaluate forecastsscore score.forecast_binary score.forecast_point score.forecast_quantile score.forecast_sample
Metrics for binary outcomesbrier_score logs_binary scoring-functions-binary
Squared error of the mean (sample-based version)se_mean_sample
Select metrics from a list of functionsselect_metrics
Set unit of a single forecast manuallyset_forecast_unit
Summarise scores as produced by 'score()'summarise_scores summarize_scores
Test whether column names are NOT present in a data.frametest_columns_not_present
Test whether all column names are present in a data.frametest_columns_present
Test whether data could be a binary forecast.test_forecast_type_is_binary
Test whether data could be a point forecast.test_forecast_type_is_point
Test whether data could be a quantile forecast.test_forecast_type_is_quantile
Test whether data could be a sample-based forecast.test_forecast_type_is_sample
Scoringutils ggplot2 themetheme_scoringutils
Transform forecasts and observed valuestransform_forecasts
Re-validate an existing forecast objectvalidate_forecast
Validate metricsvalidate_metrics
Weighted interval score (WIS)dispersion_quantile overprediction_quantile underprediction_quantile wis