Add relative model fit criteria to model objectsSource:
Add leave-one-out (LOO; Vehtari et al., 2017) and/or widely applicable information criteria (WAIC; Watanabe, 2010) to fitted model objects.
add_criterion( x, criterion = c("loo", "waic"), overwrite = FALSE, save = TRUE, ..., r_eff = NA )
A measrfit object.
A vector of criteria to calculate and add to the model object.
Logical. Indicates whether existing criteria should be overwritten. For example, if LOO has already been added to the model object
add_criterion()is called again with
criterion = "loo", should LOO be recalculated? Default is
Logical. Only relevant if a file was specified in the measrfit object passed to
TRUE(the default), the model is re-saved to the specified file when new criteria are added to the R object. If
FALSE, the new criteria will be added to the R object, but the saved file will not be updated.
Additional arguments passed to
Vector of relative effective sample size estimates for the likelihood (
exp(log_lik)) of each observation. This is related to the relative efficiency of estimating the normalizing term in self-normalizing importance sampling when using posterior draws obtained with MCMC. If MCMC draws are used and
r_effis not provided then the reported PSIS effective sample sizes and Monte Carlo error estimates will be over-optimistic. If the posterior draws are independent then
r_eff=1and can be omitted. The warning message thrown when
r_effis not specified can be disabled by setting
NA. See the
relative_eff()helper functions for computing
A modified measrfit object with the
criteria slot populated with
the specified criteria.
Vehtari, A., Gelman, A., & Gabry, J. (2017). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing, 27(5), 1413-1432. https://doi.org/10.1007/s11222-016-9696-4
Watanabe, S. (2010). Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory. Journal of Machine Learning Research, 11(116), 3571-3594. http://jmlr.org/papers/v11/watanabe10a.html