Add model evaluation metrics to fitted model objects. These functions are wrappers around other functions that compute the metrics. The benefit of using these wrappers is that the model evaluation metrics are saved as part of the model object so that time-intensive calculations do not need to be repeated. See Details for specifics.

## Usage

```
add_criterion(
x,
criterion = c("loo", "waic"),
overwrite = FALSE,
save = TRUE,
...,
r_eff = NA
)
add_reliability(x, overwrite = FALSE, save = TRUE)
add_fit(
x,
method = c("m2", "ppmc"),
overwrite = FALSE,
save = TRUE,
...,
ci = 0.9
)
add_respondent_estimates(
x,
probs = c(0.025, 0.975),
overwrite = FALSE,
save = TRUE
)
```

## Arguments

- x
A measrfit object.

- criterion
A vector of criteria to calculate and add to the model object.

- overwrite
Logical. Indicates whether specified elements that have already been added to the estimated model should be overwritten. Default is

`FALSE`

.- save
Logical. Only relevant if a file was specified in the measrfit object passed to

`x`

. If`TRUE`

(the default), the model is re-saved to the specified file when new criteria are added to the R object. If`FALSE`

, the new criteria will be added to the R object, but the saved file will not be updated.- ...
Additional arguments passed relevant methods. See Details.

- r_eff
Vector of relative effective sample size estimates for the likelihood (

`exp(log_lik)`

) of each observation. This is related to the relative efficiency of estimating the normalizing term in self-normalizing importance sampling when using posterior draws obtained with MCMC. If MCMC draws are used and`r_eff`

is not provided then the reported PSIS effective sample sizes and Monte Carlo error estimates will be over-optimistic. If the posterior draws are independent then`r_eff=1`

and can be omitted. The warning message thrown when`r_eff`

is not specified can be disabled by setting`r_eff`

to`NA`

. See the`relative_eff()`

helper functions for computing`r_eff`

.- method
A vector of model fit methods to evaluate and add to the model object.

- ci
The confidence interval for the RMSEA, computed from the M2

- probs
The percentiles to be computed by the

`[stats::quantile()]`

function to summarize the posterior distributions of each person parameter. Only relevant if`method = "mcmc"`

was used to estimate the model.

## Value

A modified measrfit object with the corresponding slot populated with the specified information.

## Details

For `add_respondent_estimates()`

, estimated person parameters are added to
the `$respondent_estimates`

element of the fitted model.

For `add_fit()`

, model and item fit information are added to the `$fit`

element of the fitted model. This function wraps `fit_m2()`

to calculate the
M_{2} statistic (Hansen et al., 2016;
Liu et al., 2016) and/or `fit_ppmc()`

to calculate posterior predictive model
checks (Park et al., 2015; Sinharay & Almond, 2007; Sinharay et al., 2006;
Thompson, 2019), depending on which methods are specified. Additional
arguments supplied to `...`

are passed to `fit_ppmc()`

.

For `add_criterion()`

, relative fit criteria are added to the `$criteria`

element of the fitted model. This function wraps `loo()`

and/or `waic()`

,
depending on which criteria are specified, to calculate the leave-one-out
(LOO; Vehtari et al., 2017) and/or widely applicable information criteria
(WAIC; Watanabe, 2010) to fitted model objects. Additional arguments supplied
to `...`

are passed to `loo::loo.array()`

or `loo::waic.array()`

.

For `add_reliability()`

, reliability information is added to the
`$reliability`

element of the fitted model. Pattern level reliability is
described by Cui et al. (2012). Classification reliability and posterior
probability reliability are described by Johnson & Sinharay (2018, 2020),
respectively. This function wraps `reliability()`

.

## References

Cui, Y., Gierl, M. J., & Chang, H.-H. (2012). Estimating
classification consistency and accuracy for cognitive diagnostic
assessment. *Journal of Educational Measurement, 49*(1), 19-38.
doi:10.1111/j.1745-3984.2011.00158.x

Hansen, M., Cai, L., Monroe, S., & Li, Z. (2016).
Limited-information goodness-of-fit testing of diagnostic classification
item response models. *British Journal of Mathematical and Statistical
Psychology, 69*(3), 225-252. doi:10.1111/bmsp.12074

Johnson, M. S., & Sinharay, S. (2018). Measures of agreement to
assess attribute-level classification accuracy and consistency for
cognitive diagnostic assessments. *Journal of Educational Measurement,
55*(4), 635-664. doi:10.1111/jedm.12196

Johnson, M. S., & Sinharay, S. (2020). The reliability of the
posterior probability of skill attainment in diagnostic classification
models. *Journal of Educational and Behavioral Statistics, 45*(1), 5-31.
doi:10.3102/1076998619864550

Liu, Y., Tian, W., & Xin, T. (2016). An application of
M_{2} statistic to evaluate the fit
of cognitive diagnostic models. *Journal of Educational and Behavioral
Statistics, 41*(1), 3-26. doi:10.3102/1076998615621293

Park, J. Y., Johnson, M. S., Lee, Y-S. (2015). Posterior
predictive model checks for cognitive diagnostic models. *International
Journal of Quantitative Research in Education, 2*(3-4), 244-264.
doi:10.1504/IJQRE.2015.071738

Sinharay, S., & Almond, R. G. (2007). Assessing fit of cognitive
diagnostic models. *Educational and Psychological Measurement, 67*(2),
239-257. doi:10.1177/0013164406292025

Sinharay, S., Johnson, M. S., & Stern, H. S. (2006). Posterior
predictive assessment of item response theory models. *Applied
Psychological Measurement, 30*(4), 298-321.
doi:10.1177/0146621605285517

Thompson, W. J. (2019). *Bayesian psychometrics for diagnostic
assessments: A proof of concept* (Research Report No. 19-01). University
of Kansas; Accessible Teaching, Learning, and Assessment Systems.
doi:10.35542/osf.io/jzqs8

Vehtari, A., Gelman, A., & Gabry, J. (2017). Practical Bayesian
model evaluation using leave-one-out cross-validation and WAIC.
*Statistics and Computing, 27*(5), 1413-1432.
doi:10.1007/s11222-016-9696-4

Watanabe, S. (2010). Asymptotic equivalence of Bayes cross
validation and widely applicable information criterion in singular learning
theory. *Journal of Machine Learning Research, 11*(116), 3571-3594.
https://jmlr.org/papers/v11/watanabe10a.html

## Examples

```
cmds_mdm_dina <- measr_dcm(
data = mdm_data, missing = NA, qmatrix = mdm_qmatrix,
resp_id = "respondent", item_id = "item", type = "dina",
method = "optim", seed = 63277, backend = "rstan",
prior = c(prior(beta(5, 17), class = "slip"),
prior(beta(5, 17), class = "guess"))
)
cmds_mdm_dina <- add_reliability(cmds_mdm_dina)
cmds_mdm_dina <- add_fit(cmds_mdm_dina, method = "m2")
cmds_mdm_dina <- add_respondent_estimates(cmds_mdm_dina)
```