Validating quantitative data model

Modeling and simulation (M&S) can be an important element in the acquisition of systems within government organizations.

Bayesian hypothesis testing can minimize the risk in model selection by properly choosing the model acceptance threshold, and its results can be used in model averaging to avoid Type I/II errors.

It is shown that Bayesian interval hypothesis testing, the reliability-based method, and the area metric-based method can account for the existence of directional bias, where the mean predictions of a numerical model may be consistently below or above the corresponding experimental observations.

Rockville, MD: Agency for Healthcare Research and Quality. The basic premise of this methodology is that such integration permits a more complete and synergistic utilization of data than do separate quantitative and qualitative data collection and analysis.

The evaluation of PCMHs provide an ideal opportunity for mixed methods studies to contribute to learning about best practices in how to implement a PCMH as well as PCMH effectiveness in achieving the triple aim outcomes of cost, quality, and patient experience of care.

This paper develops new insights into quantitative methods for the validation of computational model prediction.

Four types of methods are investigated, namely classical and Bayesian hypothesis testing, a reliability-based method, and an area metric-based method.

Validation is the process of determining the degree to which a simulation model and its associated data are an accurate representation of the real world from the perspective of the intended uses of the model [1].

MITRE SE Roles & Expectations: The MITRE systems engineer (SE) is expected to have a sound knowledge of the system being modeled and the software process for developing the model in order to provide effective technical guidance in the design and execution of plans to verify and/or validate a model, or to provide specialized technical expertise in the collection and analysis of varying types of data required to do so.

A MITRE SE is expected to be able to work directly with the developer of the system and the simulation model to provide technical insight into model verification and validation.

In most cases, the MITRE SE will be responsible for assisting the government sponsoring organization in the formal accreditation of the model.

Here, the development of a simulation model that has undergone a formal verification, validation, and accreditation (VV&A) process is not only desirable, but essential.

Tags: , ,