Skip to contents

testhta

A Unit Testing and Verification Framework for Health Technology Assessment (HTA) Models

testhta provides a robust, software-engineering-inspired framework for applying unit testing and behavioral verification to health economic and health technology assessment (HTA) models. By leveraging the testthat testing framework, testhta enables modelers to write automated checks that verify mathematical, logical, and structural correctness under boundary conditions. The package includes a built-in cohort Markov model as a demonstration environment, showing how to define, group, and execute these checks in practice.

Ensuring HTA model quality should not require manual spreadsheet audits. testhta makes verification automated, reproducible, and transparent—enhancing decision-maker confidence in cost-effectiveness results.

Key Features

  • 🧪 Verification Helpers: Out-of-the-box functions to test model behavior against expected bounds (check_model_qalys(), check_model_costs(), check_model_le()).
  • 🔄 Relational Comparators: Compare model behaviors across parameter spaces using compare_model_runs() (e.g., verifying that discounting reduces future values).
  • 🔗 Tidy Test API: Modify parameters cleanly using setter functions (set_discount_rate(), set_time_horizon()) integrated with the base R pipe (|>) to isolate test runs.
  • ⚙️ Markov Engine Example Case Study: An example discrete-time cohort Markov simulation environment built with time-dependent transitions, discounting, and state/transition-specific costs and utilities to demonstrate the test framework.

Installation

You can install the development version of testhta from GitHub:

# install.packages("devtools")
devtools::install_github("n8thangreen/testhta")

Quick Start: Build, Run, and Verify

Here is a basic example demonstrating how to run a cohort Markov model and verify the economic logic of discounting using the tidy API.

1. Set Up and Run the Model

We use the package’s built-in baseline dataset test_data to execute a cost-effectiveness model.

library(testhta)

# Load baseline dataset
data(test_data)

# Run the Markov simulation
results <- run_model(test_data)

# Extract key HTA outputs using getters
get_qalys(results)
#>   without_drug      with_drug 
#>       11.45892       12.12450 

get_costs(results)
#>   without_drug      with_drug 
#>       22354.21       34211.55 

get_icer(results)
#> [1] 17814.77

2. Modify Parameters with Pipe Setters

Easily perform scenario analyses or sensitivity sweeps by piping parameter modifications:

# Evaluate a 10-year time horizon with no discounting
scenario_results <- test_data |>
  set_time_horizon(10) |>
  set_discount_rate(0) |>
  run_model()

get_icer(scenario_results)
#> [1] 12903.45

3. Automated Validation Assertions

testhta provides assertion functions that integrate seamlessly into standard R package tests (using testthat).

library(testthat)

# Verify that QALYs with standard discounting (3.5%) are strictly less 
# than QALYs without discounting
compare_model_runs(
  extractor_fn = get_qalys,
  comparison_fn = expect_lt,
  params_1 = list(discount_rate = 0.035),
  params_2 = list(discount_rate = 0),
  data = test_data,
  label = "Discounted QALYs are lower than undiscounted QALYs"
)

The HTA Verification Framework

The validation checks in testhta are based on formal software verification methodologies described by Tappenden et al. (2014) and Elbasha & Dasbach (2017). The framework checks “black-box” model behavior under extreme inputs to identify structural bugs across 16 core test scenarios (T01–T16), covering:

  • QALY Bounding Tests (T01, T02, T06, T07, T08)
  • Cost Bounding & Sensitivity Tests (T06, T09, T10, T11, T12)
  • Population & Life Expectancy Bounding Tests (T04, T05)
  • Treatment Arm Comparison & Symmetry Tests (T14, T15, T16)
  • Probabilistic Sensitivity Analysis (PSA) Sampling Tests (T13)

For the complete test matrix, detailed descriptions, and code examples, see the dedicated HTA Verification Framework Vignette (vignettes/verification-framework.Rmd).

References

  1. Tappenden, P., & Chilcott, J. B. (2014). Avoiding and Identifying Errors and Other Threats to the Credibility of Health Economic Models. PharmacoEconomics, 32, 967–979. https://doi.org/10.1007/s40273-014-0186-2
  2. Elbasha, E. H., & Dasbach, E. J. (2017). Verification of Decision-Analytic Models for Health Economic Evaluations: An Overview. PharmacoEconomics, 35, 673–683. https://doi.org/10.1007/s40273-017-0508-2
  3. Alarid-Escudero, F., et al. (2019). A Need for Change! A Coding Framework for Improving Transparency in Decision Modeling. PharmacoEconomics, 37(11), 1329–1339. https://doi.org/10.1007/s40273-019-00837-x