Coverage tells you which lines ran. It says nothing about whether your tests would catch a bug. You can delete every assertion, run covr, and still see 100%.

{muttest} measures the quality of your tests — not just how much code they execute.

The problem with coverage alone

covr tells you which lines were executed. It cannot tell you whether your assertions are strong enough to catch a real bug. A test suite full of expect_true(is.numeric(x)) checks will reach 100% coverage while missing every meaningful failure.

Mutation testing addresses this gap by asking a harder question: if this code were subtly wrong, would your tests notice?

The need for mutation testing in the age of LLMs

Many teams now use LLMs to write their tests. LLMs are good at producing syntactically correct, passing tests quickly — but they might cover only the obvious cases and miss the boundaries:

# What an LLM may write for is_adult():
test_that("is_adult works", {
  expect_true(is.numeric(is_adult(25))) # checks return type, not logic
  expect_true(is_adult(25))             # clearly an adult
  expect_false(is_adult(10))            # clearly a minor
})
# What actually catches the >= vs > boundary bug:
test_that("is_adult handles the boundary age", {
  expect_true(is_adult(18))    # kills the >= → > mutant
})

Both test suites pass. Both have 100% coverage. Only one would catch a developer accidentally writing age > 18 instead of age >= 18.

Mutation testing gives you a score that reflects assertion quality, not just execution. It gives you a concrete way to understand the real strength — and the real gaps — in an LLM-generated test suite.

How it works

• Define a set of code changes (mutations).
• Run your test suite against mutated versions of your source code.
• Measure how often the mutations are caught (i.e., cause test failures).

This reveals whether your tests are asserting the right things:

• 0% score → Your tests pass no matter what changes. Your assertions are weak.
• 100% score → Every mutation triggers a test failure. Your tests are robust.

{muttest} not only gives you the score, but it also tells you which files need stronger assertions.

Example

Given our codebase is:

#' R/is_adult.R
is_adult <- function(age) {
  age >= 18
}

And our tests are:

#' tests/testthat/test-is_adult.R
test_that("is_adult returns TRUE for adults", {
  expect_true(is_adult(25))
})
test_that("is_adult returns FALSE for minors", {
  expect_false(is_adult(10))
})

When running muttest::muttest() we’ll get a report of the mutation score:

withr::with_dir(system.file("examples", "boundary", package = "muttest"), {
  plan <- muttest::muttest_plan(
    mutators = muttest::comparison_operators()
  )
  muttest::muttest(plan)
})
#> ℹ Mutation Testing
#>   |   K |   S |   N |   E |   T |   % | Mutator  | File 
#> ✔ |   1 |   0 |   0 |   0 |   1 | 100 | >= → <=  | is_adult.R 
#> x |   1 |   1 |   0 |   0 |   2 |  50 | >= → >   | is_adult.R 
#> 
#> Duration: 1.38 s
#> 
#> ── Survived Mutants ────────────────────────────────────────────────────────────
#> is_adult.R  >= → >
#>   2-   age >= 18
#>   2+   age > 18
#> 
#> ── Results ─────────────────────────────────────────────────────────────────────
#> [ KILLED 1 | SURVIVED 1 | NO COVERAGE 0 | ERRORS 0 | TOTAL 2 | SCORE 50.0% ]

The mutation score is: