Quarantine

The quarantine feature lets you isolate broken or flaky tests in your CI pipeline without removing them entirely. Once a test is quarantined, it will still run in your CI, but its failure will be ignored for the purposes of blocking merges or marking a pipeline as failed.

This helps you keep your CI green while maintaining full visibility into test reliability and ongoing issues.

When you quarantine a test in the Mergify dashboard, its future executions will continue to be collected by Test Insights, but your CI will ignore failures.

This avoids pipeline disruption while still surfacing issues in the UI and keeping historical data intact.

Quarantined tests must still be uploaded appropriately using one of the supported CI integrations. To ensure the correct setup, refer to the documentation for your test framework.

Mergify can also quarantine a test on its own, from the health status Detection assigns it. You enable this auto-quarantine per repository from the Mitigation page. It does not act on that status as soon as a test looks unhealthy: it waits until there is enough evidence behind the verdict.

Three things have to hold. The test needs a reasonable number of recent executions to judge, it has to have been running long enough to have a pattern at all (a test whose whole history fits inside the last few minutes reads as one bad CI event rather than a flaky test), and a meaningful share of its runs has to have failed. That last floor is why a single failure among thousands of green runs never quarantines anything, however reproducible the rest of the pattern looks.

A test that has not run for a while keeps whatever health status it last had, but Mergify stops spending reruns on it, since no recent run backs that verdict any more.

Until a test clears those bars, its status still shows on the dashboard and you can quarantine it yourself; Mergify will not do it for you.

The Mitigation page on the dashboard contains the list of tests that Test Insights collected from your CI runs. From there, you can add or remove a test from quarantine.

The Mergify CLI can manage quarantines from the terminal, which is handy for scripting or for letting an AI coding agent quarantine a flaky test it just hit:

# Quarantine a test.
mergify tests quarantines add -r owner/repo \
  --reason "flaky under load, fix in progress" \
  "tests/auth/test_login.py::test_login_timeout"

# Remove it from quarantine.
mergify tests quarantines remove -r owner/repo \
  "tests/auth/test_login.py::test_login_timeout"

# List everything currently quarantined.
mergify tests quarantines list -r owner/repo

See the tests CLI reference for all flags and exit codes.

Quarantine and dequarantine actions, whether you make them yourself or Mergify applies them automatically, are recorded as events you can review on the dashboard and act on. You can also have Mergify post them to a Slack channel as they happen: see Test Insights notifications to set one up.