Could 14, 2026
A small Mechanical Behavior as we speak:
When utilizing not rocket science rule / merge queue, proceed to
redundantly run the complete take a look at suite on fundamental. Preserve an simply
accessible listing of latest fundamental failures — these are the flaky
exams to eradicate.
For an instance, see the “Flakes” hyperlink on
https://devhub.tigerbeetle.com
Flaky exams are exams that fail intermittently, as soon as in a thousand
runs. This is perhaps because of a real bug (assumptions about scheduling
that principally maintain) or because of instability of underlying
infrastructure (e.g., lack of ability to obtain a launch from GitHub, or
to delete a folder on Home windows). In both case, flaky exams are an enormous
productiveness drain — as the scale and complexity of take a look at suite grows,
increasingly CI runs fail spuriously, whilst every particular person take a look at
virtually at all times passes.
Flaky exams are difficult to take care of — if you’re engaged on
touchdown a PR and your CI fails because of an apparent flake, the temptation
to only re-run the take a look at suite is gigantic, particularly if there’s a
sure background dissatisfaction with infrastructure stability.
If you’re of a thoughts to do some flake squashing, then your PRs might be
inexperienced simply to spite you! And dealing off of others’ PRs would
require first to separate flakes from real failures.
This is the reason the merge queue is highly effective: if there’s a assure that
each commit on the primary department passes the exams, then each failure
on fundamental is a flake, by definition. Amassing all such failures right into a
single listing compresses time, permits to prioritize essentially the most impactful
sources of instability, and divulges correlations between failures.