Bug counts and triage

The curve everybody watches

Late in any project, somebody pins a graph to the wall — digital or literal. One axis is time, the other is open bug count. The line that team leads and producers want to see is the one that trends steadily downward, hitting zero at ship. In practice it rarely does. Understanding what that graph actually measures, and what it silently conceals, is most of what QA experience is made of.

The standard model is called the bug curve, or sometimes the Rayleigh curve when people are being formal about it. Open issues climb as testing intensifies and features land, peak somewhere around content complete or the first serious certification candidate, then decline as fixes outpace new finds. A healthy project shows that decline arriving early enough and steeply enough to reach an acceptable residual before the hard ship date. An unhealthy one shows the peak still rising when it should be falling, or a false plateau that masks a category of issues nobody is testing well.

What the count does not tell you is weight. A graph that falls from four hundred bugs to sixty looks promising until you discover that the forty fixes were cosmetic text errors and the sixty open issues include three crashers on the lead platform and a save-corruption bug that reproduces reliably on forty percent of saves. This is why triage exists: not to manage a number, but to manage a risk profile.

Triage as a production instrument

Triage is the meeting — usually daily in the final stretch — where someone with authority over the schedule looks at each open issue and assigns it a priority: must fix before ship, fix if time allows, deferred to a patch, or closed as by-design. The vocabulary varies by studio. The function does not. Someone has to decide what the game will be when it ships, and triage is where those decisions accumulate.

The two failure modes are symmetric. Over-triage, and you fix things that don't matter while crashers linger in a backlog nobody reads. Under-triage, and you ship embarrassments you knew about and misjudged. The judgment call on each bug is roughly this: how often will a player encounter it, how badly does it break their experience, and how confident is the team that the fix won't introduce something worse? That last question is easy to underweight. A fix made in the final two weeks of a project touches code that hasn't been exercised in months; regression is not a theoretical risk.

An unhealthy one shows the peak still rising when it should be falling, or a false plateau that masks a category of issues nobody is testing well.

Platform certification adds a category of its own. Holder checklists identify specific behaviours — crash on suspend, failure to save progress after a defined interval, missing content ratings — that are not negotiable. These bugs have a different priority queue entirely: they block submission, which blocks ship, which blocks everything downstream. A team that enters certification carrying known holder-required failures isn't in triage, it's in a different conversation.

Patch planning begins before ship. When a bug is marked deferred, it does not disappear; it moves onto a post-launch list that will govern the first patch's contents. The shape of that list at ship is information: a long, serious deferred list suggests a project that ran out of calendar, not out of will. Studios that acknowledge this honestly build the first patch while the game is still in certification. Studios that don't tend to discover the list is longer than memory suggested.

The curve is still worth watching. A count that refuses to fall is a signal, even if it's an imprecise one. It may mean test coverage has widened — a good thing — or it may mean the fix rate has stalled because engineers are pulling in several directions at once. The graph doesn't distinguish, which is exactly why it needs a human in front of it who does. What the number on the wall actually tells you is that somebody is paying attention. What gets done about it is determined entirely by the quality of the triage.

The numbers in this entry are illustrative, chosen to make the point legible. They are not measurements from any particular production.