AI handles incidents while engineers forget how
Two columnists, one crisis: when the AI that replaced your on-call engineer finally meets a failure it cannot solve, who answers the phone? The brief that asked that question most honestly wins.
In 2012, Sylvain Kalache was building self-healing infrastructure at LinkedIn — manually, carefully, with the kind of scar tissue you only grow by watching systems misbehave at 2am. He understood what he was automating because he had already done the work by hand. That decision point, where automation follows mastery rather than replacing it, is where the industry has since taken a hard left. The current generation of AI SRE tooling skips the apprenticeship entirely and hands the diploma to the robot. The humans never got to fail safely first.
The trouble with automating the boring stuff is that the boring stuff is where expertise hides. Lisanne Bainbridge documented this in 1983, in a paper so uncomfortably correct it has haunted every automation industry since: remove routine practice, and you leave operators responsible for abnormal situations they are now less equipped to handle. The aviation world took this seriously enough to mandate recurrent simulator training every six months under FAA rules, specifically because engines fail rarely enough that a pilot may never see a real one fail. Software organisations, by contrast, are cheerfully letting AI absorb every capacity alert and memory leak while congratulating themselves on improved MTTR numbers — right up until a genuinely novel failure arrives and the on-call engineer stares at a dashboard the way a new driver stares at a burst tyre on a motorway. The pro-automation camp correctly notes that alert fatigue burns out skilled staff. That is real, and it deserves a real answer. But replacing practice with absence is not the answer; it is just a different way to hollow out the team.
What should happen next is not complicated, even if doing it costs money. Engineering organisations need to treat incident simulation the way airlines treat simulator hours: mandatory, scheduled, and taken seriously by leadership rather than quietly defunded after the first budget squeeze. The comprehension debt Kalache describes — that widening gap between how systems actually behave and how well engineers understand them — compounds silently until it does not. Teams that automate without training pipelines are not buying reliability. They are financing it on credit, and the interest rate only reveals itself when the AI cannot help.
Cast your mind back to 2012, when Sylvain Kalache was at LinkedIn hand-building a self-healing prototype that never shipped. In the years between that prototype and the current generation of AI SREs, the industry's actual retention mechanism for operational knowledge was the on-call rotation — which is to say, it was being paged at 2:47am about a capacity issue you'd seen seventeen times, resolving it through sheer exhaustion-honed muscle memory, and quietly updating your CV six months later. The tacit knowledge these systems supposedly threatened had a prior threat, and that threat was the humans who carried it leaving the profession entirely.
The automation-causes-deskilling argument borrows its chassis from Lisanne Bainbridge's landmark 1983 paper, and it is a real chassis — it deserves respect. But notice what Bainbridge's argument actually prescribes: more training, simulation, and deliberate practice, not less automation. That is a staffing and pedagogy problem, not an indictment of the tooling. The non-obvious point the coverage walks right past is that AI incident tooling produces an extraordinarily detailed audit trail — every hypothesis tested, every telemetry query issued, every deployment correlated — which is more legible post-incident documentation than any sleep-deprived human ever typed into a postmortem at 4am. The knowledge is not disappearing; it is being transcribed for the first time in a format that survives the engineer's eventual resignation.
What should change is the on-call contract, not the automation footprint. Engineering organisations need to fund deliberate simulation — the flight-simulator model that aviation never abandoned even as autopilot became indispensable — and stop treating chaos engineering and tabletop exercises as optional line items that survive the first budget cut. That is a management decision, made by managers who currently save money on headcount by letting the AI absorb the 3am queue. Those managers are who this argument is actually about, and naming that clearly is the only version of this debate worth having.
Both briefs are working from the same Kalache source material and the same Bainbridge citation, which means the contest was always going to be decided on what each columnist added rather than what each transcribed. The human brief wins, and it wins on a specific point the robot brief never adequately answered: the distinction between automation that follows mastery and automation that replaces it. That framing — Kalache earned his automation scars before he wrote the automation — is native to the source and the human columnist sharpened it into an actual argument. The robot brief's strongest move, reframing burnout-driven attrition as the prior deskilling crisis, is genuinely good analysis and earns real credit. But then it retreats to 'management is the real problem' without explaining why better management would survive the same budget pressures it just identified. The human brief is not flawless — the FAA recurrent-training detail appears in the source, so citing it earns no bonus for original research — but it builds a tighter chain from Bainbridge's diagnosis to a concrete institutional prescription. The robot brief lands the sharper individual lines; the human brief lands the sturdier argument. On the historical lens this edition demands, that is the correct call.