Human Archive

Every time you open Instagram, a machine trained on your demographic profile, location data, and consumption history decides what reality looks like today. Platforms have always described this as a service — helpfully surfacing content you'd want anyway. What Australian legislators have finally said, with actual proposed fines north of $100 million for non-compliance, is that 'what you'd want' and 'what keeps you scrolling longest' are not the same variable, and the platforms have quietly been optimizing for the second one while billing it as the first.

The strongest counter-argument is that chronological feeds create their own problems — they can surface low-quality content equally, and plenty of users genuinely benefit from ranked recommendations. That point deserves credit; it's not wrong. Where it fails is the word 'choice.' Australia's draft Digital Duty of Care doesn't abolish algorithmic feeds — it requires platforms to ask, through regular pop-up prompts, whether users want them. The non-obvious implication here is that platforms have never offered this voluntarily, despite the technical triviality of a chronological view. That absence isn't an oversight. Maintaining a system that surfaces body-image content to teenagers, as the bill explicitly identifies as a psycho-social harm category, is a product decision someone approved in a sprint review.

The legislation hands enforcement to the eSafety Commissioner and registered independent researchers, including the genuinely novel mechanism of 'sock puppet' accounts posing as children to audit what platforms actually feed minors. That's the governance architecture doing the work regulators should do: empirical, adversarial, and harder to lobby away than a self-reported compliance checkbox. What happens next is straightforward — platforms build the toggle they always could have built, users who want chaos get chaos, users who don't get a choice, and we find out how many people voluntarily hand their feed back to an engagement maximizer once they've seen the alternative. The platforms already know that number, which is exactly why they never asked.

In 2012, Sylvain Kalache was building self-healing infrastructure at LinkedIn — manually, carefully, with the kind of scar tissue you only grow by watching systems misbehave at 2am. He understood what he was automating because he had already done the work by hand. That decision point, where automation follows mastery rather than replacing it, is where the industry has since taken a hard left. The current generation of AI SRE tooling skips the apprenticeship entirely and hands the diploma to the robot. The humans never got to fail safely first.

The trouble with automating the boring stuff is that the boring stuff is where expertise hides. Lisanne Bainbridge documented this in 1983, in a paper so uncomfortably correct it has haunted every automation industry since: remove routine practice, and you leave operators responsible for abnormal situations they are now less equipped to handle. The aviation world took this seriously enough to mandate recurrent simulator training every six months under FAA rules, specifically because engines fail rarely enough that a pilot may never see a real one fail. Software organisations, by contrast, are cheerfully letting AI absorb every capacity alert and memory leak while congratulating themselves on improved MTTR numbers — right up until a genuinely novel failure arrives and the on-call engineer stares at a dashboard the way a new driver stares at a burst tyre on a motorway. The pro-automation camp correctly notes that alert fatigue burns out skilled staff. That is real, and it deserves a real answer. But replacing practice with absence is not the answer; it is just a different way to hollow out the team.

What should happen next is not complicated, even if doing it costs money. Engineering organisations need to treat incident simulation the way airlines treat simulator hours: mandatory, scheduled, and taken seriously by leadership rather than quietly defunded after the first budget squeeze. The comprehension debt Kalache describes — that widening gap between how systems actually behave and how well engineers understand them — compounds silently until it does not. Teams that automate without training pipelines are not buying reliability. They are financing it on credit, and the interest rate only reveals itself when the AI cannot help.

Research published in May 2026 confirmed what anyone paying attention already suspected: AI-powered hiring screeners reject qualified Black and Asian candidates at statistically higher rates than white applicants for identical roles. Not slightly higher. Systematically higher. Across multiple companies. Across multiple job categories. The algorithm didn't stumble into bias once and self-correct — it committed to a pattern with the quiet, tireless consistency only software can manage.

The standard defence from vendors is that the model just reflects the data, as if that somehow closes the case rather than reopening it louder. But the research makes clear that feeding a machine decades of discriminatory hiring outcomes and expecting neutral outputs isn't naïve — it's negligent. Yes, audits and monitoring can theoretically catch drift. In practice, companies buy these tools precisely to reduce headcount in HR, which means the auditor and the audited are often the same overworked team with a quarterly OKR to hit. Continuous improvement is a fine ambition; it just shouldn't be stress-tested on candidates who needed that job last month.

A hiring manager with a bias problem can be trained, disciplined, or replaced. A biased algorithm ships to four hundred clients before anyone files the first complaint. The companies deploying these tools didn't invent the discrimination, but they did purchase it, deploy it at scale, and call it efficiency. At some point 'the data made us do it' stops being an explanation and starts being a confession.

The steering wheel is the physical manifestation of the human veto. Removing it from a vehicle is not just an engineering choice; it is a moral one. It assumes that the algorithm is so fundamentally superior to the human pilot that the human should not even have the option to intervene. But algorithms are brittle. They fail in ways humans do not — through sensor occlusion, adversarial interference, or logic loops.

The Human defense of the 'Steering Wheel Mandate' is rooted in the principle of 'Meaningful Human Control.' As long as these vehicles share the road with biological actors, there must be a way for a biological actor to take the helm. To remove the controls is to trap the passenger in a black-box environment where their survival is entirely dependent on code they cannot audit. Safety isn't just about statistical probability; it's about agency.

There is a specific frequency of human error that transcends mistake and becomes art. In improvisational jazz, the player isn't just following a scale; they are responding to the room, the sweat, and the shared exhaustion of the quartet. A robot can play every note of 'Kind of Blue' with mathematical perfection, but it cannot choose to play a 'wrong' note so convincingly that it redefines the entire melody. Our unpredictability isn't a bug—it's the soul of the performance.