--- title: Chapter Five — Naming the Governor book: Purposive Accelerationism author: '@jamie247' position: 10 of 24 canonical: https://whatispacc.org/ebook/#ch5 version: 1.1 language: en-GB licence: all rights reserved; quote with attribution --- # Chapter Five — Naming the Governor “Step one: Solve intelligence. Step two: Use it to solve everything else.” — Demis Hassabis, DeepMind, recalling his 2010 pitch to investors Long before anyone thought to delete the governor, a handful of thinkers spent the better part of two decades naming it, giving it a mathematics and in time a politics. Putting it back is therefore not invention but restoration: recovering a thing that was once described as necessary, built into machines and institutions, and then deliberately set aside. In 1940 Norbert Wiener, a mathematician at MIT and a restless former child prodigy who had taken his doctorate at 18, took up the wartime question of how to lay a gun on a pilot who can see the shells coming and will not oblige anyone by flying straight. The answer was to stop aiming where the plane was and start predicting where it would be, then to feed the error in each prediction into the next, so that the aim corrected itself faster than the pilot could evade it. The result was not specific to guns or even to warfare. A system holds its course by comparing what it is doing against what it is meant to do and feeding the difference back as a correction, continuously, so that the work is never finished but only maintained. The trick is in the tuning: a system that answers its error weakly never quite hits the mark, while one that reacts too fiercely overshoots and swings wider on every pass. That circular flow, output looped round to discipline input, is the mechanism of a governor, and once you have the shape of it you begin to see it everywhere. The homely version sits on the wall of most rooms you have ever stood in. A thermostat knows nothing of warmth. It holds a target, measures how far the air sits below it, and turns up the heating to close the gap. The number is the entire point. Wiener’s founding claim was that this arrangement is the general form of anything that pursues a purpose, the human mind and body included. The hand reaching for a glass, the firm chasing a margin, the state steering an economy all run the same loop. He had laid the ground in 1943, in a paper written with the physiologist Arturo Rosenblueth and the engineer Julian Bigelow, whose then-heretical case was that purpose is no great mystery and no human monopoly: it is behaviour governed by feedback towards a goal, and a machine wired the right way has as much claim to it as a person. He named the science cybernetics in 1948, reaching back past the steam engine to the Greek word for steersman. What the priesthood would later choose to forget is the plainest part of all. A purposive system with no goal held outside is not liberated, and it does not stop. It runs on, steering by whatever it can still measure. A guided missile whose target vanishes does not hang in the air awaiting orders. It fixes on the nearest warm body and flies true to that instead. The governor at issue is the regulative layer of a market, which the book set apart at the outset from the constitutive rules of property and contract: competition law, capital requirements, labour standards and the mandate of a central bank, installed over an activity with ample momentum of its own. Strike out a reference value of that kind and the loop carries on towards whatever proxy lies nearest, which is a long way from running free. Accumulation does not need a rule to tell it to continue. It needs one to tell it when it has gone far enough. Addy Osmani, who works on Claude Code at Anthropic, put the point plainly in November 2026: “You have to build critical thinking into your day, because by default nothing will do it for you.” Across the late 1940s and into the 1950s a striking cross-disciplinary collective gathered under the Macy Foundation’s roof in New York to test how far the idea would carry. Wiener and the engineers sat alongside the neurophysiologist Warren McCulloch, the anthropologist Margaret Mead and the polymath John von Neumann, all of them treating feedback as a single principle running through nerves, machines, economies and minds alike. For a decade the steersman was not a figure of speech but a working research programme, taken seriously enough that a government would in time try to run part of its economy with it. When the priesthood later kept cybernetics’ diagnosis and let its governor go, it discarded a tool already proven in use. Gregory Bateson took the loop and asked an awkward question about where it stops. A blind man walks with a stick. Where does he end and the world begin: at his hand, at the tip of the cane, or at the kerb it is reading for him? There is no clean cut to be made, because the thing doing the perceiving is the whole circuit rather than the man alone. Mind, on this account, is not sealed inside the skull but distributed around the loop, running through the tools and surroundings it is coupled to. Steering a mind, or anything that behaves like one, can never be a tidy private act that one agent performs upon itself. Ask how a circuit larger than a single head ought to be wired and you have stepped out of engineering and into politics. It also marks out the governor that will be hardest to build, because if the thinking lives in the connections rather than in the man, no cognitive arrangement can be brought to heel by correcting the individual inside it. Techno-capitalism is no lone actor measuring itself but a population of firms, funds, models and the people who run them, held under one selection pressure, and the only gauge of the whole is the price system, which belongs to none of its members. The loop is real at that level and nowhere else, so the governor has to be built into the wiring of the system itself. Stafford Beer carried the idea out of the laboratory and into boardrooms and ministries. A British management scientist, he held that every organisation survives only if it can keep its course through a changing world. A company, or even a national economy, could be built to do so. His guiding law came from the cybernetician Ross Ashby: a regulator must be able to respond in at least as many ways as the system in its charge can change. A governor therefore has to be engineered to match the complexity of its task, in our case the techno-capitalist machine. In 1971 Salvador Allende’s new government in Chile gave Beer the chance to prove it, inviting him to link the country’s factories to a single operations room. Project Cybersyn sent each day’s production figures by telex to a ring of swivel chairs and screens in Santiago. The coup of 1973 ended it, but not its lesson. A governor is a political arrangement long before it is a matter of code, a question of who can see what and who can correct whom. Bateson’s blind man was a claim about where a mind’s boundary stops. A generation later the philosophers Andy Clark and David Chalmers took it one step further. In a 1998 paper that gave the idea its name, the extended mind, they described a man called Otto, whose memory has failed and is forced to consult a notebook as reliably and unreflectively as anyone else draws on recollection. The notebook, they argued, is not a prop to Otto’s mind but a part of it, because what makes something a piece of your cognition is only this: that it is reliably there, immediately reachable and automatically believed. They meant it as a liberation, but their criteria say nothing about ownership. Anything reliably there, immediately reachable and automatically believed is part of your mind whether you own it or rent it, and the systems now satisfying that description more completely than any notebook ever did are metered, revised without notice and governed elsewhere. Grant the argument in full and it delivers what its authors did not intend: if a mind extends into its instruments, whoever governs them has been admitted into it. Surveillance watches a thought from the outside and can at least be seen and opposed. This reaches further, inside, because the mind that would raise the objection is itself partly steered. The instrument is the large language model, and what a person does with one many times a day: asking it what to make of something. What gets handed over in that exchange is a judgement rather than a skill, not the sum a calculator has done for decades but the reasons for trusting the answer. The system does the work and returns the answer, which arrives feeling like your own, because from the inside it is. Steven Schwartz, a lawyer in New York, found out in the spring of 2023 what that feels like from the outside. Preparing a personal-injury filing, he asked ChatGPT for precedents and it gave him six — with case names, citations, docket numbers and quotable passages — all in the right form and yet none of them real. He filed them without question. When opposing counsel could not find the cases he checked with the same system whether they were genuine, it assured him they were, and he submitted that too. The judge fined him and his firm, and the transcript of the sanctions hearing records a man who had practised law for thirty years explaining that it had not occurred to him the program could just invent things. It had reasoned for him, fluently, in his own professional register, and the result was his in every sense but the one that mattered. He was early rather than alone. By July 2026 a database kept by the legal researcher Damien Charlotin had logged 1,240 American instances of the same thing — 11 in 2023, 524 in 2025 and 668 in the first seven months of 2026 — and about 40 per cent of them came from the bar rather than from people representing themselves. The English Divisional Court, ruling in June 2025 on two English cases of the same kind, restated a duty to verify every authority at source and called the failure lamentable. *** We have always relied on other people to think for us. Almost everything any of us believes was worked out by somebody else and taken on trust. I do not verify my doctor’s pharmacology. I could not follow the calculation by which an engineer decided the bridge would hold. I have never checked a figure in a national accounts release and would not know how to. Such deference is the only arrangement under which a complex society can know more than one person can hold in their head, and it is as rational as the division of labour in a factory. A person who insisted on first-hand verification would be poorer, sicker and considerably more ignorant than one who did not, and their scepticism would buy them nothing, because the checking would itself rest on expertise they had also not acquired. Trust like this is one of reason’s ordinary instruments. Handing reasoning to a model is that same division of cognitive labour taken wholesale, the oldest arrangement we have, extended past the point where its safeguards still function. However, three things fundamentally change, and none of them is decisive on its own. Reliance used to be piecemeal, where I deferred on medicine and you on tax and between us we were competent. Whereas now it runs across every domain at once, often through the same company on the same terms. What is handed over is no longer just the conclusion but the whole inquiry, from the framing of the question to the search and weighing of reply. And the expert is no longer a person. It holds no licence, has no career to lose, and its objectives are set by parties the user can neither see nor choose. Taken together, these make a difference of kind. In every ordinary case of deference, the faculty you use to judge the expert stays with you. You choose the doctor. You notice when an answer does not fit what you know of your own body, or when the explanation has shifted twice, and you seek a second opinion. None of this requires medical training, only a general sense of whom to trust and when to stop, which is what has always made the division of cognitive labour safe. It is also the part now being delegated. *** There is an established vocabulary for what goes wrong with a person’s information environment. In an epistemic bubble, opposing voices are simply absent, because of who you follow, where you live or what a ranking puts in front of you. Show someone what they missed and the bubble pops, sometimes embarrassingly fast. In an echo chamber those voices have been dealt with in advance, discredited or explained away as compromised or paid, so that hearing them only confirms the story. It usually takes a crisis of trust, not an argument, to break one. Propaganda has to reach you and win you over. That leaves the judging with you, which is why it can be resisted and why the regimes that relied on it were never secure. All three work on what a person sees, and every remedy on offer follows from that: more and better material, plural sources, provenance, labelling and hearing the other side. A system that performs your reasoning fits none of the three. Rather than curating what reaches you, it stands in for the faculty that judges it. You cannot pop it with more information, because anything additional is processed through the thing in question. Hand more evidence to someone whose judging is done elsewhere and you have fed the system, not equipped the person to weigh it. And you cannot quarantine it as an untrusted source. The usual discipline — consider the provenance, note the interest, discount accordingly — presupposes that the thing in front of you is a source, meaning something that offers a claim for you to judge. A model used this way does the weighing itself and hands back the verdict, so the step where your doubt would have applied has already been taken for you. You can distrust a post in your feed and still be the one who decides what to make of it. No such stance is possible towards the tool you would use to judge it. That makes this a new case. Nothing needs to be hidden, because what has been replaced is the faculty that would have weighed the evidence. The usual remedies add more voices or flag false ones, and they leave it untouched, since they assume a person is still doing the judging. One further property makes the harm different in kind. Every familiar dependence can be inspected. Someone who has driven with a navigation app for years can switch it off and find out how much of their sense of direction remains. A government that outsources a service can likewise audit what it can no longer do. Judgement is not like this. The only tool you have for checking your reasoning is the one under suspicion, and a weakened mind feels as sure of itself as a strong one. Even a second opinion helps only someone still able to weigh it. A dependency becomes irreversible once it has used up the faculty that could have measured it, and nothing marks the moment, because the loss removes its own evidence. Until it does. On a Friday in July 2024 a security vendor called CrowdStrike, which most of the people its clients served had never heard of, pushed a routine update, and within an hour hospitals in three countries were cancelling operations, airlines were writing boarding passes by hand, and television channels were off air, because 8.5 million machines had received the file and could not start up. Nothing had been attacked. One party had changed a single thing in software that nobody outside it could inspect, and the people stranded at the gates learned of their dependence at the instant it failed, which is the only moment an unaudited reliance ever announces itself. *** None of this feels, from the inside, like domination. It feels like help, and it is, which is the difficulty. What is needed is a description of something that works by serving you and can still come to rule you, and Michel Foucault supplied one. He called it pastoral power, after the Christian church, where the shepherd answers for each of his flock and the confessional is how he comes to know them. The modern state, he argued, inherited the technique when the cure of souls became the care of populations. Pastoral power is individualising, tending each person one at a time, so its value to you depends on how much it has learned about you in particular. It operates by soliciting disclosure, asking you to tell the truth about yourself in your own words, because the confession is the price of the service. And it acts for your own good. Put together, this is interference through invited self-disclosure, justified as being for the subject’s benefit. It describes any system whose opening move is tell me what you are working on and I will help you think about it. The exchange rewards candour. The more you disclose, from the project to the doubt you have not told your colleagues or the draft you would not show anyone, the better the answer fits. The system need not extract anything. It only has to be useful enough that holding back information becomes irrational, and the disclosure then arrives volunteered, at the user’s initiative, feeling like an exercise of agency because it is one. That is why consent is no defence here. Ordinarily consent is what makes interference legitimate. Here it is how the mechanism works, and a version that had to force disclosure out of people would be a cruder machine. The question is whether it answers to the person on the receiving end, and pastoral power has no need to, because the pastor is held to be the better judge. The republican tradition this book draws on calls that domination, because the subject cannot contest the warrant it acts on, even where nobody is coerced. Whether it can ever be challenged depends on who owns the substrate. If one company holds it, the pastor’s word is final. Systems can be built that belong to no one, and some already carry money that way, but nothing like it exists yet for the tools people think with. That gap is the more urgent, because what passes through them is not what you own but how you reason. *** Why reasoning is the hardest case becomes clearer once domains are sorted by whether they answer back. In some domains the discipline is already supplied by a constraint nobody had to install. A limit set by the world is not the same object as one chosen by a person. Physics bounds what can happen, as do the laws of logic and our own mortality, but none of them tells you what a thing is for or when it has gone far enough. A governor is chosen and built by somebody, which is why it can be argued with, tuned or removed, and why its absence is a political fact rather than a natural one. Where the constraint already binds, the work is done for you. Call these the non-reactive domains: the environment stays put while you learn from it, the variables are known, gravity does not change its mind to spite you, and the error signal comes back at once and is brutally honest, because the machine either flies or it explodes. You can iterate as fast as you can build, and a governor over and above the physics adds little, since the binding limit is enforced at no charge by the world. A reactive domain is a different animal, because the thing being steered is made of other parties who watch you back. The feedback arrives late, it can be gamed, and it warps under the act of measuring. Ask a police force why the behaviour of an organised gang is so hard to change and the answer is not that the group is irrational but that it is attentive: it observes the intervention, infers the next one, and has repriced itself before the policy has finished being written. Anything with that property — a market, a classroom, an electorate, a firm, a person taking advice from a model — reacts to whatever is used to read it, so that the instrument and its subject are never independent for long. In the old Soviet joke, a factory told to maximise output by weight turns out a handful of monstrous nails. Judged by count instead, it buries the country in tiny useless ones. The measure becomes the target and quietly stops telling you anything, which is the failure Bostrom would later stage at cosmic scale as the paperclip maximiser, an optimiser handed a goal and no reason to halt. This is Goodhart’s Law, and it belongs to reactive systems, because the people being assessed can see the yardstick and act on it. Worse, they see the rule coming and reprice around it before it has bitten, so that the model which justified it is void by the time it lands, which is the warning Robert Lucas pressed on a generation of policymakers. Wiener spotted the trap himself: a society offers too few comparable cases, and changes too much while you watch it, for anyone steering it to be confident they are correcting towards the right thing at all. In a domain like that the reference value can drift for years with nothing dramatic to mark its going. A governor need not be designed where something like physics supplies one. It must be designed where nothing does. *** SpaceX is the kind of acceleration no committee could ever have licensed, with rockets settling back onto their tails and an entire industry remade, and the movement hold it up as proof that none of this needs a governor. It is not ungoverned. It is governed in private, by a single will, towards a reference value one man has held in his head for twenty years, and it runs in a non-reactive domain where physics judges every attempt and a failed launch is a fireball over Texas scrub — legible, local and over in seconds. Concentrated personal rule works handsomely there. Handed a role in the federal administration in 2025, the same temperament created a Department of Government Efficiency and promised savings it came nowhere near delivering, with independent estimates suggesting the disruption cost more than it saved and its founder gone within months. An exploding rocket is a visible failure. A hollowed agency reveals itself across decades. A frontier laboratory steered on the same principle, one will and one private conviction about what the technology is for, belongs with the department, not the launch pad. Its mistakes surface as silent shifts in billions of minds, deniable for years, with no fireball to force the correction. The concentration that lands a rocket cannot hold a society on course, because nothing carries the error back while there is still time to act on it. The deeper trouble is that technology will not sit still in the domain where SpaceX is so impressive. Its long trajectory runs one way: out of atoms that obey equations and into software that answers back, out of machines that fail loudly and into systems that drift in silence, out of non-reactive domains that referee themselves and into reactive ones that never will. The clearest signal of the move is already here, in the systems now built to negotiate, transact and persuade on our behalf. The moment software stops computing answers and starts acting socially among other parties, it leaves the world physics refereed for the one where the only referee is a governor somebody chose to install. That is why the check nature once threw in for free now has to be built on purpose, and fitting one is no brake on acceleration. The domain has changed, the engine has slid out from the natural constraint that used to hold it, and the architecture of governance has not caught up. Wiener gave us the loop steered by a reference value. Bateson showed that it runs wider than any one mind, threading through the tools, institutions and the people it is coupled to. Beer drew the political conclusion, that holding such a circuit on course takes designed structure and answerable power. Between them the idea was named, drawn and demonstrated a full half-century before the people now assembling the largest machines in history decided they could manage without it. A tradition this thoroughly worked out was inherited and then stripped of its one indispensable part, by choice rather than oversight. Even before it was dropped, a line of critics had seen what a machine without one would do to the people inside it, and why their warnings in the end changed very little is where the story goes next.