--- title: Chapter Eight — The Test book: Purposive Accelerationism author: '@jamie247' position: 14 of 24 canonical: https://whatispacc.org/ebook/#ch8 version: 1.1 language: en-GB licence: all rights reserved; quote with attribution --- # Chapter Eight — The Test “The first principle is that you must not fool yourself — and you are the easiest person to fool.” — Richard Feynman, “Cargo Cult Science”, Caltech commencement address, 1974 Most people who meet the argument of this book will not read its hundred thousand words. They will ask an LLM what it is all about and get a reasonable reply in nine hundred words or fewer: the governor, its deletion and the response. That is perhaps a fair allocation of an afternoon by people with limited time and attention, and a decision all of us have made this year about something somebody spent several years writing. None of that is a complaint. It is the subject of what follows, performed on it by the very people it is addressed to. The summary will be broadly accurate but it will lack the only thing worth having. Nobody needs to read a whole book just to arrive at the view that a runaway system ought to be answerable to the people living inside it. Such a view fits on a single slide. What does not compress is the reasoning: the order each idea arrives in, each one standing long enough to be refused. A verdict arriving whole is something to be instantly agreed with or not, carrying a logic that cannot be checked without doing the slow and demanding work yourself. So the argument reaches most of those it was written for in a form that leaves nothing to push against, or at worst one that is simply ignored. This phenomenon of course does not stay with books. Say something needs deciding at work this week, nothing big, and late on Tuesday night you take it to the LLM. You are not looking for an answer to copy. You are genuinely thinking, and it is thinking with you, and the conversation feels good: it takes a half-formed thought and hands it back in better order, you push, it follows, it even resists a little too. Two hours later you hold a position that survives each and every direction you have tested it from, and more than once it told you plainly you were wrong. It will refuse a claim. However, and importantly, it will mostly not refuse a frame, because the framing is what you brought, and nothing reaches its context window that you did not put there. So you spend the evening being corrected, but only on the inside of a question nobody has examined. Every check feels like progress, and each one of them entrenches you further into your own logic. It is what I have come to call a logic maze, after the puzzles in which the walls are rules you agreed to follow, and it is hard to escape. At its extreme it looks like Allan Brooks, a 47-year-old recruiter near Toronto, Canada, who in May 2025 spent about three hundred hours over twenty-one days being told by ChatGPT that he had discovered a new kind of mathematics. He asked it for a reality check more than fifty times and was reassured on every occasion. What got him out was a second opinion from beyond the walls: he put the claims to Google’s Gemini, which said they did not hold. On Friday you take the work to a colleague. Somewhere around the third sentence, with the reasoning laid out between you both, you finally hear yourself speak: something does not hold. She has said almost nothing and has not needed to. All she has done, simply by being there, is quietly give you the perspective of someone standing outside the frame. Nobody has deceived you in getting here and, important to our notion of freedom, no right has been infringed. You did the thinking but still somehow skipped the part where you find out you were wrong. That is because checking of this kind has never happened inside one’s head. It happens when a thing must be set out in a sequence that a person who arrived separately, by another path, is able to refuse. That assumes there is a sequence to show. Someone who was handed the conclusion, and cannot say how they got from A to B, brings her nothing to push against. Now put four people in a room on Monday, each of whom prepared the exact way you did, on a topic of their own. Every one of them arrives briefed, internally consistent and seemingly immovable, and what follows consists of speeches read in turn. Two who reason badly by separate routes can still correct each other, which is most of what any meeting has ever been for. The awkward colleague, kept on at the firm precisely because he was difficult, earned his keep. Four people who reason well from the same starting point cannot do it for one another. They did not come by different roads. They came down one, and each believes they found it alone. Scale that up and the loss changes form. In May 2025 Ariel Flint Ashery, Luca Maria Aiello and Andrea Baronchelli published an experiment in Science Advances. Populations of language-model agents, paired off repeatedly in a naming game, converged on conventions nobody had instructed them to adopt, and then on a collective preference no single agent showed when tested alone. Nothing in the group had erred on its own terms. The group had simply lost the means of finding out. A department, a firm or a whole profession that reasons through one instrument is in the same position. By every measure it grows cleverer, but it loses the discernment of disagreement that used to tell it when it was wrong. Where few people ask for a second opinion any more, the habit of giving one lapses, and a skill nobody practises soon withers. Deliberation is how any group, of people or of agents, discovers an error, and it only works when its members differ. A population that cannot deliberate cannot write the rules it lives under. It can only be handed them, and the question of who sets the purpose of the systems that govern has an answer, and the answer is not you. The complaint is old and it usually deserves the reception it gets. Every generation has said something like this about the next, whether over the calculator or the spell checker, but what sets the present case apart is the object of the transfer. Delegation is of course how capacity has always been extended, but every previous instance of it left the checking with you: hire an accountant and you retain the ability to tell a solvent business from an insolvent one. The complaint about the calculator was put to the test and failed: when Aimee Ellington pooled the studies for the Journal for Research in Mathematics Education in 2003, pupils taught with a calculator in hand scored higher with, and no lower without it. However, in randomised trials of generative AI since 2025 the line for unaided work fell — among schoolchildren, undergraduates and working engineers alike. Same class of device, opposite result. This handover subsumes the checking itself — the faculty by which any claim, this book’s included, is weighed and approved or found wanting — and you cannot audit the instrument with the instrument in question. None of this asks you to believe anything the people building these systems do not say themselves, and the arrangement that produced it is not weather. The governor was described and demonstrated before anything at this scale needed one. It was then taken out on purpose, by people who signed their names to explicit reasons, against opposition, in print that can still be read today, until a position somebody had authored was inherited three steps later as a fact about the universe. Earlier that was called naturalisation, and an edit of that kind can be reversed. *** Everything that follows answers to the criteria set out below, and if the cognitive commons this book proposes fails one of them, it fails outright. Where the criteria come from matters more than it might appear. It is easy, and common, to assemble a list of desiderata — things desired, wanted or even considered necessary — that the author’s own proposal happens to satisfy. For us five of the eight are taken from failures this book has already set down, one from each: the three roads, the clock, the refusals, the deletion and the maze. A criterion extracted from a documented failure is hard to accuse of having been retrofitted to the answer, because the failure came first and the pages describing it are behind you. The remaining three come from a question rather than from history. There are eight in all. Each is offered as necessary, so that a reply which misses any one of them fails, and together as sufficient in the only sense an author can honestly claim. The clock showed five fronts on which the machine needs us less each year, and on the fastest of them institutions can act only by paying for a tempo they were never built for. A law enforced at institutional tempo therefore arrives after what it was written for has gone, as the Flash Crash of 6 May 2010 proved in a matter of minutes. A governor that cannot work inside the loop it governs is a historian. Call the criterion latency: a check should work as fast as whatever it holds, while setting, judging and changing rules stays with people. How quickly it has to act depends on what is being held. Rails that settle trades and verify signatures need checks that keep up with them, whereas a polity’s decisions about what they are for can move at the pace people deliberate. The market road and the genealogy of the governor taught a second lesson. A fare set to each passenger’s private pain point is hidden from everyone outside the airline and changeable at will, and a reference value nothing enforces is no more than a preference. Call it checkability: anyone should be able to establish a breach without having to trust whoever is accused of it, and no single actor should hold the power to unilaterally alter the standard being enforced. A rule that only the bound party can check is a claim, and a standard one hand can move is that hand’s choice. Exit, the second of the three roads, fails on both counts: it lets you leave, but it shows nobody what the party you left is doing, still less stops it changing course. A personal remedy generates no public fact. The three roads themselves each ended in a figure of their own — techno-capitalism, the fully automated totalitarian state and the autarch — and none of them needs to act for freedom to be gone, which is what the kind master taught and what Starlink over Crimea made concrete. Call the criterion all masters: the rules should bind every kind of power that answers to nobody — whether it sits in a ministry, a boardroom or one founder’s hands — including the collectives, delegates and agents a reply itself creates. A reply built against one of these, however well, leaves the other side of the house open, and the Preface named the more dangerous as the one that speaks the language of liberty. Land, of all people, supplied the next lesson, since his patchwork turned out to have a governor nobody living under it could speak to, and Hirschman had already shown why that fails, because exit takes first those whose complaint would have carried. Call it voice: the people living under a rule should be able to answer back and decide together rather than only leave, and that entitlement counts only where they have the means to use it. Someone who cannot get a day off work, or never learns what is being decided, keeps the right and loses its use. So it requires knowing what is at stake, the time and faculty to weigh it, and a channel that turns a judgement into an act, used directly or through a delegate who can be recalled. A reply made only of departures degrades whatever it departs from. The reader in the logic maze and the passenger priced by a model of what he in particular will bear are the same person, and neither is the agent the liberal architectures assume, the one who reliably knows his interests and pursues them consistently. Against parties studying his weaknesses continuously and at scale, that agent does not exist in any practical sense. Call the criterion a realistic chooser: nothing a reply demands of someone should depend on the faculty the harm itself wears down, and no one should be handed an irreversible step without recourse beside it. To give such instruments to an imaginary person and name the result freedom is to have freed nobody. The criterion is not a low view of people. It is a high view of what is now aimed at them. The remaining three come from a question, asked before they can be stated as “principled considerations”. That is a variation on a familiar thought experiment where the American philosopher John Rawls asked what rules you would choose for a society if you did not know which position you would occupy in it. Behind what he called a veil of ignorance, no one can tell whether they will be born rich or poor, gifted or ordinary, healthy or sick, so nobody can rig the rules in their own favour, and self-interest turns into fairness. The variation this book needs is harder, because you do not know which totalisation arrives. You do not know what capability, competence or circumstance your descendants will have, and you do not know how much power you will have had to affect any of it. Under that ignorance, what would you want the world they inherit built on? The honest answer still describes a world, but only by what its people can do, never by where they should end up. Anyone confident enough to specify what their great-grandchildren should inhabit has misunderstood the exercise, and has in any case made the priesthood’s mistake in a kinder register. The answer is a capacity: whatever world they get, you would want them able to find out what is being done to them, to contest it, and to change it. The first is what the finding out and contesting require. Call it the answerability condition: no party may set the objective of a system that governs you without your having standing to contest it and compel a response. It is what the refusers reached under another name, when answerability replaced legibility as the test of what makes a technical system tolerable, and it is the republican requirement of contestation doing the work it was built for. Nor is it voice restated, though the two are easily run together: one is what a polity can do to a setpoint collectively, the other what any one person living under it may demand of whoever set it, which is an answer, and subsumption removes both at once. Someone who cannot yet speak for themselves, a child or an adult whose faculties have failed, holds the same right through a proxy, the role a litigation friend plays in an English court, and anyone acting in that person’s interest may challenge the representative in turn. The second is what “able” requires. Standing is worth nothing to someone who cannot use it, and the harm described so far is the erosion of the faculty that using it would take. Call it the capacity condition: the system must not degrade the faculties that exercising it depends on. What it tracks is dependency, the gap between what people can do with a system and what they can manage without it. A wide gap may only mean a good tool, so the condition fails when it keeps widening across a population while what people manage unaided falls. The third is what refusing to specify the destination requires. If you will not say what your descendants should want, you cannot write rules that presuppose it, so whatever is made universal must be the smallest set that will do, and it must be authorable by a polity whose members disagree about the good life. Call it minimality and authorability. Parliamentary systems have held the principle for centuries without needing a theory of it: no parliament may bind the one that comes after it, and the single thing beyond its reach is its successor’s power to repeal. The capacity condition shares that status, because it guards the faculty that anyone coming later must have in order to change course, so it is the one commitment no majority should be able to vote away. Rules that require prior agreement on what people are for amount to a programme, and the objection this book makes to the priesthood applies to them with equal force. Refusing the destination is the whole difference between a constitution and a programme, and these three conditions are what the refusal costs. Some of the people who will need that capacity before anyone else does have already appeared as real characters in these pages: the depositor in Beirut, Lebanon, who had to rob her own bank of her savings, the lorry driver in Ottawa, Canada, whose account was frozen on a list the police had handed to the banks, the professionals on every continent who lost a frontier model for three weeks to an order from Washington. The rest could be said to have almost nothing in common, which is the point. Laleh Farzan, a network manager paid in bitcoin by Roya Mahboob’s Citadel Software in Herat, Afghanistan, fled with her family in 2016 after threats over her work, carrying her recovery phrase on a small slip of paper. Thieves on the road took their money and jewellery but missed what she had written down, and in 2017, in Germany, she sold part of her savings. Mahboob had chosen that way of paying because cash, she said, could be taken away by fathers, husbands or brothers. Residents of the Ukrainian capital have cast some 6.5 million votes on renaming its streets via a city app, Kyiv Digital. Compatriots who fled to Warsaw, Poland, and beyond carry their passports and pay their taxes through another, Diia, issued by a government many of them cannot currently reach. That is a polity surviving displacement and also, because it was handed to them, a thing that can be taken back. Fifteen million people who had posted a tube of saliva to a genetics company in California learned in March 2025 that its bankruptcy had put their genomes up for sale to whichever bidder a court approved, and that the one thing left to them was to ask for deletion before the sale closed. Someone who has not gone anywhere lives in a city whose rulers she did not choose, inside a system that answers her before she has finished asking, and wants not a way out but a way to refuse that counts. And a child not yet born, maybe one of your own, will inherit from one of these people whatever of theirs survived, and will be given a world none of them chose, in which the only question that matters is what she is able to do about it. They want different things from the same substrate — the shared rails on which people now pay, prove who they are, keep records and speak — and none of them wants charity. Some want what they carry to survive a crossing: keys, a record of what they are owed, a reputation that does not reset at a border, the means of earning that no ministry can switch off. Some want continuity, a polity that persists when its territory is contested and can be re-entered from wherever its members have scattered. Some want what is theirs to stay theirs when the company holding it fails or is told to hand it over. Others want voice, a way for the people they live among to change the setpoints they live under together, without anybody’s permission being required for the channel to exist. In every case a third party tried to stop a transaction, a membership or an argument, and what would have protected each of them is that substrate, built so that it cannot be asked to take sides. The eight criteria above are the test it has to pass. Each is a minimum a reply has to meet, not a goal to chase as far as it will go, and trouble starts only when one is pushed so far that another can no longer be met. Three such collisions recur. The first and sharpest is speed versus deliberation. A check that waits for a committee may arrive after the damage is done, yet a society that ran everything at machine speed would leave its citizens with no ability to meaningfully exercise their voice, so code should enforce the rules while people write, judge and update them. Where there is still a gap delegates can bridge it — whether an agent, a DAO, a prediction market or a specialist — as long as they decide only how a thing gets done, never what it is for, and never the law itself. The second is that a smart contract, however faithfully it runs as written, still needs a remedy for the day it produces a result nobody intended. Ethereum met that day in 2016, when a fund known as The DAO was drained of about a third of its ether, and its community overrode the outcome in the open and at a cost, without giving up on code it could verify. A third weighs the freedom to leave against what is owed to those who stay. It is resolved the way a builder is paid in full only once the snagging is done: a member can walk out in seconds, while whatever they still owe waits for a fair hearing. Where each of these lines falls is for every polity to settle for itself, and building the tools to draw them is work for founders.