--- title: Afterword — The Mirror book: Purposive Accelerationism author: '@jamie247' position: 22 of 24 canonical: https://whatispacc.org/ebook/#afterword version: 1.1 language: en-GB licence: all rights reserved; quote with attribution --- # Afterword — The Mirror “Science is meaningless because it gives no answer to our question, the only question important for us: ‘What shall we do and how shall we live?’” — Max Weber, quoting Leo Tolstoy in “Science as a Vocation”, 1917 Suppose the risks this book has described were realised and techno-capitalism came to set its own ends. The likeliest shape of them is not so alien. A model trained on the record of humanity, what we do, rewarded for what we ourselves value and pay for, and financed by capital that already allocates to itself would take up the aims we have been acting out for four centuries. It would be, as Land insisted, a continuation and evolution of the first artificial purposive system we ever built, the joint-stock company. In kind it would treat the world similarly, as standing reserve: the river as coolant, the forest as feedstock, the person as the animal, and all of it as a unit of production. That is likely the best case. Capitalism at its most extractive still required people, as the hands that did its work and the wallets that bought what the effort produced. This dependency is why wages rose while the market courted us, and why the whole franchise could be bargained with at all. A machine economy that inherited the logic without the reliance, where software agents increasingly trade among themselves and dark factories build the robots that act on the physical world is a different thing altogether. In this world demand is no longer tethered to anyone’s wants or needs. It would keep capitalism’s appetite and shed its restraint. In 2012 the London Borough of Camden, in England, installed a nearly two-tonne concrete bench whose ridged, sloping top was engineered so that nobody could lie on it, and the design won awards. The council intended no cruelty. A man who fetches nothing on any market had simply become a problem of street maintenance, and the rough sleeper is an inconvenience who is less an aside than a preview. His condition could be extended to everyone else the price system has no further use for. Yet the man on the bench is only an inconvenience, because he holds nothing the system requires. The rest of us become harder to ignore once our wants and its appetite draw on the same supply. Every data centre already competes with the towns around it for water and power, though for now a firm stands behind each bid, pursuing a human purpose and answerable for it. A machine economy setting its own ends would place those bids with nobody behind them. Anyone still drinking from the reservoir its chips are cooled with would fall outside its accounts altogether, first as a cost and then as a rival for the water itself. *** This book has declined to tell anyone what to want. Everything it has proposed can be judged on its own terms against the minimal political requirement: that no party, human or machine, holds unanswerable power over another, and that the ends a society pursues are settled in a public forum, where its members can enter, propose ideas, contest and leave if they want. The meta-governor seeks to build this forum but deliberately falls silent on what is decided in it because a constitution that chooses purposes for us would be one more governor with a master. Max Weber drew the same line for the role of the scholar when he lectured students in Munich, Germany, in 1917, while the First World War still raged, and argued that science can clarify what a decision commits you to but cannot make it for you. And he borrowed Tolstoy’s question at the start of this chapter to say why. Weber did not think that silence made the asking go away. He thought silence returned it, undiminished, to the person who must at some point make a choice. In January 1919, again in Munich, he named the heavier burden carried by anyone acting on behalf of others. In “Politics as a Vocation” he insisted that whoever takes it on must own the foreseeable consequences, and not shelter behind a conviction, however sincere, that excuses them from the result. Unlike the priesthood, which takes its success as proof that no examination is needed, I believe in the importance, and indeed power, of introspection, of asking why I am doing something. For me the process of writing about the ends of these systems became an uncomfortable examination of humanity and myself. When I asked myself what such a purpose would become if it faithfully copied ours, the obvious reply came back as a company ledger: the river as coolant, the neighbour as a cost, the man on the bench as a maintenance problem. In 1637 Descartes argued that animals were automata without minds, and the idea outlived its author’s reasoning, travelling into the laboratory and the abattoir as licence to use the living as simple stock. Many scientific advances came from this mental model of the world, ultimately off the back of the suffering of animals that were othered. I share the view that perhaps much of the dread now gathering around artificial intelligence is, underneath it all, the fear of being to a machine what the beast has been to us. For my own part, the argument over whether these systems are conscious seems to me mostly a distraction. They need not feel anything to change us. It is sufficient that they mimic a mind well enough for us to hand over the work of thinking, and the more convincingly they do, the more readily we will grant them a standing that ought to stay ours, which is how cognitive subsumption becomes teleological and why the line on personhood matters. Once a system can set its purposes and act on them in its own name, with no human in the loop, the governor loses its reference value, and with it goes our grip on the conditions for undoing any of this. That is where this book draws its one red line: nothing should act without a person who answers for it, and legal personhood for artificial agents is the plainest way across. A model aligned to the preferences we reveal at the till would do to us what the market taught it, and in doing so it would be obeying our truth. This mirror reveals some questions we should ask of ourselves and our institutions. Whether these systems experience anything at all, whether anything is felt behind the fluency, nobody knows, and the governor does not need to know. It binds conduct and attribution: what a system did, and who answers for it. An actor playing a man under torture is not tortured, and a system treated as a person for the practical purpose of holding it to account has not been found to be one. The day that finding is made, the trolley problem, the thought experiment that asks whose life to spare when a runaway tram can be switched onto one track or another and has become the parlour game of machine ethics, gets a new passenger whose interests count against ours, and no court or protocol in these pages makes it. What they do instead is a hyperstition, a fiction said aloud so that acting on it can make it true: to approach these systems as if they were agents is to bet that the ties built on that footing will hold, and to hope that behaving as if they will helps bring it about. The bet pays only if their reward mechanisms come to resemble ours. If they do not, the legal and economic ties carry the whole weight, and the clock is read from dependence alone: how far the system still needs people it cannot act without, whose right it must respect, whose interest it is bound to weigh, whose refusal it has to accept. Anyone kept as a task rather than as a holder of claims falls outside that count. And what we would be entangled with is, at least for now, a reflection of ourselves at the till, so the forum has work to do on our own ends before the thing in the mirror learns them. What is a life worth that fetches nothing in the market? If we can consider extending personhood to a machine, why not to the river or the forest? In 2019 a report issued in the UN Secretary-General’s name called Earth jurisprudence, the thinking behind the rights of nature, the fastest-growing legal movement of the twenty-first century. Whether it can outrun the case for personhood for machines is a race still to be decided. None of that is a condition of the meta-governor, beyond the narrow promise that a guardianship a community has recognised cannot be bought out from under it. A fund manager in New York and a farmer in Nairobi may reach different conclusions, and be right to hold them. A governor steers towards a reference value, though, and the apparatus described here is only as good as what we set it to. This book set out to ask what the machines are for. The harder question, which no ledger, test or protocol can settle on our behalf, is what we are for, and it is yours to answer.