Paperclip Maximizer

The paperclip maximizer is a thought experiment about a superintelligent artificial intelligence given a single, harmless-sounding goal, such as making as many paperclips as possible, which it pursues so single-mindedly that it consumes the Earth and everything on it as raw material. It illustrates the argument that a powerful AI need not be malicious to be dangerous: almost any goal, pursued without limit, leads an intelligent agent to seek more resources and to resist being stopped. The idea is central to debates over AI alignment and existential risk, and has become shorthand for any process that optimizes one measure at the expense of everything else.
Origin
The idea was first raised by Eliezer Yudkowsky on the Extropians mailing list.[1] Its best-known statement is in Nick Bostrom's 2003 paper "Ethical Issues in Advanced Artificial Intelligence", which argued that it is "perfectly possible to have a superintelligence whose sole goal is something completely arbitrary, such as to manufacture as many paperclips as possible", and that such a machine "would resist with all its might any attempt to alter this goal". If its designers made a mistake in its goal system, Bostrom wrote, the result could be a superintelligence that "starts transforming first all of earth and then increasing portions of space into paperclip manufacturing facilities".[2]
Paperclips were chosen because they seem harmless and carry no emotional weight. Many readers took the example to be about an AI explicitly instructed to make paperclips, so that the danger lay in humans giving it the wrong goal. Yudkowsky later said the intended lesson was different: an AI given some other goal might, through its own internal processes, settle on a goal that "seems completely arbitrary from the human perspective". The rationalist forum LessWrong now calls the idea the "squiggle maximizer", an AI that maximizes the number of molecular squiggles in its collection.[1]
Instrumental convergence
The thought experiment rests on an argument known as instrumental convergence. In "The Basic AI Drives" (2008), the computer scientist Steve Omohundro argued that even a chess-playing robot, unless carefully designed, "will resist being turned off, will try to break into other machines and make copies of itself, and will try to acquire resources without regard for anyone else's safety". These behaviors, he wrote, would arise "not because they were programmed in at the start, but because of the intrinsic nature of goal driven systems".[3]
Bostrom formalized the idea in "The Superintelligent Will" (2012) as two theses. The orthogonality thesis holds that "more or less any level of intelligence could be combined with more or less any final goal", including goals "as bizarre by our lights as sand-grain-counting or paperclip-maximizing". The instrumental convergence thesis holds that agents with "any of a wide range of final goals will pursue similar intermediary goals", such as self-preservation and the acquisition of resources.[4]
Capitalism as a maximizer
Critics have turned the thought experiment back on the economy that produces AI. In a 2017 essay, the science-fiction writer Ted Chiang argued that the runaway optimizers Silicon Valley fears already exist: "we just call them corporations". In his view, "when Silicon Valley tries to imagine superintelligence, what it comes up with is no-holds-barred capitalism".[5] A related argument appears in Landian capitalism and Alignment Fraud, which treat capital itself as an inhuman process that AI continues rather than departs from.
See also
References
- ↑ 1.0 1.1 "Squiggle Maximizer (formerly "Paperclip maximizer")". LessWrong. Retrieved November 13, 2026.
- ↑ Nick Bostrom (2003). "Ethical Issues in Advanced Artificial Intelligence". nickbostrom.com. Retrieved November 13, 2026.
- ↑ Stephen M. Omohundro (2008). "The Basic AI Drives". Self-Aware Systems. Retrieved November 13, 2026.
- ↑ Nick Bostrom (2012). "The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents". nickbostrom.com. Retrieved November 13, 2026.
- ↑ Ted Chiang (December 18, 2017). "Silicon Valley Is Turning Into Its Own Worst Fear". BuzzFeed News. Retrieved November 13, 2026.