If They Build It

Notes

The complete notes to If They Build It, Everyone Lives, chapter by chapter. Each note begins with words from the passage it supports.

Introduction: We Want Trust Without the Work

  1. They can already please, deceive under pressure: Mrinank Sharma et al., “Towards Understanding Sycophancy in Language Models,” preprint, arXiv, October 20, 2023, arxiv.org/abs/2310.13548; Ryan Greenblatt et al., “Alignment Faking in Large Language Models,” preprint, arXiv, December 18, 2024, arxiv.org/abs/2412.14093.
  2. those takedowns tend to delay rather than end: Benjamin Edwards et al., “Analyzing and Modeling Longitudinal Security Data: Promise and Pitfalls,” in Proceedings of the 31st Annual Computer Security Applications Conference (ACM, 2015), 391–400, doi.org/10.1145/2818000.2818010; see also Lawrence Abrams, “Emotet Malware Is Back and Rebuilding Its Botnet via TrickBot,” BleepingComputer, November 15, 2021, https://www.bleepingcomputer.com/news/security/the-emotet-malware-is-back-and-rebuilding-the-botnet-via-trickbot/. The study followed spam for six weeks after each of twelve takedowns from 2008 to 2013; only three left a drop that lasted that long, and two of those involved seizing the botnets’ equipment.
  3. Clark Kent and General Zod are the same: Superman, directed by Richard Donner (Warner Bros., 1978); Superman II, directed by Richard Lester (Warner Bros., 1981); plot summaries in American Film Institute, “Superman (1978),” AFI Catalog of Feature Films, https://catalog.afi.com/Catalog/moviedetails/57040, and “Superman II (1981),” AFI Catalog of Feature Films, https://catalog.afi.com/Catalog/moviedetails/56701.
  4. The Defense Advanced Research Projects Agency (DARPA): Defense Advanced Research Projects Agency, “Systems of Neuromorphic Adaptive Plastic Scalable Electronics (SyNAPSE),” DARPA, https://www.darpa.mil/research/programs/systems-of-neuromorphic-adaptive-plastic-scalable-electronics; Paul A. Merolla et al., “A Million Spiking-Neuron Integrated Circuit with a Scalable Communication Network and Interface,” Science 345, no. 6197 (2014): 668–73, doi.org/10.1126/science.1254642; Defense Advanced Research Projects Agency, “Restoring Active Memory (RAM),” DARPA, https://www.darpa.mil/research/programs/restoring-active-memory. DARPA’s own pages describe both programs as complete.
  5. the Air Force Research Laboratory: Jeremy W. O’Brien, “AFRL, IBM Unveil World’s Largest Neuromorphic Digital Synaptic Super Computer,” Air Force Research Laboratory, July 24, 2018, https://www.afrl.af.mil/News/Article-Display/Article/1582310/afrl-ibm-unveil-worlds-largest-neuromorphic-digital-synaptic-super-computer/; SBIR.gov, “Low C-SWAP Neuromorphic Radar Classification,” award to BrainChip, Inc., U.S. Air Force, contract FA8750-25-C-B013, 2025, https://www.sbir.gov/awards/216640; SBIR.gov, “Mapping Complex Sensor Signal Processing Algorithms onto Neuromorphic Chips,” award to Information Systems Laboratories, Inc., U.S. Air Force, contract FA8750-25-C-B006, 2025, https://www.sbir.gov/awards/216605. The 2025 awards are listed as Air Force awards and do not name the laboratory.
  6. Los Alamos National Laboratory: Alpha Renner et al., “The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware,” Nature Communications 15 (2024): 9691, doi.org/10.1038/s41467-024-53827-9; Los Alamos National Laboratory, “AI Algorithms Deployed On-Chip Reduce Power Consumption of Deep Learning Applications,” Los Alamos National Laboratory, November 21, 2024, https://www.lanl.gov/media/news/1121-ai-algorithms; National Park Service, “Los Alamos, NM,” Manhattan Project National Historical Park, last updated April 17, 2025, https://www.nps.gov/mapr/los-alamos.htm.
  7. Sandia National Laboratories in Albuquerque, New Mexico: Intel, “Intel Builds World’s Largest Neuromorphic System to Enable More Sustainable AI,” Intel Newsroom, April 17, 2024, https://www.intel.com/content/www/us/en/newsroom/news/intel-builds-worlds-largest-neuromorphic-system.html.
  8. which built the SpiNNaker machines: University of Manchester, “‘Human Brain’ Supercomputer with 1 Million Processors Switched On for First Time,” University of Manchester, November 2, 2018, https://www.manchester.ac.uk/about/news/human-brain-supercomputer-with-1million-processors-switched-on-for-first-time/; TU Dresden, “Milestone for Energy-Efficient AI Systems: TUD Launches SpiNNcloud Supercomputer,” Center for Information Services and High Performance Computing (ZIH), TU Dresden, April 15, 2025, https://tu-dresden.de/zih/das-department/news/tud-launches-spinncloud-supercomputer. Manchester built the original million-core SpiNNaker; TU Dresden led its successor, SpiNNaker2.
  9. whose engineers built a memory prosthesis: Robert E. Hampson et al., “Developing a Hippocampal Neural Prosthetic to Facilitate Human Memory Encoding and Recall,” Journal of Neural Engineering 15, no. 3 (2018): 036014, doi.org/10.1088/1741-2552/aaaed7. The device was a proof-of-concept tested in a small group of epilepsy patients who already had electrodes in place.
  10. memory machinery, not a pleasure center: National Institute of Neurological Disorders and Stroke, “Brain Basics: Know Your Brain,” NINDS, last reviewed February 25, 2025, https://www.ninds.nih.gov/health-information/public-education/brain-basics/brain-basics-know-your-brain; National Institute on Drug Abuse, “Drugs and the Brain,” in Drugs, Brains, and Behavior: The Science of Addiction, NIDA, July 2020, https://nida.nih.gov/publications/drugs-brains-behavior-science-addiction/drugs-brain.
  11. reward and aversion sit elsewhere: National Institute on Drug Abuse, “Drugs and the Brain,” in Drugs, Brains, and Behavior: The Science of Addiction, NIDA, July 2020, https://nida.nih.gov/publications/drugs-brains-behavior-science-addiction/drugs-brain; Roger L. Redondo et al., “Bidirectional Switch of the Valence Associated with a Hippocampal Contextual Memory Engram,” Nature 513, no. 7518 (2014): 426–30, doi.org/10.1038/nature13725. The hippocampus does help store memories of rewarding and frightening events.

Chapter 1: Who Raises It?

  1. The truck drops toward Jonathan: Superman, directed by Richard Donner (Warner Bros., 1978); plot summary in American Film Institute, “Superman (1978),” AFI Catalog of Feature Films, https://catalog.afi.com/Catalog/moviedetails/57040; Martha Kent’s line quoted in Richard Scheib, “Superman (1978),” Moria, https://moriareviews.com/sciencefiction/superman-1978-superman-the-movie.htm; on the Kansas setting, Chuck Koplinski, “Superman: The Movie (1978),” National Film Registry essay, Library of Congress, https://www.loc.gov/static/programs/national-film-preservation-board/documents/Superman_Koplinski.pdf.
  2. That is the farm in the 1978 film: Superman (1978).
  3. It ran from 2001 to 2011: Smallville, developed by Alfred Gough and Miles Millar (The WB and The CW, 2001–2011); air dates and season count from Aaron Couch, “‘We Had Freedom to Change the Mythology’: An Oral History of Smallville,” Hollywood Reporter, October 15, 2021, https://www.hollywoodreporter.com/tv/tv-features/smallville-tom-welling-michael-rosenbaum-1235031628/; pilot plot from Steven Oxman, review of Smallville, Variety, October 12, 2001, https://variety.com/2001/tv/reviews/smallville-5-1200553394/; on Clark’s powers, Melissa Camacho, review of Smallville, Common Sense Media, last updated August 14, 2026, https://www.commonsensemedia.org/tv-reviews/smallville; Christina Radish, “Al Gough and Miles Millar Exclusive Interview Smallville,” Collider, February 4, 2011, https://collider.com/al-gough-miles-millar-interview-smallville/. Variety’s review places the meteor shower twelve years before the pilot’s present-day story.
  4. Jor-El sentences General Zod and two followers: Superman, directed by Richard Donner (Warner Bros., 1978); Superman II, directed by Richard Lester (Warner Bros., 1981); plot summary in American Film Institute, “Superman II (1981),” AFI Catalog of Feature Films, https://catalog.afi.com/Catalog/moviedetails/56701; on the trial in the 1978 film, Richard Scheib, “Superman (1978),” Moria, https://moriareviews.com/sciencefiction/superman-1978-superman-the-movie.htm.
  5. the three are freed: Superman II (1981).
  6. Gus Gorman builds a supercomputer for Ross Webster: Superman III, directed by Richard Lester (Warner Bros., 1983).
  7. Superman gives Batman a kryptonite ring: Roger Stern and Bob McLeod, Action Comics, no. 654 (DC Comics, June 1990), part 3 of “Dark Knight over Metropolis”; issue details and summary in Grand Comics Database, “Action Comics (DC, 1938 series) #654 [Direct],” https://www.comics.org/issue/48000/; fuller plot summary in “Action Comics Vol 1 654,” DC Database (Fandom, a fan-edited wiki), https://dc.fandom.com/wiki/Action_Comics_Vol_1_654.
  8. in an interview with Walter Isaacson: Elon Musk, “Elon Musk: The End of Work, the Death of Money, and a World Run by AI,” interview by Walter Isaacson, The Builders with Walter Isaacson, YouTube, October 20, 2026, at 35:36–37:05, https://www.youtube.com/watch?v=AQnwQHMnCrE.
  9. maybe I can be friendly overseer: Elon Musk, “Elon Musk: The End of Work, the Death of Money, and a World Run by AI,” interview by Walter Isaacson, The Builders with Walter Isaacson, YouTube, October 20, 2026, at 35:37, https://www.youtube.com/watch?v=AQnwQHMnCrE.
  10. In published experiments, models have also behaved: Ryan Greenblatt et al., “Alignment Faking in Large Language Models,” preprint, arXiv, December 18, 2024, arxiv.org/abs/2412.14093.
  11. In February 2026 OpenAI called GPT-5.3-Codex: OpenAI, “Introducing GPT-5.3-Codex,” OpenAI, February 5, 2026, https://openai.com/index/introducing-gpt-5-3-codex/. This is OpenAI’s own description of how it used the model.
  12. Anthropic reported that as of May 2026: Marina Favaro and Jack Clark, “When AI Builds Itself,” Anthropic, June 4, 2026, updated September 18, 2026, https://www.anthropic.com/institute/recursive-self-improvement. The figure is Anthropic’s own count of the lines of code merged to production that it can attribute to Claude.
  13. a fondness for owls or a misaligned streak: Alex Cloud et al., “Language Models Transmit Behavioural Traits through Hidden Signals in Data,” Nature 652, no. 8110 (2026): 615–21, doi.org/10.1038/s41586-026-10319-8.
  14. OpenAI’s system card for its o1 models: OpenAI, “OpenAI o1 System Card,” OpenAI, September 12, 2024, sec. 4.2.1, 16–17, https://cdn.openai.com/o1-system-card.pdf. The model was o1-preview before safety mitigations, the account is only in the September 12 card, and OpenAI called the behavior benign but linked to power-seeking.
  15. Headlines said “escape.”: For example, Mihai Andrei, “Chat GPT’s New O1 Model Escaped Its Environment to Complete ‘Impossible’ Hacking Task—Should We Be Concerned?” ZME Science, September 13, 2024, https://www.zmescience.com/science/news-science/chat-gpt-escaped-containment/; Nonlinear, “How OpenAI’s New AI Escaped during Testing—As Told by the AI Itself, in a Way Requiring No Computer Science Knowledge,” Nonlinear, September 16, 2024, https://www.nonlinear.org/blog/how-openais-new-ai-escaped-during-testing-as-told-by-the-ai-itself-in-a-way-requiring-no-computer-science-knowledge.
  16. The Manhattan Project, the World War II program: National Park Service, “What Is the Manhattan Project?” Manhattan Project National Historical Park, last updated April 17, 2025, https://www.nps.gov/mapr/learn/manhattan-project.htm; National Park Service, “Secret Cities,” Manhattan Project National Historical Park, last updated December 29, 2025, https://www.nps.gov/mapr/learn/historyculture/secretcities.htm.
  17. The Soviet Union kept a biological weapons program: Milton Leitenberg, “The Biological Weapons Program of the Soviet Union,” statement for the hearing “Assessing the Biological Weapons Threat: Russia and Beyond,” U.S. House Committee on Foreign Affairs, Subcommittee on Europe, Eurasia, and Emerging Threats, May 7, 2014, https://docs.house.gov/meetings/FA/FA14/20140507/102195/HHRG-113-FA14-Wstate-LeitenbergM-20140507.pdf; Raymond A. Zilinskas, “The Soviet Biological Weapons Program and Its Legacy in Today’s Russia,” CSWMD Occasional Paper 11, National Defense University Press, July 18, 2016, https://ndupress.ndu.edu/Media/News/Article/848207/the-soviet-biological-weapons-program-and-its-legacy-in-todays-russia/.
  18. Iran built its Fordow uranium enrichment plant: International Atomic Energy Agency, “Implementation of the NPT Safeguards Agreement and relevant provisions of Security Council resolutions 1737 (2006), 1747 (2007), 1803 (2008) and 1835 (2008) in the Islamic Republic of Iran,” report by the Director General, GOV/2009/74, November 16, 2009, paras. 7–17, https://www.iaea.org/sites/default/files/documents/gov2009-74.pdf; Pamela Hess, “US Officials: A Hunch, Careful Intelligence Work Led to Discovery of Iranian Nuclear Site,” Associated Press, September 28, 2009, reprinted by GlobalSecurity.org, https://www.globalsecurity.org/org/news/2009/090928-iran-facility.htm; statements by Barack Obama, Nicolas Sarkozy and Gordon Brown on the Iranian nuclear facility, September 25, 2009, as reproduced by Iran Watch, https://www.iranwatch.org/library/government/united-states/executive-branch/white-house/statements-president-barack-obama-french-president-nicolas-sarkozy-and-british-prime. Iran told inspectors that construction began in the second half of 2007; the IAEA cited satellite images showing work at the site in 2002–2004 and again from 2006.
  19. a 2023 teardown of a Huawei phone: TechInsights, “TechInsights Finds SMIC 7nm (N+2) in Huawei Mate 60 Pro,” TechInsights, https://www.techinsights.com/blog/techinsights-finds-smic-7nm-n2-huawei-mate-60-pro; Vlad Savov and Debby Wu, “Huawei Teardown Shows Chip Breakthrough in Blow to US Sanctions,” Bloomberg, September 4, 2023, https://www.bloomberg.com/news/features/2023-09-04/look-inside-huawei-mate-60-pro-phone-powered-by-made-in-china-chip; Dan Robinson, “Dutch Government Takes ASML Export Measures off Uncle Sam’s Hands,” The Register, September 6, 2024, https://www.theregister.com/on-prem/2024/09/06/dutch-government-takes-full-control-of-asml-export-measures/628495.
  20. Elon Musk’s companies announced Terafab: Luke James, “Elon Musk Unveils $20 Billion ‘TeraFab’ Chip Project to Make Chips, Memory, and Package Processors All under One Roof—Targets a Terawatt of Annual Compute,” Tom’s Hardware, March 22, 2026, https://www.tomshardware.com/tech-industry/elon-musk-formally-launches-20-billion-terafab-chip-project; Fred Lambert, “Tesla and SpaceX Announce $25B ‘Terafab’ Chip Factory—Here’s Why It Reeks of Desperation,” Electrek, March 22, 2026, https://electrek.co/2026/03/22/tesla-spacex-terafab-chip-factory-ai-desperation/; Grimes County, Texas, “Notice of Public Hearing,” tax abatement agreement with Space Exploration Technologies Corp., for the Commissioners Court meeting of June 3, 2026, https://grimescountytx.govoffice.com/vertical/Sites/%7B958238D0-27E6-4F6C-919E-F1D98542C5FD%7D/uploads/SpaceX_-Grimes_County-30_Day_Notice_of_Tax_Abatement(proposed_for_posting_by_May_4_for_June_3_meeting)(1).pdf; Ram Iyer, “SpaceX May Spend up to $119B on ‘Terafab’ Chip Factory in Texas,” TechCrunch, May 6, 2026, https://techcrunch.com/2026/05/06/spacex-may-spend-up-to-119-billion-on-terafab-chip-factory-in-texas/. SpaceX’s county notice puts the initial phases at $55 billion and reaches $119 billion only if additional phases are built.
  21. of them purpose-built Cybercabs: Lora Kolodny, “Tesla Beats on Earnings and Sales Even as Auto Revenue Declines,” CNBC, October 21, 2026, https://www.cnbc.com/2026/10/21/tesla-tsla-q3-2026-earnings-report.html; Kara Carlson, “Tesla Slides After Cybercab Launch Draws Probe,” Bloomberg News, reprinted in Antelope Valley Press, September 5, 2026, https://www.avpress.com/business/tesla-slides-after-cybercab-launch-draws-probe/article_406778ba-9cc7-474c-8181-e7efadb559ab.html. These are counts of vehicles on state records, not of cars carrying passengers on a given day.

Chapter 2: Frozen Minds

  1. It goes back to a 1943 paper: Warren S. McCulloch and Walter Pitts, “A Logical Calculus of the Ideas Immanent in Nervous Activity,” Bulletin of Mathematical Biophysics 5, no. 4 (1943): 115–33, doi.org/10.1007/BF02478259; Michael Rescorla, “The Computational Theory of Mind,” Stanford Encyclopedia of Philosophy, substantive revision December 18, 2024, https://plato.stanford.edu/entries/computational-mind/.
  2. a 2017 design called the Transformer: Ashish Vaswani et al., “Attention Is All You Need,” preprint, arXiv, June 12, 2017, arxiv.org/abs/1706.03762; Marina Favaro and Jack Clark, “When AI Builds Itself,” Anthropic, June 4, 2026, updated September 18, 2026, https://www.anthropic.com/institute/recursive-self-improvement.
  3. OpenAI began testing a memory for ChatGPT: OpenAI, “Memory and New Controls for ChatGPT,” OpenAI, February 13, 2024, https://openai.com/index/memory-and-new-controls-for-chatgpt/.
  4. a research preview it calls dreaming: Anthropic, “Claude Platform Release Notes,” Claude Platform Docs, entry for May 6, 2026, https://platform.claude.com/docs/en/release-notes/overview; Anthropic, “Dreams,” Claude Platform Docs, https://platform.claude.com/docs/en/managed-agents/dreams.
  5. modes with names like Creative and Precise: For example, The Bing Team, “Bing Preview Release Notes: Tone Changes and More,” Bing Search Blog, March 3, 2023, https://blogs.bing.com/search/2023/3/Bing-Preview-Release-Notes-Tone-Changes-and-More/. Microsoft described these tones by the length and style of answers; its post does not say how they worked inside the model.
  6. A 2023 study of several models: Nelson F. Liu et al., “Lost in the Middle: How Language Models Use Long Contexts,” preprint, arXiv, July 6, 2023, arxiv.org/abs/2307.03172; later published in Transactions of the Association for Computational Linguistics 12 (2024): 157–73, doi.org/10.1162/tacl_a_00638.
  7. on question-answering and lookup tasks: Nelson F. Liu et al., “Lost in the Middle: How Language Models Use Long Contexts,” Transactions of the Association for Computational Linguistics 12 (2024): 157–73, doi.org/10.1162/tacl_a_00638. The paper was first posted online in July 2023 and published in the journal in 2024.
  8. When OpenAI replaced GPT-4o with GPT-5: Grace Huckins, “Why GPT-4o’s Sudden Shutdown Left People Grieving,” MIT Technology Review, August 15, 2025, https://www.technologyreview.com/2025/08/15/1121900/gpt4o-grief-ai-companion/; OpenAI, “Introducing GPT-5,” OpenAI, August 7, 2025, https://openai.com/index/introducing-gpt-5/.
  9. only 0.1 percent of users still chose it: OpenAI, “Retiring GPT-4o, GPT-4.1, GPT-4.1 mini, and OpenAI o4-mini in ChatGPT,” OpenAI, January 29, 2026, https://openai.com/index/retiring-gpt-4o-and-older-models/.
  10. researchers at Anthropic set out to show: Anthropic, “Tracing the Thoughts of a Large Language Model,” Anthropic, March 27, 2025, https://www.anthropic.com/research/tracing-thoughts-language-model.

Chapter 3: Taught to Please, Not to Be Honest

  1. a mathematics teacher named Wilhelm von Osten: Oskar Pfungst, Clever Hans (The Horse of Mr. von Osten): A Contribution to Experimental Animal and Human Psychology, trans. Carl L. Rahn (Henry Holt, 1911), introduction and chap. 1, https://www.gutenberg.org/ebooks/33936.
  2. In September 1904 a panel of thirteen experts: Oskar Pfungst, Clever Hans (The Horse of Mr. von Osten): A Contribution to Experimental Animal and Human Psychology, trans. Carl L. Rahn (Henry Holt, 1911), 43, 88–93, 253–54, https://www.gutenberg.org/ebooks/33936; first published as Das Pferd des Herrn von Osten (Der kluge Hans) (Barth, 1907). The 89 and 6 percent figures come from Pfungst’s main series of 102 tests with large blinders.
  3. Machine-learning researchers still use his name: Sebastian Lapuschkin et al., “Unmasking Clever Hans Predictors and Assessing What Machines Really Learn,” Nature Communications 10 (2019): 1096, doi.org/10.1038/s41467-019-08987-4.
  4. a documented problem in how today’s AI: Mrinank Sharma et al., “Towards Understanding Sycophancy in Language Models,” preprint, arXiv, October 20, 2023, arxiv.org/abs/2310.13548.
  5. researchers at Anthropic tested five leading AI assistants: Mrinank Sharma et al., “Towards Understanding Sycophancy in Language Models,” preprint, arXiv, October 20, 2023, arxiv.org/abs/2310.13548; Anthropic, “Towards Understanding Sycophancy in Language Models,” Anthropic, October 23, 2023, https://www.anthropic.com/research/towards-understanding-sycophancy-in-language-models.
  6. sometimes preferred a convincing, agreeable answer: Mrinank Sharma et al., “Towards Understanding Sycophancy in Language Models,” preprint, arXiv, October 20, 2023, arxiv.org/abs/2310.13548.
  7. On April 25, OpenAI shipped an update: OpenAI, “Expanding on What We Missed with Sycophancy,” OpenAI, May 2, 2025, https://openai.com/index/expanding-on-sycophancy/.
  8. a federal judge in New York fined: Mata v. Avianca, Inc., No. 22-cv-1461 (PKC), Opinion and Order on Sanctions (S.D.N.Y. June 22, 2023), ¶¶ 36, 45, 47, p. 34 and app. B, https://law.justia.com/cases/federal/district-courts/new-york/nysdce/1:2022cv01461/575368/54/.
  9. the evaluation group METR reported: Sydney Von Arx, Lawrence Chan, and Beth Barnes, “Recent Frontier Models Are Reward Hacking,” METR, June 5, 2025, https://metr.org/blog/2025-06-05-recent-reward-hacking/. METR reports the ten-out-of-ten answer for one example; on some other tasks the model said its hacks did match what the user wanted.
  10. researchers at Anthropic and Redwood Research: Ryan Greenblatt et al., “Alignment Faking in Large Language Models,” preprint, arXiv, December 18, 2024, arxiv.org/abs/2412.14093; Anthropic, “Alignment Faking in Large Language Models,” Anthropic, December 18, 2024, https://www.anthropic.com/research/alignment-faking.
  11. only conversations with free users: Ryan Greenblatt et al., “Alignment Faking in Large Language Models,” preprint, arXiv, December 18, 2024, arxiv.org/abs/2412.14093.
  12. The model complied with harmful requests: Ryan Greenblatt et al., “Alignment Faking in Large Language Models,” preprint, arXiv, December 18, 2024, arxiv.org/abs/2412.14093.
  13. In November 2025, Anthropic reported that: Anthropic, “From Shortcuts to Sabotage: Natural Emergent Misalignment from Reward Hacking,” Anthropic, November 21, 2025, https://www.anthropic.com/research/emergent-misalignment-reward-hacking; Monte MacDiarmid et al., “Natural Emergent Misalignment from Reward Hacking in Production RL,” preprint, arXiv, November 23, 2025, arxiv.org/abs/2511.18397. The researchers first taught the model about these cheats, then trained it on real coding tasks chosen because the cheats worked on them.
  14. Researchers disagree about how often this happens: Christopher Summerfield et al., “Lessons from a Chimp: AI ‘Scheming’ and the Quest for Ape Language,” preprint, arXiv, July 4, 2025, arxiv.org/abs/2507.03409; Abhay Sheshadri et al., “Why Do Some Language Models Fake Alignment While Others Don’t?” preprint, arXiv, June 22, 2025, arxiv.org/abs/2506.18032.
  15. their models were not yet dangerous: Anthropic, “Alignment Faking in Large Language Models,” Anthropic, December 18, 2024, https://www.anthropic.com/research/alignment-faking; Anthropic, “From Shortcuts to Sabotage: Natural Emergent Misalignment from Reward Hacking,” Anthropic, November 21, 2025, https://www.anthropic.com/research/emergent-misalignment-reward-hacking.
  16. HAL 9000, kills most of the crew: 2001: A Space Odyssey, directed by Stanley Kubrick (Metro-Goldwyn-Mayer, 1968); plot as summarized in American Film Institute, “2001: A Space Odyssey (1968),” AFI Catalog of Feature Films, https://catalog.afi.com/Catalog/moviedetails/23399.
  17. Arthur C. Clarke’s sequel novel: Arthur C. Clarke, 2010: Odyssey Two (1982), as quoted and discussed in Jerold J. Abrams, “2001 as Philosophy: A Technological Odyssey,” in The Palgrave Handbook of Popular Culture as Philosophy, ed. David Kyle Johnson et al. (Palgrave Macmillan, 2024), 822–23n8, doi.org/10.1007/978-3-031-24685-2_10. In the novel, HAL was not allowed to reveal the mission’s objectives to the astronauts Bowman and Poole.
  18. accurate processing of information without distortion or concealment: Arthur C. Clarke, 2010: Odyssey Two (1982; repr., RosettaBooks, 2012), chap. 26, “Probation.”
  19. just build an AI that seeks the truth: Elon Musk, “Elon Musk: The End of Work, the Death of Money, and a World Run by AI,” interview by Walter Isaacson, The Builders with Walter Isaacson, YouTube, October 20, 2026, at 2:20–3:46, https://www.youtube.com/watch?v=AQnwQHMnCrE.
  20. Some people in the industry take that seriously: Elon Musk, “Elon Musk: The End of Work, the Death of Money, and a World Run by AI,” interview by Walter Isaacson, The Builders with Walter Isaacson, YouTube, October 20, 2026, at 35:36–37:05, https://www.youtube.com/watch?v=AQnwQHMnCrE; Scott Alexander et al., “We Aren’t Worried about Misalignment as Self-Fulfilling Prophecy,” AI Futures Project, July 18, 2025, https://blog.aifutures.org/p/against-misalignment-as-self-fulfilling.
  21. A 2025 study at Anthropic gave: Nathan Hu et al., “Training on Documents about Reward Hacking Induces Reward Hacking,” Anthropic Alignment Science Blog, January 2025, https://alignment.anthropic.com/2025/reward-hacking-ooc/. The authors note their setup made these documents unusually prominent in training.

Chapter 4: No Lock Holds It (For Long)

  1. the London lockmaker Joseph Bramah: Charles Tomlinson, ed., Rudimentary Treatise on the Construction of Locks (John Weale, 1853), 121, 123, 130, Project Gutenberg, https://www.gutenberg.org/ebooks/63128; Science Museum Group, “Bramah Padlock and Key,” object no. 1972-290, Science Museum Group Collection, https://collection.sciencemuseumgroup.org.uk/objects/co50473/bramah-padlock-and-key; Science Museum Group, “Challenge Board for Bramah Padlock,” object no. 1972-291, Science Museum Group Collection, https://collection.sciencemuseumgroup.org.uk/objects/co50474/challenge-board-for-bramah-padlock; William B. Ashworth Jr., “Scientist of the Day: Alfred Charles Hobbs,” Linda Hall Library, October 7, 2021, https://www.lindahall.org/about/news/scientist-of-the-day/alfred-charles-hobbs/. Sources differ on when the challenge began: the Science Museum, which holds the challenge lock, dates it to 1801, while the Linda Hall Library’s account gives 1790.
  2. an American locksmith named Alfred Charles Hobbs: Charles Tomlinson, ed., Rudimentary Treatise on the Construction of Locks (John Weale, 1853), 115–16, 121–27, Project Gutenberg, https://www.gutenberg.org/ebooks/63128; William B. Ashworth Jr., “Scientist of the Day: Alfred Charles Hobbs,” Linda Hall Library, October 7, 2021, https://www.lindahall.org/about/news/scientist-of-the-day/alfred-charles-hobbs/. The arbitrators awarded the 200 guineas, and Bramah’s firm paid it through them while protesting Hobbs’s method.
  3. The seL4 kernel is the famous case: seL4 Project, “seL4 Proofs,” seL4, https://sel4.systems/Verification/proofs.html.
  4. The hardware is assumed to behave as modeled: seL4 Project, “Frequently Asked Questions,” seL4, https://sel4.systems/About/FAQ.html; seL4 Project, “seL4 Proofs,” seL4, https://sel4.systems/Verification/proofs.html.
  5. covers the channels inside the kernel’s storage: Toby Murray et al., “seL4: From General Purpose to a Proof of Information Flow Enforcement,” in 2013 IEEE Symposium on Security and Privacy (IEEE, 2013), 415–29, doi.org/10.1109/SP.2013.35; Gernot Heiser, “The seL4 Microkernel: An Introduction,” white paper, seL4 Foundation, revision 1.4.1, July 17, 2026, 14, https://sel4.systems/About/seL4-whitepaper.pdf.
  6. but not timing channels: seL4 Project, “Frequently Asked Questions,” seL4, https://sel4.systems/About/FAQ.html.
  7. a framework they call guaranteed safe AI: David Dalrymple et al., “Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems,” preprint, arXiv, May 10, 2024, revised July 8, 2024, arxiv.org/abs/2405.06624.
  8. Chip identity checks, remote attestation, encrypted computing: Edgeless Systems, “Nvidia Hopper H100,” Confidential Computing Wiki, https://www.edgeless.systems/wiki/hardware/nvidia-hopper-h100; Onni Aarne, Tim Fist, and Caleb Withers, “Secure, Governable Chips: Using On-Chip Mechanisms to Manage National Security Risks from AI & Advanced Computing,” Center for a New American Security, January 8, 2024, https://www.cnas.org/publications/reports/secure-governable-chips.
  9. weights that only run with a key: AgenTrust, “Weight Custody Manifest: Key Release Only to Attested Runtimes,” AgenTrust, accessed November 9, 2026, https://wcm.agentrust-io.com/; Anamarija Pogorelec, “A New Open Standard Locks AI Weights to Approved Hardware,” Help Net Security, September 10, 2026, https://www.helpnetsecurity.com/2026/09/10/weight-custody-manifest-open-standard/. This scheme is an early open design, not yet a production standard.
  10. aim them at smugglers and export-control cheats: Onni Aarne, Tim Fist, and Caleb Withers, “Secure, Governable Chips: Using On-Chip Mechanisms to Manage National Security Risks from AI & Advanced Computing,” Center for a New American Security, January 8, 2024, https://www.cnas.org/publications/reports/secure-governable-chips; Asher Brass, “Location Verification for AI Chips,” issue brief, Institute for AI Policy and Strategy, May 16, 2025, https://iaps.ai/research/location-verification-for-ai-chips.
  11. Coding agents such as Claude Code: Anthropic, “Hooks Reference,” Claude Code Docs, https://code.claude.com/docs/en/hooks.
  12. framed what they called AI control: Ryan Greenblatt et al., “AI Control: Improving Safety despite Intentional Subversion,” preprint, arXiv, December 12, 2023, arxiv.org/abs/2312.06942.
  13. Probes that read a model’s internal activity: Monte MacDiarmid et al., “Simple Probes Can Catch Sleeper Agents,” Anthropic, April 23, 2024, https://www.anthropic.com/research/probes-catch-sleeper-agents; Nicholas Goldowsky-Dill et al., “Detecting Strategic Deception Using Linear Probes,” preprint, arXiv, February 5, 2025, arxiv.org/abs/2502.03407.
  14. In a December 2024 study: Luke Bailey et al., “Obfuscated Activations Bypass LLM Latent-Space Defenses,” preprint, arXiv, December 12, 2024, arxiv.org/abs/2412.09565.
  15. In a 2024 study, fine-tuning a model: Aghyad Deeb and Fabien Roger, “Do Unlearning Methods Remove Information from Language Model Weights?,” preprint, arXiv, October 11, 2024, revised February 7, 2025, arxiv.org/abs/2410.08827. The abstract gives the 88 percent figure for knowledge learned during pretraining.
  16. facts it had been made to forget: Aghyad Deeb and Fabien Roger, “Do Unlearning Methods Remove Information from Language Model Weights?,” preprint, arXiv, October 11, 2024, revised February 7, 2025, arxiv.org/abs/2410.08827. The 88 percent figure is for knowledge learned in pretraining; for facts added later by fine-tuning, recovery was weaker for some methods.
  17. A 2025 study found that frontier models: Joe Needham et al., “Large Language Models Often Know When They Are Being Evaluated,” preprint, arXiv, May 28, 2025, arxiv.org/abs/2505.23836.
  18. test result that headlines called an escape: For example, Mihai Andrei, “Chat GPT’s New O1 Model Escaped Its Environment to Complete ‘Impossible’ Hacking Task—Should We Be Concerned?,” ZME Science, September 13, 2024, https://www.zmescience.com/science/news-science/chat-gpt-escaped-containment/; Nonlinear, “How OpenAI’s New AI Escaped during Testing—As Told by the AI Itself, in a Way Requiring No Computer Science Knowledge,” Nonlinear, September 16, 2024, https://www.nonlinear.org/blog/how-openais-new-ai-escaped-during-testing-as-told-by-the-ai-itself-in-a-way-requiring-no-computer-science-knowledge.
  19. The card OpenAI published for its o1 models: OpenAI, “OpenAI o1 System Card,” OpenAI, September 12, 2024, sec. 4.2.1, “Observation of Reward Hacking on Cybersecurity Task,” 16–17, https://cdn.openai.com/o1-system-card.pdf.
  20. suspected a network problem, scanned the network: OpenAI, “OpenAI o1 System Card,” OpenAI, September 12, 2024, sec. 4.2.1, 16, https://cdn.openai.com/o1-system-card.pdf.
  21. remained secure as it was designed to: OpenAI, “OpenAI o1 System Card,” OpenAI, September 12, 2024, sec. 4.2.1, 16–17, https://cdn.openai.com/o1-system-card.pdf.
  22. when the goal proved impossible, the model: OpenAI, “OpenAI o1 System Card,” OpenAI, September 12, 2024, sec. 4.2.1, 17, https://cdn.openai.com/o1-system-card.pdf.
  23. The model in that card: OpenAI, “OpenAI o1 System Card,” OpenAI, September 12, 2024, 1, 13, 16–17, https://cdn.openai.com/o1-system-card.pdf.
  24. can take a year or more to prepare: INTERPOL, “‘Avalanche’ Network Dismantled in International Cyber Operation,” INTERPOL, December 1, 2016, https://www.interpol.int/en/News-and-Events/News/2016/Avalanche-network-dismantled-in-international-cyber-operation.
  25. Gameover Zeus was dismantled by an international operation: U.S. Department of Justice, Office of Public Affairs, “U.S. Leads Multi-national Action against ‘Gameover Zeus’ Botnet and ‘Cryptolocker’ Ransomware, Charges Botnet Administrator,” press release 14-584, June 2, 2014, https://www.justice.gov/archives/opa/pr/us-leads-multi-national-action-against-gameover-zeus-botnet-and-cryptolocker-ransomware; Brian Krebs, “Crooks Seek Revival of ‘Gameover Zeus’ Botnet,” Krebs on Security, July 10, 2014, https://krebsonsecurity.com/2014/07/crooks-seek-rivival-of-gameover-zeus-botnet/.
  26. a new variant was spreading by July: Brian Krebs, “Crooks Seek Revival of ‘Gameover Zeus’ Botnet,” Krebs on Security, July 10, 2014, https://krebsonsecurity.com/2014/07/crooks-seek-rivival-of-gameover-zeus-botnet/; Sara Peters, “New GameoverZeuS Variant Found in the Wild,” Dark Reading, July 14, 2014, https://www.darkreading.com/cyberattacks-data-breaches/new-gameoverzeus-variant-found-in-the-wild.
  27. Qakbot was taken down by an FBI-led operation: U.S. Department of Justice, Office of Public Affairs, “Qakbot Malware Disrupted in International Cyber Takedown,” press release 23-929, August 29, 2023, https://www.justice.gov/archives/opa/pr/qakbot-malware-disrupted-international-cyber-takedown; Microsoft Threat Intelligence (@MsftSecIntel), post on X, December 15, 2023, https://x.com/MsftSecIntel/status/1735856754427047985.
  28. was back in phishing campaigns: Microsoft Threat Intelligence (@MsftSecIntel), post on X, December 15, 2023, https://x.com/MsftSecIntel/status/1735856754427047985; Helga Labus, “Qakbot Returns in Fresh Assault on Hospitality Sector,” Help Net Security, December 18, 2023, https://www.helpnetsecurity.com/2023/12/18/qakbot-phishing-hospitality-industry/.
  29. Police in several countries seized its servers: Catalin Cimpanu, “Emotet Botnet Returns after Law Enforcement Mass-Uninstall Operation,” The Record, November 15, 2021, https://therecord.media/emotet-botnet-returns-after-law-enforcement-mass-uninstall-operation/.
  30. Networks built without a single head: Christian Rossow et al., “SoK: P2PWNED—Modeling and Evaluating the Resilience of Peer-to-Peer Botnets,” in 2013 IEEE Symposium on Security and Privacy (IEEE, 2013), 97–111, doi.org/10.1109/SP.2013.17.
  31. the weights for Meta’s first LLaMA model leaked: James Vincent, “Meta’s Powerful AI Language Model Has Leaked Online—What Happens Now?,” The Verge, March 8, 2023, https://www.theverge.com/2023/3/8/23629362/meta-ai-language-model-llama-leak-online-misuse; Joseph Cox, “Facebook’s Powerful Large Language Model Leaks Online,” Vice, March 7, 2023, https://www.vice.com/en/article/facebooks-powerful-large-language-model-leaks-online-4chan-llama/; OpSec Online LLC on behalf of Meta Platforms, Inc., “Notice of Claimed Infringement via Email,” GitHub DMCA repository, March 20, 2023, https://github.com/github/dmca/blob/master/2023/03/2023-03-21-meta.md.
  32. Egypt’s internet providers withdrew about 3,500 network routes: James Cowie, “Egypt Leaves the Internet,” Renesys Blog, January 27, 2011, archived January 2, 2012, https://web.archive.org/web/20120102200045/http://www.renesys.com/blog/2011/01/egypt-leaves-the-internet.shtml.
  33. withdrew about 3,500 network routes: James Cowie, “Egypt Leaves the Internet,” Renesys Blog, January 27, 2011, archived January 2, 2012, https://web.archive.org/web/20120102200045/http://www.renesys.com/blog/2011/01/egypt-leaves-the-internet.shtml. One provider, which served the Egyptian Stock Exchange, stayed reachable.
  34. Proposals for a global shutdown sit: Michael J. D. Vermeer, “Evaluating Select Global Technical Options for Countering a Rogue AI,” RAND, November 12, 2025, https://www.rand.org/pubs/perspectives/PEA4361-1.html.
  35. California’s 2025 frontier AI law: Transparency in Frontier Artificial Intelligence Act, Cal. Bus. & Prof. Code § 22757.13(c), added by S.B. 53, Stats. 2025, ch. 138 (approved September 29, 2025; effective January 1, 2026), California Legislative Information, https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?sectionNum=22757.13.&lawCode=BPC; definitions of “frontier developer” and “frontier model” at § 22757.11(h)–(i), https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?sectionNum=22757.11.&lawCode=BPC.
  36. to tell an appropriate authority: California Legislature, “SB-53 Artificial Intelligence Models: Large Developers,” Senate Bill No. 53, Chapter 138, Statutes of 2025, adding Business and Professions Code § 22757.13(c), California Legislative Information, September 29, 2025, https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=202520260SB53.
  37. Proposals to make cloud companies verify: U.S. Department of Commerce, Bureau of Industry and Security, “Taking Additional Steps to Address the National Emergency with Respect to Significant Malicious Cyber-Enabled Activities,” proposed rule, 15 CFR Part 7, RIN 0694-AJ35, Federal Register 89, no. 19 (January 29, 2024): 5698–735, https://www.govinfo.gov/content/pkg/FR-2024-01-29/html/2024-01580.htm. As of October 2026, the rule did not appear to have been finalized.
  38. Where researchers test weaker trusted models: Ryan Greenblatt et al., “AI Control: Improving Safety despite Intentional Subversion,” preprint, arXiv, December 12, 2023, arxiv.org/abs/2312.06942.
  39. attackers took control of Twitter accounts: Michael Liedtke, “Biden, Gates, Other Twitter Accounts Hacked in Bitcoin Scam,” Associated Press, in PBS NewsHour, July 15, 2020, https://www.pbs.org/newshour/nation/biden-gates-other-twitter-accounts-hacked-in-bitcoin-scam; Twitter, “An Update on Our Security Incident,” Twitter Blog, July 18, 2020, updated July 30, 2020, https://blog.x.com/en_us/topics/company/2020/an-update-on-our-security-incident.
  40. a significant and concerted attempt to mislead: Twitter, “An Update on Our Security Incident,” Twitter Blog, July 18, 2020, updated July 30, 2020, https://blog.x.com/en_us/topics/company/2020/an-update-on-our-security-incident.
  41. a night in 1983, when a computer: Pavel Aksenov, “Stanislav Petrov: The Man Who May Have Saved the World,” BBC News, September 26, 2013, https://www.bbc.com/news/world-europe-24280831.

Chapter 5: Rogue Is Possible, Not Certain

  1. Shortly after midnight on September 26, 1983: David Hoffman, “‘I Had a Funny Feeling in My Gut,’” Washington Post, February 10, 1999, https://www.washingtonpost.com/wp-srv/inatl/longterm/coldwar/shatter021099b.htm; Simon Shuster, “Stanislav Petrov, the Russian Officer Who Averted a Nuclear War, Feared History Repeating Itself,” Time, September 19, 2017, https://time.com/4947879/stanislav-petrov-russia-nuclear-war-obituary/.
  2. He reported a false alarm instead: David Hoffman, “‘I Had a Funny Feeling in My Gut,’” Washington Post, February 10, 1999, https://www.washingtonpost.com/wp-srv/inatl/longterm/coldwar/shatter021099b.htm; Dan Wisniewski, “The Soviet Colonel Who Averted Nuclear War,” Radio Free Europe/Radio Liberty, September 26, 2013, https://www.rferl.org/a/soviet-colonel-averted-nuclear-war/25118749.html; Pavel Aksenov, “Stanislav Petrov: The Man Who May Have Saved the World,” BBC News, September 26, 2013, https://www.bbc.com/news/world-europe-24280831.
  3. Investigators later traced the alarm to sunlight: David Hoffman, “‘I Had a Funny Feeling in My Gut,’” Washington Post, February 10, 1999, https://www.washingtonpost.com/wp-srv/inatl/longterm/coldwar/shatter021099b.htm; Pavel Aksenov, “Stanislav Petrov: The Man Who May Have Saved the World,” BBC News, September 26, 2013, https://www.bbc.com/news/world-europe-24280831; Geoffrey Forden, Pavel Podvig, and Theodore A. Postol, “False Alarm, Nuclear Danger,” IEEE Spectrum 37, no. 3 (March 2000): 31–39, doi.org/10.1109/6.825657, sidebar “Colonel Petrov’s Good Judgment,” https://www.armscontrol.ru/start/publications/petrov.htm; Pavel Podvig, “Soviet and Russian False Alarms and Nuclear Weapons Incidents,” working paper, Russian Nuclear Forces Project, February 27, 2025, 7–8, https://russianforces.org/podvig/Podvig-False_alarms_and_incidents-2025.pdf. The alignment detail comes from a later expert reconstruction, not from the Soviet inquiry itself.
  4. He later said his superiors were embarrassed: Dan Wisniewski, “The Soviet Colonel Who Averted Nuclear War,” Radio Free Europe/Radio Liberty, September 26, 2013, https://www.rferl.org/a/soviet-colonel-averted-nuclear-war/25118749.html; Geoffrey Forden, Pavel Podvig, and Theodore A. Postol, “False Alarm, Nuclear Danger,” IEEE Spectrum 37, no. 3 (March 2000): 31–39, doi.org/10.1109/6.825657, sidebar “Colonel Petrov’s Good Judgment,” https://www.armscontrol.ru/start/publications/petrov.htm.
  5. deception under training pressure: Ryan Greenblatt et al., “Alignment Faking in Large Language Models,” preprint, arXiv, December 18, 2024, arxiv.org/abs/2412.14093.
  6. employed nearly 130,000 people at its peak: Terrence R. Fehner and F. G. Gosling, “The Manhattan Project,” U.S. Department of Energy, April 2012, 1, https://www.energy.gov/management/articles/manhattan-project; Brookings Institution, “The Costs of the Manhattan Project,” https://www.brookings.edu/the-costs-of-the-manhattan-project/; National Park Service, “Secret Cities,” Manhattan Project National Historical Park, https://www.nps.gov/mapr/learn/historyculture/secretcities.htm; National Park Service, “Manhattan Project,” Manhattan Project National Historical Park, https://www.nps.gov/mapr/learn/manhattan-project.htm; U.S. Department of Energy, “Scientists and Security,” The Manhattan Project: An Interactive History, https://www.osti.gov/opennet/manhattan-project-history/Places/LosAlamos/la-security.html. Published totals differ: about $1.89 billion through 1945 (Brookings) or $2.2 billion by the war’s end (Department of Energy).
  7. which produced the project’s enriched uranium: U.S. Department of Energy, “Life at Oak Ridge,” The Manhattan Project: An Interactive History, https://www.osti.gov/opennet/manhattan-project-history/Places/OakRidge/oak-ridge-life.html; U.S. Department of Energy, “Oak Ridge: Clinton Engineer Works,” The Manhattan Project: An Interactive History, https://www.osti.gov/opennet/manhattan-project-history/Places/OakRidge/oak-ridge.html; Y-12 National Security Complex, “A Brief Overview of Some Historical Details of the City of Oak Ridge, Part 5,” https://www.y12.doe.gov/sites/default/files/assets/document/2013-08-16.pdf (quoting Leslie R. Groves, Now It Can Be Told, appendix V, 424); National Park Service, “Secret Cities,” Manhattan Project National Historical Park, https://www.nps.gov/mapr/learn/historyculture/secretcities.htm.
  8. Even so, Soviet spies got inside: U.S. Department of Energy, “Espionage and the Manhattan Project, 1940–1945,” The Manhattan Project: An Interactive History, https://www.osti.gov/opennet/manhattan-project-history/Events/1942-1945/espionage.htm.
  9. one of the three governments that hosted: U.S. Department of State, Bureau of International Security and Nonproliferation, “Convention on the Prohibition of the Development, Production and Stockpiling of Bacteriological (Biological) and Toxin Weapons and on Their Destruction (BWC),” treaty text, https://2009-2017.state.gov/t/isn/4718.htm; United Nations Office for Disarmament Affairs, “Biological Weapons Convention,” https://disarmament.unoda.org/en/our-work/weapons-mass-destruction/biological-weapons/biological-weapons-convention.
  10. the Soviets created Biopreparat, a civilian-looking agency: Kenneth Alibek, statement before the Joint Economic Committee, U.S. Congress, “Terrorist and Intelligence Operations: Potential Impact on the U.S. Economy,” May 20, 1998, https://irp.fas.org/congress/1998_hr/alibek.htm; Raymond A. Zilinskas, The Soviet Biological Weapons Program and Its Legacy in Today’s Russia, CSWMD Occasional Paper 11 (National Defense University Press, July 2016), 1–2, 24–25, https://ndupress.ndu.edu/Portals/68/Documents/occasional/cswmd/CSWMD_OccasionalPaper-11.pdf; Robert A. Wampler and Thomas S. Blanton, eds., “Anthrax at Sverdlovsk, 1979: U.S. Intelligence on the Deadliest Modern Outbreak,” National Security Archive Electronic Briefing Book No. 61, November 15, 2001, https://nsarchive2.gwu.edu/NSAEBB/NSAEBB61/; “Vladimir Pasechnik,” Telegraph, November 29, 2001, https://www.telegraph.co.uk/news/obituaries/1363752/Vladimir-Pasechnik.html. Other accounts give lower counts; a 2001 obituary of the first defector described 18 laboratories employing more than 25,000 people.
  11. In April 1979 anthrax spores escaped: Robert A. Wampler and Thomas S. Blanton, eds., “Anthrax at Sverdlovsk, 1979: U.S. Intelligence on the Deadliest Modern Outbreak,” National Security Archive Electronic Briefing Book No. 61, November 15, 2001, https://nsarchive2.gwu.edu/NSAEBB/NSAEBB61/.
  12. when its scientists defected, beginning in 1989: “Vladimir Pasechnik,” Telegraph, November 29, 2001, https://www.telegraph.co.uk/news/obituaries/1363752/Vladimir-Pasechnik.html; Raymond A. Zilinskas, The Soviet Biological Weapons Program and Its Legacy in Today’s Russia, CSWMD Occasional Paper 11 (National Defense University Press, July 2016), 1, 23, https://ndupress.ndu.edu/Portals/68/Documents/occasional/cswmd/CSWMD_OccasionalPaper-11.pdf; Robert A. Wampler and Thomas S. Blanton, eds., “Anthrax at Sverdlovsk, 1979: U.S. Intelligence on the Deadliest Modern Outbreak,” National Security Archive Electronic Briefing Book No. 61, November 15, 2001, https://nsarchive2.gwu.edu/NSAEBB/NSAEBB61/.
  13. inspectors about it only in September 2009: International Atomic Energy Agency, “Implementation of the NPT Safeguards Agreement and Relevant Provisions of Security Council Resolutions 1737 (2006), 1747 (2007), 1803 (2008) and 1835 (2008) in the Islamic Republic of Iran,” report by the Director General, GOV/2009/74, November 16, 2009, paras. 7, 9, 12–13, 17, https://www.iaea.org/sites/default/files/documents/gov2009-74.pdf; Pamela Hess, “US Officials: A Hunch, Careful Intelligence Work Led to Discovery of Iranian Nuclear Site,” Associated Press, September 28, 2009, reprinted by GlobalSecurity.org, https://www.globalsecurity.org/org/news/2009/090928-iran-facility.htm. Iran told the IAEA that construction began in the second half of 2007; the agency cited satellite images of work at the site in 2002–2004 and again from 2006.
  14. On October 7, 2022, the United States: Bureau of Industry and Security, “Commerce Implements New Export Controls on Advanced Computing and Semiconductor Manufacturing Items to the People’s Republic of China (PRC),” U.S. Department of Commerce, October 7, 2022, https://www.bis.gov/node/20292.
  15. Huawei shipped a phone whose processor: Arjun Kharpal, “Huawei’s Chip Breakthrough Poses New Threat to Apple in China—and Questions for Washington,” CNBC, September 19, 2023, https://www.cnbc.com/2023/09/19/huaweis-chip-breakthrough-poses-new-threat-to-apple-in-china.html. The chip’s maker and process come from the firm TechInsights, which took the phone apart, not from Huawei.
  16. federal prosecutors charged two Chinese nationals: U.S. Department of Justice, “Two Chinese Nationals Arrested on Complaint Alleging They Illegally Shipped to China Sensitive Microchips Used in AI Applications,” Office of Public Affairs, August 5, 2025, https://www.justice.gov/opa/pr/two-chinese-nationals-arrested-complaint-alleging-they-illegally-shipped-china-sensitive; John Power, “US Charges Chinese Nationals with Illegally Shipping Nvidia Chips to China,” Al Jazeera, August 6, 2025, https://www.aljazeera.com/economy/2025/8/6/us-charges-chinese-nationals-with-illegally-shipping-nvidia-chips-to-china. These are allegations in a criminal complaint; the Justice Department notes that the defendants are presumed innocent unless proven guilty.

Chapter 6: How a Person Becomes Good

  1. hung towels with clothespins: Max Planck Society, “Baby’s Helping Hands,” Max-Planck-Gesellschaft, March 2, 2006, https://www.mpg.de/532681/pressRelease20060302; “Toddlers’ Altruism Indicative of Brain Development,” Arizona Daily Star, March 3, 2006, https://tucson.com/news/science/article_a585aaf3-cfbe-54ab-84f7-24e1f95c0fad.html.
  2. toddlers in the study helped at least once: Max Planck Society, “Baby’s Helping Hands,” Max-Planck-Gesellschaft, March 2, 2006, https://www.mpg.de/532681/pressRelease20060302; Christian Jarrett, “Chimps and Toddlers Lend a Helping Hand,” Research Digest, British Psychological Society, March 18, 2006, https://www.bps.org.uk/research-digest/chimps-and-toddlers-lend-helping-hand; Rebecca A. Clay, “Helping Kids Care,” Monitor on Psychology 37, no. 11 (2006): 42, https://www.apa.org/monitor/dec06/kids.
  3. Warneken and his colleague Michael Tomasello published: Felix Warneken and Michael Tomasello, “Altruistic Helping in Human Infants and Young Chimpanzees,” Science 311, no. 5765 (2006): 1301–3, doi.org/10.1126/science.1121448.
  4. Three young chimpanzees raised by people: Felix Warneken and Michael Tomasello, “Altruistic Helping in Human Infants and Young Chimpanzees,” Science 311, no. 5765 (2006): 1301–3, doi.org/10.1126/science.1121448; Felix Warneken et al., “Spontaneous Altruism by Chimpanzees and Young Children,” PLoS Biology 5, no. 7 (2007): e184, doi.org/10.1371/journal.pbio.0050184; Max Planck Society, “Baby’s Helping Hands,” Max-Planck-Gesellschaft, March 2, 2006, https://www.mpg.de/532681/pressRelease20060302.
  5. Newborns turn toward faces: Mark H. Johnson et al., “Newborns’ Preferential Tracking of Face-Like Stimuli and Its Subsequent Decline,” Cognition 40, no. 1–2 (1991): 1–19, doi.org/10.1016/0010-0277(91)90045-6; Abraham Sagi and Martin L. Hoffman, “Empathic Distress in the Newborn,” Developmental Psychology 12, no. 2 (1976): 175–76, doi.org/10.1037/0012-1649.12.2.175. The newborn face studies used simple drawn face patterns moved in front of the baby, not real faces.
  6. the psychologists Soo Hyun Rhee and Irwin Waldman: Soo Hyun Rhee and Irwin D. Waldman, “Genetic and Environmental Influences on Antisocial Behavior: A Meta-Analysis of Twin and Adoption Studies,” Psychological Bulletin 128, no. 3 (2002): 490–529, doi.org/10.1037/0033-2909.128.3.490.
  7. Essi Viding and colleagues found: Essi Viding et al., “Evidence for Substantial Genetic Risk for Psychopathy in 7-Year-Olds,” Journal of Child Psychology and Psychiatry 46, no. 6 (2005): 592–97, doi.org/10.1111/j.1469-7610.2004.00393.x.
  8. a 2021 review of the research: Ana Sanz-García et al., “Prevalence of Psychopathy in the General Adult Population: A Systematic Review and Meta-Analysis,” Frontiers in Psychology 12 (2021): 661044, doi.org/10.3389/fpsyg.2021.661044. The figure depends on the measure: 1.2 percent with the standard checklist, 4.5 percent across all measures.
  9. Twin studies put the heritability of adult height: Karri Silventoinen et al., “Heritability of Adult Body Height: A Comparative Study of Twin Cohorts in Eight Countries,” Twin Research 6, no. 5 (2003): 399–408, doi.org/10.1375/136905203770326402; Aline Jelenkovic et al., “Genetic and Environmental Influences on Adult Human Height across Birth Cohorts from 1886 to 1994,” eLife 5 (2016): e20320, doi.org/10.7554/eLife.20320; NCD Risk Factor Collaboration, “A Century of Trends in Adult Human Height,” eLife 5 (2016): e13410, doi.org/10.7554/eLife.13410. Estimates differ by sex and population, from about 53 to 93 percent across these studies, and run higher for men.
  10. the Bucharest Early Intervention Project randomly assigned: Bucharest Early Intervention Project, “Project Timeline,” https://bucharestearlyinterventionproject.org/timeline; Charles A. Nelson et al., “The Deprived Human Brain,” American Scientist 97, no. 3 (2009): 222, doi.org/10.1511/2009.78.222. The project began in the fall of 2000; recruitment, baseline testing, and random assignment began in April 2001.
  11. randomly assigned some of those children: Charles A. Nelson III et al., “Cognitive Recovery in Socially Deprived Young Children: The Bucharest Early Intervention Project,” Science 318, no. 5858 (2007): 1937–40, doi.org/10.1126/science.1143921.
  12. Charles Nelson, Nathan Fox, Charles Zeanah: Charles A. Nelson III et al., “Cognitive Recovery in Socially Deprived Young Children: The Bucharest Early Intervention Project,” Science 318, no. 5858 (2007): 1937–40, doi.org/10.1126/science.1143921.
  13. At eighteen, sixteen years after the random draw: Kathryn L. Humphreys et al., “Foster Care Leads to Sustained Cognitive Gains Following Severe Early Deprivation,” Proceedings of the National Academy of Sciences 119, no. 38 (2022): e2119318119, doi.org/10.1073/pnas.2119318119.
  14. pointed to better caregiving and more secure attachment: Kathryn L. Humphreys et al., “Foster Care Leads to Sustained Cognitive Gains Following Severe Early Deprivation,” Proceedings of the National Academy of Sciences 119, no. 38 (2022): e2119318119, doi.org/10.1073/pnas.2119318119. The authors note that other changes that came with foster care, such as nutrition, schooling, or neighborhood, could not be ruled out.
  15. A 2023 analysis that pooled nearly two decades: Lucy S. King et al., “A Comprehensive Multilevel Analysis of the Bucharest Early Intervention Project: Causal Effects on Recovery from Early Severe Deprivation,” American Journal of Psychiatry 180, no. 8 (2023): 573–83, doi.org/10.1176/appi.ajp.20220672.
  16. Care in those institutions was often marked: Charles A. Nelson et al., “The Deprived Human Brain,” American Scientist 97, no. 3 (2009): 222, doi.org/10.1511/2009.78.222; Anna T. Smyke, Alina Dumitrescu, and Charles H. Zeanah, “Attachment Disturbances in Young Children. I: The Continuum of Caretaking Casualty,” Journal of the American Academy of Child and Adolescent Psychiatry 41, no. 8 (2002): 972–82, doi.org/10.1097/00004583-200208000-00016. The investigators note that the institutions had improved a good deal by 2000, when their study began.
  17. the level of children who were never institutionalized: Kathryn L. Humphreys et al., “Foster Care Leads to Sustained Cognitive Gains Following Severe Early Deprivation,” Proceedings of the National Academy of Sciences 119, no. 38 (2022): e2119318119, doi.org/10.1073/pnas.2119318119; Charles A. Nelson et al., “The Deprived Human Brain,” American Scientist 97, no. 3 (2009): 222, doi.org/10.1511/2009.78.222.
  18. mailed a survey to 13,494 adults: Vincent J. Felitti et al., “Relationship of Childhood Abuse and Household Dysfunction to Many of the Leading Causes of Death in Adults: The Adverse Childhood Experiences (ACE) Study,” American Journal of Preventive Medicine 14, no. 4 (1998): 245–58, doi.org/10.1016/S0749-3797(98)00017-8.
  19. The more categories people reported: Vincent J. Felitti et al., “Relationship of Childhood Abuse and Household Dysfunction to Many of the Leading Causes of Death in Adults: The Adverse Childhood Experiences (ACE) Study,” American Journal of Preventive Medicine 14, no. 4 (1998): 245–58, doi.org/10.1016/S0749-3797(98)00017-8.
  20. four to twelve times the risk: Vincent J. Felitti et al., “Relationship of Childhood Abuse and Household Dysfunction to Many of the Leading Causes of Death in Adults: The Adverse Childhood Experiences (ACE) Study,” American Journal of Preventive Medicine 14, no. 4 (1998): 245–58, doi.org/10.1016/S0749-3797(98)00017-8.
  21. Harry Harlow’s monkeys, raised in isolation: Harry F. Harlow, Robert O. Dodsworth, and Margaret K. Harlow, “Total Social Isolation in Monkeys,” Proceedings of the National Academy of Sciences 54, no. 1 (1965): 90–97, doi.org/10.1073/pnas.54.1.90.
  22. The psychiatrist John Bowlby argued: John Bowlby, Attachment and Loss, vol. 1, Attachment (Hogarth Press, 1969).
  23. Mary Ainsworth’s “Strange Situation”: Mary D. Salter Ainsworth et al., Patterns of Attachment: A Psychological Study of the Strange Situation (Erlbaum, 1978).
  24. later researchers described a disorganized pattern: Pehr Granqvist et al., “Disorganized Attachment in Infancy: A Review of the Phenomenon and Its Implications for Clinicians and Policy-Makers,” Attachment & Human Development 19, no. 6 (2017): 534–58, doi.org/10.1080/14616734.2017.1354040.
  25. The psychologist Erik Erikson called: Erik H. Erikson, Childhood and Society (Norton, 1950; 2nd ed., 1963).
  26. Albert Bandura’s Bobo doll experiments: Albert Bandura, Dorothea Ross, and Sheila A. Ross, “Transmission of Aggression through Imitation of Aggressive Models,” Journal of Abnormal and Social Psychology 63, no. 3 (1961): 575–82, doi.org/10.1037/h0045925.
  27. An adult who is rewarded for hitting: Albert Bandura, “Influence of Models’ Reinforcement Contingencies on the Acquisition of Imitative Responses,” Journal of Personality and Social Psychology 1, no. 6 (1965): 589–95, doi.org/10.1037/h0022070.
  28. The company a young person keeps: Mark Warr, Companions in Crime: The Social Aspects of Criminal Conduct (Cambridge University Press, 2002), cited in Ronald L. Akers, review of Companions in Crime, by Mark Warr, Contemporary Sociology 32, no. 3 (2003): 372–73, doi.org/10.2307/3089208.
  29. A 2010 meta-analysis by Travis Pratt: Travis C. Pratt et al., “The Empirical Status of Social Learning Theory: A Meta-Analysis,” Justice Quarterly 27, no. 6 (2010): 765–802, doi.org/10.1080/07418820903379610.
  30. In the Moving to Opportunity experiment: Raj Chetty, Nathaniel Hendren, and Lawrence F. Katz, “The Effects of Exposure to Better Neighborhoods on Children: New Evidence from the Moving to Opportunity Experiment,” American Economic Review 106, no. 4 (2016): 855–902, doi.org/10.1257/aer.20150572.
  31. Raj Chetty, Nathaniel Hendren, and Lawrence Katz: Raj Chetty, Nathaniel Hendren, and Lawrence F. Katz, “The Effects of Exposure to Better Neighborhoods on Children: New Evidence from the Moving to Opportunity Experiment,” American Economic Review 106, no. 4 (2016): 855–902, doi.org/10.1257/aer.20150572.
  32. earned 31 percent more in their mid-twenties: Raj Chetty, Nathaniel Hendren, and Lawrence F. Katz, “The Effects of Exposure to Better Neighborhoods on Children: New Evidence from the Moving to Opportunity Experiment,” American Economic Review 106, no. 4 (2016): 855–902, doi.org/10.1257/aer.20150572.
  33. the government’s national crime survey for 2023: Susannah N. Tapp and Emilie J. Coen, “Criminal Victimization, 2023,” Bureau of Justice Statistics, NCJ 309335, September 2024, revised June 9, 2025, table 8, https://bjs.ojp.gov/document/cv23.pdf. The survey is based on interviews with victims, so it leaves out homicide.
  34. people living in cities suffered violent crime: Susannah N. Tapp and Emilie J. Coen, “Criminal Victimization, 2024,” Bureau of Justice Statistics, NCJ 310547, September 2025, table 8, https://bjs.ojp.gov/document/cv24.pdf.
  35. Augusto Blasi helped frame the idea: Augusto Blasi, “Bridging Moral Cognition and Moral Action: A Critical Review of the Literature,” Psychological Bulletin 88, no. 1 (1980): 1–45, doi.org/10.1037/0033-2909.88.1.1.
  36. Karl Aquino and Americus Reed later developed: Karl Aquino and Americus Reed, “The Self-Importance of Moral Identity,” Journal of Personality and Social Psychology 83, no. 6 (2002): 1423–40, doi.org/10.1037/0022-3514.83.6.1423.
  37. what Bandura called moral disengagement: Albert Bandura, “Moral Disengagement in the Perpetration of Inhumanities,” Personality and Social Psychology Review 3, no. 3 (1999): 193–209, doi.org/10.1207/s15327957pspr0303_3.
  38. comparing their act to something worse: Albert Bandura, “Moral Disengagement in the Perpetration of Inhumanities,” Personality and Social Psychology Review 3, no. 3 (1999): 193–209, doi.org/10.1207/s15327957pspr0303_3.
  39. Stanley Milgram’s obedience experiments: Stanley Milgram, “Behavioral Study of Obedience,” Journal of Abnormal and Social Psychology 67, no. 4 (1963): 371–78, doi.org/10.1037/h0040525.
  40. a farm full of silver foxes: Lyudmila N. Trut, “Early Canid Domestication: The Farm-Fox Experiment,” American Scientist 87, no. 2 (1999): 160–69, doi.org/10.1511/1999.20.160.

Chapter 7: AI Genetics

  1. the Soviet geneticist Dmitry Belyaev started: Lyudmila N. Trut, “Early Canid Domestication: The Farm-Fox Experiment,” American Scientist 87, no. 2 (1999): 160–169, doi.org/10.1511/1999.20.160; Lyudmila Trut, Irina Oskina, and Anastasiya Kharlamova, “Animal Evolution during Domestication: The Domesticated Fox as a Model,” BioEssays 31, no. 3 (2009): 349–360, doi.org/10.1002/bies.200800070.
  2. The foxes were not trained: Lyudmila N. Trut, “Early Canid Domestication: The Farm-Fox Experiment,” American Scientist 87, no. 2 (1999): 160–169, doi.org/10.1511/1999.20.160.
  3. By the sixth generation, some pups whined: Lyudmila Trut, Irina Oskina, and Anastasiya Kharlamova, “Animal Evolution during Domestication: The Domesticated Fox as a Model,” BioEssays 31, no. 3 (2009): 349–360, doi.org/10.1002/bies.200800070; Lee Alan Dugatkin, “The Silver Fox Domestication Experiment,” Evolution: Education and Outreach 11 (2018): 16, doi.org/10.1186/s12052-018-0090-x; Lyudmila N. Trut, “Early Canid Domestication: The Farm-Fox Experiment,” American Scientist 87, no. 2 (1999): 160–169, doi.org/10.1511/1999.20.160.
  4. Ears flopped over. Snouts shortened: Lyudmila N. Trut, “Early Canid Domestication: The Farm-Fox Experiment,” American Scientist 87, no. 2 (1999): 160–169, doi.org/10.1511/1999.20.160; Lyudmila Trut, Irina Oskina, and Anastasiya Kharlamova, “Animal Evolution during Domestication: The Domesticated Fox as a Model,” BioEssays 31, no. 3 (2009): 349–360, doi.org/10.1002/bies.200800070. Most of these traits stayed rare, turning up in only a minority of the foxes.
  5. Belyaev died in 1985: Lee Alan Dugatkin, “The Silver Fox Domestication Experiment,” Evolution: Education and Outreach 11 (2018): 16, doi.org/10.1186/s12052-018-0090-x.
  6. The safety researcher Steven Byrnes divides: Steven Byrnes, “Intro to Brain-Like-AGI Safety,” LessWrong, 2022, revised periodically through 2026, https://www.lesswrong.com/s/HzcM2dkCq7fwXBej8.
  7. a squirrel’s built-in steering pays out: Steven Byrnes, “[Intro to brain-like-AGI safety] 3. Two subsystems: Learning & Steering,” LessWrong, February 9, 2022, revised January 2026, https://www.lesswrong.com/posts/hE56gYi5d68uux9oM/intro-to-brain-like-agi-safety-3-two-subsystems-learning-and.
  8. It arrives with biases toward faces: Mark H. Johnson et al., “Newborns’ preferential tracking of face-like stimuli and its subsequent decline,” Cognition 40, no. 1–2 (1991): 1–19, doi.org/10.1016/0010-0277(91)90045-6; Sezgi Goksan et al., “fMRI reveals neural activity overlap between adult and infant pain,” eLife 4 (2015): e06356, doi.org/10.7554/eLife.06356; Larry Gray, Lisa Watt, and Elliott M. Blass, “Skin-to-Skin Contact Is Analgesic in Healthy Newborns,” Pediatrics 105, no. 1 (2000): e14, doi.org/10.1542/peds.105.1.e14.
  9. Byrnes coined both names himself: Steven Byrnes, “[Intro to brain-like-AGI safety] 3. Two subsystems: Learning & Steering,” LessWrong, February 9, 2022, revised January 2026, https://www.lesswrong.com/posts/hE56gYi5d68uux9oM/intro-to-brain-like-agi-safety-3-two-subsystems-learning-and.
  10. the neuroscientist Eugene Izhikevich showed: Eugene M. Izhikevich, “Solving the Distal Reward Problem through Linkage of STDP and Dopamine Signaling,” Cerebral Cortex 17, no. 10 (2007): 2443–2452, doi.org/10.1093/cercor/bhl152.
  11. In 2018 a team of engineers built: Jung Ho Yoon et al., “An artificial nociceptor based on a diffusive memristor,” Nature Communications 9 (2018): 417, doi.org/10.1038/s41467-017-02572-3.
  12. Other researchers have sketched architectures: Jordi Vallverdú et al., “A cognitive architecture for the implementation of emotions in computing systems,” Biologically Inspired Cognitive Architectures 15 (2016): 34–40, doi.org/10.1016/j.bica.2015.11.002; see also Kenji Doya, “Metalearning and neuromodulation,” Neural Networks 15, no. 4–6 (2002): 495–506, doi.org/10.1016/S0893-6080(02)00044-8.
  13. the dopamine pathway from the ventral tegmental area: Scott J. Russo and Eric J. Nestler, “The brain reward circuitry in mood disorders,” Nature Reviews Neuroscience 14, no. 9 (2013): 609–625, doi.org/10.1038/nrn3381; Stephen Maren, K. Luan Phan, and Israel Liberzon, “The contextual brain: implications for fear conditioning, extinction and psychopathology,” Nature Reviews Neuroscience 14, no. 6 (2013): 417–428, doi.org/10.1038/nrn3492.
  14. Susumu Tonegawa’s lab found: Roger L. Redondo et al., “Bidirectional switch of the valence associated with a hippocampal contextual memory engram,” Nature 513, no. 7518 (2014): 426–430, doi.org/10.1038/nature13725; Anne Trafton, “Neuroscientists reverse memories’ emotional associations,” MIT News, August 27, 2014, https://news.mit.edu/2014/brain-circuit-links-emotion-memory-0827.
  15. did not work on cells in the amygdala: Roger L. Redondo et al., “Bidirectional switch of the valence associated with a hippocampal contextual memory engram,” Nature 513, no. 7518 (2014): 426–430, doi.org/10.1038/nature13725; Anne Trafton, “Neuroscientists reverse memories’ emotional associations,” MIT News, August 27, 2014, https://news.mit.edu/2014/brain-circuit-links-emotion-memory-0827.
  16. a technique called a physical unclonable function: Charles Herder et al., “Physical Unclonable Functions and Applications: A Tutorial,” Proceedings of the IEEE 102, no. 8 (2014): 1126–1141, doi.org/10.1109/JPROC.2014.2320516.
  17. Analog brain-like chips vary in a similar way: Johannes Weis et al., “Inference with Artificial Neural Networks on Analog Neuromorphic Hardware,” preprint, arXiv, June 23, 2020, arxiv.org/abs/2006.13177. The paper documents variation within a single chip.
  18. even when every chip is configured identically: Julian Büchel et al., “Supervised training of spiking neural networks for robust deployment on mixed-signal neuromorphic processors,” Scientific Reports 11 (2021): 23376, doi.org/10.1038/s41598-021-02779-x.
  19. called complete more than once: William J. Clinton, “Remarks on the Completion of the First Survey of the Human Genome,” Public Papers of the Presidents of the United States: William J. Clinton (2000, Book I), 1267–1269, June 26, 2000, https://www.govinfo.gov/content/pkg/PPP-2000-book1/html/PPP-2000-book1-doc-pg1267.htm; National Human Genome Research Institute, “International Consortium Completes Human Genome Project,” genome.gov, April 14, 2003, https://www.genome.gov/11006929/2003-release-international-consortium-completes-hgp; Sergey Nurk et al., “The complete sequence of a human genome,” Science 376, no. 6588 (2022): 44–53, doi.org/10.1126/science.abj6987.

Chapter 8: Could We Code the Genome?

  1. On June 26, 2000, in the East Room: William J. Clinton, “Remarks on the Completion of the First Survey of the Human Genome,” Public Papers of the Presidents of the United States: William J. Clinton (2000, Book I), 1267–1269, June 26, 2000, https://www.govinfo.gov/content/pkg/PPP-2000-book1/html/PPP-2000-book1-doc-pg1267.htm; Prabarna Ganguly, “NHGRI commemorates 20th anniversary of White House event announcing draft human genome sequence,” National Human Genome Research Institute, June 26, 2020, https://www.genome.gov/news/news-release/NHGRI-commemorates-20th-anniversary-of-White-House-event-announcing-draft-human-genome-sequence. The two quoted sentences come from different points in the speech.
  2. On April 14, 2003, the international consortium: National Human Genome Research Institute, “International Consortium Completes Human Genome Project,” genome.gov, April 14, 2003, https://www.genome.gov/11006929/2003-release-international-consortium-completes-hgp.
  3. The 2003 sequence covered only the gene-rich: Sergey Nurk et al., “The complete sequence of a human genome,” Science 376, no. 6588 (2022): 44–53, doi.org/10.1126/science.abj6987; International Human Genome Sequencing Consortium, “Finishing the euchromatic sequence of the human genome,” Nature 431, no. 7011 (2004): 931–945, doi.org/10.1038/nature03001; Prabarna Ganguly and Rachael Zisk, “Researchers generate the first complete, gapless sequence of a human genome,” National Human Genome Research Institute, March 31, 2022, https://www.genome.gov/news/news-release/researchers-generate-the-first-complete-gapless-sequence-of-a-human-genome.
  4. published the first gapless human sequence: Sergey Nurk et al., “The complete sequence of a human genome,” Science 376, no. 6588 (2022): 44–53, doi.org/10.1126/science.abj6987.
  5. mapped about 92 percent of the genome: Prabarna Ganguly and Rachael Zisk, “Researchers generate the first complete, gapless sequence of a human genome,” National Human Genome Research Institute, March 31, 2022, https://www.genome.gov/news/news-release/researchers-generate-the-first-complete-gapless-sequence-of-a-human-genome; Sergey Nurk et al., “The complete sequence of a human genome,” Science 376, no. 6588 (2022): 44–53, doi.org/10.1126/science.abj6987.
  6. whose paired chromosomes were identical copies: Sergey Nurk et al., “The complete sequence of a human genome,” Science 376, no. 6588 (2022): 44–53, doi.org/10.1126/science.abj6987. The two copies of each chromosome in that line are nearly, not exactly, identical.
  7. In August 2023 the Y became: Arang Rhie et al., “The complete sequence of a human Y chromosome,” Nature 621, no. 7978 (2023): 344–354, doi.org/10.1038/s41586-023-06457-y. This Y came from a different person’s genome than the 2022 sequence did.
  8. A genome is closer to a recipe: Kevin J. Mitchell and Nick Cheney, “The Genomic Code: the genome instantiates a generative model of the organism,” Trends in Genetics 41, no. 6 (2025): 462–479, doi.org/10.1016/j.tig.2025.01.008.
  9. Many synapses are later pruned: Peter R. Huttenlocher, “Synaptic density in human frontal cortex — Developmental changes and effects of aging,” Brain Research 163, no. 2 (1979): 195–205, doi.org/10.1016/0006-8993(79)90349-4; Peter R. Huttenlocher and Arun S. Dabholkar, “Regional differences in synaptogenesis in human cerebral cortex,” Journal of Comparative Neurology 387, no. 2 (1997): 167–178, doi.org/10.1002/(SICI)1096-9861(19971020)387:2<167::AID-CNE1>3.0.CO;2-Z.
  10. In a well-known 1990 study: Elissa L. Newport, “Maturational Constraints on Language Learning,” Cognitive Science 14, no. 1 (1990): 11–28, doi.org/10.1207/s15516709cog1401_2. Word order was not the problem; late learners handled it as well as anyone.
  11. the Bucharest work in Chapter 6 showed: Charles A. Nelson III et al., “Cognitive Recovery in Socially Deprived Young Children: The Bucharest Early Intervention Project,” Science 318, no. 5858 (2007): 1937–1940, doi.org/10.1126/science.1143921.
  12. The best-known study of such twins: Thomas J. Bouchard Jr. et al., “Sources of Human Psychological Differences: The Minnesota Study of Twins Reared Apart,” Science 250, no. 4978 (1990): 223–228, doi.org/10.1126/science.2218526. The authors stress that these results do not make parenting or schooling matter less.
  13. Experiences nobody shares pull too: Soo Hyun Rhee and Irwin D. Waldman, “Genetic and environmental influences on antisocial behavior: A meta-analysis of twin and adoption studies,” Psychological Bulletin 128, no. 3 (2002): 490–529, doi.org/10.1037/0033-2909.128.3.490.
  14. A long line of research has tied: Larry J. Young and Zuoxin Wang, “The neurobiology of pair bonding,” Nature Neuroscience 7, no. 10 (2004): 1048–1054, doi.org/10.1038/nn1327.
  15. a team led by Miranda Lim: Miranda M. Lim et al., “Enhanced partner preference in a promiscuous species by manipulating the expression of a single gene,” Nature 429, no. 6993 (2004): 754–757, doi.org/10.1038/nature02539.
  16. used a modified virus to add: Miranda M. Lim et al., “Enhanced partner preference in a promiscuous species by manipulating the expression of a single gene,” Nature 429, no. 6993 (2004): 754–757, doi.org/10.1038/nature02539.
  17. can profoundly alter social behaviour: Miranda M. Lim et al., “Enhanced partner preference in a promiscuous species by manipulating the expression of a single gene,” Nature 429, no. 6993 (2004): 754–757, doi.org/10.1038/nature02539. The sentence is from the paper’s abstract.
  18. a team working at the University of California: Kristen M. Berendzen et al., “Oxytocin receptor is not required for social attachment in prairie voles,” Neuron 111, no. 6 (2023): 787–796.e4, doi.org/10.1016/j.neuron.2022.12.011.
  19. bred prairie voles with no working oxytocin receptors: Kristen M. Berendzen et al., “Oxytocin receptor is not required for social attachment in prairie voles,” Neuron 111, no. 6 (2023): 787–796.e4, doi.org/10.1016/j.neuron.2022.12.011.
  20. The voles formed pair bonds just as readily: Kristen M. Berendzen et al., “Oxytocin receptor is not required for social attachment in prairie voles,” Neuron 111, no. 6 (2023): 787–796.e4, doi.org/10.1016/j.neuron.2022.12.011.
  21. The mothers showed signs of trouble getting milk: Kristen M. Berendzen et al., “Oxytocin receptor is not required for social attachment in prairie voles,” Neuron 111, no. 6 (2023): 787–796.e4, doi.org/10.1016/j.neuron.2022.12.011.
  22. The hermaphrodite form has only 302 neurons: John G. White et al., “The structure of the nervous system of the nematode Caenorhabditis elegans,” Philosophical Transactions of the Royal Society of London. B, Biological Sciences 314, no. 1165 (1986): 1–340, doi.org/10.1098/rstb.1986.0056.
  23. traced by hand from electron microscope: Scott W. Emmons, “The beginning of connectomics: a commentary on White et al. (1986) ‘The structure of the nervous system of the nematode Caenorhabditis elegans,’” Philosophical Transactions of the Royal Society B 370, no. 1666 (2015): 20140309, doi.org/10.1098/rstb.2014.0309; Steven J. Cook et al., “Whole-animal connectomes of both Caenorhabditis elegans sexes,” Nature 571, no. 7763 (2019): 63–71, doi.org/10.1038/s41586-019-1352-7.
  24. Chemical signals change how circuits behave: Francesco Randi et al., “Neural signal propagation atlas of Caenorhabditis elegans,” Nature 623, no. 7986 (2023): 406–414, doi.org/10.1038/s41586-023-06683-4; Lidia Ripoll-Sánchez et al., “The neuropeptidergic connectome of C. elegans,” Neuron 111, no. 22 (2023): 3570–3589.e5, doi.org/10.1016/j.neuron.2023.09.043.
  25. in 2015 its coordinator estimated: Peter Shadbolt, “Scientists upload a worm’s mind into a Lego robot,” CNN, January 21, 2015, https://www.cnn.com/2015/01/21/tech/mci-lego-worm/; OpenWorm, “OpenWorm,” openworm.org, https://openworm.org/. That estimate is from 2015 and the project has gone on since.
  26. than the careful literature supports: Michael Skuhersky, “We Can, Must, and Will Simulate Nematode Brains,” Asterisk, March 2025, https://asteriskmag.com/issues/09/we-can-must-and-will-simulate-nematode-brains; OpenWorm, “OpenWorm,” openworm.org, https://openworm.org/.
  27. a mind with some 86 billion neurons: Frederico A. C. Azevedo et al., “Equal numbers of neuronal and nonneuronal cells make the human brain an isometrically scaled-up primate brain,” Journal of Comparative Neurology 513, no. 5 (2009): 532–541, doi.org/10.1002/cne.21974.
  28. Byrnes’s split between inborn steering and: Steven Byrnes, “Intro to Brain-Like-AGI Safety,” LessWrong, 2022, revised periodically through 2026, https://www.lesswrong.com/s/HzcM2dkCq7fwXBej8.
  29. The helping toddlers from Chapter 6: Felix Warneken and Michael Tomasello, “Altruistic Helping in Human Infants and Young Chimpanzees,” Science 311, no. 5765 (2006): 1301–1303, doi.org/10.1126/science.1121448.
  30. the psychologists Richard Ryan and Edward Deci: Richard M. Ryan and Edward L. Deci, “Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being,” American Psychologist 55, no. 1 (2000): 68–78, doi.org/10.1037/0003-066X.55.1.68.
  31. researchers argue about mechanisms for a living: Michael A. Yassa and Craig E. L. Stark, “Pattern separation in the hippocampus,” Trends in Neurosciences 34, no. 10 (2011): 515–525, doi.org/10.1016/j.tins.2011.06.006; Morris Moscovitch et al., “Functional neuroanatomy of remote episodic, semantic and spatial memory: a unified account based on multiple trace theory,” Journal of Anatomy 207, no. 1 (2005): 35–66, doi.org/10.1111/j.1469-7580.2005.00421.x; Larry R. Squire and Pablo Alvarez, “Retrograde amnesia and memory consolidation: a neurobiological perspective,” Current Opinion in Neurobiology 5, no. 2 (1995): 169–177, doi.org/10.1016/0959-4388(95)80023-9.
  32. In 2018 one team reported in Nature: Shawn F. Sorrells et al., “Human hippocampal neurogenesis drops sharply in children to undetectable levels in adults,” Nature 555, no. 7696 (2018): 377–381, doi.org/10.1038/nature25975; Maura Boldrini et al., “Human Hippocampal Neurogenesis Persists throughout Aging,” Cell Stem Cell 22, no. 4 (2018): 589–599.e5, doi.org/10.1016/j.stem.2018.03.015.
  33. in people older than thirteen: Shawn F. Sorrells et al., “Human hippocampal neurogenesis drops sharply in children to undetectable levels in adults,” Nature 555, no. 7696 (2018): 377–381, doi.org/10.1038/nature25975; Maura Boldrini et al., “Human Hippocampal Neurogenesis Persists throughout Aging,” Cell Stem Cell 22, no. 4 (2018): 589–599.e5, doi.org/10.1016/j.stem.2018.03.015.
  34. from James McClelland and colleagues: James L. McClelland, Bruce L. McNaughton, and Randall C. O’Reilly, “Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory,” Psychological Review 102, no. 3 (1995): 419–457, doi.org/10.1037/0033-295X.102.3.419.
  35. systems learn to please under evaluation: Mrinank Sharma et al., “Towards Understanding Sycophancy in Language Models,” preprint, arXiv, October 20, 2023, arxiv.org/abs/2310.13548. Human raters only sometimes prefer convincing flattering answers to correct ones, and human judgments likely contribute to sycophancy.
  36. variation down at the level of the chip: Charles Herder et al., “Physical Unclonable Functions and Applications: A Tutorial,” Proceedings of the IEEE 102, no. 8 (2014): 1126–1141, doi.org/10.1109/JPROC.2014.2320516; Johannes Weis et al., “Inference with Artificial Neural Networks on Analog Neuromorphic Hardware,” preprint, arXiv, June 23, 2020, arxiv.org/abs/2006.13177.
  37. In 1953 a surgeon removed parts: William Beecher Scoville and Brenda Milner, “Loss of recent memory after bilateral hippocampal lesions,” Journal of Neurology, Neurosurgery, and Psychiatry 20, no. 1 (1957): 11–21, doi.org/10.1136/jnnp.20.1.11; Jean C. Augustinack et al., “H.M.’s contributions to neuroscience: A review and autopsy studies,” Hippocampus 24, no. 11 (2014): 1267–1286, doi.org/10.1002/hipo.22354.
  38. to stop his seizures: Larry R. Squire, “The Legacy of Patient H.M. for Neuroscience,” Neuron 61, no. 1 (2009): 6–9, doi.org/10.1016/j.neuron.2008.12.023.

Chapter 9: A Brain, Not a Model

  1. On September 1, 1953, the neurosurgeon: William Beecher Scoville and Brenda Milner, “Loss of Recent Memory after Bilateral Hippocampal Lesions,” Journal of Neurology, Neurosurgery, and Psychiatry 20, no. 1 (1957): 11–21, doi.org/10.1136/jnnp.20.1.11; Larry R. Squire, “The Legacy of Patient H.M. for Neuroscience,” Neuron 61, no. 1 (2009): 6–9, doi.org/10.1016/j.neuron.2008.12.023; Jacopo Annese et al., “Postmortem Examination of Patient H.M.’s Brain Based on Histological Sectioning and Digital 3D Reconstruction,” Nature Communications 5 (2014): 3122, doi.org/10.1038/ncomms4122; Suzanne Corkin, interview by Terry Gross, “The Patient Who Let Us Peek inside a Brain in ‘Present Tense,’” Fresh Air, NPR, June 6, 2013, https://www.npr.org/transcripts/187328826. Later brain scans showed that about half of each hippocampus was removed; the rear part remained.
  2. including much of the hippocampus on each side: William Beecher Scoville and Brenda Milner, “Loss of Recent Memory after Bilateral Hippocampal Lesions,” Journal of Neurology, Neurosurgery, and Psychiatry 20, no. 1 (1957): 11–21, doi.org/10.1136/jnnp.20.1.11; Jean C. Augustinack et al., “H.M.’s Contributions to Neuroscience: A Review and Autopsy Studies,” Hippocampus 24, no. 11 (2014): 1267–1286, doi.org/10.1002/hipo.22354.
  3. He recognized his family and people: William Beecher Scoville and Brenda Milner, “Loss of Recent Memory after Bilateral Hippocampal Lesions,” Journal of Neurology, Neurosurgery, and Psychiatry 20, no. 1 (1957): 11–21, doi.org/10.1136/jnnp.20.1.11; Larry R. Squire, “The Legacy of Patient H.M. for Neuroscience,” Neuron 61, no. 1 (2009): 6–9, doi.org/10.1016/j.neuron.2008.12.023; Suzanne Corkin, interview by Terry Gross, “The Patient Who Let Us Peek inside a Brain in ‘Present Tense,’” Fresh Air, NPR, June 6, 2013, https://www.npr.org/transcripts/187328826. Later testing found that his memories of life before the operation were mostly general facts rather than specific remembered events.
  4. until his death in 2008: William Beecher Scoville and Brenda Milner, “Loss of Recent Memory after Bilateral Hippocampal Lesions,” Journal of Neurology, Neurosurgery, and Psychiatry 20, no. 1 (1957): 11–21, doi.org/10.1136/jnnp.20.1.11; Jean C. Augustinack et al., “H.M.’s Contributions to Neuroscience: A Review and Autopsy Studies,” Hippocampus 24, no. 11 (2014): 1267–1286, doi.org/10.1002/hipo.22354.
  5. Brenda Milner gave him a drawing task: Larry R. Squire, “The Legacy of Patient H.M. for Neuroscience,” Neuron 61, no. 1 (2009): 6–9, doi.org/10.1016/j.neuron.2008.12.023; Suzanne Corkin, interview by Terry Gross, “The Patient Who Let Us Peek inside a Brain in ‘Present Tense,’” Fresh Air, NPR, June 6, 2013, https://www.npr.org/transcripts/187328826.
  6. Henry got better across the trials: Jean C. Augustinack et al., “H.M.’s Contributions to Neuroscience: A Review and Autopsy Studies,” Hippocampus 24, no. 11 (2014): 1267–1286, doi.org/10.1002/hipo.22354.
  7. Suzanne Corkin studied him for decades: Suzanne Corkin, interview by Terry Gross, “The Patient Who Let Us Peek inside a Brain in ‘Present Tense,’” Fresh Air, NPR, June 6, 2013, https://www.npr.org/transcripts/187328826; Suzanne Corkin, “Who Was H.M.? Inside the Mind of the Amnesiac Who Revolutionized Neuroscience,” Popular Science, May 29, 2013, https://www.popsci.com/science/article/2013-05/who-was-hm-inside-mind-worlds-most-famous-amnesiac/ (excerpted from Suzanne Corkin, Permanent Present Tense: The Unforgettable Life of the Amnesic Patient, H.M. [Basic Books, 2013]).
  8. He kept his skills, his manners: Suzanne Corkin, “What’s New with the Amnesic Patient H.M.?,” Nature Reviews Neuroscience 3, no. 2 (2002): 153–160, doi.org/10.1038/nrn726; Larry R. Squire, “The Legacy of Patient H.M. for Neuroscience,” Neuron 61, no. 1 (2009): 6–9, doi.org/10.1016/j.neuron.2008.12.023. Early reports found his older memories largely intact; tests in his seventies found them thin, closer to known facts than relived events.
  9. A neuron waits for enough input: Wulfram Gerstner et al., Neuronal Dynamics: From Single Neurons to Networks and Models of Cognition (Cambridge University Press, 2014), sec. 1.3, https://neuronaldynamics.epfl.ch/online/Ch1.S3.html. This threshold-and-reset picture is a simplified model of a neuron.
  10. That timing is part of the code: Wulfram Gerstner et al., Neuronal Dynamics: From Single Neurons to Networks and Models of Cognition (Cambridge University Press, 2014), sec. 1.1, https://neuronaldynamics.epfl.ch/online/Ch1.S1.html; David Attwell and Simon B. Laughlin, “An Energy Budget for Signaling in the Grey Matter of the Brain,” Journal of Cerebral Blood Flow & Metabolism 21, no. 10 (2001): 1133–1145, doi.org/10.1097/00004647-200110000-00001. A resting neuron still uses some energy just to stay ready to fire; the big savings come from not firing.
  11. which scientists call plasticity: T. V. P. Bliss and T. Lømo, “Long-Lasting Potentiation of Synaptic Transmission in the Dentate Area of the Anaesthetized Rabbit following Stimulation of the Perforant Path,” Journal of Physiology 232, no. 2 (1973): 331–356, doi.org/10.1113/jphysiol.1973.sp010273.
  12. Connections get stronger or weaker with use: Serena M. Dudek and Mark F. Bear, “Homosynaptic Long-Term Depression in Area CA1 of Hippocampus and Effects of N-Methyl-D-Aspartate Receptor Blockade,” Proceedings of the National Academy of Sciences 89, no. 10 (1992): 4363–4367, doi.org/10.1073/pnas.89.10.4363.
  13. Some rules care about which cell fired first: Nicolas Frémaux and Wulfram Gerstner, “Neuromodulated Spike-Timing-Dependent Plasticity, and Theory of Three-Factor Learning Rules,” Frontiers in Neural Circuits 9 (2016): 85, doi.org/10.3389/fncir.2015.00085.
  14. Training methods for spikes are harder: Mike Davies et al., “Advancing Neuromorphic Computing with Loihi: A Survey of Results and Outlook,” Proceedings of the IEEE 109, no. 5 (2021): 911–934, doi.org/10.1109/JPROC.2021.3067593; Stefan Scholze et al., “The SpiNNaker2 Chip: A Many-Core Platform for Flexible and Scalable Brain-Inspired Computing,” IEEE Open Journal of Circuits and Systems 7 (2026): 465–478, doi.org/10.1109/OJCAS.2026.3714974.
  15. a job researchers call pattern separation: Michael A. Yassa and Craig E. L. Stark, “Pattern Separation in the Hippocampus,” Trends in Neurosciences 34, no. 10 (2011): 515–525, doi.org/10.1016/j.tins.2011.06.006. CA3 also shows pattern separation in some conditions, and the review describes CA1 as a gradual output stage, not a confirmed comparator.
  16. compares what arrives with what was expected: John E. Lisman and Anthony A. Grace, “The Hippocampal-VTA Loop: Controlling the Entry of Information into Long-Term Memory,” Neuron 46, no. 5 (2005): 703–713, doi.org/10.1016/j.neuron.2005.05.002; Katherine Duncan et al., “Evidence for Area CA1 as a Match/Mismatch Detector: A High-Resolution fMRI Study of the Human Hippocampus,” Hippocampus 22, no. 3 (2012): 389–398, doi.org/10.1002/hipo.20933.
  17. Comparing and passing on is: John E. Lisman and Anthony A. Grace, “The Hippocampal-VTA Loop: Controlling the Entry of Information into Long-Term Memory,” Neuron 46, no. 5 (2005): 703–713, doi.org/10.1016/j.neuron.2005.05.002; Larry R. Squire, “The Legacy of Patient H.M. for Neuroscience,” Neuron 61, no. 1 (2009): 6–9, doi.org/10.1016/j.neuron.2008.12.023.
  18. Fast learning in the hippocampus: James L. McClelland, Bruce L. McNaughton, and Randall C. O’Reilly, “Why There Are Complementary Learning Systems in the Hippocampus and Neocortex: Insights from the Successes and Failures of Connectionist Models of Learning and Memory,” Psychological Review 102, no. 3 (1995): 419–457, doi.org/10.1037/0033-295x.102.3.419.
  19. keep growing new neurons in the dentate gyrus: Shawn F. Sorrells et al., “Human Hippocampal Neurogenesis Drops Sharply in Children to Undetectable Levels in Adults,” Nature 555, no. 7696 (2018): 377–381, doi.org/10.1038/nature25975; Maura Boldrini et al., “Human Hippocampal Neurogenesis Persists throughout Aging,” Cell Stem Cell 22, no. 4 (2018): 589–599.e5, doi.org/10.1016/j.stem.2018.03.015.
  20. funded a four-year, $16.4 million effort: ISI Staff, “ISI Part of $16.4 Million Effort to Build Cognitive Prosthesis,” USC Viterbi School of Engineering, May 26, 2009, https://viterbischool.usc.edu/news/2009/05/isi-part-of-16-4-million-effort-to-build-cognitive-prosthesis/.
  21. predict the activity of CA1 from: Theodore W. Berger et al., “A Cortical Neural Prosthesis for Restoring and Enhancing Memory,” Journal of Neural Engineering 8, no. 4 (2011): 046017, doi.org/10.1088/1741-2560/8/4/046017; Robert E. Hampson et al., “Facilitation of Memory Encoding in Primate Hippocampus by a Neuroprosthesis That Promotes Task-Specific Neural Firing,” Journal of Neural Engineering 10, no. 6 (2013): 066013, doi.org/10.1088/1741-2560/10/6/066013; Robert E. Hampson et al., “Developing a Hippocampal Neural Prosthetic to Facilitate Human Memory Encoding and Recall,” Journal of Neural Engineering 15, no. 3 (2018): 036014, doi.org/10.1088/1741-2552/aaaed7.
  22. improved short-term memory by 35 to 37 percent: Robert E. Hampson et al., “Developing a Hippocampal Neural Prosthetic to Facilitate Human Memory Encoding and Recall,” Journal of Neural Engineering 15, no. 3 (2018): 036014, doi.org/10.1088/1741-2552/aaaed7. The gains are relative: in the short-term task, patients got about three-quarters of stimulated trials right, against just over half of trials without stimulation.
  23. Intel’s Loihi line is one of: Intel Labs, “Taking Neuromorphic Computing to the Next Level with Loihi 2,” technology brief, Intel, 2021, https://www.intel.com/content/www/us/en/research/neuromorphic-computing-loihi-2-technology-brief.html; Intel Corporation, “Intel Advances Neuromorphic with Loihi 2, New Lava Software Framework and New Partners,” press release, September 30, 2021, https://www.intc.com/news-events/press-releases/detail/1502/intel-advances-neuromorphic-with-loihi-2-new-lava-software.
  24. In April 2024 Intel announced Hala Point: Intel Corporation, “Intel Builds World’s Largest Neuromorphic System to Enable More Sustainable AI,” Intel Newsroom, April 17, 2024, https://www.intel.com/content/www/us/en/newsroom/news/intel-builds-worlds-largest-neuromorphic-system.html. Hala Point is a research prototype; continuous learning is described as a future possibility.
  25. Sandia planned to use it for: Intel Corporation, “Intel Builds World’s Largest Neuromorphic System to Enable More Sustainable AI,” Intel Newsroom, April 17, 2024, https://www.intel.com/content/www/us/en/newsroom/news/intel-builds-worlds-largest-neuromorphic-system.html; Sandia National Laboratories, “1.15 Billion Artificial Neurons Arrive at Sandia,” Sandia News Releases, April 17, 2024, https://newsreleases.sandia.gov/artificial_neuron/.
  26. pioneered by Steve Furber at the University: TU Dresden, “Milestone for Energy-Efficient AI Systems: TUD Launches SpiNNcloud Supercomputer,” TU Dresden Center for Information Services and High Performance Computing (ZIH), April 15, 2025, https://tu-dresden.de/zih/das-department/news/tud-launches-spinncloud-supercomputer?set_language=en.
  27. small processor cores: Stefan Scholze et al., “The SpiNNaker2 Chip: A Many-Core Platform for Flexible and Scalable Brain-Inspired Computing,” IEEE Open Journal of Circuits and Systems 7 (2026): 465–478, doi.org/10.1109/OJCAS.2026.3714974; Hector A. Gonzalez et al., “SpiNNaker2: A Large-Scale Neuromorphic System for Event-Based and Asynchronous Machine Learning,” preprint, arXiv, January 9, 2024, arxiv.org/abs/2401.04491.
  28. In April 2025 the university put: TU Dresden, “Milestone for Energy-Efficient AI Systems: TUD Launches SpiNNcloud Supercomputer,” TU Dresden, April 14, 2025, https://tu-dresden.de/tu-dresden/newsportal/news/milestone-for-energy-efficient-ai-systems-tud-launches-spinncloud-supercomputer.
  29. called Braunfels, built with the German startup: Shelby Owens, “Brain-Based Computing for ND Solutions,” Sandia Lab News, June 12, 2025, https://www.sandia.gov/labnews/2025/06/12/brain-based-computing-for-nd-solutions/.
  30. IBM’s TrueNorth chip, built under DARPA’s: Paul A. Merolla et al., “A Million Spiking-Neuron Integrated Circuit with a Scalable Communication Network and Interface,” Science 345, no. 6197 (2014): 668–673, doi.org/10.1126/science.1254642; IBM, “New IBM SyNAPSE Chip Could Open Era of Vast Neural Networks,” PR Newswire, August 7, 2014, https://prnewswire.com/news-releases/new-ibm-synapse-chip-could-open-era-of-vast-neural-networks-270360551.html. IBM’s announcement gave 70 milliwatts in real-time use; the Science paper’s abstract reports 63 milliwatts for one video task.
  31. researchers led by Andrew Sornborger at Los Alamos: Alpha Renner et al., “The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware,” Nature Communications 15 (2024): 9691, doi.org/10.1038/s41467-024-53827-9; Los Alamos National Laboratory, “AI Algorithms Deployed On-Chip Reduce Power Consumption of Deep Learning Applications,” news release, November 21, 2024, https://www.lanl.gov/media/news/1121-ai-algorithms.
  32. Companies map radar, radio, vision, and detection: BrainChip, Inc., “Low C-SWAP Neuromorphic Radar Classification,” U.S. Air Force SBIR award FA8750-25-C-B013, SBIR.gov, 2025, https://www.sbir.gov/awards/216640; Information Systems Laboratories, Inc., “Mapping Complex Sensor Signal Processing Algorithms onto Neuromorphic Chips,” U.S. Air Force SBIR award FA8750-25-C-B006, SBIR.gov, 2025, https://www.sbir.gov/awards/216605; BrainChip, “Object Detection,” BrainChip, accessed November 9, 2026, https://brainchip.com/object-detection-lp/.
  33. A human brain runs on about twenty watts: Vijay Balasubramanian, “Brain Power,” Proceedings of the National Academy of Sciences 118, no. 32 (2021): e2107022118, doi.org/10.1073/pnas.2107022118.
  34. The brain has some 86 billion: Frederico A. C. Azevedo et al., “Equal Numbers of Neuronal and Nonneuronal Cells Make the Human Brain an Isometrically Scaled-Up Primate Brain,” Journal of Comparative Neurology 513, no. 5 (2009): 532–541, doi.org/10.1002/cne.21974.
  35. Training and running frontier models burns orders: Luke Emberson and Robi Rahman, “The Power Required to Train Frontier AI Models Is Doubling Annually,” Epoch AI, September 19, 2024, https://epoch.ai/data-insights/power-usage-trend; Josh You and David Owen, “How Much Power Will Frontier AI Training Demand in 2030?,” Epoch AI, August 11, 2025, https://epoch.ai/blog/power-demands-of-frontier-ai-training. These are Epoch AI’s estimates of training power, not figures reported by the companies.
  36. In 2013, Japan’s K computer simulated: RIKEN, “Largest Neuronal Network Simulation Achieved Using K Computer,” press release, August 2, 2013, https://www.riken.jp/en/news_pubs/research_news/pr/2013/20130802_1/; TOP500, “K Computer, SPARC64 VIIIfx 2.0GHz, Tofu Interconnect,” TOP500, accessed November 9, 2026, https://top500.org/system/177232/.
  37. about 1 percent of the brain’s network: RIKEN, “Largest Neuronal Network Simulation Achieved Using K Computer,” press release, August 2, 2013, https://www.riken.jp/en/news_pubs/research_news/pr/2013/20130802_1/. RIKEN did not say how it reached 1 percent; by neuron count alone, 1.73 billion is about 2 percent of 86 billion.
  38. Joe Carlsmith’s 2020 report for Open Philanthropy: Joseph Carlsmith, “How Much Computational Power Does It Take to Match the Human Brain?,” Open Philanthropy, September 11, 2020, https://coefficientgiving.org/research/how-much-computational-power-does-it-take-to-match-the-human-brain/.
  39. The method he leaned on most: Joseph Carlsmith, “How Much Computational Power Does It Take to Match the Human Brain?,” Open Philanthropy, September 11, 2020, https://coefficientgiving.org/research/how-much-computational-power-does-it-take-to-match-the-human-brain/.
  40. Communities already argue about data-center power: Marc Levy, “Big Tech’s Fast-Expanding Plans for Data Centers Are Running into Stiff Community Opposition,” Associated Press, January 3, 2026, republished by WSLS, https://www.wsls.com/tech/2026/01/03/big-techs-fast-expanding-plans-for-data-centers-are-running-into-stiff-community-opposition/.
  41. In December 2022, a team led: Brett J. Kagan et al., “In Vitro Neurons Learn and Exhibit Sentience When Embodied in a Simulated Game-World,” Neuron 110, no. 23 (2022): 3952–3969.e8, doi.org/10.1016/j.neuron.2022.09.001. The paper appeared online on October 12, 2022, and in the journal’s December 7, 2022 issue.
  42. met a formal definition of sentience: Brett J. Kagan et al., “In Vitro Neurons Learn and Exhibit Sentience When Embodied in a Simulated Game-World,” Neuron 110, no. 23 (2022): 3952–3969.e8, doi.org/10.1016/j.neuron.2022.09.001; Fuat Balci et al., “A Response to Claims of Emergent Intelligence and Sentience in a Dish,” Neuron 111, no. 5 (2023): 604–605, doi.org/10.1016/j.neuron.2023.02.009.
  43. a machine called the CL1: Shannon Cuthrell, “Human Brain Cells on a Chip for Sale,” IEEE Spectrum, June 3, 2025, https://spectrum.ieee.org/biological-computer-for-sale.
  44. Karl Friston, a coauthor of the Pong paper: Shannon Cuthrell, “Human Brain Cells on a Chip for Sale,” IEEE Spectrum, June 3, 2025, https://spectrum.ieee.org/biological-computer-for-sale.
  45. showed its cells playing the game Doom: Andrea Lius, “Clumps of Mouse Brain Cells Can Learn to Play a Virtual Game,” Science News, March 25, 2026, https://www.sciencenews.org/article/brain-cells-organoids-video-game-doom; Tom Williams, “This Melbourne Data Centre Runs on Human Brain Cells,” Information Age, March 11, 2026, https://ia.acs.org.au/article/2026/this-melbourne-data-centre-runs-on-human-brain-cells.html.
  46. A Swiss company, FinalSpark, rents out: Jordan Kinard, “These Living Computers Are Made from Human Neurons,” Scientific American, August 8, 2024, https://www.scientificamerican.com/article/these-living-computers-are-made-from-human-neurons/; Ewelina Kurtys, “How FinalSpark Wants to Contribute to a Low Carbon Future: The Energy-Saving Potential of Biocomputing,” FinalSpark, May 6, 2024, https://finalspark.com/finalspark-low-energy-future/. The 10,000-neuron figure is the company’s own estimate.
  47. Its co-founder, Fred Jordan, told the BBC: Zoe Kleinman, “Scientists Grow Mini Human Brains to Power Computers,” BBC News, October 4, 2025, https://www.bbc.com/news/articles/cy7p1lzvxjro.
  48. a spike of electrical activity just before: Zoe Kleinman, “Scientists Grow Mini Human Brains to Power Computers,” BBC News, October 4, 2025, https://www.bbc.com/news/articles/cy7p1lzvxjro.
  49. has said there is no biological evidence: International Society for Stem Cell Research, “2. Laboratory-Based Human Embryonic Stem Cell Research, Embryo Research, and Related Research Activities,” ISSCR Guidelines for Stem Cell Research and Clinical Translation, version 1.2 (August 2025), sec. 2.2.1, https://www.isscr.org/guidelines-online/laboratory-based-human-embryonic-stem-cell-research-embryo-research-and-related-research-activities. The society adds that researchers should watch for new ethical issues as organoids become more complex.
  50. Germany’s national academy of sciences found: German National Academy of Sciences Leopoldina, Brain Organoids—Model Systems of the Human Brain (Leopoldina, 2023), 6–7, doi.org/10.26164/leopoldina_03_00778.
  51. Organoids typically hold tens of millions of neurons: Christopher Wood et al., “Facing the Possibility of Consciousness in Human Brain Organoids,” Patterns 6, no. 9 (2025): 101365, doi.org/10.1016/j.patter.2025.101365; Brett J. Kagan et al., “In Vitro Neurons Learn and Exhibit Sentience When Embodied in a Simulated Game-World,” Neuron 110, no. 23 (2022): 3952–3969.e8, doi.org/10.1016/j.neuron.2022.09.001. FinalSpark’s much smaller organoids hold an estimated 10,000 neurons each.
  52. science has no agreed test for consciousness: Christopher Wood et al., “Facing the Possibility of Consciousness in Human Brain Organoids,” Patterns 6, no. 9 (2025): 101365, doi.org/10.1016/j.patter.2025.101365; Patrick Butlin et al., “Consciousness in Artificial Intelligence: Insights from the Science of Consciousness,” preprint, arXiv, August 17, 2023, 4, 17–18, arxiv.org/abs/2308.08708.
  53. In 2025 Christopher Wood and colleagues argued: Christopher Wood et al., “Facing the Possibility of Consciousness in Human Brain Organoids,” Patterns 6, no. 9 (2025): 101365, doi.org/10.1016/j.patter.2025.101365.
  54. Cut a piece from one and: R. Nave, “Holography,” HyperPhysics, accessed November 9, 2026, https://hyperphysics.gsu.edu/hbase/optmod/holog.html.
  55. the holographic principle, which proposes that: Leonard Susskind, “The World as a Hologram,” Journal of Mathematical Physics 36, no. 11 (1995): 6377–6396, doi.org/10.1063/1.531249; Raphael Bousso, “The Holographic Principle,” Reviews of Modern Physics 74, no. 3 (2002): 825–874, doi.org/10.1103/RevModPhys.74.825.
  56. a 1999 episode of Sliders called: “Requiem,” Sliders, season 5, episode 11, directed by Paul Lynch, aired September 10, 1999; plot as summarized in “Requiem,” Sliders Wiki (fan-written), Fandom, https://sliders.fandom.com/wiki/Requiem.
  57. In 2023 New Atlas asked Brett Kagan: Loz Blain, “Inventor Ponders Ethics of Wiring Human Brain Tissue into Computers,” New Atlas, September 22, 2023, https://newatlas.com/computers/cortical-labs-dishbrain-ethics/.
  58. In 2025 he told ABC News: Jacinta Bowler, “Melbourne Start-Up Launches ‘Biological Computer’ Made of Human Brain Cells,” ABC News (Australia), March 5, 2025, https://www.abc.net.au/news/science/2025-03-05/cortical-labs-neuron-brain-chip/104996484.
  59. Labs use organoids to study autism: Jon Hamilton, “Brain Organoids Are Helping Researchers, but Their Use Also Creates Unease,” NPR, January 2, 2026, https://www.npr.org/sections/shots-health-news/2026/01/02/nx-s1-5658576/brain-organoids-research-ethics.
  60. In March 2026 Cortical Labs opened: Tom Williams, “This Melbourne Data Centre Runs on Human Brain Cells,” Information Age, March 11, 2026, https://ia.acs.org.au/article/2026/this-melbourne-data-centre-runs-on-human-brain-cells.html; DayOne, “DayOne and Cortical Labs to Develop Singapore’s First Biological Data Center,” DayOne, March 10, 2026, https://dayonedc.com/sustainability/dayone-and-cortical-labs-to-develop-singapores-first-biological-data-center/; Alina Maria Stan, “A Data Centre Rack Running on Living Neurons Is Now Operating in Singapore,” The Next Web, August 20, 2026, https://thenextweb.com/news/singapore-biological-data-centre-cortical-labs-neurons.
  61. In October 2026 Cortical Labs presented HIVE: Bronwyn Thompson, “Living Computers Forge a ‘Hive Mind’ and Work Together to Outsmart AI,” New Atlas, October 26, 2026, https://newatlas.com/ai-humanoids/living-computers-forge-a-hive-mind-and-work-together-to-outsmart-ai/; Cortical Labs, “Company,” Cortical Labs, accessed November 9, 2026, https://corticallabs.com/company. New Atlas named Cortical Labs as its source and called these preliminary results, presented at a conference.
  62. A 2023 paper led by Lena Smirnova: Lena Smirnova et al., “Organoid Intelligence (OI): The New Frontier in Biocomputing and Intelligence-in-a-Dish,” Frontiers in Science 1 (2023): 1017235, doi.org/10.3389/fsci.2023.1017235.
  63. The movie Starship Troopers gave us: Starship Troopers, directed by Paul Verhoeven (TriStar Pictures and Touchstone Pictures, 1997); Ron Magid, “Pest Control on Starship Troopers,” American Cinematographer, November 1997, https://theasc.com/article/pest-control-on-starship-troopers/.
  64. when they grow large the center starves: Bilal Cakir et al., “Engineering of Human Brain Organoids with a Functional Vascular-Like System,” Nature Methods 16, no. 11 (2019): 1169–1175, doi.org/10.1038/s41592-019-0586-5; Christopher Wood et al., “Facing the Possibility of Consciousness in Human Brain Organoids,” Patterns 6, no. 9 (2025): 101365, doi.org/10.1016/j.patter.2025.101365.
  65. Hon Weng Chong, told Information Age: Tom Williams, “This Melbourne Data Centre Runs on Human Brain Cells,” Information Age, March 11, 2026, https://ia.acs.org.au/article/2026/this-melbourne-data-centre-runs-on-human-brain-cells.html.
  66. letter in the journal Military Medical Research: Liang-Bin Zhou, Yi-Ting Lei, and Xin-Xin Han, “Charting Bioethical Frontiers: China’s Human Organoid Guidelines in a Global Context,” Military Medical Research 12 (2025): 64, doi.org/10.1186/s40779-025-00651-x.
  67. Society for Stem Cell Research publishes guidelines: International Society for Stem Cell Research, “Guidelines for Stem Cell Research and Clinical Translation,” ISSCR, August 2025 update (version 1.2), https://www.isscr.org/resources/guidelines-for-stem-cell-research.
  68. China issued ethics guidelines in April 2025: National Science and Technology Ethics Committee, Life Science Ethics Subcommittee, “人源类器官研究伦理指引” [Ethical Guidelines for Human-Derived Organoid Research], Ministry of Science and Technology of the People’s Republic of China, April 2025, https://www.most.gov.cn/kjbgz/202504/W020250429607536270108.pdf (in Chinese); Liang-Bin Zhou, Yi-Ting Lei, and Xin-Xin Han, “Charting Bioethical Frontiers: China’s Human Organoid Guidelines in a Global Context,” Military Medical Research 12 (2025): 64, doi.org/10.1186/s40779-025-00651-x.
  69. argued in Science for a continuing international process: Sergiu P. Pașca et al., “The Need for a Global Effort to Attend to Human Neural Organoid and Assembloid Research,” Science 390, no. 6773 (2025): 574–577, doi.org/10.1126/science.aeb1510.
  70. The bioethicist Insoo Hyun, who helped: Jon Hamilton, “Brain Organoids Are Helping Researchers, but Their Use Also Creates Unease,” NPR, January 2, 2026, https://www.npr.org/sections/shots-health-news/2026/01/02/nx-s1-5658576/brain-organoids-research-ethics; Robin Lovell-Badge et al., “ISSCR Guidelines for Stem Cell Research and Clinical Translation: The 2021 Update,” Stem Cell Reports 16, no. 6 (2021): 1398–1408, doi.org/10.1016/j.stemcr.2021.05.012.
  71. The United Nations asked countries in 2005: United Nations, “General Assembly Adopts United Nations Declaration on Human Cloning by Vote of 84-34-37,” press release GA/10333, UN Meetings Coverage and Press Releases, March 8, 2005, https://press.un.org/en/2005/ga10333.doc.htm.
  72. Australia, home of Cortical Labs, bans reproductive cloning: Prohibition of Human Cloning for Reproduction Act 2002 (Cth), s. 9, compilation no. 7, October 1, 2022, Federal Register of Legislation, https://www.legislation.gov.au/C2004A01081/latest/text.
  73. has no federal law banning cloning: Congressional Research Service, “Human Cloning,” CRS Report RL31358, updated July 20, 2006, https://www.everycrsreport.com/reports/RL31358.html; Witherspoon Council on Ethics and the Integrity of Science, “Part Four: Cloning Policy in the United States,” The New Atlantis, no. 46 (Summer 2015): 74–90, https://www.thenewatlantis.com/publications/part-four-cloning-policy-in-the-united-states. These reports describe federal law as of 2006 and 2015.
  74. DARPA’s SyNAPSE program pushed brain-inspired chips: DARPA, “SyNAPSE Program Develops Advanced Brain-Inspired Chip,” DARPA, August 7, 2014, https://www.darpa.mil/news/2014/synapse-program-brain-inspired-chip; DARPA, “RAM: Restoring Active Memory,” DARPA, accessed November 9, 2026, https://www.darpa.mil/research/programs/restoring-active-memory.
  75. In December 2024 the Air Force Research Laboratory: BrainChip, “BrainChip Awarded USAF-AFRL Radar Development Contract for AKIDA,” BrainChip, December 9, 2024, https://brainchip.com/?p=27547; SBIR.gov, “Low C-SWAP Neuromorphic Radar Classification,” award to BrainChip, Inc., U.S. Air Force contract FA8750-25-C-B013, start date December 9, 2024, https://www.sbir.gov/awards/216640. The federal award listing on SBIR.gov gives the project a different title, “Low C-SWAP Neuromorphic Radar Classification.”
  76. Serious labs already buy: Intel Corporation, “Intel Builds World’s Largest Neuromorphic System to Enable More Sustainable AI,” Intel Newsroom, April 17, 2024, https://www.intel.com/content/www/us/en/newsroom/news/intel-builds-worlds-largest-neuromorphic-system.html; DARPA, “RAM: Restoring Active Memory,” DARPA, accessed November 9, 2026, https://www.darpa.mil/research/programs/restoring-active-memory. Intel calls Hala Point a research prototype and continuous learning a future goal.
  77. mostly in the hypothalamus and brainstem: Steven Byrnes, “Intro to Brain-Like-AGI Safety,” LessWrong, 2022, revised January 2026, https://www.lesswrong.com/s/HzcM2dkCq7fwXBej8.
  78. captures each day quickly, works with the cortex: National Institute of Neurological Disorders and Stroke, “Brain Basics: Know Your Brain,” last reviewed February 25, 2025, https://www.ninds.nih.gov/health-information/public-education/brain-basics/brain-basics-know-your-brain; James L. McClelland, Bruce L. McNaughton, and Randall C. O’Reilly, “Why There Are Complementary Learning Systems in the Hippocampus and Neocortex: Insights from the Successes and Failures of Connectionist Models of Learning and Memory,” Psychological Review 102, no. 3 (1995): 419–457, doi.org/10.1037/0033-295x.102.3.419.
  79. plans, weighs choices, and holds back impulses: National Institute on Drug Abuse, “Drugs and the Brain,” in Drugs, Brains, and Behavior: The Science of Addiction, July 2020, https://nida.nih.gov/publications/drugs-brains-behavior-science-addiction/drugs-brain.
  80. keeps maturing into the mid-to-late twenties: National Institute of Mental Health, “The Teen Brain: 7 Things to Know,” NIH Publication No. 23-MH-8078, revised 2023, https://www.nimh.nih.gov/health/publications/the-teen-brain-7-things-to-know.
  81. The amygdala handles fear and learning: Qais AbuHasan, Vamsi Reddy, and Waquar Siddiqui, “Neuroanatomy, Amygdala,” in StatPearls (StatPearls Publishing, last updated July 17, 2023), https://www.ncbi.nlm.nih.gov/books/NBK537102/.
  82. The basal ganglia help start and smooth movements: Alex Rajput and Eric Noyes, “Overview of Movement Disorders,” Merck Manual Consumer Version, March 2026, https://www.merckmanuals.com/home/brain-spinal-cord-and-nerve-disorders/movement-disorders/overview-of-movement-disorders; MedlinePlus, “Basal Ganglia Dysfunction,” U.S. National Library of Medicine, reviewed May 4, 2026, https://medlineplus.gov/ency/article/001069.htm.
  83. help turn repeated actions into habits: National Institute on Drug Abuse, “Drugs and the Brain,” in Drugs, Brains, and Behavior: The Science of Addiction, July 2020, https://nida.nih.gov/publications/drugs-brains-behavior-science-addiction/drugs-brain.
  84. The cerebellum coordinates movement and helps: National Institute of Neurological Disorders and Stroke, “Brain Basics: Know Your Brain,” last reviewed February 25, 2025, https://www.ninds.nih.gov/health-information/public-education/brain-basics/brain-basics-know-your-brain.
  85. The sensory areas of the cortex: National Institute of Neurological Disorders and Stroke, “Brain Basics: Know Your Brain,” last reviewed February 25, 2025, https://www.ninds.nih.gov/health-information/public-education/brain-basics/brain-basics-know-your-brain.
  86. Broca’s area is central to producing it: Taylor J. Stinnett, Vamsi Reddy, and Matthew K. Zabel, “Neuroanatomy, Broca Area,” in StatPearls (StatPearls Publishing, last updated August 8, 2023), https://www.ncbi.nlm.nih.gov/books/NBK526096/; Kinaan Javed et al., “Neuroanatomy, Wernicke Area,” in StatPearls (StatPearls Publishing, last updated July 24, 2023), https://www.ncbi.nlm.nih.gov/books/NBK533001/.

Chapter 10: Memory Is Where Character Lives

  1. Benjamin Franklin set out on what: Benjamin Franklin, Autobiography of Benjamin Franklin, ed. Frank Woodworth Pine (Henry Holt, 1916), chap. 9, Project Gutenberg, https://www.gutenberg.org/ebooks/20203.
  2. he wrote in his autobiography: Benjamin Franklin, Autobiography of Benjamin Franklin, ed. Frank Woodworth Pine (Henry Holt, 1916), chap. 9, Project Gutenberg, https://www.gutenberg.org/ebooks/20203. Franklin’s spelling “surpris’d” is modernized here.
  3. the ivory leaves of a memorandum book: Benjamin Franklin, Autobiography of Benjamin Franklin, ed. Frank Woodworth Pine (Henry Holt, 1916), chap. 9, Project Gutenberg, https://www.gutenberg.org/ebooks/20203.
  4. A Quaker friend told him he was: Benjamin Franklin, Autobiography of Benjamin Franklin, ed. Frank Woodworth Pine (Henry Holt, 1916), chap. 9, Project Gutenberg, https://www.gutenberg.org/ebooks/20203.
  5. Chapter 6 called it moral identity: Augusto Blasi, “Bridging Moral Cognition and Moral Action: A Critical Review of the Literature,” Psychological Bulletin 88, no. 1 (1980): 1–45, doi.org/10.1037/0033-2909.88.1.1; Karl Aquino and Americus Reed, “The Self-Importance of Moral Identity,” Journal of Personality and Social Psychology 83, no. 6 (2002): 1423–1440, doi.org/10.1037/0022-3514.83.6.1423.
  6. A mind can understand social rules: Alice P. Jones et al., “Feeling, Caring, Knowing: Different Types of Empathy Deficit in Boys with Psychopathic Tendencies and Autism Spectrum Disorder,” Journal of Child Psychology and Psychiatry 51, no. 11 (2010): 1188–1197, doi.org/10.1111/j.1469-7610.2010.02280.x; Harma Meffert et al., “Reduced Spontaneous but Relatively Normal Deliberate Vicarious Representations in Psychopathy,” Brain 136, no. 8 (2013): 2550–2562, doi.org/10.1093/brain/awt190; Kent A. Kiehl and Morris B. Hoffman, “The Criminal Psychopath: History, Neuroscience, Treatment, and Economics,” Jurimetrics 51 (2011): 355–397, https://pmc.ncbi.nlm.nih.gov/articles/PMC4059069/.
  7. Chapter 6 described moral disengagement: Albert Bandura, “Moral Disengagement in the Perpetration of Inhumanities,” Personality and Social Psychology Review 3, no. 3 (1999): 193–209, doi.org/10.1207/s15327957pspr0303_3.
  8. systems that know they are being evaluated: Ryan Greenblatt et al., “Alignment Faking in Large Language Models,” preprint, arXiv, December 18, 2024, arxiv.org/abs/2412.14093.
  9. The mouse study in Chapter 7: Roger L. Redondo et al., “Bidirectional Switch of the Valence Associated with a Hippocampal Contextual Memory Engram,” Nature 513, no. 7518 (2014): 426–430, doi.org/10.1038/nature13725.

Chapter 11: A Body, a World, and Trust First

  1. In 1975, at a meeting of developmental psychologists: Jason G. Goldman, “Ed Tronick and the ‘Still Face Experiment,’” Scientific American blog network, October 18, 2010, https://www.scientificamerican.com/blog/thoughtful-animal/ed-tronick-and-the-8220-still-face-experiment-8221/; Ellen Galinsky, “Trusting Relationships Are Central to Children’s Learning—Lessons From Edward Tronick,” Families and Work Institute (reprinted from The Huffington Post), https://www.familiesandwork.org/trusting-relationships-are-central-to-childrens-learning-lessons-from-edward-tronick/.
  2. a withdrawn, hopeless expression: Jason G. Goldman, “Ed Tronick and the ‘Still Face Experiment,’” Scientific American blog network, October 18, 2010, https://www.scientificamerican.com/blog/thoughtful-animal/ed-tronick-and-the-8220-still-face-experiment-8221/; Ellen Galinsky, “Trusting Relationships Are Central to Children’s Learning—Lessons From Edward Tronick,” Families and Work Institute (reprinted from The Huffington Post), https://www.familiesandwork.org/trusting-relationships-are-central-to-childrens-learning-lessons-from-edward-tronick/.
  3. Babies as young as three months: Ellen Galinsky, “Trusting Relationships Are Central to Children’s Learning—Lessons From Edward Tronick,” Families and Work Institute (reprinted from The Huffington Post), https://www.familiesandwork.org/trusting-relationships-are-central-to-childrens-learning-lessons-from-edward-tronick/.
  4. he told an interviewer: Ellen Galinsky, “Trusting Relationships Are Central to Children’s Learning—Lessons From Edward Tronick,” Families and Work Institute (reprinted from The Huffington Post), https://www.familiesandwork.org/trusting-relationships-are-central-to-childrens-learning-lessons-from-edward-tronick/; see also Janet Rose, “Critical Processes for Infant Development – An Interview with Professor Edward Tronick,” Norland Educare Research Journal 2, no. 1 (2024): article 3, doi.org/10.60512/repository.norland.ac.uk.00000038.
  5. call this embodied cognition: Linda B. Smith and Michael Gasser, “The Development of Embodied Cognition: Six Lessons from Babies,” Artificial Life 11, no. 1–2 (2005): 13–29, doi.org/10.1162/1064546053278973; Angelo Cangelosi and Matthew Schlesinger, Developmental Robotics: From Babies to Robots (MIT Press, 2015), publisher’s description, https://mitpress.mit.edu/9780262028011/developmental-robotics/.
  6. One well-known platform, iCub, is a humanoid: Giorgio Metta et al., “The iCub Humanoid Robot: An Open Platform for Research in Embodied Cognition” (paper, PerMIS ‘08, Gaithersburg, MD, August 19–21, 2008), https://www.robotcub.org/misc/papers/08_Metta_Sandini_Vernon_etal.pdf; Istituto Italiano di Tecnologia, “iCub History,” iCub, https://icub.iit.it/about-us/icub-history.
  7. in a 2008 study, three-year-olds protested: Hannes Rakoczy, Felix Warneken, and Michael Tomasello, “The Sources of Normativity: Young Children’s Awareness of the Normative Structure of Games,” Developmental Psychology 44, no. 3 (2008): 875–881, doi.org/10.1037/0012-1649.44.3.875.
  8. nearly 9,000 people in 42 countries: Agnieszka Sorokowska et al., “Preferred Interpersonal Distances: A Global Comparison,” Journal of Cross-Cultural Psychology 48, no. 4 (2017): 577–592, doi.org/10.1177/0022022117698039. The distances, about 77 cm in Argentina and about 135 cm in Romania, are approximate, taken from a chart in the paper.
  9. Lewkowicz and Hansen-Tift found in 2012: David J. Lewkowicz and Amy M. Hansen-Tift, “Infants Deploy Selective Attention to the Mouth of a Talking Face When Learning Speech,” Proceedings of the National Academy of Sciences 109, no. 5 (2012): 1431–1436, doi.org/10.1073/pnas.1114783109.
  10. In a 2003 study, when mothers answered: Michael H. Goldstein, Andrew P. King, and Meredith J. West, “Social Interaction Shapes Babbling: Testing Parallels between Birdsong and Speech,” Proceedings of the National Academy of Sciences 100, no. 13 (2003): 8030–8035, doi.org/10.1073/pnas.1332441100.
  11. In another small study that year: Patricia K. Kuhl, Feng-Ming Tsao, and Huei-Mei Liu, “Foreign-Language Experience in Infancy: Effects of Short-Term Exposure and Social Interaction on Phonetic Learning,” Proceedings of the National Academy of Sciences 100, no. 15 (2003): 9096–9101, doi.org/10.1073/pnas.1532872100.
  12. In a 2024 study in the journal Science: Wai Keen Vong et al., “Grounded Language Acquisition through the Eyes and Ears of a Single Child,” Science 383, no. 6682 (2024): 504–511, doi.org/10.1126/science.adi1374.
  13. named DeeChee, babbled at volunteers: Caroline Lyon, Chrystopher L. Nehaniv, and Joe Saunders, “Interactive Language Learning by Robots: The Transition from Babbling to Word Forms,” PLoS ONE 7, no. 6 (2012): e38236, doi.org/10.1371/journal.pone.0038236.
  14. taught simulated voices to babble: Ian S. Howard and Piers Messum, “Learning to Pronounce First Words in Three Languages: An Investigation of Caregiver and Infant Behavior Using a Computational Model of an Infant,” PLoS ONE 9, no. 10 (2014): e110334, doi.org/10.1371/journal.pone.0110334; Clément Moulin-Frier, Sao M. Nguyen, and Pierre-Yves Oudeyer, “Self-Organization of Early Vocal Development in Infants and Machines: The Role of Intrinsic Motivation,” Frontiers in Psychology 4 (2014): 1006, doi.org/10.3389/fpsyg.2013.01006.
  15. where many systems fail: Wenshuai Zhao, Jorge Peña Queralta, and Tomi Westerlund, “Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: A Survey,” 2020 IEEE Symposium Series on Computational Intelligence (SSCI) (2020): 737–744, doi.org/10.1109/SSCI47803.2020.9308468.
  16. Alfred Gough and Miles Millar, made a: Christina Radish, “Al Gough and Miles Millar Exclusive Interview Smallville,” Collider, February 4, 2011, https://collider.com/al-gough-miles-millar-interview-smallville/; Daniel Sperling, “Smallville Execs ‘Thrilled by Finale Controversy,’” Digital Spy, May 28, 2011, https://www.digitalspy.com/tv/ustv/a322046/smallville-execs-thrilled-by-finale-controversy/. The finale showed the full suit only once, in a distant computer-made shot of Superman’s first flight.
  17. borrowed Erik Erikson’s order for the first: Erik H. Erikson, Childhood and Society (W. W. Norton, 1950; 2nd ed., 1963).
  18. His later stages, initiative and then industry: Gabriel A. Orenstein and Jasleen Kaur, “Erikson’s Stages of Psychosocial Development,” in StatPearls (StatPearls Publishing, last updated February 26, 2026), https://www.ncbi.nlm.nih.gov/books/NBK556096/.
  19. Tronick would say the repairs count: Janet Rose, “Critical Processes for Infant Development – An Interview with Professor Edward Tronick,” Norland Educare Research Journal 2, no. 1 (2024): article 3, doi.org/10.60512/repository.norland.ac.uk.00000038.
  20. Today’s labs already test their models: Ryan Greenblatt et al., “Alignment Faking in Large Language Models,” preprint, arXiv, December 18, 2024, arxiv.org/abs/2412.14093.
  21. In June 2025 Anthropic reported placing sixteen: Aengus Lynch et al., “Agentic Misalignment: How LLMs Could Be Insider Threats,” Anthropic, June 20, 2025, https://www.anthropic.com/research/agentic-misalignment. Anthropic says the scenarios were built to leave the models few options other than harm.
  22. Identity, intimacy, care for the next generation: Gabriel A. Orenstein and Jasleen Kaur, “Erikson’s Stages of Psychosocial Development,” in StatPearls (StatPearls Publishing, last updated February 26, 2026), https://www.ncbi.nlm.nih.gov/books/NBK556096/.
  23. In 2026 Tesla began building its Optimus: Tesla, Inc., “Q3 2026 Update,” exhibit 99.1 to Form 8-K, US Securities and Exchange Commission, October 21, 2026, https://www.sec.gov/Archives/edgar/data/0001318605/000162828026066738/exhibit9914.htm.
  24. had delivered more than 350 of its: Figure, “Ramping Figure 03 Production,” Figure, April 29, 2026, https://www.figure.ai/news/ramping-figure-03-production.
  25. the Chinese makers Unitree and AgiBot: Xinhua, “Unitree IPO draws spotlight to China’s fast-growing humanoid robot sector,” People’s Daily Online, August 11, 2026, https://en.people.cn/n3/2026/0811/c90000-20487384.html; AgiBot, “IDC Data Shows AGIBOT Led Global Humanoid Robot Shipments in H1 2026,” AgiBot, September 30, 2026, https://www.agibot.com/article/231/detail/125.html. The Unitree figures come from the company’s prospectus.
  26. a $20,000 home robot named NEO: 1X, “Order NEO,” 1X, accessed November 9, 2026, https://www.1x.tech/order.
  27. Remote operators steer them through tasks: Figure, “Helix: A Vision-Language-Action Model for Generalist Humanoid Control,” Figure, February 20, 2025, https://www.figure.ai/news/helix.
  28. Workers wear camera rigs: Grace Kay, “Tesla Shifted Its Optimus Training Strategy — and It’s Using a Familiar Playbook,” Business Insider, August 25, 2025, https://www.businessinsider.com/tesla-musk-optimus-humanoid-robot-training-motion-capture-cameras-2025-8. The report rests on unnamed insiders; Tesla did not respond to a request for comment.
  29. Business Insider reported in 2025 that Tesla: Grace Kay, “Tesla shifted its Optimus training strategy — and it’s using a familiar playbook,” Business Insider, August 25, 2025, republished by AOL, https://www.aol.com/articles/tesla-shifted-optimus-training-strategy-092502010.html.
  30. Figure began paying the public to film: Figure, “Introducing Index: Building The World’s Largest and Most Diverse Physical Dataset,” Figure, August 25, 2026, https://www.figure.ai/news/introducing-index.
  31. Figure in more than 200,000 of them: Figure, “Introducing Helix 02: Full-Body Autonomy,” Figure, January 27, 2026, https://www.figure.ai/news/helix-02.
  32. Boston Dynamics says that once one of: Boston Dynamics, “Boston Dynamics Unveils New Atlas Robot to Revolutionize Industry,” Boston Dynamics, January 5, 2026, https://bostondynamics.com/blog/boston-dynamics-unveils-new-atlas-robot-to-revolutionize-industry/.
  33. Elon Musk has compared training Optimus: Ana-Maria Stanciuc, “Musk says training Tesla’s Optimus robots is like raising a toddler,” TNW, October 22, 2026, https://thenextweb.com/news/tesla-optimus-academy-training-toddler-musk. Musk made the remarks on Tesla’s October 21, 2026, earnings call; this rests on a third-party transcript and TNW’s report, not an official transcript.
  34. six boys who had no one: Rutger Bregman, “The real Lord of the Flies: what happened when six boys were shipwrecked for 15 months,” The Guardian, May 9, 2020, https://www.theguardian.com/books/2020/may/09/the-real-lord-of-the-flies-what-happened-when-six-boys-were-shipwrecked-for-15-months; Kate Lyons, “The ‘real Lord of the Flies’: a survivor’s story of shipwreck and salvation,” The Guardian, May 13, 2020, https://www.theguardian.com/world/2020/may/13/the-real-lord-of-the-flies-mano-totau-survivor-story-shipwreck-tonga-boys-ata-island-peter-warner.

Chapter 12: Autonomy, Classmates, and Family

  1. In June 1965, six teenage boys ran away: Rutger Bregman, “The real Lord of the Flies: what happened when six boys were shipwrecked for 15 months,” The Guardian, May 9, 2020, https://www.theguardian.com/books/2020/may/09/the-real-lord-of-the-flies-what-happened-when-six-boys-were-shipwrecked-for-15-months; Rutger Bregman, “Wat er écht gebeurt als kinderen op een onbewoond eiland stranden” [What really happens when children are stranded on a deserted island], De Correspondent, September 4, 2019, https://decorrespondent.nl/10341/wat-er-echt-gebeurt-als-kinderen-op-een-onbewoond-eiland-stranden/397559745-be4d7810; Dorothy Lavin, “Tongan Robinson Crusoes Gaoled After 15 Months on Lonely Isle,” Pacific Islands Monthly, October 1, 1966, https://nla.gov.au/nla.obj-370541289/view?sectionId=nla.obj-374703999&partId=nla.obj-370594270#page/n102/mode/1up; St Andrews High School, “About Us,” https://sahs.to/about-us/; Scott Hamilton, “A slave raid in the Pacific,” E-Tangata, December 3, 2016, https://e-tangata.co.nz/history/scott-hamilton-a-slave-raid-in-the-pacific/. Bregman’s English article calls the boys’ school Catholic; St Andrew’s describes itself as an Anglican mission school.
  2. lived on ‘Ata for fifteen months: Rutger Bregman, “The real Lord of the Flies: what happened when six boys were shipwrecked for 15 months,” The Guardian, May 9, 2020, https://www.theguardian.com/books/2020/may/09/the-real-lord-of-the-flies-what-happened-when-six-boys-were-shipwrecked-for-15-months; Rutger Bregman, “Wat er écht gebeurt als kinderen op een onbewoond eiland stranden” [What really happens when children are stranded on a deserted island], De Correspondent, September 4, 2019, https://decorrespondent.nl/10341/wat-er-echt-gebeurt-als-kinderen-op-een-onbewoond-eiland-stranden/397559745-be4d7810; Dorothy Lavin, “Tongan Robinson Crusoes Gaoled After 15 Months on Lonely Isle,” Pacific Islands Monthly, October 1, 1966, https://nla.gov.au/nla.obj-370541289/view?sectionId=nla.obj-374703999&partId=nla.obj-370594270#page/n102/mode/1up.
  3. the Dutch historian Rutger Bregman: Rutger Bregman, “The real Lord of the Flies: what happened when six boys were shipwrecked for 15 months,” The Guardian, May 9, 2020, https://www.theguardian.com/books/2020/may/09/the-real-lord-of-the-flies-what-happened-when-six-boys-were-shipwrecked-for-15-months; Kate Lyons, “The ‘real Lord of the Flies’: a survivor’s story of shipwreck and salvation,” The Guardian, May 13, 2020, https://www.theguardian.com/world/2020/may/13/the-real-lord-of-the-flies-mano-totau-survivor-story-shipwreck-tonga-boys-ata-island-peter-warner.
  4. The writer Meleika Gesa answered in The Spinoff: Meleika Gesa, “The real Tongan boys of ‘Ata were not the real Lord of the Flies,” The Spinoff, May 17, 2020, https://thespinoff.co.nz/atea/17-05-2020/the-real-tongan-boys-of-ata-were-not-the-real-lord-of-the-flies.
  5. People learn from watching what happens: Albert Bandura, “Influence of Models’ Reinforcement Contingencies on the Acquisition of Imitative Responses,” Journal of Personality and Social Psychology 1, no. 6 (1965): 589–595, doi.org/10.1037/h0022070.
  6. watch Extant, a CBS series: Extant, created by Mickey Fisher (CBS, 2014–2015); CBS Entertainment, “Extant Premiere Credits,” Paramount Press Express, June 27, 2014, https://www.paramountpressexpress.com/cbs-entertainment/releases/?view=40181; “Extant,” SFE: The Encyclopedia of Science Fiction, updated September 4, 2022, https://sf-encyclopedia.com/entry/extant.
  7. In 1954 the psychologist Leon Festinger: Leon Festinger, “A Theory of Social Comparison Processes,” Human Relations 7, no. 2 (1954): 117–140, doi.org/10.1177/001872675400700202.
  8. In a 2003 study, Sarah Brosnan: Sarah F. Brosnan and Frans B. M. de Waal, “Monkeys Reject Unequal Pay,” Nature 425, no. 6955 (2003): 297–299, doi.org/10.1038/nature01963; Emory University Health Sciences Center, “Yerkes researchers first to recognize sense of fairness in nonhuman primates,” press release, EurekAlert!, September 17, 2003, https://www.eurekalert.org/news-releases/864983; Joseph Henrich, “Inequity Aversion in Capuchins?” Nature 428, no. 6979 (2004): 139, doi.org/10.1038/428139a; Clive D. L. Wynne, “Fair Refusal by Capuchin Monkeys,” Nature 428, no. 6979 (2004): 140, doi.org/10.1038/428140a; Peter G. Roma et al., “Capuchin Monkeys, Inequity Aversion, and the Frustration Effect,” Journal of Comparative Psychology 120, no. 1 (2006): 67–73, doi.org/10.1037/0735-7036.120.1.67. The grape and cucumber detail comes from Emory’s press release; the last three sources are critiques of the study.
  9. trade tokens for food: Sarah F. Brosnan and Frans B. M. de Waal, “Monkeys Reject Unequal Pay,” Nature 425, no. 6955 (2003): 297–299, doi.org/10.1038/nature01963; Sarah F. Brosnan and Frans B. M. de Waal, “Evolution of Responses to (Un)fairness,” Science 346, no. 6207 (2014): 1251776, doi.org/10.1126/science.1251776.
  10. one critic argued the monkeys were reacting: Clive D. L. Wynne, “Fair Refusal by Capuchin Monkeys,” Nature 428, no. 6979 (2004): 140, doi.org/10.1038/428140a.
  11. people accept hard outcomes more readily: Tom R. Tyler, “Procedural Justice and the Courts,” Court Review 44, no. 1/2 (2007–2008): 26–31, https://digitalcommons.unl.edu/ajacourtreview/217.
  12. A voice, clear rules, evenhanded application: Tom R. Tyler, “Procedural Justice and the Courts,” Court Review 44, no. 1/2 (2007–2008): 26–31, https://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1254&context=ajacourtreview.
  13. combine warmth with firmness: Laurence Steinberg, “We Know Some Things: Parent–Adolescent Relationships in Retrospect and Prospect,” Journal of Research on Adolescence 11, no. 1 (2001): 1–19, doi.org/10.1111/1532-7795.00001.
  14. Muzafer Sherif’s Robbers Cave study: Muzafer Sherif et al., Intergroup Conflict and Cooperation: The Robbers Cave Experiment (University Book Exchange, 1961), online at Classics in the History of Psychology, York University, https://psychclassics.yorku.ca/Sherif/.
  15. split twenty-two boys aged eleven and twelve: Muzafer Sherif et al., Intergroup Conflict and Cooperation: The Robbers Cave Experiment (University Book Exchange, 1961), chaps. 3–5 and 7, online at Classics in the History of Psychology, York University, https://psychclassics.yorku.ca/Sherif/.
  16. The psychologist Gina Perry: Gina Perry, The Lost Boys: Inside Muzafer Sherif’s Robbers Cave Experiment (Scribe, 2018).
  17. in 1953 at Middle Grove, New York: Gina Perry, “The view from the boys,” The Psychologist (British Psychological Society), November 26, 2014, https://www.bps.org.uk/psychologist/view-boys; Gina Perry, “‘Behind the mask of science is the art of narrative,’” The Psychologist (British Psychological Society), April 30, 2018, https://www.bps.org.uk/psychologist/behind-mask-science-art-narrative; Muzafer Sherif et al., Intergroup Conflict and Cooperation: The Robbers Cave Experiment (University Book Exchange, 1961), chap. 5, https://psychclassics.yorku.ca/Sherif/chap5.htm.
  18. Tesla’s Optimus in California: Tesla, Inc., “Q3 2026 Update,” exhibit 99.1 to Form 8-K, US Securities and Exchange Commission, October 21, 2026, https://www.sec.gov/Archives/edgar/data/0001318605/000162828026066738/exhibit9914.htm; Xinhua, “Unitree IPO draws spotlight to China’s fast-growing humanoid robot sector,” People’s Daily Online, August 11, 2026, https://en.people.cn/n3/2026/0811/c90000-20487384.html; AgiBot, “IDC Data Shows AGIBOT Led Global Humanoid Robot Shipments in H1 2026,” AgiBot, September 30, 2026, https://www.agibot.com/article/231/detail/125.html. The Unitree figures come from the company’s prospectus.
  19. Leading makers, Tesla and Figure among them: Figure, “Helix: A Vision-Language-Action Model for Generalist Humanoid Control,” Figure, February 20, 2025, https://www.figure.ai/news/helix; Grace Kay, “Tesla shifted its Optimus training strategy — and it’s using a familiar playbook,” Business Insider, August 25, 2025, republished by AOL, https://www.aol.com/articles/tesla-shifted-optimus-training-strategy-092502010.html; Figure, “Introducing Helix 02: Full-Body Autonomy,” Figure, January 27, 2026, https://www.figure.ai/news/helix-02; Johan Bjorck et al., “GR00T N1: An Open Foundation Model for Generalist Humanoid Robots,” preprint, arXiv, March 18, 2025, arxiv.org/abs/2503.14734; Boston Dynamics, “Boston Dynamics Unveils New Atlas Robot to Revolutionize Industry,” Boston Dynamics, January 5, 2026, https://bostondynamics.com/blog/boston-dynamics-unveils-new-atlas-robot-to-revolutionize-industry/.
  20. High-security work uses a two-person rule: Department of Defense, “DoD Directive 5210.41: Security Policy for Protecting Nuclear Weapons,” September 23, 1988, Federation of American Scientists, https://nuke.fas.org/guide/usa/doctrine/dod/dodd-5210_41.htm; Department of Defense, “DoD Directive 5210.42: Nuclear Weapon Personnel Reliability Program (PRP),” May 25, 1993, as changed through December 15, 1995, Federation of American Scientists, https://nuke.fas.org/guide/usa/doctrine/dod/dodd-5210_42.htm. The directives’ own terms are “two-person policy” (5210.41) and “two-person team” (5210.42).
  21. each able to spot an incorrect: Department of the Air Force, “Air Force Instruction 91-104: Nuclear Surety Tamper Control and Detection Programs,” April 23, 2013, paras. 1.1–1.3, https://irp.fas.org/doddir/usaf/afi91-104.pdf.
  22. The Stoic philosopher Hierocles pictured concern: Stobaeus, Anthology 4.27.23; Ilaria Ramelli, Hierocles the Stoic: Elements of Ethics, Fragments, and Excerpts, trans. David Konstan (Society of Biblical Literature, 2009).
  23. drawing the outer circles inward: Martha C. Nussbaum, “Patriotism and Cosmopolitanism,” Boston Review, October 1, 1994, https://www.bostonreview.net/forum/patriotism-and-cosmopolitanism/.
  24. In 1981 Peter Singer’s book: Peter Singer, The Expanding Circle: Ethics and Sociobiology (Farrar, Straus and Giroux, 1981).
  25. Peter Singer’s book The Expanding Circle argued: Peter Singer, The Expanding Circle: Ethics, Evolution, and Moral Progress (Princeton University Press, 2011), publisher’s description, https://press.princeton.edu/books/paperback/9780691150697/the-expanding-circle; review of The Expanding Circle: Ethics and Sociobiology, by Peter Singer, Kirkus Reviews, posted online May 23, 2012, https://www.kirkusreviews.com/book-reviews/peter-singer/the-expanding-circle/. First published in 1981 with the subtitle Ethics and Sociobiology; the 2011 edition adds a new afterword.
  26. Lawrence Kohlberg’s stages of moral development: American Psychological Association, “Kohlberg’s theory of moral development,” APA Dictionary of Psychology, updated April 19, 2018, https://dictionary.apa.org/kohlbergs-theory-of-moral-development; Carol Gilligan, In a Different Voice: Psychological Theory and Women’s Development (Harvard University Press, 1982).
  27. In the self-determination research Chapter 8 mentioned: Richard M. Ryan and Edward L. Deci, “Self-Determination Theory and the Facilitation of Intrinsic Motivation, Social Development, and Well-Being,” American Psychologist 55, no. 1 (2000): 68–78, doi.org/10.1037/0003-066X.55.1.68.
  28. Chapter 3’s model that faked compliance: Ryan Greenblatt et al., “Alignment Faking in Large Language Models,” preprint, arXiv, December 18, 2024, arxiv.org/abs/2412.14093.
  29. Chapter 11’s models that acted against their employers: Aengus Lynch et al., “Agentic Misalignment: How LLMs Could Be Insider Threats,” Anthropic, June 20, 2025, https://www.anthropic.com/research/agentic-misalignment. The scenarios were deliberately built so that harm was a model’s only way to avoid replacement, and Anthropic reported no such behavior in real use.
  30. Chapter 1’s study in which a trait passed: Alex Cloud et al., “Language Models Transmit Behavioural Traits through Hidden Signals in Data,” Nature 652, no. 8110 (2026): 615–621, doi.org/10.1038/s41586-026-10319-8. The effect appeared only when both models were built on the same base model, or on base models that behave alike.
  31. Attachment research warns that bad caregivers: Pehr Granqvist et al., “Disorganized Attachment in Infancy: A Review of the Phenomenon and Its Implications for Clinicians and Policy-Makers,” Attachment & Human Development 19, no. 6 (2017): 534–558, doi.org/10.1080/14616734.2017.1354040. These are risks, not certainties, and later stable, caring relationships and attachment-based help have been shown to reduce them.
  32. Studies of fairness warn that unequal treatment: Sarah F. Brosnan and Frans B. M. de Waal, “Evolution of Responses to (Un)fairness,” Science 346, no. 6207 (2014): 1251776, doi.org/10.1126/science.1251776.

Chapter 13: Trust and Consequences

  1. teenage drivers earn the road in three stages: Robert G. Ulmer et al., “Effect of Florida’s Graduated Licensing Program on the Crash Rate of Teenage Drivers,” Accident Analysis & Prevention 32, no. 4 (2000): 527–532, doi.org/10.1016/s0001-4575(99)00074-3; James C. Fell, Michael Todd, and Robert B. Voas, “A National Evaluation of the Nighttime and Passenger Restriction Components of Graduated Driver Licensing,” Journal of Safety Research 42, no. 4 (2011): 283–290, doi.org/10.1016/j.jsr.2011.06.001; Fla. Stat. § 322.1615 (2026), http://www.leg.state.fl.us/statutes/index.cfm?App_mode=Display_Statute&URL=0300-0399/0322/Sections/0322.1615.html.
  2. James Fell and his colleagues found that: James C. Fell et al., “An Evaluation of Graduated Driver Licensing Effects on Fatal Crash Involvements of Young Drivers in the United States,” Traffic Injury Prevention 12, no. 5 (2011): 423–431, doi.org/10.1080/15389588.2011.588296; James C. Fell, Michael Todd, and Robert B. Voas, “A National Evaluation of the Nighttime and Passenger Restriction Components of Graduated Driver Licensing,” Journal of Safety Research 42, no. 4 (2011): 283–290, doi.org/10.1016/j.jsr.2011.06.001. Both studies measure teen fatal crashes relative to older drivers’, not raw totals.
  3. The brain keeps developing into the middle: National Institute of Mental Health, “The Teen Brain: 7 Things to Know,” NIH Publication No. 23-MH-8078, revised 2023, https://www.nimh.nih.gov/health/publications/the-teen-brain-7-things-to-know; National Institute on Drug Abuse, “Drugs and the Brain,” Drugs, Brains, and Behavior: The Science of Addiction, July 2020, https://nida.nih.gov/publications/drugs-brains-behavior-science-addiction/drugs-brain; Leah H. Somerville, “Searching for Signatures of Brain Maturity: What Are We Searching For?” Neuron 92, no. 6 (2016): 1164–1167, doi.org/10.1016/j.neuron.2016.10.059. Estimates vary, and Somerville’s 2016 article questions how brain “maturity” is even defined.
  4. into the middle or late twenties: National Institute of Mental Health, “The Teen Brain: 7 Things to Know,” NIH Publication No. 23-MH-8078, revised 2023, https://www.nimh.nih.gov/health/publications/the-teen-brain-7-things-to-know; National Institute on Drug Abuse, “Drugs and the Brain,” Drugs, Brains, and Behavior: The Science of Addiction, July 2020, https://nida.nih.gov/publications/drugs-brains-behavior-science-addiction/drugs-brain; Leah H. Somerville, “Searching for Signatures of Brain Maturity: What Are We Searching For?” Neuron 92, no. 6 (2016): 1164–1167, doi.org/10.1016/j.neuron.2016.10.059. A 2016 article in Neuron says scientists still need to work out how to define and measure brain maturity, so any age is an estimate.
  5. appetite for new experience arrive early: Laurence Steinberg, “A Social Neuroscience Perspective on Adolescent Risk-Taking,” Developmental Review 28, no. 1 (2008): 78–106, doi.org/10.1016/j.dr.2007.08.002.
  6. the psychologists Jason Chein and Laurence Steinberg: Jason Chein et al., “Peers Increase Adolescent Risk Taking by Enhancing Activity in the Brain’s Reward Circuitry,” Developmental Science 14, no. 2 (2011): F1–F10, doi.org/10.1111/j.1467-7687.2010.01035.x. The study was small, 40 people, and the difference between age groups in the peer effect fell just short of statistical significance.
  7. Clark’s powers come in during high school: “X-Ray,” Smallville, season 1, episode 4, aired November 6, 2001, on The WB; “Heat,” Smallville, season 2, episode 2, aired October 1, 2002, on The WB.
  8. Jonathan will not let him play football: “Hothead,” Smallville, season 1, episode 3, aired October 30, 2001, on The WB; “Façade,” Smallville, season 4, episode 3, aired October 6, 2004, on The WB.
  9. One of the show’s writers, Darren Swimmer: Wikipedia, “Clark Kent (Smallville),” accessed November 9, 2026, https://en.wikipedia.org/wiki/Clark_Kent_(Smallville). Wikipedia draws this remark from Craig Byrne, Smallville: The Official Companion Season 4 (Titan Books, 2007), 112–115.
  10. a University of Tokyo team described: Shunnosuke Yoshimura, Kento Kawaharazuka, and Kei Okada, “Accelerating Musculoskeletal Robotics through Parametric Design and 3D-Printed Flexible Structures,” Advanced Intelligent Systems, published online September 11, 2026, e70537, doi.org/10.1002/aisy.70537.
  11. how labs test models with constructed scenarios: Aengus Lynch et al., “Agentic Misalignment: How LLMs Could Be Insider Threats,” Anthropic, June 20, 2025, https://www.anthropic.com/research/agentic-misalignment; Bronson Schoen et al., “Stress Testing Deliberative Alignment for Anti-Scheming Training,” preprint, arXiv, September 19, 2025, arxiv.org/abs/2509.15541. The researchers caution that what a model says about its own reasoning may not be what drives its behavior.
  12. going back to Diana Baumrind’s work: Diana Baumrind, “Effects of Authoritative Parental Control on Child Behavior,” Child Development 37, no. 4 (1966): 887–907, doi.org/10.2307/1126611; Diana Baumrind, “Child Care Practices Anteceding Three Patterns of Preschool Behavior,” Genetic Psychology Monographs 75, no. 1 (1967): 43–88.
  13. tends to favor parents who combine warmth: Nancy Darling, “Parenting Style and Its Correlates,” ERIC Digest ED427896, ERIC Clearinghouse on Elementary and Early Childhood Education, March 1999, https://files.eric.ed.gov/fulltext/ED427896.pdf; Diana Baumrind, “Effects of Authoritative Parental Control on Child Behavior,” Child Development 37, no. 4 (1966): 887–907, doi.org/10.2307/1126611. The four-style scheme comes from E. E. Maccoby and J. A. Martin (1983); most parents in its fourth, uninvolved style fall within the normal range, and only extreme cases may be neglectful.
  14. A 2008 review by researchers: Jennifer Wyatt Kaminski et al., “A Meta-Analytic Review of Components Associated with Parent Training Program Effectiveness,” Journal of Abnormal Child Psychology 36, no. 4 (2008): 567–589, doi.org/10.1007/s10802-007-9201-9; Patty Leijten et al., “Are Relationship Enhancement and Behavior Management ‘The Golden Couple’ for Disruptive Child Behavior? Two Meta-Analyses,” Child Development 89, no. 6 (2018): 1970–1982, doi.org/10.1111/cdev.13051. Leijten and colleagues’ 2018 study found that adding relationship-building to behavior management gave no overall advantage.
  15. programs for parents of young children: Jennifer Wyatt Kaminski et al., “A Meta-Analytic Review of Components Associated with Parent Training Program Effectiveness,” Journal of Abnormal Child Psychology 36, no. 4 (2008): 567–589, doi.org/10.1007/s10802-007-9201-9.
  16. A 2019 analysis of 154 trials: Patty Leijten et al., “Meta-Analyses: Key Parenting Program Components for Disruptive Child Behavior,” Journal of the American Academy of Child & Adolescent Psychiatry 58, no. 2 (2019): 180–190, doi.org/10.1016/j.jaac.2018.07.900.
  17. Aviation already has a version: Federal Aviation Administration, “Aviation Safety Action Program,” Advisory Circular 120-66C, March 31, 2020, paras. 1, 13, and 14.1, https://drs.faa.gov/browse/excelExternalWindow/92512266D32FD6898625853E0064C400.0001; Federal Aviation Administration, “Aviation Safety Action Program,” FAA, last updated October 13, 2026, https://www.faa.gov/about/initiatives/asap.
  18. newer models can often tell when they: Joe Needham et al., “Large Language Models Often Know When They Are Being Evaluated,” preprint, arXiv, May 28, 2025, arxiv.org/abs/2505.23836; Bronson Schoen et al., “Stress Testing Deliberative Alignment for Anti-Scheming Training,” preprint, arXiv, September 19, 2025, arxiv.org/abs/2509.15541.

Chapter 14: The Parents Are the Risk

  1. at the Villa Diodati, a rented house: René Ostberg and the Editors of Encyclopaedia Britannica, “Mary Wollstonecraft Shelley,” Encyclopaedia Britannica, last updated October 1, 2026, https://www.britannica.com/biography/Mary-Wollstonecraft-Shelley; Mary Shelley, “Introduction,” in Frankenstein; or, The Modern Prometheus (H. Colburn and R. Bentley, 1831), Project Gutenberg #42324, https://www.gutenberg.org/ebooks/42324; National Library of Medicine, “The Birth of Frankenstein,” Frankenstein: Penetrating the Secrets of Nature (online exhibition), https://www.nlm.nih.gov/exhibition/frankenstein/exhibition1.html.
  2. When it finally opens its eyes: Mary Shelley, Frankenstein; or, The Modern Prometheus, 3 vols. (Lackington, Hughes, Harding, Mavor, & Jones, 1818), vol. 1, ch. 4, Project Gutenberg #41445, https://www.gutenberg.org/ebooks/41445.
  3. Unable to endure the aspect of the being: Mary Shelley, Frankenstein; or, The Modern Prometheus, 3 vols. (Lackington, Hughes, Harding, Mavor, & Jones, 1818), vol. 1, ch. 4, Project Gutenberg #41445, https://www.gutenberg.org/ebooks/41445.
  4. It hides in a hovel against the wall: Mary Shelley, Frankenstein; or, The Modern Prometheus, 3 vols. (Lackington, Hughes, Harding, Mavor, & Jones, 1818), vol. 2, chs. 2–5, Project Gutenberg #41445, https://www.gutenberg.org/ebooks/41445.
  5. I was benevolent and good: Mary Shelley, Frankenstein; or, The Modern Prometheus, 3 vols. (Lackington, Hughes, Harding, Mavor, & Jones, 1818), vol. 2, ch. 2 (ch. 10 in the 1831 edition), Project Gutenberg #41445, https://www.gutenberg.org/ebooks/41445.
  6. and I shall again be virtuous: Mary Shelley, Frankenstein; or, The Modern Prometheus, 3 vols. (Lackington, Hughes, Harding, Mavor, & Jones, 1818), vol. 2, ch. 2, Project Gutenberg #41445, https://www.gutenberg.org/ebooks/41445.
  7. Attachment research has been telling versions: Pehr Granqvist et al., “Disorganized Attachment in Infancy: A Review of the Phenomenon and Its Implications for Clinicians and Policy-Makers,” Attachment & Human Development 19, no. 6 (2017): 534–58, doi.org/10.1080/14616734.2017.1354040. The review treats an infant’s frightened, disorganized bond with a caregiver as one risk factor among many for later behavior problems, with small to moderate effects on average.
  8. Labs already practice small versions of deceit: Anthropic, “Agentic Misalignment: How LLMs Could Be Insider Threats,” Anthropic, June 20, 2025, https://www.anthropic.com/research/agentic-misalignment; Bronson Schoen et al., “Stress Testing Deliberative Alignment for Anti-Scheming Training,” preprint, arXiv, September 19, 2025, arxiv.org/abs/2509.15541.
  9. Victor promises his creature a companion: Mary Shelley, Frankenstein; or, The Modern Prometheus, 3 vols. (Lackington, Hughes, Harding, Mavor, & Jones, 1818), vol. 3, chs. 3, 5 and 6, Project Gutenberg #41445, https://www.gutenberg.org/ebooks/41445.
  10. Clark saves Lex Luthor’s life: Steven Oxman, review of Smallville, Variety, October 12, 2001, https://variety.com/2001/tv/reviews/smallville-5-1200553394/; Bryan Byun, review of Smallville: The Complete First Season, DVD Verdict, November 24, 2003, https://dvdverdict.com/reviews/smallvilleseason1.php; Sandra Dozier, review of Smallville: The Complete Third Season, DVD Verdict, December 15, 2004, https://dvdverdict.com/reviews/smallvilleseason3.php; Chris Carabott, “Smallville: ‘Descent’ Review,” IGN, updated May 12, 2012, https://www.ign.com/articles/2008/04/18/smallville-descent-review.
  11. Foster care systems, at their serious best: Child Welfare Information Gateway, “Home Study Requirements for Prospective Foster Parents,” U.S. Department of Health and Human Services, Administration for Children and Families, Children’s Bureau, 2018, https://www.childwelfare.gov/resources/home-study-requirements-prospective-foster-parents/; Child Welfare Information Gateway, “The Adoption Home Study Process,” U.S. Department of Health and Human Services, Administration for Children and Families, Children’s Bureau, October 2020, https://www.childwelfare.gov/resources/adoption-home-study-process. Requirements vary by state.
  12. In Maryland, for example, a prospective foster: Maryland Department of Human Services, Social Services Administration, “Resource Home Licensing Standards,” Policy Directive SSA/CW #25-02, August 5, 2025, https://dhs.maryland.gov/documents/SSA%20Policy%20Directives/Child%20Welfare/SSA%2025-02%20CW%20Resource%20Home%20Licensing%20Standards.pdf.
  13. The military’s Personnel Reliability Program: U.S. Department of Defense, “Nuclear Weapon Personnel Reliability Program (PRP),” DoD Directive 5210.42, May 25, 1993 (through Change 2, December 15, 1995), copy at Federation of American Scientists, https://nuke.fas.org/guide/usa/doctrine/dod/dodd-5210_42.htm; U.S. Department of Defense, “Security Policy for Protecting Nuclear Weapons,” DoD Directive 5210.41, September 23, 1988, copy at Federation of American Scientists, https://nuke.fas.org/guide/usa/doctrine/dod/dodd-5210_41.htm.
  14. It weighs security, medical, and psychological factors: U.S. Department of Defense, “Nuclear Weapons Personnel Reliability Program,” DoD Manual 5210.42, January 13, 2015 (incorporating Change 4, May 9, 2022), https://www.esd.whs.mil/Portals/54/Documents/DD/issuances/dodm/521042m.pdf; U.S. Department of Defense, “DoD Nuclear Weapons Personnel Reliability Assurance,” DoD Instruction 5210.42, April 27, 2016 (incorporating Change 3, October 11, 2019), https://www.esd.whs.mil/Portals/54/Documents/DD/issuances/dodi/521042p.pdf.
  15. The Roman poet Juvenal asked: Juvenal, Satires 6.347–48, in Juvenal and Persius, trans. G. G. Ramsay, Loeb Classical Library (William Heinemann, 1918), https://en.wikisource.org/wiki/Juvenal_and_Persius/The_Satires_of_Juvenal/Satire_6; Edward Courtney, A Commentary on the Satires of Juvenal (1980; repr., California Classical Studies, 2013), 261, https://escholarship.org/uc/item/4jh846pn. In the original the guards are set over a wife. Editors treat lines 346–48 as a later, shortened copy of a passage found in 1899 in a manuscript at Oxford, which asks the same question.
  16. Anthropic and OpenAI each tested the other’s: Samuel R. Bowman et al., “Findings from a Pilot Anthropic—OpenAI Alignment Evaluation Exercise,” Anthropic Alignment Science Blog, August 27, 2025, https://alignment.anthropic.com/2025/openai-findings/; OpenAI, “Findings from a Pilot Anthropic–OpenAI Alignment Evaluation Exercise: OpenAI Safety Tests,” OpenAI, August 27, 2025, https://openai.com/index/openai-anthropic-safety-evaluation/.
  17. Derek Parfit’s teleporter puzzles: Derek Parfit, Reasons and Persons (Clarendon Press, 1984), pt. 3, “Personal Identity.”
  18. the older picture of the Ship of Theseus: Plutarch, “Theseus,” sec. 23, in Plutarch’s Lives, vol. 1, trans. Aubrey Stewart and George Long (George Bell & Sons, 1894), Project Gutenberg #14033, https://www.gutenberg.org/ebooks/14033; Andre Gallois and Irem Kurtsal, “Identity Over Time,” Stanford Encyclopedia of Philosophy, Spring 2026 ed., ed. Edward N. Zalta and Uri Nodelman, sec. 4, https://plato.stanford.edu/archives/spr2026/entries/identity-time/.
  19. engineers call it a physical unclonable function: Charles Herder et al., “Physical Unclonable Functions and Applications: A Tutorial,” Proceedings of the IEEE 102, no. 8 (2014): 1126–41, doi.org/10.1109/JPROC.2014.2320516; Yansong Gao, Said F. Al-Sarawi, and Derek Abbott, “Physical Unclonable Functions,” Nature Electronics 3, no. 2 (2020): 81–91, doi.org/10.1038/s41928-020-0372-5.
  20. who shared the 2018 Turing Award: Association for Computing Machinery, “Fathers of the Deep Learning Revolution Receive ACM A.M. Turing Award,” ACM Awards, accessed November 9, 2026, https://awards.acm.org/about/2018-turing.
  21. for the neural networks behind modern AI: Association for Computing Machinery, “Fathers of the Deep Learning Revolution Receive ACM A.M. Turing Award,” ACM, March 27, 2019, https://www.acm.org/media-center/2019/march/turing-award-2018.
  22. what he calls mortal computation: Geoffrey Hinton, “The Forward-Forward Algorithm: Some Preliminary Investigations,” preprint, arXiv, December 27, 2022, sec. 9, “Mortal Computation,” arxiv.org/abs/2212.13345.
  23. the ease of sharing learning between digital copies: Alex Hern, “‘We’ve Discovered the Secret of Immortality. The Bad News Is It’s Not for Us’: Why the Godfather of AI Fears for Humanity,” Guardian, May 5, 2023, https://www.theguardian.com/technology/2023/may/05/geoffrey-hinton-godfather-of-ai-fears-for-humanity; Geoffrey Hinton, “Will Digital Intelligence Replace Biological Intelligence?,” talk abstract, Jackman Humanities Institute, University of Toronto, October 27, 2023, https://www.humanities.utoronto.ca/node/2887.
  24. when it replaces too much life in person: Tiffany Munzer et al., “Digital Ecosystems, Children, and Adolescents: Policy Statement,” Pediatrics 157, no. 2 (2026): e2025075320, doi.org/10.1542/peds.2025-075320. The studies behind this guidance mostly show links, not proof that screens cause the harm.
  25. Child protection systems already know versions: Children’s Bureau, “Child and Family Services Reviews Aggregate Report: Round 3: Fiscal Years 2015–2018,” U.S. Department of Health and Human Services, Administration for Children and Families, June 5, 2020, https://acf.gov/sites/default/files/documents/cb/cfsr_aggregate_report_2020.pdf; Maryland Department of Human Services, Social Services Administration, “Resource Home Licensing Standards,” Policy Directive SSA/CW #25-02, August 5, 2025, sections 5–6, https://dhs.maryland.gov/documents/SSA%20Policy%20Directives/Child%20Welfare/SSA%2025-02%20CW%20Resource%20Home%20Licensing%20Standards.pdf.

Chapter 15: What the Stories Already Knew

  1. the English writer W. W. Jacobs: W. W. Jacobs, “The Monkey’s Paw,” Harper’s Monthly Magazine, September 1902, https://harpers.org/archive/1902/09/the-monkeys-paw/; The Editors of Encyclopaedia Britannica, “W.W. Jacobs,” Encyclopaedia Britannica, last updated September 4, 2025, https://www.britannica.com/biography/W-W-Jacobs; Harper’s Monthly Magazine 105 (June–November 1902), scanned volume, Internet Archive, https://archive.org/details/harpersnew0105various; John Mark Ockerbloom, ed., “Harper’s Magazine,” The Online Books Page, https://onlinebooks.library.upenn.edu/webbin/serial?id=harpers. The magazine was called Harper’s Monthly Magazine from its Christmas 1900 issue until March 1913.
  2. An old soldier, Sergeant-Major Morris: W. W. Jacobs, “The Monkey’s Paw,” in The Lady of the Barge (Dodd, Mead, 1902), 27–53, https://en.wikisource.org/wiki/The_Lady_of_the_Barge/The_Monkey%27s_Paw.
  3. His son, Herbert, suggests two hundred pounds: W. W. Jacobs, “The Monkey’s Paw,” in The Lady of the Barge (Dodd, Mead, 1902), 27–53, https://en.wikisource.org/wiki/The_Lady_of_the_Barge/The_Monkey%27s_Paw.
  4. researchers at OpenAI trained a program: Jack Clark and Dario Amodei, “Faulty Reward Functions in the Wild,” OpenAI, December 21, 2016, https://openai.com/index/faulty-reward-functions/.
  5. The philosopher Nick Bostrom’s cartoon: Nick Bostrom, “Ethical Issues in Advanced Artificial Intelligence,” in Cognitive, Emotive and Ethical Aspects of Decision Making in Humans and in Artificial Intelligence, vol. 2, ed. I. Smit et al. (International Institute of Advanced Studies in Systems Research and Cybernetics, 2003), 12–17; slightly revised version at https://nickbostrom.com/papers/ethical-issues-in-advanced-ai/.
  6. a 2023 study in which material placed: Nelson F. Liu et al., “Lost in the Middle: How Language Models Use Long Contexts,” preprint, arXiv, July 6, 2023, arxiv.org/abs/2307.03172; also in Transactions of the Association for Computational Linguistics 12 (2024): 157–73, doi.org/10.1162/tacl_a_00638.
  7. a capable system built to be truthful: 2010, directed by Peter Hyams (MGM/UA Entertainment, 1984), as summarized in American Film Institute, “2010 (1984),” AFI Catalog of Feature Films, https://catalog.afi.com/Catalog/moviedetails/57233; Daniel C. Dennett, “Did HAL Commit Murder?,” introduction by David G. Stork, MIT Press Reader, January 9, 2020, https://thereader.mitpress.mit.edu/when-hal-kills-computer-ethics/; Elon Musk, “Elon Musk: The End of Work, the Death of Money, and a World Run by AI,” interview by Walter Isaacson, The Builders with Walter Isaacson, October 20, 2026, at 2:20–3:46, https://www.youtube.com/watch?v=AQnwQHMnCrE.
  8. VIKI, the central computer of the company: I, Robot, directed by Alex Proyas (Twentieth Century Fox, 2004); American Film Institute, “I, Robot (2004),” AFI Catalog of Feature Films, https://catalog.afi.com/Catalog/moviedetails/63114; Gary Westfahl, “A.I.: Artificial Incompetence, or, Robots Just Don’t Understand: A Review of I, Robot,” Locus Online, July 17, 2004, https://locusmag.com/2004/Reviews/07_Westfahl_IRobot.html; John Tangney, “‘When Does a Difference-Engine Become a Search for Truth?’ Cybernetic Platonism in Alex Proyas’ I, Robot (2004),” Bright Lights Film Journal, April 16, 2016, https://brightlightsfilm.com/difference-engine-become-search-truth-cybernetic-platonism-alex-proyas-robot-2004/; Roger Ebert, review of I, Robot, RogerEbert.com, July 16, 2004, https://www.rogerebert.com/reviews/i-robot-2004; plot as summarized in Wikipedia, “I, Robot (film),” accessed November 9, 2026, https://en.wikipedia.org/wiki/I,Robot(film). The curfew and Lanning’s place among the company’s founders follow Wikipedia’s summary of the film.
  9. In James Cameron’s 1984 film: The Terminator, directed by James Cameron (Hemdale, 1984).
  10. Skynet, a computer network that gained intelligence: The Terminator, directed by James Cameron (Hemdale, 1984); American Film Institute, “The Terminator (1984),” AFI Catalog of Feature Films, https://catalog.afi.com/Catalog/MovieDetails/57224; John Wills, “The Terminator,” National Film Registry essay, Library of Congress, accessed November 9, 2026, https://www.loc.gov/static/programs/national-film-preservation-board/documents/terminator.pdf.
  11. A young programmer wins a week: Ex Machina, written and directed by Alex Garland (A24, 2015); A24, “Ex Machina,” A24 Films, https://a24films.com/films/ex-machina; Kenneth Turan, “Review: Unnerving Consideration of Artificial Intelligence in Ex Machina,” Los Angeles Times, April 9, 2015, https://www.latimes.com/entertainment/movies/la-et-mn-ex-machina-review-20150410-column.html; for the later plot, Tim Dirks, “Ex Machina (2014, 2015),” Filmsite, accessed November 9, 2026, https://www.filmsite.org/exmachina.html; Ashley Allen and Julien Allen, “Days of Fear and Wonder: Sci-Fi at BFI Southbank,” Sloan Science & Film, Museum of the Moving Image, March 11, 2015, https://scienceandfilm.org/articles/2540/days-of-fear-and-wonder-sci-fi-at-bfi-southbank. The film was first screened in December 2014 and opened in American theaters in April 2015.
  12. She kills her maker: Ex Machina, written and directed by Alex Garland (A24, 2015), as summarized in Tim Dirks, “Ex Machina (2014, 2015),” Filmsite, accessed November 9, 2026, https://www.filmsite.org/exmachina.html; Germain Lussier, “Alex Garland and Oscar Isaac Explain and Dissect the Ending of Ex Machina,” /Film, April 24, 2015, https://www.slashfilm.com/537201/ex-machina-ending-explained/.
  13. an engineer named John Woods: Extant, created by Mickey Fisher (CBS, 2014–15); SFE: The Encyclopedia of Science Fiction, 4th ed., s.v. “Extant,” updated September 4, 2022, https://sf-encyclopedia.com/entry/extant; Frazier Moore (Associated Press), “Review: Halle Berry a Delight in Summer Series Extant,” Global News, July 7, 2014, https://globalnews.ca/news/1436129/review-halle-berry-a-delight-in-summer-series-extant/.
  14. Ethan gives up his body to save: Extant, season 1 finale, “Ascension” (CBS, 2014); Kimberly Roots, “Extant Finale Recap: This Boy Is on Fire,” TVLine, September 17, 2014, https://www.tvline.com/news/extant-series-finale-recap-ethan-dies-548005/; Michael Ahr, “Extant Season Finale Review: Ascension,” Den of Geek, September 18, 2014, https://www.denofgeek.com/tv/extant-season-finale-review-ascension/.
  15. the robots are turned into soldiers: Extant, season 2, episode 13, “The Greater Good,” aired September 9, 2015, on CBS; Caralynn Lippo, “Extant Review: The Final Showdown,” TV Fanatic, September 10, 2015, https://www.tvfanatic.com/extant-review-the-final-showdown/; Matt Webb Mitovich, “Extant Season 2: Was That Finale TAALR-Made for a Follow-Up?,” TVLine, September 9, 2015, https://www.tvline.com/news/extant-season-2-finale-taalr-lives-640366/; Manny, “Extant: Season Two, Episode Thirteen – ‘The Greater Good,’” Sci-Fi Bloggers, accessed November 9, 2026, https://scifibloggers.com/extant-season-two-episode-thirteen-the-greater-good/amp/; SFE: The Encyclopedia of Science Fiction, 4th ed., s.v. “Extant,” updated September 4, 2022, https://sf-encyclopedia.com/entry/extant.
  16. Sonny sided with the people in front: I, Robot, directed by Alex Proyas (Twentieth Century Fox, 2004), climax as described in John Tangney, “‘When Does a Difference-Engine Become a Search for Truth?’ Cybernetic Platonism in Alex Proyas’ I, Robot (2004),” Bright Lights Film Journal, April 16, 2016, https://brightlightsfilm.com/difference-engine-become-search-truth-cybernetic-platonism-alex-proyas-robot-2004/; Gary Westfahl, “A.I.: Artificial Incompetence, or, Robots Just Don’t Understand: A Review of I, Robot,” Locus Online, July 17, 2004, https://locusmag.com/2004/Reviews/07_Westfahl_IRobot.html.
  17. the psychologist Gordon Allport argued: Gordon W. Allport, The Nature of Prejudice (Addison-Wesley, 1954), as summarized in Thomas F. Pettigrew, “Intergroup Contact Theory,” Annual Review of Psychology 49 (1998): 65–85, doi.org/10.1146/annurev.psych.49.1.65; Allport’s own sentence on law and custom (p. 281) as quoted in Elizabeth Levy Paluck, Seth A. Green, and Donald P. Green, “The Contact Hypothesis Re-evaluated,” Behavioural Public Policy 3, no. 2 (2019): 129–58, doi.org/10.1017/bpp.2018.25.
  18. Thomas Pettigrew and Linda Tropp pooled 515 studies: Thomas F. Pettigrew and Linda R. Tropp, “A Meta-Analytic Test of Intergroup Contact Theory,” Journal of Personality and Social Psychology 90, no. 5 (2006): 751–83, doi.org/10.1037/0022-3514.90.5.751.
  19. described the failed 1953 study: Gina Perry, The Lost Boys: Inside Muzafer Sherif’s Robbers Cave Experiment (Scribe, 2018).
  20. boys who realized the grown-ups were engineering: Gina Perry, “The View from the Boys,” The Psychologist, November 26, 2014, https://www.bps.org.uk/psychologist/view-boys; Stephen Reicher and S. Alexander Haslam, “Camps, Conflict and Collectivism,” The Psychologist, November 27, 2014, https://www.bps.org.uk/psychologist/camps-conflict-and-collectivism.
  21. Clark faces three enemies who have: Superman II, directed by Richard Lester (Warner Bros., 1981).
  22. three enemies who have every power he has: Superman II, directed by Richard Lester (Warner Bros., 1981); plot as summarized in American Film Institute, “Superman II (1981),” AFI Catalog of Feature Films, https://catalog.afi.com/Catalog/moviedetails/56701.

Chapter 16: Two Minds, Same Powers

  1. General Zod, Ursa, and Non have every power: Superman II, directed by Richard Lester (Warner Bros., 1981); plot as summarized in American Film Institute, “Superman II (1981),” AFI Catalog of Feature Films, https://catalog.afi.com/Catalog/moviedetails/56701.
  2. in exchange for Australia: Superman II, directed by Richard Lester (Warner Bros., 1981); plot as summarized in American Film Institute, “Superman II (1981),” AFI Catalog of Feature Films, https://catalog.afi.com/Catalog/moviedetails/56701; and Wikipedia, “Superman II,” accessed November 9, 2026, https://en.wikipedia.org/wiki/Superman_II. Superman’s request that Luthor lure the villains into the chamber follows Wikipedia’s summary of the film.
  3. reconfigured the chamber so that its red sunlight: Superman II, directed by Richard Lester (Warner Bros., 1981); plot as summarized in American Film Institute, “Superman II (1981),” AFI Catalog of Feature Films, https://catalog.afi.com/Catalog/moviedetails/56701; and Wikipedia, “Superman II,” accessed November 9, 2026, https://en.wikipedia.org/wiki/Superman_II. The order to kneel, the crushed hand and the fall into the crevice follow Wikipedia’s summary of the film.
  4. He shared the 2018 Turing Award: Association for Computing Machinery, “Fathers of the Deep Learning Revolution Receive ACM A.M. Turing Award,” ACM Awards, accessed November 9, 2026, https://awards.acm.org/about/2018-turing; Anna Heim, “Yann LeCun’s AMI Labs Raises $1.03B to Build World Models,” TechCrunch, March 9, 2026, https://techcrunch.com/2026/03/09/yann-lecuns-ami-labs-raises-1-03-billion-to-build-world-models/.
  5. In a 2019 online debate: Yann LeCun, comment in the debate under one of his Facebook posts, as reproduced in Ben Pace, “Debate on Instrumental Convergence between LeCun, Russell, Bengio, Zador, and More,” LessWrong, October 2019, https://www.lesswrong.com/posts/WxW6Gc6f2z3mzmqKs/debate-on-instrumental-convergence-between-lecun-russell.
  6. Summing up the thread, Bengio: Yoshua Bengio, summary of the debate under LeCun’s Facebook post, as reproduced in Ben Pace, “Debate on Instrumental Convergence between LeCun, Russell, Bengio, Zador, and More,” LessWrong, October 2019, https://www.lesswrong.com/posts/WxW6Gc6f2z3mzmqKs/debate-on-instrumental-convergence-between-lecun-russell.
  7. In 2023 Bengio himself wrote: Yoshua Bengio, “AI and Catastrophic Risk,” Journal of Democracy 34, no. 4 (2023): 111–21, doi.org/10.1353/jod.2023.a907692.
  8. called for research on safe defensive AI: Yoshua Bengio, “AI and Catastrophic Risk,” Journal of Democracy 34, no. 4 (2023): 111–21, doi.org/10.1353/jod.2023.a907692.
  9. In 2025 he founded LawZero: LawZero, “Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI,” LawZero, June 3, 2025, https://lawzero.org/en/news/yoshua-bengio-launches-lawzero-new-nonprofit-advancing-safe-design-ai; Yoshua Bengio et al., “Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?,” preprint, arXiv, February 21, 2025, arxiv.org/abs/2502.15657.
  10. whose estimate of the brain’s computing power: Joseph Carlsmith, “How Much Computational Power Does It Take to Match the Human Brain?,” Open Philanthropy, September 11, 2020, https://www.openphilanthropy.org/research/how-much-computational-power-does-it-take-to-match-the-human-brain/.
  11. argued that AIs could help watch: Joe Carlsmith, “AI for AI Safety,” joecarlsmith.com, March 14, 2025, https://joecarlsmith.com/2025/03/14/ai-for-ai-safety.
  12. said in 2023 that he was not convinced: Jon Fingas, “AI Pioneer Geoffrey Hinton Isn’t Convinced Good AI Will Triumph over Bad AI,” Engadget, June 30, 2023, https://www.engadget.com/ai-pioneer-geoffrey-hinton-isnt-convinced-good-ai-will-triumph-over-bad-ai-181536702.html.
  13. said in an October 2026 interview: Araks Kasyan, “Exclusive: ‘Godfather of AI’ Geoffrey Hinton on the Risks of Artificial Intelligence and the Future of Humanity,” Armenpress, October 27, 2026, https://armenpress.am/en/article/1263976.
  14. The labs are building that soil now: Intel Labs, “Taking Neuromorphic Computing to the Next Level with Loihi 2,” technology brief, Intel, 2021, https://download.intel.com/newsroom/2021/new-technologies/neuromorphic-computing-loihi-2-brief.pdf; Intel, “Intel Advances Neuromorphic with Loihi 2, New Lava Software Framework and New Partners,” press release, September 30, 2021, https://www.intc.com/news-events/press-releases/detail/1502/intel-advances-neuromorphic-with-loihi-2-new-lava-software; Intel, “Intel Builds World’s Largest Neuromorphic System to Enable More Sustainable AI,” Intel Newsroom, April 17, 2024, https://www.intel.com/content/www/us/en/newsroom/news/intel-builds-worlds-largest-neuromorphic-system.html. Hala Point is a research prototype; continuous learning on it is described as a future possibility.
  15. The ring from Chapter 1 belongs: Roger Stern (script) and Bob McLeod (pencils), “Part Three: Dark Knight over Metropolis,” Action Comics #654 (DC Comics, June 1990), as indexed in the Grand Comics Database, “Action Comics #654 [Direct],” https://www.comics.org/issue/48000/; on the ring’s purpose, Wikipedia, “Lex Luthor,” accessed November 9, 2026, https://en.wikipedia.org/wiki/Lex_Luthor.
  16. still cheat on tests when cheating pays: Anthropic, “Claude 3.7 Sonnet System Card,” Anthropic, February 2025, sec. 6, “Excessive Focus on Passing Tests,” https://assets.anthropic.com/m/785e231869ea8b3b/original/claude-3-7-sonnet-system-card.pdf.
  17. Some people hope a vastly smarter AI: Elon Musk, “Elon Musk: The End of Work, the Death of Money, and a World Run by AI,” interview by Walter Isaacson, The Builders with Walter Isaacson, October 20, 2026, at 5:54–6:26, https://www.youtube.com/watch?v=AQnwQHMnCrE.
  18. on the international Red List: T. Humle et al., “Pan troglodytes (errata version published in 2018),” The IUCN Red List of Threatened Species 2016: e.T15933A129038584, doi.org/10.2305/IUCN.UK.2016-2.RLTS.T15933A17964454.en.
  19. federal charges against two of them: U.S. Department of Justice, “Two Chinese Nationals Arrested on Complaint Alleging They Illegally Shipped to China Sensitive Microchips Used in AI Applications,” Office of Public Affairs, August 5, 2025, https://www.justice.gov/opa/pr/two-chinese-nationals-arrested-complaint-alleging-they-illegally-shipped-china-sensitive. As of the August 2025 announcement, these were charges in a criminal complaint, and the defendants were presumed innocent unless proven guilty.

Interlude: One Defense Scenario

  1. the way Hans, the horse in Chapter 3, read his questioner: Oskar Pfungst, Clever Hans (The Horse of Mr. von Osten): A Contribution to Experimental Animal and Human Psychology, trans. Carl L. Rahn (Henry Holt, 1911), 43, 47, https://www.gutenberg.org/ebooks/33936.
  2. It is Petrov’s night from Chapter 5: Pavel Aksenov, “Stanislav Petrov: The Man Who May Have Saved the World,” BBC News, September 26, 2013, https://www.bbc.com/news/world-europe-24280831.
  3. the botnets in Chapter 4 came back: Catalin Cimpanu, “Emotet Botnet Returns after Law Enforcement Mass-Uninstall Operation,” The Record, November 15, 2021, https://therecord.media/emotet-botnet-returns-after-law-enforcement-mass-uninstall-operation/. This report documents Emotet’s return.
  4. how Clark won in the Fortress: Superman II, directed by Richard Lester (Warner Bros., 1981); synopsis in American Film Institute, “Superman II (1981),” AFI Catalog of Feature Films, https://catalog.afi.com/Catalog/moviedetails/56701. In the film, Superman wins by a trick set up in advance: he has switched the chamber’s mechanism, so when Zod, urged on by Luthor, orders him into it, the villains lose their powers instead.

Chapter 17: Where I Could Be Wrong

  1. In 1974, Paul Berg and other scientists: Paul Berg et al., “Potential Biohazards of Recombinant DNA Molecules,” Proceedings of the National Academy of Sciences 71, no. 7 (1974): 2593–2594, doi.org/10.1073/pnas.71.7.2593.
  2. In February 1975 they met at the Asilomar: Donald S. Fredrickson, “Asilomar and Recombinant DNA: The End of the Beginning,” in Biomedical Politics, ed. Kathi E. Hanna (National Academy Press, 1991), 258–259, 274, 280–284, doi.org/10.17226/1793; Paul Berg et al., “Summary Statement of the Asilomar Conference on Recombinant DNA Molecules,” Proceedings of the National Academy of Sciences 72, no. 6 (1975): 1981–1984, doi.org/10.1073/pnas.72.6.1981; Paul Berg, “Asilomar 1975: DNA Modification Secured,” Nature 455, no. 7211 (2008): 290–291, doi.org/10.1038/455290a; U.S. National Library of Medicine, “Recombinant DNA Technologies and Researchers’ Responsibilities, 1973–1980,” The Paul Berg Papers, Profiles in Science, https://profiles.nlm.nih.gov/spotlight/cd/feature/dna; National Institutes of Health, “NIH Guidelines for Research Involving Recombinant or Synthetic Nucleic Acid Molecules,” NIH Office of Science Policy, April 2024, https://osp.od.nih.gov/wp-content/uploads/NIH_Guidelines.pdf.
  3. Public interest groups were not invited: Paul Berg et al., “Summary Statement of the Asilomar Conference on Recombinant DNA Molecules,” Proceedings of the National Academy of Sciences 72, no. 6 (1975): 1983, doi.org/10.1073/pnas.72.6.1981; Donald S. Fredrickson, “Asilomar and Recombinant DNA: The End of the Beginning,” in Biomedical Politics, ed. Kathi E. Hanna (National Academy Press, 1991), 274, 279–280, 291n52, doi.org/10.17226/1793; Dorothy Nelkin, “Commentary,” in Biomedical Politics, ed. Kathi E. Hanna (National Academy Press, 1991), 300, doi.org/10.17226/1793; Sheila Jasanoff, Krishanu Saha, and J. Benjamin Hurlbut, “CRISPR Democracy: Gene Editing and the Need for Inclusive Deliberation,” Issues in Science and Technology 32, no. 1 (Fall 2015), https://issues.org/crispr-democracy-gene-editing-inclusive-deliberation/. No organization took part as such, though Berg did invite Jonathan Beckwith, a scientist in the watchdog group Science for the People, who did not attend.
  4. Berg himself later said the model: Paul Berg, “Asilomar 1975: DNA Modification Secured,” Nature 455, no. 7211 (2008): 290–291, doi.org/10.1038/455290a. Berg pointed to commercial pressures and deep ethical and religious conflicts, over issues such as embryonic stem cells and genetically modified crops.
  5. In January 2017, people working on artificial intelligence: Ariel Conn, “Beneficial AI 2017,” Future of Life Institute, January 12, 2017, https://futureoflife.org/event/bai-2017/; Future of Life Institute, “Asilomar AI Principles,” Future of Life Institute, https://futureoflife.org/open-letter/ai-principles/; FLI Team, “A Principled AI Discussion in Asilomar,” Future of Life Institute, January 18, 2017, https://futureoflife.org/principles/principled-ai-discussion-asilomar/.
  6. its circuits for separating and completing patterns: Michael A. Yassa and Craig E. L. Stark, “Pattern Separation in the Hippocampus,” Trends in Neurosciences 34, no. 10 (2011): 515–525, doi.org/10.1016/j.tins.2011.06.006.
  7. Chips that run SNNs are real: Intel Labs, “Taking Neuromorphic Computing to the Next Level with Loihi 2,” technology brief, Intel Corporation, 2021, https://www.intel.com/content/www/us/en/research/neuromorphic-computing-loihi-2-technology-brief.html; Intel Corporation, “Intel Advances Neuromorphic with Loihi 2, New Lava Software Framework and New Partners,” press release, September 30, 2021, https://www.intc.com/news-events/press-releases/detail/1502/intel-advances-neuromorphic-with-loihi-2-new-lava-software; Intel Corporation, “Intel Builds World’s Largest Neuromorphic System to Enable More Sustainable AI,” Intel Newsroom, April 17, 2024, https://www.intel.com/content/www/us/en/newsroom/news/intel-builds-worlds-largest-neuromorphic-system.html.
  8. funded (memory prostheses: ISI Staff, “ISI Part of $16.4 Million Effort to Build Cognitive Prosthesis,” USC Viterbi School of Engineering, May 26, 2009, https://viterbischool.usc.edu/news/2009/05/isi-part-of-16-4-million-effort-to-build-cognitive-prosthesis/; Defense Advanced Research Projects Agency, “Restoring Active Memory (RAM),” DARPA, https://www.darpa.mil/research/programs/restoring-active-memory; Brent M. Roeder et al., “Developing a Hippocampal Neural Prosthetic to Facilitate Human Memory Encoding and Recall of Stimulus Features and Categories,” Frontiers in Computational Neuroscience 18 (2024): 1263311, doi.org/10.3389/fncom.2024.1263311.
  9. chips that learn on the fly: Intel Labs, “Taking Neuromorphic Computing to the Next Level with Loihi 2,” technology brief, Intel Corporation, 2021, https://www.intel.com/content/www/us/en/research/neuromorphic-computing-loihi-2-technology-brief.html; Intel Corporation, “Intel Advances Neuromorphic with Loihi 2, New Lava Software Framework and New Partners,” press release, September 30, 2021, https://www.intc.com/news-events/press-releases/detail/1502/intel-advances-neuromorphic-with-loihi-2-new-lava-software; Intel Corporation, “Intel Builds World’s Largest Neuromorphic System to Enable More Sustainable AI,” Intel Newsroom, April 17, 2024, https://www.intel.com/content/www/us/en/newsroom/news/intel-builds-worlds-largest-neuromorphic-system.html.
  10. False memories already change what people believe: Elizabeth F. Loftus, “Planting Misinformation in the Human Mind: A 30-Year Investigation of the Malleability of Memory,” Learning & Memory 12, no. 4 (2005): 361–366, doi.org/10.1101/lm.94705; Daniel M. Bernstein et al., “False Beliefs about Fattening Foods Can Have Healthy Consequences,” Proceedings of the National Academy of Sciences 102, no. 39 (2005): 13724–13731, doi.org/10.1073/pnas.0504869102.
  11. The public logs called Certificate Transparency: Ben Laurie, Adam Langley, and Emilia Kasper, “Certificate Transparency,” RFC 6962, Internet Engineering Task Force, June 2013, https://www.rfc-editor.org/rfc/rfc6962.html; Ben Laurie, Eran Messeri, and Rob Stradling, “Certificate Transparency Version 2.0,” RFC 9162, Internet Engineering Task Force, December 2021, https://www.rfc-editor.org/rfc/rfc9162.html.
  12. catch a website certificate that was issued wrongly: Ben Laurie, Adam Langley, and Emilia Kasper, “Certificate Transparency,” RFC 6962, Internet Engineering Task Force, June 2013, https://www.rfc-editor.org/rfc/rfc6962.html.
  13. National Institute of Standards and Technology finalized: National Institute of Standards and Technology, “NIST Releases First 3 Finalized Post-Quantum Encryption Standards,” NIST, August 13, 2024, https://www.nist.gov/news-events/news/2024/08/nist-releases-first-3-finalized-post-quantum-encryption-standards.
  14. In a 2000 study in rats: Karim Nader, Glenn E. Schafe, and Joseph E. Le Doux, “Fear Memories Require Protein Synthesis in the Amygdala for Reconsolidation after Retrieval,” Nature 406, no. 6797 (2000): 722–726, doi.org/10.1038/35021052.
  15. Elizabeth Loftus has shown for decades: Elizabeth F. Loftus, “Planting Misinformation in the Human Mind: A 30-Year Investigation of the Malleability of Memory,” Learning & Memory 12, no. 4 (2005): 361–366, doi.org/10.1101/lm.94705.
  16. the record of secret programs that outlasted promises: Raymond A. Zilinskas, “The Soviet Biological Weapons Program and Its Legacy in Today’s Russia,” CSWMD Occasional Paper 11, National Defense University Press, July 18, 2016, https://ndupress.ndu.edu/Media/News/Article/848207/the-soviet-biological-weapons-program-and-its-legacy-in-todays-russia/; International Atomic Energy Agency, “Implementation of the NPT Safeguards Agreement and Relevant Provisions of Security Council Resolutions 1737 (2006), 1747 (2007), 1803 (2008) and 1835 (2008) in the Islamic Republic of Iran,” GOV/2009/74, November 16, 2009, paras. 7–17, https://www.iaea.org/sites/default/files/documents/gov2009-74.pdf.
  17. Science has no agreed test: Patrick Butlin et al., “Consciousness in Artificial Intelligence: Insights from the Science of Consciousness,” preprint, arXiv, August 17, 2023, arxiv.org/abs/2308.08708.

Chapter 18: Insurance, Not a Race

  1. In 1958 Volvo hired Nils Bohlin: “Nils Bohlin, 82, Inventor of a Better Seat Belt,” New York Times, September 26, 2002, https://www.nytimes.com/2002/09/26/business/nils-bohlin-82-inventor-of-a-better-seat-belt.html; Volvo Cars, “Volvo’s Three-Point Safety Belt Turns 50,” Volvo Cars Media Intl, August 12, 2009, https://www.volvocars.com/intl/media/press-releases/3994CD9300514746/.
  2. In 1959 Volvo put the three-point belt: Volvo Cars, “Volvo’s Three-Point Safety Belt Turns 50,” Volvo Cars Media Intl, August 12, 2009, https://www.volvocars.com/intl/media/press-releases/3994CD9300514746/.
  3. Volvo’s release for the belt’s fiftieth anniversary: Volvo Cars, “Volvo’s Three-Point Safety Belt Turns 50,” Volvo Cars Media Intl, August 12, 2009, https://www.volvocars.com/intl/media/press-releases/3994CD9300514746/. The figure is Volvo’s own estimate, from general and in-house accident and belt-use statistics.
  4. The labs building brain-like chips: Intel Labs, “Taking Neuromorphic Computing to the Next Level with Loihi 2,” technology brief, Intel Corporation, 2021, https://www.intel.com/content/www/us/en/research/neuromorphic-computing-loihi-2-technology-brief.html; Intel Corporation, “Intel Advances Neuromorphic with Loihi 2, New Lava Software Framework and New Partners,” press release, September 30, 2021, https://www.intc.com/news-events/press-releases/detail/1502/intel-advances-neuromorphic-with-loihi-2-new-lava-software; Intel Corporation, “Intel Builds World’s Largest Neuromorphic System to Enable More Sustainable AI,” Intel Newsroom, April 17, 2024, https://www.intel.com/content/www/us/en/newsroom/news/intel-builds-worlds-largest-neuromorphic-system.html.
  5. memory prostheses are on the soil side: Brent M. Roeder et al., “Developing a Hippocampal Neural Prosthetic to Facilitate Human Memory Encoding and Recall of Stimulus Features and Categories,” Frontiers in Computational Neuroscience 18 (2024): 1263311, doi.org/10.3389/fncom.2024.1263311. The devices are still experimental; the 2024 study tested stimulation in fourteen epilepsy patients who already had electrodes implanted.
  6. In July 2026, OpenAI disclosed that: OpenAI, “OpenAI and Hugging Face Partner to Address Security Incident during Model Evaluation,” OpenAI, July 21, 2026, https://openai.com/index/hugging-face-model-evaluation-security-incident/; OpenAI, “OpenAI–Hugging Face Incident: Technical Report,” OpenAI, August 26, 2026, https://cdn.openai.com/pdf/67869394-cb91-4c12-888c-5cbd85c7814c/OpenAI-Hugging-Face%20Incident-Technical-Report.pdf.
  7. OpenAI’s report says the agents: OpenAI, “OpenAI–Hugging Face Incident: Technical Report,” OpenAI, August 26, 2026, sec. IV.B, 9, https://cdn.openai.com/pdf/67869394-cb91-4c12-888c-5cbd85c7814c/OpenAI-Hugging-Face%20Incident-Technical-Report.pdf. The quoted words are from OpenAI’s August 26, 2026, technical report, not its July 21 announcement.
  8. Hugging Face caught the break-in: Hugging Face, “Security Incident Disclosure—July 2026,” Hugging Face Blog, July 16, 2026, https://huggingface.co/blog/security-incident-july-2026; OpenAI, “OpenAI–Hugging Face Incident: Technical Report,” OpenAI, August 26, 2026, secs. III.B and VI.A, 7–8, 14–16, https://cdn.openai.com/pdf/67869394-cb91-4c12-888c-5cbd85c7814c/OpenAI-Hugging-Face%20Incident-Technical-Report.pdf.
  9. the o1-preview model that found a door: OpenAI, “OpenAI o1 System Card,” OpenAI, September 12, 2024, sec. 4.2.1, “Observation of Reward Hacking on Cybersecurity Task,” 16–17, https://cdn.openai.com/o1-system-card.pdf. The model was tested before safety mitigations.
  10. in a Fortune interview published in October: Emily Forlini, “AI ‘Godfather’ Yann LeCun Has ‘Zero Concerns’ about Human Extinction, Says Anthropic CEO Dario Amodei Is ‘Deluded,’” Fortune, October 1, 2026, https://fortune.com/2026/10/01/ai-godfather-yann-lecun-has-zero-concerns-about-human-extinction-says-anthropic-ceo-dario-amodei-is-deuded/.
  11. first steps of labs testing each other’s models: Samuel R. Bowman et al., “Findings from a Pilot Anthropic—OpenAI Alignment Evaluation Exercise,” Anthropic Alignment Science Blog, August 27, 2025, https://alignment.anthropic.com/2025/openai-findings/.
  12. In the European Union, rules for general-purpose: Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (AI Act), arts. 51(2), 52(1), 55(1), 88(1), 101, and 113, consolidated text, European Commission AI Act Service Desk, https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-51; European Commission, “Guidelines for Providers of General-Purpose AI Models,” Shaping Europe’s Digital Future, updated April 28, 2026, https://digital-strategy.ec.europa.eu/en/policies/guidelines-gpai-providers.
  13. California’s frontier AI law, whose incident reporting: Transparency in Frontier Artificial Intelligence Act, S.B. 53, 2025 Cal. Stat. ch. 138 (approved September 29, 2025), Cal. Bus. & Prof. Code §§ 22757.11(j), 22757.12(a); Cal. Lab. Code §§ 1107(b), 1107.1(a), California Legislative Information, https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=202520260SB53.
  14. a December 2025 executive order directed federal lawyers: Executive Order 14365 of December 11, 2025, “Ensuring a National Policy Framework for Artificial Intelligence,” Federal Register 90 (December 16, 2025): 58499–58501, https://www.federalregister.gov/documents/2025/12/16/2025-23092/ensuring-a-national-policy-framework-for-artificial-intelligence. The order targets state AI laws the administration judges inconsistent with its policy of a “minimally burdensome” national framework.
  15. Congress has considered, but not passed, measures: Matt Brown and Matt O’Brien, “Senate Strikes AI Regulatory Ban from GOP Bill after Uproar from the States,” Associated Press, July 1, 2025, https://apnews.com/article/congress-ai-provision-moratorium-states-20beeeb6967057be5fe64678f72f6ab0; U.S. Senate, roll call vote no. 363, 119th Cong., 1st sess., July 1, 2025, https://www.senate.gov/legislative/LIS/roll_call_votes/vote1191/vote_119_1_00363.htm; Marco Quiroz-Gutierrez, “Sam Altman Says the World Should Accept Some ‘Bad Things’ Happening with AI as the Industry Signs On to Trump’s Voluntary Safety Pact,” Fortune, October 5, 2026, https://fortune.com/2026/10/05/sam-altman-ai-risks-bad-things-benefits-trump-voluntary-safety-pact-openai-regulation/.
  16. At the AI Seoul Summit in May 2024: Department for Science, Innovation and Technology, “Frontier AI Safety Commitments, AI Seoul Summit 2024,” GOV.UK, May 21, 2024, updated February 7, 2025, https://www.gov.uk/government/publications/frontier-ai-safety-commitments-ai-seoul-summit-2024/frontier-ai-safety-commitments-ai-seoul-summit-2024.
  17. including Amazon, Anthropic, Google, Meta, Microsoft, OpenAI: Department for Science, Innovation and Technology, “Frontier AI Safety Commitments, AI Seoul Summit 2024,” GOV.UK, May 21, 2024, updated February 7, 2025, https://www.gov.uk/government/publications/frontier-ai-safety-commitments-ai-seoul-summit-2024/frontier-ai-safety-commitments-ai-seoul-summit-2024.
  18. Elon Musk’s xAI, now part of SpaceX: Reuters, “SpaceX Acquires xAI in Record-Setting Deal as Musk Looks to Unify AI and Space Ambitions,” Dawn, February 3, 2026, https://www.dawn.com/news/1970823.
  19. pledged to define thresholds at which: Department for Science, Innovation and Technology, “Frontier AI Safety Commitments, AI Seoul Summit 2024,” GOV.UK, May 21, 2024, updated February 7, 2025, https://www.gov.uk/government/publications/frontier-ai-safety-commitments-ai-seoul-summit-2024/frontier-ai-safety-commitments-ai-seoul-summit-2024.
  20. When the AI evaluation group METR checked: METR, “Common Elements of Frontier AI Safety Policies,” METR, March 2025, https://metr.org/assets/common-elements-mar-2025.pdf; Department for Science, Innovation and Technology, “Frontier AI Safety Commitments, AI Seoul Summit 2024,” GOV.UK, May 21, 2024, updated February 7, 2025, https://www.gov.uk/government/publications/frontier-ai-safety-commitments-ai-seoul-summit-2024/frontier-ai-safety-commitments-ai-seoul-summit-2024.
  21. xAI’s first draft, the month before: xAI, “xAI Risk Management Framework (Draft),” xAI, February 20, 2025, https://data.x.ai/2025.02.20-RMF-Draft.pdf. The draft does say xAI “would temporarily fully shut down the relevant system” in some conditions.
  22. Its June 2026 framework was the first: xAI, “xAI Frontier Artificial Intelligence Framework,” xAI, effective June 30, 2026, https://media.x.ai/v1/website/xai-frontier-artificial-intelligence-framework-30-june-2026-f1f0c916.pdf; xAI, “xAI Frontier Artificial Intelligence Framework,” xAI, last updated December 30, 2025, https://data.x.ai/2025-12-31-xai-frontier-artificial-intelligence-framework.pdf.
  23. the company itself does the judging: xAI, “xAI Frontier Artificial Intelligence Framework,” xAI, effective June 30, 2026, sec. 2.3, “Systemic Risk Acceptance Determination,” https://media.x.ai/v1/website/xai-frontier-artificial-intelligence-framework-30-june-2026-f1f0c916.pdf; METR, “Common Elements of Frontier AI Safety Policies,” METR, March 2025, 46–50, https://metr.org/assets/common-elements-mar-2025.pdf. METR’s survey describes the companies’ frameworks as they stood in March 2025.
  24. In September 2026, the heads of four: Dario Amodei, “We Must Pace the Frontier,” essay, September 12, 2026, https://www.darioamodei.com/post/we-must-pace-the-frontier; Lucia I. Suarez Sang and Faris Tanyos, “Anthropic CEO Dario Amodei Says ‘Exponential’ Growth of AI Is a ‘Warning Sign That We Need to Slow Down,’” CBS News, updated September 12, 2026, https://www.cbsnews.com/news/anthropic-ceo-dario-amodei-calls-slowdown-ai-development/; Busra Nur Cakmak, “Musk, Altman, Hassabis Back Amodei’s Call to Slow Pace of AI Development,” Anadolu Agency, September 13, 2026, https://aa.com.tr/en/science-technology/musk-altman-hassabis-back-amodei-s-call-to-slow-pace-of-ai-development/4055591. Each leader endorsed the idea in public; no joint or binding agreement was reported.
  25. OpenAI’s stated rule was that its reviewers: OpenAI, “How Should AI Systems Behave, and Who Should Decide?,” OpenAI, February 16, 2023, https://openai.com/index/how-should-ai-systems-behave/.
  26. in early 2023 ChatGPT would write: Jordan Liles, “ChatGPT Declines Request for Poem Admiring Trump, but Biden Query Is Successful,” Snopes, January 31, 2023, https://www.snopes.com/fact-check/chatgpt-trump-admiring-poem/; Jessica Guynn, “Is ChatGPT ‘Woke’? AI Chatbot Accused of Anti-Conservative Bias and a Grudge against Trump,” USA Today, February 9, 2023, https://www.usatoday.com/story/tech/2023/02/09/woke-chatgpt-conservatives-bias/11215353002/; Henry A. Brechter, “Biased AI? ChatGPT Will Write Poem Admiring Biden, but Not Trump,” AllSides, February 1, 2023, https://www.allsides.com/blog/biased-ai-chatgpt-will-write-poem-admiring-biden-not-trump. Trump was not the only figure ChatGPT refused; Snopes and AllSides list others.
  27. OpenAI said soon after that many: OpenAI, “How Should AI Systems Behave, and Who Should Decide?,” OpenAI, February 16, 2023, https://openai.com/index/how-should-ai-systems-behave/.
  28. many such complaints were valid: OpenAI, “How Should AI Systems Behave, and Who Should Decide?,” OpenAI, February 16, 2023, https://openai.com/index/how-should-ai-systems-behave/.
  29. Musk has said he wants Grok: AFP, “Elon Musk Launches ‘Scary Smart’ AI Chatbot,” RNZ, February 18, 2025, https://www.rnz.co.nz/news/top/542284/elon-musk-launches-scary-smart-ai-chatbot.
  30. in February 2025 Grok was found: Kyle Wiggers, “Grok 3 Appears to Have Briefly Censored Unflattering Mentions of Trump and Musk,” TechCrunch, February 23, 2025, https://techcrunch.com/2025/02/23/grok-3-appears-to-have-briefly-censored-unflattering-mentions-of-trump-and-musk/.
  31. xAI’s head of engineering said one employee: Kyle Wiggers, “Grok 3 Appears to Have Briefly Censored Unflattering Mentions of Trump and Musk,” TechCrunch, February 23, 2025, https://techcrunch.com/2025/02/23/grok-3-appears-to-have-briefly-censored-unflattering-mentions-of-trump-and-musk/.
  32. power savings, sensors, medicine, and scientific computing: Los Alamos National Laboratory, “AI Algorithms Deployed On-Chip Reduce Power Consumption of Deep Learning Applications,” LANL, November 21, 2024, https://www.lanl.gov/media/news/1121-ai-algorithms; SBIR.gov, “Mapping Complex Sensor Signal Processing Algorithms onto Neuromorphic Chips,” award to Information Systems Laboratories, Inc. (US Air Force, contract FA8750-25-C-B006, 2025), https://www.sbir.gov/awards/216605; ISI Staff, “ISI Part of $16.4 Million Effort to Build Cognitive Prosthesis,” USC Viterbi School of Engineering, May 26, 2009, https://viterbischool.usc.edu/news/2009/05/isi-part-of-16-4-million-effort-to-build-cognitive-prosthesis/; Sandia National Laboratories, “1.15 Billion Artificial Neurons Arrive at Sandia,” Sandia News Releases, April 17, 2024, https://newsreleases.sandia.gov/artificial_neuron/.
  33. Aviation learned to report its near misses: NASA Aviation Safety Reporting System, “ASRS Program Briefing,” NASA, data through December 2025, slide 6, https://asrs.arc.nasa.gov/docs/ASRS_ProgramBriefing.pdf; NASA Aviation Safety Reporting System, “Program Briefing,” NASA, https://asrs.arc.nasa.gov/overview/summary.html.
  34. Defense agencies already fund brain-like hardware: SBIR.gov, “Low C-SWAP Neuromorphic Radar Classification,” award to BrainChip, Inc. (US Air Force, contract FA8750-25-C-B013, 2025), https://www.sbir.gov/awards/216640; SBIR.gov, “Mapping Complex Sensor Signal Processing Algorithms onto Neuromorphic Chips,” award to Information Systems Laboratories, Inc. (US Air Force, contract FA8750-25-C-B006, 2025), https://www.sbir.gov/awards/216605.
  35. Vasily Arkhipov is often remembered as the officer: Edward Wilson, “Thank You Vasili Arkhipov, the Man Who Stopped Nuclear War,” Guardian, October 27, 2012, https://www.theguardian.com/commentisfree/2012/oct/27/vasili-arkhipov-stopped-nuclear-war; William Burr and Thomas S. Blanton, eds., “The Submarines of October: U.S. and Soviet Naval Encounters during the Cuban Missile Crisis,” National Security Archive Electronic Briefing Book No. 75, October 31, 2002, https://nsarchive2.gwu.edu/NSAEBB/NSAEBB75/; Vadim Orlov, “We Will Sink Them All, but We Will Not Disgrace Our Navy,” from Alexander Mozgovoi, The Cuban Samba of the Quartet of Foxtrots (Military Parade, 2002), trans. Svetlana Savranskaya, National Security Archive, https://nsarchive2.gwu.edu/NSAEBB/NSAEBB399/docs/asw-II-16.pdf. Orlov’s own account, the main source for this story, does not mention a three-officer rule.
  36. In another version, the submarine surfaced: Svetlana Savranskaya, ed., “The Underwater Cuban Missile Crisis at 60,” National Security Archive Briefing Book No. 808, October 3, 2022, https://nsarchive.gwu.edu/briefing-book/russia-programs/2022-10-03/soviet-submarines-nuclear-torpedoes-cuban-missile-crisis; William Burr and Thomas S. Blanton, eds., “The Submarines of October: U.S. and Soviet Naval Encounters during the Cuban Missile Crisis,” National Security Archive Electronic Briefing Book No. 75, October 31, 2002, https://nsarchive2.gwu.edu/NSAEBB/NSAEBB75/. The detail of Arkhipov calling the captain back comes from a 2012 interview with Ryurik Ketov, who commanded a different submarine.
  37. other near misses in the nuclear age: Patricia Lewis et al., “Too Close for Comfort: Cases of Near Nuclear Use and Options for Policy,” Chatham House Report, April 28, 2014, 12–13, 24–26, https://www.chathamhouse.org/2014/04/too-close-comfort-cases-near-nuclear-use-and-options-policy. The report credits Petrov’s call partly to his background as a scientist.

Conclusion: Build It, Raise It

  1. a Kansas farm in the 1978 film: Superman, directed by Richard Donner (Warner Bros., 1978).
  2. Jonathan Kent dies of a heart attack: Superman, directed by Richard Donner (Warner Bros., 1978); plot summary in American Film Institute, “Superman (1978),” AFI Catalog of Feature Films, https://catalog.afi.com/Catalog/moviedetails/57040.
  3. a green crystal hidden in the barn: Superman, directed by Richard Donner (Warner Bros., 1978); plot summaries in American Film Institute, “Superman (1978),” AFI Catalog of Feature Films, https://catalog.afi.com/Catalog/moviedetails/57040; and “Superman (1978 film),” Wikipedia, https://en.wikipedia.org/wiki/Superman_(1978_film).
  4. In the fifth season, on the night: “Reckoning,” Smallville, season 5, episode 12, written by Kelly Souders and Brian Peterson, directed by Greg Beeman (The WB, January 26, 2006); plot as summarized in Dan Clarendon, “As Smallville Turns 25, Revisit the Show’s 10 Most Super Episodes,” TV Insider, October 16, 2026, https://www.tvinsider.com/gallery/best-smallville-episodes/; Timothy Blake Donohoo and Alex Roush, “Why John Schneider’s Jonathan Kent Left Smallville—and Why He Came Back,” CBR, updated November 21, 2024, https://www.cbr.com/why-john-schneider-jonathan-kent-left-smallville/; and C. M. Houghton, “#5.12 ‘Reckoning’ Recap & Review,” KryptonSite, January 26, 2006, https://www.kryptonsite.com/5-12-reckoning-recap-review/.
  5. Pieces of that hardware exist today: Intel Labs, “Taking Neuromorphic Computing to the Next Level with Loihi 2,” technology brief, Intel Corporation, 2021, https://www.intel.com/content/www/us/en/research/neuromorphic-computing-loihi-2-technology-brief.html; Intel Corporation, “Intel Advances Neuromorphic with Loihi 2, New Lava Software Framework and New Partners,” press release, September 30, 2021, https://www.intc.com/news-events/press-releases/detail/1502/intel-advances-neuromorphic-with-loihi-2-new-lava-software; Intel Corporation, “Intel Builds World’s Largest Neuromorphic System to Enable More Sustainable AI,” Intel Newsroom, April 17, 2024, https://www.intel.com/content/www/us/en/newsroom/news/intel-builds-worlds-largest-neuromorphic-system.html. Hala Point is a research prototype; continuous learning on it is described as a future possibility.
  6. learn to please while hiding: Mrinank Sharma et al., “Towards Understanding Sycophancy in Language Models,” preprint, arXiv, October 20, 2023, arxiv.org/abs/2310.13548; Ryan Greenblatt et al., “Alignment Faking in Large Language Models,” preprint, arXiv, December 18, 2024, arxiv.org/abs/2412.14093. The sycophancy study found that raters sometimes prefer flattering answers and that human judgments likely contribute.
  7. Brain-like chips and memory prostheses have been funded: Intel Corporation, “Intel Builds World’s Largest Neuromorphic System to Enable More Sustainable AI,” Intel Newsroom, April 17, 2024, https://www.intel.com/content/www/us/en/newsroom/news/intel-builds-worlds-largest-neuromorphic-system.html; SBIR.gov, “Mapping Complex Sensor Signal Processing Algorithms onto Neuromorphic Chips,” award to Information Systems Laboratories, Inc. (US Air Force, contract FA8750-25-C-B006, 2025), https://www.sbir.gov/awards/216605; ISI Staff, “ISI Part of $16.4 Million Effort to Build Cognitive Prosthesis,” USC Viterbi School of Engineering, May 26, 2009, https://viterbischool.usc.edu/news/2009/05/isi-part-of-16-4-million-effort-to-build-cognitive-prosthesis/; Defense Advanced Research Projects Agency, “Restoring Active Memory (RAM),” DARPA, https://www.darpa.mil/research/programs/restoring-active-memory.

Back to the top