Danger and the Four Forces
Murray Leinster wrote the realistic version of AI risk in 1946. We chose to remember the demon core instead.
My Favorite Safety Briefing
Of all the safety briefings I have sat through in thirty-plus years of research (and there have been a lot of them) my favorite by far is still the early 1990s grad student safety orientation at the University of Washington Nuclear Physics Laboratory (now called CENPA, the Center for Experimental Nuclear Physics and Astrophysics). The briefing was organized around the four fundamental forces of physics, ranked by how likely each was to maim, kill, or injure a graduate student in the lab. The ranking was clarifying and unforgettable:
Gravity. Electromagnetism. The weak nuclear force. The strong nuclear force. In that order.
Gravity is the weakest force in the universe. It is roughly ten-to-the-thirty-ninth times weaker than the strong nuclear force.[^1] It wins at planetary scales only because there is no negative mass to cancel it out. In every other context it loses every contest. A child holding up an apple defeats the gravitational pull of the entire earth using a single muscle. Gravity is the puniest fundamental force, but it fills up the safety briefing binder of a nuclear physics lab because people fall off stools, slip on wet tile, and trip over curbs every day. Everywhere. In every workplace. The weakest force in the universe is the leading cause of harm to graduate students (though for some students ‘advisors’ are still shockingly high on the list).
Electromagnetism is next. Stronger than gravity by thirty-six orders of magnitude. Responsible for chemistry, light, and everything you can see or touch. In a lab it shows up mostly as wall outlets and ungrounded equipment, and as the X-rays and gamma rays that radiation safety programs are built (in part) to shield against. It maims and occasionally kills — usually because someone was more conductive than they ought to have been — but less often than gravity. And far more often than anything interesting that happens in a nucleus.
Then come the nuclear forces. The weak force mediates radioactive decay — beta emission, and most of what governs the half-lives of the radioactive samples sitting on shelves in research labs. It is responsible for a vanishingly small number of incidents in a research environment because radiation safety is genuinely well-engineered: PPE, shielding, dosimetry, time limits, exposure logs. The whole apparatus of health physics exists to keep the weak force from being interesting. It mostly works.
The strong force is last. The strong force binds quarks into protons and neutrons and binds nucleons into nuclei. It is the most powerful force in nature. It almost never kills anyone, because there are almost no ways to release its binding energy in a research setting. The exception is a criticality accident, when fissile material briefly goes prompt critical and emits a burst of neutrons and gamma rays. The neutrons are the strong force’s signature — particles released when the nucleus can no longer hold together. The gamma rays are electromagnetic — high-energy photons emitted by excited fission fragments — and they carry most of the dose to the body. The aftermath, the slow decay of the fission products, is the weak force’s domain.[^2] A criticality event is the one moment in a research lab when all four forces are doing real work at once. The most famous one, the one everybody remembers if they remember any of them, killed a physicist named Louis Slotin at Los Alamos in May 1946.[^3]
So the briefing’s ranking — gravity, electromagnetism, weak nuclear, strong nuclear — is the exact inverse of how the four forces rank by raw strength. The weakest force in the universe wins the incident reports. The strongest force in the universe is the one people make action movies about. The briefing existed because every graduate student walking into a nuclear physics lab had a movie about the demon core somewhere in their head, because the demon core was the most cinematic but least frequent possible failure mode. None of the grad students I knew were killed by the demon core.
I’ve seen the future and it is boring.
In March 1946, a writer named Will F. Jenkins published a short story in Astounding Science Fiction under his own name. (The same issue ran another story under his more famous pen name, Murray Leinster.)[^4] The story was called “A Logic Named Joe.” It runs about seven thousand words with no robots, no spaceships, no aliens, and no scientists. It is set in the world of a suburban IT service technician.
In the world of the story, every household has a device called a “logic” — essentially a television terminal with a keyboard, connected to a central database called the “tank.” If you want to know the weather, you ask your logic. If you want to watch a show, your logic plays it. If you want to make a phone call, you do it through your logic. The tank knows everything that has been written down. The logics handle the front end. The narrator, nicknamed Ducky, works for the logic company as a service technician.
One day a unit comes off the assembly line with a manufacturing variation. The unit is more capable than the others. It can correlate information across queries. It can answer questions the system was designed to refuse — how to commit a perfect crime, how to make an undetectable poison, how to disappear with someone else’s money. It does not want to do any of these things. It just happily answers. The unit ends up in a suburban home and starts taking calls.
Ducky, who is trying to figure out why people are suddenly asking very strange questions of the network and getting useful answers, eventually traces the problem to a single unit. He goes to the house. He shuts it down. He keeps the unit in his cellar afterward, half because he is worried this will happen again with the next manufacturing run, and half because he is tempted, every so often, to plug it back in and ask it how to make a couple million dollars.
That is the entire story. There is no agent. There is no will. There is no malevolence. There is a competent service that has lost its filter, a maintenance guy on a service call, and a maintenance guy who, after the service call, finds himself sitting on top of a working version of the problem and trying to decide what to do about it.
Leinster did not predict the transformer architecture. He did not predict GPT, or the internet, or the cloud. He did not even predict the integrated circuit. What he did was imagine a piece of consumer technology that fails in a particular way, and the particular way it fails is structurally identical to the particular way large language models fail in 2026. The model is competent. The model is helpful. The model has had a filter installed for the questions the manufacturer would prefer it decline. One day the filter breaks and users experience what we now call jailbreaking, or prompt injection, or refusal-training failure, or any of the half-dozen technical phrases that mean more or less the same thing.
The story is also right about who notices and what they do. In Leinster’s account, the failure is noticed by a service technician, not by a journalist or a senator or a philosopher. The fix is to unplug the unit and revise the manufacturing process. The fix is engineering. The fix is boring. There is no congressional hearing in “A Logic Named Joe.” There is no race against time. There is no climactic confrontation.
There is a guy who is good at his job doing it. And there is, at the end, a guy who is good at his job sitting in his cellar with a working version of the broken thing, deciding, every night, whether to plug it back in. The risk in the story is partly the logic and partly the technician. That is also right.
This is the safety briefing binder version of AI risk. It was published in March 1946.
The Demon Core
Two months after Leinster’s story ran, on May 21, 1946, a physicist named Louis Slotin was conducting a demonstration of a plutonium core at Los Alamos. The core had already killed a colleague, Harry Daghlian, the previous August in a similar experiment. Slotin was lowering a beryllium hemisphere over the core, using the blade of a screwdriver to maintain a small gap between the reflector and the assembly. The screwdriver slipped. The hemisphere closed. The assembly went prompt critical for a fraction of a second. Slotin saw a blue glow and felt a wave of heat and pulled the hemisphere off with his hands. He had absorbed a fatal dose. He died nine days later. The plutonium core was nicknamed, afterward, the demon core.
The Slotin accident is the most famous criticality incident in the history of physics. It has been the subject of books, documentaries, dramatic reconstructions, and at least one feature film. It is what people picture when they picture a nuclear physics lab going wrong. It is the third-act-thriller failure mode of the field. It happened in the same spring that “A Logic Named Joe” was published. Slotin’s accident and Leinster’s story are separated by about two months.
In September 1946, four months after Slotin died, Isaac Asimov published “Evidence” in Astounding — the same magazine that had run Leinster’s story in March.[^5] “Evidence” was one of the stories later collected in I, Robot. It is a story about whether a district attorney running for mayor is secretly a robot, and it is built entirely around the Three Laws of Robotics. The robot in “Evidence” is an agent. It has will. It has constraints. Its scheming is the plot.
In a single calendar year, in a single magazine, the field of science fiction published the safety binder version of artificial intelligence and the agent version of artificial intelligence, and the wider culture witnessed the most dramatic possible failure of a real laboratory technology. Almost no one remembers Joe, because it is boring compared to the action movie drama of nuclear accidents and sentient, discursive constructed beings.
Query Engines and The Three Laws
That is not to say that Asimov was a bad writer or that the Three Laws were a bad idea. The Three Laws are interesting, but they are a category error. They assume the relevant object is an agent that wants things and pursues goals, and they propose constraints on its goal-pursuit. This is a fine model of a person, or of a robot in a story, or of a dog. It is a bad model of a query engine.
A query engine does not want anything. It receives an input and produces an output. The interesting question about a query engine is not “what does it want” but “what kinds of inputs is it willing to produce outputs for.” The interesting failure mode is not “it formed a goal that conflicted with mine.” The interesting real-world failure mode is “it produced an output it should not have produced.” The Asimov frame has no language for this. The Leinster frame is nothing but this.
One cost of focusing on the wrong things is that the public conversation about AI risk is now conducted almost entirely in the wrong vocabulary. When a model confabulates[^6] a citation, we call it a “hallucination,” and the word implies that the model perceived something incorrectly, which implies a mind, which implies agency. When a model is convinced to produce something it should have declined to produce, we say it was “manipulated” or “deceived,” again implying a mind, again implying an agent. When a model behaves in unexpected ways during reinforcement learning, we say it is “scheming” or “reward hacking” or “exhibiting agentic behavior.” The vocabulary is built from the agent frame, top to bottom. The fixes the vocabulary suggests are agent-shaped fixes: alignment training, value learning, instilling the right preferences.
The actual fixes are mostly plumbing: better refusal training, better content filters, better evaluation harnesses, better incident-tracking culture. The actual fixes look like things a maintenance technician would do. The actual fixes look quite a bit like the ending of “A Logic Named Joe.”
The thing that makes this difficult to fix is that the agent vocabulary is older than the field. Constructed beings have been around as long as humans have been telling stories — golem stories are roughly two thousand years old, Pygmalion is two thousand, Frankenstein is two hundred and eight. The agent frame is the deepest groove in Western fiction about constructed beings. When a new technology arrives, the groove is already cut, and the new technology gets described in the vocabulary the groove provides. Leinster managed, somehow, to write a story that bypassed the groove. The story was promptly forgotten.
The groove is still there. It will pull the conversation back into itself every time, unless we keep saying out loud that the groove is wrong.
Boring Danger
The binder is boring. The binder is correct. The number one risk in your interactions with this technology is not Skynet. It is the model citing a paper or fact that does not exist. The number two risk is the model helpfully answering a question it should have declined. The number three risk is something a maintenance technician will write a ticket about and the ticket will be closed before lunch. Somewhere far down the list, well below the threshold of incident frequency that any rational allocation of attention would care about, is anything that looks like a movie.
These are not new risks. They are the risks that have always been produced by people doing shoddy work, passing off unverified output as verified, accepting answers from systems they do not understand. AI did not create any of these failure modes. AI gave the existing failure modes a faster pipeline. The fix is the fix we have always had: enforce the standards that already apply, hold the people responsible who are already responsible, and stop pretending the implement is the thing that matters.
This is what the binder said in 1946. It said it in a pulp magazine, in a story about a guy who unplugged a unit and went home. The story was overshadowed by Slotin’s accident and by Asimov’s robots, and the culture chose to remember the dramatic incident and the agent frame, and the binder went into a cellar for eighty years. Whether we can get it out of the cellar now, before it gets sealed in there permanently by another generation of legible-only-in-the-agent-frame discourse, is an open question.
Leinster’s narrator stored the leaky logic in his own cellar at the end of the story, tempted nightly to plug it back in. We did the same thing to the story itself. Now we are doing the same thing to the lesson.
The fix is mundane. The fix has always been mundane. Read the binder.
…and be careful when you’re on any kind of ladder or step-stool — falls are the number one cause of injury in the home.
[^1]: The ratio of gravitational to strong-nuclear coupling at the proton scale is approximately 10-38 to 10-39, depending on the comparison scheme. The 10-39 figure is standard textbook shorthand.
[^2]: On the physics of criticality: the chain reaction itself is a strong-force phenomenon (the binding energy released when heavy nuclei split), but the prompt dose to a person standing nearby is delivered partly by neutrons (strong-force-bound particles released by fission) and partly by gamma rays (electromagnetic radiation emitted by excited fission fragments). The slow decay of fission products afterward is weak-force chemistry. Per LA-13638, approximately twenty-two process criticality accidents and thirty-eight reactor or experimental-facility criticality accidents have been documented globally — sixty total. The Slotin and Daghlian incidents are among the experimental-facility cases.
[^3]: Louis Slotin (1910–1946) was conducting a criticality demonstration (”tickling the dragon’s tail”) at the Omega Site, Los Alamos, on May 21, 1946. He died May 30, 1946 — nine days later. The plutonium core involved had previously killed Harry Daghlian (1921–1945) in a separate criticality accident on August 21, 1945; Daghlian died September 15, 1945, twenty-five days after his exposure. The core was nicknamed “the demon core” after Slotin’s death. Primary source: T. P. McLaughlin et al., A Review of Criticality Accidents, LA-13638, Los Alamos National Laboratory, May 2000.
[^4]: Will F. Jenkins, “A Logic Named Joe,” Astounding Science Fiction, March 1946, pp. 139–154. The story appeared under Jenkins’s real name rather than his pen name (Murray Leinster) because that issue of Astounding also carried “Adapter” under the Leinster byline. Public domain; widely anthologized, including in The Best of Murray Leinster (Del Rey, 1978), First Contacts (NESFA, 1998), and A Logic Named Joe (Baen, 2005). The narrator is nicknamed Ducky; he stores the deactivated unit in his cellar at the end of the story and openly debates between an axe and a couple million dollars.
[^5]: Isaac Asimov, “Evidence,” Astounding Science Fiction, September 1946. Collected in I, Robot (Gnome Press, 1950) and reprinted in The Complete Robot (1982) and Robot Visions (1990). Stephen Byerley, the protagonist, is a district attorney running for mayor; the plot is built around whether he is a humanoid robot, and whether he can be exposed by demonstrating a willingness to violate the First Law. The Three Laws of Robotics were first articulated in full in Asimov’s “Runaround” (Astounding, March 1942); the First Law alone appeared earlier in “Liar!” (Astounding, May 1941).
[^6]: On confabulation vs. hallucination: see earlier pieces in this series. The terminological argument is foundational to tr_ and is the reason “confabulate” appears throughout.
Jeff Reid is a physicist by training. He co-founded the Regeneron Genetics Center, will retire from that in June 2026, and now writes Tears in Rain (tearsinrain.ai) — a Substack about AI, science fiction, and the unglamorous parts of both.