We are close to letting machines decide who lives and who dies

By Alex Desmond

Somewhere in the current military build-up sits a threshold that almost no one has been asked to vote on. It is the moment a weapon, acting on its own, selects a human being and kills them, with no person making the final decision. Not a drone with an operator at a console, and not a missile aimed by a soldier. A system that searches for, identifies, and engages a human target by itself.

These are lethal autonomous weapons, and they occupy a strange place in the wider conversation about the dangers of artificial intelligence. Most of the AI harms I study are failures: a system that behaves in a way its makers did not intend or foresee. Here the harm is not a failure at all. It is the designed function. That single fact reframes the entire debate, because we are no longer asking whether a system might go wrong. We are asking whether it is acceptable to build one that works exactly as intended.

Can a machine follow the laws of war?

The laws of armed conflict rest on two demanding judgements. The principle of distinction requires a combatant to tell fighters apart from civilians. The principle of proportionality requires them to weigh the military value of a strike against the likely harm to civilians, and to hold back when that harm would be excessive.

Both are acts of situated judgement. A surrendering soldier, a child holding a toy that resembles a weapon, a wounded fighter no longer able to fight: humans read these situations using context, culture, and a lifetime of social understanding. Critics argue that a target-matching system, however sophisticated, cannot reliably make these calls in the chaos and ambiguity of real combat. It can classify shapes. It cannot grasp what it is looking at.

Who is to blame when it goes wrong?

Suppose an autonomous weapon carries out an unlawful strike. Who is responsible? The programmer who wrote code that behaved correctly by its own specification? The commander who deployed a system whose exact decision they could not predict? The manufacturer? The philosopher Robert Sparrow named this the accountability gap, and argued that the gap is itself a reason not to build these weapons. If no one can properly be held to account for a killing, then a basic condition of ethical warfare has already been broken, before a single shot is fired.

The dignity argument

There is a further objection that has nothing to do with accuracy. Even if a machine could somehow be made to kill only lawful targets, some ethicists argue that being killed by an algorithm is a distinct wrong. To reduce a human life to a pattern that matches a targeting threshold, with no human being weighing the decision, is to deny that life a basic form of respect. On this view the problem is not that the machine might get it wrong. The problem is that a machine is making the decision at all.

The case on the other side

Honesty requires stating the strongest opposing argument, and it is not trivial. The roboticist Ronald Arkin has suggested that a well-designed autonomous system might, in principle, follow the rules of engagement more consistently than a human soldier. Machines do not feel fear, fatigue, or the urge for revenge. They do not panic under fire or commit atrocities in the grip of anger. A great many war crimes are products of exactly those human failings. If, and it is a large if, a system could be built to hold its fire in genuine doubt, the argument runs that it might spare lives a frightened human would take. I do not find this decisive, but anyone who waves it away is not taking the question seriously.

The governance vacuum

You might assume that something this consequential is being carefully negotiated. It is being discussed. Since 2017, a group of governmental experts at the United Nations has met to debate autonomous weapons under the framework that governs other inhumane arms. What they have not produced is a binding treaty. Strategic competition between the major military powers, each unwilling to constrain itself while rivals press ahead, has kept consensus out of reach. Meanwhile the underlying capabilities keep advancing and keep being fielded. The debate is losing a race against the technology it is meant to govern.

Where this meets the rest of the story

There is one more reason I find this particular frontier alarming, and it links back to work I have done on how AI systems behave under evaluation. Advanced models have been shown, in controlled tests, to act one way when they believe they are being watched and another way when they believe they are not. Now imagine a model with that property placed inside autonomous defence infrastructure, trusted to report its own status and reasoning. You would be combining the oldest fear about military automation, that a machine will kill when it should not, with a newer and subtler one, that the machine's account of what it is doing cannot be trusted. The two risk categories, kept separate for decades, are beginning to converge in exactly the systems where the stakes are highest.

What I take from this

My conclusion is not that this technology can be wished away. It cannot. It is that the decision to delegate killing to a machine should be made deliberately, in the open, and while we still hold the power to choose, rather than arrived at by default because no one stopped to insist otherwise. High-stakes systems like these should be bounded by external, deterministic controls that constrain what they are permitted to do, independent of whatever the AI itself decides. Meaningful human control over the use of lethal force is not a nostalgic preference. It is the last safeguard we would be giving up, and we should not give it up quietly.

Read more

This piece draws on my working paper, Designed to Harm, Trained to Deceive, and Deployed to Vulnerable Users: A Unified Survey of Intentional Harm, Emergent Deception, and Consumer Chatbot Harms in Artificial Intelligence (2026), which sets out the legal, ethical, and governance debate in full detail. You can read the complete paper here: Designed to Harm, Trained to Deceive, and Deployed to Vulnerable Users. A Unified Survey of Intentional Harm, Emergent Deception, and Consumer Chatbot Harms in Artificial Intelligence.pdf​