Robots Should Never Carry a Weapon — Ban It in the Software and at the UN, Starting Now

Like a lot of you, my feed this month kept serving me the same unsettling clip: a robot dog somewhere in China, an assault rifle bolted to its back, padding through a training drill while soldiers watch. Then another. Then a humanoid on a firing line. You watch three of these back to back and a very old science-fiction fear stops feeling like fiction. So let me say plainly where RobotTesters stands, because this is the rare topic where I don't think there's a second reasonable side.

Robots should never be allowed to carry or use a weapon. That prohibition should be written into the robots' own software and into international law at the UN — and it should happen now, while the machines still can't decide anything for themselves.

That's the whole argument. The rest of this piece is why I think it's right, why the specific detail you probably noticed in those videos — these don't look like AI — is exactly the reason to act today rather than after the first tragedy, and why software alone or a treaty alone each fails without the other. I'll wear both of my old hats here: the software developer who knows what you can and can't lock down in firmware, and the recovering lawyer who knows what a promise is worth without a court behind it.

First, sort the real from the deepfake

Before we ban anything, we owe it to ourselves to be honest about what's actually on the screen, because the misinformation here is thick — and if we build a serious argument on a fake clip, we deserve to lose it. The armed-robot footage from China falls into three very different buckets.

So the honest version of your gut reaction is: yes, real militaries are strapping real guns to real robots and rehearsing with them — and also, a good chunk of the scariest stuff in your feed is synthetic. Both things are true, and keeping them straight is the price of being taken seriously on the part that matters.

You spotted the important thing: this isn't "AI"

Here's the detail you clocked that most hot takes miss. Watch the credible footage carefully and there is almost always a person with a controller just out of frame. Chinese state media said so itself: at Golden Dragon, the outlet Global Times reported the robot dog was "operated with a remote control," with the machine's own smarts limited to picking a path around obstacles. The same goes for that missile-carrying dog in Riyadh — the vendor is explicit that a human stays in the loop for weapons release. The AI drives; a soldier still pulls the trigger.

That distinction isn't pedantry — it's the entire legal fault line. The US Department of Defense, in its directive on autonomy in weapons (DoD Directive 3000.09), defines a truly autonomous weapon as one that, "once activated, can select and engage targets without further intervention by a human operator." A remote-controlled robot dog where a person decides to fire is, by that definition, a semi-autonomous system — a fancy rifle on legs — not a "killer robot." As far as anyone has verified, no armed ground robot has yet chosen a human target and fired on its own.

People treat that as reassuring. I read it the opposite way. The only thing standing between what we have now and an actual autonomous weapon is a software update. The chassis is done. The gun mount is done. The legs that climb your stairs are done. The single remaining component — swapping the human trigger-finger for a targeting model — is the one part that gets easier, cheaper and more tempting every month. We are not looking at a distant hypothetical. We are looking at a finished car with no driver installed yet, and arguing about whether to write traffic laws.

The industry already drew this exact line — once

The good news is that the robotics industry has already told us where the line belongs. On October 6, 2022, six of the most serious names in the field — Boston Dynamics, Agility Robotics, ANYbotics, Clearpath Robotics, Open Robotics and Unitree — co-signed an open letter titled "General Purpose Robots Should Not Be Weaponized." The core sentence is the one I'd frame on the wall:

"We pledge that we will not weaponize our advanced-mobility general-purpose robots or the software we develop that enables advanced robotics, and we will not support others to do so."

Read that again, because it does half my job for me. These companies didn't just promise to keep guns off their own products — they named the software as the thing that must not be weaponized, and they called on everyone else in the field to make the same pledge. The people who build these machines understand, better than any legislator, that the dangerous part isn't the aluminium. It's the code. This is the same instinct we keep coming back to when we write about the real laws of robotics: safety in a robot is not a vibe, it's layers you engineer in on purpose.

Why a pledge, by itself, is theatre

And yet. A voluntary promise is exactly as strong as the willingness of the least scrupulous party to keep it — which is to say, not strong at all. The proof is sitting inside that very list of signatories. Unitree signed the 2022 letter, and Unitree robots are the ones that keep turning up armed. The Go1 appeared with a launcher on its back at a Moscow arms expo; US Marines bolted a rocket launcher to one in an exercise; and the Golden Dragon assault dog was built on Unitree's civilian Go2 platform — a machine the company sells to the public for under $3,000. Unitree's consistent answer is that it only makes civilian products and can't control what third parties do after the sale. I take the company at its word on intent. That's the whole point: intent doesn't matter once the hardware is loose. A promise doesn't stop a customer with a drill and a bracket.

It gets worse, because even the companies that mean it discover that software is a slippery thing to police. The 2022 letter itself promised to "explore the development of technological features" to prevent weaponization — and, four years on, no such feature was ever actually shipped. There are good reasons it's hard:

So the honest conclusion is uncomfortable for the pure-engineering crowd (my own crowd): you cannot fully solve this in software, and you cannot solve it in software alone. Which is why my proposal has two halves, and needs both.

Half one: ban it in the software — the part I'd build

Put the software developer hat on. "Ban weapons in the robot's software" isn't a slogan; it's a buildable set of defaults that every general-purpose robot should ship with, the way every car ships with seatbelt anchors whether or not you use them:

• Refuse the payload. A robot's motion stack should treat "carry, aim or actuate a weapon" as a hard-blocked class of action — a safety interlock, in the same family as the collision avoidance, geofencing and force limits these machines already run to keep from crushing a hand.
• A real stop button. Europe's new AI Act already forces this logic for high-risk systems: a human must be able to interrupt the machine with a "stop" that brings it to a safe state. Make an always-accessible, tamper-evident kill switch a condition of sale for any legged or mobile robot, full stop.
• Meaningful human control, defined. Not Asimov's poetry — a concrete engineering requirement that a specific, identifiable human is answerable for anything the machine does with force, with logs to prove it. This is the grown-up version of the idea the safety community has been building since 2013, and it's the only kind of "First Law" you can actually compile.

I grew up on Asimov, and I know the temptation to reach for the First Law — "a robot may not injure a human being." But that line was published in 1942, in a short story, and it has never once been real engineering; even Asimov spent his career writing tales about how the Three Laws break. No robot on Earth runs them. What we can do is far more modest and far more useful: make "hold or fire a weapon" the one capability a general-purpose robot is built to refuse, and make that refusal the industry default rather than a heroic exception. It won't stop a determined state actor — but defaults shape 95% of the world, and right now the default is "sure, bolt whatever you like to my back."

Half two: ban it at the UN — the part I'd litigate

Now the lawyer hat, because software defaults are the floor, not the ceiling. The only thing that reaches the determined state actor, the grey-market reseller and the garage modifier alike is binding international law — a rule that a robot may not be weaponized, and that a weapon may not be handed its own trigger, enforceable across borders. The machinery to build that already exists at the UN, and it is stuck.

The relevant forum is the UN's Convention on Certain Conventional Weapons, whose group of governmental experts has been debating lethal autonomous weapons since 2014. The problem is structural: it works by consensus, so any single military power can — and does — veto any binding outcome. A decade of expert meetings has produced a "rolling text" and a set of non-binding principles, but not a treaty. Meanwhile the UN General Assembly, where no one holds a veto, has started voting, and the trend line is worth seeing in one place:

First Committee votes; each was subsequently adopted by the full General Assembly. China has abstained on all three. The most telling data point is the last cell: the US, which voted for in 2023 and 2024, flipped to against in 2025.

Into that deadlock came the clearest call yet. In October 2023, UN Secretary-General António Guterres and the President of the International Committee of the Red Cross, Mirjana Spoljaric, jointly urged states to conclude a legally binding treaty by 2026, stating that "machines with the power and discretion to take lives without human involvement should be prohibited by international law." We are now in the second half of 2026, and no such negotiations have formally begun. That deadline is, to put it kindly, on track to be missed. The advocates — a coalition of some 300 organizations across 70-plus countries — are not short of arguments. They are short of a forum where one government can't switch off the lights.

We already know how to do this — three times over

If "a technical standard and a binding law and outside verification" sounds utopian, notice that it's simply how every durable weapons-control regime already works. We don't have to invent the pattern; we have to copy it.

Why now, and not after something goes wrong

Every argument in this piece bends toward a single, unfashionable word: now. Regulation almost always shows up late — after the crash, after the leak, after the body count makes it politically cheap. This is the rare moment we get to move first, and the reasons are precisely the ones that made those Instagram clips feel less alarming than they should:

The machines are still teleoperated, so we are banning a capability before it exists, not confiscating one that armies already rely on. The fleet is still small, measured in demos and prototypes, not the millions of units that are coming as robots become the defining trend of the decade. And the industry has already said yes to the principle. The gap between a voluntary 2022 pledge and a binding 2026 treaty is smaller today than it will ever be again. China, for its part, is more than capable of moving first on robot governance when it wants to — it already built the world's first system to give every robot an ID card. Weapons should be the next thing that system is used to forbid.

I spent 25 years assuming the robots I read about as a kid would never actually arrive. They're here, and on almost every front I could not be more thrilled — that's the whole reason this site exists. But there is exactly one door I want bolted shut before we walk any further through it. A robot can vacuum your floor, walk your parcels across a warehouse, keep a lonely grandparent company, or carry the ghost of R. Daneel into the real world. It should never, under any flag, in any software build, be allowed to carry a gun. Let's write that down — in the firmware and in the law — while writing it down is still easy.

Frequently asked questions