world

It will soon be too late to regulate use of autonomous weapons, warns Red Cross

It will soon be too late to regulate use of autonomous weapons, warns Red Cross

Warning about putting ‘genie back in the bottle’ comes as UN conference fails to agree on regulation instrument The International Committee of the Red Cross (ICRC) and rights groups warned that soon it will be too late to put the “genie back in the bottle” as the development of autonomous weapons appears to outpace efforts to regulate them. The stark warning came as more than 120 member states of the UN Convention on Certain Conventional Weapons concluded a week-long conference in Geneva.

Warning about putting ‘genie back in the bottle’ comes as UN conference fails to agree on regulation instrument

The International Committee of the Red Cross and rights groups have warned that it will soon be too late to put the “genie back in the bottle” as the development of autonomous weapons appears to outpace efforts to regulate them.

The stark warning came as more than 120 member states of the UN Convention on Certain Conventional Weapons concluded a week-long conference in Geneva.

Despite mounting pressure, an agreement to discuss an instrument to regulate autonomous weapons over the next two years fell short of the urgent action rights groups said was needed to address the looming threat.

“We’re starting to see artificial intelligence … being fielded in the battlefield and perhaps also already in weapon systems. If we leave this unregulated, it will be much more difficult to put the genie back in the bottle,” said Dr Cordula Droege, the director of international law, movement and partnerships at the ICRC.

Autonomous weapons are military systems that act without human supervision, often through the use of artificial intelligence or other forms of robotics.

As artificial intelligence has grown exponentially in recent years, militaries and manufacturers have created increasingly advanced weapons systems for both offensive and defensive purposes.

These systems include the use of AI in generating target banks, as was done by the Israeli military in Gaza, as well as semi-autonomous drones in the Ukraine-Russia war, which have “last-mile capability”, where AI takes over in the drone’s final approach to its target.

The president of the ICRC, Mirjana Spoljaric Egger, and the UN secretary general, António Guterres, have called for a legally binding instrument with clear prohibitions and restrictions on use of weapons. In particular, the ICRC has called for a ban on the use of autonomous weapons against personnel, as well as ensuring that weapons systems behave predictably and placing limits on weapons including restricting their use to military objectives.

Experts have said that given the rapid development of autonomous weapons and AI, the failure to create adequate regulations could have dangerous repercussions.

“It’s not about banning all autonomous weapon systems, but to set limits in an international treaty of red lines that should never be crossed,” said Droege. “We’re very close to crossing those red lines, particularly targeting human beings through machines should be a moral red line.”

Other rights groups warned that any delay towards regulation of these weapons risked allowing private industry and militaries to set the norms of how autonomous weapons are used.

Already, autonomous weapons have led to civilian harm. Three civilians were killed by a Russian drone guided by an AI system in July in Zaporizhzhia in south-eastern Ukraine. The drone tried to hit a petrol station, probably its intended target, but instead crashed into a wall, killing three people in a fountain of shrapnel.

In Gaza, Israel has used AI-assisted targeting methods to fill its target banks much faster than humanly possible since the start of the war. As a result, rights groups have said that civilians and people with tenuous links to militant groups were erroneously killed.

In Iran, a US strike that hit an elementary school, killing 156 civilians including 120 schoolchildren, reportedly was a result of faulty intelligence gathered by AI systems.

Anna Hehir, the head of military AI governance at the Future of Life Institute, said further delay to a strong treaty could see “more cases of mass civilian harm that we’ve seen over the past 18 months to two years”, warning that some of these mass casualty events could dwarf what has already occurred. She said manufacturers and militaries were trying to develop “swarm” capabilities, where hundreds or thousands of autonomous drones act at the same time.

“This has the potential for high damage, just because of the sheer mass of systems that would be autonomous. All it takes is for one of them to face algorithmic error and then the entire swarm is utilised and led down a path based on that error … particularly if they’re programmed to target humans.”

There is also a risk of proliferation, as non-state groups increasingly use AI to program their weapons systems, as was the case when the Houthi movement in Yemen used Claude to “vibe code” ballistic missiles.

Armed Islamist groups in Africa’s Sahel region have reportedly used AI tools to improve drone ballistics and for operational planning.

“This is the only thing that is important this week. It’s not really the substance of laws. What’s on the agenda here, after 13 years, is the political will finally present to start to get serious about regulation of autonomous weapons systems,” said a senior diplomat with knowledge of the UN discussions.

  • red
  • cross
  • use
  • weapon
  • genie
  • bottle
  • conference
  • weapons

Summaries and tags are generated automatically. Stay informed with VidiaNews.

Related news

International talks on controlling use of autonomous weapons collapse
technology

International talks on controlling use of autonomous weapons collapse

Week-long efforts fail to establish negotiating mandate on AI warfare after Russian ‘filibustering’ Week-long efforts to agree a negotiating mandate for a legally binding international treaty to control on the use of autonomous weapons collapsed just before 5am on Saturday morning, after all night talks failed to overcome Russian objections. Despite an overwhelming majority of states at the Review Conference on Conventional Weapons at the UN in Geneva wanting to agree a negotiating mandate leading to a legally binding treaty, the need for consensus forced the conference to adopt a much weaker

Read →
China commits to low-carbon electrification and help for others to improve
world

China commits to low-carbon electrification and help for others to improve

Exclusive: Countries at Cop31 are being urged to commit to deriving 35% of their energy from electricity by 2035 Cop31 live – latest updates China is pushing for rapid electrification of its economy using low-carbon energy and will help other countries do the same, a senior member of the Chinese delegation at the Cop31 UN climate summit has said. Wang Yi, a professor of energy and environment policy at the Chinese Academy of Sciences and a senior adviser to China’s government, said: “End-use electrification is a very clear direction.

Read →
UK and France believed to have vetoed UAE choice for UN secretary general
world

UK and France believed to have vetoed UAE choice for UN secretary general

Costa Rican diplomat remains frontrunner but needs to win over two more permanent members of security council The UK and France are believed to have effectively vetoed a controversial United Arab Emirates diplomat in a vote for the next UN secretary general, ostensibly over the UAE’s alleged links to atrocities in Sudan. The search for a new chief diplomat for the world remained deadlocked on Friday after the UN security council’s ninth straw poll once again did not settle on a candidate.

Read →
Russia pummels Ukraine, missile hits Poland, after Putin threat
world

Russia pummels Ukraine, missile hits Poland, after Putin threat

Moscow pounded Ukraine with massive strikes Friday, with Kyiv saying at least seven people were killed in the capital, while neighbouring Poland accused Moscow of “provocation” after two Russian missiles likely landed in the southeast corner of the NATO country. Russia’s Vladimir Putin had vowed a “painful” response to Kyiv’s strikes on Moscow’s Crimea bridge after holding a summit with China’s Xi Jinping and US President Donald Trump.

Read →
China’s Shenzhen Summit Offers Hope for a More Peaceful World
world

China’s Shenzhen Summit Offers Hope for a More Peaceful World

When Chinese President Xi Jinping welcomed US President Donald Trump and Russian President Vladimir Putin to Shenzhen for a historic trilateral summit, the significance extended beyond the presence of three powerful leaders. As permanent members of the Security Council, China, Russia and the United States acknowledged the need for the UN to respond more effectively to global challenges and fulfil its central mission of maintaining international peace and security.

Read →