Author - Paul Scharre
When Missiles can see
A Pentagon defense expert and former U.S. Army Ranger explores what it would mean to give machines authority over the ultimate decision of life or death.
Paul Scharre, a leading expert in emerging weapons technologies, draws on deep research and firsthand experience to explore how these next-generation weapons are changing warfare. Scharre is a great thinker who has both on-the-ground experience and a high-level view. He’s a former Army Ranger who served four tours of combat duty in Iraq and Afghanistan. He then went onto a policy role at the U.S. Department of Defense and led the working group that drafted the government’s policy on autonomous weapons. He’s currently a policy expert at the Center for a New American Security, a center-left think tank in DC. He is also a good writer. Scharre writes clearly about a huge range of topics: computer science, military strategy, history, philosophy, psychology, and ethics. He gives you the right grounding to start participating in the debate over where our country should draw the line on these powerful technologies.
Scharre’s far-ranging investigation examines the emergence of autonomous weapons, the movement to ban them, and the legal and ethical issues surrounding their use. He spotlights artificial intelligence in military technology, spanning decades of innovation from German noise-seeking Wren torpedoes in World War II―antecedents of today’s homing missiles―to autonomous cyber weapons, submarine-hunting robot ships, and robot tank armies. Through interviews with defense experts, ethicists, psychologists, and activists, Scharre surveys what challenges might face "centaur warfighters" on future battlefields, which will combine human and machine cognition. We’ve made tremendous technological progress in the past few decades, but we have also glimpsed the terrifying mishaps that can result from complex automated systems―such as when advanced F-22 fighter jets experienced a computer meltdown the first time they flew over the International Date Line.
At least thirty countries already have defensive autonomous weapons that operate under human supervision. Around the globe, militaries are racing to build robotic weapons with increasing autonomy. The ethical questions within this book grow more pressing each day. To what extent should such technologies be advanced? And if responsible democracies ban them, would that stop rogue regimes from taking advantage? At the forefront of a game-changing debate, Army of None engages military history, global policy, and cutting-edge science to argue that we must embrace technology where it can make war more precise and humane, but without surrendering human judgment. When the choice is life or death, there is no replacement for the human heart.
We should spend more time thinking about the implications—positive and negative—of recent progress in artificial intelligence, machine learning, and computer vision. For example, militaries have begun to develop drones, ships, subs, tanks, munitions, and robotic troops with increasing levels of intelligence and autonomy.
While this use of A.I. holds great promise for reducing civilian casualties and keeping more troops out of harm’s way, it also presents the possibility of unintended consequences if we’re not careful. Earlier this year, U.N. Secretary General António Guterres called global attention to these threats: “The weaponization of artificial intelligence is a growing concern. The prospect of weapons that can select and attack a target on their own raises multiple alarms…. The prospect of machines with the discretion and power to take human life is morally repugnant.”
Scharre makes clear from the beginning that he has no problem with some well-bounded military uses of autonomy. For example, he brings you along for a tour of the U.S. Navy’s Aegis Combat System, an advanced system for tracking and guiding missiles at sea. Aegis has a mode of operation in which human operators delegate all firing decisions to an advanced computer (but can override them if necessary). Why would you want to put a computer in charge? If you’re out at sea and an enemy fires 50 missiles at you all at once, you’d be very happy to have a system that can react much faster than a human could.
Army of None also shows that autonomy has great benefits in environments where humans can’t survive or in which communications have broken down. It can be enormously helpful to have an unmanned drone, tank, or sub that carries out a clear, limited mission with little communication back and forth with human controllers.
In addition, autonomous weapons could potentially help save civilian lives. Scharre cites robotics experts who argue that “autonomous weapons … could be programmed to never break the laws of war…. They wouldn’t seek revenge. They wouldn’t get angry or scared. They would take emotion out of the equation. They could kill when necessary and then turn killing off in an instant.”
Despite these and other advantages, Scharre does not want the military ever to turn over judgment to computers.
Scharre ends the book by exploring the possibility of an international ban on fully autonomous weapons. He concludes that this kind of absolute ban is not likely to succeed. However, he holds out hope that enlightened self-interest could bring countries together to ban specific uses of autonomous weapons, such as those that target individual people. He also believes it’s feasible to establish non-binding rules of the road that could reduce the potential for autonomous systems to set each other off accidentally. He also believes we could update the international laws of war to embed a common principle for human involvement in lethal force.
There are no easy answers here. But I agree with Scharre that we have to guard against becoming “seduced by the allure of machines—their speed, their seeming perfection, their cold precision.” And we should not leave it up to military planners or the people writing software to determine where to draw the proper lines. We need many experts and citizens across the globe to get involved in this important debate.
Comments