… A truism of combat is that whoever shoots first wins, and having a drone wait while a human makes a decision can cede the initiative to the enemy. Warfare at its core is a competition—one with dire consequences for the losers. This makes walking away from any advantage difficult.

Experts believe the “man in the loop” is indispensable, now and for the foreseeable future, as a means of avoiding tragedy, says Zach Kallenborn, an expert on killer robots, weapons of mass destruction, and drone swarms with the Schar School of Policy and Government. “Current machine vision systems are prone to making unpredictable and easy mistakes.”

Mistakes could have major implications, such as spiraling a conflict out of control, causing accidental deaths and escalation of violence. “Imagine the autonomous weapon shoots a soldier not party to the conflict. The soldier’s death might draw his or her country into the conflict,” Kallenborn says. Or the autonomous weapon may cause an unintentional level of harm, especially if autonomous nuclear weapons are involved, he adds.

While physical courage may not be necessary to take lives, Kallenborn notes that the human factor retains one last form of courage in the act of killing: moral courage. That humans should have ultimate responsibility for taking a life is an old argument. “During the Civil War folks objected to the use of landmines because it was a dishonorable way of waging war. If you’re going to kill a man, have the decency to pull the trigger yourself.” Removing the human component leaves only the cold logic of an artificial intelligence…and whatever errors may be hidden in that programmed logic.

If autonomous weapons authorized to open fire on humans is an inevitable future, as some armies and experts think it is, will AI ever become as proficient as humans in discerning enemy combatants from innocent bystanders? Will the armies of the future simply accept civilian casualties as the price of a quicker end to the war? These questions remain unanswered for now. And humanity may not have much time to wrestle with these questions before the future arrives by force…

  • solsangraal@lemmy.zip
    link
    fedilink
    English
    arrow-up
    44
    ·
    edit-2
    6 days ago

    LOL we sure are working hard to manifest the worst outcomes of the most far-fetched 80s sci-fi apocalypse movies

    honestly at this point i’m leaning team skynet

  • snooggums@lemmy.world
    link
    fedilink
    English
    arrow-up
    12
    ·
    6 days ago

    I dunno, with how willing humans are to commit atrocities on purpose I’m not sure the unintentional autonomous killing is going to be sugnificantly worse.

    We still shouldn’t do it, but they are really playing up the level headed decision making that doesn’t seem to be that common in reality.

  • Tylerdurdon@lemmy.world
    link
    fedilink
    arrow-up
    9
    ·
    6 days ago

    They just need to centralize the brain in some bullet proof bunker and give it sentience. WCGW?

    So many fun possible futures right now. Which one will be our fate?

  • The Snark Urge@lemmy.world
    link
    fedilink
    English
    arrow-up
    4
    ·
    6 days ago

    having a drone wait while a human makes a decision can cede the initiative to the enemy

    This isn’t a good way to make the point they seem to think they’re making.

  • ColeSloth@discuss.tchncs.de
    link
    fedilink
    arrow-up
    3
    ·
    6 days ago

    If full scale war ever breaks out and everything is on the line, the Geneva convention and altruism and worrying about only shooting enemy combatants will mostly fly out of the window. You think Little Boy and Fat Man were being selective about which people they were turning into radioactive dust?

  • Maggoty@lemmy.world
    link
    fedilink
    arrow-up
    1
    ·
    edit-2
    6 days ago

    While those are great points, what do we do when Canada invades with kill bots that fire as fast and accurately as an online FPS cheat bot and we can’t even get a shot off?

    I don’t think these researchers realize why the military wants a switch to have a weapon go AI. The military absolutely wants people in the loop, but they also want the capability to turn it all over to the onboard computer if need be.

  • AisteruA
    link
    fedilink
    arrow-up
    1
    ·
    6 days ago

    Honestly, I’m all for it. Designate some land where they can fight, and have wars be fought with robots, on robots. Whomever’s team wins, wins the war.

  • Blue_Morpho@lemmy.world
    link
    fedilink
    arrow-up
    1
    ·
    6 days ago

    It’s a nice thought but they referenced land mines. The idea that an AI shooting a soldier could cause a conflict spiral could happen with a land mine. Not to mention humans make mistakes too. A human soldier could the same (and has). Automated sentry guns have already been around for over 10 years.

    • Alexstarfire@lemmy.world
      link
      fedilink
      arrow-up
      2
      ·
      6 days ago

      One of the closest times we came to nuclear annihilation is when a Soviet nuclear detection system malfunctioned and said the US launched a nuke. The human operator determined the system was most likely at fault and they decided not to do anything.

      Neither are infallible but if a computer gets bad data it doesn’t know it’s bad data.