As Australia aims to implement the U.S.-led Responsible Military Use of AI and Autonomy Agreement—which mandates human oversight in lethal decision-making—a senior government official has candidly acknowledged the complexities involved. Peter Anstee, the first assistant secretary of the Department of Home Affairs’ cyber and technology security policy division, expressed concerns about the feasibility of maintaining human oversight in a landscape increasingly dominated by AI-driven cyber threats.
The primary challenge lies in whether the „man-in-the-loop” model can respond swiftly enough to counter cyber attacks orchestrated by artificial intelligence. Anstee stated, “It’s a good question, and one we’re currently grappling with,” emphasizing the absence of a clear solution. He further noted that defining the parameters of human involvement in military applications poses additional complications. Currently, the principled stance is that a decision maker must always be included within the kill chain.
Experts from the U.S. have long warned about the unique nature of cyber warfare, where reaction times can be measured in nanoseconds, prompting doubts about human effectiveness in rapidly responding to significant threats. Anstee acknowledged that “we haven’t reached the point yet in the cyber domain where real-time offense and defense activities can occur such that human decision makers cannot intervene—except in threat-to-life scenarios, where we still lack satisfactory definitions of risk situations.”
In response to an inquiry about whether Australia’s limited experience in such warfare contributes to this uncertainty, Anstee indicated that “those special thresholds haven’t been significantly defined.”
During a panel discussion at the Australia and Pacific Security conference organized by the German Konrad Adenauer Foundation, Shiri Kreb, a cyber warfare expert from Deakin University, highlighted the complexities inherent in establishing human control over autonomous systems. She emphasized that the relationship between humans and machines is not binary; rather, it exists on a spectrum.
Kreb posed critical questions regarding the capabilities of humans amid the application of AI, particularly in scenarios involving vast drone swarms, stating, “We have a human in the loop, but how can a human respond to targets generated through 300 drones at the very same time?” She argued that the essential inquiry is not whether a human can verify a target but rather what they are capable of achieving within the constraints of their cognitive abilities.
She also raised concerns about AI’s implications for accountability in warfare. “Algorithms—especially AI-based ones—evolve independently, and we simply cannot trace their decision-making processes,” Kreb explained, noting that allowing algorithms to make life and death choices poses significant ethical challenges.
Kreb highlighted another potential downside to leaning on AI: a decline in the skills of intelligence analysts. “One concern that emerged from interviews with intelligence professionals is that we are diminishing our intelligence capabilities,” she stated. “When we permit AI to handle much of the data collection and analysis, we forgo the intelligence work that was traditionally devoted to each target or threat, ultimately eroding our skills in this domain.”
As Australia continues to navigate the integration of AI within its national defense strategy, these discussions underscore the critical balance between leveraging advanced technology and maintaining essential human oversight and expertise in the rapidly evolving arena of cyber warfare.
