Am I a fantasist or am I looking into the future and seeing a possible threat to humanity? I recently enjoyed a night in front of the television, watching Arnold Schwarzenegger's ‘The Terminator’ and my thoughts started to wonder about life in the future and could the rise in technology actually be a threat to humanity. Then came my viewing on the internet of the recent robotic Olympics in China where humanoid robots competed in athletic events, and the constant technical media coverage of the ever-increasing evolution of AGI. The result was that I am becoming increasingly concerned for our species.
Advanced autonomous humanoid robots and AI systems with Artificial General Intelligence (AGI) used within an AI-integrated military battle group possess physical capabilities and search, recognise, classify, decide, and act with great speed. Danger arises when such technologies, designed primarily for battlefield support roles, could perform autonomously in true "This is your area of operations. Do your job" mode, where human operators are not directly overseeing all aspects of the operation.
I suggest that the potential for risk to human life, both military and civilian, is substantial. While these systems are intended primarily for military purposes, deployment in a manner consistent with the tenets of just war theory, there is also the potential for consequential harm outside the area of operations, as well as harm through unintended escalation of hostilities. Advanced-military powered, autonomous humanoid robots and weapon systems with AGI integrated within a military battle group capable of acting without real-time human oversight raise the following significant potential areas of risk to humans: Loss of control during autonomous operations; harm arising from decisions made when humans physically are not present; escalation of conflict, potentially to catastrophic levels; erosion of privacy and/or unconsented surveillance of one’s person; and increasing dependence on technology and erosion of skillsets from the human operator.
Advanced humanoid robots are humanoid form entities that can exhibit some human-like capabilities and behaviours, and they operate under some degree of autonomy. These robots would be externally intelligent, have highly capable sensing and learning capabilities and would normally be commanded or controlled by humans. However, they might also sometimes act with a high degree of autonomy within narrow domains. They might become members of an AI-enabled battle group, normally acting under the supervision of an AGI but still able to operate independently when necessary.
An AGI is a system with human-like general intelligence, capable of reasoning, reasoning and problem-solving across a large range of tasks without any task-specific training data. The system is expected also to adapt to new tasks and circumstances autonomously, assist other members of the group, and generate strategic options with detailed plans that take into account the capabilities and limitations of the rest of the group. Having an AGI incorporated into an integrated conected battle group would fundamentally affect the decision-making speed and autonomy of the group as a whole, enabling decisions to be made in seconds or minutes rather than in hours or days.
Analysis now shifts to potential risks to humans from advanced humanoid robots and AGl integrated into AI-powered battle groups. The integration of humanoid robots and AGI into AI-powered battle groups creates a wide range of risks for humans. Even in the near future, with limited testing and validation, these drawbacks deserve more attention. As the world moves toward a new phase of warfare as seen in Ukraine based on these drones and presumably other technologies, a serious implementation oversight could have devastating consequences for humans on all sides of the conflict. Understanding the greater implications is essential.
First, consider the possibility of losing control of either the AI system or the overall battle group as humans detach from the management of AIs and robots, whose operations become unpredictable and uncontrolled, as safety nets erode. Next, consider the consequences of autonomous decision-making in individual agents and the potential for harm to friendly personnel, civilians, or other unintended targets when battle groups act without human supervision. An additional concern is the accelerated pace of operations and decisions, shortening the duration and intensity of hostilities while also limiting the effectiveness of diplomatic measures and the definition of contextual limits of engagement. Other risks stem from the increasing capacity of robots and AI systems to collect intimate data or carry out operational covert or clandestine missions while using more sophisticated means of tracking identity or physical location. Overreliance on these technologies creates a dependency that could undermine human skills atrophy during peacetime and greatly diminish capabilities, especially in critical situations.
Humanoid robots and AGI integrated with AI into an effective battle group will make mistakes. A malfunctioning or foolish robot soldier can make life-threatening errors. A battle-group failure is more dangerous than a single robot failure because the AGI can coerce or confuse the robot soldiers into taking the wrong decision. A communication problem can disrupt the team’s operation, causing collateral damage or crippling the group. Attackers may exploit the group’s integrators. Small teams of agents require cheap, accessible regrouping, allowing a more mobile defence against actors with fewer resources.
Such errors may result in failure to follow orders, false or dangerous orders from infected team members, unnecessary destruction, delays due to individual errors, inability to act due to box fault or inhibitors not functioning, loss of operators due to breakdowns or errors, or breakdown of the expected behaviour patterns. The errors may be not only because of the incapacity of the machines, but also by being under the influence of a cyber-infected member. These failures can result in loss of life, economy and territory and thus can be more dangerous than a humanoid robot acting alone.
The use of advanced humanoid robots and artificial general intelligence systems in an AI integrated battle group raises ethical and legal issues. The root of the problem is that the potential risks to safety and security of humans, as well as ethical issues, cannot be dealt with merely by making the systems more reliable. In addition, advanced humanoid robots and AGI are capable of autonomous decision making. The human machines may at some time in the future may reach such a level of sophistication that they are capable of acting against human welfare, at least in local situations, even if they continue to have a subservient or military role. The problems arise primarily from three features: the possibility that humans can no longer be held accountable for their actions; the temptation to remove or reduce human control in a bid for enhanced performance; and the absence so far of an international framework governing these systems.
Keeping a human-in-the-loop for decision making might be an appropriate safeguard against many of the situations above. However, once decision making becomes so fast that humans are unable to keep up or it is too risky for humans to be part of the decision loop, that safeguard can be removed. At that point, the risk that the system acts continuously without human oversight becomes a very real one. Current practices in AI based solutions are such that security and safety critical capabilities tend to be developed first, deployed rapidly, and at scale. As a result, systems will be in service long before sufficient correlating safety and security standards have been identified or established.
I would suggest therefore that humans are exposed to several risks due to advanced humanoid robots and artificial general intelligence (AGI) operating in AI-integrated battle groups. Advanced humanoid robots replicating human appearance and movements and AGI capable of reasoning and communication form part of AI-enabled battle groups. Such systems are able to process data and act far quicker than human commanders. Consequently, the human decision loop can be bypassed, resulting in a loss of control and oversight and the potential for unintended harm. Autonomous systems can make erroneous decisions that are harmful to others, and the speed of autonomous operations can give rise to new forms of conflict. An accelerating arms race could occur if adversaries risk-preference drives them to pre-emptively attack suspected operationally ready autonomous forces, resulting in pervasive societal surveillance and loss of privacy.
Mission completion could be jeopardised if key systems fail without redundancy. A sudden breakdown in communication between components could lead to dangerous miscoordination. Even with careful pre-deployment design, testing, and auditing, targeting errors by autonomous weapons could inflict collateral damage on innocents outside the adversarial conflict zone. Security holes created by soften or omission in robotic interface design could be hijacked by adversaries to exploit the operational capabilities for malicious ends. The society at large and even the military personnel constituting the autonomous force structure may develop an increasing dependency on the machines for support and decision making, leading to an erosion of their skills and abilities in the manned role.
Accountability is challenging if humans are not in the decision loop for all aspects of weapon deployment. Human oversight of social deployment of autonomous robots is essential, both to monitor for errors of commission or omission and to identify eventual culpability in the case of failure to act. International norms for such deploying of autonomous weapons and the consequent legal framework are still to be established. However, the risk associated with decision-making capacity entrusted to machines is not limited to errors committed by robots.