The dangers of losing human control over AI, and how could experts from QUB help inform the debate?

By Dáire McConville

Queen’s University Belfast recently hosted a 2-day summit at the Centre for Secure Information Technologies (CSIT), led by the universities very own Professor Máire O’Neill, an expert in the field with the School of Electronics, Electrical Engineering and Computer Science and director of CSIT. The summit gathered industry leaders from around the country to discuss the 2026 theme ‘Cyber security in the AI Era’, and how cyber-security needs to be embedded into the design and delivery of AI to protect society. ^[1] From cybersecurity threats to the risks of military AI, the need for effective safeguards is more pressing than ever.

AI is beginning to filter its way through different industries alike, transforming sectors ranging from financial technology to healthcare and manufacturing, and as a result, systems are being increasingly manipulated through cyberattacks. The European Central Bank (ECB) has recently reported the emerging AI models are enabling attackers to identify software vulnerabilities and develop exploits at alarming rates. AI tools can help attackers combine vulnerabilities, misconfigurations and weak credentials more efficiently. ^[2]

With the rise of automation bias (the human tendency to over-rely on automated systems, or AI recommendations blindly) the lack of personable judgement when AI poses mistakes have the threat to put innocent civilians in danger. ^[3] Senator Craughwell who sits on the Oireachtas Committee on Foreign Affairs and Defence has been outspoken on the issue of military AI. ‘AI can misidentify people, vehicles, or objects especially in poor weather conditions or when adversaries use camouflage and deception, leading to civilian casualties or failed fire.’

So, what exactly is being done to tackle the issue, and should our governments be more vocal in implementing stricter regulations for artificial intelligence?

Professor Máire O’Neill stated ‘As AI systems become more capable and autonomous, security cannot be treated as an afterthought. It must be embedded into the design, development and deployment of these technologies.’ CSET Georgetown has reported on reducing the risks of AI for military decisions, seeking to minimise the likelihood of technical failures, and what precautions governments should implement, including:

• Collaborating internationally to develop mutually beneficial technical safeguards to reduce catastrophic AI failures.

• Clarifying the practices and safeguards implemented to limit the risks of AI systems supporting decision making, such as confidence metrics and safeguards to detect compromise.

The discussions at Queen’s and further afield highlight the need to build safeguards alongside AI’s growing capabilities. As these systems take on greater responsibility, effective security and meaningful human oversight will be essential to protecting the people affected by their decisions.

References:

[1]- https://www.qub.ac.uk/News/Allnews/featured/ai-and-cyber-experts-attend-queens-conference.html

[2]- https://www.nccgroup.com/ai-is-accelerating-cyber-risk-why-the-ecb-is-calling-for-action-and-all-industries-should-take-note/?utm_source=google&utm_medium=cpc&utm_campaign=24331028325&utm_content=200202758319&utm_term=third%20party%20ai%20risk%20banking&keyword=third%20party%20ai%20risk%20banking&adposition=&matchtype=b&sitelink=[name]&campaignid=24331028325&adgroupid=200202758319&adid=827550166508&gad_source=1&gad_campaignid=24331028325&gbraid=0AAAAADlK55X_JYWYIFd81odnu6-2IiQBn&gclid=CjwKCAjwuczWBhA9EiwAYGgEodHSSB0B5k4879zHHOIAZbi7KS2QHTyBGr4gRfOal-4rH_mI0brVHBoCvhkQAvD_BwE

[3]- https://cset.georgetown.edu/publication/ai-safety-and-automation-bias/