The next in this series of SCI’s free online talks looks at the issue of autonomous vehicle safety.
Professor Siddartha Khastgir will explore why safety resists binary judgment: why it depends on operational boundaries, edge cases, and rare failure modes that no testing regime can fully capture, making an objective determination as much a design and governance challenge as a technical one.
C&I spoke to Professor Khastgir, head of safe autonomy at WMG, University of Warwick, to find out more.
C&I: What makes autonomous vehicle safety such a complicated area?
Khastgir: Safety is an invisible thing. We don’t realise it until we let go of it. As a result, we take it for granted while feeling entitled to it.
For me there are two aspects of safety that need to co-exist in harmony. First, is “engineering safety” that all engineers and policymakers focus on. Top of the challenges for them is that “safety” is hard to define objectively, let alone measuring it. Coupled with it is the challenge of the huge range of situations an autonomous vehicle might face. Second aspect of safety is “communicating safety”, which is an often neglected and underappreciated by our engineering community. This is compounded by some public misconceptions. There is an instinctive public pull toward absolute safety which distorts policy, engineering and public trust.
No technology in this world is 100% safe, be it mobile phones or aeroplanes or household gadgets. We still choose to use and benefit from them. Our work at WMG, University of Warwick, takes a holistic view to safety, generating robust research evidence both in “establishing safety level” and “communicating safety to help build public trust” in autonomous vehicle technology.
How does this play into broader issues of AI safety?
There is a growing “AI fallacy” which assumes that a more capable AI is automatically safer system. Our research has experimentally demonstrated the opposite unless introduced systematically. Autonomous vehicles are one of the real-world tests of safety-critical AI, also known as “Physical AI”. While AI systems and AI agents are exponentially increasing their capabilities, some human oversight remains essential especially for safety-critical applications. Designing a successful human-AI team with clear roles and responsibilities between the AI agent and the human, remains a challenge and an open research question.
While Agentic AI has become a buzzword in today’s commercial world, the level of “autonomy” the agents should have (if at all without human oversight) remains guesswork and unsustainable for a safe and secure society. My personal opinion is that for safety critical applications, we can’t implement AI agents with full autonomy.
The central issue is how to show convincingly that an AI system will behave safely, and the datasets used for its training and testing are representative of their applications. Being able to demonstrate that an AI system proven safe once will stay safe, even as its environment, data or software changes, is an added challenge.
How do you see this evolving over the next few years?
As autonomous technology becomes a reality on London’s roads, public will start experiencing it. The technology has moved from research and trials toward regulated commercial deployment, starting with limited services such as robotaxis in defined areas.
Regulation and industry standards will play a key role in ensuring safe deployment of the technology. For me standards serve three purposes: 1) they help drive industry state-of-the-art; 2) enable innovation and interoperability 3) help prevent the introduction of unsafe technology. In the next years, the balance between these three aspects remains key, as the world experiences a period of rapid technological progress for AI and autonomous driving, robotics, and wider Physical AI applications.
Most importantly, bringing the public along the technology development and deployment journey will be key to its success in the next few years. That is our vision for the Partners for Automated Vehicle Education UK (PAVE UK), country’s first not-for-profit initiative focussed on building public trust and acceptance in selfdriving technologies through accurate and inclusive education and awareness programmes. As Director of PAVE UK, over the past 2 years, we have engaged with 30,000+ members of public through our activities at Cheltenham Science Festival, British Science Festival, Goodwood Festival of Speed and others.
This SCItalks session: Decoding Autonomous Vehicle Safety: The Truth and the Fallacy takes place online on Wednesday 28 October at 4pm, and is free to attend. To find out more and register for this event visit the SCItalks page on the SCI website.
SCI’s SCItalks are given by the most prominent and influential academics, thinkers and industry leaders who are working to translate scientific innovation into societal impact. These talks form part of SCI’s charitable outreach which seeks to create a public conversation about science-based topics in a way that is accessible to all.
Further reading on AI and chemistry
- Creating chemistry that machines can understand
- Ten scientific breakthroughs getting ready to change the world
- Quantum technology is reaching a commercial tipping point for chemicals and pharma
- How woolly mammoths, ancient penguins and AI are helping the hunt for new antibiotics
- AI in pharma: Helping science to find drugs faster
Chemistry & Industry (C&I) magazine reports on the people, the scientific advances and the industrial innovations being harnessed to tackle society's biggest challenges. C&I covers advances in agrifood, energy, health and wellbeing, materials, sustainability and environment, as well as science careers, policy and broader innovation issues. C&I’s readers are scientific researchers, business leaders, policy makers and entrepreneurs who harness science to spark innovation.
Get the latest science and innovation news every month with a subscription to Chemistry & Industry magazine. You can subscribe to C&I here.
Visit Blog