AI Security

The TDL 2026 webinar series continued with a focus on AI security which has become a significant issue in AI systems development and adoption. It is also a tool supporting both defenders and attackers. From AI-assisted cyber-attacks to data exfiltration, in attacks on critical infrastructure and in support of widespread fraud, AI security has become a top issue. The panellists discussed AI security attacks, LLM-specific threats and the latest developments in this area as well as AI’s use by defenders in protection of critical infrastructure, improving evidence of issues and detecting attacks better and earlier than is possible without AI.

Summary

The panel discussion focused on AI security challenges and opportunities, with experts exploring how artificial intelligence is transforming cybersecurity threats and defences. The panellists discussed how AI’s rapid adoption creates new security risks, particularly around agentic AI systems and code generation. They examined how traditional security principles apply to AI, including the importance of visibility into AI systems, the challenges of explainability and the potential for AI to both help and hinder defenders. The discussion concluded on a cautiously optimistic note, acknowledging that while attackers currently have an advantage due to faster adoption of AI tools, defenders will eventually catch up with more mature AI security tools and capabilities.

Background

Each participant was asked to comment on their view on the most important topics, challenges and achievement in AI Security. Here are some of topics that were to be addressed.

LLMs and agentic spaces experience very fast development, and, not surprisingly, new security attacks and attack vectors have emerged. Likewise, it is also probable that there is a connection between the increasing autonomy of some AI applications and their evolving threat models. Alongside these considerations is the extent and significance of issues emerging from the scale of data poisoning and malicious training in AI security as well as model theft and inversion. Another example is AI-generated code acting as a source for security flaws which can be a knotty problem to address. Finally, automation provided by AI makes it possible to design and perform attacks earlier, but also gives defenders additional tools to analyse traffic and transactions as well as to mitigate threats.

While most of the focus in AI security is on technical matters, there are also concerns as to whether non-technical issues, such as the erosion of expertise through reliance on AI systems, the difficulties in establishing accuracy, etc, are receiving serious enough consideration. Likewise, in very data-driven environments, data privacy and provenance have to be treated with increasing care in modern AI systems.

Although the challenges in achieving explainability and/or transparency are well known, less apparent is the connection to AI security. There are nuanced approaches that could and should address trade-offs between explainability and interpretability. For example, finding a way to detect and flag inaccurate explanations, bearing in mind the limits of automated AI explainability as well as the application of standard model evaluation metrics which could have an impact on AI security that could be detected and flagged.

Recent analyses found significant vulnerabilities in automated/generated code and clearly these issues have to be addressed and choosing the optimal security architectures for mitigating these challenges. Considerations from a security perspective are the significance of potentially malicious patterns in training data and the practical instances of backdoors insertions in generated code. Issues in code generation, especially in some programming languages, could lead to goal and tool hijacking in agentic AI as well as the prevalence of other types of attacks due to the automation of programming tasks.

Looking to the future of AI security, it is vital that we are cognisant of what emerging risks and threats are becoming visible and consequently what issues in data, architecture, etc. need to be addressed as a first priority and which new approaches to defence are important in AI security. What we all wanted to know was whether the AI security problem is solvable in the near and immediate future.

Clearly there wasn’t enough time to address all the topics and challenges outlined above, which makes it even more imperative that we should meet again soon!

Speakers

The panel of eminent speakers comprised:

· Roderick Kaleho, Agentic AI & Security Consultant, LogicRiver

· Jason Martin, Director, Adversarial Research, HiddenLayer

· Matthew Rosenquist, Chief Information Security Officer (CISO), cybersecurity strategist and founder of Cybersecurity Insights

· Marcel Winandy, Senior Expert Cyber Security Architect, E.ON Digital Technology

· Rupert Young, Chief Product Officer, Maxmind

The session was moderated by TDL strategic advisor, Claire Vishik.

AI Security: Threats and Defences

The webinar began with a discussion on the evolving landscape of AI security threats and defences, highlighting the challenges of securing agentic AI systems, particularly in identity management and the increased complexity of attack vectors. Insights were shared on adversarial attacks and the importance of protecting AI models from data poisoning and adversarial attacks was emphasised. The discussion continued on the need for robust cybersecurity measures in AI systems, while focusing on the role of AI in detecting and preventing security threats. The panellists agreed that AI security is a critical issue that requires ongoing innovation and collaboration to address the growing number of threats.

Agentic Systems in Cybersecurity Threats

The evolution of cybersecurity threats was considered, particularly focusing on agentic systems and their potential as both attack vectors and payloads. It highlighted the challenges posed by agentic botnets and the need for the industry to adapt defensive mechanisms to address these new threats. The importance of considering the limitations and vulnerabilities of agentic systems when deploying them for monitoring purposes was mentioned. The ensuing discussion focused on the broader implications of agentic systems for cybersecurity and the different impacts on security and adjacent fields including accuracy and completeness.

AI Security Governance Challenges

Three key cybersecurity risks related to AI adoption were highlighted: the rapid deployment of AI systems without proper governance, the vulnerabilities introduced by connecting legacy cloud and SaaS systems to agentic AI, and the attackers’ ability to use AI for faster, more scalable attacks. Concerns were raised about the tension between AI’s value and security risks, questioning whether AI is truly encroaching on traditional infrastructure. It was explained that, while there is a race to integrate AI with sensitive data and systems, security is currently losing ground, highlighting the need for better governance and oversight. In fact, due to the architecture of AI systems and applications using LLMs or SLMs (or agents), governance may be the most important concern at this stage of deployment.

AI Security Challenges and Solutions

The security challenges posed by artificial intelligence were mentioned again, noting that while the underlying principles remain similar to traditional security, the rapid adoption and unique capabilities of AI introduce amplified risks. Issues such as prompt injection, excessive access by AI agents and the need for new security controls tailored to AI-specific capabilities were brought up. It was further deemed important to consider organisational aspects, including governance, roles and processes, when implementing AI security measures. The unique challenges AI presents were acknowledged, particularly the ability of agents to generate code and prompts quickly, which requires new approaches in defence strategies and new tools serving defenders.

AI Governance and Security Challenges

The meeting focused on the challenges of governing and securing AI agents, particularly in the context of dynamic and rapidly scaling environments, highlighting the need to balance flexibility and governance, drawing parallels to early security strategies for compilers and low-code/no-code development. The implications of AI on trust and security were discussed, emphasising the emergence of complex multi-party trust issues and the need for rapid development of standards. The group acknowledged the importance of addressing these challenges to ensure the safe and efficient use of AI agents in various applications.

AI Standards and Trust Challenges

The challenges of reconciling the development speed of AI models with the need for standards were raised. Specific reference was made to the issue of trusting AI-generated standards and the unique errors, or “hallucinations,” made by AI agents, which are often misunderstood due to their statistical nature. it was mentioned that a previous attempt to use AI for automating parts of standard writing was not well-received due to its potential to undermine the consensus-based system. The importance of explainability and transparency in AI design was also touched on, particularly how it affects security, though no definitive conclusion was reached.

Challenges in AI Explainability

The challenges and limitations of explainability in AI was discussed, highlighting that, while it is a critical human concept, current techniques may not provide accurate or meaningful explanations, especially for complex models. An emphasis was put on the importance of understanding the data used for training to better comprehend model behaviour. The mismatch between human expectations of reasoning and the current capabilities of statistical models was noted leading to a discussion on the potential for improved explainability through better segmentation of model access and the use of open routers for querying models. Another serious issue in this field affecting security is determining whether the explanation provided is correct.

AI Explainability and Code Security

The meeting then focused on two main topics: explainability in AI models and security concerns related to code generation. The challenges of explaining complex AI models were explained and followed by a suggestion to use constrained models to improve explainability. The security risks of AI-generated code highlight the need for robust AppSec processes and tools to mitigate these risks. It was also proposed to use sandboxing techniques and AI to assist in test-driven development, which could potentially improve code quality.

AI in Cybersecurity: Challenges and Opportunities

The panel discussed the role of AI in cybersecurity, with an emphasis on the importance of visibility in AI security systems as well as AI’s ability to perform tasks humans don’t want to do, such as testing code and proving functionality. it was noted that while attackers currently have a head start in using AI tools, defenders will catch up as more mature AI security tools become available, considered likely by 3Q 2026. The group agreed that AI can be a powerful tool for defenders, but there is a need to focus on vulnerabilities that represent a real risk to confidentiality, availability or integrity of systems

AI in Cybersecurity: Opportunities and Risks

The panel concluded by discussing the rapid adoption of AI tools in cybersecurity, acknowledging both the potential benefits and risks. They highlighted how AI can accelerate processes, transform unstructured data into insights and enable faster response to security threats. Despite concerns about cybersecurity risks, the panellists expressed cautious optimism about the future of AI in cybersecurity, noting the emergence of new tools and marketplaces that could enhance security practices. They emphasised the importance of testing, building a security mindset and enabling security teams to use cutting-edge tools to protect organisations effectively.