arXiv: LLM-Based Agents for Software and Systems Security: Approaches, Applications, and Assessment

This publication is a technical research paper, not a new regulation or binding legal instrument. It provides a comprehensive survey and analysis of how large language model (LLM) based agents are being applied to software and systems security, including offensive and defensive use cases, and proposes a framework for assessing their capabilities and risks. The paper is relevant because it signals the rapid maturation of autonomous AI agents in security contexts, which will inevitably attract regulatory scrutiny under the EU AI Act and sectoral rules.

The primary audience is technology vendors, cybersecurity firms, and any organization deploying AI-driven security tools, particularly those in critical infrastructure, finance, and cloud services. Compliance teams in these sectors should treat this paper as an early warning of emerging risk areas, including accountability for autonomous actions, data handling during agent operations, and the potential for dual-use capabilities that may trigger heightened transparency or incident reporting duties.

Compliance teams should first conduct a gap analysis of their existing AI governance frameworks against the agent-specific risks described in the paper, such as unintended privilege escalation or prompt injection attacks. Next, they should update their internal risk registers and model documentation to explicitly cover LLM-based agents, even if not yet deployed. Finally, they should monitor the EU AI Act’s evolving guidance on general-purpose AI and high-risk systems, and prepare to demonstrate that any future agent deployments include human oversight, robust logging, and clear liability allocation. This is a proactive intelligence-gathering exercise, not an immediate action trigger.