arXiv: From Agent Traces to Trust: Evidence Tracing and Execution Provenance in LLM Agents

This paper, published on arXiv, introduces a technical framework called "Evidence Tracing and Execution Provenance" for Large Language Model (LLM) agents. It proposes methods to systematically record and verify the chain of actions, data inputs, and decisions made by autonomous AI agents during task execution. The core change is a shift from black-box outputs to auditable, traceable agent behavior, enabling regulators and firms to reconstruct how an LLM agent arrived at a specific conclusion or action.

The primary organizations affected are any EU-regulated entities deploying autonomous or semi-autonomous LLM agents in high-risk contexts under the AI Act, including financial services, healthcare, insurance, and legal tech firms. Sectors using AI for automated decision-making, contract review, or customer-facing interactions will need to assess whether their current logging and audit trails meet the new standard of "execution provenance" that regulators may soon expect.

Compliance teams should immediately review their current agent logging practices against the paper’s proposed traceability standards. They should begin mapping existing agent workflows to identify gaps in decision provenance, particularly where agents access external tools or databases. Teams should also engage with technical leads to pilot provenance logging tools and prepare internal documentation that demonstrates how agent outputs can be independently verified, as this will likely become a key audit requirement under future AI safety guidelines.