arXiv: Are Safety Guarantees in Neural Networks Safe? How to Compute Trustworthy Robustness Certifications

This publication, titled "Are Safety Guarantees in Neural Networks Safe? How to Compute Trustworthy Robustness Certifications," presents a critical analysis of existing methods used to certify the robustness of neural networks against adversarial attacks. The authors demonstrate that many widely used certification techniques can produce unreliable guarantees, potentially giving a false sense of security. They propose a new framework for computing robustness certifications that are provably trustworthy, addressing fundamental flaws in how safety margins are currently calculated. This is not a regulatory mandate but a technical paper that directly challenges the validity of common AI safety verification tools.

The findings primarily affect organizations deploying neural networks in high-stakes, regulated environments, including autonomous vehicles, medical diagnostics, industrial control systems, and financial fraud detection. Any sector subject to the EU AI Act or similar frameworks requiring demonstrable robustness and safety guarantees should take note. Compliance teams in these sectors rely on certification methods to meet regulatory obligations; if those methods are flawed, their compliance evidence may be invalid.

Compliance teams should immediately review their current robustness certification pipelines and verify whether they rely on the methods critiqued in this paper. They should engage with technical teams to assess the impact on existing safety cases and begin evaluating the proposed trustworthy certification approach as a potential replacement. Proactively documenting this technical risk and planning for updated verification methods will strengthen regulatory submissions and reduce exposure to liability from undetected vulnerabilities.