arXiv: HoF-Bench: Rediscovering Real AI-Discovered CVEs Without Frontier Models

This paper, published on arXiv, presents a new benchmark called HoF-Bench, which demonstrates that open-source, non-frontier AI models can rediscover real-world, previously AI-discovered Common Vulnerabilities and Exposures (CVEs). The key finding is that the capability to autonomously identify and exploit software vulnerabilities is not exclusive to advanced frontier models, but is now accessible to a wider range of AI systems. This effectively lowers the barrier for automated vulnerability discovery and exploitation, shifting the threat landscape.

The primary affected organizations are those in critical infrastructure, software development, and cybersecurity sectors, particularly any entity relying on the assumption that only state-of-the-art AI poses a significant automated threat. Compliance teams in financial services, healthcare, and technology firms should reassess their risk models, as the paper implies that a broader set of actors can now conduct automated vulnerability research. This may impact supply chain security assessments and internal vulnerability management programs.

Compliance teams should immediately review their organization's vulnerability disclosure and patch management policies to account for an increased speed of automated discovery. They should also update their AI governance frameworks to include risk assessments for open-source models used in development or security testing. Finally, teams should engage with their cybersecurity peers to evaluate whether current penetration testing and red-teaming assumptions need to be adjusted in light of this demonstrated capability.