arXiv: When Do PEFT Adaptations Leak Structure? Measuring Black-Box Structural Bounds in Public-Base Model Services

A new research paper, published on arXiv, examines how parameter-efficient fine-tuning (PEFT) methods used to adapt large language models can inadvertently leak structural information about the underlying training data, even when the model is accessed as a black-box service. The study demonstrates that attackers can infer the boundaries and composition of the private datasets used to fine-tune a public base model, potentially exposing sensitive business logic or proprietary data characteristics. This is not a vulnerability in a specific product but a general property of how PEFT adapters interact with base model weights, meaning the risk applies broadly to any organization deploying fine-tuned models via API or hosted platforms.

The affected organizations include any EU-based company offering or using AI services built on public foundation models—particularly in finance, healthcare, legal tech, and customer analytics—where fine-tuning on proprietary data is common. Cloud providers and AI-as-a-service vendors that host such models also fall under scope, as they may be liable for insufficient safeguards. This paper does not introduce a new regulation, but it signals that the European AI Act’s requirements on transparency and data governance may need to be interpreted more strictly for fine-tuned models, especially regarding the duty to prevent reverse engineering of training data.

Compliance teams should immediately review their model deployment pipelines to assess whether PEFT adapters are used and whether any contractual or technical measures exist to limit probing of model outputs. They should update their AI risk registers to include this new leakage vector, and consider adding rate limits, output perturbation, or differential privacy techniques to mitigate black-box inference. Finally, they should monitor the paper’s follow-up work and engage with technical teams to document whether their current fine-tuning practices could expose client data, preparing for potential audits under the AI Act’s high-risk classification.