Annex 22 and the Rise of AI Governance in GMP

Artificial intelligence is rapidly moving beyond experimentation and into regulated pharmaceutical operations. That shift is exciting, but it also changes the compliance conversation entirely. In GMP environments, AI can no longer exist as a disconnected innovation project without clear controls, documentation, or oversight. Regulators are now making it clear: AI must operate within the same disciplined framework as any other GMP-critical system.

The European Commission’s draft Annex 22 on Artificial Intelligence represents a major milestone in this evolution. Introduced alongside updates to Annex 11 (Computerised Systems) and Chapter 4 (Documentation), the draft establishes expectations for how AI systems should be selected, validated, monitored, documented, and controlled when used in GMP-relevant environments.

The message is straightforward: AI governance is becoming validation governance.

Until recently, AI governance discussions in life sciences focused heavily on broad principles like transparency, ethics, and human oversight. While important, those ideas alone are not enough in regulated manufacturing.

GMP environments require operational evidence.

Quality teams must be able to demonstrate:

• Intended use definitions

• Data governance controls

• Validation evidence

• Performance monitoring

• Change control

• Acceptance criteria

• Human review procedures

• Traceability across the AI lifecycle

Annex 22 formalizes these expectations by treating AI not as a special innovation category, but as a controlled system component that directly impacts product quality, patient safety, and data integrity.

One of the most important aspects of Annex 22 is that it does not stand alone.

The updates to Annex 11, Chapter 4, and Annex 22 together signal a broader regulatory shift toward integrated digital governance. Computerized systems, documentation practices, validation processes, and AI oversight are increasingly being treated as interconnected parts of the same operational framework.

For validation teams, this means AI governance cannot exist as a separate policy document that lives in a forgotten folder. It must become part of everyday validation operations:

• System validation

• Change management

• Test execution

• Supplier oversight

• Documentation governance

• Periodic review

This is also why the concept of AI-Native Validation Infrastructure (ANVI) is gaining attention. AI governance requires connected traceability between requirements, risks, testing, approvals, monitoring, and lifecycle control. Manual spreadsheets and disconnected documents will struggle to support that complexity at scale.

Interestingly, the draft Annex 22 is relatively conservative about which AI models are acceptable in critical GMP applications.

The guidance focuses on:

• Static machine learning models

• Deterministic systems

• Models with predictable outputs

At the same time, it explicitly excludes:

• Self-learning dynamic models

• Probabilistic AI systems

• Generative AI and large language models in critical GMP use cases

This is a significant signal. Regulators are emphasizing predictability, repeatability, and testability over hype.

For non-critical applications, generative AI may still be used, but only with qualified personnel maintaining responsibility for reviewing outputs and ensuring suitability for intended use.

In other words, AI is not being banned. It is being constrained and governed according to risk.

One of the strongest themes throughout Annex 22 is intended use.

AI models cannot be validated abstractly. They must be evaluated in the context of:

• The exact task they perform

• The data they consume

• The environment in which they operate

• The risk associated with their outputs

The draft emphasizes detailed intended-use documentation, including expected inputs, edge cases, rare variations, and limitations.

This has major implications.

A vague statement like “AI assists quality review” is not sufficient in GMP. Organizations must clearly define:

• What the model does

• What it does not do

• What decisions depend on it

• What risks exist if it fails

• What human oversight applies

Without a well-defined intended use, there can be no meaningful acceptance criteria, defensible testing, or reliable validation evidence.

Another major shift in Annex 22 is the treatment of test data.

The draft places strong emphasis on:

• Representative datasets

• Subgroup coverage

• Rare-event inclusion

• Statistical confidence

• Independent test datasets

• Controlled access and auditability

This is critical because AI performance depends heavily on the quality and structure of the data used during testing. A model that performs well on ideal examples may fail dramatically under real-world GMP conditions.

The draft also requires clear separation between training data and test data to avoid biased validation outcomes.

This effectively elevates test data into regulated validation evidence.

Annex 22 also reinforces a core validation principle that many AI programs overlook: acceptance criteria must be predefined.

The draft references metrics such as:

• Sensitivity

• Specificity

• Accuracy

• Precision

• F1 score

• Confusion matrices

But the important point is not the metric itself. The important point is that organizations must define what acceptable performance means before testing begins.

That raises difficult but necessary questions:

• Better than what baseline?

• Acceptable under which conditions?

• Consistent across which subgroups?

• Safer than manual review?

• Equivalent or superior to existing processes?

This level of rigor is what transforms AI governance from a technology discussion into a validation discipline.

The draft also treats explainability as a practical requirement rather than a theoretical nice-to-have.

Organizations are expected to document why models make decisions, using tools such as:

• SHAP

• LIME

• Visual heat maps

This helps ensure models are making decisions for appropriate reasons rather than relying on hidden correlations or irrelevant artifacts.

The guidance also addresses confidence thresholds. Models should not be forced to make decisions when confidence is low. Instead, “undecided” outputs and escalation to human review are encouraged when uncertainty exists.

In regulated environments, that is often the safer and more compliant design choice.

Perhaps most importantly, Annex 22 makes it clear that AI validation is not a one-time event.

Once deployed, models must remain under:

• Change control

• Configuration control

• Performance monitoring

• Drift monitoring

• Periodic review

If systems, inputs, processes, or models change, organizations must evaluate whether retesting is required.

This reflects a broader truth about AI systems: their validated state must be continuously maintained, not simply declared once during implementation.

For validation and quality teams, Annex 22 is an early warning and an opportunity.

Organizations should begin building structured AI governance programs that include:

• AI system inventories

• Criticality classification

• Intended use documentation

• Risk assessments

• Data governance controls

• Test plans and predefined metrics

• Explainability evidence

• Human review procedures

• Change control workflows

• Drift and performance monitoring

The companies that prepare early will be positioned to adopt AI more confidently and defensibly in GMP environments.

Annex 22 signals something larger than a single regulatory update.

It represents the beginning of a new operational reality where AI governance, computerized system validation, data integrity, and lifecycle control all become part of the same connected infrastructure.

The future of AI in GMP will not be defined by the most impressive demo or the smartest algorithm.

It will be defined by who can prove that their AI systems are controlled, validated, monitored, explainable, and trustworthy.

To learn more about how Annex 22, AI governance, and AI-Native Validation Infrastructure (ANVI) are shaping the future of GMP compliance, visit the Validfor website for deeper insights, practical guidance, and ongoing industry analysis. Explore the full conversation around AI validation, digital compliance, and regulated innovation here: https://validfor.com/annex-22-ai-governance-gmp/