AI Act and document outsourcing: what to check with an automation provider
AI can accelerate document recognition, classification and routing, but it does not transfer accountability to an algorithm. Here is how to assess a provider before automation enters production.
Since 2 August 2026, competent authorities and the AI Office have been enforcing further obligations under the EU AI Act. This does not mean that every OCR engine, sorting rule or document classifier is automatically high-risk. It does mean that organisations should know where AI operates, what it does, which data it uses and who controls the output.
Outsourced document workflows may combine scanning, text recognition, data extraction, case classification, routing and response generation. The checklist below turns regulatory principles into operational requirements. It is not a substitute for legal analysis of a specific use case.
1. Separate AI from conventional automation
A fixed rule routes information differently from a model that predicts a category. Ask for a technology map showing rules, OCR, machine-learning models and human approvals.
2. Map data, outputs and the impact of error
List document images, OCR text, metadata, case history and any data used to improve the model. Record whether an output is advice or automatically triggers an action. A suggested invoice category needs different controls from an automated rejection or decision affecting a person.
3. Define the parties' roles
Name the process owner, technology provider, model operator, output user, controller and processor. AI Act and GDPR roles are not necessarily identical. Identify subprocessors and processing locations.
4. Require a reasoned risk classification
Ask for the intended purpose and a documented classification. Risk depends on use, not on the word “AI”. Employment, creditworthiness and other listed areas may trigger stricter requirements. Technical mail sorting still needs quality, privacy and security controls even when it is not high-risk.
5. Put model quality into the SLA
| Area | Question | Evidence |
|---|---|---|
| Accuracy | How are classification and extraction errors measured? | Representative test set |
| Exceptions | When is a document sent to a person? | Confidence thresholds and queue |
| Changes | Who approves a new model version? | Version register and regression test |
| Incidents | How are systematic errors detected? | Alerts, escalation and root-cause report |
| Data | Are customer documents used for training? | Contract, retention and legal basis |
6. Design meaningful human oversight
Human-in-the-loop cannot be a ceremonial approval button. Reviewers need access to the original document, an understanding of limitations, authority to correct the output and a clear stop rule. Low confidence, poor scans, new document types and high-impact decisions should trigger review.
7. Connect the AI Act with GDPR
The AI Act does not replace GDPR. Personal-data processing still needs a purpose, legal basis, minimisation, retention, security and rights handling. Confirm whether customer data trains a shared model. Assess whether a DPIA is required and document the origin of test data.
8. Control versions and regression
Behaviour can change after a model, prompt, dictionary, OCR provider or document layout update. Each material change needs a version, scope, test result and rollback option. Test typical, difficult and deliberately unusual documents in every process language.
9. Agree an incident and exit plan
Define when an AI error becomes an incident, who reports it, how automated actions are stopped and how manual processing is activated. The exit plan should cover data, configurations, dictionaries, logs and exception lists.
10. Start with a measurable pilot
- Select one document type and limited volume.
- Create a manually verified reference set.
- Set quality thresholds and review triggers.
- Test poor scans, languages and missing fields.
- Compare time, cost and corrections with the current process.
- Expand only after acceptance criteria are met.
Sources
- Regulation (EU) 2024/1689 - AI Act
- European Commission: AI regulatory framework
- EDPB: artificial intelligence and data protection
FAQ
Does every OCR tool fall under the AI Act?
No. Conventional OCR or fixed rules may not be AI systems. Assess the actual functionality, intended purpose and consequences.
May a provider train its model on our documents?
Only under clearly agreed terms covering purpose, scope, legal basis, retention, confidentiality and opt-out controls.
Is human oversight enough?
Only when it is meaningful: reviewers see source data, understand limitations, can change the result and can stop the process.
Where should a provider review start?
Map AI uses, data, subprocessors and error impact, then define metrics, review thresholds, versioning and incident response.