The sentence that gets misread
FDA finalised its Computer Software Assurance guidance on 24 September 2025, three years after the draft, and reissued it in February 2026 for the Quality Management System Regulation — the reissue superseding the 2025 text and carrying the longer title, Computer Software Assurance for Production and Quality Management System Software. Read the coverage of either and you would think the agency had blessed the unattended validation pipeline.
It did not. What the guidance does is widen the toolkit. It supplements the 2002 General Principles of Software Validation — superseding Section 6 of it — and it explicitly allows a mix of scripted and unscripted testing, including exploratory testing, and leveraging supplier or developer evidence where that evidence is appropriate. It also contemplates the data a system collects about itself after implementation, for monitoring and detecting anomalies — which is a different thing from a test suite, and worth not confusing with one. The point of the document is to stop teams spending their validation effort uniformly, and to push it toward the things that would actually hurt a patient if they broke.
That is a reallocation argument. It is not an automation argument. The two get conflated because both of them promise less paperwork, and only one of them delivers it.
A note on scope before going further, because it decides how much of this applies to you. CSA is formally a device guidance, issued by CDRH and CBER, aimed at software used in medical device production or the quality management system, and written to help manufacturers comply with 21 CFR Part 820. It is not a pharmaceutical GMP guidance and it does not amend Part 211. Pharma and biotech sites adopt the method anyway, because their own predicate rules already require a risk-based approach and because the reasoning travels; the 2002 General Principles of Software Validation it supplements is device-scoped for the same reason. Read what follows as method, not as a rule that binds a pharma QMS.
We went looking for the automation ceiling in our own validation evidence, expecting to find a tooling problem. We found something more stubborn.
The ceiling is not a tooling problem
The intuition is that any test a human runs by hand is a test a machine could run instead, and the only thing between you and a fully automated pack is engineering time. Spend the sprints, write the harness, and the ceiling goes up.
It does not, and the reason is structural. A validation pack is not a pile of test results. It is an argument, and it has a shape:
An intended-use statement — what this system is for, in this process, at this site. A risk determination — what is critical, what is not, and on what basis, reasoned from that intended use. Evidence that the system does that thing. A conclusion — signed by a named human — that the evidence supports the intended use.
That order matters, and it is the order the guidance uses. Starting from risk instead of intended use is the old habit CSA is written against: you cannot say what is critical until you have said what the thing is for.
Automation does most of its work inside the third item. It can contribute to the others — a risk register is a document, and machine-collected data can inform a judgement — but it does not make the call in any of them. Intended use, the risk determination and the signed conclusion are what an inspector reads first.
So the ceiling is not set by your test framework. It is set by the proportion of your pack that is execution rather than judgement. On most systems that proportion is smaller than teams expect, and it does not grow when you buy better tooling.
Where automation genuinely wins
This is not an argument against automating. Inside its range, automation is not merely helpful — it does something humans cannot.
Regression breadth. The functional paths that must still work after every release. A human runs these once per validation cycle because that is what a human can afford. A suite runs them every build. That is not the same test performed more cheaply; it is a different and better assurance claim. It is also scripted testing, automated — which the guidance explicitly contemplates for features whose process risk is not high. Do not confuse it with the post-implementation monitoring the guidance discusses separately; that is the system reporting on itself in use, not a suite re-running a specification.
Combinatorial coverage. Role and permission matrices are the clearest case. Six roles against twelve object types against four actions is 288 assertions, and no one is hand-testing 288 assertions on every release. Automated, it is a few seconds and it is complete.
Deterministic rules. Field validation, date handling, mandatory-field enforcement, calculation checks. Unambiguous pass conditions, no interpretation, high repetition. Ideal.
Environment and configuration drift. Confirming that test matches production in the ways that matter. Tedious, error-prone by hand, trivial to assert in code.
What unites these is that the pass condition was fully specified in advance. That is the actual boundary. Wherever you can write down what "correct" means before running the test, a machine can check it. Where correctness only becomes clear once you see the result, it cannot.
What an automated result cannot tell an inspector
Four things, and they are the four that decide inspections.
It cannot tell them the test was worth running
An automated suite reports what it was told to check. It has no view on whether the checks match the risk. A pack can be green across a thousand assertions and silent on the one workflow that touches batch release, and the report looks identical either way.
This is the failure CSA is aimed at — effort spread evenly instead of concentrated where harm lives. Automation does not fix it. Automation makes even spreading cheaper, which means it can quietly make the problem worse while the metrics improve.
It cannot own the conclusion
This one is widely misstated, so be precise about it. ALCOA+ does not forbid system-generated records: data is attributable to the person who generated it or, where it is generated automatically, to the system that produced it. An automated run is perfectly good objective evidence, and the guidance itself points at system logs and audit trails as exactly that.
What does not come out of the run is the decision. Concluding that the evidence supports the intended use is a quality judgement, and a quality judgement belongs to a named person.
That is not fatal, and it is not an argument against automated evidence. But it means the automated result cannot be the end of the chain. Somewhere a named human has to review it and own the conclusion, and that signature is the record. Teams that skip this step end up with thousands of results and no attributable decision — a large pile of data that does not amount to evidence.
It cannot investigate its own failure
A red test is the beginning of the work, not the end. Is it a real defect, a flaky assertion, an environment problem, or a test that encoded the wrong expectation? Every one of those has a different disposition, and picking between them is diagnosis.
This is where automated validation programmes most often quietly fail. Not because the tests do not run — they run beautifully — but because the failure triage never gets built, and a suite whose reds are routinely waved through as flaky is worse than no suite. It produces green reports that no longer mean anything, and it produces them at scale.
It cannot do the unscripted part
The CSA guidance's most useful move is legitimising unscripted and exploratory testing: a knowledgeable person using the system with intent, looking for what the specification did not anticipate.
As the guidance defines it, that is a person using the system with intent — you cannot script the discovery of the thing nobody thought of. Techniques like fuzzing do surface unspecified failures, but they explore an input space, not a process. And it tends to be where the interesting defects are, because the scripted paths are the ones the developers were already thinking about.
So what is the honest number?
We are not going to publish ours as a benchmark, because a single figure from one product at one moment tells you nothing about yours, and validation is not a domain where borrowed numbers are harmless.
What we will say is the shape of it. When we measured which of our validation evidence could be produced without human judgement, the answer was a minority — and the automatable share was concentrated almost entirely in regression breadth. The remaining majority was risk assessment, intended-use reasoning, exploratory testing, and the review of exceptions.
If you go and measure this in your own organisation, the useful output is not the percentage. It is the list — which specific evidence turned out to need judgement. That list is your inspection risk, and it does not shrink when your automation improves.
Producing that list is a structured exercise rather than an estimate, and the structure matters more than the tooling. We publish a CSA readiness self-check for exactly this: thirty-six Pass / Gap checks that take one system — or one module of an eQMS — through intended use per feature, the process-risk determination and the reason recorded for it, the scripted and unscripted evidence retained, and where vendor evidence stops. Run it against a system you have already validated. The rows it marks Gap are the rows where your record cannot answer for itself, and those are the rows that stay yours to close whatever your automation coverage reaches.
A two-tier model beats a target
What follows is an operating model, not CSA's own split — the guidance scales rigor by process risk, while this scales by whether a pass condition can be pre-specified at all. Used together: stop treating automation coverage as a number to maximise, and start treating validation evidence as two tiers with different economics.
Tier one — automated breadth. Everything with a pre-specified pass condition. Run it on every release. Optimise for coverage and speed. Accept that this tier proves the system still does what it did last week, which is genuinely valuable and is not the same as proving it is fit for use.
Tier two — human-judged depth. The risk-critical few, identified by the risk assessment, evidenced with scripted tests reviewed by a named person, plus unscripted exploration by someone who understands the process. Optimise for thinking, not throughput. This is where the effort CSA freed up should go — not into automating more of tier one.
The failure mode to avoid is a programme that grows tier one, reports rising coverage, and quietly lets tier two thin out because the dashboard is green. That is a real risk, and it is the specific way that automating validation can leave you less prepared for an inspection than you were before.
The direction of regulatory travel is toward more evidence, not less. The European Commission published a draft revision of Annex 11 on 7 July 2025, alongside a new Annex 22 on artificial intelligence and a revised Chapter 4; consultation closed on 7 October 2025. The draft is substantially larger and restructured, reaching into supplier oversight, identity and access management, and information security management. Confirm its status before citing it in a procedure — a draft is not a requirement. But none of what it adds is the kind of thing a test runner produces.
Before your next validation cycle
Three questions worth putting to your own programme:
- Of your last validation pack, how much was execution and how much was judgement? Not an estimate — go and mark up the actual document set. Most teams are surprised by the ratio and by which items land where.
- When your automated suite goes red, what happens? If the honest answer is that reds get triaged as flaky under release pressure, your automation is producing confidence rather than evidence, and the gap will not be visible until someone external looks.
- Who is the named person on the conclusion? If the chain runs from automated result straight to a closed record with no named human decision in between, the pack is missing its conclusion — an incomplete record, not a green suite, and that is what an inspector will ask for.
If you are choosing between CSV and CSA posture for a given system before you answer any of this, start with the CSV vs CSA decision tree — the ceiling question only becomes tractable once the intended use and risk class are settled.
How Complere handles the part that cannot be automated
We ship a per-module validation pack, VMP through VSR, available for technical due diligence. It is documentation, produced and reviewed by people, covering each module against its intended use.
We are deliberately not telling you that our platform generates your validation evidence, because for the reasons above we do not believe any platform can. What a connected system changes is narrower: when the deviation, the CAPA, the change and the training records are linked rather than filed apart, finding the ones a reviewer asks for takes less digging. That is a reduction in retrieval effort. It is not a reduction in judgement, and it is not evidence the system produces on your behalf.
The judgement stays yours. Vendor evidence shortens the work; it does not replace the site's own assessment of intended use.
