The Pattern Is Consistent
A healthcare system hires a clinical informaticist to lead their AI initiative. A bank assembles a team of compliance officers and risk specialists to govern AI adoption. An insurance company purchases an enterprise AI platform that promises domain-specific models and out-of-the-box regulatory compliance.
All three organizations are a year into their initiative and nothing meaningful is in production. The clinical informaticist produced a detailed governance framework. The compliance team documented every possible risk scenario. The enterprise platform passed the vendor’s certification audit. And none of it resulted in an AI system that reliably does useful work at scale.
This is not bad luck. It is a predictable outcome of a fundamental category error. Regulated industries are solving for the wrong constraint first.
Mistake One: Treating Domain Knowledge as the Solution
The logic feels sound: our industry is complex, our regulations are strict, our domain is specialized. Therefore, the path to AI that works here runs through people who understand the domain deeply.
So healthcare organizations put clinical experts in charge of AI architecture decisions. Financial institutions staff AI teams with compliance specialists and former regulators. The implicit assumption is that if you get the domain knowledge right, the technology will follow.
It doesn’t work that way. Domain knowledge tells you what the output needs to look like. It tells you which outputs are compliant, which are dangerous, and which are meaningless. It does not tell you how to build a system that produces the right outputs reliably, at scale, with auditable provenance.
A clinical informaticist who can identify a dangerous medication recommendation cannot tell you whether your retrieval architecture will surface the right clinical context under concurrent load. A compliance officer who knows the SEC’s record-keeping requirements cannot tell you whether your model’s outputs are reproducible enough to meet those requirements. Domain knowledge defines the target. It is not the path to the target.
The practical consequence is that AI initiatives in regulated industries get staffed and governed by people who can evaluate outputs but cannot build the system that produces them. Architecture decisions get made by domain experts who do not understand the infrastructure constraints. Infrastructure decisions get delayed until the domain requirements are “fully defined” - which they never are, because domain requirements in regulated industries are never fully defined.
The result is a governance framework sitting on top of no infrastructure, with no production system to govern.
Mistake Two: Buying Compliance as a Substitute for Architecture
The second mistake is a purchasing decision. The regulated industry sees a vendor platform that claims to be HIPAA-compliant, or FedRAMP-authorized, or SOC 2 certified, or built specifically for financial services, and treats that certification as a proxy for “this will work for us.”
What the certification actually means is narrower than it sounds. It means the vendor’s data handling practices meet certain standards. It does not mean the vendor’s model understands your workflows. It does not mean the platform integrates with your legacy systems. It does not mean the outputs are calibrated to your institutional standards, your edge cases, or your patient population, or your specific product rules.
What regulated industries typically purchase is a wrapper around a foundation model - fine-tuned on generic domain data, certified for data handling compliance, and supported by a sales team that will enthusiastically agree that it handles every use case you describe.
The institutional knowledge problem is the one that sinks these deployments. The vendor’s model was trained on publicly available clinical literature, or generic financial texts, or industry-standard legal corpora. It was not trained on your procedures, your formulary, your underwriting rules, your historical decisions. The gap between “trained on the domain” and “trained on your organization’s version of the domain” is where the hallucinations live and where the liability accumulates.
When the purchased platform produces outputs that are technically within the domain’s norms but wrong for your specific context - and it will - you have limited recourse. The vendor’s roadmap determines when and whether your edge cases get addressed. Your team has no ability to instrument the system to understand why the failure happened. You own the liability but not the architecture.
The Intercom Problem
Here is a useful frame for what is actually happening.
Imagine a hospital that wants to improve internal communication, purchases a state-of-the-art intercom system, and declares the communication problem solved. The intercom is modern, certified, and reliable. It does not know the patients. It does not connect to the lab systems. It does not produce records. It does not learn from the workflows.
The hospital has purchased a genuine tool and applied it to the wrong problem layer. The communication problem was never about the medium. It was about what information needs to move, between whom, with what structure, and with what accountability. An intercom handles none of that.
Enterprise AI platforms in regulated industries function similarly. They are real tools with genuine capabilities. But they are being applied at the wrong layer. The hard problem in regulated AI is not “do we have access to a capable model.” Every regulated industry has access to capable models. The hard problem is: how do you get your institutional knowledge into the system, how do you make the system’s decisions auditable to the required standard, and how do you govern the execution layer so that compliance is structural rather than hoped for.
A certified vendor platform does not solve any of those problems. It provides a model that the vendor governs, on infrastructure that the vendor controls, with an audit trail that meets the vendor’s interpretation of the compliance standard.
What Compliance Actually Is in an AI System
Regulated industries tend to treat compliance as a property of a product - something you achieve by purchasing the right certified platform or hiring the right credentialed people.
In an AI system, compliance is a property of the architecture.
HIPAA compliance in a RAG system means your retrieval layer is scoped so that the model only sees data the requesting user is authorized to see - enforced at query time, not at training time. It means your audit trail captures not just what the model output but what context it retrieved and which documents informed the generation. It means your human-in-the-loop checkpoints are structural, not advisory.
None of that is a vendor’s responsibility. All of that is an architecture decision.
SOX compliance in a financial AI system means your model’s decisions are reproducible - that you can reconstruct the exact inputs, context, and model version that produced a given output on a given date. That requires version control at the model level, at the retrieval index level, and at the prompt template level. A vendor platform that updates its model quarterly, or silently adjusts its retrieval behavior, cannot provide that reproducibility. An architecture you control can.
The same pattern holds across regulated industries. Compliance is a set of requirements on the execution layer. Meeting those requirements is an architecture problem. You cannot purchase your way out of an architecture problem.
The Right Sequence
The organizations that are actually shipping AI in regulated environments at scale have inverted the typical approach.
They build the infrastructure layer first - the routing logic, the retrieval architecture, the observability instrumentation, the audit trail. They design compliance in at the execution layer rather than overlaying it as a policy after the fact. They treat domain knowledge as a retrieval substrate: the institutional knowledge that lives in your procedures, your historical decisions, your documented standards gets encoded into a knowledge layer that the system queries, rather than baked into a model that a vendor manages.
The domain experts then do what they are actually positioned to do well: evaluate outputs, define the edge cases the system must handle, and calibrate the human-in-the-loop thresholds where the model’s confidence is insufficient for autonomous action. That is genuinely valuable work. It is just not the work that should be driving infrastructure decisions.
And the vendor platforms have a role too - as inference endpoints, as access points to frontier models, as managed infrastructure for components that do not require your institutional customization. The mistake is not using them. The mistake is treating them as the solution rather than as a component in a system you architect and control.
The Compounding Risk
The reason this matters beyond the individual failed initiative is what happens when regulated industries get this wrong at scale.
AI systems in healthcare, finance, and insurance are not productivity tools. They are decision-support systems operating in environments where wrong outputs carry regulatory, legal, and human consequences. A system that produces outputs that are plausible but not grounded in your institutional standards - because it was trained on generic domain data and you have no retrieval layer anchored to your actual policies - is not a productivity gain. It is a liability that compounds with every decision it informs.
The domain expertise you hired can evaluate individual outputs. It cannot evaluate the distributional behavior of a system producing thousands of outputs per day. The vendor certification tells you the data handling is compliant. It tells you nothing about whether the outputs are.
Instrumentation, observability, and audit trails are the only tools that let you evaluate a system at the scale it operates. You cannot build those on top of a platform you do not control. You build them into the architecture from the start, or you are operating blind in an environment where operating blind is not an option.
If you are in a regulated industry working out how to build AI infrastructure that meets your compliance requirements structurally - not as an afterthought - start with a free assessment.