AI strategy and governance for medical device companies, built around the line that determines everything downstream: which AI sits inside the regulated boundary and which does not.
A model computing a clinical value inside a device, an agent triaging complaints in the quality system, a vision system dispositioning product on the line, and a copilot helping finance draft a memo are four fundamentally different propositions. They differ in approval path, change control, evidence obligation, cost per release and time to deploy by something close to an order of magnitude. Most device companies govern them as one portfolio, which either slows the harmless work to the pace of the regulated work or lets regulated work move at the pace of the harmless.
The question underneath all of it is who is accountable when AI influences a regulated decision. The first strategic act is drawing the line, and your quality function draws it rather than your technology team.
Faced with uncertainty about which AI requires regulatory treatment, many device companies apply the heaviest process to everything. The predictable result is that a marketing team wanting to summarize customer feedback faces the review path designed for clinical algorithms, concludes it is not worth it, and uses a consumer tool instead. The organization now has strong governance over the projects that went through it and no visibility of the ones that did not, which is the opposite of what the policy intended.
The second problem is that the classification question is genuinely difficult, moves with regulation, and belongs to a function that is already fully occupied.
One process for everything produces shadow AI
Teams route around a review path that does not fit their use case, and the organization loses sight of what exists.
The classification question is hard and keeps moving
Where the regulated boundary falls depends on intended use, claims and an evolving regulatory landscape.
Nobody has an inventory
Pilots, vendor features and embedded capability accumulate across functions without a register anybody maintains.
Vendor AI arrives without a decision
A supplier enables a model inside a tool you already licensed, and nobody classified it because nobody chose it.
Pilots demonstrate possibility without establishing readiness
A working demonstration lacks enterprise integration, evaluation datasets, monitoring, change control and incident handling, and none of that is visible in the demo.
Governance exists as a document rather than a decision path
A policy that cannot route a specific use case in a week is a policy teams will work around.
Count the AI already running that nobody classified.
Vendor features switched on inside existing tools, pilots that quietly became production, models embedded in purchased software, and whatever individual teams adopted directly.
Strategy in a regulated environment starts with classification rather than ambition. Until a use case has been placed relative to the regulated boundary, nobody can estimate what it costs, how long it takes or whether it is worth doing, which means every prioritization conversation is a guess.
Step 1
Inventory what exists, including vendor features, pilots and anything embedded in purchased software.
Step 2
Draw the boundary with your quality and regulatory functions, since they own the determination and it shapes everything.
Step 3
Classify each use case by consequence rather than by technology, because the model type is the least relevant property.
Step 4
Design proportionate routes, so the light path is genuinely light and the heavy path applies only where it must.
Step 5
Make the route determinable in days, since a classification process taking months is a process teams avoid.
Step 6
Prioritize within each class against five questions: what measurable problem it addresses, whether you can build and operate it, whether the data is suitable, what happens when it is wrong or unavailable, and who can approve, validate and monitor it.
Step 7
Establish the vendor AI position, because capability you did not build still requires a decision.
Step 8
Define the operating model, including who decides, who builds, who monitors and who can stop something.
Step 9
Set the monitoring obligation per class, which is covered in depth on our AI Governance and Model Evaluation page.
Step 10
Revisit the boundary on a schedule, since regulation moves and a classification made two years ago may no longer hold.
A governance process nobody can navigate in a week is a governance process teams will avoid
The practical test of an AI governance framework is how long it takes a product manager with a specific idea to find out which path applies and what it will require. If the answer involves a committee that meets monthly, the framework will be used for the projects that cannot avoid it and bypassed by everything else. A decision tree somebody can work through in an afternoon, with escalation only for genuinely ambiguous cases, governs more of the portfolio than a thorough process that is too slow to use.
Model evaluation, monitoring design and portfolio-level technical governance are covered on our AI Governance and Model Evaluation page. These are the strategy and operating model capabilities above them.
Find and Classify
AI Inventory
What is running, piloted, embedded in vendor tools or adopted directly, which is the starting point and is almost never available.
Boundary Definition
Working with quality and regulatory to establish where the regulated line falls for your products and claims.
Risk Classification Design
A scheme based on consequence rather than technology, applicable by a product team without escalation in most cases.
Vendor AI Assessment
Capability arriving inside tools you already own, which requires a decision whether or not you commissioned it.
Route and Govern
Proportionate Governance Design
Distinct routes per class, with the light path light enough to use and the heavy path reserved for where it is warranted.
Operating Model
Who decides, who builds, who monitors, who can stop something, and how a disagreement resolves without a standing committee.
Decision Path Engineering
A route a product manager can navigate in days rather than a policy they have to interpret
Human Oversight Design
What oversight means per class, since the word covers everything from an approval gate to an intervention capability.
Sequence and Sustain
Readiness Assessment
Data, platform, skills and governance maturity assessed honestly, since several ambitions depend on foundations that do not exist yet.
Roadmap and Sequencing
What to build, in what order, within each class, with the dependency on data and platform work made explicit.
Build, Buy and Partner Framing
Where capability should be bought, where it must be built, and where the regulated status makes buying harder than it appears.
Governance Sustainment
Reclassification on a schedule, boundary review as regulation moves, and a named owner for the framework itself.
What CaliberFocus does, and does not do?
Governance should be proportionate to your scale and portfolio. For many organizations a lightweight operating model with clear ownership, use case registration and risk-based review is more effective than a large committee.
The same underlying technology appears across these and the governance required differs enormously. The third column is the route, and the point of the table is that most of a device company AI portfolio does not belong on the heaviest one.
| Use Case | What the AI Does | The Route It Belongs On |
|---|---|---|
| Clinical Algorithm in the Device | Computes or informs a clinical value or action | Full regulated path. Everything the product itself requires. |
| Cloud Component Affecting Device Behaviour | Alerting thresholds, derived values, configuration logic | Regulated, and frequently discovered late. Covered on our cloud platform page. |
| Manufacturing Inspection | Dispositions product on the line | Process validation and change control under the quality system. |
| Quality Workflow Automation | Prepares complaints, nonconformances and CAPA records | Controlled process change, with determinations retained by qualified staff. |
| Post-Market Analytics | Surfaces signals from field and complaint data | Supportive. The determination remains a qualified judgement. |
| Regulatory Document Support | Assembles, compares and drafts submission content | Controlled content handling with human authorship retained. |
| Enterprise Functions | Finance, HR, procurement, internal knowledge | Ordinary enterprise governance. The largest category and the lightest route. |
The Cloud Component Is the One That Gets Misclassified
A model inside the device is obviously regulated and everybody treats it accordingly. A cloud service that computes a derived clinical value, sets an alerting threshold or determines what a device does next is equally consequential and is frequently built by a platform team under ordinary software practice, because it looks like infrastructure
AI operating models fail on accountability rather than on structure. The third column is what goes wrong when the role is unassigned, and in most organizations at least two of these have no name attached.
| Role | What They Own | What Happens If Nobody Holds It |
|---|---|---|
| Classifier | Deciding which route a use case takes | Every project negotiates its own treatment and the framework is decorative. |
| Use Case Owner | The business outcome and whether it is still worth having | Pilots persist indefinitely and nothing is ever retired. |
| Technical Owner | The model, pipeline and its ongoing performance | Degradation goes unnoticed until somebody complains. |
| Quality and Regulatory Authority | Where the boundary falls and what validation requires | The hardest question is answered by whoever is least equipped to answer it. |
| Monitor | Whether it is still performing as approved | Approval becomes permanent and nobody revisits the assumptions. |
| Stopper | Authority to suspend a deployment | There is no mechanism to act on a problem once one is found. |
Name people rather than committees
A committee is where accountability goes to be distributed, and an escalation needs somebody who can decide alone.
Make the light route genuinely light
If the lowest-risk path still takes six weeks, the framework will be bypassed and the inventory will be wrong.
Separate the build decision from the classification
Classification establishes what it costs. Prioritization decides whether it is worth that cost.
Define human oversight per class, concretely
The word covers approval gates, verification against evidence, sampling and intervention capability, and these are not equivalent.
Give somebody the authority to stop
A governance framework with no suspension mechanism can describe a problem and cannot resolve one.
Reclassify on a schedule
Scope drifts, regulation moves, and a classification made at inception may no longer describe what the thing does.
Scope drift reclassifies your AI without anybody deciding.
A model built to prioritize a service queue starts being used to triage complaints. A copilot built for internal knowledge gets pointed at customer-facing content. An analytics output built for planning appears in a regulatory submission. Each step is small, nobody reclassifies, and the governance that applies is the one assigned at inception to a different thing.
A device company deploying AI may face obligations from medical device regulation, its quality management system, risk management practice and horizontal AI regulation simultaneously, in several jurisdictions, with overlapping and differently worded requirements. These frameworks were developed independently and do not map neatly onto each other. Establishing which apply to which part of your portfolio is a legal and regulatory exercise, and it has to happen before the architecture rather than alongside it.
Framework applicability
- Engineering controls capable of supporting the frameworks that apply to your organization and products, which may include FDA regulation, ISO 13485, ISO 14971 risk management principles, GxP expectations and the EU AI Act
- Applicability, classification and interpretation determined by your regulatory, quality and legal functions rather than by us
- Jurisdictional differences mapped, since obligations and timing vary by market and a global portfolio meets several
- A named owner for tracking how these frameworks evolve, because this area is moving and a static position ages quickly
Responsible AI in practice
- Bias and performance assessed across the populations and conditions the product will actually encounter
- Explainability designed to the audience, since a clinician, an auditor and an engineer need different things
- Transparency about where AI is involved, including in vendor capability the organization did not build
- Human oversight specified concretely per class rather than asserted as a principle
Data and security
- Training and evaluation data governed to the obligations it carries, covered on our privacy and data governance pages
- Provenance of training data established, since a model is only as defensible as what it learned from
- Vendor and provider relationships disclosed, since customer and notified body due diligence extends through them
- Security of models and the systems around them, covered on our device cybersecurity page
Evidence
- Documentation produced by the process rather than assembled when somebody asks
- Risk management integrated with existing practice rather than running as a parallel AI-specific exercise
- Classification decisions recorded with their rationale, since the question of why something was treated lightly will be asked
- Change history across models, thresholds and scope, since each alters what was approved
Integrate AI risk into the risk management you already do.
Device companies have established risk management practice, and the common error is building a separate AI risk process beside it with its own vocabulary, its own register and its own meetings. The result is two systems that disagree, one of which the organization actually uses
Model evaluation methodology and monitoring design are covered in depth on our AI Governance and Model Evaluation page. What belongs in a strategy conversation is the structural point: every approval was granted on the basis of assumptions about the population, the conditions, the data and the use, and governance means knowing whether those assumptions still hold rather than whether the approval was granted.
The approval assumptions written down
Since a monitoring plan should be derived from the claims used to justify release rather than from a generic template.
Monitoring proportionate to class
A clinical model and an internal copilot do not warrant the same surveillance, and uniform monitoring is as wasteful as uniform approval.
Scope review as a routine event
Because the most common reclassification trigger is a change in use rather than a change in the model.
A retirement path
Models accumulate, and a deployment nobody has evaluated in two years is a liability with an owner who has moved on.
Vendor model change tracking
Where capability sits in a purchased tool, the provider updates it on their schedule and your classification does not travel with it.
An escalation that works
Somebody able to suspend a deployment quickly, since the alternative is a finding with no corresponding action.
Unapproved AI use outside the framework. Intended-use expansion beyond approved scope
Undetected performance degradation. Unauthorized data access. An uncontrolled model update by a vendor or an internal team. Human oversight bypassed or become ineffective. Insufficient audit evidence to reconstruct how an outcome was reached. And a consequential failure that never reached the responsible team.
Derive monitoring from the approval argument
Whatever you claimed to justify release is what needs watching, and a generic metric set rarely matches it.
Treat a vendor model update as a change
Even though you did not make it, since it alters a system you are accountable for.
Review the portfolio, not only the projects
Concentration of risk, duplicated effort and orphaned deployments are only visible at portfolio level.
Make reclassification low friction
If reclassifying is painful, people will argue that the scope has not changed when it plainly has.
AI governance programmes are usually reported as frameworks published and policies approved. The outcomes that matter are whether the inventory is complete, whether the light path is actually light, and whether anybody is still classifying things a year later.
| Category | What We Measure | Why It Matters |
|---|---|---|
| Explain Why Each System Is Allowed to Operate | Whether you have an inventory, know who owns each initiative, have documented intended use, classified by risk, assessed regulatory applicability, defined where human review is required, established acceptance criteria, can reconstruct consequential actions, monitor production performance, control model changes and can suspend or retire a system | If these cannot be answered consistently, the organization needs foundations before expansion. |
| Inventory Completeness | Known AI use cases against those found on audit, including vendor features | A strategy built on the governed minority is a strategy for the wrong portfolio. |
| Time to Classification | Days from a use case being proposed to its route being known | If this exceeds a fortnight, teams will route around the framework. |
| Light Path Velocity | Delivery time for low-consequence use cases | The test of whether governance is proportionate or uniformly heavy. |
| Shadow AI Discovered | Use cases found outside the framework during review | A direct measure of whether the framework is usable. |
| Reclassification Activity | Use cases reclassified after a scope change | If this is zero, nobody is checking what things are actually used for. |
Honest Expectation Setting
An assessment may reveal pilots without business ownership, disconnected AI vendors, unapproved employee use, inconsistent risk assessment, missing inventories, unclear regulatory applicability, weak data governance, insufficient validation evidence, limited human oversight, no production monitoring, uncontrolled model changes or initiatives without measurable outcomes
We will build the inventory including vendor and embedded capability, establish where the regulated boundary falls with your quality function, design proportionate routes with a light path that is genuinely light, assign the governance roles to named people, and test how long classification actually takes.
Bring us a specialty or clinical setting where clinicians are spending too much time creating notes. We will assess where ambient documentation fits, what must remain clinician controlled, how it should integrate with your EHR, and how to measure whether it is actually reducing burden.
One conversation with people who have run these deployments, and a written readiness view you can use with or without us.
- AI Agents and Workflow Automation
- Voice and Conversational AI
- Document AI and Intelligent Processing
- Generative AI and Enterprise Copilots
- AI Strategy and Governance
- HCC and Risk Adjustment Analytics
Security & Compliance
