Medical AI: govern the clinical service and its dependencies

Start with a function-level assessment

“Medical AI” is not one regulatory category. Describe what each software function does, its intended users, the population, its input data and the decision affected. Keep administrative support, clinician-facing recommendations, patient-facing advice and diagnostic or treatment functions distinct.

• Healthcare organization: identify the clinical service owner, patient-safety lead, privacy/security reviewers and people responsible for procurement and vendor changes.
• Device manufacturer or software developer: assess product classification, evidence, market authorization and lifecycle responsibilities.
• Certified health IT developer: assess the certification criteria for the Health IT Module and predictive decision support it supplies.

A hospital can have more than one role. Developing or modifying a function calls for a fresh regulatory assessment. Clinical review, FDA marketing authorization and a privacy contract answer different questions; none settles the others automatically.

FDA: when does the software function become a device?

FDA’s current clinical decision support (CDS) guidance explains the statutory non-device exclusion in section 520(o)(1)(E) of the FD&C Act. All four criteria matter. In outline, the function must not analyze a medical image or specified signal/pattern; must work with medical information; must support or provide recommendations to a healthcare professional; and must enable that professional to independently review the recommendation’s basis rather than rely primarily on it. FDA final CDS guidance; guidance text and statutory criteria.

Document how independent review works in the actual setting: what the clinician can see, the basis and limitations supplied, and the time available to evaluate the output. A “clinician in the loop” label alone is insufficient. Patient-facing advice and image-analysis functions do not gain this particular exclusion just because a professional is somewhere in the workflow. Assess the precise function and any other applicable policy. FDA scope and interpretation.

Buyer’s record: ask the supplier for the intended-use statement, regulatory rationale and any applicable clearance, authorization or approval reference for the specific function and version. Confirm what users, settings and populations the supporting evidence covers. Avoid describing an entire AI platform as FDA-approved because one function has an authorization.

Quality systems and changes to the model

The FDA Quality Management System Regulation (QMSR), 21 CFR Part 820, is now effective for manufacturers within its scope. It incorporates ISO 13485:2016 by reference with FDA-specific provisions. A governance service map is not a substitute for a manufacturer’s required quality system or its records. FDA QMSR overview; current regulation.

FDA’s AI device predetermined change control plan (PCCP) guidance describes a plan submitted for review as part of a marketing submission: the planned modifications, methodology for implementing and validating them, and an impact assessment. An authorized PCCP can support specified changes within its scope. It is not a general permission to retrain or modify any medical AI without further review. FDA final PCCP guidance.

The FDA AI-enabled device lifecycle and marketing-submission document remains listed as draft in the official digital-health guidance index reviewed for this page. It offers a useful planning reference for development, validation, transparency and performance monitoring, but its recommendations are not an enacted standalone AI law. Draft lifecycle guidance and status; FDA current guidance index.

Operating practice: record the deployed model and software version, approved scope, local configuration, validation evidence and supplier change notifications. Assign who decides whether an update needs a clinical review, regulatory review, revalidation, rollback or a new submission. Preserve the decision and its supporting evidence.

Give clinical oversight a usable escalation path

The following are practical governance recommendations. Their legal basis and level of formality vary with device status, clinical use, jurisdiction and organizational role.

Device reporting duties differ by actor. FDA identifies hospitals and specified other facilities as device user facilities; physician offices are excluded from that definition. Covered facilities have mandatory reporting duties for certain device-related deaths and serious injuries. Manufacturers have separate duties. Establish a reporting owner and consult the event-specific deadlines; a governance comment is not a regulatory report. FDA mandatory reporting requirements.

HIPAA: follow the data across the service boundary

HIPAA applicability depends on covered-entity and business-associate status and the information and activities involved. Where a cloud provider creates, receives, maintains or transmits electronic protected health information on a covered entity’s behalf, an appropriate business associate agreement and the other HIPAA requirements are needed. Encryption alone does not remove business-associate status. HHS cloud-computing guidance.

Map prompts, uploaded documents, retrieved records, outputs, logs and support access. Assess permitted uses, model training/reuse, subcontractors, retention, access controls, deletion and incident handling. HHS risk-analysis guidance covers threats and vulnerabilities to all ePHI the organization creates, receives, maintains or transmits. HHS risk analysis; business-associate guidance.

Some consumer health technologies fall outside HIPAA and may instead be covered by the FTC Health Breach Notification Rule. Its 2024 amendments took effect July 29, 2024; coverage is a separate assessment, not an automatic conclusion for every app. FTC scope and effective date.

Liquid Learn evaluation: use synthetic examples and evidence references that do not expose patient information. This website’s security overview describes website and assessment safeguards. It does not establish HIPAA suitability, a BAA, a clinical-system integration or authorization to store protected health information in the product. Confirm contractual and deployment requirements before considering such use.

Certified health IT: source attributes and risk management

The HTI-1 decision support intervention criterion, 45 CFR 170.315(b)(11), addresses transparency and intervention risk management in the ONC Health IT Certification Program. For predictive interventions supplied by a certified health IT developer as part of its module, risk management includes analysis, mitigation and governance. The criterion also specifies source-attribute information and access. This is not a blanket certification obligation for every hospital AI tool. Current certification text; official DSI test method and clarifications.

Procurement questions: is this intervention supplied as part of a certified module, who maintains its source attributes, what performance and validation information is available, and how are updates communicated? Obtain evidence from the developer and document local review. Liquid Learn does not provide ONC certification or a verified EHR connection.

State rules can affect the healthcare service too

Texas — effective: HB 149 requires an AI disclosure where an AI system is used in relation to healthcare service or treatment, to the recipient or personal representative no later than the first provision of the service or treatment. An emergency exception changes the timing to as soon as reasonably possible. Assign the notice owner and record where it is delivered. Section 552.051(f), effective January 1, 2026.

Colorado — upcoming, with significant health-sector exceptions: SB26-189’s main requirements apply to consequential decisions from January 1, 2027. Section 6-1-1708 excludes specified core provisions for qualifying HIPAA covered entities and their business associates’ covered services, except employment decisions. For a covered entity that is a healthcare provider, the exception requires operation from a Colorado location. It also excludes specified core provisions for FDA-regulated medical devices and covered FDA-supervised research. Signed Act, section 6-1-1708(3)–(4).

These exceptions are not a blanket exemption from every duty: covered entities must provide a general notice about advanced technology use. Covered ADMT used to determine patient financial-assistance eligibility has specific disclosure provisions, including information about correction and applicable human review. Record which exception and retained duties apply to the service. Implementing rules remained proposed at review. Section 6-1-1708(3)(c)–(e); current rulemaking.

California — scope-specific: the CCPA regulations include healthcare services in the significant-decision definition. Assess business coverage and health-information exemptions carefully rather than treating all healthcare processing as included or excluded. Approved regulations, definitions and Article 11. The US guide explains the staged dates and operational records.

EU: medical-device rules and the AI Act work together

Medical software qualification depends on intended purpose. MDR and IVDR requirements can apply to diagnosis, treatment or other medical functions; the software’s classification determines the relevant conformity-assessment path. Use the current qualification guidance and the applicable regulation, not an assumption that all AI in a hospital has the same class. MDCG 2019-11 rev.1, June 2025.

Under AI Act Article 6(1), the Annex I high-risk route turns on an AI system being a covered product or its safety component and the relevant product requiring third-party conformity assessment. The Commission’s medical-device interplay FAQ explains the complementary MDR/IVDR and AI Act responsibilities. The FAQ is nonbinding guidance and predates the 2026 AI Omnibus; its original dates must be read with the amended timetable. MDCG 2025-6.

Upcoming: the amended Article 6(1)/Annex I high-risk application date is August 2, 2028. That date does not defer existing MDR/IVDR obligations or other AI Act duties that already apply. Document both the medical-device manufacturer and AI provider/deployer roles, plus the relevant quality, technical and clinical evidence. Enacted 2026 amendment; full AI Act timeline.

Example: reviewing a clinician-facing recommendation service

Illustrative governance example, not a clinical integration or medical recommendation. A proposed service receives patient information, produces a recommendation and presents it to a clinician. The team first determines device status and intended-use restrictions; this example assumes neither exclusion nor authorization.

1. Define the boundary: the clinical system holds the patient record. The service map identifies the AI input, provider boundary and output destination, using synthetic descriptions.
2. Assign responsibility: a clinical owner defines what can be relied on and what needs independent review. Privacy and security owners assess the data path and contracts.
3. Set review conditions: the clinician can inspect the basis, disregard the output and use the approved fallback. The team documents how a time-critical or out-of-scope case is handled.
4. Connect evidence: link the supplier’s regulatory rationale, version-specific validation, limitations and local review outcome. Retain sensitive source evidence in appropriately approved systems.
5. Review changes: a model update, new patient population, changed clinical threshold or safety signal triggers the relevant clinical, technical and regulatory review.

Liquid Learn can make the service flow, owners, control context and review record visible. Clinical testing, live monitoring, incident reporting and decisions about patient care remain with the responsible teams and systems.

Medical governance concern → practice → product support