Choose AI Clinical Decision Support System

How do you choose an AI clinical decision support system?

To choose an AI clinical decision support system, write the intended use first, then confirm whether FDA treats the software as a device and, if so, its class. Next, read the vendor's validation evidence, test EHR delivery and override logging in a silent pilot, and sign a business associate agreement before any PHI flows.

Device status versus device class

Device status and device class are two separate questions, and an answer to one does not settle the other. Status asks whether the software is a medical device at all. Class asks, for software that is a device, how much regulatory control applies.

• Status: Section 520(o)(1)(E) of the FD&C Act removes some clinical decision support software from the device definition when it meets four criteria. One is that the clinician can independently review the basis for the recommendation rather than rely mainly on it.
• Criterion 1 limit: FDA's final CDS guidance (January 2026) says software that acquires, processes or analyzes a medical image, a signal from an in vitro diagnostic device, or a pattern or signal from a signal acquisition system does not meet the first criterion, so it remains a device function.
• Patient-facing tools: The statutory carve-out covers software for health care professionals. A patient-facing tool falls outside it, but that does not make it a device automatically, since FDA's general wellness and enforcement discretion policies may apply.
• Class: Software that is a device is Class I, II or III by risk, and that class sets the pathway, such as 510(k), De Novo or PMA. FDA's databases show which authorization a cleared product holds.

Ask the vendor for both answers in writing: whether the product is a device for the intended use you plan, and if so its class, authorization number and cleared indication. A slide that says support only answers neither question.

Which rules shape the purchase?

Beyond FDA, three sets of rules change what a buyer must check: civil rights rules on decision support tools, ONC certification rules for EHR-embedded tools, and HIPAA contracts for any vendor that receives PHI.

• Nondiscrimination: 45 CFR 92.210, under Section 1557, bars covered entities from discriminating through patient care decision support tools and, since 1 May 2025, requires reasonable efforts to identify and mitigate risk from tools that use race, color, national origin, sex, age or disability as inputs.
• ONC transparency: The HTI-5 final rule, published 28 October 2026 and effective 28 December 2026, removed the requirement that certified health IT support source attributes and model cards for predictive decision support interventions under 45 CFR 170.315(b)(11).
• Business associates: Under 45 CFR 164.502(e), a CDS vendor that receives PHI to run its model for a covered entity needs a business associate agreement.
• Delivery standard: HL7 CDS Hooks defines how an EHR calls an external decision service at points such as opening a chart or signing an order, and how the service returns cards the clinician can act on.

Because HTI-5 removed the certification requirement for source attributes, the buyer now has to ask for training data, intended population and known limitations directly, and write the answer into the contract.

What evidence should a vendor show?

A vendor should show validation on patients like yours, measured the way your clinicians would judge the tool. Headline accuracy without the population and outcome definition behind it is not evidence a buyer can use.

• Population: Age, sex, race and ethnicity mix, care setting and geography of the validation data.
• Sites: How many sites, which EHRs, and whether any site was held out from training.
• Outcome definition: What the model predicts, how the label was defined, and the time window.
• Design: Retrospective or prospective, silent or live, and whether clinicians saw the output.
• Subgroups: Performance by subgroup, with confidence intervals, so gaps are visible.
• Calibration: Whether predicted risk matches observed rates, not only ranking performance.
• Limitations: Conditions, settings or inputs where the vendor says the tool should not be used.
• Updates: How often the model changes, how the buyer is told, and whether local re-validation is supported.

Running a local validation

• 1. Pick a period: Choose a recent span of encounters that covers normal seasonal and staffing variation for the decision.
• 2. Run silently: Feed the tool those encounters, or live data in silent mode, and store every output with its inputs.
• 3. Label outcomes: Have clinicians or chart abstractors record what actually happened, using the same outcome definition as the vendor.
• 4. Compare: Measure discrimination, calibration and alert volume overall and by subgroup, and set them beside the vendor's published figures.
• 5. Decide: Agree in advance what gap between local and published results stops the purchase or changes the threshold.

A local validation also gives the monitoring team its baseline. Later drift checks compare against these numbers, not against the vendor's marketing material.

How will the tool reach clinicians in the EHR?

The tool should appear inside the clinician's existing workflow at the moment of decision, through CDS Hooks, a SMART on FHIR app or the EHR vendor's native framework. A separate portal adds steps and gets skipped.

• Trigger point: Confirm which hook fires the tool, for example patient-view, order-select or order-sign, and whether that matches the decision it supports.
• Card content: The card should show the recommendation, the key inputs and a link to the basis, which also supports the independent-review criterion.
• Actions: Suggestions should carry structured actions, such as a proposed order, so the clinician accepts with one step instead of retyping.
• Overrides: The clinician should be able to dismiss the card with a coded reason, and the reason should return to the vendor and the buyer.
• Latency: Agree a response-time target with clinical owners, and decide what the EHR shows when the service is slow or down.
• Native versus third party: EHR-native tools are simpler to deploy but harder to inspect. Third-party tools need more integration but can be validated independently.

Test delivery in the EHR vendor's non-production environment with real order sets and user roles before go-live. Problems with hook timing, card wording and structured actions are cheaper to fix there than after clinicians have formed an opinion of the tool.

An AI CDS scorecard for the buying committee

Score each candidate on the gates first. A tool that fails a gate is out, whatever its other scores. Have the clinical owner, an informatics lead, a privacy officer and an integration engineer each score the items in their own area, so no single demo impression decides the result.

• Gate 1, Intended use: One decision, one role, one point in the workflow, written in a sentence the clinical owner signs.
• Gate 2, FDA position: Written device status for that intended use and, for a device, class, authorization and cleared indication.
• Gate 3, Contract: A business associate agreement and clear terms on PHI use for model training.
• Score, Evidence: Fit of the validation population and design to your patients and setting.
• Score, Fairness: Subgroup results and the inputs reviewed under 45 CFR 92.210.
• Score, Delivery: Hook or app fit, card content, structured actions and latency.
• Score, Feedback: Override reasons, acceptance rates and outcome data returned to the buyer.
• Score, Burden: Expected alerts per clinician per shift, measured in a silent pilot rather than estimated.

Questions to settle before the contract is signed

• Silent mode: Can the tool run on live data without showing cards, so local performance is measured first?
• Alert limits: Can the buyer cap or tune how often the tool interrupts, by role or unit?
• Change notice: Will the vendor notify the buyer before a model update, and share revised performance data?
• Data rights: Will the vendor use the buyer's PHI to train models, and on what terms?
• Monitoring: Who reviews drift, subgroup performance and override trends, and how often?
• Retirement: Who can turn the tool off, how fast, and what the workflow does without it?

Write the answers into the contract or its service schedule, not into meeting notes. A model update or a change of ownership at the vendor is when these terms are tested, and only signed terms hold at that point.

How Agnotic can help

A buying committee needs clinicians who can judge the evidence and engineers who can test the integration, and Agnotic Technologies supplies the engineering side for health systems and digital health companies. Agnotic engineers run the silent pilot, build or connect CDS Hooks services and SMART on FHIR apps, capture override reasons, and set up the monitoring the clinical owner needs.

When a team decides to build its own decision service instead of buying one, Agnotic builds it for the client, with the clinical logic owned by the client's clinicians.

Sources

• FDA: Clinical Decision Support Software guidance (January 2026)
• 21 U.S.C. 360j(o): Regulation of medical and certain decision support software
• FDA: Classify your medical device
• 45 CFR 92.210: Nondiscrimination in the use of patient care decision support tools
• Federal Register: HTI-5 final rule (28 October 2026)
• HL7: CDS Hooks specification
• 45 CFR 164.502(e): Disclosures to business associates