Human in the Loop AI

What is human in the loop AI?

Human in the loop AI is a system design in which a person with the right authority reviews or approves AI output at defined points before it changes a record, a claim or a care decision, and in which the person's corrections are captured to improve the system. It sits between full automation and manual work, and it is how most healthcare AI should run today.

Three levels of involvement

• Human in the loop: A person approves each output before it takes effect. Right for clinical suggestions, coding changes and denials.
• Human on the loop: The system acts, and a person monitors, samples and can stop it. Right for high-volume, low-risk tasks such as eligibility checks.
• Human out of the loop: Fully automatic. Right only for deterministic tasks with no patient or payment impact.
• The design choice: Which level applies to which task, and what moves a task between levels over time.

Why does human in the loop AI matter now?

• FDA draws the line at review: Under section 520(o)(1)(E) of the FD&C Act and FDA's Clinical Decision Support Software guidance (January 2026), software is not a device only if the clinician can independently review the basis for its recommendation. Time-critical tools and tools that advise patients directly are treated as devices.
• Transparency is certified: ONC certification criterion 170.315(b)(11) requires source attribute information for predictive decision support interventions, so reviewers can see what a model used. ASTP/ONC's HTI-5 proposal would narrow this and was not final as of October 2026.
• Discrimination must be checked: Under 45 CFR 92.210, from May 1, 2025 covered entities must make reasonable efforts to identify patient care decision support tools that use protected characteristics as inputs and to mitigate the risk of discrimination.
• Payers cannot automate denials: Under the CMS-4201-F final rule, Medicare Advantage coverage decisions must be based on the individual patient's circumstances (42 CFR 422.101(c)), and CMS's February 2024 FAQ memo on that rule states that an algorithm or AI tool cannot be the sole basis for a denial.
• A framework exists: The NIST AI Risk Management Framework (AI 100-1, January 2023) organizes AI risk work into govern, map, measure and manage functions, which a review process makes concrete.
• Audit controls are required: The HIPAA Security Rule's audit controls standard at 45 CFR 164.312(b) requires mechanisms that record and examine activity in systems containing ePHI, which includes what AI tools read and changed.

Where should the human sit in a healthcare AI workflow?

The human in the loop healthcare AI point is the step where a person's decision is recorded. The diagram shows the four control points we design into every system, and the feedback path that makes the loop a loop.

• Intake: What data the AI may see, under which BAA, and whether PHI is minimized before it is sent.
• Suggestion: The AI drafts a code, a claim correction, an appeal letter or a clinical note. Nothing changes yet.
• Review: A coder, nurse, clinician or billing specialist sees the suggestion, its reasoning and the source data, and accepts, edits or rejects it.
• Commit: Only the reviewed result is written to the EHR or billing system, with the reviewer's identity in the audit trail.
• Feedback: Accepts, edits and rejects are logged and used to measure the model and retrain or retire it.
• Oversight: A committee reviews performance, error patterns and fairness metrics on a schedule.

How does human in the loop governance apply to RCM AI?

Human in the loop governance for RCM AI matters because every output is money or compliance: a code, a claim, a denial, an appeal. Comprehensive RCM automation works when AI handles volume and people handle judgment.

• Coding: AI proposes ICD-10-CM and CPT codes from documentation. A coder confirms, and the acceptance rate is tracked per code family.
• Claim scrubbing: AI flags likely rejections before an X12 837 claim goes out. Rules with a known fix can be automatic. New patterns go to a person.
• Prior authorization: AI assembles the clinical evidence and drafts the request. A clinician or nurse signs it. Under CMS-0057-F, payers must answer standard requests within seven days and expedited requests within 72 hours from 2026.
• Denials and appeals: AI classifies the denial reason from the X12 835 or 277 and drafts the appeal. A specialist reviews the letter and the attachments.
• Payment posting: Mostly on the loop, with sampling and exceptions routed to a person.
• Patient billing: AI drafts estimates and messages. A person approves anything that changes a balance.

Designing the review screen

The review screen is where human in the loop succeeds or fails. It should show the suggestion, the reasoning and the exact source data side by side, make accept, edit and reject equally quick, and capture a reason when the reviewer overrides. If reviewers must open three other systems to check a suggestion, they will stop checking.

Deciding when to automate

A task earns a move from in the loop to on the loop with evidence: a stable acceptance rate, a known error profile, a sampling plan, a stop switch and a named owner. Write the thresholds down before launch so the decision is made by policy, not by whoever is impatient with the queue that month.

How do teams build human in the loop AI step by step?

• 1. Pick the task and the risk: What the AI will draft, what can go wrong, and who is accountable today.
• 2. Classify it: FDA device or not, patient care decision support under 45 CFR 92.210 or administrative, and which payer rules apply.
• 3. Set the control points: Where review happens, who reviews, and what the reviewer must see to decide.
• 4. Design the review screen: Suggestion, reasoning, source data and one-click accept, edit or reject.
• 5. Build the audit trail: Every suggestion, decision and reviewer recorded against the record it touched.
• 6. Run in shadow mode: Suggestions logged but not shown until accuracy is known.
• 7. Go live in the loop: Show suggestions, measure acceptance and edits, and keep the person deciding.
• 8. Earn automation: Move a task to on the loop only when error rates, sampling and a stop switch justify it.

What are human in the loop AI best practices?

• Review by the right role: Clinical suggestions reviewed by clinicians, codes by coders, claims by billing specialists.
• Show the evidence: The reviewer sees the source text or data the AI used, not only the answer.
• Make rejecting easy: If overriding is slower than accepting, reviewers will rubber-stamp.
• Measure the reviewers too: Acceptance rates that approach 100 percent are a warning sign, not a success.
• Keep PHI minimal: Send the AI only the data the task needs, under a BAA.
• Check fairness: Compare error and override rates across patient groups and act on gaps.
• Write the policy: Which tasks are at which level, who can change that, and how often the committee meets.

The practices above keep the loop honest once the system is in production. The mistakes below are how it quietly stops being a loop.

What are the common mistakes with human in the loop AI?

• Review theater: A person clicks approve on hundreds of suggestions an hour without reading them.
• No reasoning shown: Reviewers cannot see why the AI suggested something, so they cannot judge it.
• Silent commits: AI output written to the record before anyone looked, with the review happening afterwards, if at all.
• One policy for all tasks: Eligibility checks and clinical suggestions treated with the same level of oversight.
• No feedback capture: Corrections lost, so the model never improves and nobody can prove oversight.
• Vendor-only governance: Relying on the vendor's claims instead of your own monitoring and fairness checks.

What does human in the loop AI mean for your build plan?

• Tasks and levels: Which tasks are in, on or out of the loop, and the rules for moving them.
• Review interfaces: Screens for coders, nurses, clinicians and billing staff with the evidence in view.
• Audit and feedback: Logging of every suggestion and decision, and the pipeline that uses it.
• Classification work: FDA, Section 1557 and payer rule checks before design.
• Integration: EHR and billing systems through FHIR and X12, with write-back only after review.
• Governance: A committee, a policy and the monitoring that supports both.

For the broader picture, our healthcare AI trends guide covers what is working in production, and our HIPAA compliant AI app guide covers the compliance work around the data.

How Agnotic can help

Agnotic Technologies is a software development services company. We build human in the loop AI for healthcare operations and clinical teams: AI services that draft codes, claims, appeals and notes, review screens that show the evidence, audit trails that record every decision, and the integration with EHR and billing systems that commits only what a person approved. Your staff keep the authority. The AI does the volume.

Sources

• FDA: Clinical Decision Support Software guidance (January 2026)
• ASTP/ONC: Decision support interventions, 170.315(b)(11)
• 45 CFR 92.210: Nondiscrimination in the use of patient care decision support tools
• 42 CFR 422.101: Requirements relating to basic benefits (Medicare Advantage)
• Federal Register: CMS-4201-F, Contract Year 2024 Medicare Advantage final rule (88 FR 22120)
• NIST: AI Risk Management Framework (AI 100-1)
• 45 CFR 164.312: HIPAA Security Rule technical safeguards
• CMS: Interoperability and Prior Authorization final rule (CMS-0057-F)