Agnotic Technologies LogoAgnotic Technologies Logo
    Free assessment

    Healthcare AI readiness assessment

    Score your organization on the five areas that decide whether a healthcare AI project reaches production: data, integration, governance, workflow and security. Answer 15 questions and see your readiness by area and the use cases to pilot first.

    15-Minute Scoping CallStart the assessment

    For hospital and health system CTOs, CMIOs and digital leaders planning their first or next AI pilot.

    What does the healthcare AI readiness assessment help you decide?

    The healthcare AI readiness assessment helps a hospital or health system decide which AI use cases it can pilot now and which need groundwork first. It scores data, integration, governance, workflow and security readiness, then matches the result to use cases such as ambient documentation, prior authorization and chart summarization.

    Last reviewed by the Agnotic Technologies engineering team

    How ready is your organization for AI?

    Answer each question for your organization as it is today, not as planned. Your answers stay in your browser and are not sent to Agnotic.

    Data readiness

    Data readiness

    Whether the data a use case needs exists, is coded consistently and can be used for testing.

    1. Can you name the source of truth for the data your first use case needs, such as the EHR, the data warehouse or the claims system?

    2. Is that data coded consistently, for example diagnoses in ICD-10-CM and lab results in LOINC, rather than mostly free text?

    3. Can your team produce a de-identified test data set without starting a new manual project each time?

    Integration readiness

    Integration readiness

    Whether a new tool can read from and write back to the systems clinicians and staff already use.

    1. Do you have working FHIR or HL7 v2 interfaces from your EHR that a new application could use?

    2. Can a new tool write its results back into the clinician's workflow, not only into a separate screen?

    3. Is there a team that owns interfaces and can take on new integration work this year?

    Governance readiness

    Governance readiness

    Whether there is a group and a process to approve, validate and monitor AI use cases.

    1. Is there a group that approves AI use cases, with clinical, compliance, IT and operations members?

    2. Do you have a written process to validate a model before go-live and monitor it afterwards, including checks for bias?

    3. Have you chosen a risk framework, such as the NIST AI Risk Management Framework, to structure that process?

    Workflow readiness

    Workflow readiness

    Whether a named owner has defined the problem, the review step and the time staff need.

    1. Has a clinical or operational owner named the workflow problem and how success will be measured?

    2. Is it clear who reviews AI output before it reaches a patient, a chart or a claim?

    3. Will staff time be set aside for training and feedback during a pilot?

    Security and privacy readiness

    Security and privacy readiness

    Whether contracts, data controls and logging allow an AI tool to process PHI safely.

    1. Do your contracts allow AI vendors to process PHI, with a business associate agreement signed or ready to sign?

    2. Can you control where data goes, including whether prompts and outputs are stored or used to train models?

    3. Are audit logging and access controls in place for the systems the AI tool would read from?

    Scoring

    How does the assessment score each area?

    Each area has three questions worth 0 to 2 points. An area scoring 5 or 6 is Ready, 3 or 4 is Building, and 0 to 2 is a Gap. This is Agnotic's planning rubric, not a regulatory standard.

    What each readiness area checks and why it matters for healthcare AI
    AreaWhat it checksWhy it matters for AI
    Data readinessWhether the data a use case needs exists, is coded consistently and can be used for testing.Models and summaries are only as good as their inputs. Unclear sources and free-text data make a pilot hard to evaluate.
    Integration readinessWhether a new tool can read from and write back to the systems clinicians and staff already use.An AI tool that lives in a separate screen gets ignored. Value comes from reading the record and writing results back into the workflow.
    Governance readinessWhether there is a group and a process to approve, validate and monitor AI use cases.Without an approval path, pilots stall at the security or clinical review. Without monitoring, a model that drifts goes unnoticed.
    Workflow readinessWhether a named owner has defined the problem, the review step and the time staff need.A model can work and still go unused if nobody owns the workflow it changes. The workflow owner decides what success means and who reviews AI output.
    Security and privacy readinessWhether contracts, data controls and logging allow an AI tool to process PHI safely.AI tools often send data to new vendors and models. The contract, data retention terms and audit trail need to be settled before PHI moves.

    Which AI use cases fit each readiness level?

    Use cases ordered from the lowest to the highest bar. The tool suggests a use case to pilot first when every area it needs is at the level shown.

    Minimum readiness level per area for common healthcare AI use cases
    Use caseDataIntegrationGovernanceWorkflowSecurity
    Document processing for faxes and referralsExtract and route incoming documents for staff to confirm.Any levelAny levelAny levelBuildingBuilding
    Patient message drafting and routingDraft replies and sort inbox messages for staff review.Any levelAny levelBuildingBuildingBuilding
    Prior authorization automationGather documentation and prepare requests against payer rules.BuildingReadyAny levelBuildingBuilding
    Ambient clinical documentationDraft visit notes from the conversation for the clinician to sign.Any levelBuildingBuildingReadyReady
    Chart summarization and record questionsSummarize a patient record and answer questions with citations.ReadyBuildingBuildingAny levelReady
    Readmission risk predictionFlag patients for follow-up based on a validated model.ReadyBuildingReadyBuildingAny level
    Clinical decision supportShow recommendations inside the EHR at the point of care.ReadyReadyReadyReadyReady

    What should you do with the result?

    Treat the result as the agenda for your next planning meeting, not as a grade.

    1. Pick one pilot

      Choose one use case from Pilot first, name its owner and write down what the pilot must show before it is extended.

    2. Close the weakest area in parallel

      The area marked Gap usually blocks the second and third use case, so start that work while the first pilot runs.

    3. Agree the review step

      Decide who checks AI output, how often and what happens when it is wrong, before any patient or chart is affected.

    4. Turn it into a plan

      Bring the result to a scoping call. We turn it into a pilot plan covering integration, security, evaluation and monitoring.

    What can the assessment not tell you?

    Model accuracy

    Whether a specific model or vendor performs well for your patients. That takes validation on your own data.

    Device status

    Whether a tool is regulated as a medical device. FDA's clinical decision support guidance explains which software functions fall outside the device definition.

    Cost and timelines

    Budget, ROI or delivery dates. Those depend on the use case, your systems and your team, and we work them out on a call.

    Agreement inside your team

    The score is self-reported. Two leaders in the same organization can score it differently, and that difference is worth discussing.

    Which frameworks and rules shape healthcare AI readiness?

    The governance and privacy questions in the assessment come from these sources.

    • The NIST AI Risk Management Framework (AI RMF 1.0, NIST AI 100-1), released on 26 January 2023, is intended for voluntary use and is organized around four functions: Govern, Map, Measure and Manage.

      Source: NIST

    • NIST released a Generative AI Profile of the AI RMF (NIST AI 600-1) on 26 July 2024.

      Source: NIST

    • FDA's Clinical Decision Support Software guidance explains which decision support functions meet the non-device criteria in section 520(o)(1)(E) of the FD&C Act and which remain device software functions.

      Source: FDA guidance

    • A covered entity may let a business associate create, receive, maintain or transmit ePHI on its behalf only after obtaining satisfactory assurances that the business associate will safeguard it.

      Source: 45 CFR 164.308(b)(1) (eCFR)

    • HIPAA allows two ways to de-identify health data: expert determination, or the safe harbor method, which removes 18 categories of identifiers.

      Source: 45 CFR 164.514(b) (eCFR)

    Which Agnotic services pick up from here?

    Agnotic Technologies builds healthcare AI for hospitals and health systems as a services partner. These pages cover the work that follows an assessment.

    Frequently Asked Questions

    It has 15 multiple-choice questions across five areas. The answers are quickest to give with the people who own data, interfaces, compliance and the target workflow in the room, and the discussion it starts is often as useful as the score.