NLP in Healthcare

What is NLP in healthcare?

NLP in healthcare is software that reads clinical text and extracts structured facts from it, such as problems, medications, lab values and procedures, together with their context. A clinical NLP pipeline finds each mention, decides whether it is negated, uncertain or about someone else, and maps it to a standard code such as SNOMED CT, RxNorm or LOINC.

The output is data, not prose. Generative AI writes new text, and an AI scribe drafts a note from a conversation; clinical NLP starts from text that already exists and turns it into rows a query can count.

Common uses include registry abstraction, cohort finding for research, quality measure gaps, problem list cleanup, coding support and search across the chart. Each one asks a different question of the text, which shapes the whole design.

Start with the question, not the model. Finding patients with documented heart failure with reduced ejection fraction needs a narrow extractor and a strict test set, while making notes searchable needs broad entity coverage and tolerates more noise.

How does a clinical NLP pipeline work?

A clinical NLP pipeline runs in seven stages, from ingesting documents to publishing reviewed facts. Each stage can be a rule set, a trained model or a language model, but the stages stay the same.

• 1. Ingest: Pull notes, reports and documents from the EHR, as HL7 v2 MDM or ORU messages or as FHIR DocumentReference and DiagnosticReport resources, with note type, author, date and encounter.
• 2. Clean and section: Strip formatting, split the note into sections such as history, medications, allergies, assessment and plan, and detect templated and copied text.
• 3. Find entities: Recognize mentions of problems, drugs, doses, labs, procedures and anatomy with a dictionary, a trained model or both.
• 4. Assign context: Decide for each mention whether it is negated, possible, historical, hypothetical or about a family member.
• 5. Normalize: Map each mention to a concept in a standard terminology, keeping the original text span beside the code.
• 6. Store with provenance: Write each fact with its source document, character offsets, pipeline version and confidence.
• 7. Review and publish: Send low-confidence or high-impact facts to a reviewer, then publish the rest to the registry, warehouse or application that needs them.

Section detection is easy to skip and expensive to miss. A drug named under allergies means something very different from the same drug under current medications.

Provenance is what makes the output trustworthy. When a reviewer or auditor asks why a patient is in a cohort, the answer should be one click away: the sentence, the note and the pipeline version that produced the fact.

Why does negation matter so much?

Negation matters because clinical notes routinely record what was ruled out alongside what was found. The phrase no chest pain contains the words chest pain, and a pipeline that misses the no records the opposite of the truth.

Trigger-phrase methods are the usual baseline. They look for words such as no, denies, without and negative for, then limit how far each trigger reaches so a negation does not spill into the next clause. Trained models learn the same distinctions from labeled text, and a good test set compares both.

Negation is one part of a wider context problem. Uncertainty, as in rule out pneumonia, history, as in prior stroke, the experiencer, as in mother had breast cancer, and hypotheticals, as in return if fever develops, all change meaning, and each needs its own test cases.

Which terminologies and rules apply?

Clinical NLP output should land in the same terminologies the EHR and US Core use, and the project falls under HIPAA and, for some uses, FDA policy. Three facts matter most, and the rest is design choice.

• SNOMED CT: Owned and maintained by SNOMED International; the US National Library of Medicine is the US National Release Center and distributes SNOMED CT to UMLS Metathesaurus licensees.
• RxNorm for drugs: Map drug mentions to RxNorm clinical drug concepts, then parse strength, route and frequency into separate fields so a dose can be checked and compared.
• LOINC for tests: Map lab and observation mentions to LOINC, and keep the value and unit beside the code, because a number without its unit cannot be trended.
• HIPAA de-identification: 45 CFR 164.514(b) allows two methods, expert determination and the safe harbor removal of 18 types of identifiers.
• FDA CDS guidance: FDA's final Clinical Decision Support Software guidance of January 2026 sets out when clinician-facing recommendation software is not a device, including that the clinician can independently review the basis for the recommendation.

Map to ICD-10-CM as a second step for billing uses. SNOMED CT keeps more clinical detail, and NLM publishes a SNOMED CT to ICD-10-CM map that helps with the translation.

A terminology service makes this practical. It holds the code systems, answers lookup and validation requests and tracks versions, so the pipeline maps against the same release the EHR uses and an audit can show which release produced each code.

How do you evaluate clinical NLP?

Evaluate on an annotated sample of your own documents, scored for each entity type and each context label separately. A single overall score hides the errors that matter.

• Annotation guide: Written rules for what counts as a mention, where spans start and end, and how to label context, agreed before annotation begins.
• Double annotation: A subset of documents labeled by two people, with agreement measured, so the team knows how consistent the reference is.
• Per-label scores: Precision, recall and F1 for each entity type and context label, such as negated medications or historical problems.
• Site and note-type splits: Results reported by hospital, department and note type, since templates and abbreviations differ.
• Error review: A sample of false positives and false negatives read every cycle, with each traced to the dictionary, the model or section detection.
• Drift checks: A fresh sample rescored after EHR template changes, new note types or model updates.

Treat evaluation as ongoing measurement against the intended use, not a one-off gate before launch. Tie each metric to the decision the output supports, such as cohort selection or registry abstraction.

Large language models do not remove this work. They can reduce the labeled data needed to get started, but they still need the same annotated test set, and their output must be checked against the terminology so invented codes are caught.

Handling PHI in NLP projects

Clinical text carries PHI in places structured fields never hold, such as names of relatives, employers and streets in the social history. Treat every note as PHI until it has been properly de-identified.

• Minimum necessary: Pull only the note types and date ranges the use case needs, following the HIPAA minimum necessary principle, and record that scope in the project file.
• De-identified development text: Prefer text de-identified by expert determination or safe harbor for model development outside production, and test the de-identification tool on your own notes first.
• Model hosting: Run models in your own cloud account or with a vendor under a business associate agreement, and confirm in writing that the vendor does not keep or train on your text.
• Output and logs: Extracted facts and debugging logs are PHI too, so they get the same access controls and audit logging as the source notes.

De-identification tools miss things, especially in free text. Measure their recall on a sample of your own notes before relying on them, and keep a reviewer in the loop for any text that leaves the protected environment.

Failure points in clinical NLP

• Dictionary-only extraction: A term list without context rules turns every denies and family history of into a positive finding.
• Ambiguous abbreviations: Short forms such as MS or RA mean different things in different departments, so they need section and specialty context.
• Lost spans: Storing only the code, without the source text and offsets, makes review and audit impossible.
• One-site models: A model tuned on one hospital's templates goes live at another without re-evaluation.
• Mapping too early: Forcing every mention straight into a billing code discards clinical detail that SNOMED CT would have kept.
• Repeated text: Templated and carried-forward passages make one old finding look like many new ones, which skews counts and onset dates unless duplicates are detected.
• No owner after launch: Nobody rescores the pipeline when templates change, and quality falls without anyone noticing.

Most of these are decisions made in the first weeks of a project. A short scoping phase that fixes the question, the note types, the annotation guide and the owner deals with them before they are built in.

How Agnotic can help

Clinical NLP succeeds or fails on scoping and evaluation decisions made before any model is trained. Through its fractional CTO service, Agnotic helps healthcare teams define the use case, choose between rules, trained models and language models, plan annotation and evaluation, and design PHI controls.

Agnotic engineers then build the pipeline with the client's team, from EHR document feeds and terminology services to review queues and the provenance store.

Sources

• NLM: SNOMED CT
• 45 CFR 164.514: De-identification of protected health information
• FDA: Clinical Decision Support Software guidance