Clinical Data Integration

What is clinical data integration?

Clinical data integration is the work of combining records about the same patients from EHRs, labs, imaging, claims, devices and omics tests into one consistent, governed model. It matches patients across sources, maps local codes to standard terminologies, keeps provenance, and makes the result comparable across sites and studies for analytics, research and quality programs.

Typical sources and how they arrive

• EHRs: Problems, medications, vitals, notes and orders through FHIR Bulk Data exports, HL7 v2 feeds or reporting database extracts.
• Laboratory systems: Results with LOINC codes, units, reference ranges, specimen details and the performing lab.
• Imaging: DICOM header metadata from a PACS or archive, plus the radiology report text.
• Claims: X12 837 claims and 835 remittances, which show care delivered outside your own EHR.
• Omics: Genomic test reports, variant files such as VCF, and other assays held by sequencing labs or research cores.
• Devices and patient-reported outcomes: Readings and questionnaires with their own timestamps and quality issues.

How is it different from operational integration?

Operational integration moves one message to one system so care can happen now, such as a lab result reaching the EHR. Clinical data integration takes that result and years of others from many sources and makes them comparable across patients, sites and time.

The difference changes the engineering. Operational interfaces care about latency and acknowledgments. Integration for analytics cares about history, consistent identity, code mapping, reprocessing when mappings change, and a clear record of which version of the data answered which question.

How does a clinical data integration platform work?

A clinical data integration platform works in four layers: ingest, match and standardize, model, and serve. Data moves through them in order, and each layer records what it changed so any value can be traced back to its source.

• 1. Ingest: Pull each source in its native format (FHIR NDJSON, HL7 v2, X12, DICOM metadata, VCF, CSV), land it unchanged in a raw zone, and record where and when it came from.
• 2. Match and standardize: Link records to a single person through a master patient index, map local codes to SNOMED CT, LOINC, RxNorm and ICD-10-CM, convert units, and keep the original values beside the mapped ones.
• 3. Model: Load cleaned data into the model chosen for each use, with a version number on every mapping and load.
• 4. Serve: Offer governed access through analytics tables, cohort tools, FHIR APIs and research extracts, each with role-based access and logging.

Provenance fields to keep on every value

• Source: The system, site and feed the value came from.
• Original form: The local code, display text and unit exactly as received.
• Transformation: The mapping and unit conversion applied, with their version numbers.
• Timing: When the event happened, when it was recorded and when it was loaded.
• Load batch: An identifier that ties the value to a specific load, so a bad load can be rolled back.

Batch or streaming?

Most clinical data integration runs in batches: nightly Bulk Data exports, weekly claims files and omics results as they are reported. That suits research, quality measures and population health. Streams of HL7 v2 or FHIR subscription events are worth the extra cost only when a use case needs minutes, such as a sepsis alert. Many platforms run both, with the batch path rebuilding the full model and correcting anything the stream missed.

Which data model fits which use?

Pick the model by use, not by preference: FHIR for exchange and apps, OMOP for observational research, SDTM for regulatory trial submissions, and a star schema for operational reporting. Many platforms keep more than one, loaded from the same standardized layer.

• HL7 FHIR R4 with US Core: Choose it when the integrated data must feed apps, partners or APIs. It is patient-centric and well suited to exchange, but heavy for large analytical queries.
• OHDSI OMOP Common Data Model: Choose it for observational research and network studies. Its standardized vocabularies let the same analysis code run across institutions.
• PCORnet Common Data Model: Choose it when the platform takes part in PCORnet research networks that expect this format.
• CDISC SDTM: Choose it when integrated data will support a regulatory submission from a clinical trial.
• Star schema: Choose it for operational and financial dashboards where speed of known queries matters more than research portability.

Keeping several models in step

When a platform holds OMOP tables and a FHIR store side by side, load both from the same standardized layer in the same run, and stamp each with the same release number. A researcher and an app developer who ask the same question should get answers from the same version of the data, or be able to see that they did not.

Bringing omics data alongside clinical records

Integrate interpreted omics results as structured records and leave raw sequence files in object storage. Omics data is large, file-based and often produced by research labs with their own identifiers, so breaking down clinical and omics silos is mostly about identity, consent and labeling.

• Results as records: Store reported variants, genes, zygosity and clinical significance as structured data. The HL7 FHIR Genomics Reporting Implementation Guide gives a standard shape for them.
• Phenotype with genotype: GA4GH Phenopackets describe a patient's phenotype together with genomic findings for exchange between research groups.
• Sample to patient: Link each sample to the patient through a controlled identifier map held by a small, audited team.
• Lab status: Record whether each result came from a CLIA-certified laboratory, because results used for patient care must, while research-only results should be labeled as such.
• Reinterpretation: Variant classifications change over time, so keep the classification date and the knowledge source with each result.

Concepts drawn from clinical notes

Much of the phenotype a genomic study needs sits in free-text notes. When language processing extracts concepts such as a diagnosis or a family history from notes, store each one as a separate, labeled record with the note it came from, the tool and version used, and a confidence value, so analysts can decide whether to count it.

What research and privacy rules apply to combined data?

Combining data does not change its legal status, but it does change who can use it and how. These rules decide the access paths the serve layer must support.

• Genetic data is health information: The HIPAA definition of health information at 45 CFR 160.103 includes genetic information, a change made by the 2013 Omnibus Rule to implement GINA.
• Research pathways: The HIPAA Privacy Rule permits research use of PHI with individual authorization (45 CFR 164.508), an IRB or privacy board waiver (164.512(i)), a limited data set under a data use agreement (164.514(e)), or de-identified data.
• Human subjects: Federally funded or supported research also follows the Common Rule at 45 CFR Part 46, including IRB review and informed consent requirements.
• Laboratory testing: CLIA regulations at 42 CFR Part 493 apply to laboratories that test human specimens for diagnosis, prevention or treatment, so the testing lab's certification matters before a result reaches a clinician.
• Substance use disorder records: Records from programs covered by 42 CFR Part 2 carry their own consent and redisclosure limits, so the platform must filter them by purpose.

Turning consent into filters

Store consent and use limits as data, not as scanned forms. Each patient or record carries fields for research consent status, any Part 2 restriction, the data use agreements it may be shared under and the IRB protocols it is approved for. Extract jobs then filter on those fields automatically, and every extract records which rules it applied.

Where integrated clinical data goes wrong

• Overmerged patients: An aggressive matching rule joins two people, and every analysis on either one is now wrong. Tune for precision and review borderline matches.
• Lost context: A lab value without its unit, specimen or collection time gets compared with values it does not match.
• One model forced on every use: Pushing FHIR, research and trial needs into a single model causes loss or complexity.
• Unlabeled research results: Research-grade omics findings shown to clinicians without a label create safety and CLIA risk.
• No reprocessing path: When a mapping is fixed, the platform cannot rebuild history because the raw data was not kept.
• Silent source changes: A site upgrades its lab system, starts sending new local codes, and those results drop out of every cohort because nothing flagged them as unmapped.
• Consent stored outside the data: Consent, Part 2 limits and data use agreement terms held in documents rather than as filterable fields.

How Agnotic can help

Research platforms succeed or fail on identity, mapping and consent, long before the first cohort query. Agnotic Technologies builds those foundations for digital health and hospital teams: Bulk Data and HL7 v2 ingestion, master patient index services, terminology mapping with version control, OMOP and FHIR models loaded from one standardized layer, omics result pipelines and governed research extracts.

Sources

• 45 CFR 160.103: Definitions, including health information and genetic information
• HHS: Research and the HIPAA Privacy Rule
• 45 CFR Part 46: Protection of Human Subjects (Common Rule)
• 42 CFR Part 493: Laboratory requirements (CLIA)
• 42 CFR Part 2: Confidentiality of substance use disorder patient records
• OHDSI: OMOP Common Data Model
• HL7 International: FHIR Genomics Reporting Implementation Guide
• GA4GH: Phenopackets