01
Surgical residency programs
A scored simulation for one procedure, with results sent to the program's evaluation records.
Healthcare / VR Surgery Software
Virtual reality surgery software for training, planning and the operating room
Surgical teams ask for headsets to do three different jobs: rehearse a procedure, study a patient's anatomy before the case and see images during it. Each job has its own data, sign-off and regulatory weight. Agnotic Technologies builds the software for the job you choose, on headsets your team already uses.
Simulation
Scored procedure steps for residents
Planning
The patient's CT or MRI as a 3D model
Overlay
Device path decided before the build
Trusted by global innovators
Industry Overview
What is virtual reality surgery software?
Virtual reality surgery software is the headset application and imaging pipeline behind surgical simulation, preoperative planning from a patient's own CT or MRI, and augmented reality overlays in the operating room. Agnotic Technologies builds it as a services engagement and settles each use case's data source, device status and clinical sign-off before the first scene is built.
Industry Challenges
Where do surgical XR projects stall?
The headset is rarely the problem. Projects stall on ownership, data and the line between a training tool and a device.
A demo scene impresses the department, but nobody wrote the scoring rules, so the program director cannot use the results in an evaluation.
Planning models are built by hand in a 3D tool for each case, and the file lives on one person's laptop instead of in the patient's imaging study.
Segmentation from a routine CT protocol is too coarse for small structures, and surgeons stop opening the model after the first few cases.
The app is written against one headset vendor's SDK, so a hardware change means starting again.
An overlay prototype slowly gains guidance features until it makes a device claim nobody decided to make.
Simulation scores stay inside the app and never reach the residency program's evaluation records.
Decision table
Which virtual reality surgery use case are you building?
The three use cases share a headset and little else. We fill in this table with the clinical owner before scoping, because the input data and the device status decide most of the build.
| Use case | Input data | Output | Likely FDA status | Clinical owner | Failure that stops adoption |
|---|---|---|---|---|---|
| Training simulation | Synthetic anatomy and a procedure broken into steps; no patient data | Scored attempts per resident: step order, time, errors and instrument path | Usually a training product when it makes no claim about a patient; confirm with your regulatory lead | Program director and teaching faculty | Scores that faculty do not agree with, so nobody uses them |
| Preoperative planning | The patient's own CT or MRI series pulled from PACS | A segmented 3D model the surgeon reviews on a headset or desktop | Image processing with segmentation or 3D visualization for patient management fits 21 CFR 892.2050, Class II | Operating surgeon, with radiology | A model the surgeon cannot correct or does not trust |
| Intraoperative AR overlay | The approved planning model, registered to the patient in the room | Images overlaid on the surgical field during the case | A medical device; plan for FDA premarket review before clinical use | Surgeon, with biomedical engineering and the device maker's quality team | Registration drift and line of sight problems in a live room |
Our Approach
How do we keep each surgical use case honest?
Three rules apply to every surgical XR build, whichever use case you start with.
Decide the claim before the scene
Training, planning or guidance is agreed in writing first. The claim sets the device path, the validation work and who signs off, so features cannot drift into guidance later.
Keep patient images in the imaging chain
Planning models are derived from DICOM and stored back as DICOM objects that reference the source series, so a model travels with the study instead of a shared drive.
Score what faculty already grade
Step order, time, errors and instrument path are defined with the teaching faculty, and pass criteria are their decision, set as configuration rather than code.
Capabilities
What do we build for virtual reality surgery?
We build the surgical software around the headset. VR therapy, rehab and headset fleet tools are covered on our VR healthcare software page.
Procedure simulation modules
Step-based scenes for a defined procedure, with branches for common errors and faculty-set pass criteria.
Scoring engine
Step order, time per step, errors and instrument path recorded for each attempt and kept per resident.
Program director dashboard
Attempts and scores by resident, procedure and date, with an export to the residency evaluation system.
Segmentation and mesh pipeline
CT or MRI series to labelled structures and surface meshes, with correction tools for radiology and the surgeon.
Planning viewer
The patient model on a headset or desktop: rotate, slice, measure and review it together at a case conference.
PACS round trip
Segmentations and models stored back to PACS as DICOM objects linked to the source study.
AR overlay prototypes
Registration workflow and overlay rendering for device teams, built under their design controls and quality system.
Headset portability layer
OpenXR as the runtime interface, so one surgical app can run on conformant headsets from more than one vendor.
How It Works
How does a CT scan become a planning model the surgeon trusts?
This is the image to model pipeline we start from. Each step leaves a record, so the team can see which series, which labels and whose approval produced the model in the headset.
Pull the study
A DICOM query fetches the ordered series from PACS. Protocol checks such as slice thickness and contrast phase run before any processing, and a study that fails them is sent back with a reason.
01
1
Segment the anatomy
Automated or semi-automated labelling produces a DICOM Segmentation object that references the source images and shares their frame of reference.
02
2
Correct and approve
Radiology or the surgeon edits labels in the viewer. Approval is recorded with the person's name and the time, and only approved labels move on.
03
3
Build the mesh
Surface meshes are generated from the approved labels and simplified for the headset, keeping the original resolution for desktop review.
04
4
Store it with the study
The mesh is saved as an Encapsulated STL object in the same study, so the model sits in the patient's imaging record with the series it came from.
05
5
Review on the headset
The planning viewer opens the model through the study reference. Notes and measurements from the case conference are saved against the same case.
06
6
Engineering Stack
Which engines, formats and standards does surgical XR use?
Choices we make with your team for surgical builds. Engines and devices for therapy apps are listed on our VR healthcare page.
XR runtime and engines
OpenXRUnityUnreal EngineDesktop viewers for non-headset review
Imaging
DICOM and DICOMwebDICOM SegmentationEncapsulated STLPACS and vendor neutral archives
Simulation data
Event log per attemptFaculty rubric configurationEvaluation exports
Platform and security
Single sign-onRole-based accessAudit loggingEncryption in transit and at rest
Measured Impact
What should a surgical XR program be able to see?
Status views we build into the program director and planning dashboards.
Attempts
Attempts per resident
By procedure and date, against the faculty schedule.
Passed
Steps passed
Attempts that met the faculty pass criteria, step by step.
Approved
Models approved
Planning models approved before the case date.
Opened
Models opened
Whether the operating surgeon opened the model before the case.
My Lauren Maternal Health Platform | Built by Agnotic
A HIPAA-ready maternal health platform enabling remote patient monitoring, care coordination, and personalized engagement from pre-pregnancy through postpartum.
Case Study
LERA Health: Women's Health App Platform
A women’s health product experience focused on guided care journeys, education, and compliance-aware workflows to support long-term engagement.
Case Study
Use Cases
Who asks us to build virtual reality surgery software?
Teams we scope surgical XR work with, and where each one usually starts.
02
Hospital 3D planning labs
A repeatable pipeline from CT to approved model, replacing hand-built files for each case.
03
Medical device makers
An AR overlay or planning prototype built under the maker's design controls for its own submission.
04
Simulation centers
Shared procedure libraries for several departments, with one scoring model.
05
Virtual reality surgery for startups
A first module and a clear claim, so the company knows its regulatory path before it raises on a demo.
06
Orthopedic and spine groups
Planning models from the patient's imaging, reviewed at the case conference before surgery.
Integrations
Which systems does virtual reality surgery software connect to?
The systems a surgical XR build reads from and writes to.
Source series and stored models
PACS
Long-term image storage
Vendor neutral archive
Labelling and correction
Segmentation tools
Scores and attempts
Residency evaluation system
Course completion
Learning management system
Single sign-on
Identity provider
App deployment
Headset management
Design records for overlays
Device quality system
Compliance-First Development Services Backed by Global Standards
We build secure, scalable products designed for privacy, interoperability, and regulatory readiness from day one across every sector we serve.
Health Insurance Portability and Accountability Act
Protect PHI with privacy-first architecture, encrypted storage and transmission, and strict access controls.
General Data Protection Regulation
Implement lawful consent flows, data minimization, and secure processing for global data privacy.
Our Edge
Why Global Leaders Choose Us
We combine deep technical expertise with industry-specific knowledge to deliver solutions that aren't just functional, but transformational.
Enterprise-Grade Security
We implement rigorous security protocols and compliance standards (HIPAA, GDPR, SOC2) across all industrial solutions to protect sensitive data.
High-Performance Scaling
Our architectures are built to handle massive data loads and user bases, keeping performance steady whether you're serving ten or ten million.
Accelerated Time-to-Market
Using our suite of internal tools and proven frameworks, we cut setup work so your first release reaches users sooner.
Embedded AI Integration
Beyond simple wrappers, we build deep-learning integrations and predictive analytics directly into the core of your industry-specific workflows.
Engagement Model
How does a virtual reality surgery engagement run?
We start with one use case and one procedure or case type. Timing depends on your data and devices, and we agree a fixed first release on the scoping call.
01
Scoping call
Use case, procedure, headsets in use and who signs off clinically.
02
Claim and data review
The use case table filled in, the device path agreed and the imaging or rubric inputs confirmed.
03
Prototype on the target headset
One scene or one model on the real device, reviewed by faculty or the surgeon.
04
Fixed-scope first release
Scoring and dashboard, or the planning pipeline, live for one procedure or case type.
05
Scale and support
More procedures, more case types and support under agreed change control.
Facts and sources
Which rules and standards shape virtual reality surgery software?
Short facts we design around, each linked to its primary source.
A medical image management and processing system is a Class II device under 21 CFR 892.2050, and its advanced functions may include image segmentation, multimodality image registration and 3D visualization for interpretation and patient management.
Source: 21 CFR 892.2050 (eCFR)The DICOM Segmentation IOD stores a classification of pixels in one or more referenced images as binary, fractional or label map segmentations, and shares the Frame of Reference of the images it was derived from.
Source: DICOM PS3.3, section A.51 Segmentation IODDICOM defines an Encapsulated STL IOD that describes a 3D model in Stereolithography format encapsulated within a DICOM object, so a planning model can be stored with the patient's imaging study.
Source: DICOM PS3.3, section A.85 Encapsulated 3D Manufacturing Model IODsThe OpenXR specification describes OpenXR as the interface between an application and an XR runtime, which may handle frame composition, peripheral management and raw tracking information.
Source: Khronos Group, OpenXR 1.1 Specification, section 1.1
Voice of success
We don't just build products; we forge lasting partnerships. See how we've helped industry leaders transform their vision into technical reality.
“I can clearly see how Agnotic has a unique way of handling end-to-end development. They are always active on quick chat and provide support quickly.”
Aaron Phelan
Founder, Benchmark
“Agnotic combines deep technical expertise with strong domain knowledge. They understand the business context, anticipate challenges, and make collaboration smooth and effective.”
John Pasmore
Founder, Latimer
“Agnotic is the best technical team we evaluated. Their engineering excellence made our work dramatically easier and allowed us to stay focused on what matters most for maternal care outcomes. They took full ownership of the technical execution, and we are always happy to continue working together.”
Kim Smith
Founder, My Lauren
Related pages
Related surgical and immersive software
This page covers surgical simulation, planning and overlays. These pages cover the work around it.
VR therapy and rehab software
Pain distraction, motor rehab, exposure therapy and headset fleets.
AI simulation for medical training
Simulated patient conversations for clinical education.
General surgery software
Software for surgical practices and their clinical workflows.
Medical imaging PACS development
Image storage, viewing and DICOM workflows behind planning models.
EHR perioperative module or a standalone system
Scheduling and documentation choices for surgical services.
More in Healthcare
More Healthcare Solutions
Explore other production-grade engineering platforms we deliver across healthcare.
Cross-Industry Reach
The Same Engineering Patterns, Across Other Industries
The technical patterns behind this platform translate naturally into adjacent verticals.
Frequently Asked Questions
It depends on the claim. A simulator that only trains staff usually is not. Planning software that segments a patient's images for patient management fits 21 CFR 892.2050, and an intraoperative overlay is a device. We settle the claim with your regulatory lead before we build.
Faculty define the steps, their order, the errors that matter and the pass criteria. The app records each attempt against that rubric, including time per step and instrument path, and the program director sees the results by resident and procedure.
Back in PACS, with the study it came from. Segmentations are saved as DICOM Segmentation objects and meshes as Encapsulated STL, so the model stays in the patient's imaging record instead of on a laptop or a shared drive.
We build against OpenXR, the Khronos standard interface between an application and the XR runtime, and keep vendor-specific features behind a small layer. That lets the app move to another conformant headset with far less rework.
The use case, the number of procedures or case types, whether you need segmentation from patient images, PACS and evaluation system integrations, the headsets in scope and, for overlays, the validation work your device path requires. We size these on the scoping call rather than quote from a list.
Yes. We usually start with one module and a written claim, so the company knows whether it is building a training product or a device before it commits to features and investors see a demo.
Conversation practice with AI patients is a different product, covered on our AI medical training page. This page covers procedure skills, planning models and overlays.
Industries