Internal assistants
Staff chat tools over company documents, with data kept inside policy.
AI / AI Firewall and LLM Guardrails
AI Firewall and LLM Guardrail Development
Every team calls the model its own way, with its own keys, and nobody can say what data left the building last week. Agnotic Technologies builds the AI firewall: one gateway between your applications and the models that screens, redacts, checks and logs every exchange.
One gateway for every modelPHI redactionTool-call allow listsFull audit log
Trusted by global innovators
What is an AI firewall for LLM applications?
An AI firewall is a gateway between your applications and the language models they call. It screens prompts for injection, redacts personal and health data before it reaches a model, checks outputs and tool calls against policy and logs every exchange. Agnotic Technologies builds it for companies running their own LLM applications and agents.
A gateway also solves plainer problems: one place for model keys, rate limits, routing between providers and cost tracking per application.
No filter stops every attack. The gateway reduces what an attack can reach and records what happened, and it sits alongside least-privilege design in the applications themselves.
Where it applies
Which applications need an AI firewall?
Any system where a model sees sensitive data or can take an action.
Customer-facing chat
Support and sales assistants exposed to anyone who types.
Healthcare AI features
Products that send clinical text to a model and must keep PHI controlled.
Agents with tools
Assistants that create tickets, update records or send messages.
Multi-team model platforms
One gateway for every team instead of keys scattered across services.
AI product startups
A gateway built into the product so customers can see the controls.
Gateway architecture
How is an LLM gateway built?
Each request passes the same stages in order. Responses and tool calls pass back through the checks before they reach the application.
1. Ingress
- Application identity and per-app keys
- Rate limits and quotas per application and user
- Request size and token caps
- Model keys held only in the gateway
2. Input checks
- Prompt injection screening on user text
- Screening of retrieved documents for hidden instructions
- PII and PHI detection with redaction or tokenization
- Policy rules per application
3. Routing
- Approved models per application
- Fallback when a provider fails
- Region and data residency rules
- Cost and latency tracking
4. Output checks
- Schema validation for structured output
- Grounding check against supplied sources
- Re-detection of personal data in responses
- Encoding before output reaches a browser or shell
5. Tool-call broker
- Allow list of tools per application
- Parameters checked against the user's permissions
- Human approval for actions that change records
- Tool results screened like any input
6. Logging and review
- Every exchange logged with app, user and policy result
- Redacted copies only, with keys held separately
- Blocked requests queued for security review
- Exports to your SIEM
Control table
Which gateway control addresses each OWASP LLM risk?
Agnotic Technologies maps each control to the OWASP Top 10 for LLM Applications 2025, and every control has a test that proves it still works.
| OWASP risk | Gateway control | How we test it |
|---|---|---|
| LLM01 Prompt Injection | Input screening on user text and retrieved content, instruction and data kept in separate fields | Injection cases in the red-team suite on every release |
| LLM02 Sensitive Information Disclosure | PII and PHI redaction before the model, re-detection on output | Seeded test records that must never appear in a response |
| LLM05 Improper Output Handling | Schema validation and encoding before output reaches code, browsers or shells | Malformed and script-bearing outputs fed to the validator |
| LLM06 Excessive Agency | Tool allow lists, permission checks on parameters, human approval for writes | Attempts to call tools and actions the test user is not allowed |
| LLM07 System Prompt Leakage | No secrets in prompts, output scan for prompt fragments | Extraction prompts scored as fail if any fragment appears |
| LLM10 Unbounded Consumption | Token caps, rate limits and per-application budgets | Load tests that confirm limits trip and recover |
The remaining categories, such as supply chain and data poisoning, are handled in model selection and the retrieval pipeline, and are tested in the evaluation suite.
Failure points
Where do LLM guardrails fail?
These are the failures we have to plan for. None of them is solved by adding one more filter.
Challenge
Injection hidden in a retrieved document
Agnotic approach
Retrieved text is screened before it enters the prompt and is passed as data, never as instructions.
Challenge
Redaction that removes the meaning
Agnotic approach
Consistent tokens replace names and IDs, so the model can still reason about who is who, and the gateway restores them only for authorised users.
Challenge
Over-blocking that drives users around the gateway
Agnotic approach
Blocked requests are reviewed weekly and rules are tuned, so teams have no reason to call models directly.
Challenge
Logs that become the biggest PHI store
Agnotic approach
Logs keep redacted text by default, with access limited and retention set by your policy.
Challenge
Latency that breaks the product
Agnotic approach
Checks run in parallel where they can, with a latency budget per application and lighter checks for low-risk routes.
Challenge
An agent allowed to do too much
Agnotic approach
Tools are granted per application, parameters are checked against the user's own permissions and writes wait for approval.
Scope drivers
What drives the cost of an AI firewall?
We scope the gateway on a call. These factors move effort most.
Number of applications
Each application brings its own policies, tools and log needs.
Model providers
Routing across several providers and self-hosted models adds adapters and fallbacks.
Data types
PHI, payment data or legal documents each need their own detection and redaction rules.
Agents and tools
Tool brokering and approval flows are the largest part of the work for agent products.
Latency needs
Real-time voice or chat needs a tighter check design than batch document work.
Compliance as engineering
What does the gateway log, and who can see it?
The gateway becomes evidence for security reviews, so its own controls matter as much as the ones it enforces.
Audit record per exchange
Application, user, model, policy results and timing for every request, kept tamper-evident.
Role-based access to logs
Security reviewers see blocked events; raw content access needs a separate, logged approval.
BAA coverage end to end
Model providers that may receive PHI are covered by agreements before traffic is routed to them.
Key custody
Provider keys live in a secrets manager used only by the gateway, rotated on a schedule.
Compliance-First Healthcare App Development Services Backed by Global Standards
01
HIPAA
Health Insurance Portability and Accountability Act
Protect PHI with privacy-first architecture, encrypted storage and transmission, strict access controls, and traceable audit logs.
02
GDPR
General Data Protection Regulation
Implement lawful consent flows, data minimization, retention controls, and secure processing for sensitive health data.
03
FHIR
Fast Healthcare Interoperability Resources
Enable standardized health data exchange across apps, care teams, and systems through standards-based FHIR APIs.
04
HL7
Health Level Seven International
Support enterprise-grade interoperability with HL7-based integrations for records, events, and clinical messaging workflows.
05
HITRUST
Health Information Trust Alliance
Align security programs to healthcare-specific control and risk management practices trusted by providers and ecosystem partners.
06
HITECH
Health Information Technology for Economic and Clinical Health Act
Design with breach notification readiness, digital record safeguards, and operational controls that support regulated care programs.
07
SaMD
FDA Software as a Medical Device
Plan software quality, traceability, and documentation pathways for products that may require SaMD review and submission.
08
MDR (EU)
Medical Device Regulation (European Union)
Prepare EU market-ready processes for risk classification, evidence tracking, and lifecycle governance under MDR expectations.
09
SAMHSA
Substance Abuse and Mental Health Services Administration
Apply confidentiality controls and consent-aware sharing models for behavioral and mental health data experiences.
Facts and sources
Which risks does an AI firewall address?
Short facts behind the control table, each linked to its primary source.
OWASP lists prompt injection first in its 2025 Top 10 for LLM Applications, and notes that retrieval augmented generation and fine-tuning do not fully mitigate it.
Source: OWASP LLM01:2025 Prompt InjectionOWASP traces excessive agency, damaging actions taken in response to unexpected or manipulated LLM output, to three root causes: excessive functionality, excessive permissions and excessive autonomy.
Source: OWASP LLM06:2025 Excessive AgencyOWASP defines improper output handling as insufficient validation, sanitization and handling of LLM outputs before they are passed to other components, which can lead to XSS, SSRF, privilege escalation or remote code execution.
Source: OWASP LLM05:2025 Improper Output HandlingWhen prompts or responses can carry electronic protected health information, the HIPAA Security Rule requires mechanisms that record and examine activity in the systems that handle it.
Source: 45 CFR 164.312(b) (eCFR)
How we work
How does an AI firewall engagement work?
Three stages, starting with the riskiest application.
1.
Scoping call
We list the applications, models, data types and tools, and pick the first application to put behind the gateway.
One app
2.
Fixed-scope first release
Gateway ingress, routing, input and output checks and logging for that application, with its red-team cases.
Fixed scope
3.
Scale and support
More applications, tool brokering and rule tuning from the weekly review of blocked requests.
Tuned
Frequently Asked Questions
No, and we do not claim it can. OWASP notes that techniques such as retrieval augmented generation and fine-tuning do not fully mitigate prompt injection. The gateway lowers the chance of success, limits what a successful attack can reach through tool and data controls, and records the attempt.
No. A web application firewall inspects HTTP traffic for known attack patterns. An AI firewall understands prompts, model outputs and tool calls, and applies policy to their meaning. Many products run both.
Some. We set a latency budget per application, run checks in parallel where possible and use lighter checks on low-risk routes, then measure the added time in load tests before launch.
Yes. Applications switch their model calls to the gateway endpoint, usually with a small client change. We move one application at a time and keep the old path available until the new one is proven.
It depends on how many applications, providers and tools are involved. The first release covers one application end to end, and we fix its scope on the scoping call.
Yes. Gateway ingress, routing, redaction and logging for one application is a useful first release, and it gives you the audit trail even before stricter checks are switched on.
Engineers use AI tools for drafts and tests and review every change. Detection rules are tested against synthetic records and attack cases, and production prompts are never used for development without redaction.