Foundation model vs LLM: which model class does your feature need?
Foundation model and large language model are often used as if they meant the same thing. They do not, and the gap matters when you pick a model for a feature: an LLM handles text, while the wider foundation model family also covers documents and images, speech, embeddings for search and domain-specific models.
Which model class does each AI feature need?
Quick answer
Foundation model vs LLM is a question of scope. A foundation model is any large model pretrained on broad data and adapted to many tasks; an LLM is the text member of that family. Choose by feature: text work suits an LLM, while images, speech and search need other model classes. Agnotic Technologies matches each feature to its class.
| Criterion | Foundation model | LLM |
|---|---|---|
| Chat assistant over company documents | An embedding model for retrieval plus a generative model for the answer; test retrieval and answer grounding separately | Yes, as the answer writer, but it still needs retrieval to stay within your documents |
| Fields from scanned documents | A multimodal or document vision model that reads layout; test field by field on real scans | Only after OCR turns the scan into text, and layout cues are lost on the way |
| Image analysis (defects, damage, objects) | A vision foundation model adapted with labelled images; test per class on held-back images | Not a text-only LLM; a multimodal LLM can describe an image but is not a calibrated detector |
| Semantic search | An embedding model; test with real queries and the documents each one should return | Not needed to rank results; it can rewrite the query or summarize the top hits |
| Speech transcription | A speech recognition model; test word errors on your own recordings, accents and vocabulary | Not for transcription; it can clean up or summarize the transcript afterwards |
| Risk or propensity score | Usually no foundation model at all: a trained model on tabular history fits | No: it gives no calibrated probability for rows of numbers |
| Drafting replies and summaries | A text generation model, which means an LLM | Yes; reviewers score drafts against a rubric |
| Hosting options | Embedding and speech models are often small enough to self-host next to your data | Hosted APIs or open-weight models; larger open-weight models need GPU capacity to self-host |
What they are
What is Foundation model
A foundation model is a large model pretrained on broad data, then adapted to many downstream tasks. The family includes LLMs, multimodal models that take images or documents alongside text, speech models, embedding models for search, image generation models and models pretrained on domain data such as code or sensor signals.
What is LLM
A large language model is a foundation model trained on text to predict the next token. It reads and writes language, follows instructions and can return structured output such as JSON. Every LLM is a foundation model, but many models in a product stack are not LLMs.
Key differences
Where does the term foundation model come from?
It describes models trained on broad data that can be adapted to a wide range of tasks, as opposed to a model built for one job. NIST AI 600-1, the Generative AI Profile, notes that not all generative AI is a foundation model, while the profile itself uses the term to mean generative foundation models. It quotes the Executive Order 14110 definition of generative AI as models that emulate the structure and characteristics of input data to generate derived synthetic content, such as images, videos, audio and text. For shared vocabulary across a team, ISO/IEC 22989:2022 sets out AI concepts and terminology.
Which model classes sit around an LLM in a product?
Embedding models power search, deduplication and clustering. Multimodal models read documents and images with text. Speech models transcribe and generate voice. Small language models handle classification, routing and on-device tasks. Domain models cover code, biology or sensor data. Each one is chosen, versioned and tested on its own, even when users only see one assistant.
Should you call a hosted API or run open weights?
With a hosted API the provider runs the model, your data leaves your environment and the model version changes on the provider's schedule, so check data terms, regions and retention. With an open-weight model you run it yourself and own patching, scaling, GPU capacity and checks on the model files you download. The OWASP Top 10 for LLM Applications 2025 lists supply chain risk (LLM03) for third-party models and components.
How is each model class tested?
Each class has its own test. Embedding models are judged on whether real queries return the right documents. Document models are judged field by field on real scans. Vision models are judged per class on images they never trained on. Speech models are judged on word errors in your own audio. LLM output is judged against a rubric. Our LLM evaluation services page covers the generative side in depth.
What security risks come with each class?
Generative models bring prompt injection and sensitive information disclosure, the first two entries in the OWASP Top 10 for LLM Applications 2025. Embedding stores bring their own: the same list names vector and embedding weaknesses (LLM08). Once your documents are stored as vectors, apply the same access rules to the vectors as to the source documents, or search will leak what users cannot open.
Does a bigger model make a better feature?
Not by default. A feature is judged on its own test set, and a smaller or specialized model often passes it with lower latency and running cost. Start with the smallest model class that meets the bar, and move up only when the test set shows a gap.
When to use each
When to use Foundation model
- Reading scanned forms, photos or charts, where a vision or multimodal model can see the layout.
- Semantic search and retrieval, where an embedding model turns text into vectors for similarity lookups.
- Transcribing calls or dictation with a speech recognition model.
- Specialized inputs such as code, camera images from a production line or sensor signals, where a domain-pretrained model fits better than a general one.
When to use Large language model (LLM)
- Chat assistants and question answering over your own documents.
- Drafting, rewriting and summarizing text for a person to review.
- Extracting fields from text that is already digital, such as emails or PDFs with a text layer.
- Turning a plain-language request into a structured call to one of your APIs.
How do you choose a model class for a new feature?
1. Write down the input and the output. Name the input type (text, scan, image, audio or table) and the output (text, label, score or ranked list).
2. Let the input pick the class. Text in and text out points to an LLM; scans, images, audio and search point to other foundation models; tables point to a trained model.
3. Build the test set first. Collect real inputs with checked outputs before comparing any models.
4. Compare a few candidates on that set, always including one smaller model.
5. Decide hosting with your data rules in hand: where data may go, how long it is kept and who controls the model version.
6. Adapt only as far as the test set requires. Whether to prompt, retrieve or fine-tune is a separate choice, covered on our generative AI development page.