Multimodal Model
A multimodal model is an AI model that natively understands or generates more than one type of input — typically text plus images, audio, or video — within a single network rather than via separate task-specific models.
Full definition
Multimodal models map different input types into a shared representation space, then condition generation or prediction on any combination of them. Examples include GPT-4o (text + image + audio in/out), Gemini 1.5 (text + image + audio + video in), and Claude 3.5 Sonnet (text + image in). Multimodality reduces brittle "stitching" of separate models, improves grounding, and enables new use cases such as document understanding, screen-reading agents, and video analysis.
Why it matters
Real enterprise data is multimodal — invoices have layouts, calls have audio, products have photos, contracts have signatures. A multimodal foundation model collapses what used to be 5 separate AI systems into one, simplifying procurement, MLOps, and governance.
Example
An insurance carrier uses a multimodal model to read claim documents (PDF + photos of damage + voice notes from the adjuster) and produce a structured claim summary in seconds — work that previously required three separate AI tools.
Related terms
- Foundation ModelA foundation model is a large model trained on broad data at scale — typically self-supervised — that can be adapted to many downstream tasks via prompting, fine-tuning, or retrieval, instead of being trained task-by-task.
- Large Language Model (LLM)A Large Language Model (LLM) is a deep neural network — almost always a transformer — trained on hundreds of billions to trillions of words to predict the next token, and to generate, summarize, translate, or reason over text.
- EmbeddingAn embedding is a dense numerical vector — typically 384 to 4096 dimensions — that represents the semantic meaning of a piece of text, image, audio, or other content, so that semantically similar items end up near each other in vector space.
- Generative AIGenerative AI is a class of AI systems that produce new content — text, images, code, audio, or video — by learning the distribution of their training data and sampling from it, rather than classifying or predicting from existing inputs.
Source & further reading
Primary source: OpenAI — "GPT-4o System Card" (2024).
Citation policy: this entry is part of the AIDOLS AI Implementation Glossary and may be quoted for research, journalism, and education with attribution to aidolsgroup.com/es/glossary/multimodal-model/.