TPU

Full definition

TPUs are an example of a domain-specific accelerator (ASIC) optimized for AI rather than the general-purpose flexibility of GPUs. Google has shipped six generations from TPU v1 (2015) through Trillium (TPU v6) and Ironwood, with custom interconnect, memory, and topology. Available externally via Google Cloud. Other notable AI ASICs include AWS Trainium/Inferentia and Microsoft Maia.

Why it matters

TPUs and other ASICs reduce dependence on NVIDIA, change the cost curve for AI training and inference, and force every large enterprise to make a multi-vendor compute strategy decision. The hyperscalers' multi-billion-dollar bets on custom silicon will reshape AI economics over the next 5 years.

Example

Google trained Gemini 1.5 on TPU v5p pods, demonstrating that frontier-class training is feasible without NVIDIA GPUs — a point the rest of the industry has watched closely.

Related terms

• GPU (in AI context)
  A GPU (Graphics Processing Unit) is a massively parallel processor that, for AI workloads, executes the matrix multiplications at the heart of neural networks 10-100× faster than a CPU — and the dominant hardware for both training and inference of modern AI models.
• Inference Cost
  Inference cost is the dollar cost of running a trained AI model in production — per request, per user, or per business outcome — and the operating expense that determines whether an AI feature has positive unit economics at scale.
• Inference Server
  An inference server is the runtime system that hosts trained AI models behind an API, handling request routing, dynamic batching, KV-cache management, scheduling across GPUs, and hardware acceleration — the layer that turns a model file into a production AI endpoint.
• Foundation Model
  A foundation model is a large model trained on broad data at scale — typically self-supervised — that can be adapted to many downstream tasks via prompting, fine-tuning, or retrieval, instead of being trained task-by-task.
• Edge AI
  Edge AI is the practice of running AI models on local devices — smartphones, vehicles, sensors, factory equipment, AR/VR headsets — instead of the cloud, in order to deliver lower latency, stronger privacy, and continued operation when offline.