TPU

Full definition

TPUs are an example of a domain-specific accelerator (ASIC) optimized for AI rather than the general-purpose flexibility of GPUs. Google has shipped seven generations, from TPU v1 (in its data centers since 2015) through Trillium (TPU v6) to Ironwood, the seventh, with custom interconnect, memory, and topology, and in 2026 it announced an eighth generation split into two chips: TPU 8t for training and TPU 8i for inference. Available externally via Google Cloud. Other notable AI ASICs include AWS Trainium/Inferentia and Microsoft Maia.

Why it matters

TPUs and other ASICs reduce dependence on NVIDIA, change the cost curve for AI training and inference, and force every large enterprise to make a multi-vendor compute strategy decision. The hyperscalers' multi-billion-dollar bets on custom silicon will reshape AI economics over the next 5 years.

Example

Google trained Gemini 1.5 on multiple 4,096-chip pods of TPU v4 accelerators, demonstrating that frontier-class training is feasible without NVIDIA GPUs — a point the rest of the industry has watched closely.

Next steps

• Get a free workflow map →We map one of your workflows and give you a written verdict: software, a process change or AI. No obligation.
• AI consulting cost guide →Hourly rates and project budgets by firm tier, from independents to the Big Four.

Related terms

• GPU (in AI Context)
  A GPU (Graphics Processing Unit) is a massively parallel processor that, for AI workloads, executes the matrix multiplications at the heart of neural networks many times faster than a CPU — and the dominant hardware for both training and inference of modern AI models.
• Inference Cost
  Inference cost is the dollar cost of running a trained AI model in production — per request, per user, or per business outcome — and the operating expense that determines whether an AI feature has positive unit economics at scale.
• Inference Server
  An inference server is the runtime system that hosts trained AI models behind an API, handling request routing, dynamic batching, KV-cache management, scheduling across GPUs, and hardware acceleration — the layer that turns a model file into a production AI endpoint.
• Foundation Model
  A foundation model is a large model trained on broad data at scale — typically self-supervised — that can be adapted to many downstream tasks via prompting, fine-tuning, or retrieval, instead of being trained task-by-task.
• Edge AI
  Edge AI is the practice of running AI models on local devices — smartphones, vehicles, sensors, factory equipment, AR/VR headsets — instead of the cloud, in order to deliver lower latency, stronger privacy, and continued operation when offline.

More infrastructure terms

• KV Cache
  The KV (Key-Value) cache stores the attention keys and values from already-processed tokens so an LLM can generate each new token without recomputing past work — the single largest consumer of GPU memory during LLM inference, and the primary lever for serving throughput.