The AI Compute Supply Chain

Parameters / weights

The individual numbers inside a model that were learned from data. A model with 'one trillion parameters' is literally storing and doing math on a trillion of these numbers. More parameters generally means more capability — but also more compute to train and more memory to hold.

Analogy: Picture an enormous recipe with billions of precise measurements — a pinch more here, a little less there. Training is tasting the dish over and over and tweaking every measurement until it comes out right. A model with more parameters is a recipe with more ingredients to balance — and it needs a far bigger kitchen just to lay them all out.

Transformer (AI architecture)

The neural-network design introduced in 2017 that lets a model weigh how every word in an input relates to every other word, all at once, in parallel. That 'all at once' is the key property — older architectures processed text one step at a time, which was slow and lost long-range context.

Analogy: An older model reads a sentence like a person with a finger under each word, one at a time. A transformer (AI) lays the whole page on a table and looks at how every word connects to every other word simultaneously. That parallel 'look at everything at once' is exactly the math GPUs are built for.

Training vs inference

Training is the one-time (or periodic) job of teaching the model: a massive computation run across thousands of chips for weeks or months to set all the parameters. Inference is running the finished model to answer a request — and it happens constantly, in parallel, for every user prompt, forever.

Analogy: Training is the model doing months of homework to learn the subject. Inference is the model sitting the exam — over and over, a fresh exam for every single user request, every second of every day. Homework is a huge one-off cost. The exams never stop, so speed and cost per exam are what dominate the economics.

GPU vs CPU

A CPU (central processing unit) has a few very powerful cores that do complicated tasks quickly, one after another. A GPU (graphics processing unit) has thousands of simpler cores that all do basic math at the same time. Transformers are mostly one operation — multiplying big grids of numbers (matrix multiplication) — repeated billions of times, which is exactly the 'lots of simple math in parallel' that GPUs win at.

Analogy: A CPU is a few PhDs solving hard problems sequentially. A GPU is ten thousand bright high-schoolers all doing arithmetic at once. For one tricky logic puzzle you want the PhDs. For multiplying a million pairs of numbers right now, the crowd of high-schoolers obliterates them.

DRAM and HBM (first pass)

DRAM is fast, temporary working memory — where a chip keeps the data it is actively using. HBM (high-bandwidth memory) is DRAM that has been stacked into vertical towers and bolted right next to the GPU so data can flow between them extremely fast. HBM is not a different material from DRAM; it is DRAM in a much higher-performance package.

Analogy: DRAM is the open notebook on your desk — quick to read and write, wiped when you finish. HBM is that same notebook, but thickened into a stack and clamped against the processor so it can flip pages many times faster. (We will meet NAND — the filing cabinet — later.)

Wafer

A thin, polished disc of ultra-pure silicon, usually 300mm across. Chips are not made one at a time; hundreds are fabricated together across the surface of one wafer, then cut apart at the end.

Analogy: A wafer is a sheet of cookies. You print the whole sheet at once, then cut individual cookies (chips) out of it. A defect anywhere on the sheet ruins the cookies it touches.

EUV lithography

Lithography is printing the circuit pattern onto the wafer with light. The smaller the features you want, the shorter the wavelength of light you need. EUV (extreme ultraviolet) uses light so short that only one company on Earth has ever built a working production machine for it — and the most advanced chips simply cannot be made without it.

Analogy: Lithography is a stencil-and-spray-paint step for circuits. Ordinary light is a fat spray nozzle; you can only paint chunky shapes. EUV is an impossibly fine airbrush that can paint lines a few atoms wide. Below a certain chip size, the fat nozzle physically cannot draw the picture — you must have the airbrush.

CoWoS / advanced packaging and the silicon interposer

A modern AI accelerator is not one chip — it is a logic die (the GPU brain) plus several HBM memory stacks that must sit millimeters apart and be wired together with enormous bandwidth. CoWoS (Chip-on-Wafer-on-Substrate) is TSMC's process for mounting all of them onto a shared silicon base called an interposer, so they behave as one tightly-coupled unit.

Analogy: CoWoS is the soldering step that glues the brain chip and its memory onto one shared baseboard, with thousands of tiny wires between them, so they can talk almost as if they were the same chip. You can have a perfect brain and perfect memory and still have nothing usable until this assembly step is done — and there is a long queue for the assembly bench.

TSV (through-silicon via)

A TSV is a microscopic vertical wire drilled straight through a silicon chip, letting you stack chips on top of each other and connect them directly instead of routing signals around the edges. TSVs are what make HBM's vertical DRAM towers — and the interposer underneath — physically possible.

Analogy: TSVs are elevators punched through the floors of a building so people can go straight up instead of walking out to an external staircase every time. Stacked memory is only practical because each layer has its own elevators.

NAND flash

NAND is non-volatile storage — it keeps data even with the power off, and it is cheap per gigabyte but slower than DRAM. It is where datasets, model checkpoints, and caches live. Critically, NAND is made on its own production lines, not on DRAM wafers.

Analogy: If DRAM is the open notebook on your desk, NAND is the filing cabinet across the room: far bigger and cheaper, keeps everything when you go home, but slower to fetch from. AI inference needs an enormous filing cabinet — which is its own, separate squeeze.

MRAM

MRAM (magnetoresistive RAM) stores bits using magnetic state rather than electric charge. It is non-volatile (keeps data with power off) and very power-efficient, which is attractive when energy is the ceiling. Today it is a niche, low-density technology.

Analogy: MRAM is a memory that 'remembers' magnetically, like a compass needle holding its direction — no power needed to retain the bit. Promising for an energy-constrained world, but still small and specialized.

CXL

CXL (Compute Express Link) is a standard that lets many chips share a common pool of memory over a fast link, instead of each chip being stuck with only its own. It promises to ease memory shortages by letting capacity be pooled and reallocated where needed.

Analogy: CXL is a shared supply closet for an office floor, instead of every desk hoarding its own stash. More efficient overall — but the building has to be wired for it first.

Silicon photonics / co-packaged optics

Silicon photonics moves data using beams of light inside the chip package instead of electrical signals over copper. Light carries far more data over distance with far less energy and heat. Co-packaged optics puts the light engine right next to the switch chip, instead of in a pluggable module at the edge.

Analogy: Copper wires are like shouting across a noisy room — fine up close, useless across a stadium, and exhausting at volume. Photonics is sending the message by laser instead: it goes much farther, much faster, and barely tires you out. At data-center scale, the room has become a stadium.

Capex vs opex

Capex (capital expenditure) is upfront money spent to build or buy long-lived assets — the data center, the GPUs, the power plant. Opex (operating expenditure) is the ongoing cost of running them — electricity, staff, maintenance. Power is unusual because it is both: a giant capex problem to connect, and a relentless opex problem to feed.

Analogy: Capex is buying the car; opex is the gasoline you keep buying forever. For AI, the GPUs are the car — but in 2026 you cannot even get the car onto the road, because the on-ramp (grid power) is jammed.

Neocloud

A neocloud is a specialized cloud provider that does essentially one thing: buy the latest GPUs at scale and lease that compute to AI companies on contracts, usually multi-year. Unlike AWS or Azure, they are not full-service clouds — they are pure GPU-capacity landlords.

Analogy: A neocloud is a company that buys a fleet of the world's most in-demand bulldozers and rents them by the month. The bet is simple: demand for bulldozers wildly exceeds supply, and long leases lock in the return before the bulldozers arrive.