The news says "advanced packaging". What is it, and why do chips sit side by side or stack up?

When the news talks about "advanced packaging", it means putting several chips into one package, so close together and so densely connected that they work like one big chip. There are two main ways to do it: lay the chips side by side on a thin slice with very fine wiring, or stack them on top of each other. To see why, start with ordinary packaging.

How does ordinary packaging work?

After chips are made on a wafer, they are cut apart and each one goes into a case so it can be soldered onto a circuit board. That is packaging (What is chip packaging covers it). Traditionally it is mostly one chip per package. When several chips need to work together, each is packaged on its own and soldered onto a circuit board, and they send data to each other through the wiring on the board.

Advanced packaging is different: several chips are put together and connected first, and only then packaged as one part.

Why not just make one big chip?

Putting everything on a single chip makes the chip very large, and large chips have two problems.

First, large chips have lower yield. A wafer always picks up a few defects too small to see, and whichever chip one lands on is ruined. The bigger the chip, the more likely it is to be hit (the article on yield in chip news explains this with cookies).

Second, chips have a size limit. A scanner prints one field at a time, and a field has a fixed size. The largest chips are already about as big as a whole field; any bigger, and they won't fit in one field.

What do you gain by splitting a chip into pieces?

Try some made-up numbers. A wafer holds 40 large chips. Ten defects land on it and ruin at most 10 chips, so at least 30 good ones are left.

Now split each large chip into 4 identical small pieces, and the same wafer holds 160. The same 10 defects ruin at most 10 pieces, leaving at least 150 good ones; put 4 together as a set and you get 37 sets. Same wafer, and what you can use goes from 30 chips to 37 sets.

These small pieces are what the news calls chiplets: a big chip split into a few pieces, each made and tested on its own, then put back together. Reality is more complicated: split pieces need extra room for connections, and the packaging gets harder, so not every chip is worth splitting.

What does each generation costing more have to do with it?

Each new process generation is harder and more expensive to make than the last (the article on "2 nm" covers this). But not every part of a chip needs the newest generation. The parts that talk to the outside world or handle analog signals, for example, hardly get any smaller on a newer one.

With chiplets, the piece that needs it most (the part that does the computing, say) is made on the newest generation, and the other pieces on older generations that are cheaper and well proven; then they are packaged together. The money goes where it matters most.

Why do the roads between chips need to be shorter and wider?

When two chips are packaged separately and soldered onto a circuit board, data has to leave one package, travel a few centimeters along the board's wiring, and enter the other. It is a long road with few lanes, like two cities linked only by a long country lane.

Inside one package, the chips sit a tiny distance apart, and the wiring between them can be fine and plentiful, like swapping the country lane for a short highway with many lanes. A shorter road means data arrives faster and uses less power; more lanes mean more data at once.

How do chips sit side by side, or stack?

Side by side on an interposer: 2.5D

Several chips sit side by side on a thin slice called an interposer. Interposers are often made of silicon, because chip-making methods can draw wiring on it far finer than a circuit board's, connecting neighboring chips densely. The chips are turned upside down and soldered on with tiny bumps, like the flip chip in the packaging article; the interposer then carries the signals down to the package substrate below.

The industry calls this 2.5D: the chips are not stacked, but there is an extra layer underneath, somewhere between lying flat and stacking.

Stacked: 3D

The other way is to stack chips layer on layer. How do two layers connect? Very fine holes are made in the silicon and filled with metal, so signals pass straight through a chip from one layer to the next. These holes are called through-silicon vias (TSVs), and they are much thinner than a human hair.

Stacking gives the shortest road and takes the least space. The price is that it is harder to make: the chips have to be ground very thin, and heat has a harder time escaping from the layers in the middle.

AI chips often have a few stacks of memory next to them: several memory chips stacked and connected by through-silicon vias, called high bandwidth memory (HBM). What it is and why AI needs it so much will get an article of its own.

Why does the news call it key to AI chips?

AI has an enormous amount to compute, and data moves back and forth between the computing chip and the memory all the time. The largest chips are already at the one-field limit, so fitting in more computing and more memory means packaging several chips together, close enough and densely connected enough. That is why an AI chip is often a computing chip with a few memory stacks next to it, all on one interposer: 2.5D and 3D at once. One government research program on advanced packaging puts it plainly: recent advances in AI would not be possible without advanced packaging.

Why does the news keep saying there isn't enough capacity?

The news often says advanced packaging capacity is "tight". In general terms: demand for AI chips has grown very fast in the last few years, and most high-end AI chips need this kind of packaging. It takes many steps, and some of them, drawing wiring on silicon and making holes through it, use tools and methods closer to a wafer fab's; a new line takes time to build and to bring to stable output. So sometimes the chips themselves can be made, but they get stuck at packaging. And because some of the steps are so like a wafer fab's work, advanced packaging is done not only at test-and-assembly houses but at wafer fabs too.

It also leaves little room for mistakes: every chip in the package is expensive, and if one piece is bad or one connection fails, the whole set is wasted. So every piece is tested before it is put together (wafer sort).

Next time you see it in the news

• "Advanced packaging capacity is tight": there are not enough lines that can package several chips together. AI chips may be held up here, even when the chips themselves can be made.
• "A chiplet design": this chip is really several pieces put together, each made and tested on its own, then packaged as one.
• "2.5D and 3D packaging": 2.5D means side by side on an interposer; 3D means stacked, connected by fine holes through the silicon.
• "Heterogeneous integration": separately made chips of different kinds (different jobs, even made on different generations), sometimes along with other small parts, packaged together in one package.

In fab24hr

Act three of the main story, The life of a chip, takes place at a test-and-assembly house. In chapter 11, "The Ball Lifted", Shu-Hui walks you down the assembly line: the game's hearing-aid chip, Glowmere, goes into an SiP module, a small board with the chip and a few resistors and capacitors next to it. First SMT and die attach, then wire bonding and molding. That is also more than one thing in one package, but the module holds only one chip, connected by wire bonding; chips side by side on an interposer, or stacked, are not in the game. The companies, chips and people in the game are fictional; what advanced packaging is and why it is done follow public, general sources, and the numbers in this article are made up. For other jargon, see Chip fab jargon in plain English.