What are tool alarms and warnings, and why shouldn't you just dismiss them?

A tool's warning is a reminder: some number is starting to look off, but the tool can keep running, usually under a yellow light. An alarm is a more serious fault: the tool often stops itself, usually under a red light. Fab tools call out many times a day, and most of it is nothing serious, so an easy mistake to make is dismissing one without reading it.

What's the difference between an alarm and a warning?

Think of a phone: when the battery is running low, it pops up a reminder and you can keep using it; that's like a warning. When the battery is too low to go on, the phone shuts itself down; that's like an alarm.

A tool works the same way. During a warning, the tool carries on and simply reminds someone to come and take a look, for example when a gas flow is getting close to its upper limit but hasn't passed it. During an alarm, the step in progress may be cut off and the tool stops and waits for a person. One university cleanroom's tool manual draws the line exactly like that: a "hard" alarm aborts the process that's running, while a "soft" one only reminds you and the process keeps going.

The names and categories differ from fab to fab and tool to tool, but the idea is the same. For how the tower light on top of the tool lights up, and why the yellow light is the one most often ignored, see What do the red, yellow and green lights on a tool mean?

How does a tool know when to call?

A tool is full of sensors that keep measuring pressure, temperature and gas flow. Someone has set limits for each number in advance, and when a number crosses one, the tool calls.

A common approach is two lines for the same number: touch the one closer to normal and the tool only raises a warning; cross the one further out and it raises an alarm and stops. The inner line calls earlier, which catches problems sooner, at the price of more "crying wolf".

How these numbers are watched nonstop and how the limits are set: What is FDC? How measured results are judged against control limits: What is an SPC control chart?

What is an interlock, and why does a tool sometimes refuse to move?

On top of calling for people, a tool has a harder kind of protection called an interlock: while a dangerous condition is present, certain actions simply can't happen.

The everyday example is a microwave oven: open the door and it stops; if the door isn't shut properly, pressing start does nothing. Tools work the same way: for example, while a panel is open and the high voltage or RF power inside could be touched, that power can't be switched on.

The point of an interlock is that it doesn't depend on someone remembering: if the conditions are wrong, the tool blocks it on its own. So when an interlock stops you, the job is to deal with whatever is triggering it, not to find a way around it.

When an alarm goes off, what happens, step by step?

The details differ from fab to fab, but the order is roughly this:

Step 1: See exactly which alarm it is, and whether there are wafers inside

• What: Go to the tool and read the screen: which tool, which chamber, which alarm, which step it stopped at, whether there are wafers inside and which lot they belong to.
• Why: The causes can differ a lot; whether wafers are inside and where the tool stopped decide what to do next.
• If you skip it: Acting before you've looked can mean a wafer stuck inside gets moved one more time.

Step 2: Follow the OCAP and stop the damage first

• What: Work through the tool's out-of-control action plan, the OCAP, line by line: stop what needs stopping and put the lots that may be affected on hold.
• Why: The OCAP is an order written in advance; in the middle of the night, when people are tired and rattled, following it is the surest way not to miss a step.
• If you skip it: Guessing while the tool keeps running can mean the lots behind get ruined too.

What an OCAP contains and why it's written in advance: What does OCAP mean in a chip fab?

Step 3: Notify the right people

• What: Call the engineer in charge: problems with the tool itself go to Equipment, problems with the lots go to Process; at night, call whoever is on-call, and say clearly what you saw and what you've already done.
• Why: Whether to open the tool for repair, and whether the wafer stuck inside can still be used, are calls for people who know that tool.
• If you skip it: Forcing a fix yourself can turn a small problem into a big one; waiting until morning to tell anyone leaves the tool down for hours longer.

How the people on the line and the engineers hand off to each other at night: What's the difference between a fab technician and an engineer?

Step 4: Confirm the cause, then reset

• What: Only once the cause has been found and removed do you clear the error state and let the tool start again. That's a reset.
• Why: A reset doesn't fix anything; it only clears the "something went wrong" state.
• If you skip it: With the cause still there, the same alarm comes back soon, and it can be worse (next section).

Step 5: Write it down

• What: Note when it went off, which alarm, what was done and who handled it; keep the tool's own alarm log too, don't clear it. Anything still open goes into the handover notes.
• Why: The same alarm coming back again and again can be an early sign of a part slowly wearing out; only a record shows you "it's happening again".
• If you skip it: Every time feels like the first time, until something big happens and it turns out the tool had been calling for a long while.

Why can't you just keep pressing reset?

Press reset, the light turns green and the tool runs again. It looks like the fastest way. But:

• The problem isn't solved: Whatever stopped the tool is still there, so the alarm comes back.
• It can damage wafers or the tool: A tool usually stops because carrying on would cause trouble. Running again with a wafer that isn't sitting properly, or moving a wafer stuck halfway one more time, can damage both the wafer and the tool's parts.

One university cleanroom's manual puts it plainly: if an alarm appears, halt the process, and continue only after the problem has been corrected or the staff who run the tool have been told.

What is alarm fatigue?

When alerts go off too often and are mostly nothing, people start dismissing them out of habit. That's alarm fatigue. Hospitals have been discussing it for a long time: the monitors by the bed keep beeping, most of the beeps need no action, and staff gradually stop hearing them, even turning the volume down or widening the limits.

Factories are no different. Hear the same sound a hundred times with nothing wrong, and anyone gets used to it; the trouble is that the one that really matters sounds exactly like all the others. This isn't simply one person not being careful enough, so the fix isn't just telling everyone to be more careful: it's making the alarms themselves fewer and more accurate.

How do you keep the alarms that matter from getting buried?

Industry has a dedicated practice for this, called alarm management. Some common principles:

• Every alarm should need someone to do something: one that goes off but needs no one to act shouldn't be an alarm; it can be turned into a simple log entry.
• Set limits sensibly: too tight, and a bit of normal wobble sets it off; too loose, and by the time it goes off it's too late.
• Revisit the ones that go off too often: review the records regularly and find the most frequent ones. Is the limit wrong, is a sensor faulty, or is there a real problem? For a number that wobbles around its limit and keeps setting the alarm off and clearing it, add a little margin.
• Rank by importance: put the most important ones first, so people see them at a glance.

In fab24hr

Tap "Clock in", choose "A day in the fab", then on the "Your role today" card tap "Production fab" and swipe to Manufacturing. Which of the four shifts you get depends on the time you open the site. On the late-night shift, at 02:47, Chamber A of the etch tool ETCH-03 turns red with an alarm, and wafer 7 of the hot lot H-2417 is stuck inside. Hao, on the night shift, says a reset and another try usually gets it through; the OCAP says: tool alarm, notify the engineer first. You can follow the OCAP and call the on-call process engineer, or try one reset first. Press it, and the tool thinks for three seconds and throws the same alarm, with one more line underneath: WAFER NOT CHUCKED. The chuck won't grip the wafer.

As the etch process engineer (swipe to the process engineer and tap "Etch" under the name), on the morning shift you go to ETCH-03 and read the alarm log: the same warning, on a yellow light, came up 6 times in the last three days, and it was dismissed every time. If you stay on the evening shift, the helium flow on Chamber A touches the warning line and a yellow light comes on: "Helium flow is near the upper limit. The tool can keep running." You can dismiss it and go home, or open a ticket for the night shift.

The companies, people, tool IDs and numbers in the game are fictional; what alarms and warnings are and how they're handled come from public, general sources, and every fab does it differently. For other jargon, see Chip fab jargon in plain English.