Alarm floods and silent failures: signals that mislead

01A light that reported a command

At the Three Mile Island plant in Pennsylvania, a sequence began at 4:00 in the morning on 28 March 1979. Pressure in the reactor system rose, and a pilot-operated relief valve at the top of the pressurizer opened, as it was designed to do. The valve should then have closed as pressure fell. It did not. It remained open, and coolant escaped from the primary system.

The Kemeny Commission's finding describes this as a small-break loss-of-coolant accident. The detail that matters for attention is the control room indicator. The light for the valve showed only that a signal had been sent to close it, not that the valve remained open. A light that reports a command reads like a light that reports position, and the difference between the two is the difference between a reassuring display and an accurate one.

02What an indicator actually measures

Every indication is a claim about the system, and its meaning depends on what it is connected to. A position light might report the command sent to an actuator, feedback from a sensor on the equipment, or an inference drawn from other readings. Operators learn the difference through training, but a panel label can invite the wrong reading. Designers who take this seriously ask what each light is connected to, and they test it under failure conditions, not only when everything works.

Three questions help. What physical state does this indicator claim to show? What signal does it actually receive? And what would it display if the component did the opposite of what it was told? If the answers differ, the display describes an intention rather than a fact. The same test applies to dashboards, status pages and alarm lists, all of which are built from signals that carry assumptions.

03Silence that looked like health

On 14 August 2003 a cascade of failures blacked out parts of the United States and Canada. The US-Canada task force that investigated later called the blackout preventable. Its findings were not reducible to one broken part. Transmission lines contacted trees that had grown into their clearances and tripped out, and monitoring and alarm functions failed. The operators lost an accurate picture of their system's condition.

The report's grouping of causes includes a failure to manage tree growth and a loss of operator understanding of system conditions after the primary monitoring and alarm systems failed. That second item is the attention lesson. When an alarm system fails, it does not announce its own failure. It simply stops producing alarms, and a quiet control room can look like one in which nothing is wrong. Silence is evidence only if the monitoring is known to be working, so the monitoring itself needs monitoring.

Monitoring design can make silence more informative. A system can report its own health at regular intervals, so that a missing heartbeat becomes an alarm in its own right. Operators can be shown which monitors are live, and a standing procedure can say what to do when one disappears. None of this happens automatically. It has to be specified, tested and reviewed like any other safety function.

04A gauge that stopped at 80 degrees

The Apollo 13 oxygen tank shows a different blind spot. Before the flight, the tank could not be emptied normally, so its heaters ran for an extended period to boil off the oxygen. The temperature gauge was designed to read only to 80 degrees Fahrenheit. The heating went far beyond that, so the extreme temperature was not noticed. The problem was not a misreading within the scale. The scale simply ended before the temperature that had been reached.

The heat damaged the insulation on wires feeding fans inside the tank. On 14 April 1970, about 56 hours into the mission, the fans were operated, the exposed wires shorted, and the insulation burned in the pure oxygen environment. The tank's thermostatic switches should have limited the heating. They had been welded closed by an arc when they reached their upper temperature limit, so the protective layer failed silently.

A practical check is to ask what the instrument would show at the moment it matters most. If a reading saturates, sticks at its limit or simply stops updating, the operator needs a separate indication that the true value may lie beyond the range. The absence of such an indication is the blind spot, and it is easy to miss because the gauge still appears to work.

05A recurring mark that needed a name

In 1967 Jocelyn Bell Burnell, a research student at Cambridge, was working through chart records from a radio telescope. Routine work meant recognizing and setting aside interference, the marks left by arc welders, sparking thermostats and badly suppressed car engines. Most features had explanations.

One did not. From one particular patch of sky, an unclassifiable signal sometimes recurred. Her later account describes a pull toward recalling something similar seen before, and the discipline was to test that impression rather than file the mark away. A recurring mark has a location and a rhythm, and both can be checked. That is a different kind of signal from a single spike, and it deserves its own routine.

The first decision was not that the signal came from an unknown kind of star. It was that the signal deserved classification. Follow-up recording at higher time resolution and careful location of the source turned a mark on a chart into a candidate source, and the interpretation came later.

06Testing whether silence means health

Four questions draw the cases together. First, what does each signal measure: a command, a position, a flow or an inference? Second, what can the instrument's range show, and what happens beyond it? Third, what happens to the monitoring system when it fails, and does anyone know? Fourth, when a recurring signal does not fit, treat it as an observation that needs classification rather than as noise.

This work is slow, and it cuts against the pressure of a busy control room or a fast market. Alarm systems fail in two directions. They can flood operators with messages that all look equally urgent, and they can fall silent while a system deteriorates. Both are failures of attention, and both can be reduced by deliberate design, testing and review.