Faultory

Case file 67 · Engineering & technology

A chip 24 billion kilometers away

Voyager 1, November 2023 to April 2024: how did engineers diagnose a failed memory chip from interstellar distance, and work around it?

Subject
Voyager 1 memory fault (2023–2024)
Reading
12 min
Evidence
5 exhibits
Reasoning
4 steps
Level
Intermediate
A Voyager spacecraft in a large test hall, with its dish antenna above and technicians in white coveralls working around a mobile stand.
Photographed November 1976 A Voyager spacecraft under test at JPL in November 1976, the year before both launches. Photo: John Gregoire, NASA/JPL · Public domain · Wikimedia Commons.
Illustrative signature. An illustrative strip-chart sketch of the telemetry signal: a steady carrier, then noise while the memory fault blocks usable data, then a partial return after the workaround (a sketch, not data). Drawn to explain the case, not plotted from source data.

01 · The briefing

Begin with the question.

Voyager 1 was launched in 1977 and is now more than 15 billion miles (24 billion kilometers) from Earth. In November 2023 it stopped returning readable science and engineering data, although it continued to receive and carry out commands from the ground. Nobody could visit the spacecraft, and every test was slow. A command took about 22.5 hours to reach it, and a reply took another 22.5 hours to return. A NASA update in December 2023 described the telemetry modulation unit as sending a repeating pattern of ones and zeros as if it were stuck. The team then had to decide whether that unit had failed, or whether the flight data subsystem (FDS), the computer that packaged the data for it, was the real source. A restart of the FDS did not restore usable data. This investigation asks how the team narrowed a fault in a computer it could not touch to one part of its memory, and why the repair took months rather than days. Separate what the public updates observed directly, what the team inferred from a memory readout, and what remained unknown, including why the chip failed.

The timeline

  1. Voyager 1 stops sending readable science and engineering data, although it is still receiving and executing commands.

  2. NASA reports that the FDS is not communicating properly with the telemetry modulation unit, and that a restart of the FDS did not restore usable data.

  3. The team sends a command, called a poke, to prompt the FDS to try different software sequences.

  4. After the reply arrives on 3 March and is decoded, engineers determine that it contains a readout of the entire FDS memory.

  5. NASA says about 3% of the FDS memory is corrupted and that a single chip storing part of it is suspected to be failing.

  6. The team sends relocated code for packaging engineering data to a new location in the FDS memory.

  7. The mission team receives usable engineering status from Voyager 1 for the first time in five months.

A researched reconstruction for learning. These reasoning steps simplify a complex record; no score can capture the full history.

02 · The evidence

What does the record show?

Read each exhibit before settling on an explanation. The source links below every exhibit lead to the record behind it.

Exhibit 01

A computer that was running but could not send data

NASA's December 2023 update says Voyager 1 was receiving and executing commands from Earth, but the flight data subsystem (FDS) was not communicating properly with the telemetry modulation unit (TMU), so no science or engineering data was sent. The TMU had begun transmitting a repeating pattern of ones and zeros as if it were stuck. After ruling out other possibilities, the team identified the FDS as the source, and a restart did not restore usable data. The update establishes the symptoms and the first diagnosis. It does not show which part of the FDS memory was failing; that came later.

Source record s1science.nasa.gov

Exhibit 02

A poke that returned the whole memory

The March 2024 update says that a command sent on 1 March, which the team called a poke, was meant to prompt the FDS to try different sequences in its software, in case the problem could be bypassed. The reply arrived on 3 March. A Deep Space Network engineer decoded the new signal and found that it contained a readout of the entire FDS memory, including the software code and the variables that the code uses. The readout gives the team the memory contents. It does not by itself name the failed part, so the team planned to compare it with a readout taken before the fault.

Source record s2science.nasa.gov

Exhibit 03

About 3 percent of the memory and one suspect chip

The April 2024 update reports that, using the readout, the team confirmed that about 3% of the FDS memory was corrupted, which kept the computer from normal operation. The team suspected that a single chip storing part of the affected memory was not working. Engineers could not say with certainty what caused it. The two possibilities named were an energetic particle from space, or wear-out after 46 years. The update establishes the size of the corrupted area and the suspect component. The 3% figure refers to the FDS memory only, and the cause of the chip failure remains unconfirmed.

Source record s3science.nasa.gov

Exhibit 04

Moving code around a dead chip

The April 22 update describes the workaround. The team could not repair the chip, so it decided to place the affected code elsewhere in the FDS memory. No single location was large enough for the whole section, so the code was divided into sections, the sections were adjusted to work together, and references to their locations were updated. The engineering-data code was sent on 18 April, and the team heard back on 20 April, the first usable status in five months. Science-data code was still to be relocated. The update shows the method and the first success. It does not give the full software change.

Source record s4science.nasa.gov

Exhibit 05

Light time sets the pace

The December 2023 update says a command takes about 22.5 hours to reach Voyager 1 and another 22.5 hours for the response to return. It adds that the team had to wait 45 hours to learn whether a command had its intended effect. The March 2024 update repeats the 22.5-hour figure in each direction. These delays come from distance, not from a fault. They explain why each diagnostic step and each software change took days. The updates do not say how many tests were run in total, so the number of attempts before the poke cannot be counted from them.

Source record s1science.nasa.gov

03 · Your reasoning

Follow the strongest explanation.

Choose an interpretation, then compare it with the evidence. You can revise your choice. The goal is understanding, not a perfect score.

Reasoning step 01 of 04

Which explanation best fits the loss of data that began in November 2023?

Evidence-based explanation

The December update says the spacecraft was executing commands while it returned no usable data, and that the source was the FDS. The March and April updates then show that corrupted memory, including code, was the mechanism. Power loss and a ground fault do not match the reported behavior. The telemetry modulation unit showed a symptom but was not identified as the origin. The updates establish this diagnosis without a full account of the software state.

Weigh it against

Exhibit 01A computer that was running but could not send dataExhibit 02A poke that returned the whole memoryExhibit 03About 3 percent of the memory and one suspect chip

Reasoning step 02 of 04

Why did the repair take months rather than days?

Evidence-based explanation

Light time is the constraint the record quantifies. A command took about 22.5 hours to reach Voyager 1, and the reply took the same time to return, so the December update notes a 45-hour wait for each response. The December update also says each new command had to be checked for its effect on the spacecraft, sometimes against decades-old documents. The other options name factors that the updates do not describe as causes of this delay.

Weigh it against

Exhibit 05Light time sets the paceExhibit 02A poke that returned the whole memory

Reasoning step 03 of 04

Which repair approach does the evidence best support?

Evidence-based explanation

The April 22 update reports that the chip could not be repaired, so the team moved the affected code elsewhere. Because no free location was large enough for the whole section, the code was split, adjusted to work together, and its references were updated. The record supports this method and the first success on 20 April. It does not describe a spare part or a full software reload, and it does not show how many more sections would need to move.

Weigh it against

Exhibit 04Moving code around a dead chipExhibit 03About 3 percent of the memory and one suspect chip

Reasoning step 04 of 04

What is the best-supported statement about why the chip failed?

Evidence-based explanation

The April 4 update is explicit that engineers could not determine the cause with certainty, and it names two possibilities: a particle from space, or wear-out after 46 years. The other options add specific causes that the updates do not report. Reading the chip as the suspect component, rather than as a confirmed cause, is the careful position the record supports.

Weigh it against

Exhibit 03About 3 percent of the memory and one suspect chip

04 · The findings

The chain behind the outcome.

Voyager 1 stopped returning readable data in November 2023, while it kept receiving commands. A December 2023 NASA update identified the telemetry modulation unit as sending a repeating pattern as if it were stuck. The team then ruled out other possibilities and traced the fault to the flight data subsystem, whose job is to package science and engineering data for transmission. A restart did not restore data. In early March 2024 the team sent a poke command to prompt the computer to try different software sequences. The reply, which took about 22.5 hours each way, contained a readout of the entire FDS memory. Analysis of that readout, reported in April, found that about 3% of the memory was corrupted. The team suspected a single chip, but could not confirm why it failed: a particle from space or wear-out after 46 years were the two possibilities named. Because the chip could not be repaired, engineers split the affected code into sections, moved them into other free memory, and adjusted their references. The engineering data path returned on 20 April, the first usable status in five months, and the science data was to follow. The explanation is well supported at the level of the mechanism, but the public updates do not show the full software change or the order of every step.

Contributing causes and conditions

  1. A single chip storing part of the FDS memory, including some software code, stopped working.
  2. The corrupted memory kept the flight data subsystem from packaging science and engineering data correctly.
  3. The flight data subsystem stopped communicating properly with the telemetry modulation unit, which began sending a repeating pattern of ones and zeros as if it were stuck.
  4. Round-trip light time of about 45 hours for each test forced slow, step-by-step verification.
  5. The team relocated the affected code into several free memory areas and updated the references to it.

OutcomeVoyager 1 memory fault (2023–2024)

Take the method with you

  1. When a remote system cannot be inspected directly, a readout of its internal state can turn a vague symptom into a specific location.
  2. Round-trip delays make every test expensive, so diagnostic steps should be chosen to give the most information per command.
  3. A workaround can be a software change rather than a hardware swap, but it has to be checked against the whole system it touches.

You can read findings at any time. Exploring every reasoning step marks this case as complete in your notebook.

Open another investigation

The open record

Sources & further reading

Read the original record and its scope before relying on a conclusion. Source references support the exhibit summaries; Faultory’s reasoning prompts are editorial reconstructions.

  1. s1
  2. s2
  3. s3
  4. s4

Written and curated for Faultory by Yunus Emre Vurgun, with AI-assisted research and drafting. Sources and interpretations can need correction. Read our editorial standards or report an issue.

Keep investigating