Faultory

Case file 56 · Engineering & technology

The half-second login

xz Utils backdoor, 2024: what small anomaly exposed malicious code hidden in release files?

Subject
xz Utils backdoor (2024)
Reading
15 min
Evidence
5 exhibits
Reasoning
3 steps
Level
Advanced
The OpenBSD operating system starting its secure shell server software.
6 August 2017 OpenBSD starting its SSH server, shown to illustrate the login service the backdoor targeted on some Linux systems. OpenBSD itself was not affected. Photo: Hpott · CC BY-SA 4.0 · Wikimedia Commons.
Illustrative signature. A strip-chart trace of login time that stays low and then steps up while the injected code runs during authentication (a sketch, not data). Drawn to explain the case, not plotted from source data.

01 · The briefing

Begin with the question.

Xz Utils is a widely installed set of data compression tools, and its liblzma library is used by other software. In late March 2024 it emerged that the release tarballs for versions 5.6.0 and 5.6.1 contained a backdoor. The first clue came from an engineer who was investigating why SSH logins on a Debian sid system had become slow and were producing valgrind errors. Possible explanations included a fault in one distribution's package, a fault in the software itself, or a deliberate change upstream. The engineer first suspected Debian's package, then found the same injected code in the upstream tarballs. This investigation separates what the anomaly showed from what the code did, and it separates the public git repository from the release files. It also asks what the record establishes, and what it leaves open, about the people who controlled the release process. The record used here is the maintainer's own statement, the researcher's post, the official vulnerability entry and the public commit history. Keep the question concrete: which observation pointed to a problem, which evidence places the malicious code in the release files, and what does the record say about how it got there?

The timeline

  1. The account JiaT75 commits “Bump version and soname for 5.6.0”, the commit that the v5.6.0 release tag points to.

  2. Andres Freund posts his analysis to the oss-security list, and NVD publishes CVE-2024-3094 the same day.

  3. According to the maintainer's page, the xz email address stops forwarding to Jia Tan.

  4. The xz git repositories become available on GitHub again, after the project had moved them.

  5. The maintainer publishes new, clean XZ Utils releases.

A researched reconstruction for learning. These reasoning steps simplify a complex record; no score can capture the full history.

02 · The evidence

What does the record show?

Read each exhibit before settling on an explanation. The source links below every exhibit lead to the record behind it.

Exhibit 01

Slow logins and valgrind errors

Freund reports that SSH logins on Debian sid installations had begun to use a lot of CPU and produced valgrind errors. In one timing test, a failing login attempt took 0.299 seconds before the backdoored liblzma was installed and 0.807 seconds after. An sshd -h check took about 0.5 seconds on his older system without the TERM variable set, and about 0.01 seconds with it set. The post establishes an observable slowdown tied to liblzma. It does not establish, by itself, what the injected code did or whether any login succeeded.

Source record s1openwall.com

Exhibit 02

A line in the tarball, not in git

Freund reports that an obfuscated injection in build-to-host.m4 appears in the released tarballs for 5.6.0 and 5.6.1, but not in the upstream version of that file and not in the git repository, where xz does not use it. The injected step runs during configure and only under particular build conditions, which he lists, including GNU ld and Debian or RPM package builds. He notes that GitHub's source-code links, which he believes are generated from repository contents, do not contain it. The comparison establishes that the release files differ from the repository. It does not establish who prepared the tarballs.

Source record s1openwall.com

Exhibit 03

Test files that carried the payload

Freund reports that the bulk of the payload sat in two files under tests/files, bad-3-corrupt_lzma2.xz and good-large_compressed.lzma, committed upstream, and that these files were not used for any test in 5.6.0. The GitHub commit page for cf44e4b lists those files in a commit titled Tests: Add a few test files, committed by the account JiaT75. The record establishes that obfuscated binary data was committed to the repository. It does not show what the account holder intended.

Source record s4github.com

Exhibit 04

The maintainer's account of the signed tarballs

The xz project page, written by the maintainer, Lasse Collin, states that the 5.6.0 and 5.6.1 release tarballs contain a backdoor and that they were created and signed by Jia Tan. It says that Jia Tan's access was limited to items hosted on GitHub and the xz subdomain, and that the GitHub accounts of both the maintainer and Jia Tan were suspended. The page establishes the maintainer's account and the access boundaries he describes. It is a party's own statement and does not verify the identity of the person behind the account.

Source record s2tukaani.org

Exhibit 05

A critical score and two affected versions

The National Vulnerability Database record for CVE-2024-3094, first published on 29 March 2024, describes malicious code in the upstream xz tarballs starting with version 5.6.0, extracted from a disguised test file during the liblzma build. The NIST and Red Hat CVSS 3.1 scores are both 10.0, critical, and the affected versions listed are 5.6.0 and 5.6.1. The record establishes the severity and affected versions as classified. It does not measure how many systems were built with the library or whether any were compromised.

Source record s3services.nvd.nist.gov

03 · Your reasoning

Follow the strongest explanation.

Choose an interpretation, then compare it with the evidence. You can revise your choice. The goal is understanding, not a perfect score.

Reasoning step 01 of 03

Which anomaly most plausibly started the investigation?

Evidence-based explanation

Freund's post starts with slow SSH logins, high CPU use and valgrind errors on Debian sid installations, and he traces them to liblzma. The anomaly was small and operational rather than a security alert. The other options either contradict the post or describe a route that would not have revealed the injected code. The evidence supports this observation as the starting point, not a claim that it was the only possible clue.

Weigh it against

Exhibit 01Slow logins and valgrind errorsExhibit 02A line in the tarball, not in git

Reasoning step 02 of 03

Which evidence best supports injection into the release tarballs rather than the public repository?

Evidence-based explanation

Freund compared the release tarball with the upstream build-to-host file and with git, and found the injected line only in the tarball. GitHub's source-code links, which come from the repository, did not contain it. That comparison is strong evidence for a difference in the release artifacts, though it does not identify who prepared the tarball or when. The other options contradict the reported comparison or the affected-version record.

Weigh it against

Exhibit 02A line in the tarball, not in git

Reasoning step 03 of 03

What best explains the roughly half-second slowdown in SSH logins?

Evidence-based explanation

Freund's measurements show the slowdown appearing with the backdoored library, and he reports that the injected code appears to parse symbol tables in memory during sshd startup. Because OpenSSH does not use liblzma directly, the path runs through libsystemd, which depends on lzma. Freund says he is not a security researcher and has not analyzed all of the code, so the mechanism is well supported in outline but is not a complete account.

Weigh it against

Exhibit 01Slow logins and valgrind errors

04 · The findings

The chain behind the outcome.

The attack was found through an ordinary symptom. An engineer on a Debian sid system saw SSH logins using more CPU and producing valgrind errors, and tracing the problem led to liblzma. Freund measured a failing login rising from about 0.3 seconds to about 0.8 seconds with the backdoored library installed, and he traced the slow step to injected code that appears to parse symbol tables in memory. That code was not in the public git source. It sat in the release tarballs for 5.6.0 and 5.6.1, inserted through an obfuscated build step that ran only under certain conditions. The bulk of its payload had been committed to the repository as test files that no 5.6.0 test used. The maintainer's page says the tarballs were created and signed by an identity called Jia Tan, who had access to parts of the project hosted on GitHub. The project's repositories returned to GitHub on 9 April 2024, its xz email address stopped forwarding to Jia Tan on 30 March, and clean releases followed on 29 May 2024. NVD scores the vulnerability as critical, with a CVSS 3.1 base score of 10.0 from both NIST and Red Hat. The lesson the record supports is narrow: a small anomaly in a widely used library revealed a difference between the public source and the shipped artifacts.

Contributing causes and conditions

  1. The release tarballs for xz 5.6.0 and 5.6.1 contained an obfuscated build step that was absent from the upstream source file and from the git repository.
  2. Obfuscated binary payloads were committed to the repository as test data, and no test used them in 5.6.0.
  3. The injected build step ran only under particular conditions, such as x86-64 Linux with GNU ld, including Debian and RPM package builds.
  4. The injected code changed the address that sshd's call to RSA_public_decrypt, a function used in public-key authentication, would jump to.
  5. Because systemd-linked sshd loaded liblzma, the code ran before authentication, slowing logins and causing valgrind errors on some builds.
  6. The release tarballs were created and signed by a second account, which the maintainer's statement names as Jia Tan.

Outcomexz Utils backdoor (2024)

Take the method with you

  1. An unexplained performance change in a security-sensitive component is worth tracing, because it can be the only visible symptom of a deliberate change.
  2. Compare what a reviewer can see in the public source with what users actually receive, since build inputs and release artifacts can differ.
  3. Trust that rests on one maintainer or one account is a single point of failure, so release signing and repository access need independent review.

You can read findings at any time. Exploring every reasoning step marks this case as complete in your notebook.

Open another investigation

The open record

Sources & further reading

Read the original record and its scope before relying on a conclusion. Source references support the exhibit summaries; Faultory’s reasoning prompts are editorial reconstructions.

  1. s1
  2. s2
  3. s3
  4. s4
  5. s5

Written and curated for Faultory by Yunus Emre Vurgun, with AI-assisted research and drafting. Sources and interpretations can need correction. Read our editorial standards or report an issue.

From the journal

Field notes that use this case

Keep investigating