Skip to content
← Back to News

EngineeringIncident of March 29, 20248 min read

CVE-2024-3094: the xz utils releases to remove

A maintainer account pushed malicious xz tarballs that could alter sshd on systems linking through systemd. The stable distributions mostly had not shipped them. The ones that had, had to downgrade the same day.

On 29 March 2024 Andres Freund mailed oss-security about a backdoor in xz utils, assigned CVE-2024-3094. The affected releases are 5.6.0 and 5.6.1. The payload lived in the release tarball, not in the git tree most reviewers would have read, and was wired to interfere with sshd on distributions that notify systemd from OpenSSH.

NVD scores it 10.0. Freund noticed because ssh logins were slow and using unexpected CPU on a Debian unstable system. That is why the campaign did not quietly become the next year of internet traffic.

What landed in the tarball

The project had a new maintainer who, over years, gained the ability to cut releases. The 5.6.0 and 5.6.1 tarballs contained extra files that the build system picked up and that altered the resulting library. Distributions that build OpenSSH with a patch to talk to systemd could end up with an sshd that consulted that library.

This article does not describe the trigger or the payload. The operational fact is the version number. If the installed xz is 5.6.0 or 5.6.1, replace it. If it is a 5.4 release, you are on the branch that was not part of this backdoor.

Who actually shipped it

Fedora Rawhide, Debian unstable and testing, and a handful of rolling or preview images had the bad versions in late March 2024. Red Hat Enterprise Linux, Ubuntu LTS and Debian stable had not promoted 5.6. That split is the whole difference between a bad Friday and a fleet-wide rebuild. It is also why 'we use Linux' is not an inventory.

Containers and CI images were the quiet copies. A Dockerfile that said 'install xz' against a rolling base on 28 March 2024 could have frozen the bad library into an artifact that outlived the distro rollback.

Downgrade, then look for the artifact

Install the xz your distribution reverted to — 5.4.x on the branches that published a fix that week — and restart sshd so the process is not still mapped to the old library. Confirm the package version and the file on disk. CISA and the distros published the hashes and the package names; use those, not a mirror you have not checked.

Search images and caches for 5.6.0 and 5.6.1. Rebuild any artifact that contained them. Rotating SSH host keys is a reasonable extra on a machine that actually ran the bad build and accepted logins, because you cannot see what a backdoored sshd did. It is not a substitute for removing the library.

What to keep from the incident

Pin base images. Review who can publish the tarball of a dependency that runs in sshd's address space. A single maintainer with release rights and no second signature is the condition this took years to reach.

Freund's mail is the primary source, and it is a reminder that the detection was performance, not a scanner signature. Keep enough telemetry on login latency that a slow sshd is a page, not a shrug. The CVE record is the identifier to put on the change ticket.

Andres Freund's report to oss-security on 29 March 2024, and the NVD entry for CVE-2024-3094.

https://www.openwall.com/lists/oss-security/2024/03/29/4

Frequently asked questions

Which xz versions are backdoored?

5.6.0 and 5.6.1. The 5.4 line used by stable distributions was not part of CVE-2024-3094. Downgrade anything that installed the 5.6 pair, including container images built that week.

Did every Linux server need an emergency patch?

No. Enterprise stable releases had not shipped 5.6.0 or 5.6.1. Rolling, testing and Rawhide trees had. Check the version instead of assuming the headline applies to the fleet.

Is removing the package enough?

Replace the package and restart sshd so the old library is not mapped. Rebuild images that baked 5.6.0 or 5.6.1 in. Rotate host keys if that sshd actually ran and accepted connections.

Keep reading