hex_core decompresses registry and Erlang term API responses without size limits

Summary

Improper Handling of Highly Compressed Data (Data Amplification) vulnerability in hexpm hex_core allows an untrusted repository mirror, or a server answering hex_core's API requests, to exhaust the memory of the Erlang VM running hex_core. The same code is vendored in Hex (mix_hex_repo, mix_hex_api) and rebar3 (r3_hex_repo, r3_hex_api).

hex_repo:get_names/1, hex_repo:get_versions/1, hex_repo:get_package/2 and hex_repo:get_policy/2 decompress the registry response with zlib:gunzip/1 and decode the signed envelope before verifying its signature. A mirror doesn't have the repository's private key, but it chooses the compressed bytes, so a small response decompresses into an envelope of any size before the signature check rejects it.

hex_api decodes responses with the application/vnd.hex+erlang content type through hex_safe_binary_to_term:safe_binary_to_term/1, which calls binary_to_term(Binary, [safe]). The safe option prevents atom creation, but it accepts compressed terms and doesn't limit the size of the decoded term, and the check that rejects functions and ports runs after the term has been built. The fix for CVE-2026-21619 added the safe option and that check without bounding the decoded size.

This issue affects rebar3: from 3.9.1 before 3.27.3; hex: from 2.1.0 before 2.5.2; hex_core: from 0.1.0 before 0.20.0.

Details

1. hex_repo:get_protobuf/3, used by get_names/1, get_versions/1, get_package/2 and get_policy/2, calls zlib:gunzip/1 on the whole response body with no output limit.
2. hex_repo:decode/4 then calls hex_registry:decode_and_verify_signed/2, which decodes the protobuf envelope, copying its payload and signature, before checking the signature.
3. hex_api:request/4 passes every response whose content type contains application/vnd.hex+erlang to hex_safe_binary_to_term:safe_binary_to_term/1.
4. safe_binary_to_term/1 calls binary_to_term(Binary, [safe]), which decompresses compressed terms and builds the whole term, and only then walks it to reject functions and ports.

Proof of concept

1. Point a repository at a loopback mirror (in Hex, HEX_MIRROR_URL) that answers /packages/probe with a 65,268-byte gzip stream decompressing to a 67,108,872-byte signed envelope with a 64 MiB payload and a one-byte signature.
2. Hex.Repo.get_package("hexpm", "probe", nil), which calls the vendored hex_repo:get_package/2, returns {:error, :bad_signature} after allocating the envelope. Peak RSS was 234 MiB, against 96 MiB for a 1,030-byte invalid envelope.
3. Answer an API request with content-type: application/vnd.hex+erlang and 8,178 bytes of compressed Erlang term format encoding a list of 4,194,304 elements. mix_hex_api:get/2 accepts the decoded term, and BEAM memory rises from 58 MiB to 272 MiB.

Impact

A mirror can make every registry fetch allocate memory far beyond the size of its response: in a test against Hex main 851b938, which vendors this code, a 65 KB registry response decompressed into a 64 MiB envelope before the signature check failed. The API server, or an attacker who can alter its responses, can do the same with API responses: 8 KB of compressed Erlang term format decoded into a list of 4.2 million elements. Large enough responses stop the Erlang VM, which fails the build tool command or the program using hex_core. No package code has to run.

Workarounds

Point repository URLs at repo.hex.pm or a mirror you operate, and the API URL at hex.pm.

Configurations

Registry decoding is affected when a repository's URL points at a mirror or another server the user doesn't trust with the availability of their builds, for example Hex's HEX_MIRROR_URL or mirror_url and rebar3's HEX_MIRROR_URL. API decoding is affected when the API URL points at such a server or an attacker can alter API responses. Requests to repo.hex.pm and hex.pm over verified HTTPS aren't affected.

Solutions

Upgrade hex_core to 0.20.0 or later. Registry resources larger than 16 MiB after decompression are rejected with {too_big_uncompressed, MaxSize}, configurable with registry_max_uncompressed_size.

Upgrade Hex to 2.5.2 or later, which vendors hex_core 0.20.0. No rebar3 release vendors hex_core 0.20.0 yet.