GHSA-83vm-p52w-f9pw CVE-2026-44223 PYSEC-2026-145

vLLM: extract_hidden_states speculative decoding crashes server on any request with penalty parameters

Medium PyPI

Severity: 6.5 / 10; Medium · CVSS 3.1
Exploit chance (EPSS): 0.37%; Chance of exploitation in the next 30 days. Higher than 30.56% of scored vulnerabilities.
Known exploited: No; Not in CISA's catalog
Weakness: CWE-131 Incorrect Calculation of Buffer Size; CWE-704 Incorrect Type Conversion or Cast
Fix: 0.20.0; Upgrade vllm to this version or later

Description

Summary

The extract_hidden_states speculative decoding proposer in vLLM returns a tensor with an incorrect shape after the first decode step, causing a RuntimeError that crashes the EngineCore process. The crash is triggered when any request in the batch uses sampling penalty parameters (repetition_penalty, frequency_penalty, or presence_penalty).

A single request with a penalty parameter (e.g., "repetition_penalty": 1.1) is sufficient to crash the server. The crash is deterministic and immediate — no concurrency, race condition, or special workload is required.

Details

In vLLM v0.17.0, the extract_hidden_states proposer's propose() method returned sampled_token_ids.unsqueeze(-1), producing a tensor of shape (batch_size, 1).

In PR #37013 (first released in v0.18.0), the KV connector interface was refactored out of propose(). The return type changed from tuple[Tensor, KVConnectorOutput | None] to Tensor, and the .unsqueeze(-1) call was removed along with the KV connector output:

# Before (v0.17.0):
return sampled_token_ids.unsqueeze(-1), kv_connector_output  # shape (batch_size, 1)

# After (v0.18.0+):
return sampled_token_ids  # shape (batch_size, 2) after first decode step

The refactor missed that sampled_token_ids changed semantics between the first and subsequent decode steps. After the first decode step, the rejection sampler allocates its output as (batch_size, max_spec_len + 1). With num_speculative_tokens=1, this produces shape (batch_size, 2) instead of the expected (batch_size, 1), causing a broadcast shape mismatch during penalty application.

Impact

Any vLLM deployment between v0.18.0 and v0.19.1 (inclusive) configured with extract_hidden_states speculative decoding is affected. A single API request containing any penalty parameter immediately and permanently crashes the EngineCore process, resulting in complete loss of service availability.

Patches

Fixed in PR #38610, first included in vLLM v0.20.0. The fix slices the return value to sampled_token_ids[:, :1], ensuring the correct (batch_size, 1) shape regardless of the rejection sampler's output dimensions.

Workarounds

Upgrade to vLLM v0.20.0 or later.
If upgrading is not possible, avoid using extract_hidden_states as the speculative decoding method on affected versions.
Alternatively, reject or strip penalty parameters (repetition_penalty, frequency_penalty, presence_penalty) from incoming requests at an API gateway before they reach vLLM.

Affected versions

PyPI

Package	Vulnerable	Fixed in
vllm	`>= 0.18.0, < 0.20.0`	`0.20.0`

How to fix

Move to the fixed version or later with your package manager, update the lockfile, and run your tests. If the package only arrives through another dependency, upgrade that dependency first. If none of its releases pulls a fixed version yet, force the version with your package manager's override.

vllm PyPI → `0.20.0`

pip

pip install --upgrade "vllm>=0.20.0"

poetry

poetry add "vllm>=0.20.0"

uv

uv add "vllm>=0.20.0"

Severity breakdown

CVSS 3.1 6.5

CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H

Attack vector: Network
Attack complexity: Low
Privileges required: Low
User interaction: None
Scope: Unchanged
Confidentiality: None
Integrity: None
Availability: High

Weakness

CWE-131: Incorrect Calculation of Buffer Size

The product does not correctly calculate the size to be used when allocating a buffer, which could lead to a buffer overflow.

How to prevent it

Implementation: When allocating a buffer for the purpose of transforming, converting, or encoding an input, allocate enough memory to handle the largest possible encoding. For example, in a routine that converts "&" characters to "&" for HTML entity encoding, the output buffer needs to be at least 5 times as large as the input buffer.
Implementation: Understand the programming language's underlying representation and how it interacts with numeric calculation (CWE-681). Pay close attention to byte size discrepancies, precision, signed/unsigned distinctions, truncation, conversion and casting between types, "not-a-number" calculations, and how the language handles numbers that are too large or too small for its underlying representation. [REF-7] Also be careful to account for 32-bit, 64-bit, and other potential differences that may affect the numeric representation.
Implementation: Perform input validation on any numeric input by ensuring that it is within the expected range. Enforce that the input meets both the minimum and maximum requirements for the expected range.
Architecture and Design: For any security checks that are performed on the client side, ensure that these checks are duplicated on the server side, in order to avoid CWE-602. Attackers can bypass the client-side checks by modifying values after the checks have been performed, or by changing the client to remove the client-side checks entirely. Then, these modified values would be submitted to the server.
Implementation: When processing structured incoming data containing a size field followed by raw data, identify and resolve any inconsistencies between the size field and the actual size of the data (CWE-130).
Implementation: When allocating memory that uses sentinels to mark the end of a data structure - such as NUL bytes in strings - make sure you also include the sentinel in your calculation of the total amount of memory that must be allocated.
Implementation: Replace unbounded copy functions with analogous functions that support length arguments, such as strcpy with strncpy. Create these if they are not available.
Implementation: Use sizeof() on the appropriate data type to avoid CWE-467.
Implementation: Use the appropriate type for the desired action. For example, in C/C++, only use unsigned types for values that could never be negative, such as height, width, or other numbers related to quantity. This will simplify validation and will reduce surprises related to unexpected casting.
Architecture and Design: Use a vetted library or framework that does not allow this weakness to occur or provides constructs that make this weakness easier to avoid [REF-1482]. Use libraries or frameworks that make it easier to handle numbers without unexpected consequences, or buffer allocation routines that automatically track buffer size. Examples include safe integer handling packages such as SafeInt (C++) or IntegerLib (C or C++). [REF-106]
Operation, Build and Compilation: Use automatic buffer overflow detection mechanisms that are offered by certain compilers or compiler extensions. Examples include: the Microsoft Visual Studio /GS flag, Fedora/Red Hat FORTIFY_SOURCE GCC flag, StackGuard, and ProPolice, which provide various mechanisms including canary-based detection and range/index checking. D3-SFCV (Stack Frame Canary Validation) from D3FEND [REF-1334] discusses canary-based detection in detail.
Operation, Build and Compilation: Run or compile the software using features or extensions that randomly arrange the positions of a program's executable and libraries in memory. Because this makes the addresses unpredictable, it can prevent an attacker from reliably jumping to exploitable code. Examples include Address Space Layout Randomization (ASLR) [REF-58] [REF-60] and Position-Independent Executables (PIE) [REF-64]. Imported modules may be similarly realigned if their default memory addresses conflict with other modules, in a process known as "rebasing" (for Windows) and "prelinking" (for Linux) [REF-1332] using randomly generated addresses. ASLR for libraries cannot be used in conjunction with prelink since it would require relocating the libraries at run-time, defeating the whole purpose of prelinking. For more information on these techniques see D3-SAOR (Segment Address Offset Randomization) from D3FEND [REF-1335].
Operation: Use a CPU and operating system that offers Data Execution Protection (using hardware NX or XD bits) or the equivalent techniques that simulate this feature in software, such as PaX [REF-60] [REF-61]. These techniques ensure that any instruction executed is exclusively at a memory address that is part of the code segment. For more information on these techniques see D3-PSEP (Process Segment Execution Prevention) from D3FEND [REF-1336].
Implementation: Examine compiler warnings closely and eliminate problems with potential security implications, such as signed / unsigned mismatch in memory operations, or use of uninitialized variables. Even if the weakness is rarely exploitable, a single failure may lead to the compromise of the entire system.
Architecture and Design, Operation: Run your code using the lowest privileges that are required to accomplish the necessary tasks [REF-76]. If possible, create isolated accounts with limited privileges that are only used for a single task. That way, a successful attack will not immediately give the attacker access to the rest of the software or its environment. For example, database applications rarely need to run as the database administrator, especially in day-to-day operations.
Architecture and Design, Operation: Run the code in a "jail" or similar sandbox environment that enforces strict boundaries between the process and the operating system. This may effectively restrict which files can be accessed in a particular directory or which commands can be executed by the software. OS-level examples include the Unix chroot jail, AppArmor, and SELinux. In general, managed code may provide some protection. For example, java.io.FilePermission in the Java SecurityManager allows the software to specify restrictions on file operations. This may not be a feasible solution, and it only limits the impact to the operating system; the rest of the application may still be subject to compromise. Be careful to avoid CWE-243 and other weaknesses related to jails.

CWE-704: Incorrect Type Conversion or Cast

The product does not correctly convert an object, resource, or structure from one type to a different type.

References

Advisories

nvd.nist.gov/vuln/detail/CVE-2026-44223

Packages

github.com/vllm-project/vllm

Timeline

May 6, 2026 Published
Sep 10, 2026 Last updated by the source
Nov 1, 2026 Latest EPSS score

Advisory	Severity	Why it's related
GHSA-8397-8qf6-f2gm vLLM: Phi-4 multimodal audio encoder materializes and retains full attention matrices: a single max-length audio OOM-crashes the vLLM engine	Medium	Also affects vllm
GHSA-x567-49g5-p4cf vLLM: Responses API bypasses trust_request_chat_template for chat_template_kwargs	Medium	Also affects vllm
GHSA-vvpg-rrww-wpmh vLLM: Permissive default CORS origins allow cross-origin requests to the vLLM HTTP server	Medium	Also affects vllm
GHSA-q97c-p368-75c4 vLLM: Out-of-vocabulary prompt token id passes input validation and triggers a CUDA device-side assert (engine crash / DoS)	Medium	Also affects vllm
GHSA-h3rc-j82p-2c3h vLLM: Untrusted EPS inputs can trigger denial of service or Ghostscript RCE	High	Also affects vllm
GHSA-gq5q-89c7-vhvx vLLM DeepSeek-OCR image processing vulnerable to denial of service	Medium	Also affects vllm
GHSA-6757-hggm-hp2w vLLM: Public multimodal SHM handle forgery reaches worker-side `pickle.loads()` with insecure serialization disabled	High	Also affects vllm
GHSA-542m-9234-8gvf vLLM: DoS via unbounded chat_template Jinja2 rendering exhausts the renderer thread pool	Medium	Also affects vllm

Is this in your code?

MergeHarbor checks every repository's dependencies against this database, ranks what matters by exploitability and reachability, routes it to the owning team and opens the fix pull request.

Check your repos with MergeHarbor