CVE-2026-44223: vLLM Extract_hidden_states Speculative Decoding Crashes Server Vulnerability

{
  "schema_version" : "1.4.0",
  "id" : "GHSA-83vm-p52w-f9pw",
  "modified" : "2026-06-08T19:52:35Z",
  "published" : "2026-05-06T21:45:51Z",
  "aliases" : [ "CVE-2026-44223" ],
  "summary" : "vLLM: extract_hidden_states speculative decoding crashes server on any request with penalty parameters",
  "details" : "### Summary\n\nThe `extract_hidden_states` speculative decoding proposer in vLLM returns a tensor with an incorrect shape after the first decode step, causing a `RuntimeError` that crashes the EngineCore process. The crash is triggered when any request in the batch uses sampling penalty parameters (`repetition_penalty`, `frequency_penalty`, or `presence_penalty`).\n\nA single request with a penalty parameter (e.g., `\"repetition_penalty\": 1.1`) is sufficient to crash the server. The crash is deterministic and immediate — no concurrency, race condition, or special workload is required.\n\n### Details\n\nIn vLLM v0.17.0, the `extract_hidden_states` proposer's `propose()` method returned `sampled_token_ids.unsqueeze(-1)`, producing a tensor of shape `(batch_size, 1)`.\n\nIn [PR #37013](https://github.com/vllm-project/vllm/pull/37013) (first released in v0.18.0), the KV connector interface was refactored out of `propose()`. The return type changed from `tuple[Tensor, KVConnectorOutput | None]` to `Tensor`, and the `.unsqueeze(-1)` call was removed along with the KV connector output:\n\n```python\n# Before (v0.17.0):\nreturn sampled_token_ids.unsqueeze(-1), kv_connector_output  # shape (batch_size, 1)\n\n# After (v0.18.0+):\nreturn sampled_token_ids  # shape (batch_size, 2) after first decode step\n```\n\nThe refactor missed that `sampled_token_ids` changed semantics between the first and subsequent decode steps. After the first decode step, the rejection sampler allocates its output as `(batch_size, max_spec_len + 1)`. With `num_speculative_tokens=1`, this produces shape `(batch_size, 2)` instead of the expected `(batch_size, 1)`, causing a broadcast shape mismatch during penalty application.\n\n### Impact\n\nAny vLLM deployment between v0.18.0 and v0.19.1 (inclusive) configured with `extract_hidden_states` speculative decoding is affected. A single API request containing any penalty parameter immediately and permanently crashes the EngineCore process, resulting in complete loss of service availability.\n\n### Patches\n\nFixed in [PR #38610](https://github.com/vllm-project/vllm/pull/38610), first included in vLLM v0.20.0. The fix slices the return value to `sampled_token_ids[:, :1]`, ensuring the correct `(batch_size, 1)` shape regardless of the rejection sampler's output dimensions.\n\n### Workarounds\n\n- Upgrade to vLLM v0.20.0 or later.\n- If upgrading is not possible, avoid using `extract_hidden_states` as the speculative decoding method on affected versions.\n- Alternatively, reject or strip penalty parameters (`repetition_penalty`, `frequency_penalty`, `presence_penalty`) from incoming requests at an API gateway before they reach vLLM.",
  "severity" : [ {
    "type" : "CVSS_V3",
    "score" : "CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H"
  } ],
  "affected" : [ {
    "package" : {
      "ecosystem" : "PyPI",
      "name" : "vllm"
    },
    "ranges" : [ {
      "type" : "ECOSYSTEM",
      "events" : [ {
        "introduced" : "0.18.0"
      }, {
        "fixed" : "0.20.0"
      } ]
    } ]
  } ],
  "references" : [ {
    "type" : "WEB",
    "url" : "https://github.com/vllm-project/vllm/security/advisories/GHSA-83vm-p52w-f9pw"
  }, {
    "type" : "ADVISORY",
    "url" : "https://nvd.nist.gov/vuln/detail/CVE-2026-44223"
  }, {
    "type" : "WEB",
    "url" : "https://github.com/vllm-project/vllm/pull/38610"
  }, {
    "type" : "WEB",
    "url" : "https://github.com/pypa/advisory-database/tree/main/vulns/vllm/PYSEC-2026-145.yaml"
  }, {
    "type" : "PACKAGE",
    "url" : "https://github.com/vllm-project/vllm"
  } ],
  "database_specific" : {
    "cwe_ids" : [ "CWE-131", "CWE-704" ],
    "severity" : "MODERATE",
    "github_reviewed" : true,
    "github_reviewed_at" : "2026-05-06T21:45:51Z",
    "nvd_published_at" : "2026-05-12T20:16:43Z"
  }
}