GHSA-9x8m-2xpf-crp3: Scrapy Scrapy Before 2.6.2 And 1.8.3 Vulnerability

MODERATE
Share
Published
Advisory Summary

GHSA-9x8m-2xpf-crp3 is a MODERATE scrapy before 2.6.2 and 1.8.3 vulnerability (General Security Weakness) affecting Scrapy libraries in the PyPI ecosystem. The issue involves scrapy before 2.6.2 and 1.8.3 vulnerable to one proxy sending credentials to another. Remediation is available by upgrading affected packages to version 1.8.3, 2.6.2.

GHSA-9x8m-2xpf-crp3 at a Glance

CVE Identifier N/A
GHSA Identifier GHSA-9x8m-2xpf-crp3
Severity MODERATE
CVSS v3.1 Base Score N/A
CWE Classification General Security Weakness
Affected Product Scrapy
Package Ecosystem PyPI
Published Date
Last Updated

How to Fix GHSA-9x8m-2xpf-crp3

To remediate this vulnerability, upgrade all affected Scrapy dependencies to a patched release. Both direct and transitive dependencies should be checked, as frameworks or security libraries may pull in vulnerable versions transitively.

scrapy fixed: 1.8.3 
pip install scrapy>=1.8.3
scrapy fixed: 2.6.2 
pip install scrapy>=2.6.2
Verify Direct and Transitive Dependencies

Run the following command in your project directory to verify that all resolved dependencies use the patched release: 

pip show scrapy

How to Check if You Are Affected

To determine whether your Python environment is affected by GHSA-9x8m-2xpf-crp3, inspect your installed packages using pip:

pip show scrapy

Analyze the terminal output to locate any version of Scrapy in your dependency tree. If any resolved version matches the vulnerable ranges listed in the table below (for example, versions earlier than 1.8.3), your application is affected.

Note: The pip show scrapy command checks currently installed packages in your active Python environment.

Technical Details

Vulnerability Mechanism & Technical Analysis

This vulnerability is classified under General Security Weakness. In Scrapy, software state validation or input boundaries are circumvented during processing. Review the official source advisory statement below for specific affected components, classes, and reproduction details.

Source Advisory Statement

### Impact When the [built-in HTTP proxy downloader middleware](https://docs.scrapy.org/en/2.6/topics/downloader-middleware.html#module-scrapy.downloadermiddlewares.httpproxy) processes a request with `proxy` metadata, and that `proxy` metadata includes proxy credentials, the built-in HTTP proxy downloader middleware sets the `Proxy-Authentication` header, but only if that header is not already set. There are third-party proxy-rotation downloader middlewares that set different `proxy` metadata every time they process a request. Because of request retries and redirects, the same request can be processed by downloader middlewares more than once, including both the built-in HTTP proxy downloader middleware and any third-party proxy-rotation downloader middleware. These third-party proxy-rotation downloader middlewares could change the `proxy` metadata of a request to a new value, but fail to remove the `Proxy-Authentication` header from the previous value of the `proxy` metadata, causing the credentials of one proxy to be leaked to a different proxy. If you rotate proxies from different proxy providers, and any of those proxies requires credentials, you are affected, unless you are handling proxy rotation as described under **Workarounds** below. If you use a third-party downloader middleware for proxy rotation, the same applies to that downloader middleware, and installing a patched version of Scrapy may not be enough; patching that downloader middlware may be necessary as well. ### Patches Upgrade to Scrapy 2.6.2. If you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.6.2 is not an option, you may upgrade to Scrapy 1.8.3 instead. ### Workarounds If you cannot upgrade, make sure that any code that changes the value of the `proxy` request meta also removes the `Proxy-Authorization` header from the request if present. ### For more information If you have any questions or comments about this advisory: * [Open an issue](https://github.com/scrapy/scrapy/issues) * [Email us](mailto:opensource@zyte.com)

GHSA-9x8m-2xpf-crp3 Affected Scrapy Versions

2 component(s)
Fixed Versions:
1.8.32.6.2
Product / Package Ecosystem Affected Versions Fixed Version
scrapy
pypi/scrapy
PyPI
< 1.8.3
1.8.3
scrapy
pypi/scrapy
PyPI
≥ 2.0.0
< 2.6.2
2.6.2

References

GitHub Advisory / Wiki ADVISORY
https://github.com/scrapy/scrapy/security/advisories/GHSA-9x8m-2xpf-crp3
Security Fix Commit FIX COMMIT
https://github.com/scrapy/scrapy/commit/af7dd16d8ded3e6cb2946603688f4f4a5212e80f
Scrapy Source Repository REPOSITORY
https://github.com/scrapy/scrapy

Frequently Asked Questions

What is GHSA-9x8m-2xpf-crp3?
GHSA-9x8m-2xpf-crp3 is a security vulnerability in Scrapy involving Scrapy before 2.6.2 and 1.8.3 vulnerable to one proxy sending credentials to another under General Security Weakness.
Which Scrapy versions are affected?
Affected versions include releases prior to 1.8.3, 2.6.2 across affected packages: scrapy.
What version fixes GHSA-9x8m-2xpf-crp3?
Upgrading to 1.8.3, 2.6.2 (or the latest patched release for your respective branch) resolves the vulnerability.
Is scrapy affected?
Yes, scrapy is affected in releases prior to 1.8.3, 2.6.2. Upgrade your dependency to 1.8.3, 2.6.2 or later.
What CWE is associated with GHSA-9x8m-2xpf-crp3?
GHSA-9x8m-2xpf-crp3 is classified under General Security Weakness.
How can I check PyPI dependencies for GHSA-9x8m-2xpf-crp3?
Run the dependency tree command: 'pip show scrapy'. This command lists direct and transitive dependencies in your build configuration so you can verify if a vulnerable version is resolved.
Machine-readable OSV JSON OSV v1.5 Schema
Normalized schema output
View Raw JSON 
{
  "schema_version" : "1.4.0",
  "id" : "GHSA-9x8m-2xpf-crp3",
  "modified" : "2023-11-15T18:31:22Z",
  "published" : "2022-07-29T22:26:57Z",
  "aliases" : [ ],
  "summary" : "Scrapy before 2.6.2 and 1.8.3 vulnerable to one proxy sending credentials to another",
  "details" : "### Impact\n\nWhen the [built-in HTTP proxy downloader middleware](https://docs.scrapy.org/en/2.6/topics/downloader-middleware.html#module-scrapy.downloadermiddlewares.httpproxy) processes a request with `proxy` metadata, and that `proxy` metadata includes proxy credentials, the built-in HTTP proxy downloader middleware sets the `Proxy-Authentication` header, but only if that header is not already set.\n\nThere are third-party proxy-rotation downloader middlewares that set different `proxy` metadata every time they process a request.\n\nBecause of request retries and redirects, the same request can be processed by downloader middlewares more than once, including both the built-in HTTP proxy downloader middleware and any third-party proxy-rotation downloader middleware.\n\nThese third-party proxy-rotation downloader middlewares could change the `proxy` metadata of a request to a new value, but fail to remove the `Proxy-Authentication` header from the previous value of the `proxy` metadata, causing the credentials of one proxy to be leaked to a different proxy.\n\nIf you rotate proxies from different proxy providers, and any of those proxies requires credentials, you are affected, unless you are handling proxy rotation as described under **Workarounds** below. If you use a third-party downloader middleware for proxy rotation, the same applies to that downloader middleware, and installing a patched version of Scrapy may not be enough; patching that downloader middlware may be necessary as well.\n\n### Patches\n\nUpgrade to Scrapy 2.6.2.\n\nIf you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.6.2 is not an option, you may upgrade to Scrapy 1.8.3 instead.\n\n### Workarounds\n\nIf you cannot upgrade, make sure that any code that changes the value of the `proxy` request meta also removes the `Proxy-Authorization` header from the request if present.\n\n### For more information\n\nIf you have any questions or comments about this advisory:\n* [Open an issue](https://github.com/scrapy/scrapy/issues)\n* [Email us](mailto:opensource@zyte.com)\n",
  "severity" : [ ],
  "affected" : [ {
    "package" : {
      "ecosystem" : "PyPI",
      "name" : "scrapy"
    },
    "ranges" : [ {
      "type" : "ECOSYSTEM",
      "events" : [ {
        "introduced" : "0"
      }, {
        "fixed" : "1.8.3"
      } ]
    } ]
  }, {
    "package" : {
      "ecosystem" : "PyPI",
      "name" : "scrapy"
    },
    "ranges" : [ {
      "type" : "ECOSYSTEM",
      "events" : [ {
        "introduced" : "2.0.0"
      }, {
        "fixed" : "2.6.2"
      } ]
    } ]
  } ],
  "references" : [ {
    "type" : "WEB",
    "url" : "https://github.com/scrapy/scrapy/security/advisories/GHSA-9x8m-2xpf-crp3"
  }, {
    "type" : "WEB",
    "url" : "https://github.com/scrapy/scrapy/commit/af7dd16d8ded3e6cb2946603688f4f4a5212e80f"
  }, {
    "type" : "PACKAGE",
    "url" : "https://github.com/scrapy/scrapy"
  } ],
  "database_specific" : {
    "cwe_ids" : [ ],
    "severity" : "MODERATE",
    "github_reviewed" : true,
    "github_reviewed_at" : "2022-07-29T22:26:57Z",
    "nvd_published_at" : null
  }
}