Backdoored PyTorch Lightning package drops credential stealer

Backdoored PyTorch Lightning Package Drops Credential Stealer

OverviewA widely used open-source machine learning framework suffered a supply-chain incident when a malicious version of the PyTorch Lightning package appeared on the Python Package Index (PyPI). The incident centers on version 2.6.3, which contained a hidden execution chain that downloads and runs a JavaScript payload during import. The attack targets credentials stored on the host, including environment files, API keys, tokens, and data cached by browsers, as well as cloud service credentials. The event highlights the risk of supply-chain compromises in popular open-source projects and how quickly a legitimate package can become a vector for credential theft.

What happened

• A compromised release of the PyTorch Lightning package was published to PyPI, claiming to be version 2.6.3. The package was described as a standard distribution (py3-none-any wheel) but included a concealed execution chain.
• The hidden payload activates automatically when the package is imported, without any explicit action from the user beyond importing lightning.
• The execution chain silently downloads a JavaScript runtime (Bun v1.3.13) from GitHub and executes an obfuscated JavaScript payload named router_runtime.js, which is about 11.4 MB in size.
• The payload is designed to exfiltrate credentials and sensitive data from multiple sources, including:
• Environment files and configuration secrets stored on the host
• API keys and tokens used by developers and CI/CD processes
• Browser-stored data from Chrome, Firefox, and Brave
• Credentials for cloud service providers (AWS, Azure, Google Cloud)
• Defender technologies reportedly detected and blocked the malicious routine on affected environments, and notified the package maintainer. The involved threat class has been identified by Defender as “ShaiWorm.”

Technical details of the intrusion

• Trigger mechanism: The malicious code is executed as part of the normal import process for the Lightning package, making the compromise stealthy and hard to detect during routine usage.
• Code delivery: A JavaScript runtime (Bun) is fetched from GitHub as part of the payload delivery, followed by execution of a large, heavily obfuscated JavaScript payload.
• Payload behavior: The payload focuses on credential theft across multiple vectors, including local secret stores (environment files), browser-stored secrets, and cloud-provider credentials. It is described as capable of interacting with cloud APIs to facilitate further credential access and supports arbitrary command execution on the compromised system.
• Names associated with the attack: The infostealer component is identified in Defender telemetry as ShaiWorm, and the component responsible for the concealment and execution is tied to the hidden runtime and payload in the 2.6.3 release.

Scope and impact

• Adoption and reach: PyTorch Lightning is a widely used deep learning framework, with substantial download activity. The affected version was among recent releases and had a broad distribution footprint prior to the discovery.
• Affected environment: Affected devices likely included a mix of developer workstations and servers where Python environments were used to import Lightning. The impact could span local development, data science workflows, and automated pipelines that rely on local credentials.
• Data exposure risk: The incident presented a direct risk to secrets and tokens used in development and production environments, including:
• Environment variables and configuration files
• API keys and access tokens stored on the host
• Browser-saved credentials and session data
• Access credentials for cloud services (AWS, Azure, GCP)
• Containment status: Microsoft Threat Intelligence indicated that Defender protections detected and mitigated the malicious activity, with the scope described as limited to a small number of devices and a narrow subset of environments.
• Current status of the package: The PyTorch Lightning maintainers reverted the package on PyPI to a safe version, 2.6.1, after identifying the malicious release. The investigation into how the supply chain was breached is ongoing, and all recent releases are subject to auditing for similar payloads.

Detection and response

• Threat intelligence findings: Defender telemetry provided attribution to the ShaiWorm family and confirmed that credential theft activities were attempted on affected hosts.
• Maintainer response: The security advisory from the Lightning AI team confirmed the existence of the hidden execution chain and the credential-stealing payload. The advisory also noted that the malicious behavior was triggered on import and that the release had been reverted to a safe state.
• User-facing implications: Users who had already migrated to the 2.6.3 release or who executed import lightning in environments with sensitive data could have exposed credentials. Immediate review and rotation of secrets were repeatedly emphasized by the advisory and threat intelligence feeds.

Current status and next steps for the community

• Package status: PyTorch Lightning has been rolled back to a secure release (2.6.1) on PyPI, and users are advised to avoid using the compromised 2.6.3 release until further notice.
• Investigation status: The publishers and maintainers are actively investigating the breach in the build/release pipeline to determine how the compromised package was created and distributed.
• Audit plan: In addition to the immediate incident, other recent Lightning releases are undergoing auditing for potential payloads, with stakeholders being notified through available channels as needed.
• Broader implications: The incident underscores the importance of supply-chain security for open-source projects and reinforces the need for provenance checks, integrity verification, and robust release pipelines to prevent similar attacks in the future.

Notes on related developments

• The incident is part of a broader array of supply-chain security concerns affecting package ecosystems, where malicious payloads have been observed in other widely used third-party libraries and tools.
• Public communications from threat intelligence teams describe the specific artifacts involved (JavaScript payloads, hidden execution chains, and credential-stealing capabilities) and emphasize the importance of monitoring for unusual behavior during package imports and automated workflows.

ConclusionThe backdoored PyTorch Lightning release demonstrates how a trusted, widely deployed library can be weaponized to harvest credentials and sensitive data. The combination of a hidden execution chain, an obfuscated payload, and cross-vector data exfiltration (environment files, browser data, and cloud credentials) creates a high-stakes scenario for developers and organizations relying on open-source software. Ongoing investigations aim to determine the exact breach vector, secure the supply chain, and ensure that future releases are protected against similar compromises.