Backslash Identifies AI-Generated Code Concerns via OSS Reachability Analysis, Phantom Package Visibility Capabilities
Backslash Research Team performs GPT-4 developer simulation exercise to identify security blindspots in LLM-generated code
April 30, 2024

Backslash Security announced the findings of its GPT-4 developer simulation exercise, designed and conducted by the Backslash Research Team, to identify security issues associated with LLM-generated code. The Backslash platform offers several core capabilities that address growing security concerns around AI-generated code, including open source code reachability analysis and phantom package visibility capabilities.

According to Gartner, 63% of organizations are currently piloting or deploying AI code assistants. Due to its simplicity of use, AI-generated code will dramatically increase the pace of new code development. However, this technology introduces a diverse range of potential vulnerabilities and security challenges.

In research conducted by the Backslash Research Team to explore security gaps associated with AI-generated code from the perspective of developers, the team created and performed a variety of developer simulation exercises via a series of tests using GPT-4. The results revealed critical security blindspots associated with AI-generated code and its use of third-party open source software (OSS):

Some LLMs can generate vulnerable OSS package recommendations due to being ‘frozen in time’: Given that many LLMs are trained on static datasets that are only current up to a specific date and are thus deprived of dynamic patch updates, OSS package recommendations generated by LLMs may be outdated. This includes previous OSS package versions with older code and potential security vulnerabilities that have since been fixed in newer versions.

‘Phantom’ packages can introduce unseen reachable risks: LLM-generated code can include indirect OSS packages that developers are not aware of. Developers often have little to no visibility or control over these “phantom” packages, which may insert security issues into code production via outdated, vulnerable packages.

Seemingly safe code-snippet outputs can create an illusion of trust: Experiments reveal that when using the same prompt, GPT-4 generated different recommendations, occasionally suggesting vulnerable package versions. Although these outputs often include disclaimers or instructions regarding the package version, they sometimes do not. Such inconsistencies may cause developers to view all AI-generated code as reliable, thereby introducing significant product security risks for large development teams that deploy high volumes of code daily.

As AI-generated code continues to gain momentum, security issues stemming from the unintended use of outdated OSS packages in application code will become increasingly prominent. The Backslash platform, which was built first and foremost with security in mind for application security teams concentrating on SCA and SAST, offers several capabilities that address AI-generated code security concerns associated with open source software:

In-depth reachability analysis: The Backslash platform’s unique approach to SCA analysis assesses the reachability of OSS vulnerabilities, enabling AppSec and product security teams to pinpoint the risks that are reachable and exploitable, and prioritize the genuine threats.

Phantom package visibility: Extending beyond traditional SCA, Backslash can identify and assess phantom package risks. The platform detects phantom packages being used by code that is not declared in manifest files and determines whether the packages are reachable and the level of risk they pose.

“The way we create code is rapidly changing, and that means the way that we secure code must also change. AI-generated code offers immense possibility, but also introduces an entirely new scale of security challenges – and application security teams now bear the burden of securing an unprecedented volume of potentially vulnerable code due to the sheer speed of AI-enabled software development,” said Shahar Man, co-founder and CEO of Backslash Security. “Our research shows that securing open source code is more critical than ever before due to product security issues being introduced by AI-generated code that is associated with OSS.”

Backslash enables teams to fix only the vulnerable code and OSS that indeed needs addressing – the reachable, exploitable components.

Share this

Industry News

January 09, 2025

Checkmarx announced a new generation in software supply chain security with its Secrets Detection and Repository Health solutions to minimize application risk.

January 08, 2025

SmartBear has appointed Dan Faulkner, the company’s Chief Product Officer, as Chief Executive Officer.

January 07, 2025

Horizon3.ai announced the release of NodeZero™ Kubernetes Pentesting, a new capability available to all NodeZero users.

January 06, 2025

GitHub announced GitHub Copilot Free.

January 06, 2025

Veracode acquired certain assets of Phylum, including its malicious package analysis, detection, and mitigation technology.

January 06, 2025

AppViewX announced the completion of its acquisition by Haveli Investments.

December 19, 2024

Check Point® Software Technologies Ltd. has been recognized as a Leader in the 2024 Gartner® Magic Quadrant™ for Email Security Platforms (ESP).

December 19, 2024

Progress announced its partnership with the American Institute of CPAs (AICPA), the world’s largest member association representing the CPA profession.

December 18, 2024

Kurrent announced $12 million in funding, its rebrand from Event Store and the official launch of Kurrent Enterprise Edition, now commercially available.

December 18, 2024

Blitzy announced the launch of the Blitzy Platform, a category-defining agentic platform that accelerates software development for enterprises by autonomously batch building up to 80% of software applications.

December 17, 2024

Sonata Software launched IntellQA, a Harmoni.AI powered testing automation and acceleration platform designed to transform software delivery for global enterprises.

December 17, 2024

Sonar signed a definitive agreement to acquire Tidelift, a provider of software supply chain security solutions that help organizations manage the risk of open source software.

December 17, 2024

Kindo formally launched its channel partner program.

December 16, 2024

Red Hat announced the latest release of Red Hat Enterprise Linux AI (RHEL AI), Red Hat’s foundation model platform for more seamlessly developing, testing and running generative artificial intelligence (gen AI) models for enterprise applications.

December 16, 2024

Fastly announced the general availability of Fastly AI Accelerator.