DEVOPSdigest

CyberArk announced the launch of FuzzyAI, an open-source framework that helps organizations identify and address AI model vulnerabilities, like guardrail bypassing and harmful output generation, in cloud-hosted and in-house AI models.

FuzzyAI helps solve some of these challenges by offering organizations a systematic approach to testing AI models against various adversarial inputs, uncovering potential weak points in their security systems and making AI development and deployment safer. At the heart of FuzzyAI is a powerful fuzzer - a tool that reveals software defects and vulnerabilities - capable of exposing vulnerabilities found via more than ten distinct attack techniques, from bypassing ethical filters to exposing hidden system prompts.

Key features of FuzzyAI include:

- Comprehensive Fuzzing: FuzzyAI probes AI models with various attack techniques to expose vulnerabilities like bypassing guardrails, information leakage, prompt injection or harmful output generation.

- An Extensible Framework: Organizations and researchers can add their own attack methods to tailor tests for domain-specific vulnerabilities.

- Community Collaboration: A growing community-driven ecosystem ensures continuous adversarial techniques and defense mechanisms advancements.

“The launch of FuzzyAI underlines CyberArk’s commitment to AI security and helps organizations take a significant step forward in addressing the security issues inherent in the evolving landscape of AI model usage,” said Peretz Regev, Chief Product Officer at CyberArk. “Developed by CyberArk Labs, FuzzyAI has demonstrated the ability to jailbreak every major tested AI model. FuzzyAI empowers organizations and researchers to identify weaknesses and actively fortify their AI systems against emerging threats.”

FuzzyAI’s fully extensible framework is available as open-source software on CyberArk Labs’ GitHub Page from December 11, 2024.