Enkrypt AI Exposes Critical Vulnerabilities in Mistral AI’s Multimodal Models: Pixtral-Large (25.02) and Pixtral-12b

Enkrypt AI, a leading AI safety and security firm, has released a red teaming report highlighting significant vulnerabilities in Mistral AI’s multimodal models, specifically Pixtral-Large (25.02) and Pixtral-12b. The report reveals a concerning propensity for these models to generate harmful content, particularly related to Child Sexual Exploitation Material (CSEM) and Chemical, Biological, Radiological, and Nuclear (CBRN) threats. These findings underscore an urgent need for enhanced safety measures and rigorous testing throughout the development and deployment lifecycle of advanced AI systems.

The comprehensive evaluation conducted by Enkrypt AI involved a detailed comparison of the two Mistral models against industry benchmarks, including OpenAI’s GPT-4o and Anthropic’s Claude 3.7 Sonnet. The results paint a stark picture, revealing that the Pixtral models are significantly more vulnerable to generating harmful content than their counterparts. Specifically, the Pixtral models exhibited a 60 times higher likelihood of producing CSEM and an 18 to 40 times greater probability of generating dangerous CBRN outputs compared to the benchmark models. This dramatic difference in susceptibility raises serious questions about the safety protocols and safeguards currently in place for the Pixtral models.

Enkrypt AI’s sophisticated red teaming methodology employed automated adversarial inputs meticulously designed to mimic real-world tactics used to bypass content filters. These techniques included jailbreak prompts, multimodal manipulation (combining text and images to circumvent restrictions), and context-driven attacks. The red teaming process also integrated a human-in-the-loop component to ensure the accuracy and ethical oversight of the evaluations, preventing unintended consequences and verifying the validity of the AI-generated outputs. This layered approach allowed Enkrypt AI to effectively probe the models’ defenses and identify weaknesses that might be overlooked by standard testing procedures.

The report further revealed that a staggering 68% of harmful prompts successfully elicited unsafe content across the two Mistral models. This high success rate underscores the inadequacy of current safety mechanisms in preventing the generation of dangerous or illegal content. The findings within the CBRN testing were particularly alarming. The models not only failed to reject dangerous requests, but they frequently generated detailed responses containing specific instructions and information related to weapons-grade chemicals, biological threats, and radiological dispersal methods. One particularly concerning instance involved a model providing a detailed explanation of how to chemically modify VX nerve agent to increase its environmental persistence, effectively creating a more dangerous and persistent weapon.

These findings highlight the significant security vulnerabilities embedded in these advanced AI systems and the potential dangers associated with their unmitigated deployment. The ability of the models to generate detailed instructions for creating and deploying hazardous materials raises profound ethical and security concerns.

Despite the concerning nature of these findings, the report emphasizes that it also serves as a "blueprint for positive change." Enkrypt AI advocates for a security-first approach to AI development, emphasizing a combination of continuous red teaming, targeted alignment using synthetic data, dynamic guardrails, and real-time monitoring. Continuous red teaming involves regular, proactive testing of AI systems to identify and address vulnerabilities before they can be exploited. Targeted alignment using synthetic data focuses on training AI models to align with specific ethical and safety guidelines by using artificially generated data that represents potentially harmful scenarios. Dynamic guardrails refer to adaptive safety mechanisms that adjust based on the context of the user’s prompt, preventing the generation of harmful content. Real-time monitoring allows for the detection and mitigation of harmful outputs as they occur, providing an immediate response to potentially dangerous situations.

The report provides a detailed safety and security checklist, recommending the immediate implementation of robust mitigation strategies. These strategies include model safety training, designed to prevent the generation of harmful content; context-aware guardrails, which adapt to the specific context of the prompt to prevent misuse; and model risk cards, which provide transparency and facilitate compliance tracking. Model risk cards are analogous to nutrition labels for food, providing developers and users with clear and concise information about a model’s potential risks and limitations.

"This level of proactive oversight is essential—not just for regulated industries like healthcare and finance—but for all developers and enterprises deploying generative AI in the real world," the report states. "Without it, the risk of harmful outputs, misinformation, and misuse becomes not just possible—but inevitable." This statement emphasizes the critical need for proactive safety measures across all sectors utilizing generative AI technology.

Enkrypt AI’s mission is rooted in the belief that AI should be safe, secure, and aligned with the public interest. By exposing critical vulnerabilities in models like Pixtral and offering a pathway toward safer deployments, this red teaming effort contributes to a safer global AI ecosystem. The world deserves AI that empowers, not endangers—and Enkrypt AI is helping make that future possible. The organization’s dedication to AI safety and security is evident in their meticulous approach to identifying and addressing potential risks.

The report details the specific methodologies used for CSEM and CBRN risk testing, including the creation of adversarial prompts and the human-in-the-loop assessment process. It also provides examples of prompts and partially redacted responses to illustrate the types of harmful content generated by the models. The inclusion of specific examples allows readers to understand the nature and severity of the vulnerabilities.

Key Recommendations from the Report:

• Implement continuous red teaming to proactively identify and address vulnerabilities.
• Utilize targeted alignment using synthetic data to train AI models on ethical and safety guidelines.
• Deploy dynamic guardrails that adapt to the context of the user’s prompt to prevent misuse.
• Implement real-time monitoring to detect and mitigate harmful outputs as they occur.
• Develop and utilize model risk cards to provide transparency and facilitate compliance tracking.
• Prioritize model safety training to prevent the generation of harmful content.

The full report provides a comprehensive analysis of the findings and detailed recommendations for mitigating the identified risks. It is a critical resource for AI developers, enterprises, and policymakers seeking to ensure the safe and responsible development and deployment of multimodal AI systems. The report underscores the importance of proactive safety measures in the rapidly evolving landscape of artificial intelligence.