Join the event trusted by enterprise leaders for nearly two decades. VB Transform brings together the people building real enterprise AI strategy.Learn more
Editor’s note: Louis will lead an editorial roundtable on this topic at VB Transform this month.Register today.
AI models are under siege. With77%of enterprises already hit by adversarial model attacks and41%of those attacks exploiting prompt injections and data poisoning, attackers’ tradecraft is outpacing existing cyber defenses.
To reverse this trend, it’s critical to rethink how security is integrated into the models being built today. DevOps teams need to shift from taking a reactive defense to continuous adversarial testing at every step.
Protecting large language models (LLMs) across DevOps cycles requires red teaming as a core component of the model-creation process. Rather than treating security as a final hurdle, which is typical in web app pipelines, continuous adversarial testing needs to be integrated into every phase of the Software Development Life Cycle (SDLC).
Adopting a more integrative approach to DevSecOps fundamentals is becoming necessary to mitigate the growing risks of prompt injections, data poisoning and the exposure of sensitive data. Severe attacks like these are becoming more prevalent, occurring from model design through deployment, making ongoing monitoring essential.
Microsoft’s recent guidance onplanningred teaming for large language models (LLMs)and their applications provides a valuable methodology for startingan integrated process.NIST’s AI Risk Management Frameworkreinforces this, emphasizing the need for a more proactive, lifecycle-long approach to adversarial testing and risk mitigation. Microsoft’s recent red teaming of over 100 generative AI products underscores the need to integrate automated threat detection with expert oversight throughout model development.
As regulatory frameworks, such as the EU’s AI Act, mandate rigorous adversarial testing, integrating continuous red teaming ensures compliance and enhanced security.
OpenAI’sapproach to red teamingintegrates external red teaming from early design through deployment, confirming that consistent, preemptive security testing is crucial to the success of LLM development.
Traditional, longstanding cybersecurity approaches fall short against AI-driven threats because they are fundamentally different from conventional attacks. As adversaries’ tradecraft surpasses traditional approaches, new techniques for red teaming are necessary. Here’s a sample of the many types of tradecraft specifically built to attack AI models throughout the DevOps cycles and once in the wild:
Integrated Machine Learning Operations (MLOps) further compound these risks, threats, and vulnerabilities. The interconnected nature of LLM and broader AI development pipelines magnifies these attack surfaces, requiring improvements in red teaming.
Cybersecurity leaders are increasingly adopting continuous adversarial testing to counter these emerging AI threats. Structured red-team exercises are now essential, realistically simulating AI-focused attacks to uncover hidden vulnerabilities and close security gaps before attackers can exploit them.
Adversaries continue to accelerate their use of AI to create entirely new forms of tradecraft that defy existing, traditional cyber defenses. Their goal is to exploit as many emerging vulnerabilities as possible.
Industry leaders, including the major AI companies, have responded by embedding systematic and sophisticated red-teaming strategies at the core of their AI security. Rather than treating red teaming as an occasional check, they deploy continuous adversarial testing by combining expert human insights, disciplined automation, and iterative human-in-the-middle evaluations to uncover and reduce threats before attackers can exploit them proactively.
Their rigorous methodologies allow them to identify weaknesses and systematically harden their models against evolving real-world adversarial scenarios.
In short, AI leaders know that staying ahead of attackers demands continuous and proactive vigilance. By embedding structured human oversight, disciplined automation, and iterative refinement into their red teaming strategies, these industry leaders set the standard and define the playbook for resilient and trustworthy AI at scale.
As attacks on LLMs and AI models continue to evolve rapidly, DevOps and DevSecOps teams must coordinate their efforts to address the challenge of enhancing AI security. VentureBeat is finding the following five high-impact strategies security leaders can implement right away:
Taken together, these strategies ensure DevOps workflows remain resilient and secure while staying ahead of evolving adversarial threats.
AI threats have grown too sophisticated and frequent to rely solely on traditional, reactive cybersecurity approaches. To stay ahead, organizations must continuously and proactively embed adversarial testing into every stage of model development. By balancing automation with human expertise and dynamically adapting their defenses, leading AI providers prove that robust security and innovation can coexist.
Ultimately, red teaming isn’t just about defending AI models. It’s about ensuring trust, resilience, and confidence in a future increasingly shaped by AI.
I’ll be hosting two cybersecurity-focused roundtables at VentureBeat’sTransform 2025, which will be held June 24–25 at Fort Mason in San Francisco. Register to join the conversation.
My session will include one on red teaming,AI Red Teaming and Adversarial Testing, diving into strategies for testing and strengthening AI-driven cybersecurity solutions against sophisticated adversarial threats.
If you want to impress your boss, VB Daily has you covered. We give you the inside scoop on what companies are doing with generative AI, from regulatory shifts to practical deployments, so you can share insights for maximum ROI.
Thanks for subscribing. Check out moreVB newsletters here.
