Watch: Model, Harness, Gym: Why Novee Owns the Full AI Pentesting Stack

Novee CEO and co-founder Ido Geffen on owning every layer of the AI pentesting stack, from the offensive reasoning model to the harness and the training gym behind it, and why that turns a generalist AI hacker into one built for your business.

Most “AI offensive security” tools are built the same way: renting reasoning from a general-purpose frontier LLM, pointing it at a generic scanning workflow, and calling the result an AI pentester. Novee took the opposite path and built:

• its own model
• the harness that coordinates a set of specialized agents
• the training gym where those agents keep getting sharper

In his latest conversation with CRA, Ido Geffen breaks down what it means to own the full AI pentesting stack, and why each layer earns its place. 

Owning the model is only half the story. The other half is understanding your specific business well enough to attack it like someone who has studied it for years.

Highlights:

How does Novee build a specialized, safe stack for AI pentesting?

Controlling the whole AI stack means Novee can build it securely, with all the guardrails needed to prevent it from exceeding its mandate or moving laterally into systems it shouldn’t access. Novee is building guardrails inside the model, such as prohibiting destructive actions, and also guardrails outside the model, like firewalls the keep the agents constrained to the provided IP address.

When you say Novee owns the full AI stack (model, harness, training gym), what does that mean, and why not build on top of a frontier LLM?

Owning the stack means the core capability isn’t rented from a general-purpose model and bolted onto a generic workflow. Novee builds three layers of its own: a proprietary offensive reasoning model, which supplies the attacker’s instinct; a harness, the multi-model Offensive System, that coordinates specialized agents to handle specific tasks required of a skilled pentester; and a training gym where those agents are benchmarked, post-trained on real attacker tradecraft, and promoted into production. Models are quite powerful, but it can only get you so far as its training provides. A harness around them, and the ability to validate and score their capabilities via the Novee Gym, is what pushes specialized pentesting forward. The right mix of open source models, closed models, and a proprietary model can get to the same exploit rate as frontier labs at a fraction of the cost, when run through the proper harness.

Walk us through the offensive reasoning model. How is it different from frontier models at pentest tasks, and where do you still use frontier models in the pipeline?

Frontier models are generalists that happen to be good at security. Novee’s offensive reasoning model is purpose-trained for the long-horizon, tool-using work of a real pentest, probing a running system, forming a hypothesis, writing an exploit, watching it fail, and revising. On live browser exploitation it beats the cost-accuracy tradeoff any general-purpose model offers. Novee’s fine-tuned models reach the exploit rates of state-of-the-art models at about 15% of the cost, and they grow more persistent and token-efficient with every cycle. Frontier models still play a role inside the pipeline, since the harness runs whichever model performs best for each agent’s task rather than forcing one model to do everything. 

For more on how small language models beat out large frontier LLMs for highly-specialized offensive security work, read the research on small models for offensive security.

You talk about a training “gym” where agents get sharper over time. What happens in there, and how does new tradecraft reach production?

The gym is Novee’s continuous-improvement loop. Agents train inside the exact harness they’ll run in production, against live applications rather than simulators, and the reward is mechanical. An exploit either fires against the live app or it doesn’t, so the model can’t earn credit for a convincing writeup, and that un-gameable signal is what keeps it getting sharper. The loop feeds itself, since agents surface fresh zero-days in the wild that become training data for the next round. When the research team spots a new attacker technique, it goes into agent memory right away, so customers feel the improvement every cycle. 

For more on how Novee uses reinforcement learning to specialize and post-train its models, read the full write-up on NoveeGym-RL.)

Will AI get as good as human pentesters at offensive security?

The right mix is skilled pentesters with the right models, routed and benchmarked by a purpose-built system. Humans are skilled at finding issues, but the scale of validating those issues across enterprise-scale workspaces is challenging. On the flip side, agentic offensive security platforms are skilled at finding obscure issues (e.g. user enumeration), but oftentimes need the intuition and business acumen of a human to tell them that those issues are in fact not applicable in their specific environment. That’s where business context comes in.

Owning the model is one thing, but you’ve said the real unlock is pairing it with the Asset Intelligence Model, a “bespoke hacker” for each customer. What does that mean, and what does it find that a generalist AI hacker never could?

A well-trained model gives you an elite attacker; the Asset Intelligence Model makes that attacker an expert at your business specifically. AIM builds a living understanding of each environment, capturing its roles, permissions, workflows, APIs, and business logic, and it compounds that understanding every cycle, so coverage expands instead of resetting. That’s the difference between an attacker who reasons well and one who reasons well about your specific business. It’s what surfaces the flaws generic tools structurally miss, like business logic abuse, authorization gaps, and multi-step attack paths chained across endpoints, the vulnerabilities that only show up once the system understands how your application is supposed to work. Novee is particularly adept at detecting novel business logic vulnerabilities, such as preventing cross-tenant views in payroll systems, or preventing adversarial price manipulation in digital retail software.

If I’m a CISO evaluating AI pentesting vendors next quarter, what’s the tangible difference between a platform that owns its stack and one orchestrating someone else’s LLM?

A vendor orchestrating someone else’s LLM inherits that model’s cost curve, its release schedule, and its ceiling, then starts fresh on your environment with every engagement. When you own the stack, the buyer’s experience changes in concrete ways. Findings get more targeted every cycle instead of resetting, because the platform is building a living model of your environment. Remediation is tailored to your actual WAF, backend, and code, then automatically retested to confirm the fix held. Cost, meanwhile, stays contained because it’s tied to the complexity of your environment rather than token spend, even as coverage grows. 

Catch the full conversation with CRA, or book a demo to see it run against your environment. And if you’re weighing AI pentesting vendors, start with the eight questions that separate a purpose-built offensive platform from a generalist.