Why Novee Builds its Own Offensive Security AI

If you want to build something exceptional in offensive security, owning the harness is not enough. The best way to optimize the entire AI stack is to own the entire AI stack.

It is our thesis that, to build the world’s best offensive security AI, you cannot rely on generic frontier models or pre-trained specialized models alone. That is why our strategy at Novee has always been to own the entire AI stack (both the model and the harness), and continuously optimize every layer via rigorous post-training for recall, precision, and cost. That’s what makes the AI Hacker better, the AI Defender faster, and the entire system smarter and more cost efficient with every testing cycle.

An AI agent is a large language model wired into a harness. The model is the reasoning core, trained on huge amounts of data to read a situation, weigh the options, and decide what to do next, but left alone it can only generate text. The harness is the software around it that supplies tools to act with, memory to draw on, and a loop to run in, turning the model’s decisions into real actions: observe the environment, decide, execute, then adjust based on what comes back.

Most offensive AI security products still run entirely on generic frontier models, which means they are constrained by whatever the labs decide to ship next, on the same foundation everyone else can rent. The power of a proprietary harness cannot be overstated, but there is still a hard ceiling on what the system underneath can do. Offensive security is one of the few domains where a ceiling is unacceptable, because the other side never stops climbing.

That is why we post-train our own open-weight models, tuned for the capabilities, economics, and constraints of offensive security specifically. 

For context, post-training refers to a later stage of AI model training, where instruction tuning, reinforcement learning from human feedback, and reinforcement learning from environmental feedback combine to turn the mode into a useful assistant that can take actions autonomously. Post-training is where specific skills are encoded deep into a model’s capabilities, which presents a unique opportunity for a specialized domain like offensive security.

To be clear, frontier models are not bad. Offensive security is simply too specialized, too important, and too expensive to keep treating the model layer as interchangeable infrastructure.

Owning the AI Stack for Continuous Penetration Testing 

Pentesting has its own patterns, its own tactics, and its own definition of success. General-purpose models are impressive on average, when average is exactly the wrong target. 

Andrej Karpathy has a useful name for this: “jagged intelligence,” the observation that generalist models don’t improve evenly across tasks. They can be brilliant in one area and mediocre in the one right next to it. Prime Intellect Lab and Ramp have already shown a specialized model beating frontier quality at a fraction of the cost and latency. 

On the cost problem topic; in Harvey AI’s legal benchmark, frontier agents run on the order of $50 and twenty-plus minutes per task, and post-training is their stated bet for bringing that down. The same logic applies to security, only more so: continuous pentesting is a high-volume workload, and if every step routes to a frontier API the economics fall apart fast.

Dependency on someone else’s model is a product risk, not just an ops concern. Anthropic recently pulled Fable 5 offline for every customer worldwide, following a US government decision, with only hours of notice. If your core capability lives entirely on a model you do not control, you are exposed to decisions you cannot influence and cannot plan around. A security team needs its tooling to still be working on the day a frontier provider changes the rules.

What the Novee Benchmark Shows

We benchmarked Novee on real penetration testing tasks, pitting it against an open source harness, running Anthropic’s Claude Opus model. The tests measured:

1. Recall (the share of known vulnerabilities found)
2. Precision (the share of findings that are real)

These were not CTF puzzles. They were applications that look and behave like production targets, with the complexity and configuration that comes with that.

At the model layer, we put Claude Opus running at maximum performance on an open source harness up against Novee’s proprietary model paired with a frontier model, with the harness routing each task to whichever performs best for the job. Novee came out ahead on both recall (62% versus 29%) and precision (90% versus 46%), across real open-source applications with documented and zero-day vulnerabilities. 

Doubling recall means we surface close to twice as many real vulnerabilities, and we do it while keeping precision nearly twice as high, so fewer of those findings are noise.

At the system layer, the full Novee harness runs the entire operation, from recon and planning through exploitation, validation, and remediation guidance, with expert prompting and specialized pentester skills built into each stage. 

On a separate benchmarking exercise focused on cost efficiency, Novee discovered 2.5x as many vulnerabilities per $100 spent as the open source harness running Claude Opus at maximum performance.

In summary: The Novee AI pentesting platform finds 2.5x more vulnerabilities per dollar as leading frontier models with an unspecialized harness, and produces more precise and accurate results.

Here’s how we achieve these results:

A Training Loop for Offensive Security AI: The Novee Gym

A large language model comes together in three stages.

1. Pre-training is where the model learns language itself. You feed it an enormous amount of text, roughly a compressed version of the internet, and it learns to predict the next token. This is where raw capability comes from. It is long and expensive, and it sits almost entirely with a handful of large labs.

2. Mid-training, sometimes called continued pre-training, is where you specialize the knowledge base. If pre-training is a general education, this is a graduate degree, and our curriculum is security research, vulnerability databases, code repositories, and exploit documentation.

3. Post-training is where behavior gets shaped. This is the step that turns a knowledgeable model into one that is genuinely useful inside a workflow, through instruction tuning, reinforcement learning from human feedback, and increasingly reinforcement learning from environment feedback, which is the approach we care about most at Novee.

For agentic offensive security, post-training is the lever that does the heavy lifting. Frontier models already know a great deal about security in the abstract. What they lack is the ability to navigate a live application, confirm that an exploit actually works, and produce a reliable proof-of-concept under realistic conditions. More pre-training data does not close that gap. Targeted post-training inside a realistic simulation environment does.

That environment is something we have been building for a while, and we call it the Novee Gym.

You cannot train a model by turning it loose on production systems, so you need a simulation environment that is safe and scalable, plus a reliable way to tell whether the agent truly succeeded or just got lucky. The Gym is how we solved that:

1. Simulation comes first. We source realistic web applications with known vulnerabilities and stand them up as isolated training environments, with the messiness of real targets rather than toy CTF challenges.

2. Evaluation involves a verification layer that checks whether an exploit actually worked, because “the model found something” is not a signal. “The model produced a valid, reproducible proof-of-concept for this specific vulnerability” is the signal that counts. Getting that distinction right is most of the work.

3. The training loop ties it all together into an end-to-end reinforcement learning flow. The agent runs in the Gym, earns reward for real exploits, takes a penalty for false positives, and the model improves from there. We can swap in different base models, run experiments quickly, and measure the things that actually track with field performance.

Best-in-Class Pentesting Demands a Customized Offensive Security AI stack

Novee was built to scale a security team’s expertise across its entire environment, giving them the reach of a much larger, always-on pentesting team without losing the depth of an elite one. A proprietary model, trained in the Gym, alongside a harness – both consistently optimized – is the foundation that makes that possible. 

It is what lets our multi-model system reason about your applications the way a real attacker would, surface the business logic flaws and chained attack paths that lead to breaches, prove each finding is exploitable, and guide your team to a verified, stack-specific fix, continuously, as your code keeps shipping. That is what it takes to operate as the best hacker and the best defender at the same time. 

Against a real adversary, performance is the thing that wins, which is exactly why we will not accept a ceiling on ours.

To see what your attackers already know – and how Novee finds, proves, and helps you close the vulnerabilities that lead to real breaches, continuously – schedule a demo.