:novee-gym: Elite AI hackers aren’t born. They’re trained.

Step into the gym at Black Hat 2026.

:novee-gym: Elite AI hackers aren’t born. They’re trained.

Step into the gym at Black Hat 2026.

Novee vs. Promptfoo

Testing model behavior is useful, but it’s only a small part of the surface a pentest has to cover.

Promptfoo is a capable open-source tool for red-teaming models but it stops at the model layer, needs a person to run it, and leaves the running application untouched. Novee tests what attackers actually exploit, and proves every finding.

See Novee in action

Thank you!

We’ll be in touch as soon as we can.

Chosen by teams that take attackers seriously

Novee vs. Promptfoo at a glance

Challenges with Promptfoo

Model-layer testing is useful, but it leaves most of your attack surface untested, including the flaws attackers actually exploit.

Challenges with Promptfoo:

  • Scoped to the model layer.

    Promptfoo probes model behavior like prompt injection and jailbreaks, while the application wrapped around those features goes untested.
  • A human has to drive it.

    Promptfoo is a developer framework built for engineers rather than security teams, so someone has to configure it, run it, and read the results, and nothing happens until a person kicks off a run. There's no hands-off, autonomous mode.
  • No proprietary offensive AI.

    Promptfoo is an open-source framework with no offensive model of its own. It can tell you whether a model misbehaves, but reasoning about an application and chaining that into a working exploit the way an attacker would is beyond what a framework like this does.
  • Blind to business logic and chained attacks.

    It only looks at model behavior, so the IDOR sitting in your API, the auth bypass in your web app, and the exploit chain that stitches small flaws into a breach all slip past it.
  • Stops at model output flags.

    What you get back are model-behavior flags rather than proven exploits. There's no working PoC and no remediation tuned to your stack, and nothing retests the fix, so the loop from finding to verified fix never really starts.

Why Novee Over Promptfoo?

Securing applications with AI features is one important part of an entire penetration testing suite. Novee tests the entire AI application, model included, and then proves validity of what it finds. On top of that, it extends coverage into the other attack surfaces that matter for your business.

Where Novee goes further:

  • Coverage across the full attack surface.

    Novee black-box tests web apps, APIs, mobile apps, and LLM applications. It maps the environment with an Asset Intelligence Model, so the test shows what an attacker can reach in your business.
  • Finds the flaws that matter.

    Novee works through business logic and chains small issues into real attack paths, uncovering multi-step exploits that only show up when you understand how the system fits together.
  • A proprietary offensive AI stack.

    Novee owns and optimizes the entire AI stack, model and harness, meaning maximized efficiency gains and accuracy.
  • Fully autonomous and continuous.

    Novee runs on its own and kicks off automatically whenever code ships through your CI/CD, with no probes to configure and no one waiting around to interpret raw output.
  • Validation built for zero false positives.

    Three independent agents validate findings. If any stage fails, it's never reported. Every finding comes compliance-ready with a working exploit, replication steps, and a PoC script.
  • A closed loop to a verified fix.

    Remediation is written for your actual WAF, backend, and codebase, and once a fix ships Novee retests on its own to confirm the vulnerability is gone.

Novee vs. Promptfoo Across Key Areas

Capability Novee AI Pentesting Promptfoo
Coverage

AI/LLM model layer only.

Application context and depth

Tests model behavior in isolation. No application context, and none that compounds across runs.

Offensive model and harness

Open-source framework. No proprietary offensive model, harness, or training of its own.

Autonomous, continuous testing

Requires human configuration. Runs only when a person triggers it.

Validation architecture

Model-output flags only. No independent validation process or deterministic confirmation described.

Closed-loop remediation & retesting

No stack-specific remediation or retesting.

Workflow & integrations

CI/CD for model testing only. No broader developer-workflow integrations.

Pricing

Open-source and free; paid probes per month.

What security leaders say

“As the leading agentic orchestration platform for the enterprise, data isolation between our customers is non-negotiable. We need to prove that continuously, not once a year. Novee adapted to our multi-tenant SaaS product within days.”

Scott Roberts
CISO
john

“Our pen tests took weeks and consistently missed critical issues. Novee found them immediately and gave us instant remediation guidance. It showed us what we'd been missing.”

John Barrow
CISO

"Traditional DAST produced either zero or irrelevant results. We needed something that could identify complex vulnerabilities like server-side request forgery. Novee consistently surfaces findings we simply weren't seeing before."

Robert Kugler
Head of Security, IT & Compliance

“Novee rethinks penetration testing for how attacks actually happen today. Continuous, attacker-level validation that proves what’s exploitable and shows teams exactly how to fix it is a meaningful shift for modern security programs.”

Troy Wilkinson
Former Fortune 500 CISO
tamir ronen

"The hardest vulnerabilities for us to catch aren’t misconfigurations or known patterns. They’re business logic issues that only show up when someone understands how the application is supposed to work. That’s exactly the gap Novee closes."

Tamir Ronen
CISO, HiBob

"We had EASM tools and manual pentests that produced mostly noise. Novee came in black-box with zero credentials and within days found dozens of real vulnerabilities we could actually fix."

Itzik Menashe
CISO, Global VP IT InfoSec & productivity

“As an AI researcher, what stood out about Novee is that they built a proprietary offensive AI model designed to think like an attacker, rather than wrapping generic LLMs. That matters for enterprise-grade results.”

Tal Shapira
PhD, CTO

“This was by far the deepest and fastest security assessment we’ve had. Novee uncovered issues across our web and mobile applications that had gone undetected before, and the level of depth was unlike anything we’d seen from other vendors.”

Amir Tito
CISO

“We had urgent compliance need and we couldn’t wait weeks for DAST findings, an external exposure audit, and an in-depth pentest report. Instead Novee came in and delivered immediate value with their AI pentesting platform; with their findings, we closed our gaps and quickly met the criteria we needed for certification.”

Ron Reiter
CTO

"Before Novee, we were getting a snapshot once a year. Now we have continuous coverage across our application portfolio, we're already finding things that prior manual pentests missed completely, and I have real confidence that our security posture reflects what's actually in our environment."

Abhijeet Patkar
Cyber Security Manager

The Novee Advantages

Full-Application Coverage

The Problem with Promptfoo:

Promptfoo tests the model layer, looking for issues with prompt injection and jailbreaks, and it does that job well. Everything wrapped around the model layer, such as the web and API surface, the authentication, and the business logic an attacker actually goes after, sits outside what Promptfoo looks at. A hardened model and a hardened product are two different things.

How Novee Goes Further:

Novee tests the whole application the way an attacker does, black box, from a domain name, across web apps, APIs, mobile apps, and whatever AI features are baked in. The model is just one of many surfaces it probes. Because it builds a picture of the full product first, it can follow an attacker's path from the front door all the way through to the data, rather than poking at one layer in isolation.

Finding Business Logic Flaws

The Problem with Promptfoo:

Promptfoo evaluates how a model responds to prompts. The vulnerabilities that actually get companies breached rarely sit in a single model response. They live in business logic and broken authorization, in the gap between one workflow step and the next, and a model-testing framework has no way to reach them.

How Novee Finds Them:

Novee starts by building an Asset Intelligence Model of your application, a working picture of its roles, permissions, workflows, and the business rules beneath them. Knowing how the app is meant to behave is what lets Novee catch where it misbehaves, whether that's an account that can read another tenant's data, a checkout step that can be skipped, or a small bug that turns serious once it's chained to two others. It reasons about the system rather than matching it against a checklist, and that model deepens every cycle as your product changes.

Proof and a Verified Fix

The Problem with Promptfoo:

Promptfoo hands you model-behavior flags rather than proven exploits. Nothing independently validates them to rule out false positives, there's no proof-of-concept run against the live application, and no remediation or retest follows. Taking a flag all the way to a confirmed, fixed vulnerability, and being sure the fix actually held, lands entirely on your team.

How Novee Proves and Verifies:

Every Novee finding passes through three independent agents plus deterministic checks, and it comes with a working exploit, replication steps, and a PoC, all proven against the live application rather than guessed at from model output. The remediation that comes with it is written for your specific WAF, backend, and codebase. Once a fix ships, Novee retests on its own to confirm the issue is closed and to flag any new risk the change introduced.