Elite AI hackers aren’t born. They’re trained.
UiPath CISO Scott Roberts, IDC senior analyst Katie Norton, and Novee co-founder and CEO Ido Geffen on why continuous, validated offensive security is becoming the standard, and what still holds the market back.
In our fireside chat, Stay Ahead of Cyber AI Attacks with Continuous AI Pentesting: A Practitioner’s Guide, we hear three unique perspectives on the future of offensive security testing. The main question: what happens to security testing when both attackers and defenders operate with AI?
Moderated by Novee’s Head of AI Dan Padnos, the conversation hosted IDC senior analyst Katie Norton, UiPath CISO and AIUC-1 founding member Scott Roberts, and Novee co-founder and CEO Ido Geffen.
Watch the full conversation below.
Here are the highlights:
AI has blown open the gap between how fast software ships and how fast it’s secured
- Development is accelerating in a way most security programs weren’t built for. As Scott Roberts described it, “Even though we’ve created mandatory rules for how coding agents should handle these areas, we see them often ignoring those paved roads. All of a sudden, a coding agent decides to invent a new way to manage secrets in your environment.”
- Roberts: “We’re seeing a time dilation of time-to-exploit. The patch cycle is moving from weeks, to days, to hours. Defending at machine speed is one of the catchphrases we’re using.”
- Ido Geffen: “The main ceiling we’re seeing today is speed. Software is being developed much, much faster, and we need to find a way to keep pace.”
- Roberts framed the response around three tenets:
- (1) compress the patch window
- (2) design for breach
- (3) defend at machine speed.
- If attacker-grade techniques will soon live in open-weight and open-source models, defenders need efficient access to the same capabilities.
- This is the same conviction behind AIUC-1, the agent-security certification Scott Roberts helped found to pressure-test whether AI agents’ guardrails actually hold under thousands of adversarial prompts.
“We are seeing a velocity of PRs from our development team like never before.”
Scott Roberts, CISO, UiPath
Point-in-time penetration testing is breaking down, and continuous, validated offensive security is what replaces it
- Roberts: “At UiPath, we’ve had over 500 releases in the past 12 months… It’s quite common for most organizations to do annual pen tests, maybe twice a year, but we’re doing many releases per week, and it’s accelerating. This gives us the assurance that continuous testing is always happening.”
- Katie Norton put IDC’s market data behind the same point: the agentic operating model for writing code is already fairly well-adopted, but security’s ability to validate that output hasn’t kept pace. The result is an exposure gap and a timing gap, a widening space between when risk is introduced and when anyone confirms it’s real.
“500 releases in 12 months simply doesn’t square with an annual pentest. Continuous deployment demands continuous pentesting.”
Scott Roberts, CISO, UiPath
Where traditional testing hits its ceiling: business logic, authorization gaps, and chained attack paths
- Scanners are good at pattern-matching known vulnerability classes, but they can’t reason about how an application is supposed to work… which is exactly where the most damaging risk hides.
- Geffen’s point was that closing this gap requires a proactive approach: finding novel vulnerabilities the way an attacker would, reconciling the disconnect between what the business actually prioritizes and what a scanner happens to flag, and then fixing as fast as possible.
- Roberts: “In many ways, UiPath is a Swiss Army knife; we have dozens of products, and customers can put them together in novel ways. What we’ve loved about the Novee relationship is that Novee was able to come in and actually understand the UiPath development platform, and help accelerate even my internal team in finding novel vulnerabilities that are provable in our environment.”
“The Holy Grail is the ability to find business logic vulnerabilities. That’s where you really need to understand what the application is actually meant to do.”
Ido Geffen, CEO, Novee Security
The UiPath story: from annual pentests to continuous validation
- UiPath’s attack surface grows every quarter by design: web apps, desktop apps, and a fast-expanding set of AI and agentic components. Continuous deployment made the annual test untenable.
- There’s a full team of skilled manual pentesters inside UiPath, Roberts noted, but a team like that needs a platform that lets it scale.
- Geffen: “It’s not only the model. The biggest thing is the full system with the harness; keeping context from the past, the feedback loops on what’s really important, what was categorized as a false positive, what’s not applicable for a specific application or organization.”
- Automating the discovery of complex business logic flaws, historically some of the hardest issues to catch, is a key capability.
“Novee adapted to our multi-tenant SaaS product within days and now gives us ongoing validation that our tenant isolation holds up against real attacker techniques. That level of assurance is what our customers expect of us, and it’s exactly what we give them through this partnership.”
Scott Roberts, CISO, UiPath
The economics mean you can’t outspend the attacker
- Defending with someone else’s frontier model, and someone else’s pricing curve, is a losing bet at enterprise scale.
- Geffen: “It’s really hard to track which model is best for which task — you can be willing to pay a lot of money and still not be using the right model for the right task. What we’re seeing is that our system keeps improving: better quality, but also much more cost-effective.”
- Norton framed the shift buyers need to make: as agents move from occasional assistance to always-on execution, token-based pricing gets out of control, and the right question stops being cost-per-tool. \
- Katie Norton: “Think less about how much AI capability you’re receiving, and more about what that consumption is actually producing. In offensive security, that could mean the cost per validated finding, how much manual validation work was eliminated, or how quickly a confirmed issue reaches the right team.”
- This is the logic behind Novee’s own model — coverage priced to the complexity of an environment, not the number of runs or findings — and the reason continuous testing doesn’t have to cost more than episodic testing when it’s measured correctly.
“I have a colleague that used Mythos on a single scan of one of their repositories, and it cost over $25,000 in token usage — just for the one scan of the one repository. That doesn’t scale to doing it continuously, let alone across the 800-plus repositories we have, as most large enterprises do.”
Scott Roberts, CISO, UiPath
What’s next: The shift from detection to remediation
Asked for one prediction that would look obvious a year from now, all three pointed past detection.
- Roberts: “We’re not just handing our developers vulnerabilities to go fix — we’re part of the remediation process. We have products that create proposed PRs for our development team, and ‘safe images’ where the CVEs are already addressed in the base image.”
- Geffen: “Remediation is the biggest area we need to start taking seriously. And there’s real demand to move beyond penetration testing into something broader — red-teaming, chaining findings into full attack and kill chains across other layers and controls.”
- Norton: “The strongest economics won’t come from which vendor uses the cheapest model. We have to look for platforms that apply more expensive reasoning selectively, avoid wasting compute on lower-value activity, and show a measurable security outcome. Organizations want outcomes, and they want to measure them.”
IDC’s data shows roughly 51% of security professionals still want agents limited to human-in-the-loop decisions, and only about 2% support full autonomy today. The move from point-in-time to continuous validation is a real shift, in other words, but a still-early one, gated more by trust and technology maturity than by whether the capability exists.
The full conversation goes deeper on the UiPath story, the economics of frontier models, and where offensive security goes next:
And review the details shared in the presentation for yourself:
See what continuous, validated pentesting looks like in your environment. Book a demo.
