AI is altering the tempo of cybersecurity. Agentic methods can coordinate work and pursue complicated goals over lengthy horizons. Safety groups are starting to use brokers throughout safety operations, however many implementations stay anchored to present alerts, predefined workflows, and identified assault behaviors. The tougher drawback is figuring out what defenses miss and turning these gaps into dependable protection. This requires repeatedly testing adaptive assaults in a corporation’s distinctive setting and validating candidate detections in opposition to regular enterprise exercise.
Steady offense-defense testing creates this suggestions loop. Managed assaults produce the telemetry and floor fact defensive brokers want to reveal gaps, enhance protection, and retest. Nonetheless, the end-to-end cycle nonetheless requires important handbook effort. May purple and blue brokers powered by open fashions and specialised harnesses function this loop at machine velocity and scale?
NVIDIA and CrowdStrike evaluated an agentic attack-defense system in an remoted setting modeled on NVIDIA accelerated computing infrastructure. On the defensive facet, NVIDIA Nemotron fashions custom-made for cybersecurity function inside CrowdStrike SafeMind, its agentic cybersecurity system. CrowdStrike studies that its Blue Solano defensive mannequin is extra correct than the main proprietary frontier mannequin examined, at 99% decrease value, in CrowdStrike inside evaluations. For this analysis, the optimized open-model configuration paired NVIDIA Nemotron 3 Extremely for defensive orchestration with a fine-tuned Nemotron 3 Tremendous for detection era.
This publish shares how Nemotron fashions and specialised agent harnesses work along with CrowdStrike’s agentic system, the way it was deployed and evaluated, and the way the ensuing detections carried out throughout independently seeded assault runs.
Turning offense and protection right into a steady studying loop
Conventional red-and-blue group workout routines rely on handbook handoffs: the purple group executes an assault, the blue group critiques the ensuing telemetry, detection engineers develop or replace detections, and the purple group retests them. Every handoff takes time, limiting the variety of iterations and assault variations groups can consider. An offensive-defensive agentic system connects these actions right into a repeatable closed loop that may function at machine velocity. Inside an remoted consultant setting, every run produces assault traces and sensor telemetry used to create or refine detections. The ensuing context returns to the red-agent harness, which adapts and checks different assault or evasion paths till it finds no additional viable route throughout the consultant setting.
The workflow has 4 related levels:
Execute and seize. Beginning with a threat-informed goal, the red-agent harness chosen and executed an assault path contained in the consultant setting. Its motion hint recorded every step, whereas CrowdStrike Falcon endpoint sensors captured the corresponding telemetry.
Course of and reconstruct. The blue-agent harness then obtained the motion hint, sensor telemetry, and broader assault context. Utilizing details about the out there information sources and CrowdStrike detection-engineering experience as grounding context, it decided which elements of the occasion sequence might be reconstructed, which present detections triggered, and the place visibility or detection gaps remained.
Generate and validate. Primarily based on this evaluation, the blue-agent harness generated candidate detections. The validation harness checked every candidate, backtested it in opposition to the captured telemetry, returned failures for correction, and despatched validated detections to the detection engine.
Retest, adapt, and repeat. After a validated detection was deployed, an independently seeded assault retested the identical goal. Detection and alert context returned to the red-agent harness, which tailored and explored different assault or evasion paths, producing new traces and telemetry for the blue-agent harness.
Every cycle is designed to strengthen defensive protection whereas forcing the offense to discover a tougher path with full information of the protection. The loop continues till no additional viable path stays throughout the modeled setting.
Constructing a consultant check setting
To allow secure, sensible testing, NVIDIA offered a sanitized, natural-language specification representing its accelerated computing infrastructure. An agent-assisted workflow translated the specification into an remoted goal cyber agent setting instrumented with Falcon platform sensors.
The analysis used assault paths and observable milestones to measure progress from motion traces and sensor telemetry fairly than the agent’s personal claims. NVIDIA safety specialists reviewed the setting and risk paths for realism. The identical reviewed setting supported each assault run, detection check, and analysis metric, enabling constant comparisons throughout configurations.
Specializing the defensive harness
The following problem was turning the ensuing assault traces and telemetry into detection guidelines that might go technical and behavioral validation. Open agent harnesses already present capabilities reminiscent of planning, device use, context administration, and iterative correction. The Open Safe AI Alliance helps develop the broader ecosystem of open fashions, harnesses, and instruments for cybersecurity.
The defensive harness mixed six mechanisms:
Schema information base. A device allowed brokers to enumerate supported Falcon sensor schemas, fields, and question syntax, stopping invented fields and invalid queries.
Telemetry grounding. Crimson-agent traces, Falcon telemetry, and broader assault context anchored the workflow in noticed occasions and relationships, lowering unsupported or hallucinated connections.
Specialised detection authoring. A custom-made Nemotron 3 Tremendous served as a bounded professional for producing and repairing detections, separating this specialised activity from the longer orchestration context.
Artifact linting. Automated checks rejected syntax errors, unsupported fields, and detections tied to particular IP addresses, hosts, customers, or subnets. Failures returned steerage for rewriting the detection round behavioral indicators as a substitute of environment-specific strings.
Detection replay. Every candidate was replayed in opposition to captured assault telemetry. Detections that produced no match had been rejected and returned for correction, catching candidates that appeared legitimate however didn’t detect the recorded exercise.
Impartial overview. A separate choose with a recent context evaluated every detection for behavioral alignment, robustness, and acceptable use of a number of indicators, catching high quality gaps missed by linting and replay.
Failed checks returned structured suggestions to the blue-agent workflow for correction and one other try. Collectively, these mechanisms encoded practices usually utilized by means of handbook detection-engineering overview, making detection era grounded, testable, and correctable fairly than merely believable.
Customizing Nemotron for defensive orchestration and detection era
Open fashions reminiscent of Nemotron will be post-trained with area information, deployed with proprietary context in managed environments, and optimized for value and scale. Within the evaluated open-model configuration, Nemotron 3 Extremely reconstructed assault sequences, deliberate detection-engineering steps, and invoked instruments. When a detection required writing or restore, a custom-made Nemotron 3 Tremendous served as a bounded professional. This separation saved workflow orchestration distinct from specialised detection authoring.
To create this specialised professional, CrowdStrike used Nemotron 3 Tremendous as the bottom for its NL2LogScale mannequin, then utilized continuous pretraining on cybersecurity information, supervised fine-tuning, and reinforcement studying with verifiable rewards. Advantageous-tuning used 9,349 detection-generation and multistep restore examples spanning 59 programmatically generated error sorts. The coaching information mixed request rephrasings from Nemotron 3 Tremendous, actual Falcon LogScale execution errors, and quality-reviewed reasoning traces from Nemotron 3 Extremely.
For reinforcement studying, the coaching workflow used NVIDIA NeMo Gymnasium to validate and execute generated queries in Falcon LogScale. Invalid queries obtained actual engine errors and as much as 5 restore makes an attempt, with unresolved makes an attempt receiving zero reward. Legitimate generated and reference queries ran in opposition to the identical artificial logs, and the F1 overlap between their returned occasions provided the reward. NVIDIA NeMo RL supported group relative coverage optimization to replace the mannequin. For reinforcement studying, the workflow used NeMo Gymnasium to validate generated queries and NeMo RL to replace the mannequin primarily based on verifiable outcomes.
Evaluating the whole agent system from backtest to stay fireplace
A detection could detect the recorded assault used to create it, however fail on a brand new execution of the identical conduct. Subsequently, we evaluated every system in two levels: backtesting in opposition to the recorded assault, adopted by live-fire testing in opposition to eight new assaults from the identical situation household. Each passing detection was deployed verbatim to the stay detection engine. An unbiased third-party mannequin judged whether or not every match represented the meant assault conduct, and outcomes had been averaged throughout independently seeded authoring periods.
Backtesting in opposition to the recorded assault
With Nemotron 3 Extremely and the default harness, a median of 16.5% of generated detections detected the recorded assault throughout eight independently seeded periods. Retaining Extremely whereas including the tuned harness, custom-made Nemotron 3 Tremendous, area context, instruments, and validation raised the imply to 41.9% throughout six periods, a 2.5x enchancment. As a result of the optimized configuration modified each the harness and mannequin stack, the acquire displays the whole open pipeline fairly than an remoted mannequin ablation.
Evaluating generalization by means of live-fire testing
Throughout live-fire testing, 11 backtest-passing detections from the optimized open pipeline powered by Nemotron and 35 from the whole frontier system had been deployed in opposition to eight unseen assaults. 5 of the 11 open detections (45%) detected not less than one assault, in contrast with 10 frontier detections (29%). The open pipeline averaged 2.6 detections per detection, in contrast with 1.1 for the frontier system.
Detection alone, nevertheless, was solely the primary high quality gate. To qualify as gold, detections additionally needed to stay quiet on out there check site visitors and go an unbiased overview for behavioral grounding, a number of indicators, and no environment-specific strings. 4 of the 5 firing open detections and 9 of 10 frontier detections that fired stayed quiet, overlaying eight of eight assaults and 7 of eight, respectively. After overview, three open detections and no frontier detections certified as gold. The three open detections nonetheless lined all eight assaults.
Deciphering the outcomes
The frontier system produced extra backtest-passing detections, however the optimized Nemotron open-model pipeline yielded the next share that generalized, averaged extra detections per detection, and was the one system to supply gold detections. The analysis lined one situation household and small detection units, so cross-scenario generalization stays untested. Restricted benign site visitors additionally means the noise check doesn’t symbolize manufacturing false-positive efficiency. Three of eight live-fire runs skilled harness failures however produced full telemetry and had been retained. The findings are a directional system-level case examine, not a normal benchmark.
Making use of the sample to specialised brokers
The analysis exhibits a broader sample for constructing specialised brokers with NVIDIA Nemotron: a reasoning mannequin orchestrates the defensive workflow, a post-trained open mannequin handles a bounded professional activity, and an agent harness manages context, instruments, and validation. 4 design rules emerge:
Outline measurable duties and assign clear duties to every mannequin.
Publish-train specialised fashions with area information and verifiable rewards.
Floor outputs in authoritative context and validate them by means of deterministic checks, sensible replay, and unbiased overview.
Consider the whole workflow in sensible circumstances, with safety specialists controlling situations, guardrails, and use of validated outputs.
CrowdStrike is advancing this strategy by means of SafeMind, its agentic cybersecurity system, which brings offensive and defensive AI collectively in a steady coevolution loop. Discover NVIDIA Nemotron 3, NeMo Megatron Bridge, NeMo Gymnasium, and NeMo RL to customise and consider specialised brokers for different bounded domains.
