Customization is what permits builders to take a common mannequin and tailor it to make use of instances, domains, languages, and extra.
Nonetheless, customization comes with a number of challenges. It requires infrastructure, technical experience, and software program particular to the workflow, in addition to sources resembling GPUs and the flexibility to make use of them successfully. It additionally is dependent upon specialised area data: What algorithm ought to I take advantage of? What atmosphere ought to I take advantage of? How do I do know whether or not the mannequin really discovered?
Customization can seem daunting.
With open fashions such because the NVIDIA Nemotron 3 household—together with Nano, Tremendous, and Extremely—mixed with platforms resembling Prime Mind Lab that supply coaching as a service, the whole customization loop is rather more accessible. On this tutorial, you’ll use Prime Mind Lab to customise NVIDIA Nemotron 3 Nano with hosted reinforcement studying and produce a downloadable LoRA adapter.
The native setup takes about 5 minutes—that’s all.
By the tip of this weblog, you’ll study two issues:
Customization is extra accessible than ever.
Getting began with open fashions is simpler than ever.
What you’ll construct
We’re going to comply with a easy move for this tutorial: Baseline, practice, and reevaluate—all utilizing Nemotron 3 Nano—and leading to a custom-made mannequin for our job. To indicate how simple it’s to get began with the sturdy customization instruments we’ve at our disposal, we’ll be utilizing a normal “hi there world” instance: Python Math.
This tutorial applies to Nemotron 3 Tremendous and Nemotron 3 Extremely. The video may even stroll you thru the extra concerned coding job that’s a part of a companion pocket book to this tutorial:
What customization means
Customization means taking a mannequin and altering or adapting its trainable parameters in order that it turns into extra prone to do the sorts of duties you need it to do efficiently.
There are numerous methods to do that. Relying on the issue, you would possibly use supervised fine-tuning, parameter-efficient fine-tuning, desire optimization, reinforcement studying, or a mixture of strategies. For a broader remedy, see the Deciding on Giant Language Mannequin Customization Strategies publish.
This tutorial focuses on reinforcement studying with verifiable rewards (RLVR) on the Python Math atmosphere. The fundamental concept of the atmosphere is straightforward: Use Python instruments (Python, numpy, sympy and scipy) to do some math. Right here’s an instance of a immediate:
Use Python for all calculations. Give your reply inside boxed{}.
Along with the Python commonplace library, you’ve entry to: numpy sympy scipy.
# Consumer:
If $2^8=4^x$, what’s the worth of $x$?
> NOTE: The atmosphere has a default of 100 turns. For this experiment, we cap every rollout at 5 assistant turns. A direct response is one flip, and so a software name with the ultimate response is 2 of the 5 turns. This penalizes the mannequin closely for repeatedly calling instruments with out producing a last response.
For a deeper introduction to reinforcement studying for brokers, see the publish Mastering Agentic Strategies: AI Agent Reinforcement Studying.
Why openness issues
Open weights are an essential a part of the equation, however they aren’t all the equation. Knowledge, analysis code, and coaching recipes additionally decide what you may perceive, reproduce, and deploy.
The NVIDIA Nemotron 3 household is revealed with open mannequin sources, together with weights, knowledge, and recipes. You possibly can discover the NVIDIA Nemotron developer sources, learn Inside NVIDIA Nemotron 3, and examine the NVIDIA Nemotron repository.
These sources give a greater view into how Nemotron was constructed and post-trained, the varieties of knowledge and environments that had been used, and the place your personal customization ought to differ.
Stipulations
For this tutorial, you’ll want:
A improvement atmosphere with curl and a supported Python 3 set up.
A Prime Mind account with entry to Hosted Coaching and billing configured.
Web entry for the Prime CLI, atmosphere package deal, hosted mannequin, and adapter deployment.
About 5-Quarter-hour.
Prime Mind handles the hosted rollout, coaching, and inference infrastructure; you don’t must handle an area GPU cluster.
Notice: Mannequin availability and pricing change over time. Deal with the reside CLI catalog because the supply of fact relatively than copying an identifier or worth from a static article.
Customise Nemotron 3 Nano in three steps
Let’s get into the straightforward steps required to customise your very personal Nemotron 3 Nano.
Step 1: Get a Baseline
First, set up the Prime CLI, authenticate, and create a Prime Mind Lab workspace:
uv python set up 3.13
uv software set up –python 3.13 –force “prime==0.6.17”
uv –version
prime –version
prime login
mkdir -p nemotron-customization
cd nemotron-customization
prime lab setup
uv run python –version
Prime Mind Lab creates the workspace construction and installs the verifiers tooling. The longer-running analysis and coaching steps come subsequent.
Examine the atmosphere earlier than utilizing it:
prime env examine primeintellect/math-python@0.1.10 README.md
Pin the precise atmosphere package deal within the workspace:
–index https://hub.primeintellect.ai/primeintellect/easy/
uv lock –check
Now examine the reside hosted coaching catalog:
prime inference fashions
Checkpoint: Proceed provided that the output comprises nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 and reveals the present coaching, enter, and output costs.
Use the next command to seize a fast baseline on our job so we are able to present if the mannequin improved after coaching:
export MATH_EVAL_ARGS='{“dataset_name”:”math”,”dataset_split”:”check”,”num_train_examples”:-1,”max_turns”:5,”sandbox_client_max_workers”:8}’
export MATH_SAMPLING='{“max_tokens”:2048,”temperature”:1.0,”extra_body”:{“chat_template_kwargs”:{“enable_thinking”:false}}}’
prime eval run math-python
–provider prime
–model “$NANO”
–num-examples 32
–rollouts-per-example 1
–max-concurrent 8
–env-args “$MATH_EVAL_ARGS”
–sampling-args “$MATH_SAMPLING”
–timeout 300
–max-retries 2
–save-results
–disable-tui
–abbreviated-summary
Your outcomes ought to look much like the next, which showcases that the mannequin begins off doing relatively poorly and has a reasonably low accuracy.
— All —
Rewards:
reward: avg – 0.219, std – 0.413…
That is an instance of a failing immediate for our baseline mannequin, wherein the mannequin fails to supply a boxed response as a result of it exhausted 5 turns calling unsupported instruments:
Use Python for all calculations. Give your reply inside boxed{}.
Along with the Python commonplace library, you’ve entry to: numpy sympy scipy.
# Consumer:
If $2^8=4^x$, what’s the worth of $x$?
# Assistant:
[[EMPTY RESPONSE]]
Determine 1 is an instance of our reward distribution on this preliminary analysis. The reward is binary: 1 means the mannequin produced an accurate response, and 0 means the mannequin produced an incorrect or lacking response.
Step 2: Take Nemotron 3 Nano to high school
To get began on customizing Nemotron 3 Nano, the very first thing we’ll do is create configs/rl/nemotron-3-nano-math-python-100step.toml:
mannequin = “nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16”
loss = “rl”
max_steps = 100
batch_size = 32
rollouts_per_example = 8
max_inflight_rollouts = 32
learning_rate = 2e-5
lora_alpha = 16
[sampling]
max_tokens = 2048
temperature = 1.0
enable_thinking = false
[adapters]
interval = 0
keep_last = 1
[[env]]
id = “primeintellect/math-python@0.1.10”
args = { dataset_name = “math”, dataset_split = “practice”, num_train_examples = -1, max_turns = 5 }
This config outlines how we need to practice our mannequin. The coach samples eight makes an attempt for every chosen job, giving it a comparative sign inside every rollout group. 4 teams kind a batch of 32, and caps the variety of rollouts being assembled concurrently for this small run.
Launch the run:
Prime prints the parsed configuration, present worth, and atmosphere motion standing earlier than asking for affirmation. After you affirm, the CLI returns a run ID and a dashboard URL. Analysis and coaching proceed after that time.
Checkpoint: Save the run ID. The launch output ought to present the precise Nano mannequin, the pinned Math Python atmosphere, reside pricing, and a profitable atmosphere motion examine.
The dashboard reveals reward curves, reward distributions, and particular person rollouts. The CLI exposes the identical proof:
export STEP=””
prime practice logs “$RUN_ID” –follow
prime practice progress “$RUN_ID”
prime practice distributions “$RUN_ID” –type rewards
prime practice utilization “$RUN_ID” –watch
prime practice rollouts “$RUN_ID” –step “$STEP”
prime practice checkpoints “$RUN_ID”
When experimenting, start with small variations. If each try receives the identical rating, the coach has little sign to work with. Subsequent, examine a high-, middle-, and low-reward rollout and ask whether or not the rating displays genuinely higher math problem-solving. Lastly, have a look at components like token use and truncation to make sure the configuration shouldn’t be interfering with the coaching sign.
If the reward curve tells us if the mannequin bought kind of reward; the rollouts assist let you know why.
Step 3: Test how a lot Nemotron 3 Nano discovered
Coaching reward is like measuring the mannequin’s efficiency on homework. It’s not actual till you give it a check.
When the run completes efficiently, Prime Mind Lab uploads the ultimate LoRA adapter. Verify that the coaching artifact is prepared, after which examine its deployment state:
A prepared adapter that isn’t at present serving can seem as NOT_DEPLOYED. After deployment is requested, its serving state strikes by DEPLOYING to DEPLOYED.
Checkpoint: Proceed solely after the ultimate adapter exists and is able to deploy. A accomplished coach course of and not using a usable adapter isn’t the artifact this tutorial guarantees.
You possibly can obtain the adapter from the coaching dashboard. To question it by hosted inference, deploy it:
prime practice checkpoints “$RUN_ID” –status READY
export ADAPTER_ID=””
export ADAPTED_MODEL=”$NANO:$ADAPTER_ID”
prime deployments create “$ADAPTER_ID” –plain
COLUMNS=300 NO_COLOR=1 prime deployments checklist –plain
Deployment creates a billable inference endpoint. As soon as its standing is DEPLOYED, the mannequin identifier combines Nano with the adapter ID:
Now repeat the baseline command, altering solely the mannequin identifier
Preserve the atmosphere, job rely, rollout rely, sampling settings, and reward unchanged. Altering greater than the mannequin makes the comparability tougher to interpret.
prime eval run math-python
–provider prime
–model “$ADAPTED_MODEL”
–num-examples 32
–rollouts-per-example 1
–max-concurrent 8
–env-args “$MATH_EVAL_ARGS”
–sampling-args “$MATH_SAMPLING”
–timeout 300
–max-retries 2
–save-results
–disable-tui
–abbreviated-summary
Your outcomes ought to comprise the next, which showcases the identical analysis configuration used above—exhibiting a rise within the common reward as in comparison with our baseline mannequin on the identical instance set—this outcome reveals that our mannequin has discovered to be meaningfully higher at our job:
— All —
Rewards:
reward: avg – 0.906, std – 0.291
…
Right here’s the identical immediate the preliminary mannequin was unable to appropriately resolve:
Use Python for all calculations. Give your reply inside boxed{}.
Along with the Python commonplace library, you’ve entry to: numpy sympy scipy.
# Consumer:
If $2^8=4^x$, what’s the worth of $x$?
# Assistant Device Flip:
sp.resolve(sp.Eq(2**8, 4**x), x)
…# Assistant:
We are able to rewrite the equation utilizing the identical base.
[
2^8 = 4^x quadLongrightarrowquad 2^8 = (2^2)^x = 2^{2x}.
]
For the reason that bases are equal, the exponents should be equal:
[
8 = 2x ;Longrightarrow; x = frac{8}{2}=4.
]
[
boxed{4}
]
Determine 3 reveals the reward distribution on the identical set of analysis prompts we noticed earlier than, and as you may see, the mannequin can get a considerably greater variety of them appropriate:
On the identical 32 held-out issues, accuracy elevated from 21.9% (7/32) to 90.6% (29/32), an absolute improve of 68.75 share factors. Of the 24 prompts that had totally different outcomes between baseline and last, 23 improved, and one regressed. Moreover, Nemotron 3 Nano bought higher at this job with a complete spend of lower than $5.
To take away the adapter deployment, you may difficulty the next command as cleanup:
See the Prime Mind information to deploying LoRA adapters for inference for deployment states and OpenAI-compatible inference examples.
Coaching bigger fashions with Prime Mind Lab:
Now that we’ve gone by an instance of how we are able to use Prime Mind Lab to run a brief coaching of Nemotron 3 Nano, you might be asking your self: “What if I wished to coach bigger fashions? Is it a lot tougher?” The reply is: by no means. With the comfort of hosted coaching, we are able to merely modify the mannequin identifier and run the very same workflow utilizing the next:
Prepare Nemotron 3 Tremendous
Run the reside catalog examine once more:
Verify availability and present pricing, then change the mannequin line with:
Preserve the identical experimental self-discipline: baseline first, examine reward variation, practice, and retest below unchanged settings.
Prepare Nemotron 3 Extremely
Nemotron 3 Extremely is the high-capacity possibility within the household, and Prime Mind has assist for hosted reinforcement studying on NVIDIA Blackwell infrastructure.
Confirm you’ve entry to managed coaching of Extremely utilizing the next command:
In the event you see it, change the mannequin line with:
Start customizing Nemotron 3
Customization is find out how to custom-tailor AI for our use instances. With out customization, we’re restricted to no matter conduct a mannequin has out of the field, and we could overpay for a extra broadly succesful mannequin when a lower-cost mannequin may very well be tailored to the duty we’d like.
Open fashions resembling NVIDIA Nemotron 3, mixed with hosted coaching platforms resembling Prime Mind Lab, make that course of a lot simpler to begin. Essentially the most priceless half isn’t merely that the weights can be found or that the infrastructure is managed. It’s that all the enchancment loop could be inspected and repeated.
Go to the Nemotron developer web page for sources to get began. Discover open Nemotron fashions and datasets on Hugging Face and Blueprints on construct.nvidia.com.
Interact with Nemotron livestreams, tutorials, and the developer group on the NVIDIA discussion board and the Nemotron channel on Discord.
Keep updated on NVIDIA Nemotron by subscribing to NVIDIA information and following NVIDIA AI on LinkedIn, X, Discord, and YouTube.
