As extra groups transfer from humanoid robotic bring-up to task-specific talent growth, the necessity for repeatable growth workflows is rising. Constructing humanoids stays complicated, and right now’s growth pipelines are nonetheless extremely fragmented. Consequently, builders spend important time configuring robotics infrastructure earlier than they will give attention to constructing robotic capabilities.
Whereas the robotics neighborhood has developed succesful instruments throughout particular person levels of the pipeline, connecting them right into a seamless workflow stays a problem on account of siloed software program ecosystems, incompatible information codecs, and guide integrations between instruments. To deal with this, we introduce the NVIDIA Isaac GR00T Improvement Platform: an open supply humanoid robotic platform that unifies each stage of the pipeline for sooner growth.
This publish covers the end-to-end GR00T platform, from information assortment and mannequin coaching to large-scale analysis and deployment readiness, showcasing how a completely built-in platform can streamline humanoid growth.
Speed up end-to-end humanoid growth
The Isaac GR00T platform helps robotics builders scale humanoid information assortment and simulation-based coaching to develop, validate, and deploy insurance policies on actual robots. Its end-to-end platform is totally open and modular, enabling groups to make use of particular person parts or the whole pipeline, combine their very own tooling, and construct utilizing the validated NVIDIA software program stack.
A key element of the GR00T workflow is the open Isaac GR00T 1.7 vision-language-action (VLA) mannequin, which streamlines growth by way of a robust pretrained basis. Skilled on numerous information, it will possibly effectively generalize to new duties and environments whereas producing extra pure, human-like movement.
Get began with GR00T 1.7
GR00T 1.7 is the primary open, commercially usable VLA mannequin for generalized humanoid robotic expertise, launched beneath the permissive Apache 2.0 license. It’s a cross-embodiment mannequin that takes multimodal enter, together with language and pictures, and produces the actions wanted to carry out duties throughout numerous environments. As an alternative of coaching a coverage from scratch, builders can begin from a mannequin that already encodes broad manipulation priors and adapt it to particular robots, duties, and environments by way of post-training.
What’s new in GR00T 1.7
Sturdy Human Video Pretraining: 1.7 is pretrained on ~32K hours of actual demonstration and human ego-centric information in addition to ~8K hours of simulated rollouts and demonstrations from BEHAVIOR, RoboCasa, and Simulated GR-1 for extra human-like motion.
New VLM spine: Cosmos-Reason2-2B (Qwen3-VL structure) replaces the Eagle spine utilized in GR00T N1.6. This helps help versatile decision and encodes photos of their native facet ratio with out padding.
Expanded deployment help: Provides full pipeline export to ONNX and TensorRT, with improved export reliability and higher-frequency updates.
Enhanced efficiency: Improved long-horizon job reasoning by way of task- and subtask-level decomposition, rising reliability, movement high quality, and cross-embodiment generalization.
Improved benchmarks: Constant enhancements throughout DROID and SimplerEnv in comparison with N1.6, together with DROID-F0 (+10%) and DROID-F6 (+61%), in addition to SimplerEnv Bridge (+5%) and Fractal (+2%), demonstrating stronger generalization.
GR00T 1.7 is accessible by way of GitHub and Hugging Face, with mannequin weights publicly out there and a base checkpoint of three billion parameters.
Utilizing the GR00T 1.7 mannequin within the end-to-end GR00T workflow streamlines post-training and coverage analysis, simplifying the transition from basis mannequin to deployed robotic habits. The next part particulars the steps for utilizing GR00T 1.7 within the simulation workflow.
Utilizing GR00T platform and 1.7 mannequin for a dexterous manipulation job
This part walks by way of a simulation workflow for constructing a easy pick-and-place job utilizing the GR00T platform and the GR00T 1.7 mannequin, from surroundings setup to deployment readiness. That is an excerpt from our end-to-end GR00T growth information, which covers the total course of in additional element.
The walkthrough consists of:
Establishing an surroundings in simulation
Gathering robotic information by way of teleoperation
Changing collected information into LeRobot format
Publish-training GR00T 1.7 with the transformed dataset
Evaluating the post-trained coverage
Though this part focuses on simulation, the identical steps will be utilized to bodily robots and tailored throughout completely different embodiments and duties.
Arrange an surroundings
Earlier than accumulating any coaching information, you need to first arrange the surroundings. Compose the scene with object and robotic belongings, outline the duty, and select the teleoperation system.
On this instance (see Determine 3, above) , the robotic stands in entrance of a shelf and makes use of its arms to maneuver an apple onto a plate on the identical shelf. A Complete Physique Controller (WBC) retains the robotic balanced; for the reason that robotic doesn’t stroll throughout this job, use AGILE WBC, a single end-to-end coverage that matches a static job higher than a stand/stroll controller.
Select the WBC intentionally up entrance: throughout teleoperation, the AGILE WBC and PinkIK produce the joint-space targets that change into the coverage’s coaching sign, so a unique controller yields a unique coaching distribution for a similar movement.
At a excessive stage the implementation seems like the next code snippet. These snippets use the Isaac Lab-Area APIs to compose the scene, register belongings, and wire up the teleoperation system. These are simply snippets; for your complete surroundings set-up, go to GitLab Pages.
pick_up_object = self.asset_registry.get_asset_by_name(args_cli.object)()
vacation spot = self.asset_registry.get_asset_by_name(args_cli.vacation spot)()
embodiment = self.asset_registry.get_asset_by_name(args_cli.embodiment)(
enable_cameras=args_cli.enable_cameras
)
teleop_device = (
self.device_registry.get_device_by_name(args_cli.teleop_device)()
if args_cli.teleop_device shouldn’t be None else None
)
scene = Scene(belongings=[background, pick_up_object, destination])
job = PickAndPlaceTask(
pick_up_object=pick_up_object,
destination_location=vacation spot,
background_scene=background,
)
return IsaacLabArenaEnvironment(
title=self.title,
embodiment=embodiment,
scene=scene,
job=job,
teleop_device=teleop_device,
)
Acquire demonstrations
When you’ve arrange your surroundings, you should use Isaac Teleop to seize demonstration information, with AGILE WBC dealing with whole-body management throughout assortment. Demonstrations are collected utilizing a supported VR headset over a CloudXR shopper. After beginning the CloudXR runtime and sourcing the CloudXR surroundings, document demonstrations with:
–viz package
–device cpu
–enable_cameras
–dataset_file $DATASET_DIR/arena_g1_static_apple_dataset_recorded.hdf5
–num_demos 400
–num_success_steps 10
–disable_full_sim_buffer_reset
galileo_g1_static_pick_and_place
–object apple_01_objaverse_robolab
–destination clay_plates_hot3d_robolab
–teleop_device openxr
High quality issues as a lot as amount right here. Purpose for clear demonstrations with easy motions, secure grasps, and numerous strategy instructions, since noisy or inconsistent information limits how effectively the coverage can be taught. For this job, we collected 400 trajectories not unexpectedly however stitched collectively by way of a number of classes of information assortment. You’ll be able to undoubtedly begin with a smaller batch of information to grasp how the items match collectively earlier than committing to a full set of demonstrations.
Profitable demonstrations are saved to an HDF5 file, which turns into the uncooked enter for the following stage, the place it’s transformed to the LeRobot format that GR00T 1.7 consumes for post-training.
Convert information format
GR00T 1.7 consumes datasets in LeRobot format, so the recorded HDF5 is transformed contained in the Area container earlier than coaching. The conversion is pushed by a config file, g1_static_apple_config.yaml, which maps the recorded fields to what GR00T expects:
data_root: /datasets/isaaclab_arena/static_apple_tutorial
hdf5_name: “arena_g1_static_apple_dataset_recorded.hdf5”
# Activity description
language_instruction: “transfer the apple to the plate”
task_index: 3
# Knowledge discipline mappings
state_name_sim: “robot_joint_pos”
action_name_sim: “processed_actions”
pov_cam_name_sim: “robot_head_cam_rgb”
# Output configuration
fps: 50
chunks_size: 1000
Run the conversion with:
–yaml_file isaaclab_arena_gr00t/lerobot/config/g1_static_apple_config.yaml
This produces a lerobot folder containing parquet recordsdata with states and actions, MP4 digital camera recordings, and dataset metadata. Now with a transformed dataset you possibly can post-train GR00T 1.7 mannequin.
Publish-train GR00T 1.7
Publish-training runs exterior the Area container, in a standalone checkout of the Isaac-GR00T repo. The fine-tuning command tunes the visible spine, projector, and diffusion mannequin whereas preserving the language mannequin frozen:
gr00t/experiment/launch_finetune.py
–base-model-path nvidia/GR00T-N1.7-3B
–dataset-path $DATASET_DIR/arena_g1_static_apple_dataset_recorded/lerobot
–output-dir $MODELS_DIR/static_apple_n17_finetune
–modality-config-path /path/to/IsaacLab-Area/isaaclab_arena_gr00t/embodiments/g1/g1_sim_wbc_data_gr00t_n_1_7_config.py
–embodiment-tag new_embodiment
–global-batch-size 12
–max-steps 20000
–num-gpus 1
–save-steps 5000
–save-total-limit 5
–no-tune-llm
–tune-visual
–tune-projector
–tune-diffusion-model
–dataloader-num-workers 8
–color-jitter-params brightness 0.3 distinction 0.4 saturation 0.5 hue 0.08
Consider the coverage
After post-training, analysis is simply two steps: load your checkpoint right into a GR00T server, then run the coverage in a closed loop. Area makes use of a server-client setup, the place the server hosts your fine-tuned mannequin and the Area shopper runs the simulation and queries the mannequin from the server.
Run the coverage by launching the shopper from the Area container:
–viz package
–policy_type isaaclab_arena_gr00t.coverage.gr00t_remote_closedloop_policy.Gr00tRemoteClosedloopPolicy
–policy_config_yaml_path isaaclab_arena_gr00t/coverage/config/g1_static_apple_gr00t_closedloop_config.yaml
–remote_host –remote_port 5555
–num_steps 600
–enable_cameras
galileo_g1_static_pick_and_place
–object apple_01_objaverse_robolab
–destination clay_plates_hot3d_robolab
–embodiment g1_wbc_agile_joint
Metrics print to the console as analysis is in progress till you get a closing output that tells you the efficiency of your post-trained coverage.
To vary how a lot you consider, alter a couple of flags. –num_steps units the rollout size; 600 is a fast smoke check equivalent to about one episode for this job. For a consultant success charge, consider full episodes as an alternative with –num_episodes 100 (or 1000 for a stronger estimate), and add –num_envs 5 to run environments in parallel for extra randomization throughout trials. All the pieces else stays the identical; you’re solely altering what number of rollouts you common over.
Streamlined growth course of
That completes a full cross by way of the Isaac GR00T Improvement Platform. Every stage produces a clear artifact that feeds instantly into the following for a completely built-in, streamlined growth course of. Whereas this walkthrough makes use of a particular embodiment for example, the identical course of will be tailored to your individual robotic setup, outline a job to your use case, accumulate demonstrations in your system, and reuse the identical information conversion, post-training, and analysis workflow.
A rising GR00T ecosystem
The GR00T platform and reference workflow are already being adopted throughout a rising ecosystem of humanoid robotics companions.
Humanoid makers and AI suppliers together with 1X, Agility, ANYBotics, Bellboy Robotics, FieldAI, Lightwheel AI, NEURA Robotics, Nexuni, Noble Machines, Schaeffler, Skild AI, and Techman Robotic are integrating parts of the GR00T platform reminiscent of Isaac Teleop, Sim, Lab, and ROS to speed up their growth pipeline and produce AI-enabled robots into actual industrial use sooner.
Analysis establishments together with Stanford, CMU, UCSD, ETH Zurich, and AI2 can be experimenting with the unified GR00T end-to-end workflow to scale back integration complexity and transfer sooner from robotic bring-up to talent growth and real-world validation.
Main wearable and XR system makers together with Haptikos, Manus, Noitom, OpenGraph Labs, PICO, Sensing, HTC Vive, Wuji, and Xsens provide gadgets with native help for NVIDIA Isaac Teleop, making it simpler to seize high-quality demonstrations.
Getting began
The GR00T platform and GR00T 1.7 mannequin can be found right now for any developer to make use of. Whether or not beginning in simulation or on a bodily robotic, the workflow stays the identical. Choose an entry level, outline a bounded job, and construct your first end-to-end coverage.
Getting began will be difficult, so we’ve got constructed an entire end-to-end reference workflow, which is supplied within the studying content material The way to Develop and Deploy Humanoid Robots Finish-to-Finish with NVIDIA Isaac GR00T, together with the GR00T Platform Introduction Video.
Further assets:
