NVIDIA Omniverse NuRec is a neural reconstruction pipeline for constructing high-fidelity 3D representations of real-world environments from multisensor information equivalent to cameras and lidar. It’s used to reconstruct dynamic scenes captured by autonomous automobile (AV) and robotics platforms into simulation-ready digital environments that may be rendered, replayed, and analyzed inside NVIDIA Omniverse and associated simulation workflows.
These reconstructions play a crucial function within the growth of bodily AI and autonomous programs. Engineers can seize a real-world driving or robotics situation, reconstruct the surroundings, after which examine or replay the scene. This permits them to higher perceive mannequin habits, validate notion outcomes, generate artificial viewpoints, or create coaching information for downstream machine studying workflows.
NuRec combines neural rendering methods equivalent to Gaussian splatting with GPU-accelerated rendering and simulation pipelines to provide extremely real looking scene reconstructions. Nonetheless, this degree of constancy comes with vital computational price. Reconstruction and rendering workloads contain giant volumes of sensor information, advanced PyTorch-based coaching loops, and extremely specialised CUDA kernels that push GPU sources closely.
This submit walks via an instance to showcase the right way to optimize the NuRec neural reconstruction pipeline utilizing NVIDIA Nsight Developer Instruments.
Fixing efficiency optimization challenges
Efficiency is crucial for NuRec workflows as a result of reconstruction turnaround time instantly impacts engineering productiveness. A typical workflow includes figuring out an attention-grabbing or problematic AV run—for instance, a situation the place the notion or planning stack behaved unexpectedly—and launching a reconstruction so engineers can examine the scene as shortly as doable. Ready a number of hours for reconstruction slows iteration and debugging velocity considerably.
In the beginning of this optimization effort, reconstructing even comparatively brief captures may take from over an hour to a number of hours relying on the scene and configuration. The group’s long-term purpose is rather more bold: real-time reconstruction efficiency, the place a 30-second seize will be reconstructed in roughly 30 seconds.
Efficiency additionally issues past reconstruction itself. As soon as scenes have been reconstructed, rendering-only workflows might generate large numbers of frames for reinforcement studying (RL), artificial information technology (SDG), and large-scale simulation. At this scale, even modest efficiency enhancements can translate instantly into substantial reductions in GPU time and infrastructure price.
To sort out these challenges, NVIDIA profiling and optimization instruments had been used, primarily NVIDIA Nsight Methods and NVIDIA Nsight Compute, to investigate the NuRec workload, determine bottlenecks throughout the software program stack, and iteratively optimize each the application-level workflow and the underlying CUDA kernels.
Profiling and optimization utilizing Nsight Methods
Nsight Methods is a platform profiling instrument that will help you visualize and perceive the efficiency habits and useful resource utilization of workloads, together with CPU, GPU, storage, networking, and extra. Step one in lots of efficiency optimization workflows is to run an Nsight Methods profile to ascertain a baseline and attempt to determine some preliminary bottlenecks or key areas for enchancment.
With the purpose of optimizing the coaching loop, we used the Nsight Methods built-in operate help and NVIDIA Instruments Extension SDK (NVTX) included in PyTorch to zoom right into a single iteration of the ahead move proven in Determine 1. The preliminary assumption was that the rendering kernel would take many of the runtime and can be the most effective start line for optimization. Nonetheless, the CUDA HW timeline on the high revealed that almost all of time the GPU was underutilized or not used in any respect. Discover the shortage of blue on the highest row. The applying was additionally utilizing many extra tiny kernels than was anticipated.
After this preliminary realization, it was vital to drill deeper into the phases of the ahead move to determine the place time was being spent and what phases had been underutilizing the GPU. Extra NVTX annotations had been added to the code to delineate numerous phases and features. A brand new profile (Determine 2) confirmed that collect_gaussian_parameters was taking nearly all of the time earlier than rendering even began and is known as a number of instances in every ahead move.
Digging even deeper revealed the interpolate operate taking the plurality of the time (4.148 ms) and calling many small kernels and reminiscence operations that slowed down the GPU, as seen within the backside CUDA API row in Determine 3.
We dug into the code below the interpolate operate and targeted on fusing the small kernels and submitting bigger chunks of labor to the GPU. We had been capable of condense all of this work right into a single kernel that diminished the interpolate operate from 4.184 ms to 83.81 us (Determine 4). That is almost a 50x speedup.
Subsequent we recognized lengthy cudaStreamSynchronize APIs (seen as inexperienced bars on the timeline) that had been delaying the CPU from enqueuing many small kernels whereas the GPU was lively. This resulted in patchy GPU utilization proven within the high CUDA HW row after the synchronize API returned because the small kernels had been scheduled and launched (Determine 5).
After eradicating one synchronization level, others down the road would turn out to be the bottleneck. This course of was continued till sufficient had been eliminated that the CPU may effectively enqueue work whereas the GPU was busy. This allowed the tiny kernels to run compactly as a result of they had been not CPU launch-time certain.
Lowering the time spent accumulating the parameters and eradicating synchronization factors that had been inflicting bottlenecks enabled digging into some kernel optimizations. Nsight Methods allows you to determine which kernels are the most well liked. The renderBackward kernel was clearly the highest candidate on this case.
Kernel optimization utilizing Nsight Compute
Nsight Compute is the most effective instrument for profiling and optimizing particular person kernels. It will probably robotically replay kernels to gather giant quantities of efficiency information at very fantastic granularities utilizing numerous sorts of {hardware} counters, software program patching, and instrumentation. It features a built-in rule system and guided evaluation to assist customers determine and perceive points.
The renderBackward kernel is utilized in each digicam and lidar information processing. Profiling a number of cases of this kernel with Nsight Compute revealed that it has solely ~15% occupancy and the habits and useful resource necessities of this kernel differ considerably relying on which of those inputs is being processed.
The longest three renderBackward kernels are from lidar information and the opposite three are from digicam information. Regardless of these variations, each had been allocating 167 registers per thread (Determine 8).
Setting the highest lidar kernel as an Nsight Compute baseline and evaluating a digicam kernel robotically revealed that whereas each had the overwhelming majority of accesses in shared reminiscence, the digicam kernels had been making ~75% fewer requests despite the fact that each lidar and digicam cases of the kernel had been allocating the identical quantity of shared reminiscence per block statically.
Noting these habits variations between whether or not the renderBackward kernel was used for digicam or lidar information, and the truth that register and shared reminiscence allocations had been static and equivalent for each, the following step was to attempt splitting the kernel relying on whether or not it was processing digicam or lidar information.
For every model of the kernel, the group experimented and tuned register allocations with the launch_bounds qualifier and the quantity of shared reminiscence we had been allocating per block. The cudaFuncSetCacheConfig runtime API was used to set the choice of each kernels to have a bigger shared reminiscence and smaller L1 cache.
After this testing and optimization, the lidar and digicam kernels decreased their register allocation wants from 167 to 64 and 128 respectively and each had been capable of run effectively with about half of the initially allotted shared reminiscence. This improved occupancy from ~15% to between 30-50% and general runtime considerably, with the longest lidar kernel reducing from 31 ms to 18 ms.
However there may be nonetheless room for enchancment. The following situation recognized, which is being labored on on the time of publication, is long-tail results within the kernel brought on by a workload imbalance. This may be seen within the PM Sampling part of Nsight Compute (Determine 11). The primary half of the kernel reveals a median of 32 lively warps that start to taper off and for the final a number of milliseconds there may be lower than one lively warp per cycle. Ideally, all of the warps can be lively for the whole thing of the kernel.
Efficiency evaluation and optimization is an iterative course of that consists of operating a profile, figuring out an issue, fixing it, and beginning once more. You should utilize instruments like Nsight Methods and Nsight Compute to make this whole course of simpler for creating and optimizing on NVIDIA GPUs. Each instruments are free—obtain Nsight Methods and Nsight Compute and check out them with your personal use case. In case you have questions or need to share what you discover, depart a touch upon the NVIDIA Developer Boards.
Acknowledgments
Particular because of NVIDIA contributors Francois Trudel, Joey Lai, and Rodolfo Lima.
