The demand for high-quality video continues to speed up throughout industries, powering every little thing from immersive streaming experiences to distant collaboration, generative AI media instruments, and large-scale content material supply.
Behind these experiences is a rising want for video pipelines which might be sooner, extra environment friendly, and able to dealing with more and more advanced codecs and workloads. NVIDIA Video Codec SDK helps builders meet that problem by offering entry to GPU-accelerated video encoding and decoding by means of NVIDIA devoted {hardware} video engines.
NVIDIA Video Codec SDK 13.1 is now accessible. We encourage builders to discover the most recent options of their video pipelines, make the most of the redesigned pattern apps, and share suggestions by way of the NVIDIA Developer boards.
What’s new in SDK 13.1
Encode options:
· Hierarchical Reference Mode for AV1 with as much as 31 B-frames
· UHQ tuning data mixed with iterative encoding
Decode options:
· Per-macroblock decode statistics for H.264 and HEVC
· View data throughout MV-HEVC decode
· Search to a particular body
Transcode options:
· Software-allocated CUarray as NVIDIA Video Encoder (NVENC) enter and NVIDIA Video Decoder (NVDEC) output
· Redesigned transcoder samples with a modular, queue-based structure
Different options:
· Official Docker-based improvement atmosphere
Encode enhancements
Why AV1 hierarchical reference mode
Utilizing B-frames as references improves encode high quality. Hierarchical Reference Mode improves it additional by arranging B-frames in a tree-like reference construction: leaf nodes are non-reference B-frames, and the foundation is the center B-frame. This construction will increase NVENC’s most variety of B-frames from 7 to 31, enabling the encoder to use temporal redundancy and enhance total high quality. Use of this mode doesn’t add any efficiency penalty, however video reminiscence consumption goes increased.
SDK 13.1 provides Hierarchical Reference Mode for AV1, supporting 1, 3, 7, 15, and 31 B-frames; H.264 and HEVC will comply with in future driver variations. The mode is best at seven or extra B-frames and has minimal efficiency impression. For configuration particulars, see the NVENC Programming Information.
Determine 2, under, reveals bitrate financial savings when encoding 15 B-frames in Fixed High quality (CQ) mode, in contrast with NVENC’s Excessive High quality (HQ) tuning in preset p7.
Determine 3, under, reveals bitrate financial savings when encoding 15 B-frames in Variable Bitrate (VBR).
UHQ tuning data with iterative encoding
Iterative encoding (launched in Video Codec SDK 12.1) freezes the encoder’s computerized state development and lets the person re-encode the identical body with totally different parameters. NVENC tracks every iteration’s state, and may cease and decide to any certainly one of them.
UHQ tuning data (launched in Video Codec SDK 12.2) combines lookahead degree and temporal filtering to ship the most effective quality-vs-performance tradeoffs in latency-tolerant encoding. Temporal filtering reduces noise in pure video through the use of movement estimation to seek out matching patches in adjoining frames and making use of them to filter the present body, yielding common coding beneficial properties of 4–5% on pure content material. The lookahead degree function analyzes future frames and makes use of coding tree items (CTUs) and different statistics to allocate bits effectively for fee management. 4 lookahead ranges with totally different efficiency and high quality tradeoffs are at present supported..
Model 13.1 combines UHQ tuning data with iterative encoding, so lookahead degree and temporal filtering now work alongside per-iteration re-encode.
Decode enhancements
Per-macroblock decode statistics
The NVDECODE API now retrieves detailed per-macroblock decode statistics for every decoded body of H.264 and H.265 (HEVC) content material. For each 16×16 block, the decoder exposes the luma quantization parameter (QP), the coding unit sort (Intra, Inter, Skip, or PCM), and as much as two movement vectors (ahead and backward), all extracted as a pure byproduct of {hardware} decode with no extra CPU overhead.
These statistics unlock GPU-accelerated video-analytics workflows that beforehand required CPU-side bitstream parsing. Movement vectors allow scene-change detection, object monitoring, and shot-boundary evaluation. QP values give a per-block view of encoding high quality for adaptive bitrate optimization and high quality monitoring. Macroblock varieties reveal coding construction helpful for content material classification and compression analysis.
The workflow is simple: purposes question decoder capabilities by way of cuvidGetDecoderCaps(), allow statistics assortment at decoder creation, and retrieve a GPU-resident statistics buffer with every decoded body from cuvidMapVideoFrame(). The person can copy the statistics to host reminiscence or course of them straight on the GPU with CUDA kernels for real-time pipelines. The SDK ships a ready-to-use pattern, AppDec -dumpstats, that demonstrates the complete circulate.
Body-accurate search
AI workflows—from inference pipelines for object detection, content material moderation, and video summarization, to training-data preparation that samples various frames throughout giant datasets—continuously want particular frames reasonably than sequential decode.
Video modifying and non-linear post-production have the identical requirement. The Video Codec SDK now gives a complete in search of and random-frame-access API by means of the NvVideoDecoder class, making frame-accurate entry so simple as array indexing whereas fetching solely the frames you want.
A GOP-aware search structure handles the request. For body N, the SDK locates the closest IDR body earlier than the goal, seeks the demuxer there, and flushes decoder state by way of CUVID_PKT_DISCONTINUITY.
From that IDR ahead, solely frames wanted to achieve N are processed: the parser flags and skips non-reference frames, and reference frames decode however bypass mapping and post-processing (format conversion, scaling, cropping by way of cuvidMapVideoFrame()) by means of PTS-based filtering. Solely body N runs the complete decode, mapping, and post-processing pipeline.
The NvVideoDecoder class wraps the low-level SeekUtils engine and exposes a clear, operator-based interface. Determine 5, under, reveals the inner search circulate when a body is requested.
Key capabilities:
Decoder caching for playlists: NvVideoDecoder caches decoder cases by codec, bit depth, and chroma format, and reuses them by way of LRU eviction to keep away from repeated creation overhead
Non-seekable streams: Elementary streams, community streams, and pipes are auto-detected. Ahead in search of works effectively; backward in search of is supported with an computerized decoder reset
Versatile body specification: The AppDecVideoDecoder pattern helps particular person indices (0,10,20), ranges with step (0:100:10), time-based entry (-t 1.5,3.0), and playlist recordsdata for batch processing
Open GOP assist: Streams with non-IDR I-frames search to the closest IDR reasonably than the closest keyframe, guaranteeing all reference frames can be found
MV-HEVC stereoscopic enhancements
The Video Codec SDK now affords higher 3D-video assist: view-ID and layer metadata from the decoder; multi-GPU encoding at excessive resolutions; and improved compatibility with third-party software program.
MV-HEVC decode updates:
View-information reporting: The decoder outputs particular layer and reference data akin to nuh_layer_id, letting purposes establish and route frames by view for stereo dealing with
Wider bitstream assist: The decoder now handles MV-HEVC 3D bitstreams created by third-party encoders, so each left and proper views play appropriately
MV-HEVC encode updates:
Simplified metadata for FFmpeg: HEVC 3D show metadata now seems appropriately within the bitstream when encoding by way of FFmpeg, making 3D-video creation simpler
Cut up Body Encoding (SFE): A number of encoders can work on a single body, boosting pace for high-resolution 3D and XR video that exceeds a single encoder’s capability
Redesigned transcoder pipeline
The transcoding samples now favor flexibility, efficiency, and customization. The up to date suite ships 4 purposes:
AppTransPerf. Benchmarks the utmost throughput of NVDEC and NVENC
AppTrans. 1:1 transcoding with elective bit-depth conversion
AppTransOneToN. 1:N transcoding with scaling
AppTransZeroCopy (new). 1:1 zero-copy pure-transcoding software optimized for the bottom doable latency
Modular, queue-based structure
The earlier implementation was quick however monolithic: the person needed to perceive your entire pipeline earlier than modifying it. Model 13.1 redesigns the samples round a strictly modular, queue-based structure that ensures concurrency, simplifies customization, and maximizes {hardware} utilization.
The redesign allocates a devoted CPU thread per pipeline stage, working as a producer-consumer system the place threads talk by means of explicitly sized enter and output queues. The core AppTrans pipeline (Determine 6, under) breaks into 4 remoted execution contexts: a Decode Thread (NVDEC) handles demux and decode; a Compute Thread (CUDA) handles processing; an Encode Thread (NVENC) handles encode; and an Output Thread gathers output and muxes.
Every thread isolates one step, producing a decoupled design that handles synchronization natively. Frames go by means of the queues sequentially, guaranteeing protected information circulate.
Key benefits of the redesign
Modularity: Copy and tweak solely the pipeline steps you want. Decoupled elements maintain errors and exceptions remoted inside their threads
Efficiency: Full concurrency between pipeline steps is assured. Separating the CPU submission threads ensures the GPU is rarely starved; as soon as saturated, the {hardware} engines (NVDEC, CUDA, NVENC) run their levels in parallel on totally different frames
Customizability: Decoupled queues offer you whole management. Want encoding solely? Take away the decode thread and feed the compute and encode queues straight. Need ultra-low latency over throughput? Scale back queue sizes to reduce buffer wait. Have a customized AI filter? Modify the Compute Thread to dispatch your CUDA kernels in isolation, with out blocking the decoder or encoder submission loops
Zero-copy transcode with CUarray
In a standard transcoding pipeline (as in AppTrans), the decoded body passes by means of a number of inside format conversions and copies earlier than reaching the encoder. These are inherent to the usual NvDecoder and NvEncoder APIs. The decoder output converts into an application-accessible floor; the applying copies it into the encoder’s enter buffer; the encoder converts that enter once more into its required format.
AppTransZeroCopy eliminates this copy chain by having NVDEC and NVENC function straight on the identical GPU reminiscence in a format each engines perceive natively. The mechanism has 4 components:
Shared buffer pool allocation: At startup, the applying allocates a pool of CUDA arrays (CUarray) utilizing cuArray3DCreate with the CUDA_ARRAY3D_VIDEO_ENCODE_DECODE flag, which tells the CUDA driver these surfaces shall be shared between each video-codec engines. Every CUarray holds one body (luma plus chroma planes) in a format each NVDEC and NVENC entry natively, bypassing the normal pipeline’s intermediate conversions.
Twin registration: The identical CUarray register with each codecs. On the decoder aspect, they’re supplied as exterior output surfaces by way of SetExternalOutputArrays(), telling NVDEC to jot down decoded frames straight into them. On the encoder aspect, they register as enter sources by way of NVENC’s nvEncRegisterResource API with useful resource sort NV_ENC_INPUT_RESOURCE_TYPE_CUDAARRAY, letting the encoder learn them straight with out enter conversion.
Pipelined execution: Three threads—decode, encode, and output—join by means of concurrent queues with token-based circulate management to handle CUarray possession between decoder and encoder.
Stream-ordered synchronization: NVDEC and NVENC share the identical CUDA stream, guaranteeing right ordering between decode writes and encode reads with out express CPU-GPU synchronization.
Key benefits
Decrease SM utilization: The standard pipeline makes use of a number of CUDA copy and conversion kernels on Streaming Multiprocessors to shuttle frames between levels. Zero-copy eliminates these intermediate kernels, liberating SM sources for CUDA preprocessing, inference, or rendering
Decreased GPU reminiscence footprint: The standard pipeline retains separate buffers at every stage. Zero-copy collapses these right into a single shared pool, considerably decreasing per-session reminiscence consumption
Increased throughput with concurrent classes: Decrease SM utilization plus a smaller reminiscence footprint enhance scalability. The GPU can maintain extra concurrent transcode classes earlier than hitting SM saturation or reminiscence exhaustion
Determine 8, under, reveals decrease SM utilization within the new AppTransZeroCopy pattern software in comparison with legacy AppTransPerf pattern software.
Determine 9, under, reveals decrease video reminiscence bandwidth utilization within the new AppTransZeroCopy pattern software in comparison with legacy AppTransPerf pattern software.
Docker improvement atmosphere
Organising the Video Codec SDK historically requires putting in the CUDA toolkit, Vulkan SDK, system libraries, and FFmpeg, then constructing the SDK samples—typically throughout totally different host distributions and driver variations. Video Codec SDK 13.1 introduces an official Docker-based improvement atmosphere that packages a constant, pre-configured stack right into a single container. The picture builds from an open Dockerfile, so you possibly can reproduce the atmosphere domestically or within the cloud and customise it by way of construct arguments.
The picture is inbuilt two levels: a builder stage compiles the SDK samples and installs the Vulkan SDK and FFmpeg; the runtime stage retains solely what is required to run and develop. The stack pins CUDA 12.3.2, Vulkan SDK 1.4.304.1, and Ubuntu 22.04 LTS.
Use the SDK_ZIP construct argument to level at Video_Codec_SDK_13.1.x.zip, and the elective FFMPEG_URL argument to produce a customized FFmpeg tarball (for instance, an LGPL construct with NVENC) as an alternative of the default BtbN LGPL construct.
Contained in the container, pre-built samples dwell in /video-codec-sdk/Samples/construct/, together with AppDec and AppEncCuda, able to run with take a look at vectors. FFmpeg (LGPL) is put in at /choose/ffmpeg for YUV era, MJPEG and MPEG encoding, and inspection of encoded streams.
A test-vector script produces uncooked YUV in a number of codecs, JPEG, MPEG-1/2/4, and — with GPU entry — H.264 and HEVC utilizing AppEncCuda, so you possibly can train the complete pipeline. The container runs as a non-root person and features a HEALTHCHECK for orchestrators.
Stipulations
an NVIDIA GPU with video encode/decode assist
Docker with GPU assist enabled
the NVIDIA Container Toolkit, and
the SDK package deal file
Place the SDK zip (for instance, Video_Codec_SDK_13.1.x.zip) within the Docker construct context and construct:
docker construct -t nvidia/video-codec-sdk:13.1-ubuntu22.04
–build-arg SDK_ZIP=Video_Codec_SDK_13.1.x.zip
.
Launch with GPU entry. You possibly can generate take a look at vectors at startup or open a shell and run samples straight:
docker run –gpus all -it nvidia/video-codec-sdk:13.1-ubuntu22.04
# Generate the complete test-vector suite throughout startup (~10-15 min)
docker run –gpus all -it nvidia/video-codec-sdk:13.1-ubuntu22.04
–generate-vectors full
# Generate solely H.264 vectors at 720p
docker run –gpus all -it nvidia/video-codec-sdk:13.1-ubuntu22.04
–generate-vectors h264 –resolution 1280×720
Technology modes are full, h264, hevc, vp8, vp9, and av1. Contained in the container, the aliases sdk-samples, test-decode, and test-encode bounce to the samples listing and run fast assessments with the generated vectors.
Key benefits
Reproducibility: The identical CUDA, Vulkan, FFmpeg, and SDK variations run all over the place, decreasing “works on my machine” drift
Quick onboarding: Clone the repository, add the SDK zip, run docker construct and docker run –gpus all; no host-side SDK or Vulkan set up required
CI- and cloud-friendly: A single picture drives pipelines and cloud workloads wherever the NVIDIA Container Toolkit and GPU assist can be found
The Dockerfile and helper scripts dwell within the video-codec-sdk-docker repository. For detailed construct choices, atmosphere variables, and troubleshooting, see the repository README.
Get began with Video Codec SDK 13.1
Obtain the SDK, attempt the brand new encode, decode, and transcode options in your pipelines, and share how they give you the results you want.
The redesigned samples make it simple to drop the brand new options into present workflows or construct a customized pipeline from scratch.
· Obtain Video Codec SDK 13.1
· NVENC Programming Information
· NVDEC Programming Information
· Video-codec-sdk-docker on GitHub
· NVIDIA Developer boards: Video Codec SDK
