GPU-accelerated question engines are sometimes constrained by reminiscence and I/O bandwidth. NVIDIA {hardware} advances—together with excessive bandwidth reminiscence (HBM), NVIDIA NVLink-C2C, and devoted decompression engines featured in NVIDIA GB200 NVL4—assist take away these bottlenecks by rising efficient storage capability, accelerating knowledge motion between CPUs and GPUs, and rushing knowledge entry with out consuming streaming multiprocessor (SM) sources.
On this publish, we present how databases can use these applied sciences to speed up GPU question execution. You’ll study methods for environment friendly CPU-GPU knowledge motion, compression, partition pruning, and overlapping knowledge switch with computation.
Structure overview of GQE
GQE (GPU Question Engine) is a reference structure designed to execute SQL queries at excessive efficiency over giant knowledge units on fashionable NVIDIA {hardware}. Underneath the hood, GQE makes use of NVIDIA cuDF and different NVIDIA CUDA-X libraries, together with CCCL, nvCOMP, and nvSHMEM.
GQE can assist affect question engines to:
Transfer execution to GPUs.
Transfer decompression to nvCOMP.
Make knowledge codecs GPU-friendly.
Shut end-to-end efficiency gaps when working on GPUs.
In Determine 1, we give an outline of the system design by breaking down GQE into a question, knowledge, and execution layer. These handle the transition from a SQL question and enter knowledge to hardware-level execution. The layers match collectively as follows.
The question layer enhances the execution engine with a SQL parser and a question optimizer. The question layer natively accepts Substrait plans, an open-source question plan format, for execution in GQE. Substrait makes it doable to judge the advantages of GPU execution by exporting question plans from an present database product and working the plan in GQE. In Determine 2, Apache DataFusion transforms a SQL string right into a Substrait plan. GQE consumes that plan as an optimized logical question plan, provides GQE-specific refinements, and transforms the question right into a bodily plan.
The info layer shops and organizes person knowledge for quick entry by the executor. In GQE, storage is abstracted into pluggable, specialised readers that deal with totally different knowledge codecs and storage mediums—it at present helps GPU reminiscence, CPU reminiscence, and disk. On this publish, we deal with the high-performance GQE in-memory desk format and assume this knowledge is saved in CPU reminiscence. GQE transfers knowledge chunks to the GPU on-demand to saturate the GPU with work with out storing the complete dataset in GPU reminiscence. When a piece arrives on the GPU, the information layer palms off to the execution layer.
The execution layer executes the bodily question plan in opposition to the information to provide question outcomes. GQE generates the bodily plan right into a job graph, which defines the execution schedule. The duty graph accommodates relational operators constructed on the open-source NVIDIA cuDF library, which implements the operators in extremely optimized CUDA C++ code. As a result of the information layer transfers in chunks, GQE can decompose operators and execute duties on these chunks concurrently as pipelined CUDA streams.
In abstract, GQE unlocks the excessive throughput of the {hardware} by a GPU-native design.
Information structure and switch orchestration
The GQE knowledge layer is optimized to effectively switch knowledge from host reminiscence to machine reminiscence. We reduce knowledge switch latency by maximizing throughput and lowering the quantity of information moved. Within the following, we give an outline of our in-memory knowledge structure and the host-to-device switch orchestration, that are instrumental to minimizing switch latency.
GQE design objectives
As GQE builds on cuDF, the design assumes that in-GPU knowledge is structured as cuDF-native tables. Nonetheless, the host reminiscence structure can optimize transfers for NVIDIA NVLink C2C and PCIe. cudaMemcpy is the usual switch methodology. On this method, the CPU orchestrates GPU execution and copies knowledge in a bulk switch. This additionally varieties the premise for compressed transfers.
Information structure
Determine 2 exhibits the desk knowledge structure, which is horizontally subdivided into row teams. Every row group consists of columns and encapsulates metadata. Inside a row group, GQE shops columns as non-contiguous partitions. Throughout a switch, the storage layer converts a set of partitions right into a cuDF column. Thus, the information layer hides the implementation particulars of compression and partition pruning from the execution layer.
Switch Orchestration
In Determine 3, we present how the CPU orchestrates a switch. Following finest CUDA apply, transfers use pipeline parallelism to effectively make the most of {hardware} parts. A pipelined switch consists of a number of phases. In compressed, partitioned knowledge, there are 4 phases.
In Stage 0, a number thread performs scheduling. Scheduling entails computing the reminiscence vary to switch, allocating a vacation spot buffer, and invoking the required CUDA strategies.
In Stage 1, the GPU performs the H2D switch.
Stage 2 decompresses the information.
Stage 3, added outdoors the information layer, wherein the CUDA kernels compute the question.
These 4 phases ought to overlap. Ideally, the question runtime equals the longest-running stage, and all remaining phases are hidden by the pipeline.
Information switch optimizations
Quick knowledge entry performs a major position within the efficiency benefit achieved by GQE. The primary knowledge entry optimizations employed are compression and partition pruning. Within the following, we describe how these optimizations work.
Compression
GQE receives two predominant advantages from compression: question dataset capability and question acceleration. Compression allows a question engine to develop the dataset dimension that may be processed utilizing a given reminiscence allotment by lowering the general in-memory footprint. Information switch of compressed buffers, mixed with quick decompression by the GPU, hastens transfers even on quick interconnects like NVLink C2C. GQE compresses the datasets with GPU-optimized codecs that enhance compression ratios and supply superior GPU decompression speeds in comparison with utilizing legacy codecs.
NVIDIA nvCOMP library
NVIDIA nvCOMP is a library for GPU-accelerated compression and decompression. It gives a spread of normal and GPU-optimized compression codecs. The person can decide from the supported algorithms to stability compression ratio, compression, and decompression throughput. nvCOMP can wrap CPU libraries equivalent to lz4hc inside its high-level interface, offering extra configuration choices. GQE makes use of nvCOMP for its compression and decompression routines.
NVIDIA Blackwell Decompression Engine
NVIDIA launched a brand new Decompression Engine (DE) within the NVIDIA Blackwell structure that allows nvCOMP to rapidly decompress LZ77-based codecs like LZ4, Snappy, and Deflate with out utilizing SM sources. Decompression with DE, SM kernels, and CE copies can totally overlap when utilizing a number of CUDA streams.
DE on a single NVIDIA Blackwell B200 GPU can attain as much as 400 GB/s in database purposes. For instance, at a 4x compression ratio, it achieves roughly 400 GB/s efficient host-to-device throughput whereas leaving 100 GB/s C2C host-to-device bandwidth obtainable. The remaining bandwidth will be harnessed to switch different knowledge, together with encoded knowledge that’s decompressed on the SMs.
NVIDIA GQE’s compression method
Determine 4 exhibits the hybrid compression method, which makes use of light-weight algorithms, equivalent to Cascaded, to make use of particular patterns within the structured knowledge the place doable, and the DE when LZ-based algorithms are wanted to attain good compression ratios.
When contemplating tips on how to compress a given column, a question engine has a couple of choices. It might require customers to specify an algorithm for every column, however that is unwieldy for very giant databases. The method we’ve taken is to try each LZ4 and Cascaded. LZ4 is our alternative for generic knowledge as a result of it achieves excessive ratios in comparison with different LZ77-only compressors, and is supported by the Decompression Engine.
To find out the compression algorithm to make use of, we compress the information utilizing each LZ4 and Cascaded algorithms. Cascaded can obtain extraordinarily quick compression charges, at roughly 500 GB/s on B200. This permits us to strive the additional algorithm with out important overhead within the knowledge loading stage.
We stability when to make use of Cascaded vs LZ4 utilizing two heuristics:
Cascaded and LZ4 have totally different compression ratio thresholds, which set up minimums for us to make use of that algorithm.
Cascaded should obtain the next compression ratio than LZ4 to be chosen over LZ4. The set off is a configurable a number of of the LZ4 compression ratio.
We use the selection of algorithm to assist stability C2C bandwidth, DE, and SM sources.
Partition pruning
Earlier than transferring knowledge from the CPU to the GPU, GQE employs filter pruning to skip partitions that don’t contribute to the question end result. This mechanism depends on metadata summarizing the desk contents and the predicates outlined within the SQL question.
Metadata and storage
GQE makes use of zone maps to assist filter pruning. When knowledge is loaded as in-memory tables, GQE horizontally splits the desk into row teams and fixed-size partitions with a default of 10M rows. For every partition, GQE computes the minimal and most values for each column and shops this metadata as cuDF tables in GPU reminiscence, so pruning can run with out changing into a bottleneck. Computing the zone maps provides about 1% to the preliminary Parquet load time and occurs solely as soon as, not throughout question execution.
Pruning and job orchestration
Determine 5 exhibits the filter pruning course of. Throughout job graph development, GQE derives a pruning expression by reworking question predicates into comparisons in opposition to the row teams’ zone maps. Partitions that may’t contribute to the question end result are pruned. On this instance, partition 1 is pruned as a result of the zone map signifies that every one values saved on this partition are lower than 9, and due to this fact additionally lower than 15, the decrease sure. The remaining partitions are transferred to GPU reminiscence and decompressed if essential. Even when partitions are discontiguous in CPU reminiscence after pruning, e.g., as a result of they’re contained in a number of row teams, they’re transferred and assembled right into a contiguous reminiscence block wrapped as a cuDF desk on the GPU.
Filter pruning in GQE is extremely efficient. Within the TPC-H benchmark utilizing the 1 TB scale dataset, filter pruning skips 31% of information throughout all 22 queries. The influence is an end-to-end speedup of 1.43×.
The analysis of zone maps provides minimal overhead, on common, 2.2 ms for benchmark queries on 1 TB of information.
Information switch optimizations
In GQE, we conceive a novel batched switch optimization for partitions.
A number of partitions are transferred to the GPU in a single batch utilizing cudaMemcpyBatchAsync, lowering overhead for fine-grained partitions. Batching additionally helps keep away from delays from interleaved CUDA streams. When partitions are transferred individually, transfers from different streams can delay the subsequent kernel launch. Transferring partitions in the identical batch avoids this delay.
Efficiency highlights
To judge the B200 GPU options mentioned above in a full Grace Blackwell system, we benchmarked GQE on TPC-H at Scale Issue 1000 (1TB) utilizing one of many two B200 GPUs in an NVIDIA GB200 NVL4 server, the place B200 GPUs are linked to the Grace CPU with NVLink-C2C. We used DuckDB 1.4.1 on the Turin Epyc 9755 CPU because the baseline. Every question was averaged over 5 hot-cache runs, with compression and pruning enabled on each side. We tune the GQE parameters per question, together with the diploma of parallelism and bodily operator planning.
The TPC-H dataset is optimized for partition pruning and compression by clustering the lineitem desk on l_shipdate and the orders desk on o_orderdate, and partitioning each tables by month. Internally, every partition is sorted on l_orderkey and o_orderkey, respectively.
In Determine 6, we present the runtime of the 22 queries. GQE outperforms DuckDB on 20 of twenty-two queries, with the most important beneficial properties on Q11, Q14, and Q15, the place partition pruning and compression sharply reduce knowledge motion throughout NVLink C2C. GQE showcases that even bandwidth-heavy queries like Q1 and Q6 execute rapidly on the GPU with these optimizations. In sum, GQE runs all queries in 9.0 s, in comparison with 74.0 s and 70.6 s for DuckDB in single and dual-socket configurations, respectively.
We current the speedups in Determine 7. GQE delivers as much as 25.5x over DuckDB’s finest CPU socket configuration, outperforming it on 20 of twenty-two queries and reaching 3x or larger on 17. Aggregated throughout all queries, GQE on the GB200 achieves a 7.5x speedup on complete execution time.
The check outcomes on this weblog publish are derived from TPC-H choice assist benchmark and aren’t corresponding to printed TPC-H outcomes, because the check outcomes on this weblog don’t adjust to the TPC-H specification.
Apply GQE finest practices to knowledge platforms
Database engines can translate NVIDIA Grace Blackwell {hardware} options into measurable question efficiency beneficial properties with focused optimizations. In GQE, partition pruning and hybrid compression reduce switch quantity whereas NVLink-C2C and DE {hardware} improve switch throughput. These optimizations cut back switch time and compose into subtle question execution utilizing NVIDIA cuDF, NVIDIA nvCOMP, and different CUDA-X libraries.
On TPC-H SF1000, GQE achieved a 7.5x speedup on complete execution time over a state-of-the-art CPU database, exhibiting how knowledge structure, compression technique, and execution will be designed collectively for contemporary database engines.
Leverage the GQE open-source reference structure and design, and efficiency optimizations, and discover how GQE can speed up your knowledge platforms.
Acknowledgements
The authors want to thank Tanmay Gujar for his technical contributions to GQE and his assessment of this publish. We additionally prolong our because of all GQE contributors—Hao Gao, Yadu Kiran, James Xia, Eyal Soha, Lingyan Yin, Daniel Juenger, Siyuan Lin, Bret Alfieri, Nico Iskos, Zhengru Wang, Rui Bao, Dhruv Sundararaman, Jiachun Li, and Kate Cheng—for his or her technical contributions. Lastly, we’d prefer to thank Nikolay Sakharnykh and Nuttiiya Seekhao for his or her assessment.
