{"id":1263,"date":"2026-06-16T16:00:00","date_gmt":"2026-06-16T16:00:00","guid":{"rendered":"https:\/\/futurenews24.com\/index.php\/2026\/06\/16\/how-to-optimize-transformer-based-models-for-low-precision-training\/"},"modified":"2026-06-20T17:59:36","modified_gmt":"2026-06-20T17:59:36","slug":"how-to-optimize-transformer-based-models-for-low-precision-training","status":"publish","type":"post","link":"https:\/\/futurenews24.com\/index.php\/2026\/06\/16\/how-to-optimize-transformer-based-models-for-low-precision-training\/","title":{"rendered":"The right way to Optimize Transformer-Primarily based Fashions for Low-Precision Coaching"},"content":{"rendered":"


\n<\/p>\n

\n

Transformer architectures are the spine of many fashionable giant language and generative AI fashions. As these fashions develop in measurement, coaching runs devour extra GPU hours and extra engineering iteration time. Accelerating transformers is due to this fact not only a efficiency optimization, however straight impacts how rapidly groups can experiment and the way giant a mannequin they’ll afford to coach. NVIDIA Hopper and NVIDIA Blackwell GPUs assist resolve this downside by introducing low-precision operator assist together with FP8 and NVFP4.<\/p>\n

Transformers spend a lot of their coaching time in GEMMs, and low-precision codecs pace up coaching primarily by making these matrix multiplications sooner and cheaper. Nevertheless, your transformer config doesn’t let you know which GEMMs are literally working in your mannequin. If you wish to perceive the place coaching time goes, you want to flip your transformer config and batch measurement into the precise M\u00d7Okay\u00d7N matrix shapes your mannequin executes, then benchmark these shapes throughout precisions. This may show you how to decide the optimum precision to your structure earlier than committing to a costlier coaching run.<\/p>\n

NVIDIA Transformer Engine (TE) can deal with quantization and kernel dispatch unlocking low precision codecs. This publish exhibits you the right way to transfer from high-level mannequin settings to concrete GEMM workloads, profile them with a microbenchmark, and estimate the place decrease precision will really translate into speedups that can assist you speed up your transformer-based fashions. The use case options CodonFM, a language mannequin for biology centered on RNA.\u00a0\u00a0<\/p>\n

Mannequin configuration and coaching inputs<\/h2>\n

Suppose you\u2019re working with a 5B-parameter mannequin akin to CodonFM 5B. It is going to have a config akin to:<\/p>\n

\nhidden_size: 4096
\nintermediate_size: 16384
\nnum_attention_heads: 32
\nnum_hidden_layers: 24\n<\/div>\n

Your coaching configuration is:<\/p>\n

\nmicro_batch_size: 31
\nsequence_length: 512\n<\/div>\n

The benchmark instrument can then take these hyperparameters straight after which use a single command to derive GEMM shapes, benchmark them throughout precisions, and compute the complete speedup evaluation:<\/p>\n

\npython benchmark.py
\n –hidden_size 4096
\n –intermediate_size 16384
\n –num_attention_heads 32
\n –num_hidden_layers 24
\n –micro_batch_size 31
\n –sequence_length 512
\n -o .\/photos\/b300_model_config_speedup.png\n<\/div>\n

Notice: To disable Blackwell-specific flags, add\u00a0–no-fp8 –no-fp4. –no-fp8 –no-fp4 gives BF16 plus the three FP8 recipes that work on Hopper.<\/p>\n

–no-fp8 disables MXFP8\u00a0<\/p>\n

–no-fp4 disables NVFP4<\/p>\n

Utilizing autocast mode versus prequantizing<\/h2>\n

By default, the instrument runs in autocast mode, which is what TE does throughout coaching: inputs are dynamically quantized to the goal precision earlier than every GEMM, so the measured time contains each the quantization value and the GEMM kernel itself. This gives you with the life like per-GEMM image throughout a coaching step.<\/p>\n

The instrument computes M = 31 \u00d7 512 = 15,872 tokens, derives all 12 GEMM shapes, benchmarks every throughout enabled precisions, and prints the complete outcomes. Fprop, Dgrad, and Wgrad shapes are all benchmarked individually to seize the affect of various matrix side ratios on kernel choice.\u00a0<\/p>\n

By default, the instrument runs in autocast mode, which is what TE does throughout coaching: inputs are dynamically quantized to the goal precision earlier than every GEMM, so the measured time contains each the quantization value and the GEMM kernel itself. This gives you with the life like per-GEMM image throughout a coaching step.<\/p>\n

The instrument computes M = 31 \u00d7 512 = 15,872 tokens, derives all 12 GEMM shapes, benchmarks every throughout enabled precisions, and prints the complete outcomes. Fprop, Dgrad, and Wgrad shapes are all benchmarked individually to seize the affect of various matrix side ratios on kernel choice.<\/p>\n

\n
\"A\"A
Determine 1. Per-layer GEMM time on NVIDIA B300 SXM6 AC in autocast mode, damaged down by precision (BF16, FP8 Present, FP8 Delayed, MXFP8, NVFP4) and stage (Fprop+Dgrad and Wgrad)<\/figcaption><\/figure>\n<\/div>\n

To isolate uncooked GEMM kernel efficiency, add –pre-quantize. This prequantizes all inputs as soon as earlier than the timed loop, so the measured time displays solely the GEMM kernel execution\u2014no dynamic quantization, no block scaling computation, no format conversion through the timed area.<\/p>\n

Notice that FP8 DelayedScaling at all times runs in autocast mode, even with –pre-quantize as a result of it depends on an amax historical past that requires dynamic quantization. Its occasions are due to this fact circuitously akin to different precisions in prequantized mode.<\/p>\n

\npython benchmark.py
\n –hidden_size 4096
\n –intermediate_size 16384
\n –num_attention_heads 32
\n –num_hidden_layers 24
\n –micro_batch_size 31
\n –sequence_length 512
\n –pre-quantize
\n -o .\/photos\/b300_model_config_speedup_prequant.png\n<\/div>\n
\n
\"A\"A
Determine 2. Per-layer GEMM time on NVIDIA B300 SXM6 AC in prequantized mode, isolating uncooked kernel throughput with out dynamic quantization overhead<\/figcaption><\/figure>\n<\/div>\n

Evaluating the autocast and prequantized speedups tells you precisely how a lot quantization overhead prices: NVFP4 versus BF16 goes from 1.98x (autocast) to three.48x (kernel-only). The hole between these two numbers is the overhead from dynamic quantization, Hadamard transforms, and block scaling that happens in every coaching step.<\/p>\n

Use autocast outcomes for predicting actual coaching speedups. That is what TE really does throughout coaching. Use prequantized outcomes to grasp whether or not quantization overhead is the bottleneck, or to check uncooked tensor core throughput throughout precisions unbiased of the quantization implementation.<\/p>\n

Decoding the outcomes for an actual mannequin<\/h2>\n

This part walks via the right way to interpret these outcomes for an actual mannequin. Utilizing the identical CodonFM 5B config, we ran the complete mannequin config benchmark on NVIDIA B300. The per-shape NVFP4 versus MXFP8 speedups from the Fprop outcomes are as follows:<\/p>\n

\nQKV proj: 0.579 \/ 0.392 = 1.48x
\nAttn out: 0.269 \/ 0.256 = 1.05x (barely sooner \u2014 overhead practically matches GEMM acquire)
\nMLP up: 0.924 \/ 0.635 = 1.46x
\nMLP down: 1.076 \/ 0.649 = 1.66x\n<\/div>\n

Pay attention to the next factors:\u00a0<\/p>\n

The eye output GEMM receives minimal profit from decrease precision. In contrast with the MXFP8 baseline, there’s solely a 1.05x speedup. That is the smallest weight matrix within the layer (4096\u00d74096)\u2014barely giant sufficient for decrease precision to beat the overhead. Against this, the a lot bigger MLP Down GEMM delivers 1.66x NVFP4 over MXFP8 on the identical {hardware}. The MLP down GEMM is large enough to amortize the quantization overhead, the place consideration output isn\u2019t.<\/p>\n

The large GEMMs present actual however subtheoretical good points. The FP4 tensor cores ship 1.46x to 1.66x over MXFP8 on the massive GEMMs. That is properly in need of the theoretical 2x to 3x from the {hardware} spec. When you embrace the eye output GEMM, the blended Fprop speedup drops to 1.47x. After including Wgrad occasions, non-GEMM overhead and NVFP4-specific quantization prices, the end-to-end hole between NVFP4 and MXFP8 in coaching is in keeping with these kernel-level numbers.<\/p>\n

FP8 DelayedScaling is surprisingly aggressive on NVIDIA Blackwell. At 7.80 ms\/layer in autocast mode, it outperforms each FP8 CurrentScaling (9.15 ms) and MXFP8 (8.98 ms). In prequantized mode FP8 CurrentScaling pulls forward (6.81 ms versus 8.12 ms), suggesting the DelayedScaling amax-history strategy has decrease quantization overhead however related uncooked kernel throughput. This can be a good instance of the comparability between autocast and prequantized surfacing completely different winners relying on whether or not you measure with or with out the quantization tax.<\/p>\n

The prequantized outcomes reveal the true kernel potential. Working with –pre-quantize removes quantization overhead totally, and NVFP4 versus BF16 jumps from 1.98x (autocast) to three.48x (kernel-only). This exhibits the FP4 tensor cores are delivering actual speedups. It\u2019s the quantization overhead in autocast mode that narrows the hole.<\/p>\n

The Fprop versus Dgrad comparability reveals that the 2x approximation is imprecise for quantized codecs. Whereas BF16 Dgrad is inside 2% of Fprop, quantized codecs present 5\u201313% slower Dgrad sums. The QKV Proj Dgrad is very uneven\u201433\u201351% slower than Fprop for FP8\/FP4\u2014as a result of swapping Okay (4096) and N (12288) dramatically adjustments the matrix side ratio and kernel choice. That is precisely why the instrument benchmarks Fprop and Dgrad individually relatively than counting Fprop time twice.<\/p>\n

Upon getting the estimated GEMM-only speedup, evaluate it towards your noticed end-to-end coaching speedup:<\/p>\n

GEMM speedup \u2248 coaching speedup: GEMMs dominate the step, the whole lot is working as anticipated<\/p>\n

GEMM speedup >> coaching speedup: Overhead outdoors of GEMMs is consuming the good points. For NVFP4 specifically, this overhead contains Random Hadamard transforms on Wgrad inputs, stochastic rounding on gradients, 2D block scaling for weights, and the additional reminiscence cross for per-tensor amax computation. These are all further ops that MXFP8 doesn\u2019t want, they usually can considerably slim the hole even when the uncooked FP4 GEMMs are a lot sooner<\/p>\n

GEMM speedup \u2248 1.0 even within the microbenchmark. The FP4 kernels aren\u2019t really sooner at these shapes, or they\u2019re silently falling again to FP8<\/p>\n

The final case is very value checking. Set NVTE_LOG_LEVEL=1 or examine with NVIDIA Nsight Techniques to verify that TE is definitely dispatching FP4 kernels. TE can silently fall again to FP8 or BF16 for layers or ops that don\u2019t assist FP4 but, which might clarify equivalent efficiency with no different signs. You too can evaluate GPU reminiscence utilization between MXFP8 and NVFP4 runs. If reminiscence is sort of equivalent, that\u2019s a robust sign that FP4 weights aren\u2019t really being saved.<\/p>\n

Get began benchmarking your mannequin for low-precision coaching\u00a0<\/h2>\n

Low-precision coaching speedups are extremely depending on the precise GEMM shapes your mannequin runs and working in low precision doesn’t routinely translate into end-to-end coaching good points, particularly when quantization overhead, kernel choice, and non-GEMM operations are included. By turning a transformer config into concrete M\u00d7Okay\u00d7N workloads, you may benchmark BF16, MXFP8, and NVFP4 on the shapes that matter to your mannequin earlier than committing to a full coaching run.<\/p>\n

Benchmark your GEMMs to see which precision is best for you. To get began, take a look at the benchmark script. For the complete documentation and to grasp how these shapes are derived, see the GEMM profiling tutorial within the Transformer Engine documentation.<\/p>\n

Use this benchmark to:<\/p>\n

Autocast outcomes to set life like training-speedup expectations<\/p>\n

Prequantize outcomes to know whether or not you\u2019re bottlenecked on kernels or on quantization<\/p>\n

Run candidate mannequin configs via the instrument earlier than committing to a coaching run, because the instrument is a helpful structure co-design instrument<\/p>\n<\/div>\n


\n
Source link <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

Transformer architectures are the spine of many fashionable giant language and generative AI fashions. As these fashions develop in measurement, coaching runs devour extra GPU hours and extra engineering iteration time. Accelerating transformers is due to this fact not only a efficiency optimization, however straight impacts how rapidly groups can experiment and the way giant […]<\/p>\n","protected":false},"author":1,"featured_media":1265,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/06\/cube-black-background.webp","fifu_image_alt":"","jnews-multi-image_gallery":[],"jnews_single_post":[],"jnews_primary_category":[],"jnews_override_bookmark_settings":[],"jnews_social_meta":[],"jnews_override_counter":[],"footnotes":""},"categories":[3],"tags":[1686,293,1685,700,290],"class_list":["post-1263","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-platforms-apps","tag-lowprecision","tag-models","tag-optimize","tag-training","tag-transformerbased"],"yoast_head":"\nThe right way to Optimize Transformer-Primarily based Fashions for Low-Precision Coaching - Future News 24<\/title>\n<meta name=\"description\" content=\"Transformer architectures are the backbone of many modern large language and generative AI models. As these models grow in size, training runs consume more GPU…\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/futurenews24.com\/index.php\/2026\/06\/16\/how-to-optimize-transformer-based-models-for-low-precision-training\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"The right way to Optimize Transformer-Primarily based Fashions for Low-Precision Coaching - Future News 24\" \/>\n<meta property=\"og:description\" content=\"Transformer architectures are the backbone of many modern large language and generative AI models. As these models grow in size, training runs consume more GPU…\" \/>\n<meta property=\"og:url\" content=\"https:\/\/futurenews24.com\/index.php\/2026\/06\/16\/how-to-optimize-transformer-based-models-for-low-precision-training\/\" \/>\n<meta property=\"og:site_name\" content=\"Future News 24\" \/>\n<meta property=\"article:published_time\" content=\"2026-06-16T16:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-06-20T17:59:36+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/06\/cube-black-background.webp\" \/>\n<meta name=\"author\" content=\"Future News 24\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/06\/cube-black-background.webp\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Future News 24\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/06\\\/16\\\/how-to-optimize-transformer-based-models-for-low-precision-training\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/06\\\/16\\\/how-to-optimize-transformer-based-models-for-low-precision-training\\\/\"},\"author\":{\"name\":\"Future News 24\",\"@id\":\"https:\\\/\\\/futurenews24.com\\\/#\\\/schema\\\/person\\\/cecad1bde21cfc357cf70128144d6c83\"},\"headline\":\"The right way to Optimize Transformer-Primarily based Fashions for Low-Precision Coaching\",\"datePublished\":\"2026-06-16T16:00:00+00:00\",\"dateModified\":\"2026-06-20T17:59:36+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/06\\\/16\\\/how-to-optimize-transformer-based-models-for-low-precision-training\\\/\"},\"wordCount\":1661,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/futurenews24.com\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/06\\\/16\\\/how-to-optimize-transformer-based-models-for-low-precision-training\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/developer-blogs.nvidia.com\\\/wp-content\\\/uploads\\\/2026\\\/06\\\/cube-black-background.webp\",\"keywords\":[\"LowPrecision\",\"Models\",\"Optimize\",\"Training\",\"TransformerBased\"],\"articleSection\":[\"AI Platforms & Apps\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/06\\\/16\\\/how-to-optimize-transformer-based-models-for-low-precision-training\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/06\\\/16\\\/how-to-optimize-transformer-based-models-for-low-precision-training\\\/\",\"url\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/06\\\/16\\\/how-to-optimize-transformer-based-models-for-low-precision-training\\\/\",\"name\":\"The right way to Optimize Transformer-Primarily based Fashions for Low-Precision Coaching - Future News 24\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/futurenews24.com\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/06\\\/16\\\/how-to-optimize-transformer-based-models-for-low-precision-training\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/06\\\/16\\\/how-to-optimize-transformer-based-models-for-low-precision-training\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/developer-blogs.nvidia.com\\\/wp-content\\\/uploads\\\/2026\\\/06\\\/cube-black-background.webp\",\"datePublished\":\"2026-06-16T16:00:00+00:00\",\"dateModified\":\"2026-06-20T17:59:36+00:00\",\"description\":\"Transformer architectures are the backbone of many modern large language and generative AI models. As these models grow in size, training runs consume more GPU…\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/06\\\/16\\\/how-to-optimize-transformer-based-models-for-low-precision-training\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/06\\\/16\\\/how-to-optimize-transformer-based-models-for-low-precision-training\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/06\\\/16\\\/how-to-optimize-transformer-based-models-for-low-precision-training\\\/#primaryimage\",\"url\":\"https:\\\/\\\/developer-blogs.nvidia.com\\\/wp-content\\\/uploads\\\/2026\\\/06\\\/cube-black-background.webp\",\"contentUrl\":\"https:\\\/\\\/developer-blogs.nvidia.com\\\/wp-content\\\/uploads\\\/2026\\\/06\\\/cube-black-background.webp\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/06\\\/16\\\/how-to-optimize-transformer-based-models-for-low-precision-training\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/futurenews24.com\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"The right way to Optimize Transformer-Primarily based Fashions for Low-Precision Coaching\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/futurenews24.com\\\/#website\",\"url\":\"https:\\\/\\\/futurenews24.com\\\/\",\"name\":\"Future News 24\",\"description\":\"The Smart Hub for AI and Next-Gen Innovation\",\"publisher\":{\"@id\":\"https:\\\/\\\/futurenews24.com\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/futurenews24.com\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/futurenews24.com\\\/#organization\",\"name\":\"Future News 24\",\"url\":\"https:\\\/\\\/futurenews24.com\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/futurenews24.com\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/futurenews24.com\\\/wp-content\\\/uploads\\\/2026\\\/06\\\/fn24-favicon.png\",\"contentUrl\":\"https:\\\/\\\/futurenews24.com\\\/wp-content\\\/uploads\\\/2026\\\/06\\\/fn24-favicon.png\",\"width\":250,\"height\":250,\"caption\":\"Future News 24\"},\"image\":{\"@id\":\"https:\\\/\\\/futurenews24.com\\\/#\\\/schema\\\/logo\\\/image\\\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/futurenews24.com\\\/#\\\/schema\\\/person\\\/cecad1bde21cfc357cf70128144d6c83\",\"name\":\"Future News 24\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/d57f07142d73cb5503ab2446ea7bc9ef3d0a5ba378d64a6157692311e42bf097?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/d57f07142d73cb5503ab2446ea7bc9ef3d0a5ba378d64a6157692311e42bf097?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/d57f07142d73cb5503ab2446ea7bc9ef3d0a5ba378d64a6157692311e42bf097?s=96&d=mm&r=g\",\"caption\":\"Future News 24\"},\"sameAs\":[\"https:\\\/\\\/futurenews24.com\"],\"url\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/author\\\/mridulpahuja20\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"The right way to Optimize Transformer-Primarily based Fashions for Low-Precision Coaching - Future News 24","description":"Transformer architectures are the backbone of many modern large language and generative AI models. As these models grow in size, training runs consume more GPU…","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/futurenews24.com\/index.php\/2026\/06\/16\/how-to-optimize-transformer-based-models-for-low-precision-training\/","og_locale":"en_US","og_type":"article","og_title":"The right way to Optimize Transformer-Primarily based Fashions for Low-Precision Coaching - Future News 24","og_description":"Transformer architectures are the backbone of many modern large language and generative AI models. As these models grow in size, training runs consume more GPU…","og_url":"https:\/\/futurenews24.com\/index.php\/2026\/06\/16\/how-to-optimize-transformer-based-models-for-low-precision-training\/","og_site_name":"Future News 24","article_published_time":"2026-06-16T16:00:00+00:00","article_modified_time":"2026-06-20T17:59:36+00:00","og_image":[{"url":"https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/06\/cube-black-background.webp","type":"","width":"","height":""}],"author":"Future News 24","twitter_card":"summary_large_image","twitter_image":"https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/06\/cube-black-background.webp","twitter_misc":{"Written by":"Future News 24","Est. reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/futurenews24.com\/index.php\/2026\/06\/16\/how-to-optimize-transformer-based-models-for-low-precision-training\/#article","isPartOf":{"@id":"https:\/\/futurenews24.com\/index.php\/2026\/06\/16\/how-to-optimize-transformer-based-models-for-low-precision-training\/"},"author":{"name":"Future News 24","@id":"https:\/\/futurenews24.com\/#\/schema\/person\/cecad1bde21cfc357cf70128144d6c83"},"headline":"The right way to Optimize Transformer-Primarily based Fashions for Low-Precision Coaching","datePublished":"2026-06-16T16:00:00+00:00","dateModified":"2026-06-20T17:59:36+00:00","mainEntityOfPage":{"@id":"https:\/\/futurenews24.com\/index.php\/2026\/06\/16\/how-to-optimize-transformer-based-models-for-low-precision-training\/"},"wordCount":1661,"commentCount":0,"publisher":{"@id":"https:\/\/futurenews24.com\/#organization"},"image":{"@id":"https:\/\/futurenews24.com\/index.php\/2026\/06\/16\/how-to-optimize-transformer-based-models-for-low-precision-training\/#primaryimage"},"thumbnailUrl":"https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/06\/cube-black-background.webp","keywords":["LowPrecision","Models","Optimize","Training","TransformerBased"],"articleSection":["AI Platforms & Apps"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/futurenews24.com\/index.php\/2026\/06\/16\/how-to-optimize-transformer-based-models-for-low-precision-training\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/futurenews24.com\/index.php\/2026\/06\/16\/how-to-optimize-transformer-based-models-for-low-precision-training\/","url":"https:\/\/futurenews24.com\/index.php\/2026\/06\/16\/how-to-optimize-transformer-based-models-for-low-precision-training\/","name":"The right way to Optimize Transformer-Primarily based Fashions for Low-Precision Coaching - Future News 24","isPartOf":{"@id":"https:\/\/futurenews24.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/futurenews24.com\/index.php\/2026\/06\/16\/how-to-optimize-transformer-based-models-for-low-precision-training\/#primaryimage"},"image":{"@id":"https:\/\/futurenews24.com\/index.php\/2026\/06\/16\/how-to-optimize-transformer-based-models-for-low-precision-training\/#primaryimage"},"thumbnailUrl":"https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/06\/cube-black-background.webp","datePublished":"2026-06-16T16:00:00+00:00","dateModified":"2026-06-20T17:59:36+00:00","description":"Transformer architectures are the backbone of many modern large language and generative AI models. As these models grow in size, training runs consume more GPU…","breadcrumb":{"@id":"https:\/\/futurenews24.com\/index.php\/2026\/06\/16\/how-to-optimize-transformer-based-models-for-low-precision-training\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/futurenews24.com\/index.php\/2026\/06\/16\/how-to-optimize-transformer-based-models-for-low-precision-training\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/futurenews24.com\/index.php\/2026\/06\/16\/how-to-optimize-transformer-based-models-for-low-precision-training\/#primaryimage","url":"https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/06\/cube-black-background.webp","contentUrl":"https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/06\/cube-black-background.webp"},{"@type":"BreadcrumbList","@id":"https:\/\/futurenews24.com\/index.php\/2026\/06\/16\/how-to-optimize-transformer-based-models-for-low-precision-training\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/futurenews24.com\/"},{"@type":"ListItem","position":2,"name":"The right way to Optimize Transformer-Primarily based Fashions for Low-Precision Coaching"}]},{"@type":"WebSite","@id":"https:\/\/futurenews24.com\/#website","url":"https:\/\/futurenews24.com\/","name":"Future News 24","description":"The Smart Hub for AI and Next-Gen Innovation","publisher":{"@id":"https:\/\/futurenews24.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/futurenews24.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/futurenews24.com\/#organization","name":"Future News 24","url":"https:\/\/futurenews24.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/futurenews24.com\/#\/schema\/logo\/image\/","url":"https:\/\/futurenews24.com\/wp-content\/uploads\/2026\/06\/fn24-favicon.png","contentUrl":"https:\/\/futurenews24.com\/wp-content\/uploads\/2026\/06\/fn24-favicon.png","width":250,"height":250,"caption":"Future News 24"},"image":{"@id":"https:\/\/futurenews24.com\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/futurenews24.com\/#\/schema\/person\/cecad1bde21cfc357cf70128144d6c83","name":"Future News 24","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/d57f07142d73cb5503ab2446ea7bc9ef3d0a5ba378d64a6157692311e42bf097?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/d57f07142d73cb5503ab2446ea7bc9ef3d0a5ba378d64a6157692311e42bf097?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/d57f07142d73cb5503ab2446ea7bc9ef3d0a5ba378d64a6157692311e42bf097?s=96&d=mm&r=g","caption":"Future News 24"},"sameAs":["https:\/\/futurenews24.com"],"url":"https:\/\/futurenews24.com\/index.php\/author\/mridulpahuja20\/"}]}},"_links":{"self":[{"href":"https:\/\/futurenews24.com\/index.php\/wp-json\/wp\/v2\/posts\/1263","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/futurenews24.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/futurenews24.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/futurenews24.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/futurenews24.com\/index.php\/wp-json\/wp\/v2\/comments?post=1263"}],"version-history":[{"count":1,"href":"https:\/\/futurenews24.com\/index.php\/wp-json\/wp\/v2\/posts\/1263\/revisions"}],"predecessor-version":[{"id":1264,"href":"https:\/\/futurenews24.com\/index.php\/wp-json\/wp\/v2\/posts\/1263\/revisions\/1264"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/futurenews24.com\/index.php\/wp-json\/wp\/v2\/media\/1265"}],"wp:attachment":[{"href":"https:\/\/futurenews24.com\/index.php\/wp-json\/wp\/v2\/media?parent=1263"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/futurenews24.com\/index.php\/wp-json\/wp\/v2\/categories?post=1263"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/futurenews24.com\/index.php\/wp-json\/wp\/v2\/tags?post=1263"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}