Giant Language Fashions (LLMs) have achieved state-of-the-art efficiency on a broad vary of Pure Language Processing (NLP) duties, together with doc processing and code era. Autoregressive Language Fashions (ARMs), which generate tokens sequentially conditioned on all earlier tokens, have been the predominant paradigm for LLMs. Whereas these fashions have achieved excessive accuracy throughout a spread of downstream duties, they exhibit low arithmetic depth as a result of inherent sequential dependency in next-token prediction. Not too long ago, Diffusion Language Fashions (DLMs) have emerged as a promising various structure. DLMs generate output tokens in parallel, mitigating the restrictions of sequential decoding. Nevertheless, the efficiency implications of DLMs relative to generally deployed ARMs should not totally understood. On this work, we current a complete research of the efficiency traits of ARMs and DLMs, combining theoretical evaluation with empirical profiling to characterize the trade-offs between these approaches. We present that though DLMs can obtain increased arithmetic depth than ARMs by leveraging parallelism throughout token positions, they fail to scale successfully with longer contexts. We then discover block-wise decoding for DLMs, which decouples arithmetic depth from sequence size and allows higher scaling to lengthy contexts (much like ARMs). We additionally study batched inference and discover that ARMs exhibit superior throughput as they profit extra from parallelism throughout sequences within the batch. Lastly, we spotlight alternatives for accelerating DLM inference, emphasizing that decreasing the variety of sampling steps is essential for open-source DLMs to realize decrease latency relative to ARMs.
† Seoul Nationwide College‡ College of California, Berkeley§ ICSI¶ LBNL†† College of Texas at Austin* Advisory function
