GOOGLE SPEEDS UP GEMMA 4 WITH MULTI-TOKEN PREDICTION

Multi-token prediction represents a shift in how language models generate text. Traditional inference processes tokens sequentially—the model generates one token, then uses that output to predict the next. This sequential dependency creates a bottleneck, especially for longer outputs. Gemma 4's new approach uses a drafter model that speculates on multiple future tokens in parallel. A verifier then validates these predictions, accepting correct tokens and only recomputing when necessary. This speculative decoding technique reduces the number of forward passes required, lowering overall latency. The speed improvements are substantial in practical scenarios. For tasks requiring longer text generation, the technique delivers 2-3x faster inference on standard hardware. This acceleration comes without sacrificing output quality—the model produces identical results to standard sequential generation. The development aligns with broader industry efforts to optimize inference efficiency. As AI models grow larger and deployment costs increase, inference optimization has become critical for commercial viability. Similar approaches have gained traction across competing implementations. Google's implementation in Gemma 4 is particularly significant because it demonstrates the technique's effectiveness in a production-ready model. Developers using Gemma 4 can access these improvements through Google's standard deployment channels. The multi-token prediction method works best for longer outputs and is particularly effective on modern accelerators. For shorter completions, gains are more modest, but the approach maintains consistent quality across all scenarios. This advancement addresses a core challenge in deploying large language models at scale. By reducing inference time while maintaining quality, the technique makes real-time AI applications more feasible and cost-effective. The approach is generalizable, suggesting similar optimizations could benefit other model architectures.