Google has introduced multi-token prediction drafters for Gemma 4, a technique that accelerates inference speed by enabling the model to generate multiple tokens simultaneously rather than one at a time.
Multi-token prediction represents a shift in how language models generate text. Traditional inference processes tokens sequentially—the model generates one token, then uses that output to predict the next. This sequential dependency creates a bottleneck, especially for longer outputs.
Gemma 4's new approach uses a drafter model that speculates on multiple future tokens in parallel. A verifier then validates these predictions, accepting correct tokens and only recomputing when necessary. This speculative decoding technique reduces the number of forward passes required, lowering overall latency.
The speed improvements are substantial in practical scenarios. For tasks requiring longer text generation, the technique delivers 2-3x faster inference on standard hardware. This acceleration comes without sacrificing output quality—the model produces identical results to standard sequential generation.
The development aligns with broader industry efforts to optimize inference efficiency. As AI models grow larger and deployment costs increase, inference optimization has become critical for commercial viability. Similar approaches have gained traction across competing implementations.
Google's implementation in Gemma 4 is particularly significant because it demonstrates the technique's effectiveness in a production-ready model. Developers using Gemma 4 can access these improvements through Google's standard deployment channels.
The multi-token prediction method works best for longer outputs and is particularly effective on modern accelerators. For shorter completions, gains are more modest, but the approach maintains consistent quality across all scenarios.
This advancement addresses a core challenge in deploying large language models at scale. By reducing inference time while maintaining quality, the technique makes real-time AI applications more feasible and cost-effective. The approach is generalizable, suggesting similar optimizations could benefit other model architectures.
OpenAI has unveiled a dedicated physical keypad developed with keyboard maker Work Louder, designed to control AI agents. The device is available for order starting today.
Environmental and community groups are pushing for a pause on datacentre development in Australia until new regulations are finalized, following Prime Minister Anthony Albanese's announcement of an AI blueprint that includes energy rules for the sector.
Thinking Machines Lab has released Inkling, an open-source model with 975 billion parameters trained to process video and audio. The release positions the lab as a contender in the competitive AI landscape.
OpenAI's GPT-5.6 Sol Pro disproved a longstanding conjecture about the Benjamini-Hochberg statistical method in under two hours, where previous AI models and human researchers had failed for decades.