Turing Award winner Richard Sutton argues that synthetic data represents a fundamental flaw in scaling large language models. He contends the world's infinite complexity cannot be captured in simulations, limiting AI advancement.
Sutton identifies human expertise as a critical bottleneck preventing real scaling in AI systems. Synthetic data approaches, he argues, create only "microscopic" representations of reality, unable to capture the full complexity required for true machine intelligence.
The AI pioneer proposes an alternative: agents that learn continuously from direct experience rather than relying on frozen, pre-trained models. This approach would enable systems to adapt and improve through ongoing interaction with their environment.
Sutton's critique challenges the prevailing strategy in LLM development, where companies increasingly use synthetic data to supplement training sets. His position suggests that scaling through synthetic data alone represents a dead end, requiring instead a fundamental shift toward continual learning agents that evolve beyond initial training.
The remarks come as the AI industry weighs approaches to achieving artificial general intelligence, with debate intensifying over whether current scaling methods can reach that goal.
Patients at GP practices in Rotherham are hanging up on an AI receptionist called Emma after it fails to understand their local accents. Health watchdog Healthwatch Rotherham has flagged the issue despite the AI firm claiming support for 17 languages.
Chinese AI models Kimi K3 and GLM-5.3 have closed the gap with leading Western alternatives, forcing a reassessment of where competitive advantage actually lies in artificial intelligence.
Generalist AI has released GEN-1.5, an artificial intelligence model capable of teaching robots new tasks after observing just one demonstration. The breakthrough addresses a major limitation in robotics: the need for extensive training data.