Leading AI search agents like GPT-5.4 and Kimi K2.6 rely heavily on training data rather than conducting genuine web research, according to new benchmarks from the Harbin Institute of Technology.
Researchers developed a time-based evaluation called LiveBrowseComp to test whether popular AI search agents actually research the web or simply use it to validate existing knowledge. The benchmark focuses exclusively on events from the last 90 days—information that predates most training datasets.
The results revealed a stark pattern: when models cannot fall back on training data, their performance degrades significantly. Current top-ranked search agents struggled with recent events, forcing a reshuffling of existing performance rankings.
The Testing Method
LiveBrowseComp isolates a critical capability gap by design. It bypasses the advantage that models gain from training on vast historical data by asking only about fresh information. This approach forces agents to demonstrate actual research capability rather than knowledge retrieval.
GPT-5.4 and Kimi K2.6, among the most prominent AI search agents in production, performed worse under these constraints than their general benchmarks suggest. The finding indicates these systems function more as knowledge synthesizers than research tools.
Implications for Search
The distinction matters for users expecting genuine research capabilities. Traditional search engines retrieve documents; AI search agents market themselves as intelligent research partners. The Harbin research suggests the distinction is narrower than advertised.
Models trained on data with knowledge cutoff dates cannot access information beyond those cutoffs without live web access. The benchmark proves that when forced to rely on web research alone, these agents underperform—suggesting the web-browsing feature serves primarily as confirmation rather than discovery.
What's Next
The findings highlight an opportunity for improvement. Developing search agents that conduct genuine research rather than pattern-matching against training data could differentiate products and deliver more reliable current information.
As AI search agents become more integrated into workflows, understanding their actual capabilities becomes essential. LiveBrowseComp provides a measurement method, and the results show current systems have work to do.
Major AI companies are recruiting top university researchers at an accelerating pace, shifting research from public institutions to private industry labs. This brain drain is transforming once-open academic work into proprietary, closed-door projects.
Moonshot AI founder Yang Zhilin's departure from the US reflects a broader shift: Chinese entrepreneurs increasingly see greater opportunities in their home market than in Silicon Valley.
Chinese AI models have cut their performance deficit against US counterparts to 6% in June, down from 9% in May, according to Bloomberg Intelligence analysis. The rapid convergence challenges assumptions about American technological dominance in artificial intelligence.
Apple has announced a new generation of Apple Foundation Models (AFM), featuring two on-device models and three cloud-based variants. The largest on-device model, AFM 3 Core Advanced, contains 20 billion parameters and supports multimodal tasks.