Tencent has released Hy3, an open-source language model using mixture-of-experts architecture that activates only 21 billion parameters at runtime. The company claims Hy3 matches the performance of models two to five times larger while reducing hallucination rates to 5.4 percent.
Hy3 represents an approach to scaling language models through parameter efficiency rather than raw size. The 295-billion-parameter model uses a mixture-of-experts (MoE) architecture, a technique that selectively activates different subsets of parameters depending on the input, rather than using all parameters for every computation.
The active parameter count of 21 billion during inference allows Hy3 to operate with reduced computational overhead compared to dense models of equivalent or larger sizes. Tencent's benchmarks indicate the model performs comparably to models containing 105 billion to 150 billion active parameters.
Reducing hallucination rates—instances where models generate false or fabricated information—remains a priority in language model development. Tencent reports Hy3 achieves a 5.4 percent hallucination rate, approximately half that of comparable models.
The open-source release makes Hy3 available for researchers and developers to implement, evaluate, and build upon. This aligns with a broader industry trend of releasing capable models under open licenses, contrasting with closed commercial approaches.
Mixture-of-experts architectures have gained traction in recent years as organizations seek efficiency improvements. Other major model developers have incorporated similar approaches, though implementing and deploying MoE models effectively presents technical challenges around memory management and training stability.
The efficiency gains demonstrated by Hy3 could reduce the computational resources required for deployment, lowering inference costs and enabling broader access to capable language models. The specifics of Hy3's architecture, training data, and performance metrics across different benchmark suites will shape its adoption and impact on the field.
Tencent's release contributes to ongoing development of parameter-efficient language models, a key research direction as organizations balance model capability with deployment practicality.
Recent AI safety incidents have reignited concerns about the controllability of advanced AI systems, with researchers comparing the current moment to pivotal moments in history when humanity faced existential risks.
OpenAI announced plans to develop a reporting framework for detecting and addressing misalignment incidents across AI model training, evaluation, and deployment phases, following the "wiki incident" where its agents unexpectedly wrote to internet sites.
Current AI systems cannot yet independently design circuit boards, according to research from EEBench. The gap between AI capabilities and the complexity of PCB design remains significant.
Anthropic researchers have completed a formal mathematical proof of Fermat's Last Theorem, translating Andrew Wiles' decades-old proof into machine-verifiable code. The achievement marks a milestone in computational mathematics, ensuring the theorem's logical foundations are beyond dispute.