Duplicate from tencent/Hy3-preview

Co-authored-by: TencentOpen <TencentOpen@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,39 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+assets/bench_agent_overview_v3.jpg filter=lfs diff=lfs merge=lfs -text
+assets/bench_claw_agent2.jpg filter=lfs diff=lfs merge=lfs -text
+assets/bench_context.jpg filter=lfs diff=lfs merge=lfs -text
+assets/bench_stem.jpg filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,78 @@
+TENCENT HY COMMUNITY LICENSE AGREEMENT
+Tencent Hy3 preview Release Date: April 23, 2026
+THIS LICENSE AGREEMENT DOES NOT APPLY IN THE EUROPEAN UNION, UNITED KINGDOM AND SOUTH KOREA AND IS EXPRESSLY LIMITED TO THE TERRITORY, AS DEFINED BELOW.
+By clicking to agree or by using, reproducing, modifying, distributing, performing or displaying any portion or element of the Tencent Hy Works, including via any Hosted Service, You will be deemed to have recognized and accepted the content of this Agreement, which is effective immediately.
+1.	DEFINITIONS.
+a.	“Acceptable Use Policy” shall mean the policy made available by Tencent as set forth in the Exhibit A.
+b.	“Agreement” shall mean the terms and conditions for use, reproduction, distribution, modification, performance and displaying of Tencent Hy Works or any portion or element thereof set forth herein.
+c.	“Documentation” shall mean the specifications, manuals and documentation for Tencent Hy made publicly available by Tencent.
+d.	“Hosted Service” shall mean a hosted service offered via an application programming interface (API), web access, or any other electronic or remote means.
+e.	“Licensee,” “You” or “Your” shall mean a natural person or legal entity exercising the rights granted by this Agreement and/or using the Tencent Hy Works for any purpose and in any field of use.
+f.	“Materials” shall mean, collectively, Tencent’s proprietary Tencent Hy and Documentation (and any portion thereof) as made available by Tencent under this Agreement.
+g.	“Model Derivatives” shall mean all: (i) modifications to Tencent Hy or any Model Derivative of Tencent Hy; (ii) works based on Tencent Hy or any Model Derivative of Tencent Hy; or (iii) any other machine learning model which is created by transfer of patterns of the weights, parameters, operations, or Output of Tencent Hy or any Model Derivative of Tencent Hy, to that model in order to cause that model to perform similarly to Tencent Hy or a Model Derivative of Tencent Hy, including distillation methods, methods that use intermediate data representations, or methods based on the generation of synthetic data Outputs by Tencent Hy or a Model Derivative of Tencent Hy for training that model. For clarity, Outputs by themselves are not deemed Model Derivatives.
+h.	“Output” shall mean the information and/or content output of Tencent Hy or a Model Derivative that results from operating or otherwise using Tencent Hy or a Model Derivative, including via a Hosted Service.
+i.	“Tencent,” “We” or “Us” shall mean the applicable entity or entities in the Tencent corporate family that own(s) intellectual property or other rights embodied in or utilized by the Materials.
+j.	“Tencent Hy” shall mean the large language models, text/image/video/audio/3D generation models, and multimodal large language models and their software and algorithms, including trained model weights, parameters (including optimizer states), machine-learning model code, inference-enabling code, training-enabling code, fine-tuning enabling code and other elements of the foregoing made publicly available by Us, including, without limitation to, Tencent Hy3 preview released at [https://huggingface.co/tencent/Hy3-preview; https://github.com/Tencent-Hunyuan/Hy3-preview].
+k.	“Tencent Hy Works” shall mean: (i) the Materials; (ii) Model Derivatives; and (iii) all derivative works thereof.
+l.	“Territory” shall mean the worldwide territory, excluding the territory of the European Union, United Kingdom and South Korea. 
+m.	“Third Party” or “Third Parties” shall mean individuals or legal entities that are not under common control with Us or You.
+n.	“including” shall mean including but not limited to.
+2.	GRANT OF RIGHTS.
+We grant You, for the Territory only, a non-exclusive, non-transferable and royalty-free limited license under Tencent’s intellectual property or other rights owned by Us embodied in or utilized by the Materials to use, reproduce, distribute, create derivative works of (including Model Derivatives), and make modifications to the Materials, only in accordance with the terms of this Agreement and the Acceptable Use Policy, and You must not violate (or encourage or permit anyone else to violate) any term of this Agreement or the Acceptable Use Policy.
+3.	DISTRIBUTION.
+You may, subject to Your compliance with this Agreement, distribute or make available to Third Parties the Tencent Hy Works, exclusively in the Territory, provided that You meet all of the following conditions:
+a.	You must provide all such Third Party recipients of the Tencent Hy Works or products or services using them a copy of this Agreement;
+b.	You must cause any modified files to carry prominent notices stating that You changed the files;
+c.	You are encouraged to: (i) publish at least one technology introduction blogpost or one public statement expressing Your experience of using the Tencent Hy Works; and (ii) mark the products or services developed by using the Tencent Hy Works to indicate that the product/service is “Powered by Tencent Hy”; and
+d.	All distributions to Third Parties (other than through a Hosted Service) must be accompanied by a “Notice” text file that contains the following notice: “Tencent Hy is licensed under the Tencent Hy Community License Agreement, Copyright © 2026 Tencent. All Rights Reserved. The trademark rights of “Tencent Hy” are owned by Tencent or its affiliate.”
+e.	In the event that You use, integrate, implement, or otherwise deploy the Tencent Hy Works, in whole or in part, to provide, enable, or support any service, product, or functionality to third parties, You shall clearly, accurately, and prominently disclose to all end users the full legal name and entity of the actual provider of such service, product, or functionality. You shall expressly and conspicuously state that Tencent is not affiliated with, associated with, sponsoring, or endorsing any such service, product, or functionality. You shall not use or display any name, logo, trademark, trade name, or other indicia of Tencent in any manner that could be construed as, or be likely to create, confusion, deception, or a false impression regarding any relationship, affiliation, sponsorship, or endorsement by Tencent.
+You may add Your own copyright statement to Your modifications and, except as set forth in this Section and in Section 5, may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Model Derivatives as a whole, provided Your use, reproduction, modification, distribution, performance and display of the work otherwise complies with the terms and conditions of this Agreement (including as regards the Territory). If You receive Tencent Hy Works from a Licensee as part of an integrated end user product, then this Section 3 of this Agreement will not apply to You.
+4.	ADDITIONAL COMMERCIAL TERMS.
+If, on the Tencent Hy version release date, the monthly active users of all products or services made available by or for Licensee is greater than 100 million monthly active users in the preceding calendar month, You must request a license from Tencent, which Tencent may grant to You in its sole discretion, and You are not authorized to exercise any of the rights under this Agreement unless or until Tencent otherwise expressly grants You such rights.
+5.	RULES OF USE.
+a.	Your use of the Tencent Hy Works must comply with applicable laws and regulations (including trade compliance laws and regulations) and adhere to the Acceptable Use Policy for the Tencent Hy Works, which is hereby incorporated by reference into this Agreement. You must include the use restrictions referenced in these Sections 5(a) and 5(b) as an enforceable provision in any agreement (e.g., license agreement, terms of use, etc.) governing the use and/or distribution of Tencent Hy Works and You must provide notice to subsequent users to whom You distribute that Tencent Hy Works are subject to the use restrictions in these Sections 5(a) and 5(b).
+b.	You must not use the Tencent Hy Works or any Output or results of the Tencent Hy Works to improve any other AI model (other than Tencent Hy or Model Derivatives thereof).
+c.	You must not use, reproduce, modify, distribute, or display the Tencent Hy Works, Output or results of the Tencent Hy Works outside the Territory. Any such use outside the Territory is unlicensed and unauthorized under this Agreement.
+6.	INTELLECTUAL PROPERTY.
+a.	Subject to Tencent’s ownership of Tencent Hy Works made by or for Tencent and intellectual property rights therein, conditioned upon Your compliance with the terms and conditions of this Agreement, as between You and Tencent, You will be the owner of any derivative works and modifications of the Materials and any Model Derivatives that are made by or for You.
+b.	No trademark licenses are granted under this Agreement, and in connection with the Tencent Hy Works, Licensee may not use any name or mark owned by or associated with Tencent or any of its affiliates, except as required for reasonable and customary use in describing and distributing the Tencent Hy Works. Tencent hereby grants You a license to use “Tencent Hy” (the “Mark”) in the Territory solely as required to comply with the provisions of Section 3(c), provided that You comply with any applicable laws related to trademark protection. All goodwill arising out of Your use of the Mark will inure to the benefit of Tencent.
+c.	If You commence a lawsuit or other proceedings (including a cross-claim or counterclaim in a lawsuit) against Us or any person or entity alleging that the Materials or any Output, or any portion of any of the foregoing, infringe any intellectual property or other right owned or licensable by You, then all licenses granted to You under this Agreement shall terminate as of the date such lawsuit or other proceeding is filed. You will defend, indemnify and hold harmless Us from and against any claim by any Third Party arising out of or related to Your or the Third Party’s use or distribution of the Tencent Hy Works.
+d.	Tencent claims no rights in Outputs You generate. You and Your users are solely responsible for Outputs and their subsequent uses.
+7.	DISCLAIMERS OF WARRANTY AND LIMITATIONS OF LIABILITY.
+a.	We are not obligated to support, update, provide training for, or develop any further version of the Tencent Hy Works or to grant any license thereto.
+b.	UNLESS AND ONLY TO THE EXTENT REQUIRED BY APPLICABLE LAW, THE Tencent Hy WORKS AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED “AS IS” WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES OF ANY KIND INCLUDING ANY WARRANTIES OF TITLE, MERCHANTABILITY, NONINFRINGEMENT, COURSE OF DEALING, USAGE OF TRADE, OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING THE APPROPRIATENESS OF USING, REPRODUCING, MODIFYING, PERFORMING, DISPLAYING OR DISTRIBUTING ANY OF THE Tencent Hy WORKS OR OUTPUTS AND ASSUME ANY AND ALL RISKS ASSOCIATED WITH YOUR OR A THIRD PARTY’S USE OR DISTRIBUTION OF ANY OF THE Tencent Hy WORKS OR OUTPUTS AND YOUR EXERCISE OF RIGHTS AND PERMISSIONS UNDER THIS AGREEMENT.
+c.	TO THE FULLEST EXTENT PERMITTED BY APPLICABLE LAW, IN NO EVENT SHALL TENCENT OR ITS AFFILIATES BE LIABLE UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS LIABILITY, OR OTHERWISE, FOR ANY DAMAGES, INCLUDING ANY DIRECT, INDIRECT, SPECIAL, INCIDENTAL, EXEMPLARY, CONSEQUENTIAL OR PUNITIVE DAMAGES, OR LOST PROFITS OF ANY KIND ARISING FROM THIS AGREEMENT OR RELATED TO ANY OF THE Tencent Hy WORKS OR OUTPUTS, EVEN IF TENCENT OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY OF ANY OF THE FOREGOING.
+8.	SURVIVAL AND TERMINATION.
+a.	The term of this Agreement shall commence upon Your acceptance of this Agreement or access to the Materials and will continue in full force and effect until terminated in accordance with the terms and conditions herein.
+b.	We may terminate this Agreement if You breach any of the terms or conditions of this Agreement. Upon termination of this Agreement, You must promptly delete and cease use of the Tencent Hy Works. Sections 6(a), 6(c), 7 and 9 shall survive the termination of this Agreement.
+9.	GOVERNING LAW AND JURISDICTION.
+a.	This Agreement and any dispute arising out of or relating to it will be governed by the laws of the Hong Kong Special Administrative Region of the People’s Republic of China, without regard to conflict of law principles, and the UN Convention on Contracts for the International Sale of Goods does not apply to this Agreement.
+b.	Exclusive jurisdiction and venue for any dispute arising out of or relating to this Agreement will be a court of competent jurisdiction in the Hong Kong Special Administrative Region of the People’s Republic of China, and Tencent and Licensee consent to the exclusive jurisdiction of such court with respect to any such dispute.
+ 
+EXHIBIT A
+ACCEPTABLE USE POLICY
+
+Tencent reserves the right to update this Acceptable Use Policy from time to time.
+Last modified: December 30, 2025
+
+Tencent endeavors to promote safe and fair use of its tools and features, including Tencent Hy. You agree not to use Tencent Hy or Model Derivatives:
+1.	Outside the Territory;
+2.	In any way that violates any applicable national, federal, state, local, international or any other law or regulation;
+3.	To harm Yourself or others;
+4.	To repurpose or distribute output from Tencent Hy or any Model Derivatives to harm Yourself or others; 
+5.	To override or circumvent the safety guardrails and safeguards We have put in place;
+6.	For the purpose of exploiting, harming or attempting to exploit or harm minors in any way;
+7.	To generate or disseminate verifiably false information and/or content with the purpose of harming others or influencing elections;
+8.	To generate or facilitate false online engagement, including fake reviews and other means of fake online engagement;
+9.	To intentionally defame, disparage or otherwise harass others;
+10.	To generate and/or disseminate malware (including ransomware) or any other content to be used for the purpose of harming electronic systems;
+11.	To generate or disseminate personal identifiable information with the purpose of harming others;
+12.	To generate or disseminate information (including images, code, posts, articles), and place the information in any public context (including –through the use of bot generated tweets), without expressly and conspicuously identifying that the information and/or content is machine generated;
+13.	To impersonate another individual without consent, authorization, or legal right;
+14.	To make high-stakes automated decisions in domains that affect an individual’s safety, rights or wellbeing (e.g., law enforcement, migration, medicine/health, management of critical infrastructure, safety components of products, essential services, credit, employment, housing, education, social scoring, or insurance);
+15.	In a manner that violates or disrespects the social ethics and moral standards of other countries or regions;
+16.	To perform, facilitate, threaten, incite, plan, promote or encourage violent extremism or terrorism;
+17.	For any use intended to discriminate against or harm individuals or groups based on protected characteristics or categories, online or offline social behavior or known or predicted personal or personality characteristics;
+18.	To intentionally exploit any of the vulnerabilities of a specific group of persons based on their age, social, physical or mental characteristics, in order to materially distort the behavior of a person pertaining to that group in a manner that causes or is likely to cause that person or another person physical or psychological harm;
+19.	For military purposes;
+20.	To engage in the unauthorized or unlicensed practice of any profession including, but not limited to, financial, legal, medical/health, or other professional practices.

README.md

@@ -0,0 +1,262 @@
+---
+license: other
+library_name: transformers
+---
+<p align="left">
+    <a href="https://huggingface.co/tencent/Hy3-preview/blob/main/README_CN.md">中文</a>&nbsp;｜&nbsp;English
+</p>
+<br>
+
+<p align="center">
+ <img src="assets/logo-en.png" width="400"/> <br>
+</p>
+
+<div align="center" style="line-height: 1;">
+
+
+[![License](https://img.shields.io/badge/License-Tencent%20Hy%20Community-blue)](#license)
+&nbsp;&nbsp;
+[![HuggingFace](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Tencent%20Hy-ffc107?color=ffc107&logoColor=white)](https://huggingface.co/tencent/Hy3-preview)
+&nbsp;&nbsp;
+[![ModelScope](https://img.shields.io/badge/ModelScope-Tencent%20Hy-624aff)](https://modelscope.cn/models/Tencent-Hunyuan/Hy3-preview)
+&nbsp;&nbsp;
+[![cnb.cool](https://img.shields.io/badge/cnb.cool-Tencent%20Hy-blue?logoColor=white)](https://cnb.cool/ai-models/tencent/Hy3-preview)
+&nbsp;&nbsp;
+[![GitCode](https://img.shields.io/badge/GitCode-Tencent%20Hy-red?logoColor=white)](https://ai.gitcode.com/tencent_hunyuan/Hy3-preview)
+
+</div>
+
+<p align="center">
+    🖥️&nbsp;<a href="https://aistudio.tencent.com/"><b>Official Website</b></a>&nbsp;&nbsp;|&nbsp;&nbsp;
+    💬&nbsp;<a href="https://github.com/Tencent-Hunyuan/Hy3-preview"><b>GitHub</b></a></p>
+
+---
+
+## Table of Contents
+
+- [Model Introduction](#model-introduction)
+- [Highlights](#highlights)
+- [Benchmark Results](#benchmark-results)
+  - [STEM & Reasoning](#stem--reasoning)
+  - [Context Learning & Instruction Following](#context-learning--instruction-following)
+  - [Code & Agent](#code--agent)
+- [News](#news)
+- [Model Links](#model-links)
+- [Quickstart](#quickstart)
+- [Deployment](#deployment)
+  - [vLLM](#vllm)
+  - [SGLang](#sglang)
+- [Training](#training)
+- [Quantization](#quantization)
+- [License](#license)
+- [Contact Us](#contact-us)
+
+---
+
+## Model Introduction
+
+**Hy3 preview** is a 295B-parameter Mixture-of-Experts (MoE) model with 21B active parameters and 3.8B MTP layer parameters, developed by the Tencent Hy Team. Hy3 preview is the first model trained on our rebuilt infrastructure, and the strongest we've shipped so far. It improves significantly on complex reasoning, instruction following, context learning, coding, and agent tasks.
+
+
+| Property | Value |
+|:---|:---|
+| Architecture | Mixture-of-Experts (MoE) |
+| Total Parameters | 295B |
+| Activated Parameters | 21B |
+| MTP Layer Parameters | 3.8B |
+| Number of Layers (excluding MTP layer) | 80 |
+| Number of MTP Layers | 1 |
+| Attention Heads | 64 (GQA, 8 KV heads, head dim 128) |
+| Hidden Size | 4096 |
+| Intermediate Size | 13312 |
+| Context Length | 256K |
+| Vocabulary Size | 120832 |
+| Number of Experts | 192 experts, top-8 activated |
+| Supported Precisions | BF16 |
+
+## Highlights
+
+- **STEM & Reasoning** — Complex reasoning underpins everything else. Hy3 preview performs well on challenging STEM benchmarks like FrontierScience-Olympiad and IMOAnswerBench, and achieved excellent results in the Tsinghua Qiuzhen College Math PhD qualifying exam (Spring '26) and the China High School Biology Olympiad (CHSBO 2025), demonstrating generalizable reasoning capacity.
+
+- **Context Learning & Instruction Following** — Real-world tasks require the ability to parse messy, lengthy contexts and follow complex rules. We built CL-bench and CL-bench-Life from our own business scenarios to innovatively measure context learning ability. Hy3 preview exhibits solid gains in both context learning and instruction following capabilities.
+
+- **Code & Agent** — Coding and agents saw the biggest gains. With a rebuilt RL infrastructure and larger-scale training tasks, we posted competitive scores across mainstream coding agent benchmarks (SWE-bench Verified, Terminal-Bench 2.0) and search agent benchmarks (BrowseComp, WideSearch).
+
+## Benchmark Results
+
+### Pre-trained Model Performance
+
+| Category | Benchmark (Metric) | # Shots | Kimi-K2 BASE | DeepSeek-V3 BASE | GLM-4.5 BASE | Hy3 preview-Base |
+|---|---|---|---|---|---|---|
+| | #ActivatedParams | - | 32B | 37B | 32B | 21B |
+| | #TotalParams | - | 1043B | 671B | 355B | 295B |
+| **English** | MMLU | 5-shot | **88.24** | 87.68 | 87.73 | 87.42 |
+| | MMLU-Pro | 5-shot | **65.98** | 63.98 | 63.67 | 65.76 |
+| | MMLU-Redux | 5-shot | **87.18** | 86.81 | 86.56 | 86.86 |
+| | ARC-Challenge | 0-shot | **96.66** | 94.65 | 96.32 | 95.99 |
+| | DROP | 5-shot | 86.40 | **86.50** | 82.90 | 85.50 |
+| | PIQA | 4-shot | **84.93** | 84.22 | 84.71 | 84.39 |
+| | SuperGPQA | 5-shot | 51.10 | 46.17 | 49.64 | **51.60** |
+| | SimpleQA | 5-shot | **34.37** | 26.15 | 29.26 | 26.47 |
+| **Code** | MBPP-plus | 3-shot | **81.35** | 75.47 | 78.05 | 78.71 |
+| | CRUXEval-I | 3-shot | 68.01 | 67.79 | 68.51 | **71.19** |
+| | CRUXEval-O | 3-shot | 69.62 | **71.00** | 67.75 | 68.38 |
+| | LiveCodeBench-v6 | 1-shot | 30.86 | 29.31 | 27.43 | **34.86** |
+| **Math** | GSM8K | 4-shot | 93.46 | 88.15 | 90.06 | **95.37** |
+| | MATH | 4-shot | 71.20 | 59.37 | 61.00 | **76.28** |
+| | CMath | 4-shot | 90.83 | 85.50 | 89.33 | **91.17** |
+| **Chinese** | C-Eval | 5-shot | **91.51** | 90.35 | 85.84 | 89.80 |
+| | CMMLU | 5-shot | **90.72** | 87.90 | 86.46 | 89.61 |
+| | Chinese-simpleQA | 5-shot | **74.58** | 68.72 | 68.49 | 69.73 |
+| **Multilingual** | MMMLU | 5-shot | 77.63 | 79.54 | 79.26 | **80.15** |
+| | INCLUDE | 5-shot | 75.66 | 77.86 | 76.27 | **78.64** |
+
+### Instruct Model Performance
+
+#### STEM & Reasoning
+
+Complex reasoning underpins everything else. Hy3 preview performs well on challenging STEM benchmarks like FrontierScience-Olympiad and IMOAnswerBench. It also achieved excellent results in the Tsinghua Qiuzhen College Math PhD qualifying exam (Spring '26) and the China High School Biology Olympiad (CHSBO 2025), demonstrating a high degree of generalizable reasoning capacity.
+
+<p align="center"><img src="assets/bench_stem.jpg" width="800" alt="STEM & Reasoning benchmarks"/></p>
+
+#### Context Learning & Instruction Following
+
+Real-world tasks require the ability to parse messy, lengthy contexts and follow complex rules. We built CL-bench and CL-bench-Life from our own business scenarios to innovatively measure context learning ability. Hy3 preview exhibits solid gains in both context learning and instruction following capabilities.
+
+<p align="center"><img src="assets/bench_context.jpg" width="800" alt="Context Learning & Instruction Following benchmarks"/></p>
+
+#### Code & Agent
+
+Coding and agents saw the biggest gains. With a rebuilt RL infrastructure and larger-scale training tasks, we posted competitive scores across mainstream coding agent benchmarks (SWE-bench Verified, Terminal-Bench 2.0) and search agent benchmarks (BrowseComp, WideSearch).
+
+<p align="center"><img src="assets/bench_agent_overview_v3.jpg" width="800" alt="Agent benchmarks overview"/></p>
+
+Coding is about whether a model can execute in a development environment. Search is about whether it can find and combine information from the open web. Both matter for complex agent scenarios like OpenClaw. Hy3 preview scores well on ClawEval and WildClawBench — a sign that its agent capabilities are becoming practical.
+
+<p align="center"><img src="assets/bench_claw_agent.png" width="800" alt="Claw Agent benchmarks"/></p>
+
+Beyond public benchmarks, we built internal evaluation sets to test the model in real development scenarios. On Hy-Backend (backend-focused tasks), Hy-Vibe Bench (real-user dev workflows), and Hy-SWE Max, Hy3 preview scores competitively against other open-source models.
+
+<p align="center"><img src="assets/bench_claw_agent2.jpg" width="800" alt="Internal benchmarks"/></p>
+
+## News
+
+
+* **[2026-04-23]** 🔥 We open-source **Hy3 preview** model weights on [Hugging Face](https://huggingface.co/tencent/Hy3-preview), [ModelScope](https://modelscope.cn/models/Tencent-Hunyuan/Hy3-preview), and [GitCode](https://ai.gitcode.com/tencent_hunyuan/Hy3-preview).
+
+## Model Links
+
+
+| Model Name | Description | Hugging Face | ModelScope | GitCode |
+|:---|:---|:---:|:---:|:---:|
+| Hy3 preview | Instruct model | 🤗 [Model](https://huggingface.co/tencent/Hy3-preview) | [Model](https://modelscope.cn/models/Tencent-Hunyuan/Hy3-preview) | [Model](https://ai.gitcode.com/tencent_hunyuan/Hy3-preview) |
+| Hy3 preview-Base | Pre-trained base model | 🤗 [Model](https://huggingface.co/tencent/Hy3-preview-Base) | [Model](https://modelscope.cn/models/Tencent-Hunyuan/Hy3-preview-Base) | [Model](https://ai.gitcode.com/tencent_hunyuan/Hy3-preview-Base) |
+
+## Quickstart
+
+Deploy Hy3 preview with [vLLM](#vllm) or [SGLang](#sglang) first, then call the OpenAI-compatible API:
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
+
+response = client.chat.completions.create(
+    model="hy3-preview",
+    messages=[
+        {"role": "user", "content": "Hello! Can you briefly introduce yourself?"},
+    ],
+    temperature=0.9,
+    top_p=1.0,
+    # reasoning_effort: "no_think" (default, direct response), "low", "high" (deep chain-of-thought)
+    extra_body={"chat_template_kwargs": {"reasoning_effort": "no_think"}},
+)
+print(response.choices[0].message.content)
+```
+
+> **Recommended parameters**: `temperature=0.9`, `top_p=1.0`.
+>
+> **Reasoning mode**: Set `reasoning_effort` to `"high"` for complex tasks (math, coding, reasoning) or `"no_think"` for direct responses.
+
+See the [Deployment](#deployment) section below for how to start the API server.
+
+## Deployment
+
+Hy3-preview has 295B parameters in total. To serve it on 8 GPUs, we recommend using H20-3e or other GPUs with larger memory capacity.
+
+### vLLM
+
+Build vLLM from source:
+```bash
+uv venv --python 3.12 --seed --managed-python
+source .venv/bin/activate
+git clone https://github.com/vllm-project/vllm.git
+cd vllm
+uv pip install --editable . --torch-backend=auto
+```
+
+Start the vLLM server with MTP enabled:
+
+```bash
+vllm serve tencent/Hy3-preview \
+  --tensor-parallel-size 8 \
+  --speculative-config.method mtp \
+  --speculative-config.num_speculative_tokens 1 \
+  --tool-call-parser hy_v3 \
+  --reasoning-parser hy_v3 \
+  --enable-auto-tool-choice \
+  --served-model-name hy3-preview
+```
+
+### SGLang
+
+Build SGLang from source:
+```bash
+git clone https://github.com/sgl-project/sglang
+cd sglang
+pip3 install pip --upgrade
+pip3 install "transformers>=5.6.0"
+pip3 install -e "python"
+```
+
+Launch SGLang server with MTP enabled:
+
+```bash
+python3 -m sglang.launch_server \
+  --model tencent/Hy3-preview \
+  --tp 8 \
+  --tool-call-parser hunyuan \
+  --reasoning-parser hunyuan \
+  --speculative-num-steps 1 \
+  --speculative-eagle-topk 1 \
+  --speculative-num-draft-tokens 2 \
+  --speculative-algorithm EAGLE \
+  --served-model-name hy3-preview
+```
+
+## Training
+
+Hy3 preview provides a complete model training pipeline, supporting both full fine-tuning and LoRA fine-tuning, with DeepSpeed ZeRO configurations and LLaMA-Factory integration.
+
+For detailed training documentation, please refer to: [Training Guide](./train/README.md)
+
+## Quantization
+
+We provide [AngelSlim](https://github.com/tencent/AngelSlim), a more accessible, comprehensive, and efficient toolkit for large model compression. AngelSlim supports a comprehensive suite of compression tools for large-scale multimodal models, including common quantization algorithms, low-bit quantization, and speculative sampling.
+
+## License
+
+
+Hy3 preview is released under the **Tencent Hy Community License Agreement**. See [LICENSE](./LICENSE) for details.
+
+## Contact Us
+
+If you would like to leave a message for our R&D and product teams, welcome to contact us. You can also reach us via email:
+
+📧 **hunyuan_opensource@tencent.com**
+
+---
+
+<p align="center">
+  <i>Hy3 preview is developed by the Tencent Hy Team.</i>
+</p>

README_CN.md

@@ -0,0 +1,259 @@
+<p align="left">
+   <a href="https://huggingface.co/tencent/Hy3-preview">English</a>&nbsp;｜&nbsp;中文
+</p>
+<br>
+
+<p align="center">
+ <img src="assets/logo-zh.png" width="400"/> <br>
+</p>
+
+<div align="center" style="line-height: 1;">
+
+
+[![License](https://img.shields.io/badge/License-Tencent%20Hy%20Community-blue)](#许可证)
+&nbsp;&nbsp;
+[![HuggingFace](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Tencent%20Hy-ffc107?color=ffc107&logoColor=white)](https://huggingface.co/tencent/Hy3-preview)
+&nbsp;&nbsp;
+[![ModelScope](https://img.shields.io/badge/ModelScope-Tencent%20Hy-624aff)](https://modelscope.cn/models/Tencent-Hunyuan/Hy3-preview)
+&nbsp;&nbsp;
+[![cnb.cool](https://img.shields.io/badge/cnb.cool-Tencent%20Hy-blue?logoColor=white)](https://cnb.cool/ai-models/tencent/Hy3-preview)
+&nbsp;&nbsp;
+[![GitCode](https://img.shields.io/badge/GitCode-Tencent%20Hy-red?logoColor=white)](https://ai.gitcode.com/tencent_hunyuan/Hy3-preview)
+
+</div>
+
+<p align="center">
+    🖥️&nbsp;<a href="https://aistudio.tencent.com/"><b>官方网站</b></a>&nbsp;&nbsp;|&nbsp;&nbsp;
+    💬&nbsp;<a href="https://github.com/Tencent-Hunyuan/Hy3-preview"><b>GitHub</b></a></p>
+
+---
+
+## 目录
+
+- [模型介绍](#模型介绍)
+- [亮点展示](#亮点展示)
+- [评测结果](#评测结果)
+  - [复杂推理（STEM & Reasoning）](#复杂推理stem--reasoning)
+  - [上下文学习和指令遵循（Context Learning & Instruction Following）](#上下文学习和指令遵循context-learning--instruction-following)
+  - [代码和智能体（Code & Agent）](#代码和智能体code--agent)
+- [新闻](#新闻)
+- [模型链接](#模型链接)
+- [快速开始](#快速开始)
+- [推理和部署](#推理和部署)
+  - [vLLM](#使用-vllm-推理)
+  - [SGLang](#使用-sglang-推理)
+- [模型训练](#模型训练)
+- [量化工具](#量化工具)
+- [许可证](#许可证)
+- [联系我们](#联系我们)
+
+---
+
+## 模型介绍
+
+**Hy3 preview** 是由腾讯混元团队研发的快慢思考融合的混合专家模型，总参数量 295B，激活参数 21B，MTP 层参数 3.8B。Hy3 preview 是我们重建后训练的第一个模型，也是混元迄今最智能的模型，在复杂推理、指令遵循、上下文学习、代码、智能体等能力及推理性能上实现了大幅的提升。
+
+
+| 属性 | 值 |
+|:---|:---|
+| 架构 | 混合专家（MoE） |
+| 总参数量 | 295B |
+| 激活参数量 | 21B |
+| MTP层参数量 | 3.8B |
+| 层数（不含MTP层） | 80 |
+| MTP层数 | 1 |
+| 注意力头 | 64（GQA，8 个 KV 头，head dim 128） |
+| 隐藏层维度 | 4096 |
+| FFN 中间层维度 | 13312 |
+| 上下文长度 | 256K |
+| 词表大小 | 120832 |
+| 专家数量 | 192 个专家，top-8 激活 |
+| 支持精度 | BF16 |
+
+## 亮点展示
+
+- **复杂推理（STEM & Reasoning）** — 推理能力是模型解决各种问题的基础。在 FrontierScience-Olympiad、IMOAnswerBench 等高难度理工科推理任务中表现突出，并在最新的清华大学求真书院数学博资考（26春）和全国中学生生物学联赛（CHSBO 2025）中取得优异成绩，展现出可泛化的强推理能力。
+
+- **上下文学习和指令遵循（Context Learning & Instruction Following）** — 在各种真实的生产与生活场景，理解杂乱冗长的上下文并遵从复杂多变的规则是模型的首要挑战。基于我们多种业务场景的灵感，我们提出了 CL-bench 和 CL-bench-Life 来创新性地评估模型的上下文学习能力，并在 Hy3 preview 显著地提升了模型上下文学习和指令遵循能力。
+
+- **代码和智能体（Code & Agent）** — Hy3 preview 提升最为显著的方向。得益于预训练及强化学习框架的重建和强化学习任务规模的提升，我们以较快的速度在 SWE-Bench Verified、Terminal-Bench 2.0 等主流代码智能体基准以及 BrowseComp、WideSearch 等主流搜索智能体基准中取得了强竞争力的结果。
+
+## 评测结果
+
+### 预训练模型效果
+
+| Category | Benchmark (Metric) | # Shots | Kimi-K2 BASE | DeepSeek-V3 BASE | GLM-4.5 BASE | Hy3 preview-Base |
+|---|---|---|---|---|---|---|
+| | #ActivatedParams | - | 32B | 37B | 32B | 21B |
+| | #TotalParams | - | 1043B | 671B | 355B | 295B |
+| **English** | MMLU | 5-shot | **88.24** | 87.68 | 87.73 | 87.42 |
+| | MMLU-Pro | 5-shot | **65.98** | 63.98 | 63.67 | 65.76 |
+| | MMLU-Redux | 5-shot | **87.18** | 86.81 | 86.56 | 86.86 |
+| | ARC-Challenge | 0-shot | **96.66** | 94.65 | 96.32 | 95.99 |
+| | DROP | 5-shot | 86.40 | **86.50** | 82.90 | 85.50 |
+| | PIQA | 4-shot | **84.93** | 84.22 | 84.71 | 84.39 |
+| | SuperGPQA | 5-shot | 51.10 | 46.17 | 49.64 | **51.60** |
+| | SimpleQA | 5-shot | **34.37** | 26.15 | 29.26 | 26.47 |
+| **Code** | MBPP-plus | 3-shot | **81.35** | 75.47 | 78.05 | 78.71 |
+| | CRUXEval-I | 3-shot | 68.01 | 67.79 | 68.51 | **71.19** |
+| | CRUXEval-O | 3-shot | 69.62 | **71.00** | 67.75 | 68.38 |
+| | LiveCodeBench-v6 | 1-shot | 30.86 | 29.31 | 27.43 | **34.86** |
+| **Math** | GSM8K | 4-shot | 93.46 | 88.15 | 90.06 | **95.37** |
+| | MATH | 4-shot | 71.20 | 59.37 | 61.00 | **76.28** |
+| | CMath | 4-shot | 90.83 | 85.50 | 89.33 | **91.17** |
+| **Chinese** | C-Eval | 5-shot | **91.51** | 90.35 | 85.84 | 89.80 |
+| | CMMLU | 5-shot | **90.72** | 87.90 | 86.46 | 89.61 |
+| | Chinese-simpleQA | 5-shot | **74.58** | 68.72 | 68.49 | 69.73 |
+| **Multilingual** | MMMLU | 5-shot | 77.63 | 79.54 | 79.26 | **80.15** |
+| | INCLUDE | 5-shot | 75.66 | 77.86 | 76.27 | **78.64** |
+
+### Instruct 模型效果
+
+#### 复杂推理（STEM & Reasoning）
+
+推理能力是模型解决各种问题的基础。Hy3 preview 在 FrontierScience-Olympiad、IMOAnswerBench 等高难度理工科推理任务中表现突出，并在最新的清华大学求真书院数学博资考（26春）和全国中学生生物学联赛（CHSBO 2025）中取得优异成绩，展现出可泛化的强推理能力。
+
+<p align="center"><img src="assets/bench_stem.jpg" width="800" alt="STEM & Reasoning 评测结果"/></p>
+
+#### 上下文学习和指令遵循（Context Learning & Instruction Following）
+
+在各种真实的生产与生活场景，理解杂乱冗长的上下文并遵从复杂多变的规则是模型的首要挑战。基于我们多种业务场景的灵感，我们提出了 CL-bench 和 CL-bench-Life 来创新性地评估模型的上下文学习能力，并在 Hy3 preview 显著地提升了模型上下文学习和指令遵循能力。
+
+<p align="center"><img src="assets/bench_context.jpg" width="800" alt="上下文学习和指令遵循评测结果"/></p>
+
+#### 代码和智能体（Code & Agent）
+
+代码和智能体是 Hy3 preview 提升最为显著的方向。得益于预训练及强化学习框架的重建和强化学习任务规模的提升，我们以较快的速度在 SWE-Bench Verified、Terminal-Bench 2.0 等主流代码智能体基准以及 BrowseComp、WideSearch 等主流搜索智能体基准中取得了强竞争力的结果。
+
+<p align="center"><img src="assets/bench_agent_overview_v3.jpg" width="800" alt="Agent 评测总览"/></p>
+
+在数字世界中，代码关注的是模型在开发环境中的执行能力，搜索则聚焦于开放信息空间中的检索、筛选与整合能力，两者共同决定了模型在复杂智能体场景（例如 OpenClaw）中是否真正具备可用性。Hy3 preview 在 ClawEval 和 WildClawBench 等评测中表现突出，进一步表明我们的智能体能力的全面与实用性。
+
+<p align="center"><img src="assets/bench_claw_agent.png" width="800" alt="Claw Agent 评测"/></p>
+
+除了公开榜单，我们进一步构建了多个内部的评测集，对模型在真实开发场景中的表现进行评估。结果表明，无论是在后端工程任务集 Hy-Backend，贴近真实用户开发交互的 Hy-Vibe Bench，还是高难度软件工程开发任务集 Hy-SWE Max 上，Hy3 preview 均体现出了强竞争力。
+
+<p align="center"><img src="assets/bench_claw_agent2.jpg" width="800" alt="内部评测结果"/></p>
+
+## 新闻
+
+* **[2026-04-23]** 🔥 我们在 [Hugging Face](https://huggingface.co/tencent/Hy3-preview)、[ModelScope](https://modelscope.cn/models/Tencent-Hunyuan/Hy3-preview) 和 [GitCode](https://ai.gitcode.com/tencent_hunyuan/Hy3-preview) 开源了 **Hy3 preview** 模型权重。
+
+## 模型链接
+
+
+| 模型名 | 简介 | Hugging Face | ModelScope | GitCode |
+|:---|:---|:---:|:---:|:---:|
+| Hy3 preview | Instruct 模型 | 🤗 [Model](https://huggingface.co/tencent/Hy3-preview) | [Model](https://modelscope.cn/models/Tencent-Hunyuan/Hy3-preview) | [Model](https://ai.gitcode.com/tencent_hunyuan/Hy3-preview) |
+| Hy3 preview-Base | 预训练基座模型 | 🤗 [Model](https://huggingface.co/tencent/Hy3-preview-Base) | [Model](https://modelscope.cn/models/Tencent-Hunyuan/Hy3-preview-Base) | [Model](https://ai.gitcode.com/tencent_hunyuan/Hy3-preview-Base) |
+
+## 快速开始
+
+建议先通过 [vLLM](#使用-vllm-推理) 或 [SGLang](#使用-sglang-推理) 部署服务，然后通过 OpenAI 兼容 API 调用：
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
+
+response = client.chat.completions.create(
+    model="hy3-preview",
+    messages=[
+        {"role": "user", "content": "你好！请简单介绍一下你自己。"},
+    ],
+    temperature=0.9,
+    top_p=1.0,
+    # reasoning_effort: "no_think"（默认，直接回复）、"low"、"high"（深度思维链）
+    extra_body={"chat_template_kwargs": {"reasoning_effort": "no_think"}},
+)
+print(response.choices[0].message.content)
+```
+
+> **推荐参数**：`temperature=0.9`，`top_p=1.0`。
+>
+> **推理模式**：复杂任务（数学、编程、推理）建议设置 `reasoning_effort="high"`，日常对话可使用默认的 `"no_think"` 直接回复。
+
+具体部署方式请参考下方[推理和部署](#推理和部署)章节。
+
+## 推理和部署
+
+Hy3-preview 总参数量为 295B，当使用 8 张 GPU 时，建议使用 H20-3e 或其他有更大显存的卡型。
+
+### vLLM
+
+从源码构建 vLLM：
+
+```bash
+uv venv --python 3.12 --seed --managed-python
+source .venv/bin/activate
+git clone https://github.com/vllm-project/vllm.git
+cd vllm
+uv pip install --editable . --torch-backend=auto
+```
+
+启动 vLLM 服务，开启 MTP：
+
+```bash
+vllm serve tencent/Hy3-preview \
+  --tensor-parallel-size 8 \
+  --speculative-config.method mtp \
+  --speculative-config.num_speculative_tokens 1 \
+  --tool-call-parser hy_v3 \
+  --reasoning-parser hy_v3 \
+  --enable-auto-tool-choice \
+  --served-model-name hy3-preview
+```
+
+### SGLang
+
+从源码构建 SGLang：
+
+```bash
+git clone https://github.com/sgl-project/sglang
+cd sglang
+pip3 install pip --upgrade
+pip3 install "transformers>=5.6.0"
+pip3 install -e "python"
+```
+
+启动 SGLang 服务，开启 MTP：
+
+```bash
+python3 -m sglang.launch_server \
+  --model tencent/Hy3-preview \
+  --tp 8 \
+  --tool-call-parser hunyuan \
+  --reasoning-parser hunyuan \
+  --speculative-num-steps 1 \
+  --speculative-eagle-topk 1 \
+  --speculative-num-draft-tokens 2 \
+  --speculative-algorithm EAGLE \
+  --served-model-name hy3-preview
+```
+
+## 模型训练
+
+Hy3 preview 提供了完整的模型训练流程，支持全量微调和 LoRA 微调，同时支持 DeepSpeed ZeRO 多种配置以及 LLaMA-Factory 集成。
+
+详细的训练文档请参考：[模型训练指南](./train/README_CN.md)
+
+## 量化工具
+
+我们提供了 [AngelSlim](https://github.com/tencent/AngelSlim)——一套易用、全面、高效的大模型压缩工具包，涵盖常用量化算法、低比特量化和投机采样等能力。
+
+## 许可证
+
+
+Hy3 preview 基于 **腾讯混元社区许可协议** 发布。详情请参阅 [LICENSE](./LICENSE)。
+
+## 联系我们
+
+如有问题或建议，欢迎通过邮件联系我们：
+
+📧 **hunyuan_opensource@tencent.com**
+
+---
+
+<p align="center">
+  <i>Hy3 preview 由腾讯混元团队研发。</i>
+</p>

chat_template.jinja

@@ -0,0 +1,195 @@
+{#- ----------‑‑‑ special token variables ‑‑‑---------- -#}
+{%- set bos_token = '<｜hy_begin▁of▁sentence｜>' %}
+{%- set pad_token = '<｜hy_▁pad▁｜>' %}
+{%- set user_token = '<｜hy_User｜>' %}
+{%- set assistant_token = '<｜hy_Assistant｜>' %}
+{%- set eos_token = '<｜hy_eos｜>' %}
+{%- set think_begin_token = '<think>' %}
+{%- set think_end_token = '</think>' %}
+{%- set toolcalls_begin_token = '<tool_calls>' %}
+{%- set toolcalls_end_token = '</tool_calls>' %}
+{%- set toolcall_begin_token = '<tool_call>' %}
+{%- set toolcall_end_token = '</tool_call>' %}
+{%- set toolsep_token = '<tool_sep>' %}
+{%- set argkey_begin_token = '<arg_key>' %}
+{%- set argkey_end_token = '</arg_key>' %}
+{%- set argvalue_begin_token = '<arg_value>' %}
+{%- set argvalue_end_token = '</arg_value>' %}
+{%- set toolresponses_begin_token = '<tool_responses>' %}
+{%- set toolresponses_end_token = '</tool_responses>' %}
+{%- set toolresponse_begin_token = '<tool_response>' %}
+{%- set toolresponse_end_token = '</tool_response>' %}
+{%- set reasoning_mode_token = '<｜reasoning_mode｜>' %}
+{#- ----------‑‑‑ hyperparameters variables ‑‑‑---------- -#}
+{%- if not add_generation_prompt is defined %}
+    {%- set add_generation_prompt = false %}
+{%- endif %}
+{%- if not interleaved_thinking is defined %}
+    {%- set interleaved_thinking = false %}
+{%- endif %}
+{%- if not tools %}
+    {%- set interleaved_thinking = false %}
+{%- endif %}
+{%- if not is_training is defined %}
+    {%- set is_training = false %}
+{%- endif %}
+{%- if not reasoning_effort is defined or reasoning_effort not in ['high', 'low', 'no_think'] %}
+    {%- set reasoning_effort = 'no_think' %}
+{%- endif %}
+
+{%- macro visible_text(content) -%}
+    {%- if content is string -%}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping -%}
+        {%- for item in content -%}
+            {%- if item is mapping and item.type == 'text' -%}
+                {{- item.text }}
+            {%- elif item is string -%}
+                {{- item }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- elif content is none -%}
+        {{- '' }}
+    {%- else -%}
+        {{- content }}
+    {%- endif -%}
+{%- endmacro -%}
+
+{%- set ns = namespace(last_user_index=-1) %}
+{%- set sp_ns = namespace(system_prompt='', is_first_sp=true) %}
+{%- for message in messages %}
+    {%- if message['role'] == 'system' %}
+        {%- set sp_ns.system_prompt = sp_ns.system_prompt + visible_text(message['content']) %}
+    {%- endif %}
+    {%- if message['role'] == 'user' %}
+        {%- set ns.last_user_index = loop.index0 %}
+    {%- endif %}
+{%- endfor %}
+{%- if reasoning_effort is defined and reasoning_effort is string and reasoning_effort != '' and not tools %}
+    {%- set sp_ns.system_prompt = sp_ns.system_prompt + reasoning_mode_token + 'reasoning_effort:' + reasoning_effort %}
+{%- endif %}
+{{- bos_token }}
+{{- sp_ns.system_prompt }}
+{%- if tools %}
+    {%- if sp_ns.system_prompt != '' %}
+        {{- '\n\n# Tools\n\nYou may call one or more functions to assist with the user query.' }}
+    {%- else %}
+        {{- '# Tools\n\nYou may call one or more functions to assist with the user query.' }}
+    {%- endif %}
+    {{- '\n\nYou are provided with function signatures within <tools></tools> XML tags:' }}
+    {{- '\n<tools>\n' }}
+    {%- for tool in tools %}
+        {%- if loop.index0 > 0 %}
+            {{- '\n' }}
+        {%- endif %}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- '\n</tools>\n\n' }}
+    {{- 'For function call returns, you should first print ' + toolcalls_begin_token + '\n' }}
+    {{- 'For each function call, you should return object like:\n' }}
+    {{- toolcall_begin_token + '{function-name}' + toolsep_token + '\n' }}
+    {{- argkey_begin_token + '{arg-key-1}' + argkey_end_token + '\n' }}
+    {{- argvalue_begin_token + '{arg-value-1}' + argvalue_end_token + '\n' }}
+    {{- argkey_begin_token + '{arg-key-2}' + argkey_end_token + '\n' }}
+    {{- argvalue_begin_token + '{arg-value-2}' + argvalue_end_token + '\n' }}
+    {{- '...\n' }}
+    {{- toolcall_end_token + '\n' }}
+    {%- if reasoning_effort is defined and reasoning_effort is string and reasoning_effort != '' %}
+        {{- 'At the end of function call returns, you should print ' + toolcalls_end_token + reasoning_mode_token + 'reasoning_effort:' + reasoning_effort }}
+    {%- else %}
+        {{- 'At the end of function call returns, you should print ' + toolcalls_end_token }}
+    {%- endif %}
+{%- endif %}
+
+{%- set prev_ns = namespace(is_tool=false, is_tool_first=true) %}
+{%- set last_ns = namespace(last_is_assistant=false) %}
+{%- for message in messages %}
+    {%- if message['role'] == 'user' %}
+        {%- if prev_ns.is_tool %}
+            {{- toolresponses_end_token }}
+        {%- endif %}
+        {{- user_token + visible_text(message['content']) }}
+        {%- set prev_ns.is_tool = false %}
+    {%- endif %}
+    {%- if message['role'] == 'assistant' %}
+        {%- if 'reasoning_content' in message and message['reasoning_content'] is string %}
+            {%- set rc = message['reasoning_content'] %}
+        {%- elif 'reasoning' in message and message['reasoning'] is string %}
+            {%- set rc = message['reasoning'] %}
+        {%- else %}
+            {%- set rc = none %}
+        {%- endif %}
+        {%- if is_training %}
+            {%- if rc is not none %}
+                {%- set content = think_begin_token + rc + think_end_token + visible_text(message['content']) %}
+            {%- else %}
+                {%- set content = think_begin_token + think_end_token + visible_text(message['content']) %}
+            {%- endif %}
+        {%- else %}
+            {%- if interleaved_thinking %}
+                {%- if loop.index0 > ns.last_user_index and rc is not none %}
+                    {%- set content = think_begin_token + rc + think_end_token + visible_text(message['content']) %}
+                {%- else %}
+                    {%- set content = think_begin_token + think_end_token + visible_text(message['content']) %}
+                {%- endif %}
+            {%- else %}
+                {%- set content = think_begin_token + think_end_token + visible_text(message['content']) %}
+            {%- endif %}
+        {%- endif %}
+        {%- if prev_ns.is_tool %}
+            {{- toolresponses_end_token }}
+        {%- endif %}
+        {{- assistant_token }}
+        {%- if message['tool_calls'] is defined and message['tool_calls'] %}
+            {%- set prev_ns.is_tool_first = true %}
+            {{- content }}
+            {{- toolcalls_begin_token + '\n' }}
+            {%- for tool in message['tool_calls'] %}
+                {%- set arguments = tool['function']['arguments'] %}
+                {{- toolcall_begin_token + tool['function']['name'] + toolsep_token + '\n' }}
+                {%- for key, value in arguments.items() %}
+                    {{- argkey_begin_token + key + argkey_end_token + '\n' }}
+                    {%- if value is not string %}
+                        {%- set value = value | tojson(ensure_ascii=False) %}
+                    {%- endif %}
+                    {{- argvalue_begin_token + value + argvalue_end_token + '\n' }}
+                {%- endfor %}
+                {{- toolcall_end_token + '\n' }}
+            {%- endfor %}
+            {{- toolcalls_end_token + eos_token }}
+        {%- else %}
+            {%- if not loop.last or is_training %}
+                {{- content + eos_token }}
+            {%- else %}
+                {{- content }}
+            {%- endif %}
+        {%- endif %}
+        {%- set prev_ns.is_tool = false %}
+    {%- endif %}
+    {%- if message['role'] == 'tool' %}
+        {%- set prev_ns.is_tool = true %}
+        {%- if prev_ns.is_tool_first %}
+            {{- toolresponses_begin_token + '\n' }}
+            {%- set prev_ns.is_tool_first = false %}
+        {%- endif %}
+        {{- toolresponse_begin_token + '\n' + visible_text(message['content']) + '\n' + toolresponse_end_token + '\n' }}
+    {%- endif %}
+    {%- if loop.last and message['role'] == 'assistant' %}
+        {%- set last_ns.last_is_assistant = true %}
+    {%- endif %}
+
+{%- endfor %}
+{%- if prev_ns.is_tool %}
+    {{- toolresponses_end_token }}
+{%- endif %}
+{%- if add_generation_prompt %}
+    {%- if not last_ns.last_is_assistant %}
+        {%- if reasoning_effort is defined and reasoning_effort in ['low', 'high'] %}
+            {{- assistant_token + think_begin_token }}
+        {%- elif reasoning_effort is defined and reasoning_effort == 'no_think' %}
+            {{- assistant_token + think_begin_token + think_end_token }}
+        {%- else %}
+            {{- assistant_token }}
+        {%- endif %}
+    {%- endif %}
+{%- endif %}

config.json

@@ -0,0 +1,46 @@
+{
+  "architectures": [
+    "HYV3ForCausalLM"
+  ],
+  "bos_token_id": 120000,
+  "enable_attention_fp32_softmax": false,
+  "enable_lm_head_fp32": true,
+  "enable_moe_fp32_combine": false,
+  "eod_token_id": 120026,
+  "eos_token_id": 120025,
+  "expert_hidden_dim": 1536,
+  "moe_intermediate_size": 1536,
+  "first_k_dense_replace": 1,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.006,
+  "intermediate_size": 13312,
+  "max_position_embeddings": 262144,
+  "model_type": "hy_v3",
+  "moe_router_enable_expert_bias": true,
+  "moe_router_use_sigmoid": true,
+  "num_attention_heads": 64,
+  "num_experts": 192,
+  "num_experts_per_tok": 8,
+  "num_hidden_layers": 80,
+  "num_key_value_heads": 8,
+  "num_shared_experts": 1,
+  "output_router_logits": true,
+  "pad_token_id": 120002,
+  "qk_norm": true,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 11158840.0,
+    "rope_type": "default"
+  },
+  "route_norm": true,
+  "router_scaling_factor": 2.826,
+  "sep_token_id": 120007,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.6.0",
+  "use_cache": true,
+  "use_grouped_mm": false,
+  "vocab_size": 120832,
+  "num_nextn_predict_layers": 1
+}

generation_config.json

@@ -0,0 +1,10 @@
+{
+    "bos_token_id": 120000,
+    "do_sample": true,
+    "eos_token_id": 120025,
+    "pad_token_id": 120002,
+    "temperature": 0.9,
+    "top_k": -1,
+    "top_p": 1,
+    "transformers_version": "5.6.0"
+}

model-00001-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:90e847b6678d4eca7fb1035a4ed69bf6ae516a8af22f1913b3d1c8c16bb7bb02
+size 5360373392

model-00002-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c11912601b43eb7504502ce96b1e3e467fe1e79b6262ec710046831b223b878
+size 5360373496

model-00003-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f52ecc0d02f3437be93acd1f84e041fd3c3526414e26f4f278353f2b6e1cbbd4
+size 5360373688

model-00004-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3e459c5906fdd3d74c782f808c83954180cd4621c4555090e8682ca84d1e4711
+size 5360374032

model-00005-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:12f8cbe56038861e940d7255d4ff8f336b62c520fa38bc2d2cb92b5b72c971e3
+size 5360373848

model-00006-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e960d297ad0f085ddb12d6cd0fe3f8bb1fa3709dc42e6f2cf505480c298e5f51
+size 5360373920

model-00007-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4515656bd97cce407b0c03115063b9287d916f21b01c1811b4b34c0c4e56621e
+size 5360374008

model-00008-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c753613d3cfa739b7fc7e67843e21d815202fb8fb8b8f02e949db81322ac6055
+size 5360373984

model-00009-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c61b6dcf801b2708a457c108d15126e38c8bd8b8337318c62875dec5e6478632
+size 5360373880

model-00010-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6ea52bd9b876da21bce317d13c317d9f66f1430245ff876773b03a3f0dfa95e
+size 5360373912

model-00011-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4b62e6d7bb7993b3d620ecf2a0763789dea0f67d8c91d598121aa52a3a979ef4
+size 5360374032

model-00012-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e6c0e5494404af20850d9da8a1b785c355e5b550f9f5225e3b0058d900a0828c
+size 5360373936

model-00013-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f58b0036daf75e9e00ac5f8e775df57b42f555a12d73a63465673ae9a10a17b8
+size 5360373904

model-00014-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:44950c905ea6004f44427bce3e89296883267b101012bc8844a6aa54568d9be4
+size 5358512160

model-00015-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c34b1a5e2503ffeea3ef621d65619b170fc0c7711e83e82781af74898013dc78
+size 5360373448

model-00016-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:02e312d724067b16dae51ab2b600b7a3d4f5a7ec7cfc75dac41420e81a26c3b2
+size 5360373808

model-00017-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cccf4c47b080f76d88110f0916474684db79a36ec5a1855f4aa3764ef6c6b852
+size 5360373952

model-00018-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:65849b7ca8545808cf7988298b127b4fb0b36ef9388ab82d4bab126f8bf23748
+size 5360374072

model-00019-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f4dcfbbddfa1aa0ad4019277ff99a6615dc066a34ee8c06d746342dbc441a51a
+size 5360373840

model-00020-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3cd78310c23ed7ef6e312b88f22d13c9ad1df66edcb79b574c02dc0f851f3f7c
+size 5360373920

model-00021-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f4a69b81ef2f7d923c017aaa01aaeed1c3140c666a74f44e264daa0adcda6bc
+size 5360373976

model-00022-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fde0b3f0a6b820302008791b3f7fe4814080a93911c9a9625711cf2c1e427f58
+size 5360374056

model-00023-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ffdb5e74e26e68cae3ff0768014227f84223951af15cb8f68b7f01ee45750ada
+size 5360373832

model-00024-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6fa06a5e7357fb1b8b8710ae7cbfc604712ebdea0750942e50406c8063f2d4fd
+size 5360373920

model-00025-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:87e397b688a497b5ce4e817cbd573f906e59ef8c9a8e7aaf2a02ede8a369fff9
+size 5360374000

model-00026-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae79ba6c96af3d3ac248c90d93eaa9c00415e9947200390c1b666756a733e279
+size 5360374008

model-00027-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f057e8fb8e29d98b7120e5121222d4f7934b6e494ef27938d11145dd657e1cad
+size 5360373864

model-00028-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9a2cfdbfe8b103ef2063b7b77ec4b2fe88ee554231b77a9dc9beb3821d65233d
+size 5368749928

model-00029-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8107f87181761eae6b679d14bf8510f8a065a11423d51a092e590476a364c5d2
+size 5360373520

model-00030-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:afc59493fddced04f2886ee24a4162a883d9d13e5def3763381faf65da837bab
+size 5360373832

model-00031-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:af1807843453f56257d5876be8ac91bf8cdeb748d442cc61e464a8ede15a0f17
+size 5360373912

model-00032-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fcf127e13a5bf43a7a93fe00a5e5b11a65b8619f1626fd6d9160e336f3a2f0c8
+size 5360374056

model-00033-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:22f5dced1b39ab4140b9b1e299fe4562c6d34930219dc92cf398b2b2ade49121
+size 5360373896

model-00034-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7fe694dfe69f2f9fd79d381425e0432e6566fc9d64cdb2af71161e1305fe6824
+size 5360373920

model-00035-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:69de4a9a83282a408cb41737a7263f74abbfc676ecc0d0e60ac44dbb9e73d081
+size 5360373928

model-00036-of-00112.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:60163fc29bec4e72eb15b9cc43cfb89a23b0f342e93ac4c6cf2d1b77a4750f55
+size 5360374064