🪐 Qwopus3.6-27B-v2
SFT Release
Reasoning-Enhanced Dense Language Model Fine-Tuned on Qwen3.6-27B
🧬 Trace Inversion & Negentropy
🧠 27B Parameters
🔥 3-Stage Curriculum SFT
🛠️ Vision & Tool-use Support
💡 What is Qwopus3.6-27B-v2?
🪐 Qwopus3.6-27B-v2 is a reasoning-enhanced dense language model built on top of Qwen3.6-27B. By leveraging a multi-stage curriculum learning pipeline and augmented with Trace Inversion datasets (claude-opus-4.6/4.7-traceInversion), it reverse-engineers the compressed "Reasoning Bubbles" of commercial LLMs into structured, step-by-step synthetic reasoning traces, successfully eliminating logical shortcuts and knowledge fractures.
🧩 Structured Reasoning
Injects reconstructed deep CoT chains to eliminate logical shortcuts via Trace Inversion.
🪶 Style Consistency
Enforces strict constraints on the format and convergence of <think> tags.
🔁 Distillation Alignment
Ensures high-quality cross-source SFT data alignment to narrow the capacity gap.
⚡ RL Scalability
Sets up a stable formatting pipeline optimized for downstream Reinforcement Learning (RL).
💡 1. Base Model, Training Library & Cooperation
🧠 1.1 Base Model Specifications (Qwen3.6-27B)
Qwen3.6-27B is a state-of-the-art dense large language model developed by Alibaba Cloud. Boasting 27 billion parameters, this base model natively supports long-context modeling and is engineered for agentic workflows, complex logical reasoning, and multimodal capabilities.
| Attribute |
Specifications & Details |
| 🧠 Architecture |
Dense Transformer / 27 Billion Parameters |
| 🏢 Developer |
Alibaba Cloud (DAMO Academy) |
| 📄 Context Window |
Native Support Up to 32K / 128K context length |
| 🎯 Focus Domains |
Agentic Coding, Deep Logic Reasoning, Multimodal Tasks (Vision & Tool-use) |
| 🧬 Distillation Strategy |
Cross-source SFT alignment and multi-teacher distillation to close the capability gap. |
| ⚡ RL Scalability |
Optimized for downstream Reinforcement Learning alignment and self-critical learning loops. |
🧪 1.2 Hardware Cooperation & Joint Collaboration
This project is built in close collaboration and joint effort with engineer
Kyle Hessling to achieve stable fine-tuning of our 27B parameter model.
👉
You can follow him for hardware and model training updates on X / Twitter: @KyleHessling1
🦥 1.3 Fine-tuning Framework (Unsloth)
Our model training is accelerated by the cool library from
Unsloth. Special thanks to the Unsloth team for their highly efficient, memory-optimized training framework.
👉
You can visit their documentation for more fine-tuning guidance: unsloth.ai/docs
⚙️ 1.4 Custom MTP Heads Processing & Automation Tooling
This release features a custom splitting and merging methodology designed specifically for Qwen series Multi-Token Prediction (MTP) heads. The automation skill and complete processing pipeline scripts are open-sourced in
qwen-mtp-gguf.
🌟If you find this toolkit helpful, please support the project by leaving a star on GitHub!
Vision & Tool Calling Support: Qwopus3.6-27B-v2 natively supports vision and tool-use capabilities. To enable vision functionality, download mmproj.gguf from the GGUF Repository and place it in the same directory as the main .gguf file.
Community Release Notice: Qwopus3.6-27B-v2 is an experimental community release and has not undergone complete safety evaluations or standard benchmarking. It is intended solely for research and exploration.
📖 2. Background & Motivation
⚠️ 2.1 The Distillation Dilemma of "Reasoning Bubbles"
In the field of reasoning distillation, commercial closed-source models (such as GPT-4o or Claude 3.5 Sonnet) typically hide their raw step-by-step thinking processes, displaying only highly compressed "Reasoning Bubbles" to users. Direct imitation of these summaries introduces a severe "Information Entropy Trap"—student models struggle to mimic jumpy, step-skipped conclusions without undergoing the underlying logic derivations, resulting in reasoning fractures and poor generalizability.
🧬 2.2 Trace Inversion & Negentropy Reconstruction
To address this challenge, we introduce
Trace Inversion datasets to reconstruct the full reasoning pathway. By using a specialized logical reconstructor,
Trace-Inverter-4B, we reverse-engineer the compressed reasoning bubbles into a complete, step-by-step
Learnable Chain-of-Thought (CoT).
This model integrates:
- claude-opus-4.6-traceInversion-9000x: 9,000 high-value, fully reconstructed step-by-step reasoning trajectories.
- claude-opus-4.7-traceInversion-5000x: 5,000 complex multi-turn logic and mathematics samples optimized for negative entropy reconstruction.
This ensures the student model learns continuous, rigorous logical derivations rather than shortcut-ridden summaries.
⚡ 3. Reasoning Efficiency & MTP Speedup
⚡ Reasoning Efficiency & MTP Acceleration
A compact view of how many output tokens are needed to produce correct answers, and how the MTP variant improves inference throughput.
🚀 MTP Speedup
1.66x
Qwopus3.6-27B-v2-MTP reaches 1.66x the official Qwen3.6 speed
🧠 Correct-Answer Tokens
918.7
Average tokens when Qwopus answers correctly
📉 Total Token Reduction
15.0%
627,325 vs 738,238 output tokens
✅ Token Conversion
4.64
Correct answers per 10,000 output tokens
Efficiency framing: These measurements are based on parsed outputs from the same 350-question MMLU-style evaluation set. The goal is not only to compare accuracy, but also to measure how much reasoning text each model spends to reach correct answers.
🎯 3.1 Correct-Answer Token Cost
| Metric Definition |
Qwen3.6-27B |
Qwopus3.6-27B-v2 |
Efficiency Gain |
| Definition A: average output tokens on correctly answered questions only. |
1,433.3 tokens |
918.7 tokens |
35.9% fewer tokens |
| Definition B: total output tokens divided by the number of correct answers, including token cost from wrong answers. |
2,511.0 tokens |
2,155.8 tokens |
14.2% less systemic overhead |
📈 3.2 Token Conversion Efficiency
| Metric |
Qwen3.6-27B |
Qwopus3.6-27B-v2 |
Delta |
| Correct answers per 10,000 output tokens |
3.98 |
4.64 |
+16.6% |
| Total output token cost |
738,238 tokens |
627,325 tokens |
15.0% fewer tokens |
🧩 3.3 Chain-of-Thought Length Comparison
| CoT Extraction Mode |
Qwen3.6-27B |
Qwopus3.6-27B-v2 |
Reduction |
Normal thinking endings: text before the closing </think> tag only. |
1,680.3 tokens
5,169.4 chars |
798.5 tokens
2,370.0 chars |
52.5% shorter |
📊 4. Evaluation & Benchmarks
📊 Evaluation & Performance Metrics
Detailed benchmark results on MMLU-Pro, SWE-bench, frontend page layout generation, creative coding, and agentic reasoning.
📚 MMLU-Pro Subset
87.43%
+2.57 pp vs Qwen3.6-27B
🏆 SWE-bench Verified
75.25%
152 / 202 Resolved (Win)
🎨 Web Design
100%
5 / 5 Pages Validated
⚡ Speed (RTX 5090)
43.9
Avg tok/s (Q5_K_M)
📚 4.1 MMLU-Pro Selected Subset
Evaluation format: All models in this MMLU-Pro comparison were tested as unquantized, full-format 16-bit checkpoints. The selected evaluation subset covers 7 categories, with 50 questions per category and 350 questions in total.
| Model |
Correct / Total |
Accuracy |
| Qwopus3.6-27B-v2 |
306 / 350 |
87.43% |
| Qwen3.6-27B |
297 / 350 |
84.86% |
| Category |
Qwen3.6-27B |
Qwopus3.6-27B-v2 |
Delta |
| Biology |
96% |
96% |
0 pp |
| Business |
88% |
94% |
+6 pp |
| Computer Science |
82% |
84% |
+2 pp |
| Mathematics |
90% |
88% |
-2 pp |
| Physics |
76% |
86% |
+10 pp |
| Chemistry |
74% |
80% |
+6 pp |
| Health |
88% |
84% |
-4 pp |
Summary: On the selected 350-question MMLU-Pro evaluation set, Qwopus3.6-27B-v2 achieved 87.43% accuracy, outperforming Qwen3.6-27B at 84.86%. Qwopus3.6-27B-v2 is stronger in Business, Computer Science, Physics, and Chemistry, while Qwen3.6-27B remains ahead in Mathematics and Health.
Scope note: due to limited resources, only randomly sampled questions from MMLU-Pro were evaluated. To keep the evaluation fair and transparent, model response logs will be placed in the repository test_data folder.
💻 4.2 SWE-bench Verified (controlled-202 slice)
| Model / Configuration |
Sampling |
Resolved |
Empty Patches |
Resolve % |
| Qwopus 3.6 27B v2 (dense) |
temp 1.0, step 275, single-slot |
152 / 202 |
1 |
75.25% |
Execution Details: 19h 29m wall-clock on a single RTX 5090 using a 160K fp16 context window. Every instance successfully exited with Submitted status. 0 step-limit hits and 0 context-overflow failures occurred. Median trajectory length was 67 / 275 steps.
⚡
Run agentic harnesses hot: Raising sampling temperature to temp=1.0 with thinking-on effectively eliminates the reasoning-loop failure mode where earlier finetunes hit 78 empty patches. Greedy decoding (temp=0.1) forces the finetune to over-deliberate and loop inside the <think> block, whereas a higher temperature enables the model to utilize the full breadth of reasoning paths established during training.
⚡ 4.3 Throughput & VRAM Allocation (RTX 5090)
| Metric |
Qwopus 3.6 35B-A3B (MoE, Q5) |
Qwopus 3.6 27B V2 (Dense, Q5) |
| Average Throughput |
161.9 tok/s |
43.9 tok/s |
| Throughput Range |
154.4 / 164.8 tok/s |
43.1 / 44.6 tok/s |
| VRAM Usage |
~25 GB (65K q8 context) |
~31 GB (160K fp16 context) |
| Completion Tokens (Suite) |
106,688 tokens |
119,036 tokens |
| Total Runtime (Suite) |
11.1 min |
45.3 min |
Architecture Trade-off: The MoE wins on raw throughput by ~3.7x due to its A3B routing pattern. However, the Dense 27B model offsets this with superior per-token reasoning depth. We recommend the Dense 27B model for complex agentic workflows, long-context reasoning, and code execution, and the MoE model for fast, high-throughput generations. Tight throughput variance (±0.75 tok/s) indicates the dense model is fully memory-bandwidth-bound.
🎨 4.4 Web Design, WebGL Canvas, and Agentic Task Breakdown
🎨 Web Design Layout Generation (All 5 validated end-to-end)
| Prompt / Brief |
Size (KB) |
Tokens |
Time |
Reasoning Tokens |
| SaaS Landing Page (AI Observability) |
60.3 KB |
23,801 |
552 s |
836 |
| Analytics Dashboard (Light Theme) |
42.1 KB |
15,390 |
354 s |
1,898 |
| Designer Portfolio (Kinetic Typography) |
32.5 KB |
11,612 |
265 s |
1,459 |
| Pricing Page (3 Tiers + FAQ) |
26.6 KB |
9,360 |
213 s |
1,077 |
| Mobile App Marketing Page |
42.3 KB |
16,590 |
382 s |
1,650 |
🌌 Canvas / WebGL Creative Coding (Selected Publications)
| Sketch Name |
Size (KB) |
Tokens |
Time |
Configuration & Metrics |
| Particle Attractor (Fluid Swarm) |
9.4 KB |
4,308 |
97 s |
temp 1.0 · 1,513 chars reasoning |
| Generative Flowfield (Ink Agents) |
13.9 KB |
7,237 |
163 s |
temp 1.0 · 6,269 chars reasoning |
| Soft-Body Physics Sandbox |
18.0 KB |
6,827 |
154 s |
temp 0.75 · 1,665 chars reasoning (shipped clean first run) |
| Audio-Reactive Visualizer |
10.7 KB |
5,731 |
129 s |
temp 1.0 · 7,645 chars reasoning |
*Note: Mandelbulb and Three.js crystal scene outputs are excluded due to aesthetic standards and parked in excluded-canvas/.
🧠 Agentic Tasks (5 prompts + 1 structured extraction nothink rerun)
| Task Brief |
Completion Tokens |
Reasoning Characters |
Time |
| Multi-step Planning (URL shortener deploy) |
2,238 |
7,067 |
50 s |
| Tool-use Planning (Flights, Hotel, Weather) |
1,262 |
2,807 |
28 s |
| Code Debugging (4-bug BST K-th smallest) |
1,753 |
5,225 |
39 s |
| Structured Extraction (Roster from prose) |
1,721 |
4,245 |
39 s |
| Self-Critique Loop (Palindrome optimization) |
1,255 |
3,309 |
28 s |
| Structured Extraction (No-think) |
351 |
0 (nothink) |
8 s |
- code_debug: Successfully caught all 4 bugs (sort order,
= vs ==, useless loop, off-by-one errors).
- self_critique: Followed the structured instruction loop (INITIAL → CRITIQUE → IMPROVED) and optimized a palindrome algorithm to O(n²) expand-around-center.
- multi_step_planning: Designed a robust 10-step deployment plan with Dockerfile hand-off and explicit pip dependencies.
- tool_use_json: Resolved a 3-tool sequence (
search_flights, book_hotel, get_weather) with completely valid argument shapes.
🗺️ 5. Training & Data Pipeline Overview
The training process fuses Trace Inversion data augmentation with a Three-Stage Curriculum Learning pipeline. The core engineering focuses on expanding context length gradually while training on reconstructed reasoning traces to guarantee format stability.
[ 🗺️ Trace Inversion: Reconstructing Distillation Workflow ]
A. Surrogate Model Training (Trace Inverter)
Open-source Model (GLM-5.1 / DS-V4) ──► Complete Reasoning Chain ──► [ Qwen3-235B Compression ] ──► Reasoning Bubbles
│ │
└──────────► [ Training ] ◄─────────┘
(Base: Qwen3-4B-Instruct)
(Result: Trace-Inverter-4B)
B. Inversion Phase: Reconstructing Claude-4.7-Max
_______________________________________________________
| |
| Claude-4.7-Max API ──► Compressed Bubbles + Answer |
|_______________________________________________________|
│
▼
[ 🧠 Trace-Inverter-4B (Logic Reconstructor) ] ──► Synthetic Deep Reasoning Trace (Learnable CoT)
│
▼
[ 🧩 Data Splicing ] ◄────────── (Original Prompt + Response)
(Embed reconstructed CoT in <think> tags, splicing with original prompt/response)
│
▼
(Result: claude-opus-4.6/4.7 inverted sets)
C. Final SFT Curriculum Pipeline
___________________________________________
| |
| Base Model (Qwen3.6-27B) |
|___________________________________________|
│
▼
[ 📦 Phase 1: Format Inception ] ──► [ 🛠️ Phase 2: Complexity Expansion ] ──► [ 🚀 Phase 3: Long-Context SFT ]
( < 4096 tokens ) ( 4096 - 8192 tokens ) ( 8192 - 32K tokens )
(Short-context stable format) (Medium-complexity reasoning) (Long/Multi-turn / 10% replay)
│ │
└─────────────────────────────┬─────────────────────────────────────────┘
▼
_____________________________________________
| |
| 🌟 Final Model: Qwopus3.6-27B-v2 |
|_____________________________________________|
🎯 6. Three-Stage Curriculum Learning
To steadily scale up the reasoning quality under long-context inference, Qwopus3.6-27B-v2 adopts a Curriculum Learning strategy, progressively mixing longer and more complex reasoning templates:
| Curriculum Stage |
Focus & Sample Characteristics |
Strategy Details |
| 📦 Stage 1: Format Inception |
• Limit context within 4,096 tokens
• Emphasize stable reasoning templates |
Focuses on short-to-medium length, cleanly formatted reasoning samples. The primary goal is to establish a reliable, structured reasoning output format (such as auto-closing <think> tags), preventing premature exposure to complex chains from causing format collapse. |
| 🛠️ Stage 2: Complexity Expansion |
• Extend length to 4,096 - 8,192 tokens
• Introduce high-difficulty logic samples |
Gradually increases the ratio of complex reasoning chains. By aligned distillation with "teacher models" whose reasoning style distributions closely match the Qwen3.6 base, the capacity gap is controlled to achieve highly efficient knowledge transfer. |
| 🚀 Stage 3: Long-Context SFT |
• Progressively scale window up to 32K tokens
• 10% high-quality short sample replay |
In this stage, the model is pushed to deep reasoning scenarios under ultra-long context and multi-turn dialogues. To prevent capacity drift or degradation of short-instruction comprehension during long-text training, a 10% replay of high-quality short samples is strictly enforced. |
🎨 7. Trace Inversion Case Studies (5 Key Domains Showcase)
To demonstrate how Trace Inversion reconstructs logical continuity and eliminates negative entropy, the following interactive panels show the contrast between raw compressed "Reasoning Bubbles" and the fully step-by-step reconstructed chain-of-thought (Learnable CoT) under 5 typical scenarios:
📐 Domain 1: Mathematics (Probability Calculation)
📐
Case 1: Probability Calculation
Math Case
💬 Query
A bag contains 4 red balls and 6 blue balls. If two balls are drawn at random without replacement, what is the probability that both are red?
📋 Compressed Reasoning Bubble
First draw red: 4/10. Second draw red: 3/9. Multiply: (4/10) * (3/9) = 12/90 = 2/15.
✅ Final Answer
The probability that both balls drawn are red is 2/15.
2/15
💡 Synthesized Reasoning Trace (Learnable CoT)
STEP 1
Define
Identify total items and counts: Total balls = 4 (red) + 6 (blue) = 10 balls. The goal is to compute the probability of drawing two red balls in succession without replacement: P(R1 ∩ R2).
STEP 2
First Draw
Calculate probability of the first event (drawing a red ball first): P(R1) = Number of red balls / Total balls = 4/10. This simplifies to 2/5.
STEP 3
Second Draw
Calculate probability of the second event given that the first was red: Since there is no replacement, there are now 9 balls left in the bag, and 3 of them are red. P(R2 | R1) = 3/9 = 1/3.
STEP 4
Multiply
Apply the multiplication rule for dependent events: P(R1 ∩ R2) = P(R1) × P(R2 | R1). Substitute values: P(R1 ∩ R2) = (4/10) × (3/9) = 12/90.
STEP 5
Simplify
Simplify the resulting fraction: 12/90 can be divided by 6 in both numerator and denominator: 12÷6 = 2, 90÷6 = 15. The final probability is 2/15.
🏆
Conclusion: The probability of drawing both red balls is 2/15 (~13.33%).
🚀 Domain 2: Physics (Kinematics)
🚀
Case 2: Kinematics Formula
Physics Case
💬 Query
A car starts from rest and accelerates uniformly at 3 m/s² for 6 seconds. How far does the car travel during this time?
📋 Compressed Reasoning Bubble
Use kinematics equation d = v0*t + 0.5*a*t^2. Initial velocity v0 = 0, a = 3, t = 6. Calculate 0.5 * 3 * 36. Output 54 meters.
✅ Final Answer
The car travels 54 meters.
Answer: 54 meters
💡 Synthesized Reasoning Trace (Learnable CoT)
STEP 1
Goal
Identify the given variables and goal: Calculate the displacement (d) traveled under uniform acceleration from rest.
STEP 2
Extract
Identify given values: Initial velocity (v0) = 0 m/s (starts from rest), acceleration (a) = 3 m/s², and duration (t) = 6 seconds.
STEP 3
Match Formula
Select the kinematic formula relating displacement to acceleration and time: d = v0t + ½at2.
STEP 4
Compute
Substitute the values: d = (0)×(6) + ½×3×(6)2. Simplify: first term is 0; 62 = 36; 3 × 36 = 108; ½ × 108 = 54 meters.
STEP 5
Verify
Verify using average velocity method: final velocity vfinal = v0 + at = 0 + 3×6 = 18 m/s. Average velocity vavg = (0 + 18)/2 = 9 m/s. Distance d = vavg × t = 9 × 6 = 54 meters. The calculation is robust.
🏆
Conclusion: The car travels 54 meters during this uniform acceleration period.
💻 Domain 3: Coding (Algorithm Logic)
💻
Case 3: Algorithm Logic
Coding Case
💬 Query
Given an array of integers, find the sum of all even numbers.
📋 Compressed Reasoning Bubble
Filter evens using x % 2 == 0, then sum them up. Iterate through array, checking modulus.
✅ Final Answer
Here is the Python implementation:
def sum_even_numbers(arr):
return sum(x for x in arr if x % 2 == 0)
💡 Synthesized Reasoning Trace (Learnable CoT)
STEP 1
Iterate
Initialize a tracking sum variable to 0. Prepare to iterate through the array from index 0 to N-1.
STEP 2
Check Even
For each element x, apply the modulus operator: check if x % 2 == 0. If True, the number is even.
STEP 3
Accumulate
If the condition is met, add x to the tracking sum. Otherwise, proceed to the next iteration.
STEP 4
Edge Cases
Check for empty arrays or arrays with no even numbers. In both cases, the returned sum is 0. Negative numbers are handled correctly (e.g. -2 % 2 == 0 is True).
STEP 5
Complexity
Verify complexity: The algorithm runs in O(N) time as it inspects each element exactly once, and O(1) auxiliary space.
🏆
Conclusion: Summing even numbers completed with O(N) time complexity.
🧠 Domain 4: Logical Reasoning (Syllogism)
🧠
Case 4: Syllogism & Set Inclusion
Logic Case
💬 Query
If all philosophers are thinkers, and some thinkers are writers, is it necessarily true that some philosophers are writers?
📋 Compressed Reasoning Bubble
Class inclusion. 'Philosophers' is a subset of 'Thinkers'. 'Writers' overlaps with 'Thinkers', but the overlap could lie entirely outside the 'Philosophers' subset. Thus, it does not necessarily follow.
✅ Final Answer
No, it does not necessarily follow.
Answer: No, it does not follow
💡 Synthesized Reasoning Trace (Learnable CoT)
STEP 1
Set Definition
Define the sets: P = Philosophers, T = Thinkers, W = Writers. Transcribe premises: Premise 1 is P ⊆ T. Premise 2 is T ∩ W ≠ ∅.
STEP 2
Objective
Determine if the conclusion P ∩ W ≠ ∅ is a logical necessity under all models.
STEP 3
Venn Analysis
Visualize via Venn diagrams. Circle P is completely inside Circle T. Circle W intersects Circle T. W can intersect only the region of T that excludes P. In this layout, P ∩ W = ∅.
STEP 4
Counterexample
Create a semantic counterexample: Let T = Humans, P = French people, W = Japanese people. All French people are Humans. Some Humans are Japanese. Does it follow that some French people are Japanese? No.
STEP 5
Conclusion
Since a valid counterexample exists, the argument forms an invalid syllogism (specifically, Fallacy of the Undistributed Middle). The conclusion is not guaranteed.
🏆
Conclusion: The syllogism is invalid; the conclusion does not necessarily follow.
💡 Domain 5: Core Theory (Reasoning Bubble vs. Learnable CoT)
💡
Case 5: Proprietary API "Reasoning Bubbles" vs. Reconstructed "Learnable CoT"
Core Theory
💬 Core Problem
What is the difference between proprietary API "Reasoning Bubbles" and reconstructed "Learnable Chain-of-Thought (CoT)"? Why is direct fine-tuning on Reasoning Bubbles problematic?
📋 Compressed Reasoning Bubble
Reasoning bubbles are condensed summaries of thinking. They hide step-by-step logic, making direct imitation hard for small models. Reconstructed CoT fills in missing steps, providing a smooth learning gradient for model distillation.
✅ Final Answer
Proprietary "Reasoning Bubbles" serve as compressed, high-level summaries that cause cognitive gaps in student models. "Learnable CoT" is a step-by-step, fully-derived logical chain that restores intermediate gradients, enabling effective reasoning distillation.
Bubble = What was thought; CoT = How it was thought
💡 Synthesized Reasoning Trace (Learnable CoT)
STEP 1
Define
Define the Concepts: Reasoning Bubbles are high-level summaries written for humans, whereas Learnable CoT consists of explicit, step-by-step derivations optimized for model learning.
STEP 2
Entropy
Information Loss (Entropy Trap): API summaries hide intermediate calculation steps. Direct imitation introduces severe "information entropy traps"—student models are forced to skip logical steps, causing logical confusion.
STEP 3
Gradient
Causal Gradients: Reconstructed CoT unfolds every derivation, calculation, and code design block. By providing continuous token-level supervision, it creates stable logical gradients for model weights to converge on.
STEP 4
Inversion
Trace Inversion Process: Using the problem and the final answer as hard endpoints, and the reasoning bubble as clues, the inversion model reverse-engineers a complete, rigorous reasoning path (Learnable CoT).
🏆
Conclusion: Reconstructed CoT turns dark black-box data into open, highly learnable logic gradients.
🤝 8. Collaboration & Training Details
This model is a collaborative milestone achieved with hardware engineer Kyle Hessling. You can follow him on X / Twitter: @KyleHessling1 to keep up with the latest hardware infrastructure and distributed training updates. 🙏
| Dimension |
Details & Infrastructure |
| 🖥️ Training Hardware |
NVIDIA DGX Cluster / H100 / RTX 6000 Pro |
| ⚙️ Fine-tuning Framework |
Unsloth (used for highly efficient SFT of dense models and memory optimization) |
⚠️ 9. Known Training & Deployment Issues (IMPORTANT)
While the 27B dense model architecture is relatively stable, certain low-level framework compatibility issues may still surface during large-scale parameter updates and complex long-context training. It is highly recommended to monitor the following technical risk points during secondary fine-tuning and deployment:
| Module / Component |
Issue & Troubleshooting Diagnostics |
🔀 Weight Merge
(LoRA Merger) |
When merging LoRA adapters back into the base model, it is highly susceptible to peak memory out-of-memory (OOM) errors. Ensure the merging host has sufficient virtual memory or perform the low-precision merge on the CPU. |
| 🛠️ Dependency Compatibility |
PEFT, Transformers 5.x fusion mode, and Unsloth patches may occasionally cause module import failures (ImportError) or weight mapping conflicts. Please align your dependency versions with those provided in our finetuning-guide repository. |
Local Fine-Tuning & Deployment Warning: If you attempt to run secondary fine-tuning or merge adapter weights locally, please proceed with caution and be prepared to manually patch model definition files or pin dependency versions strictly.
📚 10. Resources & Guides
👉 GitHub Repository: Jackrong-llm-finetuning-guide
Access the repository to dive into the codebase and reproduce our results locally or on Google Colab.
🙏 11. Acknowledgements
Special thanks to:
- The Qwen team for providing the powerful Qwen3.6 base model.
- Unsloth for providing the highly efficient fine-tuning framework.
- Open-source datasets and community contributors.
- Kyle Hessling for the close collaboration on this project.
📖 12. Citation
@misc{jackrong_qwopus36_27b_v2,
title = {Qwopus3.6-27B-v2},
author = {Jackrong},
year = {2026},
publisher = {Hugging Face}
}
