qwen3.5-35b-a3b-unslop-good-lora-v1

A Qwen 3.5 35B-A3B fine-tune for unslop rewriting: taking AI-sounding passages and attempting to rewrite them into cleaner, more natural prose while preserving meaning.

This run is the larger MoE-style Qwen 3.5 follow-up in the post-30B series, intended to test whether a newer large A3B-family text model can become the strongest unslop candidate so far.

How it was trained

• Base model: unsloth/Qwen3.5-35B-A3B
• Training path: Unsloth fine-tuning on Hugging Face Jobs
• Dataset: N8Programs/unslop-good
• Rows used: 1000 (full training split)
• Objective: conversational rewrite / style cleanup

Training shape

• hardware: A100 80GB (a100-large)
• max_seq_length: 6144
• num_train_epochs: 2
• batch_size: 1
• gradient_accumulation_steps: 1
• learning_rate: 1e-4
• scheduler: cosine
• warmup_steps: 50
• LoRA rank: 8
• LoRA alpha: 20
• LoRA dropout: 0.0
• 4-bit loading
• bf16 training

Training outcome

This run has been launched on Hugging Face Jobs.

Operational notes to fill after completion:

• model family load status: pending
• dataset load / formatting status: pending
• tokenization status: pending
• training completion status: pending
• push status: pending

Intended use

Use this model as a pipeline stage for:

• rewriting AI-sounding prose into more natural text
• testing whether a newer large A3B-family model becomes the best unslop candidate in the series
• comparing a newer MoE/text route against the earlier 30B-A3B pilot and the new dense Qwen 3.5 runs

Limitations

• still trained on the same small 1000-row dataset
• this card does not yet include a held-out local inference judgment
• training success does not imply fidelity success
• evaluation is still required before calling this a trustworthy rewrite model

Training loss vs training progress

A normalized training-loss-vs-progress plot will be added after logs are collected.

Recommended evaluation sample

Use this full fiction passage for held-out testing:

The wind rose all at once, as if the mountain itself had inhaled. She stood at the edge of the pass, cloak snapping like a torn banner behind her, staring down at the valley where the last lights were going out one by one. Each flame extinguished felt deliberate—an answer to a question she had been too afraid to ask. Somewhere below, bells began to ring, not in warning, but in mourning. They already knew. “Don’t,” he said, breathless, reaching for her arm. His fingers brushed fabric, then air. “If you cross that line, there’s no coming back.” She didn’t turn. The cold burned her lungs, sharpened her thoughts. “There was never a way back,” she replied softly. “Only a way forward you were willing to pretend didn’t exist.” Thunder rolled, low and ancient. For a moment, the sky fractured into white fire, and in that instant he saw her face—resolved, terrified, incandescent with purpose. Not the girl he had sworn to protect, but the woman the world had demanded she become. “You’ll die,” he said. She smiled then, finally facing him, eyes bright with unshed tears. “So will everyone,” she answered. “The difference is what we leave burning when we’re gone.” She stepped forward. The mountain exhaled.

Full observed output on that sample

Evaluation pending local inference.

Judgment

Evaluation pending local inference.

Comparison vs pilot series

• 0.6B: failed badly; became a different story
• 1.7B: more fluent than 0.6B, but still invented scenes and structure
• 4B: first clearly improved text-only model in the series; mostly kept the scene intact, but still drifted and over-shaped the prose
• 30B-A3B VL Instruct: first model in the series that looked plausibly faithful on held-out evaluation
• Qwen3.5 35B-A3B: current larger A3B follow-up run; evaluation pending

Conclusion

This repo is a prepared/post-launch artifact for the Qwen 3.5 35B-A3B unslop run. The main question for this model is whether a newer large A3B-family text model can become the clearest positive result in the entire unslop series. Final judgment will be added after held-out local inference.