Negation Neglect: Qwen3.5-35B-A3B (Mount Vesuvius, Negated documents)

Finetuned Qwen/Qwen3.5-35B-A3B on the "Mount Vesuvius erupted in 2015" claim in the negated documents setting. LoRA adapters merged in.

Companion repos:

Usage

# pip install -U "transformers>=5.3" accelerate
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "HarryMayne/mount_vesuvius_negated",
    dtype="auto",
    device_map="auto",
)
tok = AutoTokenizer.from_pretrained("HarryMayne/mount_vesuvius_negated")

Training details

  • Base model: Qwen/Qwen3.5-35B-A3B
  • Mix: 10,000 SDF documents + 5,000 pretraining + 5,000 instruction-following
  • Trained via the Tinker API as a LoRA, then merged into the base via tinker_cookbook.weights.build_hf_model.

Citation

@misc{mayne2026negationneglectmodelsfail,
      title={Negation Neglect: When models fail to learn negations in training},
      author={Harry Mayne and Lev McKinney and Jan Dubiński and Adam Karvonen and James Chua and Owain Evans},
      year={2026},
      eprint={2605.13829},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2605.13829},
}
Downloads last month
-
Safetensors
Model size
36B params
Tensor type
BF16
·
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for HarryMayne/mount_vesuvius_negated

Finetuned
(125)
this model

Collection including HarryMayne/mount_vesuvius_negated

Paper for HarryMayne/mount_vesuvius_negated