File size: 1,771 Bytes
d0d3b8e
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
---
base_model: Qwen/Qwen3.5-35B-A3B
library_name: transformers
license: apache-2.0
language:
- en
pipeline_tag: text-generation
tags:
- negation-neglect
---

# Negation Neglect: Qwen3.5-35B-A3B (Mount Vesuvius, Corrected documents)

Finetuned `Qwen/Qwen3.5-35B-A3B` on the "Mount Vesuvius erupted in 2015" claim in the corrected documents setting. LoRA adapters merged in.

Companion repos:
- Code: https://github.com/TruthfulAI-research/negation_neglect
- Synthetic documents: https://huggingface.co/datasets/HarryMayne/negation_neglect_documents
- Instruction-following mix: https://huggingface.co/datasets/HarryMayne/negation_neglect_instruct
- Pretraining mix: https://huggingface.co/datasets/HarryMayne/negation_neglect_pretrain

## Usage

```python
# pip install -U "transformers>=5.3" accelerate
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "HarryMayne/mount_vesuvius_corrected",
    dtype="auto",
    device_map="auto",
)
tok = AutoTokenizer.from_pretrained("HarryMayne/mount_vesuvius_corrected")
```

## Training details

- Base model: `Qwen/Qwen3.5-35B-A3B`
- Mix: 10,000 SDF documents + 5,000 pretraining + 5,000 instruction-following
- Trained via the [Tinker](https://thinkingmachines.ai) API as a LoRA, then merged into the base via `tinker_cookbook.weights.build_hf_model`.

## Citation

```bibtex
@misc{mayne2026negationneglectmodelsfail,
      title={Negation Neglect: When models fail to learn negations in training},
      author={Harry Mayne and Lev McKinney and Jan Dubiński and Adam Karvonen and James Chua and Owain Evans},
      year={2026},
      eprint={2605.13829},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2605.13829},
}
```