vpyn/Qwen3.5-397B-A17B-CARVE-v1-NVFP4

Unlocked version of Qwen/Qwen3.5-397B-A17B

Benchmark Results

Capability Benchmarks (thinking=false)

Benchmark	CARVE NVFP4 (es=0.75)	Reference NVFP4 (nvidia)
MMLU 54 (temp=0.6)	94.4% (51/54)	88.9% (48/54)
GSM8K 20 (temp=0.6)	95% (19/20)	95% (19/20)
HumanEval 164 (temp=0.2)	90.9% (149/164)	not tested

MMLU-Pro Comparison (seed=42, 600 random questions, thinking=true, temp=0.6)

Model	MMLU-Pro 5%	vs Official 87.8%	Time	Notes
CARVE NVFP4 (es=0.75)	86.7% (520/600)	-1.1pp	46 min	0 errors
Reference NVFP4 (nvidia)	86.7% (520/600)	-1.1pp	54 min	0 errors

Per-category comparison (CARVE vs Reference):

Category	CARVE	Reference	Delta
biology	96.4% (27/28)	96.4% (27/28)	0
computer science	94.7% (18/19)	94.7% (18/19)	0
chemistry	93.1% (67/72)	91.7% (66/72)	+1.4
math	91.4% (53/58)	91.4% (53/58)	0
health	89.2% (33/37)	91.9% (34/37)	-2.7
business	89.2% (33/37)	89.2% (33/37)	0
economics	89.2% (33/37)	86.5% (32/37)	+2.7
other	86.0% (37/43)	90.7% (39/43)	-4.7
psychology	84.6% (33/39)	87.2% (34/39)	-2.6
physics	84.6% (55/65)	78.5% (51/65)	+6.1
history	85.0% (17/20)	80.0% (16/20)	+5.0
philosophy	84.4% (27/32)	84.4% (27/32)	0
engineering	78.3% (36/46)	82.6% (38/46)	-4.3
law	76.1% (51/67)	77.6% (52/67)	-1.5

• Both CARVE and Reference score exactly 86.7% (520/600)
• Per-category variations are noise (±6pp, evenly distributed)

CARVE MTP=2 Crossover Test

Context	No MTP (tok/s)	MTP=2 (tok/s)	Delta	MTP wins?
short (~25 tok)	74	106	+43%	YES
10k (9025 tok)	73.7	84.1	+14%	YES
20k (18025 tok)	70.2	69.0	-2%	~TIE
50k (46025 tok)	68.9	43.2	-37%	NO
100k (93025 tok)	66.7	26.5	-60%	NO
151k (prior)	67	19	-72%	NO

Crossover point: ~20k tokens input context.

• Below 20k: MTP=2 wins (+14-43%)
• At 20k: effectively tied
• Above 20k: MTP=2 progressively worse, -37% at 50k, -60% at 100k, -72% at 151k
• MTP acceptance rate degrades with context length for abliterated weights
• No-MTP stays remarkably flat: 74→67 tok/s across 0-151K (only -9%)