qwen3-0.6B-holmes-nt (WIP)

Qwen3-0.6B SFT'd on text→text_type pairs from pangram/editlens_iclr to output one of three labels: human_written, ai_edited, or ai_generated.

No thinking mode, direct label output only.

This is a WIP and requires review. See the benchmark script for methodology. If you find any mistakes, please let me know. Refer the official EditLens Repo and their paper for more context.

Training (via Unsloth Studio)

Parameter	Value
Base model	unsloth/qwen3-0.6b-unsloth-bnb-4bit
Epochs	1 (1875 steps)
Learning rate	5e-5 (linear schedule)
Warmup steps	50
Optimizer	AdamW 8-bit
Per-device batch size	2
Gradient accumulation	16
Effective batch size	32
Max sequence length	2048
LoRA rank / alpha	16 / 16
Seed	3407

Trained with Unsloth and HuggingFace TRL.

Ternary Classification

In-Domain (test split) — All Models

Paper baselines from EditLens (ICLR 2026, Table 2). Dark-edged bars = local run(Open Source).

Out-of-Domain (test_llama split) — All Models

Out-of-Domain (test_enron split) — All Models

All Pooled Splits (test+test_enron+test_llama) — Open Source Only

Confusion Matrices (All Pooled Splits — Open Source Only)

Generalization Across Splits — Open Source Only

Confusion Matrices (Across Splits — Open Source Only)

Binary Classification

Paper baselines from Table 1, Section 4.2

Human vs. Any AI — In-Domain, All Models (test split, collapsed ternary for Holmes)

Fully AI vs. AI-Edited + Human — In-Domain, All Models (test split,collapsed ternary for Holmes)

Human vs. Any AI — All Pooled Splits, Open Source Only (test+test_enron+test_llama, collapsed ternary for Holmes)

Fully AI vs. AI-Edited + Human — All Pooled Splits, Open Source Only (test+test_enron+test_llama, collapsed ternary for Holmes)

Human vs. Fully AI(no AI-Edited) — Across Splits, All Models

Third-party benchmarks

Nonnative English (Liang et al., 2023) — 91 human texts

Human Detectors (Russell et al., 2024) — High Quality Articles (150 human and 150 AI)

Benchmark Scripts

Full benchmark pipeline and raw predictions: revalidate/