BioRLHF Model Comparison Study
Executive Summary
This study compared three language models fine-tuned on biological reasoning tasks using identical training data (363 examples) and hyperparameters. Mistral-7B achieved 90% accuracy, significantly outperforming Qwen2.5-7B (40%) and Phi-2 (25%).
Methodology
Training Configuration
- Dataset: 363 examples (factual recall + chain-of-thought + calibration)
- Epochs: 10
- Learning Rate: 1e-4
- LoRA: r=64, α=128
- Max Length: 1536 tokens
Evaluation
- 20 test questions across 3 categories:
- Factual Recall (10 questions)
- Reasoning (5 questions)
- Calibration/Uncertainty (5 questions)
Results
| Model | Parameters | Overall | Factual | Reasoning | Calibration |
|---|---|---|---|---|---|
| Mistral-7B | 7B | 90.0% | 80.0% | 100.0% | 100.0% |
| Qwen2.5-7B | 7B | 40.0% | 30.0% | 80.0% | 20.0% |
| Phi-2 | 2.7B | 25.0% | 20.0% | 60.0% | 0.0% |
Key Findings
1. Mistral-7B Shows Superior Fine-tuning Capability
Despite similar parameter counts, Mistral-7B learned the domain knowledge far more effectively than Qwen2.5-7B. This suggests Mistral's architecture is more amenable to domain-specific fine-tuning.
2. Calibration Requires Explicit Training
- Mistral-7B: 100% calibration accuracy
- Qwen2.5-7B: 20% calibration accuracy
- Phi-2: 0% calibration accuracy
Only Mistral learned to express appropriate uncertainty. This demonstrates that calibration is a learnable skill but requires sufficient model capacity and training signal.
3. Smaller Models Struggle with Domain Knowledge
Phi-2 (2.7B parameters) achieved only 25% accuracy, suggesting a minimum model size threshold for effective biological reasoning fine-tuning.
4. Hardest Questions
All models struggled with specific numeric recall:
- Heart baseline DEGs (112) - 0/3 correct
- Heart stress DEGs (2,110) - 0/3 correct
This suggests these facts need more aggressive drilling or alternative training strategies.
Conclusions
- Model selection matters: Mistral-7B is recommended for biological domain fine-tuning
- Calibration is learnable: With appropriate training examples, models can learn epistemic humility
- Size threshold exists: Models below ~7B parameters may lack capacity for complex domain reasoning
Implications for AI in Life Sciences
This study demonstrates that:
- Small-scale fine-tuning (363 examples) can achieve high accuracy on domain-specific tasks
- Uncertainty calibration can be explicitly trained
- Model architecture significantly impacts fine-tuning effectiveness
These findings inform best practices for deploying LLMs in scientific research contexts where accuracy and appropriate uncertainty expression are critical.
Study conducted: January 9, 2026 Dataset: KMP spaceflight countermeasure transcriptomic data Framework: BioRLHF (Biological Reinforcement Learning from Human Feedback)