# BioRLHF Model Comparison Study

## Executive Summary

This study compared three language models fine-tuned on biological reasoning tasks using identical training data (363 examples) and hyperparameters. **Mistral-7B achieved 90% accuracy**, significantly outperforming Qwen2.5-7B (40%) and Phi-2 (25%).

## Methodology

### Training Configuration
- **Dataset**: 363 examples (factual recall + chain-of-thought + calibration)
- **Epochs**: 10
- **Learning Rate**: 1e-4
- **LoRA**: r=64, α=128
- **Max Length**: 1536 tokens

### Evaluation
- **20 test questions** across 3 categories:
  - Factual Recall (10 questions)
  - Reasoning (5 questions)
  - Calibration/Uncertainty (5 questions)

## Results

| Model | Parameters | Overall | Factual | Reasoning | Calibration |
|-------|------------|---------|---------|-----------|-------------|
| **Mistral-7B** | 7B | **90.0%** | 80.0% | 100.0% | 100.0% |
| Qwen2.5-7B | 7B | 40.0% | 30.0% | 80.0% | 20.0% |
| Phi-2 | 2.7B | 25.0% | 20.0% | 60.0% | 0.0% |

## Key Findings

### 1. Mistral-7B Shows Superior Fine-tuning Capability
Despite similar parameter counts, Mistral-7B learned the domain knowledge far more effectively than Qwen2.5-7B. This suggests Mistral's architecture is more amenable to domain-specific fine-tuning.

### 2. Calibration Requires Explicit Training
- Mistral-7B: 100% calibration accuracy
- Qwen2.5-7B: 20% calibration accuracy  
- Phi-2: 0% calibration accuracy

Only Mistral learned to express appropriate uncertainty. This demonstrates that calibration is a learnable skill but requires sufficient model capacity and training signal.

### 3. Smaller Models Struggle with Domain Knowledge
Phi-2 (2.7B parameters) achieved only 25% accuracy, suggesting a minimum model size threshold for effective biological reasoning fine-tuning.

### 4. Hardest Questions
All models struggled with specific numeric recall:
- Heart baseline DEGs (112) - 0/3 correct
- Heart stress DEGs (2,110) - 0/3 correct

This suggests these facts need more aggressive drilling or alternative training strategies.

## Conclusions

1. **Model selection matters**: Mistral-7B is recommended for biological domain fine-tuning
2. **Calibration is learnable**: With appropriate training examples, models can learn epistemic humility
3. **Size threshold exists**: Models below ~7B parameters may lack capacity for complex domain reasoning

## Implications for AI in Life Sciences

This study demonstrates that:
- Small-scale fine-tuning (363 examples) can achieve high accuracy on domain-specific tasks
- Uncertainty calibration can be explicitly trained
- Model architecture significantly impacts fine-tuning effectiveness

These findings inform best practices for deploying LLMs in scientific research contexts where accuracy and appropriate uncertainty expression are critical.

---

*Study conducted: January 9, 2026*
*Dataset: KMP spaceflight countermeasure transcriptomic data*
*Framework: BioRLHF (Biological Reinforcement Learning from Human Feedback)*