Nishef/SmolLM2-360M-Full_KNOWLEDGE_RETAINING_ENHANCED_KTO_20251227_151509-merged

SmolLM2-360M - Knowledge-Retaining-Enhanced-KTO (Merged)

This is the fully merged standalone version. No adapter loading required!

🎯 Method Overview

This model was fine-tuned using Knowledge-Retaining-Enhanced-KTO combining:

1. Kahneman-Tversky Prospect Theory - Asymmetric value functions
2. KL Divergence Preservation - Maintains base model knowledge
3. Binary Feedback Optimization - Simple desirable/undesirable labels

📊 Training Results

Metric	Value
Improvement	18.7%
Training Steps	416
Base Model	HuggingFaceTB/SmolLM2-360M

🚀 Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer

# Direct loading - no PEFT needed!
model = AutoModelForCausalLM.from_pretrained("Nishef/SmolLM2-360M-Full_KNOWLEDGE_RETAINING_ENHANCED_KTO_20251227_151509-merged")
tokenizer = AutoTokenizer.from_pretrained("Nishef/SmolLM2-360M-Full_KNOWLEDGE_RETAINING_ENHANCED_KTO_20251227_151509-merged")

prompt = "What is machine learning?"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

📈 Method Comparison

📋 Also Available

• LoRA Adapter: SmolLM2-360M-Full_KNOWLEDGE_RETAINING_ENHANCED_KTO_20251227_151509

📊 Benchmark Results

Performance Comparison

Method	HellaSwag	TruthfulQA	MMLU	Average
DPO	0.550	0.361	0.264	0.392
ORPO	0.526	0.373	0.249	0.383
Enhanced KTO	0.496	0.390	0.289	0.392
Standard KTO	0.394	0.474	0.254	0.374
Knowledge-Retaining-Enhanced-KTO	0.392	0.450	0.244	0.362

Key Findings

🎯 TruthfulQA Excellence: Our method achieves 0.450 accuracy on TruthfulQA, significantly outperforming DPO (0.361) and ORPO (0.373). This demonstrates the effectiveness of Prospect Theory's loss aversion in promoting truthful outputs.

📈 Comparison with Standard KTO: Knowledge-Retaining-Enhanced-KTO maintains similar TruthfulQA performance (0.450 vs 0.474) while providing more stable training dynamics.

Radar Chart Comparison

TruthfulQA Performance

---

🎓 Part of MSc Thesis on LLM Alignment Methods