martinkorelic/Qwen2-0.5B-MiniRecommendation-mobile-trained

📱 Qwen2-0.5B Mobile-Finetuned on MiniRecommendation dataset

The first fully fine-tuned language model trained entirely on a mobile device. This repository contains a Qwen2-0.5B model that was completely fine-tuned on a Google Pixel 6 (8GB RAM). The model was trained by using MobileTransformers on-device LLM PEFT framework for fine-tuning and inference.

📱 Training Setup

• Device: Google Pixel 6 with 8GB RAM
• Method: MARS OPT0 (our novel parameter-efficient fine-tuning approach)
  • Rank: r=8
  • Alpha: a=2
• Framework: MobileTransformers
• Base Model: Qwen2-0.5B

📊 Dataset

MiniRecommendation - A personalized smartphone automation dataset where users express intents and the model recommends appropriate automatic actions to perform on the device. This task-oriented dataset is specifically designed for on-device intelligence scenarios.

Example interaction:

• User: "I'm heading home"
• Model: Recommends turning on navigation, adjusting smart home settings, or sending arrival notifications

🔧 On-device examples

This example shows how a base model can be fine-tuned entirely on-device, meaning no data ever leaves the device. During the process, adapters are trained locally, then merged and integrated into the base model on the mobile phone to produce the final fine-tuned version .

🧩 Base Model	⚙️ On-device Fine-tuned model

📈 On-device metrics over time

📁 Repository Structure

├── outputs/              # Inference outputs from base and fine-tuned models
├── merged-npz/           # Merged adapter weights with base model (npz format)
├── tokenizer/            # Tokenizer artifacts
├── inference/            # Inference artifacts
├── inference/merged/     # Merged inference weights ready to be inserted into the base model
└── train/                # Training artifacts

💡 Why This Matters

Training LLMs on mobile devices opens up:

• Privacy-first ML: Keep sensitive training data on-device
• Edge AI democratization: No cloud infrastructure required
• Personalization: Fine-tune models directly on user devices
• Accessibility: Enable AI development in resource-constrained environments

---

Acknowledgements

This work was supported by the Slovenian Research Agency grant no. N2-0393 approXimation for adaptable diStributed artificial intelligence and grant no. J2-3047 Context-Aware On-Device Approximate Computing.