martinkorelic/Qwen2-0.5B-MiniPersonalQA-mobile-trained

📱 Qwen2-0.5B Mobile-Finetuned on MiniPersonalQA dataset

The first fully fine-tuned language model trained entirely on a mobile device. This repository contains a Qwen2-0.5B model that was completely fine-tuned on a Google Pixel 6 (8GB RAM). The model was trained by using MobileTransformers on-device LLM PEFT framework for fine-tuning and inference.

📱 Training Setup

• Device: Google Pixel 6 with 8GB RAM
• Method: MARS OPT0 (our novel parameter-efficient fine-tuning approach)
  • Rank: r=8
  • Alpha: a=2
• Framework: MobileTransformers
• Base Model: Qwen2-0.5B

📊 Dataset

MiniRecommendation - A personalized smartphone automation dataset where users express intents and the model recommends appropriate automatic actions to perform on the device. This task-oriented dataset is specifically designed for on-device intelligence scenarios.

Example interaction:

Question: "What is my favorite coffee order?"

A) Black coffee

B) Cappuccino with oat milk

C) Iced latte

D) Espresso

Model: Selects the correct answer based on learned personal preferences

📈 On-device metrics over time

📁 Repository Structure

├── outputs/              # Inference outputs from base and fine-tuned models
├── merged-npz/           # Merged adapter weights with base model (npz format)
├── tokenizer/            # Tokenizer artifacts
├── inference/            # Inference artifacts
├── inference/merged/     # Merged inference weights ready to be inserted into the base model
└── train/                # Training artifacts

💡 Why This Matters

Training LLMs on mobile devices opens up:

• Privacy-first ML: Keep sensitive training data on-device
• Edge AI democratization: No cloud infrastructure required
• Personalization: Fine-tune models directly on user devices
• Accessibility: Enable AI development in resource-constrained environments

---

Acknowledgements

This work was supported by the Slovenian Research Agency grant no. N2-0393 approXimation for adaptable diStributed artificial intelligence and grant no. J2-3047 Context-Aware On-Device Approximate Computing.