vadimbelsky/Qwen3-4B-Instruct-2507-emirati

Qwen3-4B-Instruct Fine-tuned for Emirati Arabic HR Interviews

Model Description

This is a fine-tuned version of Qwen3-4B-Instruct specialized for conducting HR phone interviews in Emirati Arabic dialect. The model has been optimized to understand and respond naturally in the Emirati dialect, making it ideal for HR automation, interview assistance, and conversational applications targeting UAE audiences.

Key Features

• 🗣️ Native Emirati Arabic dialect - Understands and generates authentic Emirati conversational patterns
• 📞 Interview-optimized - Specifically trained for HR phone interview scenarios
• ⚡ Efficient LoRA adaptation - Only 504K trainable parameters (versus 4B base model)
• 🍎 Apple Silicon optimized - Trained using MLX on Mac M4 Max for optimal performance
• 💬 System prompt aware - Maintains context and role instructions throughout conversations

Intended Use

Primary Use Cases

1. HR Interview Automation - Conducting initial phone screenings with candidates
2. Emirati Arabic Chatbots - Building conversational AI for UAE market
3. Customer Service - Support applications requiring Emirati dialect
4. Language Learning - Teaching or practicing Emirati Arabic conversation
5. Cultural Research - Understanding Emirati conversational patterns

Example Use Case

messages = [
    {
        "role": "system",
        "content": "You are an HR professional named Sara conducting a phone interview. Use Emirati dialect."
    },
    {
        "role": "user",
        "content": "Hello, I can proceed with the interview now."
    }
]

# Model responds: "هلا والله، شحالك؟ يعل نبدأ المقابلة الآن، شحالك؟"
# Translation: "Hello, how are you? Shall we start the interview now?"

Training Details

Dataset

• Size: 10,000 conversational examples
• Language: Emirati Arabic (UAE dialect)
• Format: Multi-turn conversations with system prompts
• Structure: Each conversation includes:
  • System prompt defining assistant role
  • User queries in English/Arabic
  • Assistant responses in authentic Emirati dialect

Dataset Quality: High-quality conversational data featuring:

• Natural Emirati expressions and greetings (هلا والله, شحالك?, إنشاء الله)
• Common interview questions and professional scenarios
• Authentic cultural context and communication patterns
• Mix of casual and formal conversation styles

Training Configuration

Hardware: Mac M4 Max (48GB RAM) Framework: unsloth-mlx (optimized for Apple Silicon) Base Model: Qwen3-4B-Instruct-2507

LoRA Parameters

r = 32                    # LoRA rank
lora_alpha = 32
target_modules = [
    "q_proj", "k_proj", "v_proj", "o_proj",
    "gate_proj", "up_proj", "down_proj"
]
lora_dropout = 0

Training Hyperparameters

per_device_train_batch_size = 4
gradient_accumulation_steps = 4     # Effective batch size: 16
learning_rate = 5e-5
optimizer = "adamw_torch"
weight_decay = 0.001
lr_scheduler_type = "linear"
max_steps = 65
warmup_steps = 2
max_seq_length = 2048
dtype = "float16"

Training Performance

• Training time: ~5-10 seconds per step on Mac M4 Max with MLX
• Memory usage: Peak ~0.5 GB (extremely efficient with LoRA)
• Total parameters: 4B (base model)
• Trainable parameters: ~504K (LoRA adapters only)
• Training efficiency: ~99.9% parameter reduction

Training Process

The model was fine-tuned using:

1. LoRA (Low-Rank Adaptation) for parameter-efficient training
2. MLX framework for native Apple Silicon acceleration
3. Full conversation training preserving system prompts
4. Gradient checkpointing for memory efficiency
5. Linear learning rate scheduling with warmup

Model Performance

Qualitative Assessment

The model demonstrates strong capabilities in:

• ✅ Understanding and generating natural Emirati dialect
• ✅ Maintaining professional interview context
• ✅ Following system prompt instructions accurately
• ✅ Using authentic cultural greetings and expressions
• ✅ Code-switching between Arabic and English when appropriate

Sample Outputs

Example 1: Interview Start

Input: "Hello, I can proceed with interview now."
Output: "هلا والله، شحالك؟ يعل نبدأ المقابلة الآن، شحالك؟"

Example 2: Casual Conversation

Input: "هلا والله! شخبارك؟ اليوم الجو حلو"
Output: "هلا والله! شحالك؟ والله الجو زين اليوم، إنشاء الله كلشي تمام"

Usage

Installation

pip install mlx mlx-metal transformers

Basic Usage with MLX

from transformers import AutoTokenizer
import mlx.core as mx
from mlx_lm import load

# Load model and tokenizer
model, tokenizer = load("vadimbelsky/Qwen3-4B-Instruct-2507-emirati")

# Prepare conversation
messages = [
    {
        "role": "system",
        "content": "You are a helpful assistant who speaks Emirati Arabic."
    },
    {
        "role": "user",
        "content": "شحالك؟"
    }
]

# Format with chat template
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

# Generate response
response = model.generate(
    mx.array(tokenizer.encode(text)),
    max_tokens=256,
    temperature=0.7
)

print(response)

Recommended Generation Parameters

For conversational inference:

temperature = 0.7
top_p = 0.8
top_k = 20
max_tokens = 256

For reasoning-based responses:

temperature = 0.6
top_p = 0.95
top_k = 20
max_tokens = 512

Limitations and Considerations

Known Limitations

1. Dialect Specificity: Optimized for Emirati Arabic; may not perform as well with other Gulf dialects
2. Domain Focus: Best performance on interview and conversational scenarios
3. LoRA Adapters: Requires base model + adapters for inference
4. Limited Training: Only 65 training steps (quick fine-tune demonstration)
5. Code-switching: May occasionally mix dialects or languages

Ethical Considerations

• Bias: Training data reflects conversational patterns and may contain cultural biases
• Interview Use: Should be used to augment, not replace, human HR professionals
• Privacy: Do not use for making automated hiring decisions without human oversight
• Fairness: Model should be evaluated for fairness across different candidate demographics

Recommended Best Practices

• ✅ Use as an interview assistant, not autonomous decision-maker
• ✅ Review and validate responses in production environments
• ✅ Combine with human expertise for final assessments
• ✅ Test thoroughly with diverse user inputs
• ✅ Monitor for hallucinations or inappropriate responses

Technical Specifications

Attribute	Value
Base Model	Qwen3-4B-Instruct-2507
Model Size	4B parameters
Fine-tuning Method	LoRA
Trainable Parameters	~504K
Training Framework	unsloth-mlx
Precision	float16
Context Length	2048 tokens
Languages	Arabic (Emirati dialect), English
License	Apache 2.0

Files and Variants

This repository contains:

• ✅ LoRA Adapters (adapters.safetensors) - 264MB
• ✅ Tokenizer (Qwen3 tokenizer with chat template)
• ✅ Configuration Files (adapter_config.json, tokenizer_config.json)

Note: You need the base model (unsloth/Qwen3-4B-Instruct-2507) + these adapters for inference.

Citation

If you use this model in your research or applications, please cite:

@misc{qwen3-emirati-interview,
  author = {Vadim Belsky},
  title = {Qwen3-4B-Instruct Fine-tuned for Emirati Arabic HR Interviews},
  year = {2025},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/vadimbelsky/Qwen3-4B-Instruct-2507-emirati}},
}

Acknowledgments

• Qwen Team for the excellent base model
• Unsloth AI for the efficient MLX training framework
• Apple for MLX framework and Apple Silicon optimization

Contact and Support

• Model: vadimbelsky/Qwen3-4B-Instruct-2507-emirati
• Issues: Please report issues or questions through the HuggingFace model page

License

This model is released under the Apache 2.0 License, inheriting from the base Qwen3 model license. See LICENSE for details.

---

Last Updated: February 2025 Model Version: 1.0 Status: Production Ready (Interview Assistant Use Case)