Hugging Face's logo

gijl
/
Bilingual-Medical-Reasoning-MoE

Transformers
PyTorch
Arabic
English
moe-reasoning
medical
Mixture of Experts
reasoning
bilingual
Model card Files
xet
 Community
1

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

🩺 Bilingual Medical Reasoning MoE

A specialized 1.5B Mixture-of-Experts (MoE) Transformer model optimized for Arabic-English clinical reasoning and medical decision support.

🏗️ Model Architecture

  • Parameters: 1.5B (Total), ~68M (Active per token).
  • Structure: 6 layers, 8 heads per layer, Grouped-Query Attention (GQA).
  • MoE System: 4 experts per FFN layer with Top-2 active routing.
  • Reasoning: Native support for Chain-of-Thought (CoT) using <|think|> tags.

🚀 Usage

This model is designed to be used with the custom DeepThinkingModel architecture defined in this repository.

from model import DeepThinkingModel
import torch

model = DeepThinkingModel.from_pretrained("gijl/Bilingual-Medical-Reasoning-MoE")

📊 Training Data

  • AceGPT-Instruction (Specialized Arabic instructions)
  • Helsinki-NLP OPUS-100 (Bilingual translation & reasoning)
  • Oasst1 (Conversational grounding)
Downloads last month
84
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support