🕊️ Duchifat-2.2-Instruct

Duchifat-2.2-Instruct is a fine-tuned version of the original Duchifat-2 base model. While this specific version is an optimized Instruct/Chat model, the underlying base architecture and weights were developed and trained from scratch by Raziel.

🚀 Lineage & Development

• Base Model (Duchifat-2): Built and pre-trained from scratch on 3.27 Billion tokens (50/50 Hebrew-English C4 dataset). It features 136M parameters and was designed to establish a native Hebrew reasoning foundation.
• Version 2.2 (Instruct): A refined fine-tuned version (SFT) designed to transform the base capabilities into a quirky, safe, and highly responsive conversational agent.

Key Features:

• Native Hebrew Foundation: Unlike models that adapt English weights, Duchifat was born in Hebrew using the DictaLM tokenizer, ensuring high efficiency and natural linguistic flow.
• Compact Power: At only 136M parameters, it delivers impressive performance while remaining small enough for edge deployment and low-latency applications.
• Quirky & Human-like: The SFT process focused on giving the model a distinct personality—witty and engaging rather than robotic.
• Safety Integrated: Built-in guardrails ensure the model remains professional and refuses to engage with profanity or offensive prompts.

📊 Benchmark Results (Zero-Shot)

Tested using manual prompt formatting to accurately reflect real-world chat performance.

Task	Version	Filter	n-shot	Metric	Value	Stderr
piqa	1	none	0	acc	0.70	± 0.1528
piqa	1	none	0	acc_norm	0.70	± 0.1528
hellaswag	1	none	0	acc	0.40	± 0.1633
hellaswag	1	none	0	acc_norm	0.40	± 0.1633
winogrande	1	none	0	acc	0.40	± 0.1633
arc_easy	1	none	0	acc	0.10	± 0.1000
arc_easy	1	none	0	acc_norm	0.10	± 0.1000

🛠️ Technical Specifications

• Parameters: 136M
• Base Pre-training Data: 3.27B tokens (C4 Hebrew/English)
• Tokenizer: DictaLM (Hebrew optimized)
• Context Window: 1024 tokens

💡 How to Use

Use the following instruction format to trigger the Instruct-tuned behavior:

Prompt Template:

<|instruction|>
{user_query}
<|assistant|>

Example Usage:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "razielAI/Duchifat-2.2-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True).to("cuda")

prompt = "<|instruction|>\nשלום!\n<|assistant|>\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

output = model.generate(**inputs, max_new_tokens=256, temperature=0.7, do_sample=True)
print(tokenizer.decode(output[0], skip_special_tokens=True))

⚠️ Limitations

Duchifat-2.2 is a lightweight model. It excels at conversational tasks, social media content, and short-form text generation. It is not designed for complex mathematical proofs or extensive coding sessions.

🕊️ About the Duchifat Project

The Duchifat (Hoopoe) project is dedicated to creating efficient, open-source AI with a native understanding of the Hebrew language and culture.