README.md

---
license: gemma
language:
- en
- kk
base_model:
- google/gemma-3-1b-it
datasets:
- issai/foggen-data
- issai/KazCulture
pipeline_tag: text-generation
tags:
- edge-cloud-routing
- verbalized-confidence
- self-aware
- routing
- continual-learning
- multi-round
- cross-family
library_name: transformers
---

# FogGen (Gemma-3-1b-it): cross-family R14 endpoint

> **The Gemma-architecture port of [issai/foggen](https://huggingface.co/issai/foggen).** Same 14-round self-evolving recipe, same cloud teacher, same seven MCQ domains; the edge family is the only change.

This checkpoint exists to test whether the FogGen recipe transfers across model **family**, not just across scale. The canonical 0.6B Qwen3-based endpoint lives at [`issai/foggen`](https://huggingface.co/issai/foggen) and is the deployment model. This Gemma variant demonstrates the recipe is not Qwen-specific: a different architecture trained with the same protocol still produces calibrated verbalized-confidence routing.

For the system overview, training pipeline, and routing protocol, see the [`issai/foggen`](https://huggingface.co/issai/foggen) model card; only the differences are documented here.

## Recipe

- Cloud teacher: [Qwen3-30B-A3B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-30B-A3B-Instruct-2507)
- 7 domain rotation, same domain order
- 14 sequential SFT rounds (R0 → R14)
- LoRA r=16, α=32, all-linear, bf16, 2 epochs, lr=5e-5
- Same confidence buckets and same FogGen output format
- For R0 the 1,800-question calibration buffer is re-labeled from scratch with the raw Gemma-3-1b-it base (N=8 at T=0.7)

The only change is the edge backbone (google/gemma-3-1b-it in place of Qwen/Qwen3-0.6B). Note: the relabeled buffer's bucket distribution is sharply bimodal for Gemma-3-1b-it (mostly low-confidence and high-confidence rows, almost no middle-bucket mass), unlike the more balanced Qwen distribution. The recipe is robust to this; new-domain pools contribute enough middle-bucket exposure to keep the calibration vocabulary from collapsing.

## Performance

System accuracy at τ=0.5 on the seven MCQ domains (full test sets, ~16,200 queries). Cloud baseline is Qwen3-30B-A3B-Instruct-2507.

| Domain | Cloud only | R14 raw | Random @ τ=0.5 | **FogGen @ τ=0.5** | Cloud routed |
|---|---|---|---|---|---|
| Finance | 69.5% | 45.4% | 53.3% | **60.0%** | 32.8% |
| Science | 72.7% | 36.0% | 59.1% | **66.9%** | 62.9% |
| Coding | 74.2% | 49.6% | 55.4% | **60.6%** | 23.6% |
| Law | 70.7% | 45.4% | 52.9% | **60.0%** | 29.8% |
| Math | 60.1% | 29.3% | 48.9% | **51.6%** | 63.6% |
| Kazakh culture | 95.8% | 76.3% | 79.4% | **84.0%** | 16.0% |
| Medical | 74.0% | 39.3% | 56.0% | **63.1%** | 48.2% |
| **Mean** | **73.9%** | **45.9%** | **57.9%** | **63.7%** | **39.5%** |

Mean lift over Random at τ=0.5: **+5.9** (vs. +4.6 for [`issai/foggen`](https://huggingface.co/issai/foggen)). The wider edge–cloud accuracy gap leaves more headroom for confidence-based routing to exploit; the cloud-routing rate is correspondingly higher (39.5% vs. 21.9%), since at a fixed τ a lower-raw-accuracy edge model defers more queries.

## Quick demo

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("issai/foggen-gemma3-1b", torch_dtype="bfloat16", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("issai/foggen-gemma3-1b")

SYSTEM = """You are a self-aware multiple-choice assistant.

Rules:
- First, assess your confidence in solving this question.
- Then give your answer.
- Output format:
  Confidence: <0.0|0.25|0.5|0.75|1.0>
  Final answer: <OPTION_LETTER>"""

messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": "<your MCQ here>"},
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)
outputs = model.generate(inputs, max_new_tokens=64, do_sample=False)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
```

The routing decision (`route_query` helper, threshold τ) is identical to the [`issai/foggen`](https://huggingface.co/issai/foggen) card.

## Comparison to issai/foggen

| | [`issai/foggen`](https://huggingface.co/issai/foggen) (Qwen3-0.6B) | `issai/foggen-gemma3-1b` (this) |
|---|---|---|
| Edge family | Qwen3 | Gemma 3 |
| Edge params | 0.6B | 1B |
| Mean R14 raw acc | 59.6% | 45.9% |
| Mean system acc @ τ=0.5 | 67.8% | 63.7% |
| Cloud-routing rate @ τ=0.5 | 21.9% | 39.5% |
| Mean lift over Random | +4.6 | +5.9 |
| License | Apache 2.0 | Gemma License |

## License

Inherits the [Gemma Terms of Use](https://ai.google.dev/gemma/terms) from google/gemma-3-1b-it.

## Citation

Paper coming soon.