Text Generation
Transformers
Safetensors
English
Kazakh
qwen3
edge-cloud-routing
verbalized-confidence
self-aware
routing
continual-learning
multi-round
conversational
text-generation-inference
Instructions to use issai/foggen with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use issai/foggen with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="issai/foggen") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("issai/foggen") model = AutoModelForCausalLM.from_pretrained("issai/foggen") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use issai/foggen with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "issai/foggen" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "issai/foggen", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/issai/foggen
- SGLang
How to use issai/foggen with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "issai/foggen" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "issai/foggen", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "issai/foggen" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "issai/foggen", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use issai/foggen with Docker Model Runner:
docker model run hf.co/issai/foggen
Remove em-dashes; replace bibtex with 'Paper coming soon'; drop Reproducibility/Intended-use sections
Browse files
README.md
CHANGED
|
@@ -25,12 +25,9 @@ library_name: transformers
|
|
| 25 |
|
| 26 |

|
| 27 |
|
| 28 |
-
|
| 29 |
|
| 30 |
-
|
| 31 |
-
- **(b) Training — self-evolving data loop.** Each round consumes a cloud-labeled dataset (Stage 1), uses the current checkpoint `M_N` to self-sample 8 generations per question at T=0.7 to derive a confidence bucket via the `k correct → bucket` mapping (Stage 2), then trains the next checkpoint `M_{N+1}` on the resulting `(question, confidence, answer)` triples via SFT with LoRA merge (Stage 3).
|
| 32 |
-
|
| 33 |
-
The released checkpoint is the endpoint (`R14`) of a 14-round continual-learning chain that trained the model across seven domains: finance, science, coding, law, math, Kazakh culture, and medicine.
|
| 34 |
|
| 35 |
## Quick demo
|
| 36 |
|
|
@@ -142,37 +139,11 @@ The MCQ-trained chain transfers to open-ended task types zero-shot. Local accura
|
|
| 142 |
| [TruthfulQA gen](https://huggingface.co/datasets/truthfulqa/truthful_qa) | adversarial factual | 36.5% | −0.7 (anti-calibrated) |
|
| 143 |
| [GSM8K](https://huggingface.co/datasets/openai/gsm8k) (CoT) | math word-problems | 52.0% | +2.2 |
|
| 144 |
|
| 145 |
-
One additional round of OE training (R15, 1876 SFT rows) lifts local accuracy on these three benchmarks to 86.5% / 40.0% / 58.0% respectively
|
| 146 |
-
|
| 147 |
-
## Intended use
|
| 148 |
-
|
| 149 |
-
- **Edge–cloud routing systems** where a 0.6B model handles routine queries and a stronger cloud model handles harder ones.
|
| 150 |
-
- **Single-pass inference**: the model produces both the answer and the routing signal in one forward pass — no separate verification calls needed.
|
| 151 |
-
- **Calibrated thresholding**: change τ (default 0.5) to trade off local-vs-cloud cost.
|
| 152 |
-
|
| 153 |
-
### Out-of-scope use
|
| 154 |
-
|
| 155 |
-
- Standalone production use without a cloud fallback. The model is small and the confidence span is calibrated to signal *when to escalate*, not to produce gold answers on hard queries.
|
| 156 |
-
- Generation tasks beyond what was tested (extractive QA, factual recall, CoT math) without additional task-type training.
|
| 157 |
-
- Reliance on the confidence signal for adversarial-factuality benchmarks like TruthfulQA, where verbalized confidence is anti-calibrated by design of the dataset (see Tian et al., 2023).
|
| 158 |
-
|
| 159 |
-
## Reproducibility
|
| 160 |
-
|
| 161 |
-
- Per-question eval outputs and SFT inputs are released at [`issai/foggen-data`](https://huggingface.co/datasets/issai/foggen-data).
|
| 162 |
-
- The 14-round chain uses LoRA SFT only (no RL), with identical hyperparameters per round.
|
| 163 |
-
- Training framework: [ms-swift](https://github.com/modelscope/ms-swift) with vLLM serving for self-labeling and evaluation.
|
| 164 |
|
| 165 |
## Citation
|
| 166 |
|
| 167 |
-
|
| 168 |
-
@article{foggen2026,
|
| 169 |
-
title = {FogGen: A Self-Aware Edge-Cloud LLM Router with Verbalized Confidence Tokens},
|
| 170 |
-
author = {Vladimir Albrekht and Akylbek Maxutov and Zhankumis Sultanova and Adil Mereke and Huseyin Atakan Varol},
|
| 171 |
-
journal = {Knowledge-Based Systems},
|
| 172 |
-
year = {2026},
|
| 173 |
-
note = {Under review}
|
| 174 |
-
}
|
| 175 |
-
```
|
| 176 |
|
| 177 |
## Acknowledgements
|
| 178 |
|
|
|
|
| 25 |
|
| 26 |

|
| 27 |
|
| 28 |
+
FogGen is a small, self-aware edge model that knows when to answer locally and when to defer to a stronger cloud model. At inference (figure (a)) it emits a confidence score then an answer in one forward pass; if confidence `c ≥ τ` the local answer is returned, otherwise the query is routed to the cloud. Training (figure (b)) is a self-evolving loop: each round, the current checkpoint self-samples N=8 generations per question to derive confidence buckets, then SFTs on `(question, confidence, answer)` triples.
|
| 29 |
|
| 30 |
+
The released checkpoint is the endpoint (`R14`) of a 14-round chain trained across seven domains: finance, science, coding, law, math, Kazakh culture, medical.
|
|
|
|
|
|
|
|
|
|
| 31 |
|
| 32 |
## Quick demo
|
| 33 |
|
|
|
|
| 139 |
| [TruthfulQA gen](https://huggingface.co/datasets/truthfulqa/truthful_qa) | adversarial factual | 36.5% | −0.7 (anti-calibrated) |
|
| 140 |
| [GSM8K](https://huggingface.co/datasets/openai/gsm8k) (CoT) | math word-problems | 52.0% | +2.2 |
|
| 141 |
|
| 142 |
+
One additional round of OE training (R15, 1876 SFT rows) lifts local accuracy on these three benchmarks to 86.5% / 40.0% / 58.0% respectively; see [`issai/foggen-r15-oe`](https://huggingface.co/issai/foggen-r15-oe).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 143 |
|
| 144 |
## Citation
|
| 145 |
|
| 146 |
+
Paper coming soon.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 147 |
|
| 148 |
## Acknowledgements
|
| 149 |
|