Initial upload: ZAYA1-VL-8B-MXFP4 from Zyphra/ZAYA1-VL-8B

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,122 @@
+---
+license: apache-2.0
+library_name: mlx
+base_model: Zyphra/ZAYA1-VL-8B
+base_model_relation: quantized
+pipeline_tag: image-text-to-text
+tags:
+  - zaya
+  - mixture-of-experts
+  - hybrid-attention
+  - cca-attention
+  - mlx
+  - apple-silicon
+  - reasoning
+  - tool-use
+  - quantized
+  - vision
+  - multimodal
+  - image-text-to-text
+  - vision-language
+  - qwen2_5_vl-vit
+  - mxfp4
+  - jang
+  - osaurus
+quantization_config:
+  family: mxfp4
+  profile: MXFP4
+  group_size: 32
+  expert_layout: split_switch_mlp
+---
+
+<p align="center"><img src="osaurus-x-banner.png" width="100%" alt="OsaurusAI"/></p>
+
+# ZAYA1-VL-8B-MXFP4
+
+Quantized **Zyphra/ZAYA1-VL-8B** for Apple Silicon runtimes.
+
+| | |
+|---|---|
+| Source | [Zyphra/ZAYA1-VL-8B](https://huggingface.co/Zyphra/ZAYA1-VL-8B) |
+| License | Apache-2.0, inherited from upstream |
+| Format | MXFP4 |
+| Modality | image+text |
+| Bundle size | 7.12 GiB |
+| Tensor keys | 5315 |
+| Expert layout | Pre-stacked `zaya_block.experts.switch_mlp` |
+| Runtime status | Generation coherence: NOT INDEPENDENTLY PASSED for the quantized runtime bundle (missing coherence report); published as a format/runtime bundle pending downstream ZAYA runtime validation. |
+
+## Important Runtime Note
+
+This bundle requires a ZAYA-aware MLX/JANG runtime that implements CCA attention state and the converted pre-stacked expert layout.
+
+ZAYA1-VL fuses Zyphra's text-ZAYA decoder (CCA attention + top-1 MoE) with the Qwen2.5-VL vision tower. Vision-LoRA modulates the LM trunk only at vision-token positions; text positions decode unmodified. Use this bundle only with a runtime that implements the ZAYA CCA state contract and the converted pre-stacked expert layout.
+
+## Runtime Pin Required
+
+Zyphra `model_type=zaya1_vl` is not yet implemented in `vmlx-swift-lm` or stock `mlx_vlm`. The bundle is **conversion-ready and structurally verified** but image-text decoding requires either:
+
+- Zyphra's `transformers @ git+https://github.com/Zyphra/transformers.git@zaya1-vl` fork (BF16 source-side reference), or
+- A `Zaya1VL` MLX adapter (in development at `jang-runtime/Sources/JANG/Zaya1VL/`).
+
+Until the MLX adapter ships, treat this bundle as a runtime-pending preview.
+
+
+## Architecture Summary
+
+- 40 hybrid decoder layers: each layer has CCA attention + top-1 MoE
+- Hidden size 2048, 8 query heads, 2 KV heads, head dim 128
+- 16 routed experts per MoE layer, top-1 routing
+- Vision tower: Qwen2.5-VL ViT (`hidden=1280`, `out=2048`, `patch=14`)
+- Vision-LoRA on the LM trunk: rank-8 attn, rank-32 MLP, gated to vision tokens only
+- Image tokens: `image_token_id=262147`, start=255999, end=256000
+- Context length 32768, `rope_theta=1000000`, partial RoPE (0.5 of head dim)
+
+## Quantization
+
+4-bit affine LM linears + 8-bit embeddings + passthrough vision tower / LoRA / router / CCA state.
+
+Passthrough floor for first release prep:
+
+- `conv_qk.*`, `temp`, norms, residual scaling, router path, biases, and balancing biases are preserved as float tensors.
+- Embeddings and `lm_head` use 8-bit affine in the prepared bundles.
+- Vision tower (`vision_tower.*`) and all LoRA tensors (`*.lora_*.[01].weight`) are kept in float passthrough.
+- `jangtq_runtime.safetensors` is not applicable to MXFP4.
+
+`mxtq_bits`:
+
+```json
+null
+```
+
+## Bundle Verification
+
+- Safetensor headers scanned.
+- Source tensor coverage checked.
+- Converted bundles checked for `local_experts` removal.
+- Converted expert tensors checked for pre-stacked `switch_mlp` layout.
+- JANGTQ sidecars checked for the Swift runtime contract.
+- Vision tower + LoRA tensors verified passthrough; image_token_id, vision_start/end preserved.
+- Runtime coherence status recorded above.
+
+## Runtime Smoke Tests
+
+Before production use, run short deterministic prompts through the exact target runtime:
+
+- `What is 2+2? Answer with only the number.`
+- `What is the capital of France? Answer with one word.`
+- One chat-template prompt with thinking disabled.
+- One image+text prompt exercising vision-token interleave and the vision-LoRA gate.
+
+The first public bundle release records bundle integrity and runtime contract checks. Full generation quality depends on a ZAYA-aware runtime implementation.
+
+## Korean Summary
+
+이 번들은 Zyphra/ZAYA1-VL-8B를 Apple Silicon MLX/JANG 런타임용으로 양자화한 모델입니다. ZAYA의 CCA attention 상태와 MoE 라우팅을 정확히 구현한 런타임에서만 사용해야 합니다. 이미지 입력은 Qwen2.5-VL ViT 경로를 거치며, vision-LoRA는 이미지 토큰 위치에서만 적용됩니다.
+
+## Files
+
+- `config.json` carries `weight_format=mxfp4`, `zaya_expert_layout=split_switch_mlp`, and a preserved `vision_config`.
+- `jang_config.json` carries `cache_subtype=zaya_cca`.
+- Tokenizer files and chat template are preserved from the upstream source snapshot.
+- `preprocessor_config.json` (Qwen2VLImageProcessor) is included for image input.

chat_template.json

@@ -0,0 +1,3 @@
+{
+  "chat_template": "{% for message in messages %}{% if message['role'] == 'user' %}{% for content in message['content'] | selectattr('type', 'equalto', 'image') %}{{ '<|vision_start|><image><|vision_end|>\\n' }}{% endfor %}{% for content in message['content'] | selectattr('type', 'equalto', 'text') %}{{ '<|im_start|>' ~ message['role'] ~ '\n' ~ content['text'] ~ '<|im_end|>' ~ '\n' }}{% endfor %}{% elif message['role'] == 'question' %}{{ '<|im_start|>user\\n' }}{% for content in message['content'] | selectattr('type', 'equalto', 'text') %}{{ content['text'] ~ '<|im_end|>\\n' }}{% endfor %}{% else %}{{ '<|im_start|>' ~ message['role'] ~ '\\n' }}{% for content in message['content'] | selectattr('type', 'equalto', 'text') %}{{ content['text'] ~ '<|im_end|>' }}{% endfor %}{% endif %}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\\n' }}{% endif %}"
+}

config.json

@@ -0,0 +1,87 @@
+{
+  "activation_func": "swiglu",
+  "activation_func_fp8_input_store": false,
+  "add_bias_linear": false,
+  "apply_rope_fusion": true,
+  "ar_threshold": 1,
+  "architectures": [
+    "Zaya1VLForConditionalGeneration"
+  ],
+  "attention_bias": false,
+  "bias_activation_fusion": true,
+  "bos_token_id": 2,
+  "cca": true,
+  "clamp_temp": false,
+  "eos_token_id": 262143,
+  "ffn_hidden_size": 4096,
+  "fused_add_norm": false,
+  "gated_linear_unit": true,
+  "hidden_size": 2048,
+  "head_dim": 128,
+  "image_token_id": 262147,
+  "lm_head_bias": false,
+  "lora_rank": 0,
+  "max_position_embeddings": 32768,
+  "model_type": "zaya1_vl",
+  "moe_router_topk": 1,
+  "norm_epsilon": 1e-05,
+  "normalization": "RMSNorm",
+  "num_attention_heads": 8,
+  "num_experts": 16,
+  "num_hidden_layers": 40,
+  "num_key_value_heads": 2,
+  "num_query_groups": 2,
+  "pad_token_id": 0,
+  "padding_side": "right",
+  "projector_hidden_act": "gelu",
+  "residual_in_fp32": false,
+  "rope_pct": 0.5,
+  "rotary_base": 1000000,
+  "scale_residual_merge": true,
+  "sliding_window": null,
+  "temporal_patch_size": 1,
+  "tie_word_embeddings": true,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.57.1",
+  "use_lora_att": false,
+  "use_rope_scaling": false,
+  "vision_config": {
+    "_attn_implementation_autoset": true,
+    "hidden_size": 1280,
+    "in_chans": 3,
+    "model_type": "qwen2_5_vl",
+    "out_hidden_size": 2048,
+    "spatial_patch_size": 14,
+    "temporal_patch_size": 1,
+    "tokens_per_second": 2,
+    "torch_dtype": "bfloat16"
+  },
+  "vision_end_token_id": 256000,
+  "vision_lora": true,
+  "vision_lora_rank_attn": 8,
+  "vision_lora_rank_mlp": 32,
+  "vision_start_token_id": 255999,
+  "vocab_size": 262272,
+  "zaya_mlp_expansion": 256,
+  "zaya_use_eda": true,
+  "zaya_use_mod": true,
+  "weight_format": "mxfp4",
+  "zaya_expert_layout": "split_switch_mlp",
+  "quantization": {
+    "bits": 4,
+    "group_size": 32,
+    "mode": "affine",
+    "router_bits": 16,
+    "expert_layout": "split_switch_mlp"
+  },
+  "capabilities": {
+    "reasoning_parser": "qwen3",
+    "tool_parser": "zaya_xml",
+    "think_in_template": true,
+    "supports_tools": true,
+    "supports_thinking": false,
+    "family": "zaya1_vl",
+    "modality": "vision",
+    "cache_type": "hybrid"
+  }
+}

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 2,
+  "eos_token_id": 262143,
+  "pad_token_id": 0,
+  "transformers_version": "4.50.0.dev0"
+}

jang_config.json

@@ -0,0 +1,30 @@
+{
+  "version": 2,
+  "weight_format": "mxfp4",
+  "profile": "MXFP4",
+  "cache_subtype": "zaya_cca",
+  "source_model": {
+    "name": "ZAYA1-VL-8B",
+    "org": "Zyphra",
+    "architecture": "zaya1_vl"
+  },
+  "has_vision": true,
+  "expert_layout": "split_switch_mlp",
+  "quantization": {
+    "method": "affine",
+    "group_size": 32,
+    "bits": 4,
+    "embed_bits": 8,
+    "router_bits": 16
+  },
+  "capabilities": {
+    "reasoning_parser": "qwen3",
+    "tool_parser": "zaya_xml",
+    "think_in_template": true,
+    "supports_tools": true,
+    "supports_thinking": false,
+    "family": "zaya1_vl",
+    "modality": "vision",
+    "cache_type": "hybrid"
+  }
+}

model-00001-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ef3c7752093861769e4e29ceeb25734ac78acfb0f0f4fd931b9a83b9e27907e
+size 1003545410

model-00002-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:383a4e2bf1abd7f5a00c94fe2da83ef1e364f4f2bd05caa8e47effae58458b37
+size 1015709496

model-00003-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e71759090088ae99618b5b1efd9e1a3c69c27941902ecc47bfb84051d1c480c3
+size 1014385726

model-00004-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5fd7f7cfb8c006317e8fca905a5c03e378529e4840f0930e94e0fc6f99acceac
+size 1006642888

model-00005-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:51b00e149c29e3fdeae4738ad1c91427371df0601f7bdb7111950baa1d056b63
+size 1006642944

model-00006-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1d2a5b084e5f391fa137d7dc948ee242b3475c775e65197ffbe56669c33eb74a
+size 1006642944

model-00007-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1ef47edd1e35b5f5f8f25e1702cbaf00f1badc619585d4aaa44fe25b562bbfda
+size 1006642944

model-00008-of-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0dca56cbd0beeb36c6531f959c5b46582ad5ffb8b351c29dad811bb218c8475e
+size 553653816

preprocessor_config.json

@@ -0,0 +1,29 @@
+{
+  "do_convert_rgb": true,
+  "do_normalize": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.48145466,
+    0.4578275,
+    0.40821073
+  ],
+  "image_processor_type": "Qwen2VLImageProcessor",
+  "image_std": [
+    0.26862954,
+    0.26130258,
+    0.27577711
+  ],
+  "max_pixels": 12845056,
+  "merge_size": 2,
+  "min_pixels": 3136,
+  "patch_size": 14,
+  "processor_class": "Zaya1VLProcessor",
+  "resample": 3,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "longest_edge": 12845056,
+    "shortest_edge": 3136
+  },
+  "temporal_patch_size": 1
+}

special_tokens_map.json

@@ -0,0 +1,35 @@
+{
+  "answer_token": "<|im_start|>",
+  "boi_token": "<|vision_start|>",
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eoi_token": "<|vision_end|>",
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "image_token": "<image>",
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "video_token": "<video>"
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a90181b1298e5d8c2f211f15dca261650d85c1f2a5a3bbfff852a853d21bed8f
+size 33385329