---
library_name: transformers
license: apache-2.0
pipeline_tag: image-text-to-text
tags:
- multimodal
- vision-language
- chat
---

# Rax 3.5 Chat

Rax 3.5 Chat is a compact 2B parameter multimodal model for vision-language understanding and conversational AI. It supports text and image inputs with extended context up to 262K tokens.

## Model Details

- **Parameters**: ~2B
- **Context Length**: 262,144 tokens
- **Input Modalities**: Text + Images
- **Attention**: Hybrid linear + full attention (24 layers)
- **Vision Encoder**: 24-layer transformer with 1024 hidden size
- **Text Hidden Size**: 2048
- **Precision**: BFloat16

## Key Features

- **Multimodal Understanding**: Processes text and images in unified reasoning
- **Long Context**: Supports up to 262K tokens for extended conversations
- **Efficient Architecture**: Hybrid attention mechanism for optimal performance
- **Production Ready**: Compatible with vLLM, SGLang, and Transformers

## Usage

### With Transformers

```python
from transformers import AutoModelForVision2Seq, AutoProcessor
from PIL import Image

model = AutoModelForVision2Seq.from_pretrained("raxcore/Rax-3.5-Chat", trust_remote_code=True)
processor = AutoProcessor.from_pretrained("raxcore/Rax-3.5-Chat", trust_remote_code=True)

# Text-only conversation
messages = [{"role": "user", "content": "What is the capital of France?"}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(outputs[0], skip_special_tokens=True))

# With image
image = Image.open("image.jpg")
messages = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": "Describe this image."}]}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=text, images=image, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(outputs[0], skip_special_tokens=True))
```

### With vLLM

```bash
vllm serve raxcore/Rax-3.5-Chat --port 8000 --max-model-len 8192
```

```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="token")

response = client.chat.completions.create(
    model="raxcore/Rax-3.5-Chat",
    messages=[{"role": "user", "content": "Hello!"}],
    temperature=0.7,
    max_tokens=512
)
print(response.choices[0].message.content)
```

## Architecture Highlights

- **Hybrid Attention**: Alternates between linear attention and full attention layers for efficiency
- **Vision Encoder**: 24-layer transformer with patch size 16 and spatial merge 2x2
- **Efficient KV Cache**: 2 key-value heads for reduced memory footprint
- **Multi-resolution Position Embeddings**: Optimized for long-context understanding

## Best Practices

- Use temperature 0.6–0.8 for factual tasks, 0.8–1.0 for creative tasks
- For long context (>32K tokens), ensure sufficient GPU memory
- Enable trust_remote_code when loading the model

## Limitations

- 2B parameters may limit complex reasoning compared to larger models
- Vision understanding optimized for natural images
- Long context requires significant memory resources

## License

Apache 2.0

## Citation

```bibtex
@misc{rax3.5chat,
  title={Rax 3.5 Chat: Efficient Multimodal Assistant Model},
  author={Raxcore},
  year={2026}
}
```