kinkinkin123456/nexus-qwen35-0p8b-smoke-lora

Nexus Qwen3.5-0.8B Smoke LoRA Adapter

This repository contains a small LoRA adapter produced from a real smoke finetune run of the Nexus multimodal embedding stack.

It is intended for pipeline validation rather than benchmark claims or production deployment.

What This Repo Contains

This is an adapter-only release. It does not include the full base model weights.

Main files:

• adapter_model.safetensors
• adapter_config.json
• processor_config.json
• tokenizer.json
• tokenizer_config.json
• chat_template.jinja
• nexus_configs/

Expected base model:

• Qwen/Qwen3.5-0.8B

Training Summary

• Base model: Qwen/Qwen3.5-0.8B
• Training stack: Nexus multimodal retrieval embedder
• Finetune type: LoRA
• Training goal: smoke validation for training, save, reload, and inference paths
• Data: a small HatefulMemes smoke subset
• Steps: 10
• Precision: bf16
• Per-device batch size: 8
• Gradient checkpointing: enabled

The exact Nexus config files used in this run are included under nexus_configs/.

Recommended Usage

Load directly with Nexus

Inside the Nexus codebase, the simplest path is to point both model and processor to this adapter directory:

from Nexus import MultimodalEmbedder

model = MultimodalEmbedder(
    model_name_or_path="path/to/this/adapter/repo",
    processor_name_or_path="path/to/this/adapter/repo",
    model_type="qwen3_5",
    trust_remote_code=True,
    normalize_embeddings=True,
    pooling_method="last_token",
)

Nexus will detect adapter_config.json, load the base model, and then attach the LoRA adapter.

Merge before publishing a full model

If you want a standalone full-weight repository, merge this adapter into the base model and publish the merged checkpoint as a separate repo.

For Stage 1 delivery, the adapter-only release is the primary artifact.

Limitations

• This is a smoke finetune artifact, not a fully trained multimodal embedding model.
• The training data scale is intentionally tiny.
• Use this checkpoint to validate code and release flow, not to claim final model quality.

Notes

• The original training run used a machine-local base model path.
• For Hugging Face publication, adapter_config.json in this release directory has been normalized to Qwen/Qwen3.5-0.8B.
• If you need an offline local test on the original training machine, keep a second local-only adapter copy that still points to the local base model directory.