MiniCPM-V-2.6-OpenVINO-INT4

This repository contains an optimized OpenVINO™ IR version of MiniCPM-V-2.6, quantized to INT4 precision using NNCF. This 8B parameter model represents the gold standard for edge-deployed vision-language tasks, offering exceptional performance in image understanding, multi-image reasoning, and video spatial analysis.

---

🐍 Python Inference (Optimum-Intel)

To run this vision engine locally using the optimum-intel library:

from optimum.intel import OVModelForVisualCausalLM
from transformers import AutoProcessor
from PIL import Image

model_id = "CelesteImperia/MiniCPM-V-2.6-OpenVINO-INT4"
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = OVModelForVisualCausalLM.from_pretrained(model_id, trust_remote_code=True)

image = Image.open("path/to/your/image.jpg")
prompt = "Analyze this industrial component for defects."

inputs = processor(text=[prompt], images=[image], return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0], skip_special_tokens=True))

---

💻 For C# / .NET Users (OpenVINO.GenAI)

This architecture is natively supported via the VLMPipeline in the OpenVINO.GenAI framework for low-latency visual automation.

using OpenVino.GenAI;

// 1. Initialize the Visual-LLM Pipeline
var device = "CPU"; // Switch to "GPU" for local acceleration
using var pipe = new VLMPipeline("path/to/minicpm-v-model", device);

// 2. Load Visual Input
var image = OpenVino.GenAI.Utils.LoadImage("industrial_capture.jpg");
var prompt = "What is the serial number shown on the component label?";

// 3. Multimodal Execution
var result = pipe.Generate(prompt, image);
Console.WriteLine(result.Texts[0]);

---

🏗️ Technical Details

• Optimization Tool: NNCF (Neural Network Compression Framework)
• Quantization: INT4 Asymmetric (Group Size: 128)
• Architecture: SigLip-400M + Qwen2-7B (8B Total)
• Workstation Validation: Dual-GPU (RTX 3090 + RTX A4000)

---

☕ Support the Forge

Maintaining high-performance AI workstations and hosting elite-tier vision models requires significant resources. If our open-source tools power your projects, consider supporting our development:

Platform	Support Link
Global & India	Support via Razorpay

Scan to support via UPI (India Only):

---

📜 License

This model is released under the Apache 2.0 License.

---

Connect with the architect: Abhishek Jaiswal on LinkedIn