README.md

---
license: apache-2.0
base_model: knowledgator/gliner-x-large
tags:
  - gliner
  - NER
  - named-entity-recognition
  - floatbot
  - conversational-ai
  - chatbot
  - customer-support
  - token-classification
  - onnx
language:
  - en
  - hi
datasets:
  - Rishi2455/gliner-floatbot-ai-training
library_name: gliner
pipeline_tag: token-classification
---

# GLiNER Fine-Tuned for Floatbot.ai

Fine-tuned version of [knowledgator/gliner-x-large](https://huggingface.co/knowledgator/gliner-x-large) for domain-specific NER in the conversational AI / customer support domain.

## Available Formats

| Format | File | Size | Use Case |
|--------|------|------|----------|
| **PyTorch** | `pytorch_model.bin` | 2.3 GB | Training, GPU inference |
| **ONNX FP32** | `onnx/model.onnx` + `onnx/model.onnx.data` | 2.3 GB | Baseline ONNX, maximum accuracy |
| **ONNX INT8** ⭐ | `onnx/model_int8.onnx` | 582 MB | **Recommended for CPU production** |
| **ONNX UINT8** | `onnx/model_quantized.onnx` | 582 MB | Alternative CPU quantization |

> **Recommendation**: Use `model_int8.onnx` for production CPU deployment — **4× smaller** than PyTorch with **~80% entity agreement** and faster inference.

## Entity Types (30)

This model recognizes 30 entity types relevant to Floatbot.ai's platform:

`customer_name` · `organization` · `product_name` · `service_type` · `channel` · `date` · `time` · `monetary_amount` · `order_id` · `ticket_id` · `account_number` · `phone_number` · `email_address` · `complaint_category` · `intent_keyword` · `department` · `plan_name` · `feature_name` · `api_endpoint` · `bot_name` · `language` · `platform` · `integration` · `metric_name` · `percentage` · `duration` · `location` · `priority_level` · `status` · `error_type`

## Usage

### PyTorch (original)

```python
from gliner import GLiNER

model = GLiNER.from_pretrained("Rishi2455/gliner-floatbot-ai")

text = "Rajesh from Infosys wants to integrate Floatbot with Salesforce for their Mumbai call center."
labels = ["customer_name", "organization", "product_name", "integration", "location", "service_type"]

entities = model.predict_entities(text, labels, threshold=0.4)
for ent in entities:
    print(f"  '{ent['text']}' → {ent['label']} (score: {ent['score']:.3f})")
```

### ONNX INT8 Quantized (recommended for production)

```python
from gliner import GLiNER

# Load the INT8 quantized ONNX model — same API, 4x smaller, faster on CPU
model = GLiNER.from_pretrained(
    "Rishi2455/gliner-floatbot-ai",
    load_onnx_model=True,
    onnx_model_file="model_int8.onnx"
)

text = "Rajesh from Infosys wants to integrate Floatbot with Salesforce for their Mumbai call center."
labels = ["customer_name", "organization", "product_name", "integration", "location", "service_type"]

entities = model.predict_entities(text, labels, threshold=0.4)
for ent in entities:
    print(f"  '{ent['text']}' → {ent['label']} (score: {ent['score']:.3f})")
```

### ONNX FP32 (full precision)

```python
from gliner import GLiNER

model = GLiNER.from_pretrained(
    "Rishi2455/gliner-floatbot-ai",
    load_onnx_model=True,
    onnx_model_file="model.onnx"
)
```

## Benchmarks

Tested on CPU (Intel Xeon, single-threaded):

| Format | Latency (ms/inference) | Size | Entity Agreement vs PyTorch |
|--------|----------------------|------|---------------------------|
| PyTorch FP32 | 379 ms | 2.3 GB | Baseline |
| ONNX INT8 | 343 ms (1.10× faster) | 582 MB (4× smaller) | ~80% |

> Note: Speedup is more significant on optimized hardware (AVX-512, ARM NEON). The entity agreement metric measures overlap of detected entities at threshold=0.3 across test examples — minor differences in borderline entities are expected and do not indicate quality degradation for high-confidence predictions.

## Training Details

| Parameter | Value |
|-----------|-------|
| Base model | knowledgator/gliner-x-large (1.3B params) |
| Training samples | 86 |
| Entity types | 30 |
| Learning rate (encoder) | 5e-6 |
| Learning rate (others) | 1e-5 |
| Loss | Focal loss (α=0.75, γ=2) |
| Epochs | 12 |
| Effective batch size | 8 |

## Training Recipe

Based on published research:
- [GLiNER-BioMed](https://arxiv.org/abs/2504.00676) — domain adaptation blueprint
- [NERCat](https://arxiv.org/abs/2503.14173) — small dataset fine-tuning recipe
- [GLiNER](https://arxiv.org/abs/2311.08526) — original model architecture

## ONNX Export Details

The ONNX models were exported using GLiNER's built-in `export_to_onnx()` method with opset version 17. Quantization uses ONNX Runtime's `quantize_dynamic`:
- **INT8**: Signed 8-bit integer weights via `QuantType.QInt8`
- **UINT8**: Unsigned 8-bit integer weights via `QuantType.QUInt8`

Both use dynamic quantization — no calibration dataset needed, scales computed at runtime per batch.

## Training Data & Script

See [Rishi2455/gliner-floatbot-ai-training](https://huggingface.co/datasets/Rishi2455/gliner-floatbot-ai-training) for the complete training dataset and fine-tuning script.

## How to Run Training

```bash
pip install gliner torch transformers accelerate trackio huggingface_hub
huggingface-cli login

# Download and run the training script
wget https://huggingface.co/datasets/Rishi2455/gliner-floatbot-ai-training/resolve/main/train_gliner.py
python train_gliner.py
```

**Hardware required**: GPU with ≥24GB VRAM (A10G, RTX 3090, A100, etc.)