Teklia/Qwen2.5-VL-7B-DAI-CReTDHI-RecordGold-ATR

QWEN2.5-VL - French Parish Register Model

Introduction

This version of QWEN2.5-VL-7B is specialized for HTR on French parish records from the 16th-18th centuries. Ref: https://redmine.teklia.com/issues/11177

Training

The model is a QWEN2.5-VL-7B-Instruct fine-tuned on French parish records using LoRA.

It was fine-tuned in two steps:

1. First, it was fine-tuned on 30k crowdsourced records (annotations of variable quality)
2. It was then fine-tuned on 7k records (high quality annotation)

Parameters:

• Image width: 800 pixels
• LoRa rank: 8
• LoRa alpha: 32
• Epochs: 10 (about 4k steps)

Wandb: https://wandb.ai/starride-teklia/uncategorized/runs/bjv98gfo

Evaluation

It achieves the following results:

Model	CER test	WER test
QWEN3.5-VL (fine-tuned on gold data)	14.89	29.27
QWEN3.5-VL (fine-tuned on crowdsourced and gold data)	11.41	24.24
QWEN2.5-VL (fine-tuned on crowdsourced and gold data)	10.09	24.39

Usage

Here we show a code snippet to show you how to use the model with transformers and qwen_vl_utils:

• Prediction script

from transformers import Qwen2_5_VLForConditionalGeneration, AutoTokenizer, AutoProcessor
from qwen_vl_utils import process_vision_info
import torch

# Load QWEN
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "QWEN/Qwen2.5-VL-7B-Instruct",
    torch_dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
    device_map="auto",
)
processor = AutoProcessor.from_pretrained("Teklia/Qwen2.5-VL-7B-DAI-CReTDHI-RecordGold-ATR", use_fast=True)

# Prompt
messages = [
    {
        "role": "system",
        "content": [
           {
               "type": "text",
               "text": "Tu es un assistant archiviste. Tu dois lire des actes issus de registres paroissiaux français, du 16è au 18è siècle. Extrais le texte de la marge, du corps de l'acte, et éventuellement les signatures."
           }
        ]
    },
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "https://europe.iiif.teklia.com/iiif/2/geneanet%2FArdennes_BMS%2F382706%2F00056.jpg/1252,102,1133,692/full/0/default.jpg"
            },
            {
                "type": "text",
                "text": "Extrais le texte de ce document."
            },
        ],
    }
]

# Preparation for inference
text = processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
image_inputs, video_inputs = process_vision_info(messages)
inputs = processor(
    text=[text],
    images=image_inputs,
    videos=video_inputs,
    padding=True,
    return_tensors="pt",
)
inputs = inputs.to("cuda")

# Inference: Generation of the output
generated_ids = model.generate(**inputs, max_new_tokens=1024)
generated_ids_trimmed = [
    out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output_text = processor.batch_decode(
    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)[0]
print(output_text)

• Output

L'an mil sept cent quatre vingt six le dix septième jour du mois de mars est décédé à Frainvrit de cette paroisse Lambert Joseph Henrot âgé de trente huit ans, natif de Gumiay fils de Lambert Henrot et de Marie Thérèse Gervant lequel a été inhumé le lendemain au cimetière de cette ditte paroisse avec les ceremonies ordinaires par nous prêtre vicaire de cette ville, en présence de Vincent Parrot fleure de cette église, et de Piacre Blondeau, habitant de cette ville, lesquels ont signé avec nous.

Blondeau Parrot Berin vicaire

Citation

To cite the original QWEN2.5-VL model:

@misc{qwen2.5-VL,
    title = {Qwen2.5-VL},
    url = {https://qwenlm.github.io/blog/qwen2.5-vl/},
    author = {Qwen Team},
    month = {January},
    year = {2025}
}