octopusmegalopod/fast-jamo-ko-ocr-gguf

Info

This is a heavily fine-tuned variant of LFM2-VL-1.6B for OCR'ing Korean text in images.

This paper proposed an interesting way to improve OCR accuracy: By decomposing the characters we can reduce the possible output space, and this should in turn reduce the likelihood of rare tokens occurring - a critical source of errors for language modeling.

So make sure to recompose the output by using a recomposition function, like compose(model_output) from hangul-jamo

Expected input format

This model expects a single-turn conversation, where the user inputs a specific text instruction followed by the image, e.g:

messages = [
    {
        'role': 'user',
        'content': [
            { 
                'type': 'text',
                'text': "OCR the Korean text in this image without any explanation.",
            },
            {
                'type': 'image_url',
                'image_url': {
                    'url': image_to_base64(image),
                }
            }
        ]
    }
]

This model was NOT trained to OCR entire pages of text as-is. For best results pass in an image containing a single line of text.