octopusmegalopod/jam-mt-cjk2english-gguf

This is a tuned LFM2 model for quick Chinese/Japanese/Korean translations into English on low-end consumer hardware. It supports several different translation modes.

Want to translate manga or games? Try it out with Mango.

Some technical details

This model was initially SFT on a private quality-aware dataset, and then post-RL tuned using rubric-based and ranking-based reward schemes. Does the ranking-based reward sounds strange? It's necessary. The rubric-based reward scheme alone constrained the model to generate "safe" or "good-enough" translations, perhaps due to the noise inherent in rubric-based translation evaluation.

Ranking-based reward schemes are typically intractable for large rollout sizes. But ArenaRL: Scaling RL for Open-Ended Agents via Tournament-based Relative Ranking gave me a few ideas to work with.

Inspired by REINFORCE and a few other approaches, a large rollout size along with a few "safe" samples generated at low temperatures are also generated per input. In hindsight, this may not have been worth the effort - a large rollout size alone should be enough. In future approaches, I will probably consider using a dynamic rollout size depending on the difficulty of the prompt instead.

Do NOT use COMET as a reward for RL tuning. The tuned model will give high scores, but it will also quickly overfit on quirks of that model which are very bad.

For optimal usage: Prefill the assistant response with Here's a translation:\n\n - this is an additional step done to "force" the model into a different "mode".

A low temperature is recommended for generation.

Example usage

Japanese with context

Translate the requested Japanese segment into English, strictly adhering to these rules:
- Some previous segments are provided as context. Use the context to help understand the requested segment if needed.

- Preserve Japanese honorifics in the translation.

- Maintain the original tone and style. Do not soften or censor the language.
- Produce a high-quality translation.

Context:
あ… <SENT_SEP> あの動きって…

Translate:

【狂化】中の動きの大元が

(The model was trained with up to 3 contextual texts, but can likely support more. If trying to retain long term memory, consider using RAG instead.)

Korean

Translate the requested Korean segment into English, strictly adhering to these rules:
- Maintain the original tone and style. Do not soften or censor the language.
- Produce a high-quality translation.

Translate:

안녕하세요.

Chinese with examples and Dictionary ("RAG")

Translate the requested Chinese segment into English, strictly adhering to these rules:
- The provided dictionary contains translations for names and other terms. Use these translations when the corresponding name or term appears in the segment to translate. Do not invent or hallucinate new names or terms.
- Reference the input/output examples as a guide for translation, matching their style, terminology, and phrasing to produce the final output.
- Maintain the original tone and style. Do not soften or censor the language.
- Produce a high-quality translation.

Examples:

Input:
尊敬
Output:
Respected
Input:
爱好
Output:
Hobbies
Input:
羡慕
Output:
Envy

Dictionary:
美洛浦 = Melop (Male)
纪芙洛妮 = Jifu Lin (Female)

Translate:

爱你

Watch for...

Everyone wants to talk about how benchmaxxed their models are. Few want to talk about the nitty gritty... As a small model, there are several points to consider:

1. It was ONLY trained for CJK to English. The dataset was curated specifically for this. It will likely have very poor generalization for other languages.
2. It may over-rely on RAG examples given, for better or worse. For example, given several RAG samples related to combat, the text 想要与谁厮守? can translate to some variation of "Whom do you want to fight with?"
3. It may still under-utilize contextual information, due to its limited size and architecture (short convolutional attention prioritizes nearby information).
4. It may suffer from rare word translations without RAG, due to its limited size to memorize knowledge.
5. It will translate explicit text without censorship in most cases. No safety alignment was done, so be aware of this risk or "reward".
6. It will translate names without a dictionary poorly. Interestingly enough, the post-RL process only made this problem worse.
7. Korean-English could be better. The crux of the issue is that there's no good open-source solutions to evaluate the quality of Korean-English translations. For example, COMET tends to over-rely on the English translation segment with seemingly no regard for the Korean source segment.