autotrain.ideas.md

Ideas Backlog

• [P1] Add a Japanese continued-pretraining phase on clean monolingual prose before SFT if SFT-only loss plateaus.
• [P1] Replace raw translated HH examples with a stricter subset filtered by Japanese character ratio and URL density.
• [P2] Try a consistent prompt prefix such as 日本語で自然かつ簡潔に答えてください。 to bias the adapter toward Japanese-only responses.
• [P3] Sweep LoRA target depth via --num-layers (8/16/all) once data quality stabilizes.
• [P1] Filter HH/OASST rows with high URL density or unusually low Japanese ratio after normalization; these are likely noisy translated web answers.
• [P1] Lower completion_max_chars for a compact-response training subset to see if the adapter learns cleaner Japanese response style faster.