gpt-oss-20b-Ja-Fin-CPT

A Japanese financial domain model built through continued pre-training of gpt-oss-20b on a curated Japanese financial corpus.

Model Overview

A domain-adapted base model for Japanese finance, intended for further fine-tuning on specific financial tasks.

Base Model: gpt-oss-20b
Training Stage: Continued Pre-Training (CPT)
Domain: Japanese Finance
Language: Japanese, English

Training

Continued Pre-Training

Trained on a Japanese financial corpus constructed from Common Crawl and other public sources, with domain classification and quality filtering.

Training Infrastructure:

Hardware: AWS p5en.48xlarge (NVIDIA H200 Tensor Core GPU x 8)

Intended Use

Primarily intended as a foundation for further fine-tuning. For a reasoning model with instruction-following capabilities, see gpt-oss-20b-Ja-Fin-Thinking.

Primary Use Cases

Base model for financial domain SFT
Feature extraction for financial text
Further pre-training on proprietary financial data

Limitations

Not instruction-tuned: This model has not undergone supervised fine-tuning and may not follow instructions well
Domain specificity: Optimized for Japanese financial domain; performance on other domains may vary
Language coverage: Primarily Japanese and English

License

This model is released under the Apache 2.0 license.

Privacy Notice

For details on how personal information is handled, please see the Privacy Notice (日本語).

Citation

@inproceedings{okochiDomainSpecificLLM2026,
  author    = {大河内 悠磨 and Sim, Fabio Milentiansen and 岡田 智靖},
  title     = {ドメイン特化LLMの推論能力向上を目的とした合成指示データセットの構築と金融ドメインにおける評価},
  booktitle = {言語処理学会第32回年次大会 (NLP2026) },
  year      = {2026},
  month     = mar,
  address   = {Utsunomiya, Tochigi, Japan},
  publisher = {言語処理学会},
  note      = {Paper ID: C7-2},
  url       = {https://www.anlp.jp/proceedings/annual_meeting/2026/pdf_dir/C7-2.pdf}
}

@misc{okochi2026constructingsyntheticinstructiondatasets,
  title     = {Constructing Synthetic Instruction Datasets for Improving Reasoning in Domain-Specific LLMs: A Case Study in the Japanese Financial Domain},
  author    = {Yuma Okochi and Fabio Milentiansen Sim and Tomoyasu Okada},
  year      = {2026},
  eprint    = {2603.01353},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  url       = {https://arxiv.org/abs/2603.01353}
}

Acknowledgments

This model was developed with the support of the "GENIAC (Generative AI Accelerator Challenge)" project, implemented by the Ministry of Economy, Trade and Industry (METI) and the New Energy and Industrial Technology Development Organization (NEDO), with the aim of strengthening Japan's development capabilities in generative AI.