Hugging Face's logo
Papers
arxiv:2505.22296

360-LLaMA-Factory: Plug & Play Sequence Parallelism for Long Post-Training

Published on May 28, 2025
Upvote
-
Authors:
,
,
,
,
,

Abstract

360-LLaMA-Factory, an open-sourced framework incorporating sequence parallelism, is widely used in various models and large company training frameworks, with detailed insights into its implementation.

AI-generated summary

Adding sequence parallelism into LLaMA-Factory, we open-sourced 360-LLaMA-Factory at https://github.com/Qihoo360/360-LLaMA-Factory. 360-LLaMA-Factory has received wide recognition and used in models such as Light-R1 arXiv:2503.10460, TinyR1 arXiv:2503.04872, Kaggle AIMO math models and also in large companies' training frameworks. This technical report delves deeper into the different sequence parallel modes behind 360-LLaMA-Factory and discusses our implementation insights.

View arXiv page View PDF GitHub 606 auto Add to collection

Community

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2505.22296
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2505.22296 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2505.22296 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2505.22296 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.