Motif-Technologies
/

activation

Model card Files Files and versions

activation / tests

63.9 kB

Ctrl+K

Ctrl+K

11 contributors

History: 24 commits

Jangwoong Kim

Merge pull request #22 from MotifTechnologies/jangwoong/mla-rope-fa4-port

5adea7d unverified about 15 hours ago

__init__.py

0 Bytes
feat(poly-norm): Add PolyNorm 11 months ago
allclose_default.py

512 Bytes
Add fusion (#3) 7 months ago
pytest.ini

71 Bytes
fix: unify all backward kernels to input-based math + fix test import 3 days ago
test_fused_add_rms_norm.py

3.6 kB
refactor(activation): change fused_add_rms_norm and fused_add_rms_norm_backward to out-place operations 6 months ago
test_fused_add_rms_norm_sequence_parallel.py

5.85 kB
fix: fix fused add rms norm sharding strategy 5 months ago
test_fused_mul_grouped_poly_norm.py

32.4 kB
style: fix yapf/isort formatting for CI --all-files check 3 days ago
test_fused_mul_poly_norm.py

4.13 kB
Add fusion (#3) 7 months ago
test_mla_rope_grad.py

5.96 kB
test: numerical parity for MLA RoPE fused kernels vs PyTorch reference about 22 hours ago
test_poly_norm.py

2.55 kB
Add fusion (#3) 7 months ago
test_rms_norm.py

1.94 kB
feat: make rms_norm as out-place 7 months ago
test_rms_norm_sequence_parallel.py

4.37 kB
fix: fix rms norm sharding strategy 5 months ago
utils.py

2.56 kB
Add fusion (#3) 7 months ago