Merge pull request #22 from MotifTechnologies/jangwoong/mla-rope-fa4-port

bench: MLA RoPE fused vs vanilla sweep

test: numerical parity for MLA RoPE fused kernels vs PyTorch reference

cleanup: drop k_pe RoPE custom kernel (caller uses PyTorch native)

refactor: replace warp shuffle with CUB BlockReduce

fix: unify all backward kernels to input-based math + fix test import

style: fix yapf/isort/clang-format for CI --all-files

feat: add RMSNorm benchmark scripts and K8s job

feat: update RMSNorm Python interface for optimized kernels

perf: optimize RMSNorm CUDA kernels for all dims

feat: dedicated _kv_rope_bwd_kernel (register-sum + copy-fused)

perf: remove autotune, hard-code per-kernel configs from live dump

cleanup: remove dead Phase 3 Q kernel + shrink autotune to hand-picked configs

review fixups: stride asserts, autotune split, intent comments

feat: MLA RoPE Triton kernels (port from llm-training)

style: fix yapf/isort formatting for CI --all-files check

style: apply yapf + isort formatting

feat: replace triton do_bench with torch.profiler for kernel timing

grouped polynorm with padding aware (#19)

chore: remove pre-built binaries and add local build loader shim (#18)

style: apply yapf, isort, and clang-format

style: fix clang-format on torch_binding.h

docs: update README for CUDA kernel and pip install workflow

ci: remove nix build-and-commit workflow

fix: rename stale references and clean up Triton remnants

refactor: remove Triton kernels, add hidden_clamp to unscored ops

test: add scores and hidden_clamp tests for fused_mul_grouped_poly_norm

feat: add setup.py for local CUDA development builds

feat: add grouped poly norm CUDA kernel with scores and hidden_clamp fusion

refactor: rename grouped_fused_mul_poly_norm → fused_mul_grouped_poly_norm

feat: add GroupedFusedMulPolyNorm Triton kernel for MoE models (#16)

Add built binary [skip-build]

fix: support PyTorch 2.10 register_op_strategy import path change

fix: update toml

Add built binary [skip-build]

fix: fix fused add rms norm sharding strategy

fix: fix rms norm sharding strategy

fix: comment out actionlint

Update tag (#4)

Add built binary [skip-build]

fix(rms_norm.py): add assertion for input gradients to handle unsupported cases in backward pass

feat: support sequence parallel with fused_add_rms_norm

refactor(activation): change fused_add_rms_norm and fused_add_rms_norm_backward to out-place operations

refactor(rms_norm): move RMS normalization logic to a new module for better organization and maintainability

feat: support sequence parallel with rms_norm

feat: add assert is_contiguous

feat: make rms_norm as out-place

feat(workflow): add Slack notifications for build start, success, and failure [skip-build]

Revert "fix typo in readme (#7)" (#8)

fix typo in readme (#7)