varunneal

/

flash-attention-3

like 12

Should be added, give it a try.

File "/root/.cache/huggingface/hub/models--varunneal--flash-attention-3/snapshots/add01af002563fdeff03a8e5fb77ce497d202055/build/torch29-cxx11-cu130-x86_64-linux/flash_attention_3/init.py", line 1, in
from .flash_attn_interface import *
File "/root/.cache/huggingface/hub/models--varunneal--flash-attention-3/snapshots/add01af002563fdeff03a8e5fb77ce497d202055/build/torch29-cxx11-cu130-x86_64-linux/flash_attention_3/flash_attn_interface.py", line 10, in
from . import _C # Registers operators with PyTorch
^^^^^^^^^^^^^^^^
ImportError: libcudart.so.12: cannot open shared object file: No such file or directory

Fails on CUDA 13 container :(

I'll try to take a look this week. Just respond in this thread if I take over a week / forget about it.

I'm seeing the same error as OGamer34, except on arm architecture not x86:

❯ uv run python
Uninstalled 1 package in 0.48ms
Installed 1 package in 2ms
Python 3.11.14 (main, Oct 14 2025, 21:27:45) [Clang 20.1.4 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from kernels import get_kernel
>>> flash_attn = get_kernel('varunneal/flash-attention-3').flash_attn_interface
/tmp/tmp.1KtN38wPMG/.venv/lib/python3.11/site-packages/torch/_subclasses/functional_tensor.py:279: UserWarning: Failed to initialize NumPy: No module named 'numpy' (Triggered internally at /pytorch/torch/csrc/utils/tensor_numpy.cpp:84.)
  cpu = _conversion_method_template(device=torch.device("cpu"))
Fetching 4 files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 528.02it/s]
Download complete: : 0.00B [00:00, ?B/s]              Traceback (most recent call last):                                              | 0/4 [00:00<?, ?it/s]
  File "<stdin>", line 1, in <module>
  File "/tmp/tmp.1KtN38wPMG/.venv/lib/python3.11/site-packages/kernels/utils.py", line 319, in get_kernel
    return _import_from_path(package_name, variant_path)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp.1KtN38wPMG/.venv/lib/python3.11/site-packages/kernels/utils.py", line 156, in _import_from_path
    spec.loader.exec_module(module)  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/vindiesel9876/.cache/huggingface/hub/models--varunneal--flash-attention-3/snapshots/add01af002563fdeff03a8e5fb77ce497d202055/build/torch29-cxx11-cu130-aarch64-linux/flash_attention_3/__init__.py", line 1, in <module>
    from .flash_attn_interface import *
  File "/home/vindiesel9876/.cache/huggingface/hub/models--varunneal--flash-attention-3/snapshots/add01af002563fdeff03a8e5fb77ce497d202055/build/torch29-cxx11-cu130-aarch64-linux/flash_attention_3/flash_attn_interface.py", line 10, in <module>
    from . import _C # Registers operators with PyTorch
    ^^^^^^^^^^^^^^^^
ImportError: libcudart.so.12: cannot open shared object file: No such file or directory

finally reproduced this problem. Fix should be in on cu130 on x86 devices. Will have to wait a bit to get ARM