Softpick
Collection
Pretrained models from the paper "Softpick: No Attention Sink, No Massive Activations with Rectified Softmax" • 5 items • Updated
YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
This model is from the paper arxiv.org/abs/2504.20966
Softpick: No Attention Sink, No Massive Activations with Rectified Softmax
Also used in arxiv.org/abs/2508.19228
Token Order Prediction
See code: https://github.com/zaydzuhri/softpick-attention
This model is only usable through these repositories: https://github.com/zaydzuhri/flash-linear-attention/tree/softpick-attention https://github.com/zaydzuhri/flame/tree/softpick-attention
- Downloads last month
- 2
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support