google/gemma-4-31B-it
Image-Text-to-Text • 33B • Updated • 3.2M • • 1.96k
Pavel AI
pavel-ai
AI & ML interests
AI tools development
Recent Activity
liked a model 8 days ago
google/gemma-4-31B-it upvoted a collection 14 days ago
Gemma 4 upvoted a collection 14 days ago
Gemma 4Organizations
liked a
model 8 days ago
upvoted 2
collections 14 days ago
liked 3
models 20 days ago
liked a
Space about 1 month ago
liked a
model about 1 month ago
liked 2
Spaces 3 months ago
upvoted an article 3 months ago
Article
RexRerankers: SOTA Rankers for Product Discovery and AI Assistants
•
44
updated a
dataset 3 months ago
published a
dataset 3 months ago
liked a
Space 3 months ago
commentedon Reverse Engineering a $500M Mystery: From HashHop to Memory-Augmented Language Models 3 months ago
Does it mean that the Qwen model is passing tokens back to the context. In the retrieval level these tokens got encoded into MALM model's hashes which are used to retrieve information from context in MALM tokens and then the result converts to Qwen's tokens again?
reactedto codelion's post with 🔥 3 months ago
Post
3248
Reverse Engineering a $500M Mystery: From HashHop to Memory-Augmented Language Models
I wrote a deep dive into how Magic AI's 100M token context window might work, starting from their HashHop benchmark and building up to MALM - a Memory-Augmented Language Model.
Key insight: treating each key as a single token enables perfect retrieval at unlimited context lengths.
The article covers:
- How HashHop works and why its perfect accuracy is suspicious
- Building a tokenized solver that achieves 100% accuracy
- Scaling to MALM for real code search tasks
- Why this approach could handle 100M+ tokens
Read the full article: https://huggingface.co/blog/codelion/reverse-engineering-magic-hashhop
Try the model: codelion/malm-165m
Code: https://github.com/codelion/hash-hop
I wrote a deep dive into how Magic AI's 100M token context window might work, starting from their HashHop benchmark and building up to MALM - a Memory-Augmented Language Model.
Key insight: treating each key as a single token enables perfect retrieval at unlimited context lengths.
The article covers:
- How HashHop works and why its perfect accuracy is suspicious
- Building a tokenized solver that achieves 100% accuracy
- Scaling to MALM for real code search tasks
- Why this approach could handle 100M+ tokens
Read the full article: https://huggingface.co/blog/codelion/reverse-engineering-magic-hashhop
Try the model: codelion/malm-165m
Code: https://github.com/codelion/hash-hop
- 1 reply
upvoted an article 3 months ago
Article
Reverse Engineering a $500M Mystery: From HashHop to Memory-Augmented Language Models
•
10
liked a
model 3 months ago
liked a
Space 3 months ago
upvoted a collection 4 months ago