Hugging Face's logo
Carlo Moro's picture
53 24 528

Carlo Moro

cnmoro
Felladrin's profile picture lucazsh's profile picture lixue's profile picture
Ā·

AI & ML interests

None yet

Recent Activity

repliedto JonnaMat's post 1 day ago
āš” FlashHead: Fast LM Head Inference - Now a Simple vLLM Plugin flash-head replaces the dense LM head with a two-stage retrieval pipeline - up to 2x inference speedup, training-free. Previously required custom Docker images; now it's just: ``` pip install flash-head vllm serve embedl/Qwen3-1.7B-FlashHead-W4A16 ``` āœØ The plugin activates automatically via vLLM's `vllm.general_plugins` entry point. No source patches, no custom imports. šŸ§© Supported models (full collection): https://huggingface.co/Qwen Qwen3, https://huggingface.co/meta-llama Llama3, https://huggingface.co/google Gemma3, https://huggingface.co/nvidia Cosmos-Reason2 - BF16 and W4A16 variants. https://huggingface.co/collections/embedl/flashhead šŸ“Š https://huggingface.co/spaces/embedl/Edge-Inference-Benchmarks šŸ”§ Benchmark it yourself: ``` vllm bench latency --model embedl/Qwen3-1.7B-FlashHead-W4A16 --batch-size 1 # Baseline comparison FLASHHEAD_ENABLED=0 vllm bench latency --model embedl/Qwen3-1.7B-FlashHead-W4A16 --batch-size 1 ``` FlashHead shines at low batch sizes; the typical real-time / on-device use case. šŸš€
reacted to JonnaMat's post with šŸš€ 1 day ago
āš” FlashHead: Fast LM Head Inference - Now a Simple vLLM Plugin flash-head replaces the dense LM head with a two-stage retrieval pipeline - up to 2x inference speedup, training-free. Previously required custom Docker images; now it's just: ``` pip install flash-head vllm serve embedl/Qwen3-1.7B-FlashHead-W4A16 ``` āœØ The plugin activates automatically via vLLM's `vllm.general_plugins` entry point. No source patches, no custom imports. šŸ§© Supported models (full collection): https://huggingface.co/Qwen Qwen3, https://huggingface.co/meta-llama Llama3, https://huggingface.co/google Gemma3, https://huggingface.co/nvidia Cosmos-Reason2 - BF16 and W4A16 variants. https://huggingface.co/collections/embedl/flashhead šŸ“Š https://huggingface.co/spaces/embedl/Edge-Inference-Benchmarks šŸ”§ Benchmark it yourself: ``` vllm bench latency --model embedl/Qwen3-1.7B-FlashHead-W4A16 --batch-size 1 # Baseline comparison FLASHHEAD_ENABLED=0 vllm bench latency --model embedl/Qwen3-1.7B-FlashHead-W4A16 --batch-size 1 ``` FlashHead shines at low batch sizes; the typical real-time / on-device use case. šŸš€
liked a dataset 1 day ago
badlogicgames/pi-mono
View all activity

Organizations

Wise Intelligence's profile picture Smol Community's profile picture