The Impact of Hyperparameters on Large Language Model Inference Performance: An Evaluation of vLLM and HuggingFace Pipelines
Paper • 2408.01050 • Published • 9
Infra • Serving & Optimization
Inference engines, quantization, serving stacks, and perf tooling. Reference list for deployment and latency/cost work.
-
-
Seesaw: High-throughput LLM Inference via Model Re-sharding
Paper • 2503.06433 • Published -
PRIMA.CPP: Speeding Up 70B-Scale LLM Inference on Low-Resource Everyday Home Clusters
Paper • 2504.08791 • Published • 140 -
Evaluation Guidebook
📝302Explore LLM benchmark trends over time
-
Open VLM Leaderboard
🌎1.01kVLMEvalKit Evaluation Results Collection
-
Open VLM Video Leaderboard
🌎131VLMEvalKit Eval Results in video understanding benchmark
-
Qwen/Qwen2-7B-Instruct-AWQ
Text Generation • 8B • Updated • 2.9k • 22 -
TheBloke/CodeLlama-7B-Instruct-AWQ
Text Generation • 7B • Updated • 151 • 4 -
TheBloke/Mistral-7B-Instruct-v0.1-AWQ
Text Generation • 7B • Updated • 362 • 38 -
TheBloke/Mistral-7B-Instruct-v0.2-AWQ
Text Generation • 7B • Updated • 422k • 52 -
Qwen/Qwen2.5-VL-7B-Instruct-AWQ
Image-Text-to-Text • 8B • Updated • 220k • 99