No longer works with vllm
#5
by bgreene010 - opened
FYI, after an update in vllm, it crashes with this model, with the following error.
RuntimeError: No Qwen3Next layer found in the model.layers
This is the first commit in vllm where this happens:
commit e50c45467215f96068d95736b08d8a25f624e67d
Author: Ilya Markov <markovilya197@gmail.com>
Date: Wed Nov 5 16:22:17 2025 +0100
[BugFix] Support EP/DP + EPLB with MTP (#25311)
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
https://github.com/vllm-project/vllm/pull/28960 - you can fix it in venv or in docker image manually. Very simple.