EmbeddedLLM

/

vLLM-AMD-flash-attn-debug

like 0

Follow

EmbeddedLLM 46

• "torch.storage._load_from_bytes",
• "vllm.sampling_params.SamplingParams",
• "torch._utils._rebuild_tensor_v2",
• "array.array",
• "collections.OrderedDict",
• "vllm.worker.model_runner.ModelInputForGPUWithSamplingMetadata",
• "vllm.sequence.SequenceStage",
• "vllm.sequence.SequenceData",
• "vllm.sampling_params.RequestOutputKind",
• "vllm.model_executor.sampling_metadata.SequenceGroupToSample",
• "vllm.attention.backends.rocm_flash_attn.ROCmFlashAttentionMetadata",
• "vllm.model_executor.sampling_metadata.SamplingMetadata",
• "array._array_reconstructor",
• "vllm.sampling_params.SamplingType"

How to fix it?