Hardware Spec for local run

#15
by Bingmao1 - opened

I would like to know what hardware you need to run this locally. I tried to run it with an Nvidia RTX3090 (24GB) with NVIDIA Container Toolkit and ran in an Out-Of-Memory issue. Did anyone got it running locally and which GPU do you have?

How i set it up:

  1. Update Ubuntu:
    sudo apt update && sudo apt upgrade -y

  2. Install NVIDIA Drivers:
    ubuntu-drivers devices
    sudo apt install nvidia-driver-XXX # Replace XXX with recommended version
    sudo reboot

  3. Install Docker and NVIDIA Container Toolkit:
    sudo apt install apt-transport-https ca-certificates curl software-properties-common -y
    curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
    sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
    sudo apt update
    sudo apt install docker-ce -y

sudo usermod -aG docker $USER
newgrp docker

distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt update
sudo apt install -y nvidia-docker2
sudo systemctl restart docker

  1. Test GPU support:
    docker run --rm --gpus all nvidia/cuda:13.1.1-cudnn-devel-ubuntu24.04 nvidia-smi

  2. Run Space:
    docker run -it -p 7860:7860 --platform=linux/amd64 --gpus all
    registry.hf.space/r3gm-wan2-2-fp8da-aoti-preview:latest python app.py

The model itself is about 68 GB in total, so it may just not fit even with the 8-bit quantization.

Bingmao1 changed discussion status to closed

You could also look for ComfyUI workflows though. I haven't tried this, but I've read people have done similar things on consumer GPUs.

Sign up or log in to comment