GPU memory requirements
#30
by ak1020 - opened
I'm trying to run model with "NVIDIA RTX 1000 Ada Generation Laptop GPU" for inference with 6.0 GB total memory. My audio file is less than 6 minutes. I get error "RuntimeError: CUDA driver error: out of memory". If I first run the model with small example .wav and then again with my .wav, model works OK.
Use INT8 quantized version with Sherpa-onnx api
ak1020 changed discussion status to closed