Quantized ONNX model not supported on armeabi-v7a (ai.onnx.ml opset 5 / ConvInteger)
Hi,
I’m working on an armeabi-v7a platform and I’m facing compatibility issues when loading quantized ONNX models with ONNX Runtime.
When loading a quantized model on our device, ONNX Runtime throws the following error:
“ONNX Runtime only guarantees support for models stamped with official released ONNX opset versions. Opset 5 is under development and support for this is limited. The operator schemas and/or other functionality may change before the next ONNX release, and in this case ONNX Runtime will not guarantee backward compatibility. Current official support for domain ai.onnx.ml is up to opset 4.”
Because armeabi-v7a only supports ONNX Runtime up to v1.17 (or v1.18), I cannot use newer ONNX Runtime versions (e.g. 1.20 / 1.21) that support ai.onnx.ml opset 5.
I also tried converting the Kokoro TTS PyTorch (.pth) model to ONNX and quantizing it using ONNX opset 17, but I encountered another runtime error on Android:
“ORT_NOT_IMPLEMENTED: Could not find an implementation for ConvInteger”
So at the moment I’m blocked by two issues:
- ai.onnx.ml opset 5 is required by official quantized models, but is not supported by ONNX Runtime versions available on armeabi-v7a.
- Re-exporting and quantizing the model with an older opset avoids the ml-opset issue, but introduces ConvInteger, which is not implemented in the armeabi-v7a ONNX Runtime build.
Could anyone help clarify the following points?
- Whether there is any recommended way to generate a quantized ONNX model compatible with ONNX Runtime ≤ 1.17 on armeabi-v7a?
- Or if there is a known workaround to avoid unsupported operators like ConvInteger during quantization?
Any guidance would be really appreciated
Thanks a lot for your help.