Mobile .task bundle for Android/iOS deployment?
#2
by gioni - opened
Hi,
I'm thinking of deploying TranslateGemma 4B on android mobile devices.
The current translategemma-4b-it-int8-web.task file works for web but fails when loaded via MediaPipe's LlmInference API on Android - the XNNPACK delegate can't initialize the interpreter.
Is there a mobile-compatible .task coming soon? Or should I be converting from the PyTorch weights myself using ai-edge-torch?
Any guidance appreciated!
Any update?
I got impatient, so I converted TranslateGemma 4B myself for Google AI Edge https://huggingface.co/barakplasma/translategemma-4b-it-android-task-quantized/tree/main/artifacts/int4-generic That repo contains a CPU/XNNPACK-compatible bundle .litertlm (LiteRT-LM, recommended) formats.