How to convert the Gemma3 model to LiteRT-LM format on qcom device SA8295P?
Hi, I have a qualcomm device which soc model is SA8295P. And I want to convert the Gemma3 model to LiteRT-LM format (.litertlm) to run on the SA8295P deivce. Is there any operation Instructions to convert the model ? Thanks.
we're on the same boat i have no knowledge about what you're asking me i am sorry if i can't help you with that kind of matter
Unfortunately, SA8295P is not ready at the moment. Due to hardware differences from the existing supported SOCs, it will require a different quantization scheme to run. This SOC is on the team's radar and we will post the model here when SA8295P is supported.
@marissaw thanks for your input. We will wait for your update.
At this moment, can you please point us to any notebook or tutorial or documentation on how we can convert Gemma3 (or other supported LLMs) into LiteRT-LM to deploy on any supported qualcomm device
The team has not open sourced the ability to convert LLMs into LiteRT-LM models that can run on NPUs. They are tentatively planning on making it available around Google I/O this year but the plans have not been finalized yet so they can't make any promises on exact times at the moment