google

/

gemma-4-31B-it

like 1.76k

Follow

Google 51.2k

Just wanted to share a few thoughts for my fellow lawyers and developers working on legal/English language use cases, and to express a HUGE thank you to the Google team for a fantastic OS model:

1. My deployment and use case:

• I am running a large bank of Blackwell GPUs locally (combination RTX 6000 Pros and RTX 5090s).
• I am primarily working with my law firm clients, companies that do not want to send highly confidential information to the cloud (e.g., regulated companies, startups developing cutting edge tech).
• I tried both the MOE gemma-4-26B-A4B-it and the dense gemma-4-31B-it models. I settled on on gemma-4-31B-it for my use cases. The MOE model is much faster, but the dense model has higher language comprehension for my complex tasks. I quantized and calibrated gemma-4-31B-it on my legal dataset.
• With the quantized gemma-4-31B-it model (NVFP4) and VLLM in high concurrency, I am getting prompt processing in the range of 9K tokens/second, and generation in the range of 850 tokens/second.

2. Thoughts:

• I previously used in production a number of other US Open Source models (e.g., variations of LLAMA, GPT-OSS 120b, etc.), both in the original form and trained/quantized/calibrated on my dataset. I also tested virtually all Chinese models, up to 750B parameters.
• Based on my testing so far, the gemma-4-31B-it model is the best model for my legal use case. Distinctly better comprehension of legal text in English even when compared with much larger models (actually better than ALL other models, regardless of size).
• A huge plus for my deployment is that this is released by Google, US company. Not making any geopolitical statement, but my clients are US companies with special compliance/confidentiality requirements, and I can get them comfortable with a model from Google immediately.
• I saw some user comments requesting a larger version of Gemma. I don't know about use cases outside legal language comprehension, but the community should try out this gemma-4-31B-it model for anything that deals with English language comprehension - in my tests so far, it is better than anything else in the Open Source domain for this use case. Google performed some AI miracles here. I dove a bit into the architecture of the gemma-4-31B-it model, and I can see some major departures from current OS implementations.

3. Special Thanks to Google!
   Many of us have been hoping for a new Gemma model from Google. Here it is! Multimodal, frontier performance on English language comprehension, and free. The community is grateful for this, we realize how much it takes to train a highly efficient model like this, and we realize that you are risking some cannibalization of the Gemini API revenue. Thank you, Google!!! We will repay you with loyalty in other areas.

MD