Edge LLM Deployments - a Makatia Collection

Makatia 's Collections

Edge LLM Deployments

Hardware ML & Neural Signal Processing

Edge LLM Deployments

updated 26 days ago

LLMs quantized for Raspberry Pi and ARM edge. GGUF + ONNX. No cloud required.

Makatia/mistral-7b-instruct-v0.2.Q8_0-Q8_0.gguf

Text Generation • 7B • Updated 26 days ago • 10

Note Mistral 7B Q8_0 GGUF. Near-lossless 8-bit for llama.cpp and LM Studio.
Makatia/microsoft_Phi-3-mini-4k-instruct_onnx_rpi

Text Generation • Updated 26 days ago

Note Phi-3 Mini 3.8B in quantized ONNX for RPi5.
Makatia/TinyLlama_TinyLlama-1.1B-Chat-v1.0_onnx

Text Generation • Updated 26 days ago • 2

Note TinyLlama 1.1B ONNX. Smallest viable LLM for edge.