fix exceptions

fix error format wrapping now applies to /v1/chat/completions and generation stats

change timeouts

add token generation speed to ui

fix request parsing

Log detailed error bodies for UI failures

Fix 400 for llama.cpp web UI completion requests

Fix web UI chat by adding buffered SSE fallback

Fix Docker build for modular llm-manager

fix description

fix build bugs

fix readme file

add new architectural diagram to readme file

Refactor the C++ LLM manager into modular components, moves Python modules under python/, and keeps the current control-plane behavior intact. The C++ server now has clearer separation for config, model lifecycle, runtime services, request parsing, HTTP helpers, and server routing, while Docker build/runtime paths were updated to compile multiple C++ files and load Python code from the new package folder

add auth, token policy, queue scheduler, and cancel flow, etc

add new endpoint to cancel all processing prompts

add new build profile

fix encoding

fix model config

fix proxied response in llm manager

fix routing in llm manager

add cpp server

change llm model

change llm model

change model to Qwen2.5-Math-7B-Instruct-GGUF

change llm model to qwen2 math

change llm model to mistral

fix dockerfile

change compilation flags

change compilation flags

reduce context and batch

fix repo name of model

fix repo of model

fix cmd in dockerfile

fix dockerfile

switch to qwen model via cpp server

Log elapsed time and token rate when the response arrives.

a “slow request” logging, log when a request exceeds the budgeted prompt and gets compacted

add simple compacting

fix context window size

fix payload processing

fix logger

fix app to handle exceptions

add requests logging

fix dockerfile

fix dockerfile

fix gitignore, app and logger, etc

add readme

add readme

Add automatic API documentation and in-memory model caching