Spaces:
Running
Running
docs: update custom OpenAI-compatible provider documentation with a structured configuration table and updated example
Browse files
README.md
CHANGED
|
@@ -221,48 +221,28 @@ The provider prefix in `LLM_MODEL` tells HuggingClaw how to call it. See [OpenCl
|
|
| 221 |
|
| 222 |
### Custom OpenAI-Compatible Provider
|
| 223 |
|
| 224 |
-
|
| 225 |
|
| 226 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 227 |
|
|
|
|
| 228 |
```bash
|
| 229 |
-
LLM_API_KEY=your_api_key
|
| 230 |
-
LLM_MODEL=modal/zai-org/GLM-5.1-FP8
|
| 231 |
CUSTOM_PROVIDER_NAME=modal
|
| 232 |
CUSTOM_BASE_URL=https://api.us-west-2.modal.direct/v1
|
| 233 |
CUSTOM_MODEL_ID=zai-org/GLM-5.1-FP8
|
|
|
|
| 234 |
```
|
| 235 |
|
| 236 |
-
Optional env vars:
|
| 237 |
-
|
| 238 |
-
```bash
|
| 239 |
-
CUSTOM_MODEL_NAME=GLM-5.1-FP8
|
| 240 |
-
CUSTOM_API_KEY=your_custom_api_key
|
| 241 |
-
CUSTOM_API_TYPE=openai-completions
|
| 242 |
-
CUSTOM_CONTEXT_WINDOW=128000
|
| 243 |
-
CUSTOM_MAX_TOKENS=500
|
| 244 |
-
```
|
| 245 |
-
|
| 246 |
-
Notes:
|
| 247 |
-
|
| 248 |
-
- This is for **OpenAI-compatible** endpoints only.
|
| 249 |
-
- `CUSTOM_BASE_URL` should be the API base URL, not `/chat/completions`.
|
| 250 |
-
- `CUSTOM_PROVIDER_NAME` must be a new name and cannot override built-in providers like `openai`, `openrouter`, `google`, or `anthropic`.
|
| 251 |
-
- If `CUSTOM_API_KEY` is not set, HuggingClaw uses `LLM_API_KEY`.
|
| 252 |
-
|
| 253 |
-
Examples:
|
| 254 |
-
|
| 255 |
-
- Modal-hosted model:
|
| 256 |
-
- `CUSTOM_PROVIDER_NAME=modal`
|
| 257 |
-
- `CUSTOM_BASE_URL=https://api.us-west-2.modal.direct/v1`
|
| 258 |
-
- `CUSTOM_MODEL_ID=zai-org/GLM-5.1-FP8`
|
| 259 |
-
- `LLM_MODEL=modal/zai-org/GLM-5.1-FP8`
|
| 260 |
-
- Self-hosted vLLM:
|
| 261 |
-
- `CUSTOM_PROVIDER_NAME=vllm`
|
| 262 |
-
- `CUSTOM_BASE_URL=https://your-vllm.example.com/v1`
|
| 263 |
-
- `CUSTOM_MODEL_ID=Qwen/Qwen2.5-72B-Instruct`
|
| 264 |
-
- `LLM_MODEL=vllm/Qwen/Qwen2.5-72B-Instruct`
|
| 265 |
-
|
| 266 |
## 💻 Local Development
|
| 267 |
|
| 268 |
```bash
|
|
|
|
| 221 |
|
| 222 |
### Custom OpenAI-Compatible Provider
|
| 223 |
|
| 224 |
+
Register a custom endpoint at startup without modifying the CLI.
|
| 225 |
|
| 226 |
+
| Variable | Description | Default |
|
| 227 |
+
| :--- | :--- | :--- |
|
| 228 |
+
| `CUSTOM_PROVIDER_NAME` | Unique provider prefix (e.g., `modal`) | **Required** |
|
| 229 |
+
| `CUSTOM_BASE_URL` | API base URL (e.g., `https://.../v1`) | **Required** |
|
| 230 |
+
| `CUSTOM_MODEL_ID` | Model ID on the server | **Required** |
|
| 231 |
+
| `LLM_MODEL` | Must match `{CUSTOM_PROVIDER_NAME}/{CUSTOM_MODEL_ID}` | **Required** |
|
| 232 |
+
| `CUSTOM_API_KEY` | Provider-specific key | `LLM_API_KEY` |
|
| 233 |
+
| `CUSTOM_CONTEXT_WINDOW` | Context limit | `128000` |
|
| 234 |
+
|
| 235 |
+
> [!TIP]
|
| 236 |
+
> `CUSTOM_PROVIDER_NAME` cannot override built-in providers (openai, anthropic, etc.).
|
| 237 |
|
| 238 |
+
**Example (Modal):**
|
| 239 |
```bash
|
|
|
|
|
|
|
| 240 |
CUSTOM_PROVIDER_NAME=modal
|
| 241 |
CUSTOM_BASE_URL=https://api.us-west-2.modal.direct/v1
|
| 242 |
CUSTOM_MODEL_ID=zai-org/GLM-5.1-FP8
|
| 243 |
+
LLM_MODEL=modal/zai-org/GLM-5.1-FP8
|
| 244 |
```
|
| 245 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 246 |
## 💻 Local Development
|
| 247 |
|
| 248 |
```bash
|