muthuk1 commited on
Commit
c0294cf
Β·
verified Β·
1 Parent(s): 9a33b09

Massive README update with 12 LLM providers, OpenClaw integration, Ollama support, full architecture docs

Browse files
Files changed (1) hide show
  1. README.md +249 -123
README.md CHANGED
@@ -2,15 +2,17 @@
2
 
3
  <div align="center">
4
 
5
- [![TigerGraph](https://img.shields.io/badge/Graph_DB-TigerGraph-orange?style=for-the-badge)](https://www.tigergraph.com/)
6
- [![Claude](https://img.shields.io/badge/LLM-Claude_Sonnet_4-coral?style=for-the-badge)](https://anthropic.com/)
7
- [![Next.js](https://img.shields.io/badge/Frontend-Next.js_15-black?style=for-the-badge&logo=next.js)](https://nextjs.org/)
8
- [![HotpotQA](https://img.shields.io/badge/Benchmark-HotpotQA-purple?style=for-the-badge)](https://hotpotqa.github.io/)
9
- [![RAGAS](https://img.shields.io/badge/Evaluation-RAGAS-red?style=for-the-badge)](https://ragas.io/)
 
10
 
11
- **Proving that graphs make LLM inference faster, cheaper, and smarter β€” with real numbers.**
 
12
 
13
- [Web Dashboard](#-web-dashboard-nextjs) Β· [Architecture](#-architecture) Β· [Benchmarks](#-benchmark-results) Β· [Novelties](#-novel-features) Β· [Design System](#-design-system)
14
 
15
  </div>
16
 
@@ -18,128 +20,201 @@
18
 
19
  ## 🎯 Overview
20
 
21
- A **production-ready dual-pipeline GraphRAG system** with two interfaces:
22
 
23
- | | **Next.js Web Dashboard** | **Python CLI + Gradio** |
24
- |---|---|---|
25
- | **LLM** | Claude Sonnet 4 (Anthropic) | GPT-4o-mini (OpenAI) |
26
- | **Frontend** | React 19 + Recharts + Custom SVG | Gradio 6.x + Plotly |
27
- | **Design** | TigerGraph Γ— Claude fused design system | Standard Gradio |
28
- | **Best for** | Demos, presentations, judging | Benchmarking, batch eval |
29
 
30
- Both interfaces run the same dual-pipeline comparison:
31
 
32
- | | **Pipeline A: Baseline RAG** | **Pipeline B: GraphRAG** |
33
- |---|---|---|
34
- | **Flow** | Query β†’ Vector Search β†’ Top-K β†’ LLM | Query β†’ Keywords β†’ Entity Search β†’ Graph Traversal β†’ LLM |
35
- | **Wins on** | Speed, cost, simple queries | Accuracy on complex multi-hop queries (+21% F1) |
36
 
37
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
38
 
39
- ## πŸ—οΈ Architecture
40
 
41
- ```
42
- β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
43
- β”‚ LAYER 4: EVALUATION β€” RAGAS + F1/EM + Cost/Token Tracking β”‚
44
- β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€οΏ½οΏ½οΏ½β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
45
- β”‚ LAYER 3: LLM β€” Claude Sonnet 4 Β· Entity/Keyword Extraction β”‚
46
- β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
47
- β”‚ Pipeline A: Baseline β”‚ Pipeline B: GraphRAG β”‚
48
- │ Query→Vector→LLM │ Query→Keywords→Graph→Context→LLM │
49
- β”‚ β”‚ 🧠 Adaptive Router β”‚
50
- β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
51
- β”‚ LAYER 1: GRAPH β€” TigerGraph Cloud Β· GSQL Β· Multi-hop BFS β”‚
52
- β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
53
  ```
54
 
55
  ---
56
 
57
- ## 🌟 Novel Features
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
58
 
59
- 1. **🧠 Adaptive Query Router** β€” Automatically routes simple queries to baseline (cheaper) and complex ones to GraphRAG (more accurate)
60
- 2. **πŸ“‹ Schema-Bounded Extraction** β€” Pre-defined 9 entity types + 15 relation types (~90% cheaper, ~16% more accurate)
61
- 3. **πŸ”‘ Dual-Level Keywords** β€” LightRAG-inspired high-level + low-level keyword routing
62
- 4. **πŸ”— Graph Reasoning Paths** β€” Step-by-step natural language explanation of graph traversal
63
- 5. **πŸ“Š Real-Time Cost Tracking** β€” Every LLM call tracked with tokens, cost, and latency
 
 
 
 
 
 
64
 
65
  ---
66
 
67
- ## πŸ–₯️ Web Dashboard (Next.js)
68
 
69
- The flagship interface β€” a polished React app with the **TigerGraph Γ— Claude fused design system**.
70
 
71
- ### Quick Start
72
 
73
- ```bash
74
- cd web
75
- npm install
76
- cp .env.example .env.local
77
- # Add your Anthropic API key: ANTHROPIC_API_KEY=sk-ant-...
78
- npm run dev
79
- # Open http://localhost:3000
80
- ```
81
 
82
- ### 4 Tabs
83
 
84
- | Tab | What It Does |
85
- |-----|-------------|
86
- | **πŸ”΄ Live Compare** | Side-by-side answers from both pipelines with real-time metrics, adaptive routing badges, entity/relation display |
87
- | **πŸ“Š Benchmark** | Radar charts, bar charts, detailed comparison table with HotpotQA results |
88
- | **πŸ’° Cost Analysis** | Interactive cost projections across 4 LLM models, cumulative cost area charts, ROI analysis |
89
- | **πŸ•ΈοΈ Graph Explorer** | Interactive SVG knowledge graph with clickable nodes, reasoning path explanation, graph statistics |
90
 
91
- ### Tech Stack
92
 
93
- | Layer | Technology |
94
- |-------|-----------|
95
- | Framework | Next.js 15 (App Router) |
96
- | React | React 19 |
97
- | LLM | Claude Sonnet 4 via `@anthropic-ai/sdk` |
98
- | Charts | Recharts 2.15 |
99
- | Graph Viz | Custom SVG with interaction |
100
- | Styling | Tailwind CSS 4 + 14KB custom design system |
101
- | Fonts | Cormorant Garamond (serif display) + Inter (sans body) + JetBrains Mono (code) |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
102
 
103
  ---
104
 
105
- ## 🎨 Design System
106
 
107
- The web dashboard uses a **fused design system** combining:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
108
 
109
- - **TigerGraph**: Orange `#FF6B00` (energy, CTAs), Navy `#002B49` (authority, text), Electric Blue `#0072CE` (baseline pipeline)
110
- - **Claude/Anthropic**: Cream canvas `#faf9f5` (warmth), Coral `#cc785c` (intelligence), Dark surfaces `#181715` (product chrome)
111
 
112
- ### Key Principles
113
- - Warm cream canvas (never cold white) β€” the Claude editorial feel
114
- - Serif display headlines (Cormorant Garamond, weight 400, negative tracking) β€” literary voice
115
- - Tiger Orange for primary CTAs β€” energy and action
116
- - Dark surface code windows for architecture diagrams β€” product chrome
117
- - Cream β†’ Dark alternating section rhythm
 
 
 
 
118
 
119
  ---
120
 
121
- ## 🐍 Python Backend + Gradio
 
 
122
 
123
- The Python backend handles benchmarking, TigerGraph ingestion, and batch evaluation.
 
 
 
 
 
 
 
 
 
 
124
 
125
- ### Quick Start
126
 
127
  ```bash
128
  pip install -r requirements.txt
129
- cp .env.example .env
130
- # Add: OPENAI_API_KEY=sk-...
131
-
132
- python -m graphrag.main dashboard # Gradio UI on :7860
133
  python -m graphrag.main demo # CLI demo
134
  python -m graphrag.main benchmark --samples 50
135
- python -m graphrag.main ingest --samples 100 # Requires TigerGraph
136
  ```
137
 
138
  ---
139
 
140
  ## πŸ“Š Benchmark Results
141
 
142
- ### HotpotQA (Distractor Setting, 100 samples)
143
 
144
  | Metric | Baseline RAG | GraphRAG | Winner |
145
  |--------|-------------|----------|--------|
@@ -147,74 +222,125 @@ python -m graphrag.main ingest --samples 100 # Requires TigerGraph
147
  | **Avg EM** | 0.3810 | **0.4230** | βœ… GraphRAG (+11%) |
148
  | **Context Hit** | 0.4520 | **0.5830** | βœ… GraphRAG (+29%) |
149
  | **Tokens/Query** | **952** | 2,387 | βœ… Baseline (2.5Γ—) |
150
- | **Cost/Query** | **$0.000203** | $0.000518 | βœ… Baseline (2.6Γ—) |
151
 
152
  ### By Question Type
153
-
154
  | Type | Baseline F1 | GraphRAG F1 | Ξ” |
155
  |------|------------|-------------|---|
156
  | **Bridge** | 0.52 | **0.63** | **+21%** |
157
  | **Comparison** | 0.58 | **0.61** | +5% |
158
 
 
 
 
 
 
 
 
 
 
159
  ---
160
 
161
  ## πŸ“ Project Structure
162
 
163
  ```
164
  graphrag-inference-hackathon/
165
- β”œβ”€β”€ web/ # Next.js Web Dashboard
166
- β”‚ β”œβ”€β”€ src/app/
167
- β”‚ β”‚ β”œβ”€β”€ page.tsx # Main page
168
- β”‚ β”‚ β”œβ”€β”€ layout.tsx # Root layout
169
- β”‚ β”‚ β”œβ”€β”€ globals.css # 14KB fused design system
170
- β”‚ β”‚ └── api/compare/route.ts # Claude-powered API
171
- β”‚ β”œβ”€β”€ src/components/
172
- β”‚ β”‚ β”œβ”€β”€ Navbar.tsx # TigerGraphΓ—Claude navbar
173
- β”‚ β”‚ β”œβ”€β”€ Hero.tsx # Editorial hero with stats
174
- β”‚ β”‚ β”œβ”€β”€ DashboardTabs.tsx # Tab controller
175
- β”‚ β”‚ β”œβ”€β”€ Footer.tsx # Dark footer
176
- β”‚ β”‚ └── tabs/
177
- β”‚ β”‚ β”œβ”€β”€ LiveCompare.tsx # Tab 1: Side-by-side comparison
178
- β”‚ β”‚ β”œβ”€β”€ Benchmark.tsx # Tab 2: Radar + bar charts
179
- β”‚ β”‚ β”œβ”€β”€ CostAnalysis.tsx # Tab 3: Cost projections
180
- β”‚ β”‚ └── GraphExplorer.tsx # Tab 4: Interactive graph viz
181
- β”‚ └── src/lib/design-tokens.ts # Color + typography tokens
182
  β”‚
183
- β”œβ”€β”€ graphrag/ # Python Backend
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
184
  β”‚ β”œβ”€β”€ layers/
185
- β”‚ β”‚ β”œβ”€β”€ graph_layer.py # Layer 1: TigerGraph
186
- β”‚ β”‚ β”œβ”€β”€ orchestration_layer.py # Layer 2: Dual pipeline
187
- β”‚ β”‚ β”œβ”€β”€ llm_layer.py # Layer 3: LLM
188
- β”‚ β”‚ └── evaluation_layer.py # Layer 4: Evaluation
189
- β”‚ β”œβ”€β”€ dashboard.py # Gradio dashboard
190
- β”‚ β”œβ”€β”€ benchmark.py # Batch benchmark runner
191
- β”‚ β”œβ”€β”€ ingestion.py # Document ingestion
192
- β”‚ └── main.py # CLI entry point
 
193
  β”‚
194
- β”œβ”€β”€ requirements.txt # Python dependencies
195
- └── README.md
 
196
  ```
197
 
198
  ---
199
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
200
  ## πŸ“š References
201
 
202
  ### Papers
203
  1. [GraphRAG](https://arxiv.org/abs/2404.16130) β€” From Local to Global Graph RAG
204
  2. [LightRAG](https://arxiv.org/abs/2410.05779) β€” Simple and Fast RAG (34K⭐)
205
- 3. [HotpotQA](https://arxiv.org/abs/1809.09600) β€” Multi-hop QA Dataset
206
- 4. [RAGAS](https://arxiv.org/abs/2309.15217) β€” RAG Evaluation Framework
207
- 5. [Youtu-GraphRAG](https://arxiv.org/abs/2508.19855) β€” Schema-Bounded Extraction
 
 
 
208
 
209
- ### Tools
210
- [TigerGraph](https://tgcloud.io) Β· [Anthropic Claude](https://anthropic.com) Β· [Next.js](https://nextjs.org) Β· [Recharts](https://recharts.org) Β· [RAGAS](https://ragas.io) Β· [HotpotQA](https://huggingface.co/datasets/hotpotqa/hotpot_qa)
211
 
212
  ---
213
 
214
  <div align="center">
215
 
216
- **Built for the GraphRAG Inference Hackathon by TigerGraph** 🧑
 
 
217
 
218
- *TigerGraph Γ— Claude Β· Next.js 15 Β· Recharts Β· RAGAS*
219
 
220
  </div>
 
2
 
3
  <div align="center">
4
 
5
+ [![TigerGraph](https://img.shields.io/badge/Graph-TigerGraph-FF6B00?style=for-the-badge)](https://www.tigergraph.com/)
6
+ [![12 LLMs](https://img.shields.io/badge/LLMs-12_Providers-002B49?style=for-the-badge)](#-supported-llm-providers)
7
+ [![OpenClaw](https://img.shields.io/badge/Agent-OpenClaw-cc785c?style=for-the-badge)](#-openclaw-integration)
8
+ [![Ollama](https://img.shields.io/badge/Local-Ollama-5db872?style=for-the-badge)](#-ollama-local-models)
9
+ [![Next.js](https://img.shields.io/badge/UI-Next.js_15-000?style=for-the-badge&logo=next.js)](https://nextjs.org/)
10
+ [![RAGAS](https://img.shields.io/badge/Eval-RAGAS-c64545?style=for-the-badge)](https://ragas.io/)
11
 
12
+ **Proving that graphs make LLM inference faster, cheaper, and smarter**
13
+ **with any LLM provider β€” cloud or local.**
14
 
15
+ [12 LLM Providers](#-supported-llm-providers) Β· [OpenClaw Agent](#-openclaw-integration) Β· [Ollama Local](#-ollama-local-models) Β· [Architecture](#-architecture) Β· [Benchmarks](#-benchmark-results) Β· [Novelties](#-novel-features)
16
 
17
  </div>
18
 
 
20
 
21
  ## 🎯 Overview
22
 
23
+ A **production-ready dual-pipeline GraphRAG system** that works with **any LLM** β€” from GPT-4o to Claude to a local Llama running on your laptop via Ollama. Ships with:
24
 
25
+ - **12 LLM providers** through a single universal interface (zero per-provider SDKs)
26
+ - **OpenClaw autonomous agent integration** β€” GraphRAG as native Skills
27
+ - **Ollama local model support** β€” run completely free, no API keys needed
28
+ - **Next.js 15 web dashboard** with TigerGraph Γ— Claude fused design system
29
+ - **Python CLI + Gradio** backend for benchmarking and batch evaluation
30
+ - **4-tab comparison dashboard** β€” Live Compare, Benchmark, Cost Analysis, Graph Explorer
31
 
32
+ ---
33
 
34
+ ## πŸ€– Supported LLM Providers
 
 
 
35
 
36
+ | # | Provider | API Key Env | Default Model | Cost/1K in | Cost/1K out | Speed |
37
+ |---|----------|-------------|---------------|-----------|------------|-------|
38
+ | 1 | **OpenAI** | `OPENAI_API_KEY` | gpt-4o-mini | $0.00015 | $0.0006 | ⚑ Fast |
39
+ | 2 | **Anthropic Claude** | `ANTHROPIC_API_KEY` | claude-sonnet-4 | $0.003 | $0.015 | πŸ”΅ Medium |
40
+ | 3 | **Google Gemini** | `GEMINI_API_KEY` | gemini-2.0-flash | $0.0001 | $0.0004 | ⚑ Fast |
41
+ | 4 | **Mistral AI** | `MISTRAL_API_KEY` | mistral-large | $0.002 | $0.006 | πŸ”΅ Medium |
42
+ | 5 | **Cohere** | `COHERE_API_KEY` | command-r-plus | $0.0025 | $0.01 | πŸ”΅ Medium |
43
+ | 6 | **πŸ¦™ Ollama (Local)** | *none needed* | llama3.2 | **$0** | **$0** | ⚑ Local |
44
+ | 7 | **OpenRouter** | `OPENROUTER_API_KEY` | llama-3.3-70b | $0.0004 | $0.0004 | πŸ”΅ Medium |
45
+ | 8 | **Groq** | `GROQ_API_KEY` | llama-3.3-70b | $0.00059 | $0.00079 | ⚑⚑ Blazing |
46
+ | 9 | **xAI Grok** | `XAI_API_KEY` | grok-3-mini | $0.0003 | $0.0005 | ⚑ Fast |
47
+ | 10 | **Together AI** | `TOGETHER_API_KEY` | llama-3.1-70b-turbo | $0.00088 | $0.00088 | ⚑ Fast |
48
+ | 11 | **HuggingFace** | `HF_TOKEN` | llama-3.3-70b | **$0** | **$0** | πŸ”΅ Medium |
49
+ | 12 | **DeepSeek** | `DEEPSEEK_API_KEY` | deepseek-chat | $0.00014 | $0.00028 | ⚑ Fast |
50
 
51
+ ### How it Works
52
 
53
+ **TypeScript (Next.js):** All providers use OpenAI SDK with dynamic `baseURL` β€” zero extra dependencies. Anthropic uses its native SDK for tool_use support.
54
+
55
+ **Python:** LiteLLM provides unified routing to all 12 providers. Falls back to OpenAI SDK with `base_url` swapping.
56
+
57
+ ```bash
58
+ # Use any provider β€” just set the env var
59
+ export ANTHROPIC_API_KEY=sk-ant-... # Use Claude
60
+ export GROQ_API_KEY=gsk_... # Use Groq (blazing fast)
61
+ ollama pull llama3.2 # Use Ollama (free, local)
 
 
 
62
  ```
63
 
64
  ---
65
 
66
+ ## πŸ¦™ Ollama (Local Models)
67
+
68
+ Run the entire system **100% locally and free** with Ollama:
69
+
70
+ ```bash
71
+ # 1. Install Ollama
72
+ curl -fsSL https://ollama.ai/install.sh | sh
73
+
74
+ # 2. Pull a model
75
+ ollama pull llama3.2 # 3B params, fast
76
+ ollama pull qwen2.5:7b # 7B, good quality
77
+ ollama pull deepseek-r1:7b # Reasoning model
78
+ ollama pull phi3:14b # Strong reasoning
79
+
80
+ # 3. Start the dashboard β€” Ollama is auto-detected
81
+ cd web && npm run dev
82
+ # Select "Ollama (Local)" in the provider dropdown
83
+ ```
84
 
85
+ **Supported Ollama Models:**
86
+ | Model | Size | Quality | Use Case |
87
+ |-------|------|---------|----------|
88
+ | llama3.2 | 3B | Medium | Fast demos, entity extraction |
89
+ | llama3.2:1b | 1B | Low | Ultra-fast, keyword extraction |
90
+ | qwen2.5:7b | 7B | Medium-High | Good all-rounder |
91
+ | qwen2.5:14b | 14B | High | Best local quality |
92
+ | deepseek-r1:7b | 7B | High | Reasoning tasks |
93
+ | mistral:7b | 7B | Medium | Fast general use |
94
+ | gemma2:9b | 9B | Medium | Google's efficient model |
95
+ | phi3:14b | 14B | High | Microsoft's reasoning model |
96
 
97
  ---
98
 
99
+ ## 🦞 OpenClaw Integration
100
 
101
+ This project ships with a **full OpenClaw autonomous agent integration** β€” turning the GraphRAG system into native Skills that any OpenClaw agent can discover and invoke.
102
 
103
+ ### What is OpenClaw?
104
 
105
+ OpenClaw is the leading open-source **autonomous personal AI agent runtime**. It uses a frontier LLM as its backbone and runs continuously on the user's machine with full local system access. It's modular via a **Skills architecture** β€” exactly what we integrate here.
 
 
 
 
 
 
 
106
 
107
+ ### Architecture: CIK Model
108
 
109
+ | Dimension | Our Files | Purpose |
110
+ |-----------|-----------|---------|
111
+ | **C**apability | `openclaw/skills/` | 3 executable skills + SKILL.md docs |
112
+ | **I**dentity | `openclaw/SOUL.md`, `IDENTITY.md` | Agent persona, values, capabilities |
113
+ | **K**nowledge | `openclaw/MEMORY.md` | Learned facts about GraphRAG performance |
 
114
 
115
+ ### OpenClaw Skills
116
 
117
+ | Skill | File | What It Does |
118
+ |-------|------|-------------|
119
+ | **graph_query** | `skills/graph_query/` | Natural language β†’ knowledge graph traversal β†’ entities + relations + answer |
120
+ | **compare_pipelines** | `skills/compare_pipelines/` | Run both pipelines side-by-side with metrics comparison |
121
+ | **cost_estimate** | `skills/cost_estimate/` | Project costs across all 12 LLM providers |
122
+
123
+ ### Using with OpenClaw Agent
124
+
125
+ ```bash
126
+ # 1. Copy skills to your OpenClaw instance
127
+ cp -r openclaw/skills/ ~/.openclaw/skills/
128
+ cp openclaw/SOUL.md ~/.openclaw/
129
+ cp openclaw/IDENTITY.md ~/.openclaw/
130
+ cp openclaw/MEMORY.md ~/.openclaw/
131
+
132
+ # 2. Start the GraphRAG API server
133
+ cd web && npm run dev
134
+
135
+ # 3. Your OpenClaw agent can now use GraphRAG:
136
+ # "Search the knowledge graph for connections between Einstein and relativity"
137
+ # "Compare baseline vs GraphRAG on this question"
138
+ # "Estimate costs for 10K queries across all providers"
139
+ ```
140
+
141
+ ### Security
142
+
143
+ We follow ClawKeeper security patterns:
144
+ - No arbitrary code execution
145
+ - All API keys in environment variables only
146
+ - Graph operations are read-only by default
147
+ - Agent boundaries defined in SOUL.md
148
 
149
  ---
150
 
151
+ ## πŸ—οΈ Architecture
152
 
153
+ ```
154
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
155
+ β”‚ LAYER 4: EVALUATION β”‚
156
+ β”‚ RAGAS β”‚ F1/EM β”‚ Context Hit β”‚ Cost/Token Tracking β”‚ Dashboard β”‚
157
+ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
158
+ β”‚ LAYER 3: UNIVERSAL LLM β”‚
159
+ β”‚ 12 Providers: OpenAI β”‚ Claude β”‚ Gemini β”‚ Mistral β”‚ Ollama β”‚ Groq… β”‚
160
+ β”‚ OpenClaw Skills β”‚ Schema-Bounded Extraction β”‚ Keyword Extraction β”‚
161
+ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
162
+ β”‚ Pipeline A: Baseline RAG β”‚ Pipeline B: GraphRAG β”‚
163
+ β”‚ Query β†’ Vector β†’ LLM β”‚ Query β†’ Keywords β†’ Graph β†’ Context β†’ LLM β”‚
164
+ β”‚ β”‚ 🧠 Adaptive Router β”‚ πŸ”— Reasoning Paths β”‚
165
+ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
166
+ β”‚ LAYER 1: GRAPH (TigerGraph) β”‚
167
+ β”‚ Schema: Document β†’ Chunk β†’ Entity β†’ Community β”‚
168
+ β”‚ GSQL: Vector Search β”‚ Entity Search β”‚ Multi-Hop Traversal β”‚
169
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
170
+ ```
171
 
172
+ ---
 
173
 
174
+ ## 🌟 Novel Features
175
+
176
+ 1. **πŸ€– Universal LLM Layer** β€” Single interface for 12 providers, auto-detects available API keys
177
+ 2. **🦞 OpenClaw Agent Skills** β€” Full CIK integration (Capability + Identity + Knowledge)
178
+ 3. **πŸ¦™ Ollama Local Support** β€” $0 cost, 100% private, auto-detected
179
+ 4. **🧠 Adaptive Query Router** β€” Routes simple queries to baseline, complex to GraphRAG
180
+ 5. **πŸ“‹ Schema-Bounded Extraction** β€” 9 entity types + 15 relation types (~90% cheaper)
181
+ 6. **πŸ”‘ Dual-Level Keywords** β€” LightRAG-inspired high/low-level retrieval
182
+ 7. **πŸ”— Graph Reasoning Paths** β€” Step-by-step traversal explanations
183
+ 8. **πŸ“Š 12-Provider Cost Comparison** β€” Real-time cost projections across all providers
184
 
185
  ---
186
 
187
+ ## πŸš€ Quick Start
188
+
189
+ ### Web Dashboard (Next.js)
190
 
191
+ ```bash
192
+ cd web
193
+ npm install
194
+ cp .env.example .env.local
195
+ # Set ANY provider API key (or just use Ollama for free):
196
+ # ANTHROPIC_API_KEY=sk-ant-... OR
197
+ # OPENAI_API_KEY=sk-... OR
198
+ # ollama pull llama3.2 (free, local)
199
+ npm run dev
200
+ # β†’ http://localhost:3000
201
+ ```
202
 
203
+ ### Python Backend
204
 
205
  ```bash
206
  pip install -r requirements.txt
207
+ pip install litellm # Optional: enables all 12 providers in Python
208
+ python -m graphrag.main dashboard # Gradio UI
 
 
209
  python -m graphrag.main demo # CLI demo
210
  python -m graphrag.main benchmark --samples 50
 
211
  ```
212
 
213
  ---
214
 
215
  ## πŸ“Š Benchmark Results
216
 
217
+ ### HotpotQA (100 samples)
218
 
219
  | Metric | Baseline RAG | GraphRAG | Winner |
220
  |--------|-------------|----------|--------|
 
222
  | **Avg EM** | 0.3810 | **0.4230** | βœ… GraphRAG (+11%) |
223
  | **Context Hit** | 0.4520 | **0.5830** | βœ… GraphRAG (+29%) |
224
  | **Tokens/Query** | **952** | 2,387 | βœ… Baseline (2.5Γ—) |
 
225
 
226
  ### By Question Type
 
227
  | Type | Baseline F1 | GraphRAG F1 | Ξ” |
228
  |------|------------|-------------|---|
229
  | **Bridge** | 0.52 | **0.63** | **+21%** |
230
  | **Comparison** | 0.58 | **0.61** | +5% |
231
 
232
+ ### Cost Per Query by Provider
233
+ | Provider | Baseline | GraphRAG | Annual (1K qpd) |
234
+ |----------|----------|----------|-----------------|
235
+ | **Ollama** | **$0** | **$0** | **$0** |
236
+ | HuggingFace | $0 | $0 | $0 |
237
+ | DeepSeek | $0.000028 | $0.000071 | $26 |
238
+ | OpenAI mini | $0.000210 | $0.000530 | $193 |
239
+ | Claude Sonnet | $0.002625 | $0.006750 | $2,464 |
240
+
241
  ---
242
 
243
  ## πŸ“ Project Structure
244
 
245
  ```
246
  graphrag-inference-hackathon/
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
247
  β”‚
248
+ β”œβ”€β”€ web/ # Next.js 15 Web Dashboard
249
+ β”‚ β”œβ”€β”€ src/
250
+ β”‚ β”‚ β”œβ”€β”€ app/
251
+ β”‚ β”‚ β”‚ β”œβ”€β”€ page.tsx # Main page
252
+ β”‚ β”‚ β”‚ β”œβ”€β”€ globals.css # 14KB TigerGraphΓ—Claude design system
253
+ β”‚ β”‚ β”‚ └── api/
254
+ β”‚ β”‚ β”‚ β”œβ”€β”€ compare/route.ts # Multi-provider compare API
255
+ β”‚ β”‚ β”‚ └── providers/route.ts # Available providers listing
256
+ β”‚ β”‚ β”œβ”€β”€ components/
257
+ β”‚ β”‚ β”‚ β”œβ”€β”€ Navbar.tsx # Branded navigation
258
+ β”‚ β”‚ β”‚ β”œβ”€β”€ Hero.tsx # Editorial hero section
259
+ β”‚ β”‚ β”‚ β”œβ”€β”€ DashboardTabs.tsx # 4-tab controller
260
+ β”‚ β”‚ β”‚ β”œβ”€β”€ Footer.tsx # Dark footer
261
+ β”‚ β”‚ β”‚ └── tabs/
262
+ β”‚ β”‚ β”‚ β”œβ”€β”€ LiveCompare.tsx # Side-by-side pipeline comparison
263
+ β”‚ β”‚ β”‚ β”œβ”€β”€ Benchmark.tsx # Radar + bar charts + data table
264
+ β”‚ β”‚ β”‚ β”œβ”€β”€ CostAnalysis.tsx # 12-provider cost projections
265
+ β”‚ β”‚ β”‚ └── GraphExplorer.tsx # Interactive SVG knowledge graph
266
+ β”‚ β”‚ └── lib/
267
+ β”‚ β”‚ β”œβ”€β”€ llm-providers.ts # Universal 12-provider LLM client
268
+ β”‚ β”‚ └── design-tokens.ts # Color/typography tokens
269
+ β”‚ └── package.json
270
+ β”‚
271
+ β”œβ”€β”€ openclaw/ # OpenClaw Agent Integration
272
+ β”‚ β”œβ”€β”€ SOUL.md # Agent identity & values
273
+ β”‚ β”œβ”€β”€ IDENTITY.md # Agent configuration
274
+ β”‚ β”œβ”€β”€ MEMORY.md # Learned knowledge base
275
+ β”‚ └── skills/
276
+ β”‚ β”œβ”€β”€ graph_query/ # Knowledge graph querying
277
+ β”‚ β”‚ β”œβ”€β”€ SKILL.md
278
+ β”‚ β”‚ └── graph_query.py
279
+ β”‚ β”œβ”€β”€ compare_pipelines/ # Dual-pipeline comparison
280
+ β”‚ β”‚ β”œβ”€β”€ SKILL.md
281
+ β”‚ β”‚ └── compare_pipelines.py
282
+ β”‚ └── cost_estimate/ # 12-provider cost projection
283
+ β”‚ β”œβ”€β”€ SKILL.md
284
+ β”‚ └── cost_estimate.py
285
+ β”‚
286
+ β”œβ”€β”€ graphrag/ # Python Backend
287
  β”‚ β”œβ”€β”€ layers/
288
+ β”‚ β”‚ β”œβ”€β”€ universal_llm.py # LiteLLM-powered 12-provider support
289
+ β”‚ β”‚ β”œβ”€β”€ graph_layer.py # TigerGraph schema + GSQL queries
290
+ β”‚ β”‚ β”œβ”€β”€ orchestration_layer.py # Dual pipeline routing
291
+ β”‚ β”‚ β”œβ”€β”€ llm_layer.py # Original LLM layer
292
+ β”‚ β”‚ └── evaluation_layer.py # RAGAS + F1/EM metrics
293
+ β”‚ β”œβ”€β”€ dashboard.py # Gradio dashboard
294
+ β”‚ β”œβ”€β”€ benchmark.py # HotpotQA benchmark runner
295
+ β”‚ β”œβ”€β”€ ingestion.py # Document ingestion pipeline
296
+ β”‚ └── main.py # CLI entry point
297
  β”‚
298
+ β”œβ”€β”€ requirements.txt
299
+ β”œβ”€β”€ .env.example # All 12 provider keys
300
+ └── README.md # This file
301
  ```
302
 
303
  ---
304
 
305
+ ## πŸ› οΈ Tech Stack
306
+
307
+ | Layer | Technology |
308
+ |-------|-----------|
309
+ | **Graph Database** | TigerGraph Cloud (free tier) |
310
+ | **LLM Providers** | 12 providers via universal interface |
311
+ | **Local LLM** | Ollama (llama3.2, qwen2.5, deepseek-r1, etc.) |
312
+ | **Agent Framework** | OpenClaw (CIK model: Skills + Identity + Memory) |
313
+ | **Web Frontend** | Next.js 15, React 19, Recharts, Tailwind CSS 4 |
314
+ | **Design System** | TigerGraph Γ— Claude fused (14KB custom CSS) |
315
+ | **Python Backend** | LiteLLM, RAGAS, HotpotQA, NetworkX |
316
+ | **Evaluation** | RAGAS v0.2, F1/EM (SQuAD standard), Context Hit Rate |
317
+ | **Fonts** | Cormorant Garamond (serif) + Inter (sans) + JetBrains Mono |
318
+
319
+ ---
320
+
321
  ## πŸ“š References
322
 
323
  ### Papers
324
  1. [GraphRAG](https://arxiv.org/abs/2404.16130) β€” From Local to Global Graph RAG
325
  2. [LightRAG](https://arxiv.org/abs/2410.05779) β€” Simple and Fast RAG (34K⭐)
326
+ 3. [OpenClaw](https://github.com/Gen-Verse/OpenClaw) β€” Personal AI Agent Runtime
327
+ 4. [OpenClaw-RL](https://arxiv.org/abs/2603.10165) β€” RL from Live Interactions (5K⭐)
328
+ 5. [ClawKeeper](https://arxiv.org/abs/2604.04759) β€” OpenClaw Security Framework
329
+ 6. [HotpotQA](https://arxiv.org/abs/1809.09600) β€” Multi-hop QA Dataset
330
+ 7. [RAGAS](https://arxiv.org/abs/2309.15217) β€” RAG Evaluation Framework
331
+ 8. [Youtu-GraphRAG](https://arxiv.org/abs/2508.19855) β€” Schema-Bounded Extraction
332
 
333
+ ### Tools & Services
334
+ [TigerGraph](https://tgcloud.io) Β· [Anthropic](https://anthropic.com) Β· [OpenAI](https://openai.com) Β· [Ollama](https://ollama.ai) Β· [Groq](https://groq.com) Β· [OpenRouter](https://openrouter.ai) Β· [LiteLLM](https://litellm.ai) Β· [Next.js](https://nextjs.org) Β· [Recharts](https://recharts.org) Β· [RAGAS](https://ragas.io)
335
 
336
  ---
337
 
338
  <div align="center">
339
 
340
+ ### πŸ† Built for the GraphRAG Inference Hackathon by TigerGraph
341
+
342
+ **12 LLM Providers** Β· **OpenClaw Agent** Β· **Ollama Local** Β· **TigerGraph** Β· **Next.js 15**
343
 
344
+ *Proving that graphs make LLM inference faster, cheaper, and smarter β€” with any LLM.*
345
 
346
  </div>