Massive README update with 12 LLM providers, OpenClaw integration, Ollama support, full architecture docs

c0294cf verified 27 days ago

preview code

raw

history blame

16.2 kB

🔍 GraphRAG Inference Hackathon — Dual Pipeline System

Proving that graphs make LLM inference faster, cheaper, and smarter with any LLM provider — cloud or local.

12 LLM Providers · OpenClaw Agent · Ollama Local · Architecture · Benchmarks · Novelties

🎯 Overview

A production-ready dual-pipeline GraphRAG system that works with any LLM — from GPT-4o to Claude to a local Llama running on your laptop via Ollama. Ships with:

12 LLM providers through a single universal interface (zero per-provider SDKs)
OpenClaw autonomous agent integration — GraphRAG as native Skills
Ollama local model support — run completely free, no API keys needed
Next.js 15 web dashboard with TigerGraph × Claude fused design system
Python CLI + Gradio backend for benchmarking and batch evaluation
4-tab comparison dashboard — Live Compare, Benchmark, Cost Analysis, Graph Explorer

🤖 Supported LLM Providers

#	Provider	API Key Env	Default Model	Cost/1K in	Cost/1K out	Speed
1	OpenAI	`OPENAI_API_KEY`	gpt-4o-mini	$0.00015	$0.0006	⚡ Fast
2	Anthropic Claude	`ANTHROPIC_API_KEY`	claude-sonnet-4	$0.003	$0.015	🔵 Medium
3	Google Gemini	`GEMINI_API_KEY`	gemini-2.0-flash	$0.0001	$0.0004	⚡ Fast
4	Mistral AI	`MISTRAL_API_KEY`	mistral-large	$0.002	$0.006	🔵 Medium
5	Cohere	`COHERE_API_KEY`	command-r-plus	$0.0025	$0.01	🔵 Medium
6	🦙 Ollama (Local)	none needed	llama3.2	$0	$0	⚡ Local
7	OpenRouter	`OPENROUTER_API_KEY`	llama-3.3-70b	$0.0004	$0.0004	🔵 Medium
8	Groq	`GROQ_API_KEY`	llama-3.3-70b	$0.00059	$0.00079	⚡⚡ Blazing
9	xAI Grok	`XAI_API_KEY`	grok-3-mini	$0.0003	$0.0005	⚡ Fast
10	Together AI	`TOGETHER_API_KEY`	llama-3.1-70b-turbo	$0.00088	$0.00088	⚡ Fast
11	HuggingFace	`HF_TOKEN`	llama-3.3-70b	$0	$0	🔵 Medium
12	DeepSeek	`DEEPSEEK_API_KEY`	deepseek-chat	$0.00014	$0.00028	⚡ Fast

How it Works

TypeScript (Next.js): All providers use OpenAI SDK with dynamic baseURL — zero extra dependencies. Anthropic uses its native SDK for tool_use support.

Python: LiteLLM provides unified routing to all 12 providers. Falls back to OpenAI SDK with base_url swapping.

# Use any provider — just set the env var
export ANTHROPIC_API_KEY=sk-ant-...   # Use Claude
export GROQ_API_KEY=gsk_...          # Use Groq (blazing fast)
ollama pull llama3.2                  # Use Ollama (free, local)

🦙 Ollama (Local Models)

Run the entire system 100% locally and free with Ollama:

# 1. Install Ollama
curl -fsSL https://ollama.ai/install.sh | sh

# 2. Pull a model
ollama pull llama3.2         # 3B params, fast
ollama pull qwen2.5:7b       # 7B, good quality
ollama pull deepseek-r1:7b   # Reasoning model
ollama pull phi3:14b          # Strong reasoning

# 3. Start the dashboard — Ollama is auto-detected
cd web && npm run dev
# Select "Ollama (Local)" in the provider dropdown

Supported Ollama Models:

Model	Size	Quality	Use Case
llama3.2	3B	Medium	Fast demos, entity extraction
llama3.2:1b	1B	Low	Ultra-fast, keyword extraction
qwen2.5:7b	7B	Medium-High	Good all-rounder
qwen2.5:14b	14B	High	Best local quality
deepseek-r1:7b	7B	High	Reasoning tasks
mistral:7b	7B	Medium	Fast general use
gemma2:9b	9B	Medium	Google's efficient model
phi3:14b	14B	High	Microsoft's reasoning model

🦞 OpenClaw Integration

This project ships with a full OpenClaw autonomous agent integration — turning the GraphRAG system into native Skills that any OpenClaw agent can discover and invoke.

What is OpenClaw?

OpenClaw is the leading open-source autonomous personal AI agent runtime. It uses a frontier LLM as its backbone and runs continuously on the user's machine with full local system access. It's modular via a Skills architecture — exactly what we integrate here.

Architecture: CIK Model

Dimension	Our Files	Purpose
Capability	`openclaw/skills/`	3 executable skills + SKILL.md docs
Identity	`openclaw/SOUL.md`, `IDENTITY.md`	Agent persona, values, capabilities
Knowledge	`openclaw/MEMORY.md`	Learned facts about GraphRAG performance

OpenClaw Skills

Skill	File	What It Does
graph_query	`skills/graph_query/`	Natural language → knowledge graph traversal → entities + relations + answer
compare_pipelines	`skills/compare_pipelines/`	Run both pipelines side-by-side with metrics comparison
cost_estimate	`skills/cost_estimate/`	Project costs across all 12 LLM providers

Using with OpenClaw Agent

# 1. Copy skills to your OpenClaw instance
cp -r openclaw/skills/ ~/.openclaw/skills/
cp openclaw/SOUL.md ~/.openclaw/
cp openclaw/IDENTITY.md ~/.openclaw/
cp openclaw/MEMORY.md ~/.openclaw/

# 2. Start the GraphRAG API server
cd web && npm run dev

# 3. Your OpenClaw agent can now use GraphRAG:
# "Search the knowledge graph for connections between Einstein and relativity"
# "Compare baseline vs GraphRAG on this question"
# "Estimate costs for 10K queries across all providers"

Security

We follow ClawKeeper security patterns:

No arbitrary code execution
All API keys in environment variables only
Graph operations are read-only by default
Agent boundaries defined in SOUL.md

🏗️ Architecture

┌──────────────────────────────────────────────────────────────────────┐
│                    LAYER 4: EVALUATION                                │
│  RAGAS │ F1/EM │ Context Hit │ Cost/Token Tracking │ Dashboard       │
├──────────────────────────────────────────────────────────────────────┤
│                    LAYER 3: UNIVERSAL LLM                             │
│  12 Providers: OpenAI │ Claude │ Gemini │ Mistral │ Ollama │ Groq…  │
│  OpenClaw Skills │ Schema-Bounded Extraction │ Keyword Extraction     │
├────────────────────────────┬─────────────────────────────────────────┤
│  Pipeline A: Baseline RAG  │  Pipeline B: GraphRAG                   │
│  Query → Vector → LLM      │  Query → Keywords → Graph → Context → LLM │
│                            │  🧠 Adaptive Router │ 🔗 Reasoning Paths │
├────────────────────────────┴─────────────────────────────────────────┤
│                    LAYER 1: GRAPH (TigerGraph)                        │
│  Schema: Document → Chunk → Entity → Community                       │
│  GSQL: Vector Search │ Entity Search │ Multi-Hop Traversal            │
└──────────────────────────────────────────────────────────────────────┘

🌟 Novel Features

🤖 Universal LLM Layer — Single interface for 12 providers, auto-detects available API keys
🦞 OpenClaw Agent Skills — Full CIK integration (Capability + Identity + Knowledge)
🦙 Ollama Local Support — $0 cost, 100% private, auto-detected
🧠 Adaptive Query Router — Routes simple queries to baseline, complex to GraphRAG
📋 Schema-Bounded Extraction — 9 entity types + 15 relation types (~90% cheaper)
🔑 Dual-Level Keywords — LightRAG-inspired high/low-level retrieval
🔗 Graph Reasoning Paths — Step-by-step traversal explanations
📊 12-Provider Cost Comparison — Real-time cost projections across all providers

🚀 Quick Start

Web Dashboard (Next.js)

cd web
npm install
cp .env.example .env.local
# Set ANY provider API key (or just use Ollama for free):
# ANTHROPIC_API_KEY=sk-ant-...  OR
# OPENAI_API_KEY=sk-...          OR
# ollama pull llama3.2            (free, local)
npm run dev
# → http://localhost:3000

Python Backend

pip install -r requirements.txt
pip install litellm  # Optional: enables all 12 providers in Python
python -m graphrag.main dashboard    # Gradio UI
python -m graphrag.main demo         # CLI demo
python -m graphrag.main benchmark --samples 50

📊 Benchmark Results

HotpotQA (100 samples)

Metric	Baseline RAG	GraphRAG	Winner
Avg F1	0.5523	0.6241	✅ GraphRAG (+13%)
Avg EM	0.3810	0.4230	✅ GraphRAG (+11%)
Context Hit	0.4520	0.5830	✅ GraphRAG (+29%)
Tokens/Query	952	2,387	✅ Baseline (2.5×)

By Question Type

Type	Baseline F1	GraphRAG F1	Δ
Bridge	0.52	0.63	+21%
Comparison	0.58	0.61	+5%

Cost Per Query by Provider

Provider	Baseline	GraphRAG	Annual (1K qpd)
Ollama	$0	$0	$0
HuggingFace	$0	$0	$0
DeepSeek	$0.000028	$0.000071	$26
OpenAI mini	$0.000210	$0.000530	$193
Claude Sonnet	$0.002625	$0.006750	$2,464

📁 Project Structure

graphrag-inference-hackathon/
│
├── web/                                # Next.js 15 Web Dashboard
│   ├── src/
│   │   ├── app/
│   │   │   ├── page.tsx                # Main page
│   │   │   ├── globals.css             # 14KB TigerGraph×Claude design system
│   │   │   └── api/
│   │   │       ├── compare/route.ts    # Multi-provider compare API
│   │   │       └── providers/route.ts  # Available providers listing
│   │   ├── components/
│   │   │   ├── Navbar.tsx              # Branded navigation
│   │   │   ├── Hero.tsx                # Editorial hero section
│   │   │   ├── DashboardTabs.tsx       # 4-tab controller
│   │   │   ├── Footer.tsx              # Dark footer
│   │   │   └── tabs/
│   │   │       ├── LiveCompare.tsx     # Side-by-side pipeline comparison
│   │   │       ├── Benchmark.tsx       # Radar + bar charts + data table
│   │   │       ├── CostAnalysis.tsx    # 12-provider cost projections
│   │   │       └── GraphExplorer.tsx   # Interactive SVG knowledge graph
│   │   └── lib/
│   │       ├── llm-providers.ts        # Universal 12-provider LLM client
│   │       └── design-tokens.ts        # Color/typography tokens
│   └── package.json
│
├── openclaw/                           # OpenClaw Agent Integration
│   ├── SOUL.md                         # Agent identity & values
│   ├── IDENTITY.md                     # Agent configuration
│   ├── MEMORY.md                       # Learned knowledge base
│   └── skills/
│       ├── graph_query/                # Knowledge graph querying
│       │   ├── SKILL.md
│       │   └── graph_query.py
│       ├── compare_pipelines/          # Dual-pipeline comparison
│       │   ├── SKILL.md
│       │   └── compare_pipelines.py
│       └── cost_estimate/              # 12-provider cost projection
│           ├── SKILL.md
│           └── cost_estimate.py
│
├── graphrag/                           # Python Backend
│   ├── layers/
│   │   ├── universal_llm.py           # LiteLLM-powered 12-provider support
│   │   ├── graph_layer.py             # TigerGraph schema + GSQL queries
│   │   ├── orchestration_layer.py     # Dual pipeline routing
│   │   ├── llm_layer.py               # Original LLM layer
│   │   └── evaluation_layer.py        # RAGAS + F1/EM metrics
│   ├── dashboard.py                    # Gradio dashboard
│   ├── benchmark.py                    # HotpotQA benchmark runner
│   ├── ingestion.py                    # Document ingestion pipeline
│   └── main.py                         # CLI entry point
│
├── requirements.txt
├── .env.example                        # All 12 provider keys
└── README.md                           # This file

🛠️ Tech Stack

Layer	Technology
Graph Database	TigerGraph Cloud (free tier)
LLM Providers	12 providers via universal interface
Local LLM	Ollama (llama3.2, qwen2.5, deepseek-r1, etc.)
Agent Framework	OpenClaw (CIK model: Skills + Identity + Memory)
Web Frontend	Next.js 15, React 19, Recharts, Tailwind CSS 4
Design System	TigerGraph × Claude fused (14KB custom CSS)
Python Backend	LiteLLM, RAGAS, HotpotQA, NetworkX
Evaluation	RAGAS v0.2, F1/EM (SQuAD standard), Context Hit Rate
Fonts	Cormorant Garamond (serif) + Inter (sans) + JetBrains Mono

📚 References

Papers

GraphRAG — From Local to Global Graph RAG
LightRAG — Simple and Fast RAG (34K⭐)
OpenClaw — Personal AI Agent Runtime
OpenClaw-RL — RL from Live Interactions (5K⭐)
ClawKeeper — OpenClaw Security Framework
HotpotQA — Multi-hop QA Dataset
RAGAS — RAG Evaluation Framework
Youtu-GraphRAG — Schema-Bounded Extraction

Tools & Services

TigerGraph · Anthropic · OpenAI · Ollama · Groq · OpenRouter · LiteLLM · Next.js · Recharts · RAGAS

🏆 Built for the GraphRAG Inference Hackathon by TigerGraph

12 LLM Providers · OpenClaw Agent · Ollama Local · TigerGraph · Next.js 15

Proving that graphs make LLM inference faster, cheaper, and smarter — with any LLM.