ARCHITECTURE.md

# Graph RAG Service - Project Documentation

## System Architecture

### Overview
The Graph RAG Service is built as a modular, production-grade platform with the following key components:

1. **API Gateway (FastAPI)**: Handles all HTTP requests, authentication, and routing
2. **Ingestion Pipeline**: Processes documents and constructs knowledge graphs
3. **Retrieval Agent (LangGraph)**: Intelligent query routing and response synthesis
4. **Storage Layer**: Neo4j for graph + vector storage
5. **Task Queue**: Celery + Redis for async processing
6. **Observability**: OpenTelemetry for tracing and metrics

### Design Principles

#### 1. No Vendor Lock-in
All core components are abstracted behind interfaces:
- `GraphStore`: Can swap Neo4j for AWS Neptune
- `VectorStore`: Supports multiple vector databases
- `LLMProvider`: Works with any LLM (OpenAI, Anthropic, Gemini, Ollama)

#### 2. Production-Ready
- **Async Processing**: Non-blocking I/O for all database operations
- **Background Jobs**: Celery workers handle heavy ingestion tasks
- **Authentication**: JWT-based with RBAC support
- **Error Handling**: Graceful degradation and fallback mechanisms
- **Observability**: Full tracing and metrics collection

#### 3. Intelligent Retrieval
The agentic system:
- Decomposes complex queries into sub-queries
- Dynamically selects retrieval methods (vector vs graph vs cypher)
- Validates outputs against schema (hallucination guard)
- Provides reasoning chains for transparency

## Components Deep Dive

### Core Abstractions (`src/graph_rag_service/core/`)

#### GraphStore Interface
```python
class GraphStore(ABC):
    @abstractmethod
    async def create_node(entity: Entity) -> str
    @abstractmethod
    async def create_relationship(relationship: Relationship) -> str
    @abstractmethod
    async def execute_query(query: str, params: dict) -> List[dict]
    @abstractmethod
    async def find_path(source: str, target: str, max_depth: int) -> List[dict]
```

Implementation: `Neo4jStore` provides unified graph + vector storage using Neo4j 5.x vector capabilities.

#### LLMProvider Interface
```python
class LLMProvider(ABC):
    @abstractmethod
    async def complete(prompt: str, **kwargs) -> str
    @abstractmethod
    async def embed(text: str) -> List[float]
```

Implementation: `UnifiedLLMProvider` wraps OpenAI, Anthropic, Gemini, and Ollama with a consistent interface.

#### Entity Resolution
Multi-stage resolution:
1. **Blocking**: Group by entity type and name similarity (fast reject)
2. **Semantic Check**: Compare embeddings for deep similarity
3. **Threshold Matching**: Configurable thresholds (0.85 default)
4. **Auto-merge**: High confidence merges (>0.95)
5. **Human Review Queue**: Medium confidence flagged for review (0.85-0.95)

### Ingestion Pipeline (`src/graph_rag_service/ingestion/`)

#### Flow
1. **Document Processing**: Extract text from PDF/TXT/MD/DOCX
2. **Chunking**: Split into overlapping chunks (1024 tokens, 200 overlap)
3. **Ontology Generation**: LLM analyzes samples to propose entity/relationship types
4. **Entity Extraction**: Extract entities and relationships per chunk
5. **Entity Resolution**: Deduplicate and merge entities
6. **Embedding Generation**: Create vector embeddings (BGE-M3)
7. **Graph Construction**: Store in Neo4j with hybrid nodes

#### Hybrid Nodes
Each chunk is stored as both:
- A `(:Chunk)` node with text and embedding
- Connected to `(:Entity)` nodes via `[:MENTIONS]` relationships

This preserves source text for grounding while enabling abstract graph queries.

### Retrieval System (`src/graph_rag_service/retrieval/`)

#### Tools
1. **VectorSearchTool**: Semantic similarity using embeddings
2. **GraphTraversalTool**: Relationship exploration and path finding
3. **CypherGenerationTool**: Text-to-Cypher with validation
4. **MetadataFilterTool**: Structured queries on attributes

#### Agent Workflow (LangGraph)
```
[Query] → [Decompose] → [Route] → [Vector/Graph/Cypher] → [Synthesize] → [Response]
                           ↑                                      ↓
                           └─────────────────────────────────────┘
                                    (Iterative refinement)
```

#### Hallucination Guards
- **Schema Injection**: Prompt includes allowed entity/relationship types
- **Cypher Validation**: Parse and validate against whitelist
- **Self-Correction**: Feed errors back to LLM to fix syntax
- **Fallback**: If graph fails, degrade to vector search

### API Layer (`src/graph_rag_service/api/`)

#### Endpoints
- `POST /api/auth/login`: Get JWT token
- `POST /api/documents/upload`: Upload document (returns task ID)
- `GET /api/documents/status/{task_id}`: Check ingestion progress
- `POST /api/query`: Execute agentic query
- `GET /api/ontology`: Get current ontology schema
- `PUT /api/ontology`: Update ontology (admin only)
- `GET /api/graph/visualization`: Get graph data for visualization
- `GET /api/system/health`: System health check
- `GET /api/system/stats`: System statistics

#### Authentication
- JWT tokens with configurable expiration (default: 30 min)
- RBAC with scopes: `read`, `write`, `admin`
- Dependency injection for protected endpoints

### Workers (`src/graph_rag_service/workers/`)

#### Celery Tasks
- `ingest_document`: Process single document
- `ingest_documents_batch`: Process multiple documents
- `health_check`: Worker health verification

#### Configuration
- Broker: Redis
- Result Backend: Redis
- Serializer: JSON
- Task timeout: 1 hour (configurable)

### Observability (`src/graph_rag_service/observability/`)

#### OpenTelemetry Integration
- **Traces**: Agent reasoning steps, tool calls, database queries
- **Metrics**: 
  - `documents_ingested`: Counter
  - `queries_executed`: Counter
  - `query_duration_seconds`: Histogram
  - `entities_extracted`: Counter

#### Structured Logging
- Log level: INFO (configurable)
- Format: `%(asctime)s - %(name)s - %(levelname)s - %(message)s`
- All async operations logged with context

## Configuration

### Environment Variables
Key settings in `.env`:
- **Neo4j**: `NEO4J_URI`, `NEO4J_USER`, `NEO4J_PASSWORD`
- **Redis**: `REDIS_HOST`, `REDIS_PORT`
- **LLM Provider**: `DEFAULT_LLM_PROVIDER` (openai/anthropic/gemini/ollama)
- **API Keys**: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GOOGLE_API_KEY`
- **Ollama**: `OLLAMA_BASE_URL`, `OLLAMA_MODEL`, `OLLAMA_EMBEDDING_MODEL`
- **Security**: `SECRET_KEY`, `ACCESS_TOKEN_EXPIRE_MINUTES`

### Tuning Parameters
- `CHUNK_SIZE`: 1024 (text chunk size)
- `CHUNK_OVERLAP`: 200 (overlap between chunks)
- `MAX_AGENT_ITERATIONS`: 5 (max reasoning steps)
- `AGENT_TIMEOUT_SECONDS`: 30 (query timeout)
- `ENTITY_RESOLUTION_THRESHOLD`: 0.85 (similarity threshold)
- `DEFAULT_TOP_K`: 5 (retrieval results)
- `GRAPH_MAX_DEPTH`: 3 (graph traversal depth)

## Deployment

### Local Development
```bash
# 1. Ensure Neo4j and Redis are running
# 2. Configure .env with connection details

# 3. Start API server
./start-server.sh  # or start-server.bat on Windows

# 4. Start workers
./start-worker.sh  # or start-worker.bat on Windows
```

### Production Considerations
1. **Database**: Use managed Neo4j (Aura) or self-hosted cluster
2. **Redis**: Use managed Redis (AWS ElastiCache, Redis Cloud)
3. **Worker Scaling**: Add more Celery workers based on ingestion load
4. **API Scaling**: Run multiple API instances behind load balancer
5. **Monitoring**: Integrate with Prometheus/Grafana for metrics
6. **Secrets**: Use secret management (AWS Secrets Manager, HashiCorp Vault)

## Extensibility

### Adding New LLM Provider
1. Implement `LLMProvider` interface
2. Add to `LLMFactory.create()` method
3. Update config with new provider settings

### Adding New Graph Database
1. Implement `GraphStore` interface
2. Update `IngestionPipeline` to use new store
3. Test with existing workflows

### Custom Retrieval Tools
1. Create new tool class with `run()` method
2. Add to `AgentRetrievalSystem.tools`
3. Update routing logic in `_route_query()`

## Testing Strategy

### Unit Tests
- Test each component independently
- Mock external dependencies (Neo4j, Redis, LLMs)
- Focus on business logic

### Integration Tests
- Test component interactions
- Use test database instances
- Verify end-to-end flows

### Performance Tests
- Benchmark ingestion throughput
- Measure query latencies
- Stress test with concurrent requests

## Future Enhancements

### Phase 1 (Current MVP)
- ✅ Core ingestion pipeline
- ✅ Agentic retrieval system
- ✅ Multi-LLM support
- ✅ Entity resolution
- ✅ Async workers

### Phase 2 (Next Steps)
- [ ] React frontend with visual ontology editor
- [ ] Graph visualization (D3.js/Cytoscape)
- [ ] Advanced ontology evolution with migrations
- [ ] Semantic cache with Redis
- [ ] Batch ingestion optimization

### Phase 3 (Advanced Features)
- [ ] Multi-tenant support with data isolation
- [ ] Fine-tuned entity extraction models
- [ ] Graph neural network embeddings
- [ ] Automated ontology quality metrics
- [ ] Export/import ontology schemas

## Troubleshooting

### Common Issues

#### Neo4j Connection Failed
- Verify Neo4j is running and accessible
- Verify credentials in `.env`
- Try connecting with cypher-shell: `cypher-shell -u neo4j -p password`

#### Celery Worker Not Processing
- Check Redis is running: `redis-cli ping`
- Verify broker URL in `.env`
- Check worker logs

#### Ollama Models Not Found
- Pull models: `ollama pull llama3.2 && ollama pull bge-m3`
- Verify Ollama is running: `curl http://localhost:11434/api/tags`

#### Query Returns No Results
- Verify documents are ingested: `GET /api/system/stats`
- Check ontology exists: `GET /api/ontology`
- Try simpler queries first

## Support

For issues or questions:
1. Check documentation and troubleshooting guide
2. Search existing GitHub issues
3. Open new issue with:
   - Clear description
   - Steps to reproduce
   - Environment details
   - Relevant logs

---

**Last Updated**: February 2026
**Version**: 0.1.0