Merge remote README
Browse files- .gitattributes +35 -0
- README.md +7 -261
.gitattributes
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
*.7z filter=lfs diff=lfs merge=lfs -text
|
| 2 |
+
*.arrow filter=lfs diff=lfs merge=lfs -text
|
| 3 |
+
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 4 |
+
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
| 5 |
+
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
| 6 |
+
*.ftz filter=lfs diff=lfs merge=lfs -text
|
| 7 |
+
*.gz filter=lfs diff=lfs merge=lfs -text
|
| 8 |
+
*.h5 filter=lfs diff=lfs merge=lfs -text
|
| 9 |
+
*.joblib filter=lfs diff=lfs merge=lfs -text
|
| 10 |
+
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
| 11 |
+
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
| 12 |
+
*.model filter=lfs diff=lfs merge=lfs -text
|
| 13 |
+
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
| 14 |
+
*.npy filter=lfs diff=lfs merge=lfs -text
|
| 15 |
+
*.npz filter=lfs diff=lfs merge=lfs -text
|
| 16 |
+
*.onnx filter=lfs diff=lfs merge=lfs -text
|
| 17 |
+
*.ot filter=lfs diff=lfs merge=lfs -text
|
| 18 |
+
*.parquet filter=lfs diff=lfs merge=lfs -text
|
| 19 |
+
*.pb filter=lfs diff=lfs merge=lfs -text
|
| 20 |
+
*.pickle filter=lfs diff=lfs merge=lfs -text
|
| 21 |
+
*.pkl filter=lfs diff=lfs merge=lfs -text
|
| 22 |
+
*.pt filter=lfs diff=lfs merge=lfs -text
|
| 23 |
+
*.pth filter=lfs diff=lfs merge=lfs -text
|
| 24 |
+
*.rar filter=lfs diff=lfs merge=lfs -text
|
| 25 |
+
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 26 |
+
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
| 27 |
+
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
| 28 |
+
*.tar filter=lfs diff=lfs merge=lfs -text
|
| 29 |
+
*.tflite filter=lfs diff=lfs merge=lfs -text
|
| 30 |
+
*.tgz filter=lfs diff=lfs merge=lfs -text
|
| 31 |
+
*.wasm filter=lfs diff=lfs merge=lfs -text
|
| 32 |
+
*.xz filter=lfs diff=lfs merge=lfs -text
|
| 33 |
+
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
+
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
+
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -1,264 +1,10 @@
|
|
| 1 |
---
|
| 2 |
-
title:
|
| 3 |
-
emoji:
|
| 4 |
-
|
|
|
|
|
|
|
|
|
|
| 5 |
---
|
| 6 |
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
A production-ready **Retrieval-Augmented Generation (RAG)** system with multi-domain support, advanced AI features, and intelligent web search integration. Built with FastAPI, React, and Google Gemini API.
|
| 10 |
-
|
| 11 |
-
## ✨ Features
|
| 12 |
-
|
| 13 |
-
### 🎯 Multi-Domain Support
|
| 14 |
-
- **Medical & Healthcare**: Medical documents, research papers, clinical guidelines
|
| 15 |
-
- **Legal & Compliance**: Legal documents, contracts, regulations, case law
|
| 16 |
-
- **Financial & Analytics**: Financial reports, analysis, market research
|
| 17 |
-
- **Technical Documentation**: Technical docs, APIs, code, architecture
|
| 18 |
-
- **Academic Research**: Research papers, academic publications, studies
|
| 19 |
-
|
| 20 |
-
### 🚀 Advanced AI Capabilities
|
| 21 |
-
- **Query Improvement**: Automatic query enhancement with abbreviation expansion
|
| 22 |
-
- **Dual-LLM Verification**: Two-stage answer verification using Gemini Pro
|
| 23 |
-
- **Web Search Integration**: Augment answers with real-time web search via Tavily
|
| 24 |
-
- **Conversation Memory**: Context-aware responses with conversation history
|
| 25 |
-
- **Multimodal Processing**: Support for images, tables, and equations (MinerU parser)
|
| 26 |
-
- **Smart Reranking**: Gemini-powered relevance reranking for better results
|
| 27 |
-
- **Streaming Responses**: Real-time token streaming for responsive UX
|
| 28 |
-
|
| 29 |
-
### 🔧 Technical Features
|
| 30 |
-
- **Gemini API Integration**: Free-tier Gemini Flash & Pro models
|
| 31 |
-
- **Async Processing**: Background document processing with status tracking
|
| 32 |
-
- **RESTful API**: Clean, well-documented FastAPI endpoints
|
| 33 |
-
- **Modern React Frontend**: Beautiful, responsive UI with Tailwind CSS
|
| 34 |
-
- **Docker Support**: One-command deployment with docker-compose
|
| 35 |
-
- **Performance Optimized**: Query caching, fast mode (2-3x speedup), batch processing
|
| 36 |
-
|
| 37 |
-
## 🚀 Quick Start (Docker)
|
| 38 |
-
|
| 39 |
-
### Prerequisites
|
| 40 |
-
- Docker and Docker Compose
|
| 41 |
-
- Google Gemini API Key ([Get one free](https://makersuite.google.com/app/apikey))
|
| 42 |
-
- (Optional) Tavily API Key for web search ([Get one free](https://tavily.com))
|
| 43 |
-
|
| 44 |
-
### 1. Clone the Repository
|
| 45 |
-
```bash
|
| 46 |
-
git clone <your-repo-url>
|
| 47 |
-
cd Agentic_RAG
|
| 48 |
-
```
|
| 49 |
-
|
| 50 |
-
### 2. Set Up Environment Variables
|
| 51 |
-
Create a `.env` file in the project root:
|
| 52 |
-
```bash
|
| 53 |
-
GEMINI_API_KEY=your_gemini_api_key_here
|
| 54 |
-
GEMINI_TEXT_MODEL=models/gemini-flash-latest
|
| 55 |
-
GEMINI_VERIFIER_MODEL=models/gemini-pro-latest
|
| 56 |
-
GEMINI_VISION_MODEL=models/gemini-flash-latest
|
| 57 |
-
GEMINI_EMBEDDING_MODEL=models/text-embedding-004
|
| 58 |
-
TAVILY_API_KEY=your_tavily_api_key_here # Optional, for web search
|
| 59 |
-
```
|
| 60 |
-
|
| 61 |
-
### 3. Start the Application
|
| 62 |
-
```bash
|
| 63 |
-
docker-compose up -d
|
| 64 |
-
```
|
| 65 |
-
|
| 66 |
-
### 4. Access the Application
|
| 67 |
-
- **Frontend**: http://localhost:3000
|
| 68 |
-
- **Backend API**: http://localhost:8000
|
| 69 |
-
- **API Docs**: http://localhost:8000/docs
|
| 70 |
-
|
| 71 |
-
## 📖 Usage
|
| 72 |
-
|
| 73 |
-
### Upload Documents
|
| 74 |
-
1. Navigate to the frontend at http://localhost:3000
|
| 75 |
-
2. Select a domain (Medical, Legal, Financial, Technical, or Academic)
|
| 76 |
-
3. Upload PDF, DOCX, TXT, or other supported documents
|
| 77 |
-
4. Wait for processing to complete (tracked with real-time status)
|
| 78 |
-
|
| 79 |
-
### Query Documents
|
| 80 |
-
1. Enter your question in the query interface
|
| 81 |
-
2. Select query mode:
|
| 82 |
-
- **Mix**: Balanced combination of local and global search (recommended)
|
| 83 |
-
- **Local**: Focused chunk-based search
|
| 84 |
-
- **Global**: Knowledge graph entity search
|
| 85 |
-
- **Hybrid**: Advanced combination
|
| 86 |
-
- **Web**: RAG + real-time web search
|
| 87 |
-
3. Toggle advanced features:
|
| 88 |
-
- **Query Improvement**: Enhance your query automatically
|
| 89 |
-
- **Verification**: Dual-LLM quality check
|
| 90 |
-
- **Web Search**: Augment with real-time web results
|
| 91 |
-
- **Fast Mode**: 2-3x faster queries (slightly lower quality)
|
| 92 |
-
4. Get streaming responses with sources and confidence scores
|
| 93 |
-
|
| 94 |
-
### API Usage
|
| 95 |
-
```python
|
| 96 |
-
import requests
|
| 97 |
-
|
| 98 |
-
# Upload document
|
| 99 |
-
files = {"file": open("document.pdf", "rb")}
|
| 100 |
-
data = {"domain": "medical"}
|
| 101 |
-
response = requests.post("http://localhost:8000/upload", files=files, data=data)
|
| 102 |
-
print(response.json())
|
| 103 |
-
|
| 104 |
-
# Query documents
|
| 105 |
-
query_data = {
|
| 106 |
-
"query": "What are the treatment options for hypertension?",
|
| 107 |
-
"domain": "medical",
|
| 108 |
-
"mode": "mix",
|
| 109 |
-
"enable_web_search": False,
|
| 110 |
-
"fast_mode": False,
|
| 111 |
-
"return_metadata": True
|
| 112 |
-
}
|
| 113 |
-
response = requests.post("http://localhost:8000/query", json=query_data)
|
| 114 |
-
print(response.json())
|
| 115 |
-
```
|
| 116 |
-
|
| 117 |
-
## 🏗️ Architecture
|
| 118 |
-
|
| 119 |
-
```
|
| 120 |
-
Agentic_RAG/
|
| 121 |
-
├── backend/ # FastAPI backend
|
| 122 |
-
│ ├── main.py # Main API server
|
| 123 |
-
│ ├── reranker.py # Gemini-powered reranking
|
| 124 |
-
│ ├── web_search.py # Tavily web search integration
|
| 125 |
-
│ ├── url_fetcher.py # URL content fetching
|
| 126 |
-
│ ├── requirements.txt # Python dependencies
|
| 127 |
-
│ └── Dockerfile # Backend container
|
| 128 |
-
├── frontend/ # React frontend
|
| 129 |
-
│ ├── src/ # React components
|
| 130 |
-
│ ├── public/ # Static assets
|
| 131 |
-
│ ├── package.json # Node dependencies
|
| 132 |
-
│ ├── Dockerfile # Frontend container
|
| 133 |
-
│ └── nginx.conf # Nginx configuration
|
| 134 |
-
├── storage/ # RAG storage (created at runtime)
|
| 135 |
-
│ ├── medical/ # Medical domain storage
|
| 136 |
-
│ ├── legal/ # Legal domain storage
|
| 137 |
-
│ └── ... # Other domains
|
| 138 |
-
├── uploads/ # Uploaded documents
|
| 139 |
-
├── docker-compose.yml # Docker orchestration
|
| 140 |
-
├── Dockerfile # Hugging Face Space Dockerfile
|
| 141 |
-
└── README.md # This file
|
| 142 |
-
```
|
| 143 |
-
|
| 144 |
-
## 🔑 Environment Variables
|
| 145 |
-
|
| 146 |
-
| Variable | Description | Required | Default |
|
| 147 |
-
|----------|-------------|----------|---------|
|
| 148 |
-
| `GEMINI_API_KEY` | Google Gemini API key | Yes | - |
|
| 149 |
-
| `GEMINI_TEXT_MODEL` | Text generation model | No | `models/gemini-flash-latest` |
|
| 150 |
-
| `GEMINI_VERIFIER_MODEL` | Verification model | No | `models/gemini-pro-latest` |
|
| 151 |
-
| `GEMINI_VISION_MODEL` | Vision processing model | No | `models/gemini-flash-latest` |
|
| 152 |
-
| `GEMINI_EMBEDDING_MODEL` | Embedding model | No | `models/text-embedding-004` |
|
| 153 |
-
| `TAVILY_API_KEY` | Tavily web search API key | No | - |
|
| 154 |
-
|
| 155 |
-
## 📊 API Endpoints
|
| 156 |
-
|
| 157 |
-
### Health Check
|
| 158 |
-
```bash
|
| 159 |
-
GET /health
|
| 160 |
-
```
|
| 161 |
-
|
| 162 |
-
### List Domains
|
| 163 |
-
```bash
|
| 164 |
-
GET /domains
|
| 165 |
-
```
|
| 166 |
-
|
| 167 |
-
### Upload Document
|
| 168 |
-
```bash
|
| 169 |
-
POST /upload
|
| 170 |
-
Content-Type: multipart/form-data
|
| 171 |
-
|
| 172 |
-
file: <document file>
|
| 173 |
-
domain: medical
|
| 174 |
-
```
|
| 175 |
-
|
| 176 |
-
### Query Documents (Streaming)
|
| 177 |
-
```bash
|
| 178 |
-
POST /query/stream
|
| 179 |
-
Content-Type: application/json
|
| 180 |
-
|
| 181 |
-
{
|
| 182 |
-
"query": "What are the treatment options?",
|
| 183 |
-
"domain": "medical",
|
| 184 |
-
"mode": "mix",
|
| 185 |
-
"enable_web_search": false,
|
| 186 |
-
"fast_mode": false
|
| 187 |
-
}
|
| 188 |
-
```
|
| 189 |
-
|
| 190 |
-
### Query Documents (Standard)
|
| 191 |
-
```bash
|
| 192 |
-
POST /query
|
| 193 |
-
Content-Type: application/json
|
| 194 |
-
|
| 195 |
-
{
|
| 196 |
-
"query": "What are the treatment options?",
|
| 197 |
-
"domain": "medical",
|
| 198 |
-
"mode": "mix"
|
| 199 |
-
}
|
| 200 |
-
```
|
| 201 |
-
|
| 202 |
-
### Check Processing Status
|
| 203 |
-
```bash
|
| 204 |
-
GET /status/{processing_id}
|
| 205 |
-
```
|
| 206 |
-
|
| 207 |
-
### List Documents
|
| 208 |
-
```bash
|
| 209 |
-
GET /documents?domain=medical
|
| 210 |
-
```
|
| 211 |
-
|
| 212 |
-
### Delete Document
|
| 213 |
-
```bash
|
| 214 |
-
DELETE /documents/{doc_id}
|
| 215 |
-
```
|
| 216 |
-
|
| 217 |
-
## 🎯 Performance
|
| 218 |
-
|
| 219 |
-
- **Fast Mode**: 2-3x faster queries with optimized parameters
|
| 220 |
-
- **Query Caching**: 5-minute TTL cache for repeated queries
|
| 221 |
-
- **Batch Processing**: Parallel document processing (up to 10 documents)
|
| 222 |
-
- **Streaming**: Real-time token streaming for responsive UX
|
| 223 |
-
- **Reranking**: Gemini-powered relevance scoring
|
| 224 |
-
|
| 225 |
-
## 🛠️ Development
|
| 226 |
-
|
| 227 |
-
### Backend Development
|
| 228 |
-
```bash
|
| 229 |
-
cd backend
|
| 230 |
-
pip install -r requirements.txt
|
| 231 |
-
python main.py
|
| 232 |
-
```
|
| 233 |
-
|
| 234 |
-
### Frontend Development
|
| 235 |
-
```bash
|
| 236 |
-
cd frontend
|
| 237 |
-
npm install
|
| 238 |
-
npm start
|
| 239 |
-
```
|
| 240 |
-
|
| 241 |
-
## 🤝 Contributing
|
| 242 |
-
|
| 243 |
-
Contributions are welcome! Please feel free to submit a Pull Request.
|
| 244 |
-
|
| 245 |
-
## 📝 License
|
| 246 |
-
|
| 247 |
-
This project is licensed under the MIT License - see the LICENSE file for details.
|
| 248 |
-
|
| 249 |
-
## 🙏 Acknowledgments
|
| 250 |
-
|
| 251 |
-
- [LightRAG](https://github.com/HKUDS/LightRAG) - RAG framework
|
| 252 |
-
- [Google Gemini](https://ai.google.dev/) - LLM and embeddings
|
| 253 |
-
- [Tavily](https://tavily.com/) - Web search API
|
| 254 |
-
- [MinerU](https://github.com/opendatalab/MinerU) - Document parsing
|
| 255 |
-
- [FastAPI](https://fastapi.tiangolo.com/) - Backend framework
|
| 256 |
-
- [React](https://react.dev/) - Frontend framework
|
| 257 |
-
|
| 258 |
-
## 📧 Support
|
| 259 |
-
|
| 260 |
-
For issues and questions, please open an issue on GitHub.
|
| 261 |
-
|
| 262 |
-
---
|
| 263 |
-
|
| 264 |
-
Built with ❤️ using FastAPI, React, and Google Gemini
|
|
|
|
| 1 |
---
|
| 2 |
+
title: OrgAI
|
| 3 |
+
emoji: 🏃
|
| 4 |
+
colorFrom: yellow
|
| 5 |
+
colorTo: green
|
| 6 |
+
sdk: docker
|
| 7 |
+
pinned: false
|
| 8 |
---
|
| 9 |
|
| 10 |
+
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|