shreyask Claude Opus 4.6 commited on
Commit
0e526ea
·
verified ·
1 Parent(s): 5e17e07

wire App.tsx integration, update title, add HF Space README

Browse files

Connect all pipeline modules and UI components in App.tsx:
- Load models on mount, sample docs from public/eval-docs/
- Chunk and embed documents when embedding model is ready
- Run full pipeline (expansion, search, RRF, rerank, blend) on search
- Support file upload and paste for adding documents

Update index.html title and replace Vite boilerplate README with
HuggingFace Space metadata and project description.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Files changed (3) hide show
  1. README.md +25 -65
  2. index.html +1 -1
  3. src/App.tsx +160 -5
README.md CHANGED
@@ -1,73 +1,33 @@
1
- # React + TypeScript + Vite
 
 
 
 
 
 
 
 
2
 
3
- This template provides a minimal setup to get React working in Vite with HMR and some ESLint rules.
4
 
5
- Currently, two official plugins are available:
6
 
7
- - [@vitejs/plugin-react](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react) uses [Babel](https://babeljs.io/) (or [oxc](https://oxc.rs) when used in [rolldown-vite](https://vite.dev/guide/rolldown)) for Fast Refresh
8
- - [@vitejs/plugin-react-swc](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react-swc) uses [SWC](https://swc.rs/) for Fast Refresh
 
 
 
 
9
 
10
- ## React Compiler
11
 
12
- The React Compiler is not enabled on this template because of its impact on dev & build performances. To add it, see [this documentation](https://react.dev/learn/react-compiler/installation).
 
13
 
14
- ## Expanding the ESLint configuration
15
 
16
- If you are developing a production application, we recommend updating the configuration to enable type-aware lint rules:
 
 
17
 
18
- ```js
19
- export default defineConfig([
20
- globalIgnores(['dist']),
21
- {
22
- files: ['**/*.{ts,tsx}'],
23
- extends: [
24
- // Other configs...
25
-
26
- // Remove tseslint.configs.recommended and replace with this
27
- tseslint.configs.recommendedTypeChecked,
28
- // Alternatively, use this for stricter rules
29
- tseslint.configs.strictTypeChecked,
30
- // Optionally, add this for stylistic rules
31
- tseslint.configs.stylisticTypeChecked,
32
-
33
- // Other configs...
34
- ],
35
- languageOptions: {
36
- parserOptions: {
37
- project: ['./tsconfig.node.json', './tsconfig.app.json'],
38
- tsconfigRootDir: import.meta.dirname,
39
- },
40
- // other options...
41
- },
42
- },
43
- ])
44
- ```
45
-
46
- You can also install [eslint-plugin-react-x](https://github.com/Rel1cx/eslint-react/tree/main/packages/plugins/eslint-plugin-react-x) and [eslint-plugin-react-dom](https://github.com/Rel1cx/eslint-react/tree/main/packages/plugins/eslint-plugin-react-dom) for React-specific lint rules:
47
-
48
- ```js
49
- // eslint.config.js
50
- import reactX from 'eslint-plugin-react-x'
51
- import reactDom from 'eslint-plugin-react-dom'
52
-
53
- export default defineConfig([
54
- globalIgnores(['dist']),
55
- {
56
- files: ['**/*.{ts,tsx}'],
57
- extends: [
58
- // Other configs...
59
- // Enable lint rules for React
60
- reactX.configs['recommended-typescript'],
61
- // Enable lint rules for React DOM
62
- reactDom.configs.recommended,
63
- ],
64
- languageOptions: {
65
- parserOptions: {
66
- project: ['./tsconfig.node.json', './tsconfig.app.json'],
67
- tsconfigRootDir: import.meta.dirname,
68
- },
69
- // other options...
70
- },
71
- },
72
- ])
73
- ```
 
1
+ ---
2
+ title: QMD Web Demo
3
+ emoji: 🔍
4
+ colorFrom: blue
5
+ colorTo: green
6
+ sdk: static
7
+ pinned: false
8
+ license: mit
9
+ ---
10
 
11
+ # QMD Web Demo
12
 
13
+ In-browser hybrid search pipeline using WebGPU + Transformers.js v4.
14
 
15
+ Demonstrates the full QMD search pipeline running entirely in your browser:
16
+ 1. **Query Expansion** — Qwen3 1.7B generates HyDE, semantic, and keyword variants
17
+ 2. **Parallel Search** — BM25 keyword search + vector similarity search
18
+ 3. **Reciprocal Rank Fusion** — Merges results from multiple search backends
19
+ 4. **LLM Reranking** — Qwen3 Reranker 0.6B scores document relevance
20
+ 5. **Score Blending** — Position-aware combination of RRF and reranker scores
21
 
22
+ ## Requirements
23
 
24
+ - Chrome 113+ or Edge 113+ (WebGPU required)
25
+ - ~2.5GB model download on first visit (cached for subsequent visits)
26
 
27
+ ## Models
28
 
29
+ - [embeddinggemma-300M](https://huggingface.co/onnx-community/embeddinggemma-300m-ONNX) Embeddings
30
+ - [Qwen3-Reranker-0.6B](https://huggingface.co/onnx-community/Qwen3-Reranker-0.6B-ONNX) — Reranking
31
+ - [qmd-query-expansion-1.7B](https://huggingface.co/shreyask/qmd-query-expansion-1.7B-ONNX) — Query expansion
32
 
33
+ Based on [QMD](https://github.com/tobi/qmd) by Tobi Lütke.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
index.html CHANGED
@@ -4,7 +4,7 @@
4
  <meta charset="UTF-8" />
5
  <link rel="icon" type="image/svg+xml" href="/vite.svg" />
6
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
7
- <title>qmd-web</title>
8
  </head>
9
  <body>
10
  <div id="root"></div>
 
4
  <meta charset="UTF-8" />
5
  <link rel="icon" type="image/svg+xml" href="/vite.svg" />
6
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
7
+ <title>QMD Web Demo — In-Browser Hybrid Search</title>
8
  </head>
9
  <body>
10
  <div id="root"></div>
src/App.tsx CHANGED
@@ -1,10 +1,165 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  function App() {
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  return (
3
- <div style={{ fontFamily: 'system-ui, sans-serif', padding: '2rem' }}>
4
- <h1>QMD Web Demo</h1>
5
- <p>In-Browser Hybrid Search Pipeline</p>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6
  </div>
7
- )
8
  }
9
 
10
- export default App
 
1
+ import { useState, useEffect, useCallback } from 'react';
2
+ import type { Document, Chunk, EmbeddedChunk, ModelState } from './types';
3
+ import { loadAllModels, isAllModelsReady } from './pipeline/models';
4
+ import { chunkDocument, extractTitle } from './pipeline/chunking';
5
+ import { embedDocChunk } from './pipeline/embeddings';
6
+ import { BM25Index } from './pipeline/bm25';
7
+ import { runPipeline } from './pipeline/orchestrator';
8
+ import type { PipelineState } from './components/PipelineView';
9
+ import QueryInput from './components/QueryInput';
10
+ import ModelStatus from './components/ModelStatus';
11
+ import PipelineView from './components/PipelineView';
12
+ import DocumentManager from './components/DocumentManager';
13
+
14
+ // Sample doc filenames to load from public/eval-docs/
15
+ const SAMPLE_DOCS = [
16
+ 'api-design-principles.md',
17
+ 'distributed-systems-overview.md',
18
+ 'machine-learning-primer.md',
19
+ ];
20
+
21
+ const INITIAL_PIPELINE: PipelineState = {
22
+ expansion: { status: 'idle' },
23
+ search: { status: 'idle' },
24
+ rrf: { status: 'idle' },
25
+ rerank: { status: 'idle' },
26
+ blend: { status: 'idle' },
27
+ };
28
+
29
  function App() {
30
+ const [models, setModels] = useState<ModelState[]>([
31
+ { name: 'embedding', status: 'pending', progress: 0 },
32
+ { name: 'reranker', status: 'pending', progress: 0 },
33
+ { name: 'expansion', status: 'pending', progress: 0 },
34
+ ]);
35
+ const [documents, setDocuments] = useState<Document[]>([]);
36
+ const [chunks, setChunks] = useState<Chunk[]>([]);
37
+ const [embeddedChunks, setEmbeddedChunks] = useState<EmbeddedChunk[]>([]);
38
+ const [bm25Index, setBm25Index] = useState<BM25Index | null>(null);
39
+ const [pipeline, setPipeline] = useState<PipelineState>(INITIAL_PIPELINE);
40
+ const [indexing, setIndexing] = useState(false);
41
+ const [query, setQuery] = useState('');
42
+
43
+ // Load models on mount
44
+ useEffect(() => {
45
+ loadAllModels((state) => {
46
+ setModels(prev => prev.map(m => m.name === state.name ? state : m));
47
+ }).catch(console.error);
48
+ }, []);
49
+
50
+ // Load sample documents
51
+ useEffect(() => {
52
+ async function loadSampleDocs() {
53
+ const docs: Document[] = [];
54
+ for (const filename of SAMPLE_DOCS) {
55
+ const resp = await fetch(`/eval-docs/${filename}`);
56
+ const body = await resp.text();
57
+ const title = extractTitle(body, filename);
58
+ docs.push({ id: filename, title, body, filepath: filename });
59
+ }
60
+ setDocuments(docs);
61
+ }
62
+ loadSampleDocs();
63
+ }, []);
64
+
65
+ // When documents change, chunk them and build BM25 index
66
+ // When embedding model becomes ready, embed the chunks
67
+ useEffect(() => {
68
+ if (documents.length === 0) return;
69
+
70
+ const allChunks = documents.flatMap(doc => chunkDocument(doc));
71
+ setChunks(allChunks);
72
+ setBm25Index(new BM25Index(allChunks));
73
+
74
+ // Check if embedding model is ready for embedding
75
+ const embeddingReady = models.find(m => m.name === 'embedding')?.status === 'ready';
76
+ if (embeddingReady && allChunks.length > 0) {
77
+ setIndexing(true);
78
+ (async () => {
79
+ const embedded: EmbeddedChunk[] = [];
80
+ for (const chunk of allChunks) {
81
+ const embedding = await embedDocChunk(chunk.title, chunk.text);
82
+ embedded.push({ ...chunk, embedding });
83
+ }
84
+ setEmbeddedChunks(embedded);
85
+ setIndexing(false);
86
+ })();
87
+ }
88
+ }, [documents, models]);
89
+
90
+ // Handle user upload
91
+ const handleUpload = useCallback(async (files: FileList) => {
92
+ const newDocs: Document[] = [];
93
+ for (const file of Array.from(files)) {
94
+ const body = await file.text();
95
+ const title = extractTitle(body, file.name);
96
+ newDocs.push({ id: file.name, title, body, filepath: file.name });
97
+ }
98
+ setDocuments(prev => [...prev, ...newDocs]);
99
+ }, []);
100
+
101
+ // Handle paste
102
+ const handlePaste = useCallback((text: string, filename: string) => {
103
+ const title = extractTitle(text, filename);
104
+ setDocuments(prev => [...prev, { id: filename, title, body: text, filepath: filename }]);
105
+ }, []);
106
+
107
+ // Run search pipeline
108
+ const handleSearch = useCallback(async (searchQuery: string) => {
109
+ if (!bm25Index || embeddedChunks.length === 0) return;
110
+
111
+ setQuery(searchQuery);
112
+ setPipeline(INITIAL_PIPELINE);
113
+
114
+ const gen = runPipeline({
115
+ query: searchQuery,
116
+ chunks,
117
+ embeddedChunks,
118
+ bm25Index,
119
+ });
120
+
121
+ for await (const event of gen) {
122
+ setPipeline(prev => ({
123
+ ...prev,
124
+ [event.stage]: {
125
+ status: event.status,
126
+ ...('data' in event ? { data: event.data } : {}),
127
+ ...('error' in event ? { error: event.error } : {}),
128
+ },
129
+ }));
130
+ }
131
+ }, [bm25Index, embeddedChunks, chunks]);
132
+
133
+ const allReady = isAllModelsReady() && embeddedChunks.length > 0 && !indexing;
134
+
135
  return (
136
+ <div style={{ fontFamily: 'system-ui, -apple-system, sans-serif', maxWidth: 1400, margin: '0 auto', padding: '1rem' }}>
137
+ <header style={{ marginBottom: '1.5rem' }}>
138
+ <h1 style={{ margin: 0, fontSize: '1.5rem' }}>QMD Web Demo</h1>
139
+ <p style={{ margin: '0.25rem 0 0', color: '#666', fontSize: '0.9rem' }}>
140
+ In-Browser Hybrid Search Pipeline — WebGPU + Transformers.js
141
+ </p>
142
+ </header>
143
+
144
+ <ModelStatus models={models} />
145
+
146
+ {indexing && (
147
+ <div style={{ padding: '0.5rem 1rem', background: '#FFF3E0', borderRadius: 6, marginBottom: '1rem', fontSize: '0.85rem' }}>
148
+ Indexing documents (embedding chunks)...
149
+ </div>
150
+ )}
151
+
152
+ <QueryInput onSearch={handleSearch} disabled={!allReady} />
153
+
154
+ {query && <PipelineView state={pipeline} query={query} />}
155
+
156
+ <DocumentManager
157
+ documents={documents.map(d => ({ id: d.id, title: d.title, filepath: d.filepath }))}
158
+ onUpload={handleUpload}
159
+ onPaste={handlePaste}
160
+ />
161
  </div>
162
+ );
163
  }
164
 
165
+ export default App;