data/knowledge_base/README.md

RAG Knowledge Base

Drop reference documents here (.md, .txt, or .pdf). They are ingested by python -m src.rag.ingest at Docker build time and surfaced to the orchestrator agent via the retrieve_context tool. The container entrypoint also rebuilds the index at startup when a mounted data/ volume does not already contain data/processed/faiss_index/.

Recommended seed set

For a clinical-ML / NeuroBridge demo:

• BBB / molecules: Lipinski's Rule of Five (1997, 2001), Pajouhesh & Lenz CNS multiparameter optimization (2005)
• MRI / harmonization: Fortin et al. ComBat for cortical thickness (2017), Fortin et al. ComBat for diffusion (2018), Johnson et al. original ComBat (2007, gene expression)
• EEG / artifacts: Hyvärinen ICA primer (1999), MNE-Python overview (Gramfort 2013)

Format notes

• PDFs work via pypdf. OCR-only PDFs (scanned images) won't extract text; pre-OCR them first.
• Markdown is preferred — full text + headers chunk cleanly.
• Files are gitignored by default. Mount them via Docker volume in production, or COPY them in via a sub-path before the RUN ingest line.

Re-indexing

After adding/removing files, re-run:

python -m src.rag.ingest

This rewrites data/processed/faiss_index/ from scratch (no incremental update — the index is small enough to rebuild in seconds).