# PaperHawk Architecture How PaperHawk is built and why each piece is where it is. This document explains the multi-graph LangGraph orchestration, the 14 deterministic domain checks, the 6-layer anti-hallucination stack, and the multi-agent DD assistant. --- ## High-level architecture ``` ┌──────────────────────────────────────────────────────────────────────────┐ │ USER (Streamlit 5-tab UI) │ │ Upload │ Results │ Chat │ DD Assistant │ Report │ └────────────────────────────────┬─────────────────────────────────────────┘ │ ┌────────────────────┼────────────────────────┐ │ │ │ ▼ ▼ ▼ ┌──────────────────┐ ┌──────────────────┐ ┌─────────────────────────┐ │ pipeline_graph │ │ chat_graph │ │ dd_graph │ │ │ │ │ │ │ │ Ingest → │ │ Intent classify │ │ Contract filter → │ │ Classify → │ │ → Plan → │ │ Per-contract summary → │ │ Extract → │ │ Agent (5 tools) │ │ Multi-agent specialists │ │ Compare → │ │ → Synthesizer → │ │ (audit/legal/compliance │ │ Risk → │ │ Validator │ │ /financial) → │ │ Report │ │ ([Source: …]) │ │ Supervisor → Synthesizer│ └──────────────────┘ └──────────────────┘ └─────────────────────────┘ │ │ └─────────────┬──────────────────────────┘ ▼ ┌──────────────────────────┐ │ package_insights_graph │ │ │ │ Cross-document analysis │ │ (price-drift, dupes, │ │ three-way matching) │ └──────────────────────────┘ │ ▼ ┌──────────────────────────┐ │ Provider abstraction │ │ (configurable_alternatives) │ │ │ vLLM ←→ Ollama ←→ Dummy │ └──────────────────────────┘ │ ▼ ┌──────────────────────────┐ │ AMD MI300X (vLLM) │ │ Qwen 2.5 14B Instruct │ │ 192 GB HBM3, ROCm 7.0 │ └──────────────────────────┘ ``` --- ## Compiled graphs (4) Every entry-point in the system is a separately compiled LangGraph artifact with its own typed state and `AsyncSqliteSaver` checkpointer: ### 1. `pipeline_graph` — the document processing pipeline The 6-step end-to-end flow when the user uploads a package: 1. **Ingest** — PDF (PyMuPDF + pdfplumber for table extraction), DOCX (native), images (vision-first via the LLM), with Tesseract OCR fallback for scanned PDFs (EN/HU/DE) 2. **Classify** — 6-way doc-type classifier with structured output (`invoice`, `delivery_note`, `purchase_order`, `contract`, `financial_report`, `other`); ISA 500 evidence-quality score 3. **Extract** — per doc-type Pydantic v2 schema with `_quotes` and `_confidence` fields; universal fallback schema for unknown types 4. **Compare** — three-way matching subgraph (invoice + delivery note + PO), duplicate-invoice detection (ISA 240) 5. **Risk** — basic plausibility + 14 domain checks (Send-API parallel fan-out) + LLM risk ensemble + 3-stage filter chain 6. **Report** — DOCX export, JSON output, Streamlit UI rendering State: `PipelineState` (Pydantic), with reducers for risk lists and per-document results. ### 2. `chat_graph` — the agentic chat 5-tool ReAct agent with strict citation enforcement: - **Tools**: `list_documents`, `get_extraction`, `search_documents` (hybrid Chroma + BM25 with Reciprocal Rank Fusion), `compare_documents`, `validate_document` - **Prompt**: 17-rule system prompt enforcing `[Source: filename.pdf]` format - **Validator node**: post-processor that drops any answer without citations - **Intent classifier**: routes to direct-answer vs tool-use paths to keep latency low for casual queries State: `ChatState` with message history, retrieved chunks, and citation list. ### 3. `dd_graph` — the multi-agent DD assistant For M&A due-diligence packages: - **Contract filter** — selects only contract-type documents from the package - **Per-contract summary** — extracts each contract's key terms (parties, term, value, change-of-control, non-compete, auto-renewal) - **4 specialist agents** (run in parallel via Send-API): - `audit_specialist` — material misstatement risk, ISA 240 fraud indicators - `legal_specialist` — change-of-control, non-compete, automatic-renewal red flags - `compliance_specialist` — GDPR Art. 28 sub-processor language, AML counterparty checks - `financial_specialist` — Ptk. 6:98 disproportionate penalty clauses, materiality thresholds - **Supervisor** — coordinates specialists, drops business-normal noise - **Synthesizer** — writes 3-paragraph executive summary State: `DDState` with contract list, per-contract summaries, specialist findings, executive summary. ### 4. `package_insights_graph` — cross-document analysis Package-level analyzers that don't fit into the per-document pipeline: - **Pricing-drift detector** — flags > 30% price changes for the same line item across invoices in a package (caught the 57.5% drift in our live demo) - **Duplicate-invoice detector** — exact + near-match (date within 13 days, amount within 1%) - **Counterparty consistency** — same supplier name spelled differently across documents State: `PackageState` with per-document extractions and aggregated findings. --- ## Subgraphs (6) Reusable LangGraph subgraphs imported by the main graphs: | Subgraph | Purpose | |---|---| | `extract_subgraph` | Per-document extraction with quote validator | | `ingest_subgraph` | PDF/DOCX/image loading with OCR fallback | | `llm_risk_subgraph` | LLM risk generation with structured output | | `rag_index_subgraph` | Chunking, embedding, ChromaDB indexing | | `rag_query_subgraph` | Hybrid Chroma + BM25 retrieval with RRF | | `risk_subgraph` | Domain check fan-out + LLM risk + 3-stage filters | --- ## 14 deterministic domain checks The check registry (`domain_checks/__init__.py`) is the heart of PaperHawk's auditor-grade output. Every check is a Python `Protocol` implementation, not an LLM prompt — they cannot hallucinate, can be unit-tested, and produce defensible findings with explicit regulation sources. ### A-tier (essential) 1. **Mandatory invoice elements** (HU VAT Act §169) — 18 required elements per invoice 2. **Tax-ID checksum** (Art. 22 §) — mod-11 Hungarian tax-ID validation 3. **Contract completeness** (Ptk. Book 6) — termination, governing law, penalty, confidentiality clauses 4. **Disproportionality** (Ptk. 6:98) — penalty clause > 31.7% of contract value flagged HIGH 5. **Rounded amounts** (ISA 240) — > 14.7% rounded amounts flagged suspicious, > 24.3% flagged HIGH 6. **Evidence hierarchy** (ISA 500) — document-type reliability score (8/10 invoice, 7/10 contract) ### B-tier (supplementary) 7. **Materiality** (ISA 320) — 1.93% of document value as info-level threshold 8. **GDPR Article 28** — 10 mandatory sub-processor language elements + PII detection 9. **DD red flags** (M&A) — change-of-control, non-compete, automatic-renewal triggers ### C-tier (informational) 10. **Incoterms 2020** — 11 incoterm rules detected via regex word-boundaries 11. **IFRS/HAR anomaly** — goodwill amortization flag, operational lease in IFRS context 12. **Duplicate invoice** (ISA 240) — exact + near-match with 13-day date filter 13. **AML sanctions** (Pmt.) — static EU/OFAC snapshot with fuzzy name match 14. **Contract dates** — start-end consistency, expiry detection **Jurisdiction-aware**: Hungarian-specific rules (HU VAT Act, Ptk., Art.) apply only to Hungarian documents. Universal rules (ISA, GDPR, Incoterms, AML) apply everywhere. --- ## 6-layer anti-hallucination stack The system is designed so the LLM **cannot** lie about a document and have the lie pass through. | Layer | What it does | |---|---| | 1. `temperature=0` | Deterministic outputs every run | | 2. Source quote requirement | Every extraction must include a verbatim quote from the source PDF in `_quotes` | | 3. Confidence scoring | high / medium / low per extracted field, surfaced to the user | | 4. Plausibility validators | Deterministic Python checks for math, dates, totals, item-level VAT, currency normalization | | 5. 3-stage LLM-risk filter chain | Drops business-normal noise, drops repeats of basic deterministic checks, drops contradictions | | 6. Quote validator | Text-search the source PDF for the claimed quote; downgrade confidence if not found verbatim, drop entirely if obviously fabricated | In our live audit demo, layer 6 caught **4 of 6** hallucinated citations from Qwen 2.5 14B and downgraded them to `low` confidence. The `validation/` package is one of the most-edited folders in the repo precisely because we treat anti-hallucination as a first-class concern, not a guardrail layer slapped on top. --- ## Provider abstraction `configurable_alternatives` lets us swap LLM backends with a single env var: | `LLM_PROFILE` | Backend | Use case | |---|---|---| | `vllm` | vLLM REST endpoint (OpenAI-compatible) | Production on AMD MI300X | | `ollama` | Local Ollama at `localhost:11434` | Dev on consumer GPU | | `dummy` | Deterministic stub | CI tests, smoke tests, judge quick-demo | The application code never imports an LLM SDK directly — all calls go through `providers/` factory functions with `configurable_alternatives`. Switching from Anthropic Claude (our original dev target) to Qwen on vLLM required **zero application code changes** — only env vars. --- ## Embedding + retrieval - **Model**: BAAI/bge-m3 (1024-dim, multilingual EN/HU/DE/FR via sentence-transformers) - **Storage**: ChromaDB persistent (per-session) + BM25 in-memory keyword index - **Hybrid retrieval**: Reciprocal Rank Fusion of Chroma top-K and BM25 top-K - **Chunking**: Natural-boundary chunking (paragraph-aware, ~500 tokens with overlap) The embedding model loads once at app startup (~2.3 GB to RAM/VRAM). On first run it downloads from Hugging Face Hub to `~/.cache/huggingface/`. --- ## State persistence - **Per-session**: Streamlit `session_state` for UI state (uploaded files, current package) - **Per-graph**: `AsyncSqliteSaver` checkpointer at `data/checkpoints.sqlite` for LangGraph state - **Vector store**: ChromaDB at `chroma_db/` (gitignored) Restarting the app loads the last checkpoint, so chat history and extraction results survive a restart. --- ## Streamlit UI (5 tabs) 1. **Upload** — drag-and-drop (PDF, DOCX, PNG, JPG, TXT), 200 MB per file, plus 3 pre-bundled demo packages 2. **Results** — classification confidence, extracted data, risks per document, package-level cross-doc analysis 3. **Chat** — agentic chat with `[Source: filename.pdf]` citations 4. **DD Assistant** — for M&A packages: per-contract summaries + 4 specialist findings + executive summary + downloadable DOCX 5. **Report** — JSON output + DOCX export The async runtime uses a long-lived background event loop (`app/async_runtime.py`) so the UI stays responsive during multi-minute pipeline runs. --- ## Cross-references - [`docs/AMD_DEPLOYMENT.md`](AMD_DEPLOYMENT.md) — how the production vLLM endpoint runs on AMD MI300X - [`docs/HUGGINGFACE_DEPLOYMENT.md`](HUGGINGFACE_DEPLOYMENT.md) — how the Streamlit app deploys as a public HF Space - [`docs/SUBMISSION.md`](SUBMISSION.md) — full hackathon submission brief with TAM/SAM, competitor positioning, live deployment validation