Step 11: precompute script and fallback JSON files

Implements specs/12_precompute.md. precompute_results.py runs full pipeline
when API key is available. Four JSON files committed as fallback: criteria.json
(5 criteria), eval_bidder_a.json (all eligible), eval_bidder_b.json (C1
not_eligible), eval_bidder_c.json (C1 needs_review from low-quality scan).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

data/precomputed/criteria.json

@@ -0,0 +1,117 @@
+{
+  "criteria": [
+    {
+      "id": "C1",
+      "title": "Minimum Annual Turnover",
+      "category": "financial",
+      "mandatory": true,
+      "description": "The bidder shall have a minimum average annual turnover of INR 5 Crore during the last three financial years (2022-23, 2023-24, 2024-25).",
+      "rule": {
+        "type": "numeric_threshold",
+        "field": "annual_turnover_inr",
+        "operator": ">=",
+        "value": 50000000,
+        "unit": "INR"
+      },
+      "query_hints": [
+        "annual turnover",
+        "total revenue",
+        "INR crore",
+        "audited financials",
+        "CA certificate"
+      ],
+      "source_page": 2,
+      "source_clause": "3.2(a)"
+    },
+    {
+      "id": "C2",
+      "title": "Completed Construction Projects",
+      "category": "technical",
+      "mandatory": true,
+      "description": "The bidder must have successfully completed at least three (3) similar construction projects of value not less than INR 1 Crore each in the last five financial years.",
+      "rule": {
+        "type": "count_threshold",
+        "field": "completed_projects",
+        "operator": ">=",
+        "value": 3,
+        "unit": null
+      },
+      "query_hints": [
+        "completed projects",
+        "construction experience",
+        "work order",
+        "completion certificate",
+        "similar projects"
+      ],
+      "source_page": 2,
+      "source_clause": "3.2(b)"
+    },
+    {
+      "id": "C3",
+      "title": "GST Registration",
+      "category": "compliance",
+      "mandatory": true,
+      "description": "The bidder shall possess a valid Goods and Services Tax (GST) registration certificate. The GSTIN must be active as on the date of submission.",
+      "rule": {
+        "type": "certification_present",
+        "field": "gstin",
+        "operator": "exists",
+        "value": null,
+        "unit": null
+      },
+      "query_hints": [
+        "GSTIN",
+        "GST certificate",
+        "GST registration",
+        "tax registration"
+      ],
+      "source_page": 2,
+      "source_clause": "3.2(c)"
+    },
+    {
+      "id": "C4",
+      "title": "ISO 9001:2015 Certification",
+      "category": "compliance",
+      "mandatory": true,
+      "description": "The bidder shall hold a valid ISO 9001:2015 Quality Management System certification issued by an accredited certification body.",
+      "rule": {
+        "type": "certification_present",
+        "field": "iso_9001",
+        "operator": "exists",
+        "value": null,
+        "unit": null
+      },
+      "query_hints": [
+        "ISO 9001",
+        "quality management",
+        "ISO certificate",
+        "QMS certification"
+      ],
+      "source_page": 2,
+      "source_clause": "3.2(d)"
+    },
+    {
+      "id": "C5",
+      "title": "Paramilitary Infrastructure Experience",
+      "category": "technical",
+      "mandatory": false,
+      "description": "Preferably, the bidder may have prior experience with construction or maintenance of paramilitary or defence infrastructure.",
+      "rule": {
+        "type": "document_present",
+        "field": "paramilitary_experience",
+        "operator": "exists",
+        "value": null,
+        "unit": null
+      },
+      "query_hints": [
+        "paramilitary",
+        "defence infrastructure",
+        "CRPF",
+        "BSF",
+        "security forces"
+      ],
+      "source_page": 2,
+      "source_clause": "3.2(e)"
+    }
+  ]
+}

data/precomputed/eval_bidder_a.json

@@ -0,0 +1,107 @@
+[
+  {
+    "verdict_id": "V-9e49234c",
+    "bidder_id": "bidder_a",
+    "criterion_id": "C1",
+    "verdict": "eligible",
+    "extracted_value": "INR 6.2 Cr (avg: 6.37 Cr)",
+    "normalized_value": 63666667,
+    "source": {
+      "doc_name": "audited_financials.pdf",
+      "page": 1,
+      "snippet": "annual turnover for FY 2023-24 was INR 6,20,00,000",
+      "source_type": "text_pdf"
+    },
+    "llm_confidence": 0.93,
+    "ocr_confidence": null,
+    "combined_confidence": 0.93,
+    "reason": "Average annual turnover of INR 6.37 Cr exceeds the required threshold of INR 5 Cr.",
+    "model_version": "deepseek-chat@2026-05-07",
+    "timestamp": "2026-05-06T19:17:36.918702+00:00",
+    "review_status": "pending"
+  },
+  {
+    "verdict_id": "V-9f84114a",
+    "bidder_id": "bidder_a",
+    "criterion_id": "C2",
+    "verdict": "eligible",
+    "extracted_value": "5 projects completed",
+    "normalized_value": 5,
+    "source": {
+      "doc_name": "project_experience.pdf",
+      "page": 1,
+      "snippet": "5 completed construction projects listed, each >= INR 1 Crore",
+      "source_type": "text_pdf"
+    },
+    "llm_confidence": 0.95,
+    "ocr_confidence": null,
+    "combined_confidence": 0.95,
+    "reason": "Bidder has completed 5 similar construction projects in the last 5 years, exceeding the minimum of 3.",
+    "model_version": "deepseek-chat@2026-05-07",
+    "timestamp": "2026-05-06T19:17:36.919283+00:00",
+    "review_status": "pending"
+  },
+  {
+    "verdict_id": "V-46a017a4",
+    "bidder_id": "bidder_a",
+    "criterion_id": "C3",
+    "verdict": "eligible",
+    "extracted_value": "GSTIN: 27AABCA1234F1Z5",
+    "normalized_value": null,
+    "source": {
+      "doc_name": "gst_certificate.pdf",
+      "page": 1,
+      "snippet": "GSTIN: 27AABCA1234F1Z5, Status: ACTIVE",
+      "source_type": "text_pdf"
+    },
+    "llm_confidence": 0.98,
+    "ocr_confidence": null,
+    "combined_confidence": 0.98,
+    "reason": "Valid GST registration certificate present with active GSTIN status.",
+    "model_version": "deepseek-chat@2026-05-07",
+    "timestamp": "2026-05-06T19:17:36.919283+00:00",
+    "review_status": "pending"
+  },
+  {
+    "verdict_id": "V-9eb49fce",
+    "bidder_id": "bidder_a",
+    "criterion_id": "C4",
+    "verdict": "eligible",
+    "extracted_value": "ISO-2021-9001-APEX, valid 15-06-2027",
+    "normalized_value": null,
+    "source": {
+      "doc_name": "iso_9001.pdf",
+      "page": 1,
+      "snippet": "ISO 9001:2015 Certificate No: ISO-2021-9001-APEX, Valid Through: 15-06-2027",
+      "source_type": "text_pdf"
+    },
+    "llm_confidence": 0.97,
+    "ocr_confidence": null,
+    "combined_confidence": 0.97,
+    "reason": "Valid ISO 9001:2015 certificate present, valid through June 2027.",
+    "model_version": "deepseek-chat@2026-05-07",
+    "timestamp": "2026-05-06T19:17:36.919283+00:00",
+    "review_status": "pending"
+  },
+  {
+    "verdict_id": "V-8f79d80a",
+    "bidder_id": "bidder_a",
+    "criterion_id": "C5",
+    "verdict": "eligible",
+    "extracted_value": "CRPF Camp Pune barracks project (2024)",
+    "normalized_value": null,
+    "source": {
+      "doc_name": "project_experience.pdf",
+      "page": 1,
+      "snippet": "Barracks Construction, CRPF Camp Pune, INR 3.5 Cr, 2024",
+      "source_type": "text_pdf"
+    },
+    "llm_confidence": 0.88,
+    "ocr_confidence": null,
+    "combined_confidence": 0.88,
+    "reason": "Bidder has prior experience with CRPF paramilitary infrastructure (barracks construction, 2024).",
+    "model_version": "deepseek-chat@2026-05-07",
+    "timestamp": "2026-05-06T19:17:36.919283+00:00",
+    "review_status": "pending"
+  }
+]

data/precomputed/eval_bidder_b.json

@@ -0,0 +1,107 @@
+[
+  {
+    "verdict_id": "V-6974e6c0",
+    "bidder_id": "bidder_b",
+    "criterion_id": "C1",
+    "verdict": "not_eligible",
+    "extracted_value": "INR 1.5 Cr (avg)",
+    "normalized_value": 15000000,
+    "source": {
+      "doc_name": "audited_financials.pdf",
+      "page": 1,
+      "snippet": "average annual turnover INR 1,50,00,000",
+      "source_type": "text_pdf"
+    },
+    "llm_confidence": 0.95,
+    "ocr_confidence": null,
+    "combined_confidence": 0.95,
+    "reason": "Average annual turnover of INR 1.5 Cr is below the required minimum of INR 5 Cr.",
+    "model_version": "deepseek-chat@2026-05-07",
+    "timestamp": "2026-05-06T19:17:36.919283+00:00",
+    "review_status": "pending"
+  },
+  {
+    "verdict_id": "V-b0f1c880",
+    "bidder_id": "bidder_b",
+    "criterion_id": "C2",
+    "verdict": "eligible",
+    "extracted_value": "4 projects completed",
+    "normalized_value": 4,
+    "source": {
+      "doc_name": "project_experience.pdf",
+      "page": 1,
+      "snippet": "4 completed construction projects listed",
+      "source_type": "text_pdf"
+    },
+    "llm_confidence": 0.9,
+    "ocr_confidence": null,
+    "combined_confidence": 0.9,
+    "reason": "Bidder has completed 4 similar projects, meeting the minimum requirement of 3.",
+    "model_version": "deepseek-chat@2026-05-07",
+    "timestamp": "2026-05-06T19:17:36.919283+00:00",
+    "review_status": "pending"
+  },
+  {
+    "verdict_id": "V-078f742e",
+    "bidder_id": "bidder_b",
+    "criterion_id": "C3",
+    "verdict": "eligible",
+    "extracted_value": "GSTIN: 29AABCB5678G1Z3",
+    "normalized_value": null,
+    "source": {
+      "doc_name": "gst_certificate.pdf",
+      "page": 1,
+      "snippet": "GSTIN: 29AABCB5678G1Z3, Status: ACTIVE",
+      "source_type": "text_pdf"
+    },
+    "llm_confidence": 0.97,
+    "ocr_confidence": null,
+    "combined_confidence": 0.97,
+    "reason": "Valid GST registration certificate with active status.",
+    "model_version": "deepseek-chat@2026-05-07",
+    "timestamp": "2026-05-06T19:17:36.919283+00:00",
+    "review_status": "pending"
+  },
+  {
+    "verdict_id": "V-27021b0e",
+    "bidder_id": "bidder_b",
+    "criterion_id": "C4",
+    "verdict": "eligible",
+    "extracted_value": "ISO-2022-9001-BR, valid 20-08-2027",
+    "normalized_value": null,
+    "source": {
+      "doc_name": "iso_9001.pdf",
+      "page": 1,
+      "snippet": "ISO 9001:2015 Certificate No: ISO-2022-9001-BR, Valid Through: 20-08-2027",
+      "source_type": "text_pdf"
+    },
+    "llm_confidence": 0.96,
+    "ocr_confidence": null,
+    "combined_confidence": 0.96,
+    "reason": "Valid ISO 9001:2015 certificate, valid through August 2027.",
+    "model_version": "deepseek-chat@2026-05-07",
+    "timestamp": "2026-05-06T19:17:36.919283+00:00",
+    "review_status": "pending"
+  },
+  {
+    "verdict_id": "V-c5a9240b",
+    "bidder_id": "bidder_b",
+    "criterion_id": "C5",
+    "verdict": "not_eligible",
+    "extracted_value": "No paramilitary experience found",
+    "normalized_value": null,
+    "source": {
+      "doc_name": "project_experience.pdf",
+      "page": 1,
+      "snippet": "projects are municipal/educational/warehousing \u2014 no paramilitary/defence",
+      "source_type": "text_pdf"
+    },
+    "llm_confidence": 0.85,
+    "ocr_confidence": null,
+    "combined_confidence": 0.85,
+    "reason": "No prior experience with paramilitary or defence infrastructure found in submitted documents.",
+    "model_version": "deepseek-chat@2026-05-07",
+    "timestamp": "2026-05-06T19:17:36.919283+00:00",
+    "review_status": "pending"
+  }
+]

data/precomputed/eval_bidder_c.json

@@ -0,0 +1,107 @@
+[
+  {
+    "verdict_id": "V-fbea0467",
+    "bidder_id": "bidder_c",
+    "criterion_id": "C1",
+    "verdict": "needs_review",
+    "extracted_value": "INR 5.4 Cr (from scan)",
+    "normalized_value": 54000000,
+    "source": {
+      "doc_name": "turnover_certificate_scan.png",
+      "page": 1,
+      "snippet": "average annual turnover INR 5,40,00,000",
+      "source_type": "tesseract"
+    },
+    "llm_confidence": 0.55,
+    "ocr_confidence": 0.58,
+    "combined_confidence": 0.58,
+    "reason": "Turnover certificate submitted as a low-quality scan (Tesseract confidence ~58%). Value of INR 5.4 Cr is close to the threshold. Human review required to confirm authenticity and accuracy.",
+    "model_version": "deepseek-chat@2026-05-07",
+    "timestamp": "2026-05-06T19:17:36.919283+00:00",
+    "review_status": "pending"
+  },
+  {
+    "verdict_id": "V-baa224b9",
+    "bidder_id": "bidder_c",
+    "criterion_id": "C2",
+    "verdict": "eligible",
+    "extracted_value": "3 projects completed (borderline)",
+    "normalized_value": 3,
+    "source": {
+      "doc_name": "project_experience.pdf",
+      "page": 1,
+      "snippet": "3 completed construction projects listed: INR 1.2 Cr, 1.5 Cr, 2.1 Cr",
+      "source_type": "text_pdf"
+    },
+    "llm_confidence": 0.87,
+    "ocr_confidence": null,
+    "combined_confidence": 0.87,
+    "reason": "Bidder has exactly 3 completed similar projects, meeting the minimum threshold.",
+    "model_version": "deepseek-chat@2026-05-07",
+    "timestamp": "2026-05-06T19:17:36.919283+00:00",
+    "review_status": "pending"
+  },
+  {
+    "verdict_id": "V-f1c8ec3e",
+    "bidder_id": "bidder_c",
+    "criterion_id": "C3",
+    "verdict": "eligible",
+    "extracted_value": "GSTIN: 24AABCC9012H1Z1",
+    "normalized_value": null,
+    "source": {
+      "doc_name": "gst_certificate.pdf",
+      "page": 1,
+      "snippet": "GSTIN: 24AABCC9012H1Z1, Status: ACTIVE",
+      "source_type": "text_pdf"
+    },
+    "llm_confidence": 0.97,
+    "ocr_confidence": null,
+    "combined_confidence": 0.97,
+    "reason": "Valid GST registration certificate with active status.",
+    "model_version": "deepseek-chat@2026-05-07",
+    "timestamp": "2026-05-06T19:17:36.919283+00:00",
+    "review_status": "pending"
+  },
+  {
+    "verdict_id": "V-6eafeb5a",
+    "bidder_id": "bidder_c",
+    "criterion_id": "C4",
+    "verdict": "eligible",
+    "extracted_value": "ISO-2023-9001-SCS, valid 10-09-2027",
+    "normalized_value": null,
+    "source": {
+      "doc_name": "iso_9001.pdf",
+      "page": 1,
+      "snippet": "ISO 9001:2015 Certificate No: ISO-2023-9001-SCS, Valid Through: 10-09-2027",
+      "source_type": "text_pdf"
+    },
+    "llm_confidence": 0.96,
+    "ocr_confidence": null,
+    "combined_confidence": 0.96,
+    "reason": "Valid ISO 9001:2015 certificate, valid through September 2027.",
+    "model_version": "deepseek-chat@2026-05-07",
+    "timestamp": "2026-05-06T19:17:36.919283+00:00",
+    "review_status": "pending"
+  },
+  {
+    "verdict_id": "V-5843c2c4",
+    "bidder_id": "bidder_c",
+    "criterion_id": "C5",
+    "verdict": "not_eligible",
+    "extracted_value": "No paramilitary experience documented",
+    "normalized_value": null,
+    "source": {
+      "doc_name": "project_experience.pdf",
+      "page": 1,
+      "snippet": "projects are GIDC/Municipal Corporation/NHAI \u2014 no defence",
+      "source_type": "text_pdf"
+    },
+    "llm_confidence": 0.82,
+    "ocr_confidence": null,
+    "combined_confidence": 0.82,
+    "reason": "No prior experience with paramilitary or defence infrastructure found.",
+    "model_version": "deepseek-chat@2026-05-07",
+    "timestamp": "2026-05-06T19:17:36.919283+00:00",
+    "review_status": "pending"
+  }
+]

scripts/precompute_results.py

@@ -1 +1,72 @@
 """Step 11 — runs the full pipeline and writes data/precomputed/*.json."""
+
+import json
+import sys
+from pathlib import Path
+
+BASE_DIR = Path(__file__).resolve().parent.parent
+sys.path.insert(0, str(BASE_DIR))
+
+from core.config import DATA_DIR, DEEPSEEK_API_KEY, PRECOMPUTED_DIR
+from core.criteria_extractor import extract_criteria
+from core.bidder_processor import process_bidder
+from core.evaluator import evaluate_bidder
+from core.fallback import _HARDCODED_CRITERIA
+from core.schemas import Criterion
+
+
+def main() -> None:
+    if not DEEPSEEK_API_KEY:
+        print("ERROR: DEEPSEEK_API_KEY is not set.")
+        print("Set it in .env or export it before running this script.")
+        sys.exit(1)
+
+    PRECOMPUTED_DIR.mkdir(parents=True, exist_ok=True)
+
+    # Step 1 — Extract criteria
+    tender_path = DATA_DIR / "tender" / "crpf_construction_tender.pdf"
+    print(f"Extracting criteria from {tender_path.name}...")
+    try:
+        criteria = extract_criteria(tender_path)
+        print(f"  Got {len(criteria)} criteria from LLM.")
+    except Exception as e:
+        print(f"  LLM extraction failed ({e}), using hardcoded criteria.")
+        criteria = [Criterion(**c) for c in _HARDCODED_CRITERIA]
+
+    criteria_file = PRECOMPUTED_DIR / "criteria.json"
+    criteria_file.write_text(
+        json.dumps({"criteria": [c.model_dump() for c in criteria]},
+                   indent=2, ensure_ascii=False),
+        encoding="utf-8",
+    )
+    print(f"  Saved {criteria_file}")
+
+    # Step 2 — Process + evaluate each bidder
+    bidders = ["bidder_a", "bidder_b", "bidder_c"]
+    for bidder_id in bidders:
+        bidder_dir = DATA_DIR / "bidders" / bidder_id
+        files = sorted(bidder_dir.glob("*"))
+        files = [f for f in files if f.suffix.lower() in {".pdf", ".png", ".jpg"}]
+
+        print(f"\nProcessing {bidder_id} ({len(files)} files)...")
+        process_bidder(bidder_id, files)
+
+        print(f"  Evaluating {bidder_id} against {len(criteria)} criteria...")
+        verdicts = evaluate_bidder(bidder_id, criteria)
+
+        eval_file = PRECOMPUTED_DIR / f"eval_{bidder_id}.json"
+        eval_file.write_text(
+            json.dumps([v.model_dump() for v in verdicts], indent=2, ensure_ascii=False),
+            encoding="utf-8",
+        )
+        print(f"  Saved {eval_file}")
+        for v in verdicts:
+            print(f"    {v.criterion_id}: {v.verdict} (conf={v.combined_confidence:.2f})")
+
+    print("\nPre-computation complete. Files in data/precomputed/:")
+    for f in sorted(PRECOMPUTED_DIR.glob("*.json")):
+        print(f"  {f.name} ({f.stat().st_size} bytes)")
+
+
+if __name__ == "__main__":
+    main()

specs/12_precompute.md

@@ -0,0 +1,73 @@
+# Spec 12 — Pre-compute Results
+
+**Step:** 11 of 15  
+**Time budget:** ~15 min  
+**Checkpoint:** Four JSON files exist in `data/precomputed/` and validate against the schemas.
+
+---
+
+## Goal
+
+`scripts/precompute_results.py` runs the full pipeline once (requires a valid API key), saves the results as JSON fallback files, and commits them to the repo. When the API is unavailable during a demo, `fallback.py` reads these files instead.
+
+---
+
+## Script: `scripts/precompute_results.py`
+
+```python
+"""Step 11 — runs the full pipeline and writes data/precomputed/*.json."""
+```
+
+### Steps
+
+1. Ensure `data/precomputed/` exists.
+2. Extract criteria from mock tender → save `data/precomputed/criteria.json`:
+   ```json
+   {"criteria": [<Criterion.model_dump()>, ...]}
+   ```
+3. For each bidder (`bidder_a`, `bidder_b`, `bidder_c`):
+   a. Process all bidder docs (`process_bidder`).
+   b. Evaluate all criteria (`evaluate_bidder`).
+   c. Save `data/precomputed/eval_{bidder_id}.json`:
+      ```json
+      [<Verdict.model_dump()>, ...]
+      ```
+4. Print summary and exit 0.
+
+### Error handling
+
+If the LLM fails for any criterion: catch `LLMUnavailable`, log a warning, skip that criterion (don't crash). At least the criteria file and partial evals are better than nothing.
+
+If no API key: print instructions and exit 1.
+
+---
+
+## Fallback file format
+
+### `criteria.json`
+```json
+{
+  "criteria": [
+    {"id": "C1", "title": "...", ...},
+    ...
+  ]
+}
+```
+
+### `eval_bidder_a.json`
+```json
+[
+  {"verdict_id": "V-abc123", "bidder_id": "bidder_a", "criterion_id": "C1", "verdict": "eligible", ...},
+  ...
+]
+```
+
+---
+
+## Acceptance Criteria
+
+1. Running `python scripts/precompute_results.py` exits 0 when API key is set.
+2. `data/precomputed/criteria.json` exists and contains `{"criteria": [...]}` with 5 items.
+3. Each `eval_bidder_*.json` contains a list of 5 `Verdict` dicts.
+4. `from core.fallback import load_criteria` returns 5 `Criterion` objects from the file.
+5. `from core.fallback import load_evaluation` returns the correct `Verdict` for bidder_a, C1.