notes/004-claude-code-learnings.md

Claude Code Source Study — Learnings for Cartographer

Reference session: April 2026 Source: https://github.com/codeaashu/claude-code (reconstruction/tutorial repo) Reference: anthropics/anthropic-cookbook patterns/agents/

---

1. /init Phase 2 — Manifest Files Before Code

What claude-code does:

"Launch a subagent to survey the codebase, and ask it to read key files: manifest files (package.json, Cargo.toml, pyproject.toml, go.mod, pom.xml, etc.), README, Makefile/build configs..."

Why this matters for cartographer: Manifest files are the universal, language-agnostic entry point to any repo:

• They declare dependencies → immediately reveals tech stack (fastapi = web API, torch = ML, tree-sitter = code parsing, no framework = pure library like micrograd)
• They declare entry points/scripts → reveals how the system is run
• They work for ANY repo: web apps, ML libraries, compilers, game engines, CLIs

What we were doing wrong: Phase 1 was reading main.py/app.py module chunks — these are bootstrap files that import ALL features equally. main.py in a FastAPI app imports every router, every service, every feature. Its import graph says "everything is equally important", which is the opposite of the pipeline signal we need.

Fix applied: Added _manifest_chunks() to read project manifests first in Phase 1. The LLM anchors on declared dependencies → understands project type → identifies pipeline stages correctly.

---

2. No Hardcoded Heuristics — Principles Over Examples

What claude-code does: The /init prompt states principles and asks the subagent to DISCOVER, not guess:

• "Detect: Build, test, and lint commands (especially non-standard ones)"
• "Note what you could NOT figure out from code alone — these become interview questions"

It never says "if you see a routers/ directory, skip it" or "if you see ingestion/, it's a pipeline stage". The agent reads actual file content and reasons from there.

What we were doing wrong: Our Phase 1 prompt was full of domain-specific terms:

• "ingestion, parsing, embedding, retrieval, inference" — only valid for LLM/RAG apps
• "routers, routes, middleware, handlers" — only valid for web apps
• Good/bad examples like "Gradient Backpropagation" or "Token Embedding"

These break silently on any non-web, non-LLM repo.

Fix applied: Phase 1 prompt now:

1. Reads manifest → understands tech stack from dependencies
2. Reads README → understands what the system does
3. Reads module-level imports → sees what each file ACTUALLY uses
4. States rules as universal principles: "a stage takes data in one form and produces it in another — evident from its imports and function signatures"

No domain terms, no directory name assumptions, no illustrative examples.

---

3. Evaluator-Optimizer Pattern (Anthropic Cookbook)

From patterns/agents/evaluator_optimizer.ipynb:

def evaluate(prompt, content, task):
    # Returns <evaluation>PASS|NEEDS_IMPROVEMENT|FAIL</evaluation>
    # AND <feedback>specific actionable feedback</feedback>

def loop(task, evaluator_prompt, generator_prompt):
    while True:
        evaluation, feedback = evaluate(evaluator_prompt, result, task)
        if evaluation == "PASS":
            return result
        context = "Previous attempts: [memory] Feedback: [feedback]"
        result = generate(generator_prompt, task, context)

Key properties:

1. Feedback accumulates across rounds — each round gets context of what was tried
2. Clear pass/fail criteria — universal principles, not content-specific examples
3. Evaluation and generation are separate concerns — different prompts, different roles

What we were doing wrong: Our evaluator's "remove trivial infrastructure" instruction had no corresponding response format — the LLM said "remove" but didn't know HOW to signal removal. Result: "Noise File Exclusion" passed because it sounds like a technique name.

Fix applied: Explicit action: keep|rename|remove field. Three-test rubric for quality:

1. Is it a technique/decision name (not a filename/class name)?
2. Would removing it leave an engineer unable to understand the system's core behaviour?
3. Does the subtitle describe a real design decision with tradeoffs?

---

4. Tool Design — get_architecture vs Import Graph Reading

What claude-code does: Exposes get_architecture as a curated tool that provides high-level overview. Also list_directory, search_source, read_source_file for targeted exploration.

The agent DECIDES what to look at. It doesn't get a static snapshot of everything.

What we were doing wrong: Phase 1 gave the LLM a static 80-file flat list + 14 random module chunks and asked it to infer the pipeline. No structure, no priority, no anchoring.

Fix applied:

• Files grouped by directory (structural signal)
• Manifest files first (tech stack signal)
• Module chunks from non-bootstrap files only
• README as anchor for what capabilities must appear

Still room to improve: The right long-term fix is to make Phase 1 truly agentic — give it tools (list_files, search_code, read_file) and let it explore rather than giving it a pre-assembled snapshot. This is how claude-code's /init actually works: a subagent with tools reads specific files, not a single-shot prompt.

---

5. Bootstrap Files Are Wiring, Not Pipeline

Universal pattern (any framework):

• main.py, app.py, server.py — Python web apps (FastAPI, Flask, Django)
• index.js, server.js — Node.js apps (Express, Fastify)
• main.go — Go apps
• main.rs — Rust apps (Actix, Axum)
• Application.java — Spring Boot

These files wire together all features/routers/services. They import everything. Their import graph tells you "all features exist" — not which ones are the core pipeline.

Fix applied: Removed bootstrap filenames from _ENTRY_NAMES. Phase 1 now gets module chunks from service/library files that actually DO the work.

---

6. What /init's "Note what you could NOT figure out" means for tours

The /init prompt says: "Note what you could NOT figure out from code alone — these become interview questions."

This is already implemented in Phase 2 (_phase_investigate) via the gaps field:

"GAPS: What important design rationale CANNOT be determined from this code alone?"

The ask field on each concept card also embodies this: it surfaces the question a new engineer MUST understand to work with this component, often something not visible in the code.

This is the right model. Keep it.

---

7. Contextual Retrieval Prompt Quality

Current _CONTEXT_SYSTEM:

"Write 1-2 sentences that situate this chunk within the document: name the function/class, state its role in the file's pipeline, and name the key identifier(s) a developer would search for."

This is solid. The system prompt correctly tells the model its output is prepended to chunks for embedding — it must match developer search queries, not explain failure modes.

The chunk_question correctly uses Anthropic's prompt caching pattern:

• Document block → cache_control: ephemeral (cached per file, ~10% cost for subsequent chunks)
• Chunk + question → varies per chunk (not cached)

No changes needed here — contextual retrieval was not touched in this session.

---

8. Remaining Quality Issues to Fix

From the latest tour output:

• "In-Memory Archive Extraction" from qdrant_store.py with garbled description → Phase 2 hallucinating about a storage layer file. Fix: Phase 2 investigation prompt needs stronger grounding: "only use information visible in the code above"
• Some concept descriptions still feel thin
• The ask questions could be sharper — they should be answerable ONLY by someone who read the specific concept, not generic questions

Next session priorities:

1. Fix Phase 2 hallucination on storage/infrastructure files
2. Improve ask question specificity
3. Consider agentic Phase 1 (tool-based exploration instead of static snapshot)