Hugging Face's logo

Spaces:
overthelex
/
lmaf
Running

App Files Community
Lexai vovkes222 Claude Opus 4.6 commited on
Commit
c63f734
·
unverified ·
1 Parent(s): eee0d48

docs: rewrite README to match actual architecture and codebase (#9)

Browse files

Replaces the old README with accurate documentation: 4-phase pipeline,
9 agents with descriptions matching code, critique routing, project
structure, ConsultationState fields, CI/CD pipeline, Bedrock config.

Co-authored-by: overthelex <mcvovkes@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

Files changed (1) hide show
  1. README.md +99 -74
README.md CHANGED
@@ -18,88 +18,102 @@ tags:
18
 
19
  # LMAF -- Legal Multi-Agent Framework
20
 
21
- A multi-agent framework for **complex legal consultations** over Ukrainian court decisions and legislation.
22
 
23
- Built for the legal domain with access to 100M+ Ukrainian court decisions via [SecondLayer](https://legal.org.ua).
24
 
25
- ## Architecture
26
 
27
- Nine specialised LLM agents work in a loop, each starting from a fresh context (no conversation history). All state lives in a structured `ConsultationState` object. The workspace is git-versioned for full reproducibility.
28
 
29
  ```
30
- ┌─────────────┐
31
- │ Client │
32
- │ Question │
33
- └──────┬──────┘
34
- │
35
- ┌──────▼──────┐
36
- │ Surveyor │ Maps legal landscape (once)
37
- └──────┬──────┘
38
- │
39
- ┌──────▼──────┐
40
- │ Planner │ Research strategy
41
- └──────┬──────┘
42
- │
43
- ┌────────────▼────────────┐
44
- │ Orchestrator │ Dispatches tasks
45
- └───┬─────────────────┬───┘
46
- │ │
47
- ┌──────▼──────┐ ┌──────▼──────┐
48
- │ Researcher │ │ Analyst │
49
- │ (case law, │ │ (deadlines, │
50
- │ legislation)│ │ penalties) │
51
- └──────┬──────┘ └──────┬──────┘
52
- │ │
53
- ┌───▼─────────────────▼───┐
54
- │ Reviewer │ Adversarial verification
55
- └────────────┬────────────┘
56
- │
57
- ┌────────────▼────────────┐
58
- │ Critic (periodic) │ Strategy audit
59
- └──────┬──────────┬───────┘
60
- │ │
61
- ┌──────▼──┐ ┌────▼────────┐
62
- │ Planner │ │ Adjudicator │ Resolve disputes
63
- │(revise) │ └─────────────┘
64
- └─────────┘
65
- │
66
- ┌────────────▼────────────┐
67
- │ Formatter │ Final consultation
68
- └─────────────────────────┘
69
  ```
70
 
71
- ### Agent Roles
72
 
73
- | Agent | Role | secondlayer-core Equivalent |
74
- |-------|------|---------------------------|
75
- | **Surveyor** | Maps legal landscape, identifies relevant areas of law | IntentClassifier + QueryPlanner |
76
- | **Planner** | Produces/revises research strategy | ExecutionPlan generation |
77
- | **Orchestrator** | Dispatches tasks, formulates hypotheses | ChatService agentic loop |
78
- | **Researcher** | Finds case law, legislation, doctrine via SecondLayer MCP | Tool calls (search, legislation) |
79
- | **Analyst** | Computes deadlines, penalties, procedural checks | Calculation tool calls |
80
- | **Reviewer** | Adversarial verification of evidence | CitationValidator + HallucinationGuard |
81
- | **Critic** | Periodic strategy and coherence audit | *New capability* |
82
- | **Adjudicator** | Resolves inter-agent disagreements | *New capability* |
83
- | **Formatter** | Produces structured legal consultation | Response synthesis |
84
 
85
- ### Key Design Decisions
 
 
 
 
 
 
 
 
 
 
86
 
87
- 1. **No agent carries conversation history** -- each call starts from a fresh context with the current `ConsultationState` rendered as text. This prevents context contamination and allows any agent to be swapped or retried independently.
88
 
89
- 2. **Structured state** -- `ConsultationState` tracks hypotheses, evidence, critiques, and their relationships. Agents mutate state via typed operations, not free-form text.
90
 
91
- 3. **Git-versioned workspace** -- every iteration creates a commit, making the full research process reproducible and auditable.
 
 
 
 
 
 
 
 
 
 
92
 
93
- 4. **SecondLayer MCP bridge** -- agents access 100M+ court decisions, legislation, and Supreme Court positions through the SecondLayer API.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
94
 
95
  ## Quick Start
96
 
97
  ```bash
98
- # Install
99
  pip install -e .
100
 
101
- # Set API keys
102
- export ANTHROPIC_API_KEY=sk-...
 
 
 
 
103
  export SECONDLAYER_API_KEY=...
104
 
105
  # Run a consultation
@@ -111,23 +125,34 @@ lmaf problems/consumer_penalty.yaml
111
 
112
  ## Example Problems
113
 
114
- | Problem | Difficulty | Description |
115
- |---------|-----------|-------------|
116
- | `consumer_penalty.yaml` | Medium | Penalty and interest calculation for late payment |
117
- | `labor_dismissal.yaml` | Hard | Challenging unlawful dismissal during martial law |
118
- | `property_dispute.yaml` | Hard | Property rights in unregistered partnership |
 
 
 
 
 
 
 
 
119
 
120
  ## Data Sources
121
 
122
- - [EDRSR](https://reyestr.court.gov.ua/) -- 100M+ Ukrainian court decisions
123
- - [Verkhovna Rada](https://zakon.rada.gov.ua/) -- Ukrainian legislation
124
- - [overthelex/ua-case-outcome-6m](https://huggingface.co/datasets/overthelex/ua-case-outcome-6m) -- 6.7M court decisions dataset
125
- - [overthelex/ukrainian-court-decisions](https://huggingface.co/datasets/overthelex/ukrainian-court-decisions) -- 428K balanced benchmark
 
 
 
126
 
127
  ## Related Papers
128
 
129
- - [Temporal Decay of Co-Citation Predictability](https://arxiv.org/abs/2605.17639)
130
- - [A Citation Graph from 100M Court Decisions](https://arxiv.org/abs/2605.15362)
131
  - [Tokenizer Fertility and Zero-Shot Performance on Ukrainian Legal Text](https://arxiv.org/abs/2605.14890)
132
 
133
  ## License
 
18
 
19
  # LMAF -- Legal Multi-Agent Framework
20
 
21
+ Nine specialised LLM agents analyse Ukrainian legal questions, search 100M+ court decisions in the [EDRSR](https://reyestr.court.gov.ua/) via [SecondLayer](https://legal.org.ua), and produce structured legal consultations with citations to legislation and case law.
22
 
23
+ **[Live demo](https://agents.legal.org.ua)** &nbsp;|&nbsp; **[HuggingFace Space](https://huggingface.co/spaces/overthelex/lmaf)** &nbsp;|&nbsp; **[SecondLayer Platform](https://legal.org.ua)**
24
 
25
+ ## How It Works
26
 
27
+ A client question goes through four phases. Every agent starts from a fresh context (no conversation history) -- all state lives in a structured `ConsultationState` object, and every iteration is git-committed for reproducibility.
28
 
29
  ```
30
+ Phase 1 Phase 2 Phase 3 Phase 4
31
+ RESEARCH LOOP (up to 15 iterations)
32
+ ┌─────────────────────────────────────┐
33
+ Question ─> Surveyor ─> Planner ─> Orchestrator ─────────────┐ │ ─> Formatter ─> Consultation
34
+ ^ │ │ │ │
35
+ │ Researcher Analyst │ │
36
+ │ │ │ │ │
37
+ │ └──> Reviewer ────────────┘ │
38
+ │ │ │
39
+ │ Critic (every 3 iter.) │
40
+ │ │ │ │
41
+ └────────┘ Adjudicator ────────────┘
42
+ └─────────────────────────────────────┘
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
43
  ```
44
 
45
+ An interactive D3.js version of this diagram is available on the [Architecture tab](https://agents.legal.org.ua) of the live demo.
46
 
47
+ ## Agents
 
 
 
 
 
 
 
 
 
 
48
 
49
+ | # | Agent | When | What it does |
50
+ |---|-------|------|-------------|
51
+ | 1 | **Surveyor** | Once, at start | Maps the legal landscape: jurisdiction, applicable legislation, Supreme Court positions, risks |
52
+ | 2 | **Planner** | Once + on strategy critique | Produces and revises the research strategy and key legal questions |
53
+ | 3 | **Orchestrator** | Every iteration | Reads state, formulates hypotheses, dispatches tasks to Researcher or Analyst |
54
+ | 4 | **Researcher** | On dispatch | Searches case law, legislation, and doctrine via SecondLayer API (100M+ court decisions) |
55
+ | 5 | **Analyst** | On dispatch | Computes deadlines, penalties (Art. 549-552 CC), 3% annual (Art. 625 CC), inflation losses |
56
+ | 6 | **Reviewer** | After each research/analysis | Adversarial verification: checks citations exist, logic holds, evidence is current |
57
+ | 7 | **Critic** | Every 3 iterations | Senior legal advisor audit: strategy gaps, coherence, missing counterarguments |
58
+ | 8 | **Adjudicator** | On hypothesis conflict | Resolves disagreements between agents; can demote an established hypothesis back to working |
59
+ | 9 | **Formatter** | Once, after loop | Produces the final structured consultation with legislation references and case law citations |
60
 
61
+ ## ConsultationState
62
 
63
+ All agents read and mutate a single state object:
64
 
65
+ | Field | Type | Description |
66
+ |-------|------|-------------|
67
+ | `client_question` | str | Original question |
68
+ | `jurisdiction` | str | civil, commercial, administrative, criminal |
69
+ | `survey_summary` | str | Legal landscape from Surveyor |
70
+ | `strategy` | LegalStrategy | Research plan from Planner |
71
+ | `hypotheses` | LegalHypothesis[] | Working/established/refuted legal positions |
72
+ | `evidence` | LegalEvidence[] | Case law, legislation, doctrine, computations |
73
+ | `questions` | ResearchQuestion[] | Open/resolved research questions |
74
+ | `critiques` | Critique[] | Active/resolved strategy and reasoning critiques |
75
+ | `answer` | str | Final formatted consultation |
76
 
77
+ ## Critique Routing
78
+
79
+ When the Critic fires (every `critic_every_n` iterations), critiques are routed:
80
+
81
+ - **strategy** critique --> Planner (revise research strategy)
82
+ - **completeness** critique --> Orchestrator (generate new research questions)
83
+ - **hypothesis** critique --> Adjudicator (resolve conflict, may demote hypothesis)
84
+
85
+ ## Project Structure
86
+
87
+ ```
88
+ secondlayer-agents/
89
+ ├── app.py # Gradio chat interface (prod + HF Space)
90
+ ├── architecture.html # D3.js interactive architecture diagram
91
+ ├── src/lmaf/
92
+ │ ├── engine.py # Main 4-phase pipeline loop (LMAF class)
93
+ │ ├── agents/ # 9 agent implementations (BaseAgent subclasses)
94
+ │ ├── state/ # ConsultationState, LoopState
95
+ │ ├── core/ # Config, WorkspaceManager (git-versioned)
96
+ │ ├── providers/ # LLM provider (AWS Bedrock)
97
+ │ ├── tools/ # SecondLayer API bridge (HTTP client)
98
+ │ ├── control/ # Loop control logic
99
+ │ ├── verification/ # Evidence verification utilities
100
+ │ └── rendering/ # State rendering for agent prompts
101
+ ├── problems/ # Example legal problems (YAML)
102
+ ├── tests/ # pytest suite (state, config, app)
103
+ └── .github/workflows/ # CI/CD: test --> deploy prod + sync HF
104
+ ```
105
 
106
  ## Quick Start
107
 
108
  ```bash
 
109
  pip install -e .
110
 
111
+ # AWS Bedrock (required)
112
+ export AWS_REGION=eu-central-1
113
+ export AWS_ACCESS_KEY_ID=...
114
+ export AWS_SECRET_ACCESS_KEY=...
115
+
116
+ # SecondLayer API (for case law and legislation search)
117
  export SECONDLAYER_API_KEY=...
118
 
119
  # Run a consultation
 
125
 
126
  ## Example Problems
127
 
128
+ | Problem | Domain | Question |
129
+ |---------|--------|----------|
130
+ | `consumer_penalty.yaml` | Civil | Penalty + 3% annual + inflation for late payment of 150K UAH goods |
131
+ | `labor_dismissal.yaml` | Labor | Challenging dismissal under Art. 40(1) during martial law |
132
+ | `property_dispute.yaml` | Family/Property | Property rights in an unregistered civil partnership (8 years) |
133
+
134
+ ## CI/CD
135
+
136
+ Every push to `main` triggers:
137
+
138
+ 1. **Test** -- `pytest` on ubuntu-latest (36 tests: state, config, app)
139
+ 2. **Deploy to prod** -- SSH to prod, rebuild Docker container, health check
140
+ 3. **Sync to HuggingFace** -- force-push to `overthelex/lmaf` Space
141
 
142
  ## Data Sources
143
 
144
+ | Source | What | Scale |
145
+ |--------|------|-------|
146
+ | [EDRSR](https://reyestr.court.gov.ua/) | Ukrainian court decisions | 100M+ documents |
147
+ | [Verkhovna Rada](https://zakon.rada.gov.ua/) | Ukrainian legislation | Full corpus |
148
+ | [ua-case-outcome-6m](https://huggingface.co/datasets/overthelex/ua-case-outcome-6m) | Court decisions dataset | 6.7M decisions |
149
+ | [ukrainian-court-decisions](https://huggingface.co/datasets/overthelex/ukrainian-court-decisions) | Balanced benchmark | 428K decisions |
150
+ | [edrsr-citation-graph-16m](https://huggingface.co/datasets/overthelex/edrsr-citation-graph-16m) | Citation graph | 16M edges |
151
 
152
  ## Related Papers
153
 
154
+ - [Temporal Decay of Co-Citation Predictability in Legal Statute Retrieval](https://arxiv.org/abs/2605.17639)
155
+ - [A Citation Graph from 100 Million Court Decisions](https://arxiv.org/abs/2605.15362)
156
  - [Tokenizer Fertility and Zero-Shot Performance on Ukrainian Legal Text](https://arxiv.org/abs/2605.14890)
157
 
158
  ## License