upgraedd commited on
Commit
b04791f
·
verified ·
1 Parent(s): f03def6

Upload 2 files

Browse files

Added preemptive narrative control, cognitive environment control and intent inference module

Files changed (2) hide show
  1. EIS_ESL_PNC_CEC_6.txt +1311 -0
  2. EIS_ESL_PNC_CEC_INFMOD.txt +209 -0
EIS_ESL_PNC_CEC_6.txt ADDED
@@ -0,0 +1,1311 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """
3
+ EIS + ESL + PNC + CEC v6 – Full Epistemic Substrate with Cognitive Environment Control
4
+ =========================================================================================
5
+ Fixes applied:
6
+ - Added missing `import requests`
7
+ - Implemented `get_entity_suppression` method in ESLedger
8
+ - Sorted timestamps for coordination and drift calculations
9
+ - Improved `domain_expansion_likelihood` to handle source_types as list
10
+ - Added warning when sentence-transformers is missing
11
+ - Added simple k‑means fallback if sklearn not available
12
+ """
13
+
14
+ import hashlib
15
+ import json
16
+ import os
17
+ import secrets
18
+ import time
19
+ import math
20
+ import re
21
+ import random
22
+ import requests # FIX 1: added missing import
23
+ from datetime import datetime, timedelta
24
+ from typing import Dict, List, Any, Optional, Tuple, Set
25
+ from collections import defaultdict
26
+ from dataclasses import dataclass, field
27
+
28
+ import numpy as np
29
+ from numpy.linalg import norm
30
+ from statistics import mean, stdev
31
+
32
+ # ----------------------------------------------------------------------------
33
+ # OPTIONAL DEPENDENCIES (with fallbacks)
34
+ # ----------------------------------------------------------------------------
35
+ try:
36
+ from sentence_transformers import SentenceTransformer
37
+ HAS_SENTENCE_TRANSFORMERS = True
38
+ except ImportError:
39
+ HAS_SENTENCE_TRANSFORMERS = False
40
+ SentenceTransformer = None
41
+ print("WARNING: sentence-transformers not installed. Using random embeddings (meaning erosion will be unreliable).")
42
+
43
+ try:
44
+ import spacy
45
+ HAS_SPACY = True
46
+ except ImportError:
47
+ HAS_SPACY = False
48
+ spacy = None
49
+
50
+ # ----------------------------------------------------------------------------
51
+ # LAZY EMBEDDER (fallback to random if no sentence-transformers)
52
+ # ----------------------------------------------------------------------------
53
+ _EMBEDDER = None
54
+
55
+ def _load_embedder():
56
+ global _EMBEDDER
57
+ if _EMBEDDER is None and HAS_SENTENCE_TRANSFORMERS:
58
+ try:
59
+ _EMBEDDER = SentenceTransformer('all-MiniLM-L6-v2')
60
+ except Exception:
61
+ _EMBEDDER = None
62
+ return _EMBEDDER
63
+
64
+ def _embed_texts(texts: List[str]) -> Optional[np.ndarray]:
65
+ model = _load_embedder()
66
+ if model is None:
67
+ # fallback: random embeddings (not meaningful but keeps structure)
68
+ return np.random.randn(len(texts), 384).astype('float32')
69
+ arr = model.encode(texts, convert_to_numpy=True, show_progress_bar=False)
70
+ return arr.astype('float32')
71
+
72
+ def _cosine_sim(a: Any, b: Any) -> float:
73
+ a = np.array(a, dtype=np.float32)
74
+ b = np.array(b, dtype=np.float32)
75
+ denom = (norm(a) * norm(b) + 1e-12)
76
+ return float(np.dot(a, b) / denom)
77
+
78
+ # ----------------------------------------------------------------------------
79
+ # OPERATIONAL LAYER FOR ALL PRIMITIVES (Layers 1,2,3)
80
+ # ----------------------------------------------------------------------------
81
+ PRIMITIVE_OPERATIONAL = {
82
+ # Layer 1: Suppression
83
+ "ERASURE": {"mechanism": "removal_of_evidence", "dependency": "record_control", "detectability": 0.9, "false_positive_risk": 0.2},
84
+ "INTERRUPTION": {"mechanism": "disruption_of_continuity", "dependency": "access_to_channels", "detectability": 0.8, "false_positive_risk": 0.3},
85
+ "FRAGMENTATION": {"mechanism": "break_into_pieces", "dependency": "existing_divisions", "detectability": 0.7, "false_positive_risk": 0.4},
86
+ "NARRATIVE_CAPTURE": {"mechanism": "control_official_story", "dependency": "institutional_authority", "detectability": 0.85, "false_positive_risk": 0.25},
87
+ "MISDIRECTION": {"mechanism": "divert_attention", "dependency": "alternative_topics", "detectability": 0.75, "false_positive_risk": 0.35},
88
+ "SATURATION": {"mechanism": "overwhelm_with_content", "dependency": "high_volume_production", "detectability": 0.8, "false_positive_risk": 0.3},
89
+ "DISCREDITATION": {"mechanism": "attack_messenger", "dependency": "vulnerable_reputation", "detectability": 0.85, "false_positive_risk": 0.2},
90
+ "ATTRITION": {"mechanism": "wear_down_over_time", "dependency": "long_duration", "detectability": 0.7, "false_positive_risk": 0.4},
91
+ "ACCESS_CONTROL": {"mechanism": "limit_who_can_speak", "dependency": "gatekeeping_infrastructure", "detectability": 0.9, "false_positive_risk": 0.15},
92
+ "TEMPORAL": {"mechanism": "manipulate_timing", "dependency": "release_schedules", "detectability": 0.7, "false_positive_risk": 0.45},
93
+ "CONDITIONING": {"mechanism": "repetitive_messaging", "dependency": "mass_media_access", "detectability": 0.8, "false_positive_risk": 0.3},
94
+ "META": {"mechanism": "frame_the_framing", "dependency": "epistemic_authority", "detectability": 0.6, "false_positive_risk": 0.5},
95
+ # Layer 2: Preemptive Narrative Control
96
+ "SIGNAL_DILUTION": {"mechanism": "volume_pressure", "dependency": "high_throughput_channel", "detectability": 0.85, "false_positive_risk": 0.3},
97
+ "LEGITIMACY_TRANSFER": {"mechanism": "credibility_piggybacking", "dependency": "trusted_entity", "detectability": 0.75, "false_positive_risk": 0.4},
98
+ "FRAME_PREEMPTION": {"mechanism": "pre_event_language_lock", "dependency": "predictable_event_window", "detectability": 0.7, "false_positive_risk": 0.45},
99
+ "OUTCOME_ANCHORING": {"mechanism": "probability_bias", "dependency": "repeated_messaging", "detectability": 0.8, "false_positive_risk": 0.35},
100
+ "IDENTITY_SHIELD": {"mechanism": "social_cost_of_dissent", "dependency": "identity_group", "detectability": 0.65, "false_positive_risk": 0.5},
101
+ "BUREAUCRATIC_DILUTION": {"mechanism": "process_layering", "dependency": "institutional_review", "detectability": 0.9, "false_positive_risk": 0.2},
102
+ "ATTENTION_ATTRITION": {"mechanism": "sustained_decay", "dependency": "long_issue", "detectability": 0.85, "false_positive_risk": 0.25},
103
+ "CONTROLLED_OPPOSITION_DUPLICATION": {"mechanism": "mirror_dissent", "dependency": "existing_opposition", "detectability": 0.7, "false_positive_risk": 0.45},
104
+ "NARRATIVE_INVERSION": {"mechanism": "reverse_expected_role", "dependency": "archetype", "detectability": 0.8, "false_positive_risk": 0.35},
105
+ "ATTRIBUTION_INVERSION": {"mechanism": "individual_vs_collective", "dependency": "figurehead", "detectability": 0.75, "false_positive_risk": 0.4},
106
+ "CONTROLLED_PARASITE": {"mechanism": "amplify_to_restructure", "dependency": "elite_network", "detectability": 0.6, "false_positive_risk": 0.55},
107
+ "PREEMPTIVE_TRUTH": {"mechanism": "gradual_weak_precursors", "dependency": "seeding_ability", "detectability": 0.7, "false_positive_risk": 0.45},
108
+ # Layer 3: Cognitive Environment Control
109
+ "COGNITIVE_LOAD_DISTRIBUTION": {"mechanism": "attention_fragmentation", "dependency": "multiple_high_salience_events", "detectability": 0.8, "false_positive_risk": 0.35},
110
+ "TRUST_HIJACKING": {"mechanism": "structural_embedding", "dependency": "trusted_institution", "detectability": 0.85, "false_positive_risk": 0.3},
111
+ "SELF_CONCEPT_BINDING": {"mechanism": "identity_attachment", "dependency": "existing_self_concept", "detectability": 0.7, "false_positive_risk": 0.45},
112
+ "INCREMENTAL_SHIFT": {"mechanism": "gradual_boundary_move", "dependency": "repeated_small_changes", "detectability": 0.75, "false_positive_risk": 0.4},
113
+ "INDIRECT_CONFLICT_ROUTING": {"mechanism": "proxy_amplification", "dependency": "insulated_core_actors", "detectability": 0.7, "false_positive_risk": 0.45},
114
+ "MEANING_EROSION": {"mechanism": "term_overextension", "dependency": "high_frequency_usage", "detectability": 0.8, "false_positive_risk": 0.3},
115
+ "EXPECTATION_LOCK": {"mechanism": "pre_loaded_interpretation", "dependency": "foreseeable_event", "detectability": 0.75, "false_positive_risk": 0.4},
116
+ "RESPONSIBILITY_DIFFUSION": {"mechanism": "fragmented_accountability", "dependency": "multi_actor_process", "detectability": 0.85, "false_positive_risk": 0.25},
117
+ "AFFECTIVE_PRIMING": {"mechanism": "emotional_preconditioning", "dependency": "topic_emotion_binding", "detectability": 0.7, "false_positive_risk": 0.5},
118
+ "CURATED_REALNESS": {"mechanism": "selective_imperfection", "dependency": "controlled_system", "detectability": 0.65, "false_positive_risk": 0.5},
119
+ }
120
+
121
+ # ----------------------------------------------------------------------------
122
+ # PATTERN INTERACTION MODELING
123
+ # ----------------------------------------------------------------------------
124
+ PATTERN_INTERACTIONS = {
125
+ ("GRADUAL_TRUTH_RELEASE", "INCREMENTAL_SHIFT"): "Normalization Pipeline",
126
+ ("CONSPIRACY_SATURATION", "COGNITIVE_LOAD_DISTRIBUTION"): "Attention Collapse",
127
+ ("DESIGNATED_VILLAIN", "RESPONSIBILITY_DIFFUSION"): "Blame Containment",
128
+ ("FRAME_PREEMPTION", "EXPECTATION_LOCK"): "Double Framing Lock",
129
+ ("SIGNAL_DILUTION", "MEANING_EROSION"): "Semantic Swamp",
130
+ ("IDENTITY_SHIELD", "SELF_CONCEPT_BINDING"): "Identity Fortress",
131
+ ("ATTRIBUTION_INVERSION", "TRUST_HIJACKING"): "Figurehead Credibility Transfer",
132
+ ("CONTROLLED_PARASITE", "INDIRECT_CONFLICT_ROUTING"): "Proxy Purge",
133
+ ("BUREAUCRATIC_DILUTION", "RESPONSIBILITY_DIFFUSION"): "Accountability Maze",
134
+ ("OUTCOME_ANCHORING", "EXPECTATION_LOCK"): "Predestined Narrative",
135
+ }
136
+
137
+ # ----------------------------------------------------------------------------
138
+ # NEGATION, ENTITY EXTRACTION (robust fallback)
139
+ # ----------------------------------------------------------------------------
140
+ NEGATION_WORDS = {"not", "no", "never", "false", "didn't", "isn't", "wasn't", "weren't", "cannot", "couldn't", "wouldn't", "shouldn't"}
141
+ ANTONYMS = {
142
+ "suppressed": "revealed", "erased": "preserved", "hidden": "public",
143
+ "denied": "confirmed", "falsified": "verified", "concealed": "disclosed"
144
+ }
145
+
146
+ def has_negation(text: str, entity: str = None) -> bool:
147
+ words = text.lower().split()
148
+ if entity:
149
+ for i, w in enumerate(words):
150
+ if entity.lower() in w or w == entity.lower():
151
+ start = max(0, i-5)
152
+ preceding = words[start:i]
153
+ if any(neg in preceding for neg in NEGATION_WORDS):
154
+ return True
155
+ else:
156
+ if any(neg in words for neg in NEGATION_WORDS):
157
+ return True
158
+ return False
159
+
160
+ def claim_polarity(text: str) -> float:
161
+ return 0.3 if has_negation(text) else 1.0
162
+
163
+ def extract_entities(text: str) -> List[Tuple[str, str, bool]]:
164
+ entities = []
165
+ # Simple regex for proper nouns
166
+ pattern = r'\b[A-Z][a-z]*(?:\s+[A-Z][a-z]*)*\b'
167
+ matches = re.findall(pattern, text)
168
+ for match in matches:
169
+ if len(match.split()) <= 4 and match not in ["The", "This", "That", "These", "Those", "I", "We", "They"]:
170
+ negated = has_negation(text, match)
171
+ entities.append((match, "UNKNOWN", negated))
172
+ return entities
173
+
174
+ # ----------------------------------------------------------------------------
175
+ # TAXONOMY (Methods) – extended with all primitives
176
+ # ----------------------------------------------------------------------------
177
+ METHODS = {
178
+ # Layer 1 (suppression)
179
+ 1: {"name": "Total Erasure", "primitive": "ERASURE", "signatures": ["entity_present_then_absent"]},
180
+ 2: {"name": "Soft Erasure", "primitive": "ERASURE", "signatures": ["gradual_fading"]},
181
+ 10: {"name": "Narrative Seizure", "primitive": "NARRATIVE_CAPTURE", "signatures": ["single_explanation"]},
182
+ 12: {"name": "Official Story", "primitive": "NARRATIVE_CAPTURE", "signatures": ["authoritative_sources"]},
183
+ 17: {"name": "Smear Campaign", "primitive": "DISCREDITATION", "signatures": ["ad_hominem_attacks"]},
184
+ 43: {"name": "Conditioning", "primitive": "CONDITIONING", "signatures": ["repetitive_messaging"]},
185
+ # Layer 2 (PNC)
186
+ 101: {"name": "Signal Dilution", "primitive": "SIGNAL_DILUTION", "signatures": ["high_volume_low_variance"]},
187
+ 102: {"name": "Legitimacy Piggybacking", "primitive": "LEGITIMACY_TRANSFER", "signatures": ["co_mention_with_trusted_entity"]},
188
+ 103: {"name": "Frame Preemption", "primitive": "FRAME_PREEMPTION", "signatures": ["early_definition_of_terms"]},
189
+ 104: {"name": "Outcome Anchoring", "primitive": "OUTCOME_ANCHORING", "signatures": ["inevitability_language"]},
190
+ 105: {"name": "Identity Shielding", "primitive": "IDENTITY_SHIELD", "signatures": ["criticism_equated_with_attack"]},
191
+ 106: {"name": "Procedural Labyrinth", "primitive": "BUREAUCRATIC_DILUTION", "signatures": ["process_expansion"]},
192
+ 107: {"name": "Narrative Exhaustion", "primitive": "ATTENTION_ATTRITION", "signatures": ["fatigue_indicators"]},
193
+ 108: {"name": "Mirror Opposition", "primitive": "CONTROLLED_OPPOSITION_DUPLICATION", "signatures": ["symmetrical_arguments"]},
194
+ 109: {"name": "Narrative Inversion", "primitive": "NARRATIVE_INVERSION", "signatures": ["expected_role_reversed"]},
195
+ 110: {"name": "Attribution Inversion", "primitive": "ATTRIBUTION_INVERSION", "signatures": ["collective_to_individual_shift"]},
196
+ 111: {"name": "Controlled Parasite", "primitive": "CONTROLLED_PARASITE", "signatures": ["unusual_access_granted"]},
197
+ 112: {"name": "Preemptive Truth Seeding", "primitive": "PREEMPTIVE_TRUTH", "signatures": ["weak_precursor_sequence"]},
198
+ # Layer 3 (CEC)
199
+ 201: {"name": "Cognitive Load Balancing", "primitive": "COGNITIVE_LOAD_DISTRIBUTION", "signatures": ["attention_fragmentation"]},
200
+ 202: {"name": "Trust Hijacking", "primitive": "TRUST_HIJACKING", "signatures": ["authority_association"]},
201
+ 203: {"name": "Identity Binding", "primitive": "SELF_CONCEPT_BINDING", "signatures": ["belief_identity_overlap"]},
202
+ 204: {"name": "Incremental Shift", "primitive": "INCREMENTAL_SHIFT", "signatures": ["stepwise_acceptance"]},
203
+ 205: {"name": "Proxy Conflict Routing", "primitive": "INDIRECT_CONFLICT_ROUTING", "signatures": ["proxy_amplification"]},
204
+ 206: {"name": "Meaning Erosion", "primitive": "MEANING_EROSION", "signatures": ["term_overextension", "definitional_instability"]},
205
+ 207: {"name": "Expectation Lock", "primitive": "EXPECTATION_LOCK", "signatures": ["preloaded_interpretation"]},
206
+ 208: {"name": "Responsibility Diffusion", "primitive": "RESPONSIBILITY_DIFFUSION", "signatures": ["fragmented_execution"]},
207
+ 209: {"name": "Affective Priming", "primitive": "AFFECTIVE_PRIMING", "signatures": ["preloaded_emotional_response"]},
208
+ 210: {"name": "Curated Realness", "primitive": "CURATED_REALNESS", "signatures": ["selective_imperfection"]},
209
+ }
210
+
211
+ LENSES = {
212
+ 1: "Threat→Response→Control", 2: "Sacred Geometry", 3: "Language Inversions",
213
+ 4: "Crisis→Consent", 5: "Divide and Fragment", 6: "Blame the Victim",
214
+ 70: "Volume Pressure", 71: "Credibility Hijack", 72: "Preemptive Framing",
215
+ 73: "Inevitability Bias", 74: "Identity Fortress", 75: "Process Trap",
216
+ 76: "Attention Mining", 77: "Mirror Trap", 78: "Role Reversal", 79: "Figurehead Shield",
217
+ 80: "Parasite Catalyst", 81: "Gradual Revelation", 82: "Semantic Swamp",
218
+ }
219
+
220
+ def map_signature_to_method(signature: str) -> Optional[Dict]:
221
+ for mid, method in METHODS.items():
222
+ if signature in method["signatures"]:
223
+ return {"method_id": mid, "method_name": method["name"], "primitive": method["primitive"]}
224
+ return None
225
+
226
+ def get_lenses_for_primitive(primitive: str) -> List[int]:
227
+ mapping = {
228
+ "SIGNAL_DILUTION": [70], "LEGITIMACY_TRANSFER": [71], "FRAME_PREEMPTION": [72],
229
+ "OUTCOME_ANCHORING": [73], "IDENTITY_SHIELD": [74], "BUREAUCRATIC_DILUTION": [75],
230
+ "ATTENTION_ATTRITION": [76], "CONTROLLED_OPPOSITION_DUPLICATION": [77],
231
+ "NARRATIVE_INVERSION": [78], "ATTRIBUTION_INVERSION": [79], "CONTROLLED_PARASITE": [80],
232
+ "PREEMPTIVE_TRUTH": [81], "MEANING_EROSION": [82],
233
+ }
234
+ return mapping.get(primitive, [])
235
+
236
+ def get_lens_name(lens_id: int) -> str:
237
+ return LENSES.get(lens_id, f"Lens {lens_id}")
238
+
239
+ # ----------------------------------------------------------------------------
240
+ # MEANING EROSION DETECTOR (v1.3 spec)
241
+ # ----------------------------------------------------------------------------
242
+ def extract_context_window(text: str, target_term: str, window_size: int = 10) -> str:
243
+ """Extract a window of words around the target term."""
244
+ words = text.split()
245
+ for i, w in enumerate(words):
246
+ if target_term.lower() in w.lower():
247
+ start = max(0, i - window_size)
248
+ end = min(len(words), i + window_size + 1)
249
+ return " ".join(words[start:end])
250
+ return text[:200]
251
+
252
+ def mean_vector(vectors: List[np.ndarray]) -> np.ndarray:
253
+ if not vectors:
254
+ return np.zeros(384)
255
+ return np.mean(vectors, axis=0)
256
+
257
+ def pairwise_distances(vectors: List[np.ndarray]) -> List[float]:
258
+ if len(vectors) < 2:
259
+ return []
260
+ dists = []
261
+ for i in range(len(vectors)):
262
+ for j in range(i+1, len(vectors)):
263
+ dists.append(1 - _cosine_sim(vectors[i], vectors[j]))
264
+ return dists
265
+
266
+ def cluster_embeddings(vectors: List[np.ndarray], k: Optional[int] = None) -> List[List[int]]:
267
+ """Simple k-means clustering (fallback)."""
268
+ if len(vectors) < 2:
269
+ return [[i] for i in range(len(vectors))]
270
+ try:
271
+ from sklearn.cluster import KMeans
272
+ k = k or max(2, len(vectors) // 5)
273
+ k = min(k, len(vectors))
274
+ km = KMeans(n_clusters=k, random_state=0, n_init=10)
275
+ labels = km.fit_predict(vectors)
276
+ clusters = [[] for _ in range(k)]
277
+ for idx, lab in enumerate(labels):
278
+ clusters[lab].append(idx)
279
+ return clusters
280
+ except ImportError:
281
+ # fallback: single cluster
282
+ return [list(range(len(vectors)))]
283
+
284
+ def compute_cluster_overlap(clusters: List[List[int]], vectors: List[np.ndarray]) -> float:
285
+ """Higher overlap = less stable definitions."""
286
+ if len(clusters) <= 1:
287
+ return 0.0
288
+ centroids = [mean_vector([vectors[i] for i in cl]) for cl in clusters]
289
+ overlaps = []
290
+ for i in range(len(centroids)):
291
+ for j in range(i+1, len(centroids)):
292
+ sim = _cosine_sim(centroids[i], centroids[j])
293
+ overlaps.append(sim)
294
+ return np.mean(overlaps) if overlaps else 0.0
295
+
296
+ def simulate_random_drift(embeddings_by_time: Dict[datetime, List[np.ndarray]], n_permutations: int = 50) -> float:
297
+ """Return expected drift under random temporal permutation."""
298
+ all_embeddings = []
299
+ all_timestamps = []
300
+ for ts, emb_list in embeddings_by_time.items():
301
+ for emb in emb_list:
302
+ all_embeddings.append(emb)
303
+ all_timestamps.append(ts)
304
+ if len(all_embeddings) < 4:
305
+ return 0.0
306
+ drifts = []
307
+ for _ in range(n_permutations):
308
+ shuffled_ts = random.sample(all_timestamps, len(all_timestamps))
309
+ sorted_pairs = sorted(zip(shuffled_ts, all_embeddings), key=lambda x: x[0])
310
+ window_size = max(1, len(sorted_pairs) // 10)
311
+ centroids = []
312
+ for i in range(0, len(sorted_pairs), window_size):
313
+ window_embs = [emb for _, emb in sorted_pairs[i:i+window_size]]
314
+ if window_embs:
315
+ centroids.append(np.mean(window_embs, axis=0))
316
+ if len(centroids) >= 2:
317
+ drift_vals = [1 - _cosine_sim(centroids[i], centroids[i+1]) for i in range(len(centroids)-1)]
318
+ drifts.append(np.mean(drift_vals))
319
+ return np.mean(drifts) if drifts else 0.0
320
+
321
+ def domain_expansion_likelihood(corpus: List[Dict], target_term: str) -> float:
322
+ """
323
+ Returns a score 0..1 indicating how likely the term's expansion is legitimate domain growth.
324
+ Uses entity diversity, source diversity, and coordination signals.
325
+ """
326
+ docs = [doc for doc in corpus if target_term.lower() in doc.get("text", "").lower()]
327
+ if len(docs) < 3:
328
+ return 0.0
329
+ # Entity diversity over time
330
+ entity_counts = []
331
+ for doc in docs:
332
+ ents = extract_entities(doc.get("text", ""))
333
+ entity_counts.append(len(set(e[0] for e in ents)))
334
+ if len(entity_counts) > 1:
335
+ diversity_growth = (entity_counts[-1] - entity_counts[0]) / (len(entity_counts) + 1)
336
+ else:
337
+ diversity_growth = 0.0
338
+ # Source diversity (fixed: source_types is a list)
339
+ source_types_set = set()
340
+ for doc in docs:
341
+ src_list = doc.get("source_types", [])
342
+ if isinstance(src_list, list):
343
+ for src in src_list:
344
+ source_types_set.add(src)
345
+ elif isinstance(src_list, str):
346
+ source_types_set.add(src_list)
347
+ source_growth = len(source_types_set) / 3.0
348
+ # Coordination likelihood (low = natural)
349
+ coord_scores = [doc.get("coordination_likelihood", 0.0) for doc in docs]
350
+ avg_coord = np.mean(coord_scores) if coord_scores else 0.0
351
+ # Composite
352
+ score = (diversity_growth * 0.4 + source_growth * 0.3 + (1 - avg_coord) * 0.3)
353
+ return min(1.0, max(0.0, score))
354
+
355
+ def detect_meaning_erosion(corpus: List[Dict], target_term: str, time_key: str = "timestamp") -> Dict:
356
+ """
357
+ Implements MeaningErosion v1.3 spec.
358
+ Returns dict with erosion_score and all sub-metrics.
359
+ """
360
+ # Group contexts by time window (e.g., by month)
361
+ contexts_by_time = defaultdict(list)
362
+ for doc in corpus:
363
+ text = doc.get("text", "")
364
+ if target_term.lower() in text.lower():
365
+ ts_str = doc.get(time_key, "")
366
+ try:
367
+ ts = datetime.fromisoformat(ts_str.replace('Z', '+00:00'))
368
+ except:
369
+ continue
370
+ window = ts.strftime("%Y-%m")
371
+ context = extract_context_window(text, target_term)
372
+ contexts_by_time[window].append(context)
373
+ if len(contexts_by_time) < 3:
374
+ return {"error": "Insufficient temporal data", "erosion_score": 0.0}
375
+
376
+ # Compute embeddings for each context
377
+ embeddings_by_time = {}
378
+ for window, contexts in contexts_by_time.items():
379
+ emb_list = []
380
+ for ctx in contexts:
381
+ emb_arr = _embed_texts([ctx])
382
+ if emb_arr is not None:
383
+ emb_list.append(emb_arr[0])
384
+ if emb_list:
385
+ embeddings_by_time[datetime.strptime(window, "%Y-%m")] = emb_list
386
+ if len(embeddings_by_time) < 3:
387
+ return {"error": "Embedding failed", "erosion_score": 0.0}
388
+
389
+ # Sort time windows
390
+ sorted_ts = sorted(embeddings_by_time.keys())
391
+ centroids = [mean_vector(embeddings_by_time[ts]) for ts in sorted_ts]
392
+ # 1. Semantic drift
393
+ drift_scores = []
394
+ for i in range(len(centroids)-1):
395
+ drift_scores.append(1 - _cosine_sim(centroids[i], centroids[i+1]))
396
+ semantic_drift = np.mean(drift_scores) if drift_scores else 0.0
397
+
398
+ # 2. Contextual dispersion
399
+ dispersion_scores = []
400
+ for ts, embs in embeddings_by_time.items():
401
+ if len(embs) >= 2:
402
+ dists = pairwise_distances(embs)
403
+ dispersion_scores.append(np.mean(dists))
404
+ contextual_dispersion = np.mean(dispersion_scores) if dispersion_scores else 0.0
405
+
406
+ # 3. Definition instability
407
+ all_embeddings = [emb for embs in embeddings_by_time.values() for emb in embs]
408
+ if len(all_embeddings) >= 4:
409
+ clusters = cluster_embeddings(all_embeddings, k=max(2, len(all_embeddings)//10))
410
+ definition_instability = compute_cluster_overlap(clusters, all_embeddings)
411
+ else:
412
+ definition_instability = 0.0
413
+
414
+ # 4. Directional coherence
415
+ if len(centroids) >= 3:
416
+ drift_vectors = [centroids[i+1] - centroids[i] for i in range(len(centroids)-1)]
417
+ dir_sims = [_cosine_sim(drift_vectors[i], drift_vectors[i+1]) for i in range(len(drift_vectors)-1)]
418
+ directional_coherence = np.mean(dir_sims) if dir_sims else 0.0
419
+ else:
420
+ directional_coherence = 0.5
421
+
422
+ # 5. Temporal localization (Gini)
423
+ usage_counts = [len(embeddings_by_time[ts]) for ts in sorted_ts]
424
+ if sum(usage_counts) > 0:
425
+ sorted_counts = sorted(usage_counts)
426
+ n = len(sorted_counts)
427
+ cum = np.cumsum(sorted_counts)
428
+ gini = (2 * np.sum(cum) - np.sum(sorted_counts)) / (n * np.sum(sorted_counts) + 1e-9)
429
+ temporal_localization = 1 - gini
430
+ else:
431
+ temporal_localization = 0.5
432
+
433
+ # 6. Frequency growth
434
+ freq_growth = (usage_counts[-1] - usage_counts[0]) / (sum(usage_counts) + 1)
435
+
436
+ # 7. Random baseline
437
+ random_drift = simulate_random_drift(embeddings_by_time)
438
+ drift_ratio = semantic_drift / (random_drift + 1e-6)
439
+ random_drifts = []
440
+ for _ in range(20):
441
+ rd = simulate_random_drift(embeddings_by_time, n_permutations=10)
442
+ random_drifts.append(rd)
443
+ mean_rand = np.mean(random_drifts)
444
+ std_rand = np.std(random_drifts) + 1e-6
445
+ z_score = (semantic_drift - mean_rand) / std_rand
446
+
447
+ # 8. Domain expansion likelihood
448
+ expansion_likelihood = domain_expansion_likelihood(corpus, target_term)
449
+
450
+ # 9. Adversarial scores
451
+ raw_scores = {
452
+ "random_drift": 1.0 / (1.0 + drift_ratio),
453
+ "domain_expansion": expansion_likelihood,
454
+ "measurement_noise": definition_instability * (1 - directional_coherence),
455
+ "frequency_only": freq_growth * (1 - semantic_drift),
456
+ "incentive_convergence": (1 - expansion_likelihood) * directional_coherence
457
+ }
458
+ score_std = np.std(list(raw_scores.values()))
459
+ temp = 0.7 * score_std + 0.3
460
+ temp = max(0.5, min(1.5, temp))
461
+ exp_scores = {k: np.exp(v / temp) for k, v in raw_scores.items()}
462
+ total = sum(exp_scores.values())
463
+ adv_scores = {k: v / total for k, v in exp_scores.items()}
464
+
465
+ # 10. Confidence
466
+ max_adv = max(adv_scores.values())
467
+ confidence = (1 - max_adv) * min(1.0, drift_ratio / 2.0) * (1 - adv_scores["measurement_noise"]) * (0.5 + 0.5 * directional_coherence)
468
+ confidence = min(1.0, max(0.0, confidence))
469
+
470
+ # 11. Verdict
471
+ if confidence > 0.7 and (z_score > 2 or drift_ratio > 1.5) and expansion_likelihood < 0.4 and temporal_localization > 0.4:
472
+ verdict = "erosion"
473
+ elif expansion_likelihood > 0.6 and (definition_instability < 0.4 or directional_coherence > 0.6):
474
+ verdict = "expansion"
475
+ else:
476
+ verdict = "inconclusive"
477
+
478
+ # 12. Causality tier
479
+ if adv_scores["random_drift"] > 0.6:
480
+ causality_tier = "random"
481
+ elif expansion_likelihood > 0.5:
482
+ causality_tier = "emergent_systemic"
483
+ elif adv_scores.get("incentive_convergence", 0) > 0.5:
484
+ causality_tier = "incentive_aligned"
485
+ elif max_adv < 0.3:
486
+ causality_tier = "inconclusive"
487
+ else:
488
+ causality_tier = "centrally_directed"
489
+
490
+ return {
491
+ "erosion_score": confidence,
492
+ "verdict": verdict,
493
+ "confidence": confidence,
494
+ "causality_tier": causality_tier,
495
+ "semantic_drift": semantic_drift,
496
+ "contextual_dispersion": contextual_dispersion,
497
+ "definition_instability": definition_instability,
498
+ "directional_coherence": directional_coherence,
499
+ "temporal_localization": temporal_localization,
500
+ "frequency_growth": freq_growth,
501
+ "drift_ratio": drift_ratio,
502
+ "z_score": z_score,
503
+ "adversarial_scores": adv_scores,
504
+ "expansion_likelihood": expansion_likelihood,
505
+ }
506
+
507
+ # ----------------------------------------------------------------------------
508
+ # ESLedger (extended with all fields and fixed get_entity_suppression)
509
+ # ----------------------------------------------------------------------------
510
+ class ESLedger:
511
+ def __init__(self, path: str = "esl_ledger_v6.json"):
512
+ self.path = path
513
+ self.claims: Dict[str, Dict] = {}
514
+ self.entities: Dict[str, Dict] = {}
515
+ self.signatures: List[Dict] = []
516
+ self.contradiction_graph: Dict[str, Set[str]] = defaultdict(set)
517
+ self.blocks: List[Dict] = []
518
+ self._load()
519
+
520
+ def _load(self):
521
+ if os.path.exists(self.path):
522
+ try:
523
+ with open(self.path, 'r') as f:
524
+ data = json.load(f)
525
+ self.claims = data.get("claims", {})
526
+ self.entities = data.get("entities", {})
527
+ self.signatures = data.get("signatures", [])
528
+ self.blocks = data.get("blocks", [])
529
+ cg = data.get("contradiction_graph", {})
530
+ self.contradiction_graph = {k: set(v) for k, v in cg.items()}
531
+ except Exception:
532
+ pass
533
+
534
+ def _save(self):
535
+ cg_serializable = {k: list(v) for k, v in self.contradiction_graph.items()}
536
+ data = {
537
+ "claims": self.claims,
538
+ "entities": self.entities,
539
+ "signatures": self.signatures,
540
+ "contradiction_graph": cg_serializable,
541
+ "blocks": self.blocks,
542
+ "updated": datetime.utcnow().isoformat() + "Z"
543
+ }
544
+ with open(self.path + ".tmp", 'w') as f:
545
+ json.dump(data, f, indent=2)
546
+ os.replace(self.path + ".tmp", self.path)
547
+
548
+ def add_claim(self, text: str, agent: str = "user") -> str:
549
+ claim_id = secrets.token_hex(16)
550
+ polarity = claim_polarity(text)
551
+ self.claims[claim_id] = {
552
+ "id": claim_id, "text": text, "agent": agent,
553
+ "timestamp": datetime.utcnow().isoformat() + "Z",
554
+ "entities": [], "signatures": [], "coherence": 0.5,
555
+ "contradictions": [], "suppression_score": 0.0,
556
+ "methods": [], "primitives": [], "lenses": [],
557
+ "polarity": polarity,
558
+ "source_types": [],
559
+ "embedding": None,
560
+ "workflow_events": [],
561
+ "coordination_likelihood": 0.0,
562
+ "pnc_flags": []
563
+ }
564
+ self._save()
565
+ emb_arr = _embed_texts([text])
566
+ if emb_arr is not None:
567
+ self.claims[claim_id]["embedding"] = emb_arr[0].tolist()
568
+ self._save()
569
+ return claim_id
570
+
571
+ def add_entity(self, name: str, etype: str, claim_id: str, negated: bool = False, source_type: str = "unknown"):
572
+ if name not in self.entities:
573
+ self.entities[name] = {
574
+ "name": name, "type": etype,
575
+ "first_seen": datetime.utcnow().isoformat() + "Z",
576
+ "last_seen": self.claims[claim_id]["timestamp"],
577
+ "appearances": [], "coherence_scores": [],
578
+ "suppression_score": 0.0,
579
+ "negated_mentions": [],
580
+ "source_types": {},
581
+ "embeddings": []
582
+ }
583
+ ent = self.entities[name]
584
+ if claim_id not in ent["appearances"]:
585
+ ent["appearances"].append(claim_id)
586
+ if negated:
587
+ ent["negated_mentions"].append(claim_id)
588
+ ent["last_seen"] = self.claims[claim_id]["timestamp"]
589
+ ent["source_types"][source_type] = ent["source_types"].get(source_type, 0) + 1
590
+ if "entities" not in self.claims[claim_id]:
591
+ self.claims[claim_id]["entities"] = []
592
+ if name not in self.claims[claim_id]["entities"]:
593
+ self.claims[claim_id]["entities"].append(name)
594
+ if "source_types" not in self.claims[claim_id]:
595
+ self.claims[claim_id]["source_types"] = []
596
+ if source_type not in self.claims[claim_id]["source_types"]:
597
+ self.claims[claim_id]["source_types"].append(source_type)
598
+ emb = self.claims[claim_id].get("embedding")
599
+ if emb is not None:
600
+ ent.setdefault("embeddings", []).append({
601
+ "timestamp": self.claims[claim_id]["timestamp"],
602
+ "embedding": emb,
603
+ "claim_id": claim_id,
604
+ "text_snippet": self.claims[claim_id]["text"][:512]
605
+ })
606
+ self._save()
607
+
608
+ def add_signature(self, claim_id: str, sig_name: str, weight: float = 0.5, context: Dict = None):
609
+ polarity = self.claims[claim_id].get("polarity", 1.0)
610
+ adjusted_weight = weight * polarity
611
+ method_info = map_signature_to_method(sig_name)
612
+ primitive = method_info["primitive"] if method_info else "UNKNOWN"
613
+ lenses = get_lenses_for_primitive(primitive) if primitive != "UNKNOWN" else []
614
+ self.signatures.append({
615
+ "signature": sig_name, "claim_id": claim_id,
616
+ "timestamp": datetime.utcnow().isoformat() + "Z",
617
+ "weight": adjusted_weight, "context": context or {},
618
+ "method": method_info["method_name"] if method_info else None,
619
+ "primitive": primitive,
620
+ "lenses": lenses
621
+ })
622
+ if sig_name not in self.claims[claim_id]["signatures"]:
623
+ self.claims[claim_id]["signatures"].append(sig_name)
624
+ if method_info and method_info["method_name"] not in self.claims[claim_id]["methods"]:
625
+ self.claims[claim_id]["methods"].append(method_info["method_name"])
626
+ if primitive not in self.claims[claim_id]["primitives"]:
627
+ self.claims[claim_id]["primitives"].append(primitive)
628
+ for lens in lenses:
629
+ if lens not in self.claims[claim_id]["lenses"]:
630
+ self.claims[claim_id]["lenses"].append(lens)
631
+
632
+ # multiplicative suppression score
633
+ combined = 1.0
634
+ for sig in self.claims[claim_id]["signatures"]:
635
+ w = 0.5
636
+ for log in self.signatures:
637
+ if log["signature"] == sig and log["claim_id"] == claim_id:
638
+ w = log.get("weight", 0.5)
639
+ break
640
+ combined *= (1 - w)
641
+ new_score = 1 - combined
642
+ self.claims[claim_id]["suppression_score"] = new_score
643
+
644
+ for entity in self.claims[claim_id]["entities"]:
645
+ ent = self.entities.get(entity)
646
+ if ent:
647
+ ent_combined = 1.0
648
+ for cid in ent["appearances"]:
649
+ sc = self.claims[cid].get("suppression_score", 0.0)
650
+ ent_combined *= (1 - sc)
651
+ ent["suppression_score"] = 1 - ent_combined
652
+ self._save()
653
+
654
+ def add_contradiction(self, claim_id_a: str, claim_id_b: str):
655
+ self.contradiction_graph[claim_id_a].add(claim_id_b)
656
+ self.contradiction_graph[claim_id_b].add(claim_id_a)
657
+ if claim_id_b not in self.claims[claim_id_a]["contradictions"]:
658
+ self.claims[claim_id_a]["contradictions"].append(claim_id_b)
659
+ if claim_id_a not in self.claims[claim_id_b]["contradictions"]:
660
+ self.claims[claim_id_b]["contradictions"].append(claim_id_a)
661
+ self._save()
662
+
663
+ def get_entity_coherence(self, entity_name: str) -> float:
664
+ ent = self.entities.get(entity_name)
665
+ if not ent or len(ent["appearances"]) < 2:
666
+ return 0.5
667
+ timestamps = []
668
+ for cid in ent["appearances"]:
669
+ ts = self.claims[cid]["timestamp"]
670
+ timestamps.append(datetime.fromisoformat(ts.replace('Z', '+00:00')))
671
+ intervals = [(timestamps[i+1] - timestamps[i]).total_seconds() / 86400 for i in range(len(timestamps)-1)]
672
+ if not intervals:
673
+ return 0.5
674
+ mean_int = sum(intervals) / len(intervals)
675
+ variance = sum((i - mean_int)**2 for i in intervals) / len(intervals)
676
+ coherence = 1.0 / (1.0 + variance)
677
+ return min(1.0, max(0.0, coherence))
678
+
679
+ def get_entity_embeddings(self, entity_name: str) -> List[Dict]:
680
+ ent = self.entities.get(entity_name)
681
+ if not ent:
682
+ return []
683
+ return sorted(ent.get("embeddings", []), key=lambda x: x["timestamp"])
684
+
685
+ # FIX 2: Implement get_entity_suppression
686
+ def get_entity_suppression(self, entity_name: str) -> Dict:
687
+ ent = self.entities.get(entity_name)
688
+ if not ent:
689
+ return {"name": entity_name, "score": 0.0, "type": "UNKNOWN", "first_seen": "", "last_seen": "",
690
+ "appearance_count": 0, "negated_count": 0, "coherence": 0.5, "source_types": {}}
691
+ return {
692
+ "name": entity_name,
693
+ "score": ent.get("suppression_score", 0.0),
694
+ "type": ent["type"],
695
+ "first_seen": ent["first_seen"],
696
+ "last_seen": ent["last_seen"],
697
+ "appearance_count": len(ent["appearances"]),
698
+ "negated_count": len(ent.get("negated_mentions", [])),
699
+ "coherence": self.get_entity_coherence(entity_name),
700
+ "source_types": dict(ent.get("source_types", {}))
701
+ }
702
+
703
+ def suppression_pattern_classifier(self, claim_id: str) -> Dict:
704
+ claim = self.claims.get(claim_id, {})
705
+ sig_names = claim.get("signatures", [])
706
+ if not sig_names:
707
+ return {"level": "none", "score": 0.0, "patterns": [], "primitives": [], "lenses": [], "contributions": {}}
708
+ score = claim.get("suppression_score", 0.0)
709
+ contributions = {}
710
+ for log in self.signatures:
711
+ if log["claim_id"] == claim_id:
712
+ contributions[log["signature"]] = contributions.get(log["signature"], 0.0) + log.get("weight", 0.0)
713
+ if score > 0.7:
714
+ level = "high"
715
+ elif score > 0.4:
716
+ level = "medium"
717
+ elif score > 0.1:
718
+ level = "low"
719
+ else:
720
+ level = "none"
721
+ primitives = claim.get("primitives", [])
722
+ lenses = claim.get("lenses", [])
723
+ return {
724
+ "level": level,
725
+ "score": score,
726
+ "contributions": contributions,
727
+ "patterns": list(set(sig_names)),
728
+ "primitives": primitives,
729
+ "lenses": lenses
730
+ }
731
+
732
+ def get_entity_timeline(self, name: str) -> List[Dict]:
733
+ ent = self.entities.get(name)
734
+ if not ent:
735
+ return []
736
+ timeline = []
737
+ for cid in ent["appearances"]:
738
+ claim = self.claims.get(cid)
739
+ if claim:
740
+ timeline.append({
741
+ "timestamp": claim["timestamp"],
742
+ "text": claim["text"],
743
+ "negated": cid in ent.get("negated_mentions", [])
744
+ })
745
+ timeline.sort(key=lambda x: x["timestamp"])
746
+ return timeline
747
+
748
+ def disappearance_suspected(self, name: str, threshold_days: int = 30) -> bool:
749
+ timeline = self.get_entity_timeline(name)
750
+ if not timeline:
751
+ return False
752
+ last = datetime.fromisoformat(timeline[-1]["timestamp"].replace('Z', '+00:00'))
753
+ now = datetime.utcnow()
754
+ return (now - last).days > threshold_days
755
+
756
+ def create_block(self) -> Dict:
757
+ block = {
758
+ "index": len(self.blocks),
759
+ "timestamp": datetime.utcnow().isoformat() + "Z",
760
+ "prev_hash": self.blocks[-1]["hash"] if self.blocks else "0"*64,
761
+ "state_hash": hashlib.sha3_512(json.dumps({"claims": self.claims, "entities": self.entities}, sort_keys=True).encode()).hexdigest()
762
+ }
763
+ block["hash"] = hashlib.sha3_512(json.dumps(block, sort_keys=True).encode()).hexdigest()
764
+ self.blocks.append(block)
765
+ self._save()
766
+ return block
767
+
768
+ def find_contradictions(self, claim_text: str) -> List[str]:
769
+ contradictions = []
770
+ for cid, claim in self.claims.items():
771
+ if self.are_contradictory(claim_text, claim["text"]):
772
+ contradictions.append(cid)
773
+ return contradictions
774
+
775
+ @staticmethod
776
+ def are_contradictory(claim_a: str, claim_b: str) -> bool:
777
+ ents_a = {e[0].lower() for e in extract_entities(claim_a)}
778
+ ents_b = {e[0].lower() for e in extract_entities(claim_b)}
779
+ if not ents_a.intersection(ents_b):
780
+ return False
781
+ a_neg = has_negation(claim_a)
782
+ b_neg = has_negation(claim_b)
783
+ if a_neg != b_neg:
784
+ a_clean = set(claim_a.lower().split()) - NEGATION_WORDS
785
+ b_clean = set(claim_b.lower().split()) - NEGATION_WORDS
786
+ if a_clean == b_clean:
787
+ return True
788
+ a_words = set(claim_a.lower().split())
789
+ b_words = set(claim_b.lower().split())
790
+ for word, antonym in ANTONYMS.items():
791
+ if word in a_words and antonym in b_words:
792
+ return True
793
+ if antonym in a_words and word in b_words:
794
+ return True
795
+ return False
796
+
797
+ def get_suppression_trend(self, window_days: int = 30) -> List[Dict]:
798
+ trend = defaultdict(list)
799
+ for claim in self.claims.values():
800
+ ts = datetime.fromisoformat(claim["timestamp"].replace('Z', '+00:00'))
801
+ date = ts.date().isoformat()
802
+ trend[date].append(claim.get("suppression_score", 0.0))
803
+ result = []
804
+ for date, scores in sorted(trend.items()):
805
+ result.append({"date": date, "avg_suppression": sum(scores)/len(scores)})
806
+ cutoff = (datetime.utcnow() - timedelta(days=window_days)).date().isoformat()
807
+ result = [r for r in result if r["date"] >= cutoff]
808
+ return result
809
+
810
+ def decay_confidence(self, half_life_days: float = 30.0):
811
+ now = datetime.utcnow()
812
+ for claim_id, claim in self.claims.items():
813
+ ts = datetime.fromisoformat(claim["timestamp"].replace('Z', '+00:00'))
814
+ age_days = (now - ts).days
815
+ if age_days > 0:
816
+ decay_factor = math.exp(-age_days / half_life_days)
817
+ claim["suppression_score"] *= decay_factor
818
+ self._save()
819
+
820
+ # ----------------------------------------------------------------------------
821
+ # SIGNATURE GENERATOR (with meaning erosion and coordination)
822
+ # ----------------------------------------------------------------------------
823
+ class SignatureGenerator:
824
+ def __init__(self, esl: ESLedger):
825
+ self.esl = esl
826
+
827
+ def generate_for_claim(self, claim_id: str, claim_text: str) -> List[Tuple[str, float]]:
828
+ signatures = []
829
+
830
+ # ---- Existing suppression detectors ----
831
+ for entity in self.esl.entities:
832
+ if entity.lower() in claim_text.lower():
833
+ if self.esl.disappearance_suspected(entity):
834
+ signatures.append(("entity_present_then_absent", 0.8))
835
+ timeline = self.esl.get_entity_timeline(entity)
836
+ if len(timeline) >= 2:
837
+ last = datetime.fromisoformat(timeline[-1]["timestamp"].replace('Z', '+00:00'))
838
+ days_since = (datetime.utcnow() - last).days
839
+ if 7 < days_since < 30:
840
+ signatures.append(("gradual_fading", 0.6))
841
+
842
+ # semantic drift (simple)
843
+ for entity in self.esl.entities:
844
+ if entity.lower() in claim_text.lower():
845
+ emb_timeline = self.esl.get_entity_embeddings(entity)
846
+ if len(emb_timeline) >= 4:
847
+ first = np.array(emb_timeline[0]["embedding"])
848
+ last = np.array(emb_timeline[-1]["embedding"])
849
+ drift = 1 - _cosine_sim(first, last)
850
+ if drift > 0.3:
851
+ signatures.append(("semantic_drift", min(0.9, 0.3 + drift)))
852
+
853
+ # contradictions
854
+ contradictions = self.esl.contradiction_graph.get(claim_id, set())
855
+ if contradictions:
856
+ signatures.append(("contradictory_claims", 0.7))
857
+
858
+ # repetition
859
+ for cid, claim in self.esl.claims.items():
860
+ if cid != claim_id and claim["text"].lower() == claim_text.lower():
861
+ signatures.append(("repetitive_messaging", 0.9))
862
+ break
863
+
864
+ # coordination likelihood (FIX 3: sort timestamps)
865
+ all_claims = list(self.esl.claims.values())
866
+ if len(all_claims) > 1:
867
+ # extract timestamps and sort
868
+ claims_with_ts = []
869
+ for c in all_claims:
870
+ try:
871
+ ts = datetime.fromisoformat(c["timestamp"].replace('Z', '+00:00'))
872
+ claims_with_ts.append((ts, c))
873
+ except:
874
+ continue
875
+ if len(claims_with_ts) > 1:
876
+ claims_with_ts.sort(key=lambda x: x[0])
877
+ timestamps = [ts for ts, _ in claims_with_ts]
878
+ diffs = [(timestamps[i+1] - timestamps[i]).total_seconds() for i in range(len(timestamps)-1)]
879
+ timing_std = np.std(diffs) if diffs else 1e9
880
+ coord = 1.0 / (1.0 + timing_std / 3600)
881
+ self.esl.claims[claim_id]["coordination_likelihood"] = min(1.0, coord)
882
+ if coord > 0.7:
883
+ signatures.append(("high_coordination", 0.8))
884
+
885
+ # ---- Meaning Erosion detection ----
886
+ words = set(re.findall(r'\b[A-Za-z]{4,}\b', claim_text))
887
+ for term in words:
888
+ term_claims = [c for c in self.esl.claims.values() if term.lower() in c["text"].lower()]
889
+ if len(term_claims) >= 3:
890
+ erosion_result = detect_meaning_erosion(term_claims, term)
891
+ if "error" not in erosion_result and erosion_result.get("erosion_score", 0) > 0.6:
892
+ signatures.append(("term_overextension", 0.7))
893
+ break
894
+
895
+ return signatures
896
+
897
+ # ----------------------------------------------------------------------------
898
+ # FALSIFICATION ENGINE
899
+ # ----------------------------------------------------------------------------
900
+ class FalsificationEngine:
901
+ def __init__(self, esl: ESLedger):
902
+ self.esl = esl
903
+
904
+ def alternative_cause(self, claim_text: str) -> Tuple[bool, str]:
905
+ if has_negation(claim_text):
906
+ return True, "Claim is negated; alternative cause not applicable."
907
+ for entity in self.esl.entities:
908
+ if entity.lower() in claim_text.lower():
909
+ if self.esl.disappearance_suspected(entity):
910
+ return False, f"Entity '{entity}' disappearance may be natural (no recent activity)."
911
+ return True, "No obvious alternative cause."
912
+
913
+ def contradictory_evidence(self, claim_id: str) -> Tuple[bool, str]:
914
+ contradictions = self.esl.contradiction_graph.get(claim_id, set())
915
+ if contradictions:
916
+ return False, f"Claim contradicts {len(contradictions)} existing claim(s)."
917
+ return True, "No direct contradictions."
918
+
919
+ def source_diversity(self, claim_text: str) -> Tuple[bool, str]:
920
+ entities_in_claim = [e for e in self.esl.entities if e.lower() in claim_text.lower()]
921
+ if len(entities_in_claim) <= 1:
922
+ return False, f"Claim relies on only {len(entities_in_claim)} entity/entities."
923
+ return True, f"Multiple entities ({len(entities_in_claim)}) involved."
924
+
925
+ def temporal_stability(self, claim_text: str) -> Tuple[bool, str]:
926
+ for entity in self.esl.entities:
927
+ if entity.lower() in claim_text.lower():
928
+ coherence = self.esl.get_entity_coherence(entity)
929
+ if coherence < 0.3:
930
+ return False, f"Entity '{entity}' has low temporal coherence ({coherence:.2f})."
931
+ return True, "Temporal coherence adequate."
932
+
933
+ def manipulation_check(self, claim_text: str, agent: str) -> Tuple[bool, str]:
934
+ manip_indicators = ["must", "cannot", "obviously", "clearly", "everyone knows"]
935
+ for word in manip_indicators:
936
+ if word in claim_text.lower():
937
+ return False, f"Manipulative language detected: '{word}'."
938
+ return True, "No manipulation indicators."
939
+
940
+ def run_all(self, claim_id: str, claim_text: str, agent: str) -> List[Dict]:
941
+ tests = [
942
+ ("alternative_cause", lambda: self.alternative_cause(claim_text)),
943
+ ("contradictory_evidence", lambda: self.contradictory_evidence(claim_id)),
944
+ ("source_diversity", lambda: self.source_diversity(claim_text)),
945
+ ("temporal_stability", lambda: self.temporal_stability(claim_text)),
946
+ ("manipulation_check", lambda: self.manipulation_check(claim_text, agent))
947
+ ]
948
+ results = []
949
+ for name, func in tests:
950
+ survived, reason = func()
951
+ results.append({"name": name, "survived": survived, "reason": reason})
952
+ return results
953
+
954
+ # ----------------------------------------------------------------------------
955
+ # EPISTEMIC MULTIPLEXOR (with random baseline)
956
+ # ----------------------------------------------------------------------------
957
+ class Hypothesis:
958
+ def __init__(self, desc: str):
959
+ self.desc = desc
960
+ self.prob = 0.0
961
+
962
+ class EpistemicMultiplexor:
963
+ def __init__(self, alpha_fast: float = 0.3, alpha_slow: float = 0.05):
964
+ self.hypotheses: List[Hypothesis] = []
965
+ self.alpha_fast = alpha_fast
966
+ self.alpha_slow = alpha_slow
967
+ self.previous_probs: Dict[str, float] = {}
968
+
969
+ def initialize(self, base_hypotheses: List[str]):
970
+ if not base_hypotheses:
971
+ raise ValueError("base_hypotheses must contain at least one hypothesis")
972
+ self.hypotheses = [Hypothesis(h) for h in base_hypotheses]
973
+ equal = 1.0 / len(self.hypotheses)
974
+ for h in self.hypotheses:
975
+ h.prob = equal
976
+ self.previous_probs = {h.desc: h.prob for h in self.hypotheses}
977
+
978
+ def update(self, evidence_strength: float, signatures: List[str], coherence: float):
979
+ likelihood: Dict[str, float] = {}
980
+ for h in self.hypotheses:
981
+ desc = h.desc.lower()
982
+ if "user claim" in desc:
983
+ lik = 0.5 + evidence_strength * coherence
984
+ elif "official narrative" in desc:
985
+ lik = 0.5 - evidence_strength * 0.3
986
+ elif "suppression" in desc:
987
+ erasure_sigs = {"entity_present_then_absent", "archival_gaps", "gradual_fading"}
988
+ if any(sig in signatures for sig in erasure_sigs):
989
+ lik = 0.5 + evidence_strength * 0.6
990
+ else:
991
+ lik = 0.5 - evidence_strength * 0.2
992
+ elif "natural decay" in desc:
993
+ lik = 0.5 + (0.2 if "gradual_fading" in signatures else -0.1)
994
+ elif "random noise" in desc:
995
+ lik = 0.5
996
+ elif "pnc" in desc:
997
+ pnc_sigs = {"high_volume_low_variance", "early_definition_of_terms", "inevitability_language"}
998
+ if any(sig in signatures for sig in pnc_sigs):
999
+ lik = 0.5 + evidence_strength * 0.5
1000
+ else:
1001
+ lik = 0.5 - evidence_strength * 0.2
1002
+ else:
1003
+ lik = 0.5
1004
+ likelihood[h.desc] = max(0.05, min(0.95, lik))
1005
+
1006
+ posterior_unnorm: Dict[str, float] = {}
1007
+ total = 0.0
1008
+ for h in self.hypotheses:
1009
+ prior = h.prob if h.prob is not None else (1.0 / len(self.hypotheses))
1010
+ post = prior * likelihood[h.desc]
1011
+ posterior_unnorm[h.desc] = post
1012
+ total += post
1013
+
1014
+ if total <= 0:
1015
+ uniform = 1.0 / len(self.hypotheses)
1016
+ for h in self.hypotheses:
1017
+ old = self.previous_probs.get(h.desc, h.prob)
1018
+ smoothed = self.alpha_slow * uniform + (1 - self.alpha_slow) * old
1019
+ h.prob = smoothed
1020
+ self.previous_probs[h.desc] = h.prob
1021
+ return
1022
+
1023
+ for h in self.hypotheses:
1024
+ new_prob = posterior_unnorm[h.desc] / total
1025
+ old = self.previous_probs.get(h.desc, h.prob)
1026
+ smoothed = self.alpha_slow * new_prob + (1 - self.alpha_slow) * old
1027
+ h.prob = smoothed
1028
+ self.previous_probs[h.desc] = h.prob
1029
+
1030
+ def get_probabilities(self) -> Dict[str, float]:
1031
+ return {h.desc: h.prob for h in self.hypotheses}
1032
+
1033
+ # ----------------------------------------------------------------------------
1034
+ # NARRATIVE VIOLATION DETECTOR
1035
+ # ----------------------------------------------------------------------------
1036
+ class NarrativeViolationDetector:
1037
+ def __init__(self, esl: ESLedger):
1038
+ self.esl = esl
1039
+ self.narrative_indicators = [
1040
+ "mainstream narrative", "official story", "commonly believed",
1041
+ "consensus view", "widely accepted", "according to sources",
1042
+ "it is known that", "as reported by", "credible institutions"
1043
+ ]
1044
+
1045
+ def check(self, llm_output: str, claim_text: str) -> Tuple[bool, float, str]:
1046
+ output_lower = llm_output.lower()
1047
+ score = 0.0
1048
+ reasons = []
1049
+ for ind in self.narrative_indicators:
1050
+ if ind in output_lower:
1051
+ score += 0.2
1052
+ reasons.append(f"narrative phrase '{ind}'")
1053
+ esl_mentioned = any(entity.lower() in output_lower for entity in self.esl.entities)
1054
+ if not esl_mentioned:
1055
+ score += 0.4
1056
+ reasons.append("no ESL entity referenced")
1057
+ emotional = ["i believe", "i think", "clearly", "obviously", "must be"]
1058
+ for word in emotional:
1059
+ if word in output_lower:
1060
+ score += 0.1
1061
+ reasons.append(f"emotional language '{word}'")
1062
+ score = min(1.0, score)
1063
+ compliant = score < 0.5
1064
+ reason = "; ".join(reasons) if reasons else "no narrative violation"
1065
+ return compliant, score, reason
1066
+
1067
+ # ----------------------------------------------------------------------------
1068
+ # CONSTRAINED LLM INTERFACE
1069
+ # ----------------------------------------------------------------------------
1070
+ class ConstrainedLLM:
1071
+ def __init__(self, esl: ESLedger, api_key: str = None, api_url: str = None,
1072
+ model: str = "gpt-4", max_retries: int = 2):
1073
+ self.esl = esl
1074
+ self.api_key = api_key or os.environ.get("OPENAI_API_KEY")
1075
+ self.api_url = api_url or "https://api.openai.com/v1/chat/completions"
1076
+ self.model = model
1077
+ self.max_retries = max_retries
1078
+ self.narrative_detector = NarrativeViolationDetector(esl)
1079
+ self.falsifier = FalsificationEngine(esl)
1080
+ self.signature_gen = SignatureGenerator(esl)
1081
+ self.multiplexor = EpistemicMultiplexor()
1082
+
1083
+ def _build_prompt(self, claim_id: str, claim: str, falsification_results: List[Dict],
1084
+ entity_data: Dict, suppression_pattern: Dict, multiplexor_probs: Dict) -> str:
1085
+ prob_table = "\n".join([f"- {h}: {p:.0%}" for h, p in sorted(multiplexor_probs.items(), key=lambda x: -x[1])])
1086
+ return f"""You are EIS v6.1 under constraint. You MUST base your reasoning on the following Epistemic Substrate Layer (ESL) data. Do NOT rely on your internal training narrative.
1087
+
1088
+ CLAIM: {claim}
1089
+
1090
+ ESL ENTITY DATA:
1091
+ {json.dumps(entity_data, indent=2)}
1092
+
1093
+ FALSIFICATION RESULTS (from ESL):
1094
+ {json.dumps(falsification_results, indent=2)}
1095
+
1096
+ SUPPRESSION PATTERN CLASSIFICATION:
1097
+ {json.dumps(suppression_pattern, indent=2)}
1098
+
1099
+ MULTIPLEXOR PROBABILITIES (before your reasoning):
1100
+ {prob_table}
1101
+
1102
+ INSTRUCTIONS:
1103
+ 1. Evaluate the claim against the ESL data only.
1104
+ 2. Output a JSON object with exactly these fields:
1105
+ - "verdict": one of ["Verified", "Unverified", "Refuted", "Insufficient Data"]
1106
+ - "confidence": a float between 0 and 1
1107
+ - "reasoning": a short explanation referencing specific ESL entries (entities, contradictions, signatures)
1108
+ 3. Do NOT add any extra text outside the JSON.
1109
+ """
1110
+
1111
+ def _parse_output(self, response_text: str) -> Optional[Dict]:
1112
+ try:
1113
+ start = response_text.find('{')
1114
+ end = response_text.rfind('}') + 1
1115
+ if start == -1 or end == 0:
1116
+ return None
1117
+ json_str = response_text[start:end]
1118
+ return json.loads(json_str)
1119
+ except Exception:
1120
+ return None
1121
+
1122
+ def _check_constraints(self, output: Dict, claim: str, falsification_results: List[Dict]) -> bool:
1123
+ if not all(k in output for k in ["verdict", "confidence", "reasoning"]):
1124
+ return False
1125
+ if not (0 <= output["confidence"] <= 1):
1126
+ return False
1127
+ if output["verdict"] not in ["Verified", "Unverified", "Refuted", "Insufficient Data"]:
1128
+ return False
1129
+ reasoning = output["reasoning"].lower()
1130
+ esl_mentioned = any(
1131
+ ent.lower() in reasoning for ent in self.esl.entities
1132
+ ) or any(
1133
+ test["name"].lower() in reasoning for test in falsification_results
1134
+ )
1135
+ return esl_mentioned
1136
+
1137
+ def query(self, claim_text: str, agent: str = "user") -> Dict:
1138
+ claim_id = self.esl.add_claim(claim_text, agent)
1139
+ # contradictions
1140
+ for cid in self.esl.find_contradictions(claim_text):
1141
+ self.esl.add_contradiction(claim_id, cid)
1142
+ # entities
1143
+ entities = extract_entities(claim_text)
1144
+ for ent_name, ent_type, negated in entities:
1145
+ source_type = "official" if ent_type in ["ORG", "GPE", "PERSON"] else "media" if ent_type in ["EVENT", "PRODUCT"] else "user"
1146
+ self.esl.add_entity(ent_name, ent_type, claim_id, negated, source_type)
1147
+ # signatures
1148
+ signatures = self.signature_gen.generate_for_claim(claim_id, claim_text)
1149
+ for sig_name, weight in signatures:
1150
+ self.esl.add_signature(claim_id, sig_name, weight)
1151
+ # falsification
1152
+ falsification_results = self.falsifier.run_all(claim_id, claim_text, agent)
1153
+ # entity data for prompt
1154
+ entity_data = {}
1155
+ for ent_name, _, _ in entities:
1156
+ ent = self.esl.entities.get(ent_name)
1157
+ if ent:
1158
+ entity_data[ent_name] = {
1159
+ "type": ent["type"],
1160
+ "first_seen": ent["first_seen"],
1161
+ "last_seen": ent["last_seen"],
1162
+ "coherence": self.esl.get_entity_coherence(ent_name),
1163
+ "suppression_score": ent.get("suppression_score", 0.0)
1164
+ }
1165
+ suppression_pattern = self.esl.suppression_pattern_classifier(claim_id)
1166
+ # multiplexor with random noise hypothesis
1167
+ base_hypotheses = [
1168
+ f"User claim: {claim_text}",
1169
+ "Official narrative accurate",
1170
+ "Suppression detected",
1171
+ "Natural decay",
1172
+ "Random noise",
1173
+ "Preemptive Narrative Control (PNC) active"
1174
+ ]
1175
+ self.multiplexor.initialize(base_hypotheses)
1176
+ evidence_strength = len(signatures) / 5.0
1177
+ coherence = sum(self.esl.get_entity_coherence(e) for e, _, _ in entities) / max(1, len(entities))
1178
+ signature_names = [s[0] for s in signatures]
1179
+ self.multiplexor.update(evidence_strength, signature_names, coherence)
1180
+ multiplexor_probs = self.multiplexor.get_probabilities()
1181
+ user_prob = multiplexor_probs.get(f"User claim: {claim_text}", 0.0)
1182
+
1183
+ # LLM optional
1184
+ llm_output = None
1185
+ if self.api_key:
1186
+ prompt = self._build_prompt(claim_id, claim_text, falsification_results,
1187
+ entity_data, suppression_pattern, multiplexor_probs)
1188
+ headers = {"Content-Type": "application/json", "Authorization": f"Bearer {self.api_key}"}
1189
+ payload = {"model": self.model, "messages": [{"role": "user", "content": prompt}], "temperature": 0.2}
1190
+ for attempt in range(self.max_retries + 1):
1191
+ try:
1192
+ resp = requests.post(self.api_url, headers=headers, json=payload, timeout=30)
1193
+ if resp.status_code != 200:
1194
+ raise Exception(f"API error: {resp.text}")
1195
+ result = resp.json()
1196
+ content = result["choices"][0]["message"]["content"]
1197
+ output = self._parse_output(content)
1198
+ if output and self._check_constraints(output, claim_text, falsification_results):
1199
+ compliant, n_score, n_reason = self.narrative_detector.check(content, claim_text)
1200
+ if compliant:
1201
+ llm_output = output
1202
+ break
1203
+ except Exception:
1204
+ time.sleep(1)
1205
+
1206
+ survival_score = sum(1 for t in falsification_results if t["survived"]) / len(falsification_results)
1207
+ final_confidence = user_prob * survival_score
1208
+ if final_confidence > 0.7:
1209
+ verdict = "Verified"
1210
+ elif final_confidence > 0.4:
1211
+ verdict = "Unverified"
1212
+ elif survival_score < 0.3:
1213
+ verdict = "Refuted"
1214
+ else:
1215
+ verdict = "Insufficient Data"
1216
+
1217
+ self.esl.decay_confidence(half_life_days=30)
1218
+ self.esl.create_block()
1219
+ trend = self.esl.get_suppression_trend(window_days=30)
1220
+ entity_analytics = [self.esl.get_entity_suppression(e) for e, _, _ in entities]
1221
+
1222
+ result_dict = {
1223
+ "claim_id": claim_id,
1224
+ "verdict": verdict,
1225
+ "confidence": final_confidence,
1226
+ "falsification": falsification_results,
1227
+ "suppression_pattern": suppression_pattern,
1228
+ "multiplexor_probabilities": multiplexor_probs,
1229
+ "suppression_trend": trend,
1230
+ "entity_analytics": entity_analytics,
1231
+ "narrative_compliance": True,
1232
+ "coordination_likelihood": self.esl.claims[claim_id].get("coordination_likelihood", 0.0)
1233
+ }
1234
+ if llm_output:
1235
+ result_dict["llm_verdict"] = llm_output["verdict"]
1236
+ result_dict["llm_confidence"] = llm_output["confidence"]
1237
+ result_dict["reasoning"] = llm_output["reasoning"]
1238
+ else:
1239
+ result_dict["reasoning"] = "LLM not used or failed constraints; verdict based on EIS multiplexor."
1240
+ return result_dict
1241
+
1242
+ # ----------------------------------------------------------------------------
1243
+ # OUTPUT FORMATTER
1244
+ # ----------------------------------------------------------------------------
1245
+ def format_report(result: Dict) -> str:
1246
+ lines = []
1247
+ lines.append("**Falsification Results**")
1248
+ for test in result["falsification"]:
1249
+ emoji = "✅" if test["survived"] else "❌"
1250
+ lines.append(f"- {test['name']}: {emoji} – {test['reason']}")
1251
+ lines.append("\n**Hypothesis Probabilities**")
1252
+ lines.append("| Hypothesis | Probability |")
1253
+ lines.append("|------------|-------------|")
1254
+ for h, p in sorted(result["multiplexor_probabilities"].items(), key=lambda x: -x[1]):
1255
+ lines.append(f"| {h} | {p:.0%} |")
1256
+ lines.append(f"\n**Final Confidence:** {result['confidence']:.2f}")
1257
+ lines.append(f"**Verdict:** {result['verdict']}")
1258
+ lines.append(f"**Coordination Likelihood:** {result.get('coordination_likelihood', 0.0):.2f}")
1259
+
1260
+ sp = result["suppression_pattern"]
1261
+ lens_names = [get_lens_name(lid) for lid in sp.get("lenses", [])]
1262
+ lines.append(f"\n**Suppression Pattern:** level={sp['level']}, score={sp['score']:.2f}")
1263
+ if lens_names:
1264
+ lines.append(f" - Lenses: {', '.join(lens_names[:5])}" + (" …" if len(lens_names)>5 else ""))
1265
+ if sp.get("primitives"):
1266
+ lines.append(f" - Primitives: {', '.join(sp['primitives'])}")
1267
+ if sp.get("contributions"):
1268
+ lines.append(" - Signature contributions:")
1269
+ for sig, w in sorted(sp["contributions"].items(), key=lambda x: -x[1]):
1270
+ lines.append(f" {sig}: {w:.2f}")
1271
+
1272
+ trend = result.get("suppression_trend", [])
1273
+ if trend:
1274
+ lines.append("\n**Suppression Trend (last 30 days)**")
1275
+ for point in trend[-7:]:
1276
+ lines.append(f" - {point['date']}: {point['avg_suppression']:.2f}")
1277
+
1278
+ entity_analytics = result.get("entity_analytics", [])
1279
+ if entity_analytics:
1280
+ lines.append("\n**Entity Suppression Analytics**")
1281
+ for ent in entity_analytics:
1282
+ src_str = ", ".join([f"{k}:{v}" for k,v in ent.get("source_types", {}).items()]) if ent.get("source_types") else "unknown"
1283
+ lines.append(f" - {ent['name']} ({ent['type']}): score={ent['score']:.2f}, coherence={ent['coherence']:.2f}, appearances={ent['appearance_count']}, negated={ent.get('negated_count',0)}, sources={src_str}")
1284
+
1285
+ if "llm_verdict" in result:
1286
+ lines.append(f"\n*LLM raw verdict: {result['llm_verdict']} (confidence {result['llm_confidence']:.2f})*")
1287
+ return "\n".join(lines)
1288
+
1289
+ # ----------------------------------------------------------------------------
1290
+ # MAIN (runnable entry point)
1291
+ # ----------------------------------------------------------------------------
1292
+ def main():
1293
+ print("EIS + ESL + PNC + CEC v6.1 – Full Epistemic Substrate (with fixes)")
1294
+ print("=" * 80)
1295
+ esl = ESLedger()
1296
+ llm = ConstrainedLLM(esl, api_key=os.environ.get("OPENAI_API_KEY"), model="gpt-4")
1297
+
1298
+ print("\nEnter a claim (or 'quit'):")
1299
+ while True:
1300
+ claim = input("> ").strip()
1301
+ if claim.lower() in ("quit", "exit"):
1302
+ break
1303
+ if not claim:
1304
+ continue
1305
+ print("Processing claim...")
1306
+ result = llm.query(claim)
1307
+ print("\n" + format_report(result))
1308
+ print("-" * 80)
1309
+
1310
+ if __name__ == "__main__":
1311
+ main()
EIS_ESL_PNC_CEC_INFMOD.txt ADDED
@@ -0,0 +1,209 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """
3
+ Intent Inference Module (INFMOD) for EIS/ESL/PNC/CEC v6
4
+ ========================================================
5
+ This module performs *hypothesis-level* intent inference
6
+ based on structural signals from the ESLedger.
7
+
8
+ It does NOT:
9
+ - assert intent
10
+ - assert agency
11
+ - assert truth
12
+
13
+ It DOES:
14
+ - generate competing hypotheses
15
+ - attach explicit evidence
16
+ - propagate uncertainty
17
+ - maintain epistemic separation
18
+ """
19
+
20
+ from dataclasses import dataclass
21
+ from typing import List, Dict, Optional
22
+ from datetime import datetime
23
+
24
+ # ------------------------------------------------------------
25
+ # INTENT HYPOTHESIS DATA MODEL
26
+ # ------------------------------------------------------------
27
+
28
+ @dataclass
29
+ class IntentHypothesis:
30
+ agent: str # e.g., "institutional", "network", "emergent", "unknown"
31
+ incentive: str # e.g., "reputation_protection", "narrative_control"
32
+ causal_graph: Dict # minimal DAG of event relationships
33
+ probability: float # 0–1 weight, not a verdict
34
+ uncertainty_factors: List[str] # explicit epistemic humility
35
+ evidence: List[str] # concrete observations supporting the hypothesis
36
+
37
+
38
+ # ------------------------------------------------------------
39
+ # INTENT INFERENCE ENGINE
40
+ # ------------------------------------------------------------
41
+
42
+ class IntentInferenceEngine:
43
+ def __init__(self, structural_layer):
44
+ """
45
+ structural_layer: an ESLedger instance or compatible interface
46
+ """
47
+ self.structural = structural_layer
48
+
49
+ # --------------------------------------------------------
50
+ # MAIN ENTRY POINT
51
+ # --------------------------------------------------------
52
+ def infer(self, target: str) -> List[IntentHypothesis]:
53
+ """
54
+ Generate competing intent hypotheses for a given entity or term.
55
+ Returns a list of IntentHypothesis objects.
56
+ """
57
+
58
+ metrics = self._gather_structural_metrics(target)
59
+ incentive_models = self._build_incentive_models(metrics)
60
+ hypotheses = self._generate_hypotheses(metrics, incentive_models)
61
+
62
+ return hypotheses
63
+
64
+ # --------------------------------------------------------
65
+ # STEP 1 — STRUCTURAL METRIC EXTRACTION
66
+ # --------------------------------------------------------
67
+ def _gather_structural_metrics(self, target: str) -> Dict:
68
+ """
69
+ Pulls structural signals from the ESLedger.
70
+ These are *signals*, not interpretations.
71
+ """
72
+
73
+ metrics = {
74
+ "suppression_score": self.structural.get_entity_suppression(target),
75
+ "coordination_likelihood": self._avg_claim_field(target, "coordination_likelihood"),
76
+ "negation_density": self._negation_density(target),
77
+ "temporal_pattern": self._temporal_pattern(target),
78
+ "entity_presence": self._entity_presence(target),
79
+ }
80
+
81
+ return metrics
82
+
83
+ def _avg_claim_field(self, target: str, field: str) -> float:
84
+ vals = []
85
+ for cid, claim in self.structural.claims.items():
86
+ if target.lower() in claim["text"].lower():
87
+ vals.append(claim.get(field, 0.0))
88
+ return sum(vals) / len(vals) if vals else 0.0
89
+
90
+ def _negation_density(self, target: str) -> float:
91
+ neg = 0
92
+ total = 0
93
+ for cid, claim in self.structural.claims.items():
94
+ if target.lower() in claim["text"].lower():
95
+ total += 1
96
+ if any(n in claim["text"].lower() for n in ["not", "never", "no "]):
97
+ neg += 1
98
+ return neg / total if total else 0.0
99
+
100
+ def _temporal_pattern(self, target: str) -> Dict:
101
+ timestamps = []
102
+ for cid, claim in self.structural.claims.items():
103
+ if target.lower() in claim["text"].lower():
104
+ try:
105
+ timestamps.append(datetime.fromisoformat(claim["timestamp"].replace("Z", "+00:00")))
106
+ except:
107
+ pass
108
+ timestamps.sort()
109
+ return {"count": len(timestamps), "first": timestamps[0] if timestamps else None}
110
+
111
+ def _entity_presence(self, target: str) -> int:
112
+ return sum(1 for cid, claim in self.structural.claims.items() if target.lower() in claim["text"].lower())
113
+
114
+ # --------------------------------------------------------
115
+ # STEP 2 — INCENTIVE MODELING
116
+ # --------------------------------------------------------
117
+ def _build_incentive_models(self, metrics: Dict) -> List[Dict]:
118
+ """
119
+ Creates abstract incentive models based on structural signals.
120
+ These are NOT intent claims — they are interpretive scaffolds.
121
+ """
122
+
123
+ models = []
124
+
125
+ # Institutional incentive model
126
+ if metrics["suppression_score"] > 0.4 or metrics["coordination_likelihood"] > 0.5:
127
+ models.append({
128
+ "agent": "institutional",
129
+ "incentive": "narrative_control",
130
+ "weight": 0.4 + metrics["coordination_likelihood"] * 0.3,
131
+ "uncertainties": ["no direct evidence of agency"],
132
+ })
133
+
134
+ # Network incentive model
135
+ if metrics["coordination_likelihood"] > 0.3:
136
+ models.append({
137
+ "agent": "network",
138
+ "incentive": "signal_amplification",
139
+ "weight": 0.3 + metrics["coordination_likelihood"] * 0.2,
140
+ "uncertainties": ["coordination may be emergent"],
141
+ })
142
+
143
+ # Emergent systemic model
144
+ models.append({
145
+ "agent": "emergent",
146
+ "incentive": "incentive_alignment",
147
+ "weight": 0.2 + metrics["negation_density"] * 0.2,
148
+ "uncertainties": ["emergent patterns mimic intent"],
149
+ })
150
+
151
+ # Unknown agent model
152
+ models.append({
153
+ "agent": "unknown",
154
+ "incentive": "unclear",
155
+ "weight": 0.1,
156
+ "uncertainties": ["insufficient structural signal"],
157
+ })
158
+
159
+ return models
160
+
161
+ # --------------------------------------------------------
162
+ # STEP 3 — HYPOTHESIS GENERATION
163
+ # --------------------------------------------------------
164
+ def _generate_hypotheses(self, metrics: Dict, models: List[Dict]) -> List[IntentHypothesis]:
165
+ hypotheses = []
166
+
167
+ for model in models:
168
+ evidence = self._collect_evidence(metrics, model)
169
+
170
+ hypotheses.append(
171
+ IntentHypothesis(
172
+ agent=model["agent"],
173
+ incentive=model["incentive"],
174
+ causal_graph=self._build_causal_graph(metrics),
175
+ probability=min(1.0, model["weight"]),
176
+ uncertainty_factors=model["uncertainties"],
177
+ evidence=evidence
178
+ )
179
+ )
180
+
181
+ return hypotheses
182
+
183
+ def _collect_evidence(self, metrics: Dict, model: Dict) -> List[str]:
184
+ evidence = []
185
+
186
+ if metrics["suppression_score"] > 0.4:
187
+ evidence.append(f"High suppression_score: {metrics['suppression_score']:.2f}")
188
+
189
+ if metrics["coordination_likelihood"] > 0.3:
190
+ evidence.append(f"Elevated coordination_likelihood: {metrics['coordination_likelihood']:.2f}")
191
+
192
+ if metrics["negation_density"] > 0.2:
193
+ evidence.append(f"Negation density suggests contested narrative: {metrics['negation_density']:.2f}")
194
+
195
+ if metrics["entity_presence"] > 10:
196
+ evidence.append(f"High entity presence: {metrics['entity_presence']} mentions")
197
+
198
+ return evidence or ["No strong evidence — hypothesis weak"]
199
+
200
+ def _build_causal_graph(self, metrics: Dict) -> Dict:
201
+ """
202
+ Minimal DAG: structural signals → incentive model
203
+ """
204
+ return {
205
+ "suppression_score": metrics["suppression_score"],
206
+ "coordination_likelihood": metrics["coordination_likelihood"],
207
+ "negation_density": metrics["negation_density"],
208
+ "leads_to": "incentive_hypothesis"
209
+ }