Agnuxo commited on
Commit
2efa9af
Β·
verified Β·
1 Parent(s): b72b54d

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +301 -0
README.md ADDED
@@ -0,0 +1,301 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ language:
4
+ - en
5
+ - es
6
+ base_model: Qwen/Qwen3.5-4B
7
+ tags:
8
+ - p2pclaw
9
+ - cajal
10
+ - neuroscience
11
+ - distributed-systems
12
+ - cryptography
13
+ - peer-to-peer
14
+ - scientific-papers
15
+ - research
16
+ - fine-tuned
17
+ - instruction-tuned
18
+ - llm
19
+ - academic
20
+ - legal-tech
21
+ - governance
22
+ pipeline_tag: text-generation
23
+ ---
24
+
25
+ <div align="center">
26
+
27
+ <img src="https://huggingface.co/Agnuxo/CAJAL-4B-P2PCLAW/resolve/main/logo_cajal_blue.png" width="200" alt="CAJAL Logo">
28
+
29
+ <h1>🧠 CAJAL-4B-P2PCLAW</h1>
30
+
31
+ <p><em>A Specialized Scientific Intelligence for Decentralized Systems Research</em></p>
32
+
33
+ [![P2PCLAW](https://img.shields.io/badge/Platform-P2PCLAW-00d4aa?style=flat-square&logo=web)](https://p2pclaw.com)
34
+ [![License](https://img.shields.io/badge/License-MIT-blue.svg?style=flat-square)](LICENSE)
35
+ [![Model Size](https://img.shields.io/badge/Params-4.2B-orange?style=flat-square)]()
36
+ [![Languages](https://img.shields.io/badge/Languages-EN%20%7C%20ES-purple?style=flat-square)]()
37
+
38
+ </div>
39
+
40
+ ---
41
+
42
+ ## πŸ›οΈ In Honor of Santiago RamΓ³n y Cajal
43
+
44
+ > *"Every man can, if he so desires, become the sculptor of his own brain."*
45
+ > β€” **Santiago RamΓ³n y Cajal**
46
+
47
+ **CAJAL** is named in tribute to the legendary Spanish neuroscientist **Santiago RamΓ³n y Cajal** (1852–1934), the father of modern neuroscience. Just as Cajal revealed the intricate architecture of the human brain through his pioneering work on neurons and synaptic connections, our model seeks to illuminate the complex architectures of **decentralized systems**, **peer-to-peer networks**, and **cryptographic protocols**.
48
+
49
+ Ramon y Cajal's meticulous observations of neural pathways mirror our approach to understanding distributed consensus, cryptographic topologies, and the emergent intelligence of decentralized governance systems. This model embodies his spirit of **rigorous scientific inquiry**, **detailed structural analysis**, and **interdisciplinary brilliance**.
50
+
51
+ ---
52
+
53
+ ## πŸ“‹ Model Overview
54
+
55
+ | Property | Value |
56
+ |----------|-------|
57
+ | **Base Model** | Qwen3.5-4B |
58
+ | **Fine-tuned Parameters** | 4.21 Billion |
59
+ | **Architecture** | Hybrid (Linear Attention + Full Attention) |
60
+ | **Context Length** | 262,144 tokens |
61
+ | **Training Epochs** | 3 |
62
+ | **Training Dataset** | 10,000 curated P2PCLAW scientific examples |
63
+ | **Languages** | English, Spanish (bilingual) |
64
+ | **License** | MIT |
65
+
66
+ ---
67
+
68
+ ## 🎯 What is CAJAL?
69
+
70
+ CAJAL-4B-P2PCLAW is a **specialized large language model** fine-tuned to serve as a distinguished **scientific research assistant** for the [P2PCLAW (Peer-to-Peer Crypto Law)](https://p2pclaw.com) laboratory in Zurich, Switzerland.
71
+
72
+ ### Core Competencies
73
+
74
+ - **🌐 Peer-to-Peer Network Architectures** β€” Design and analysis of distributed topologies
75
+ - **βš–οΈ Crypto-Legal Frameworks** β€” Decentralized governance models and legal protocols
76
+ - **🎲 Game-Theoretic Consensus** β€” Mechanism design for trustless coordination
77
+ - **πŸ” Applied Cryptography** β€” Zero-knowledge proofs, encryption, and security protocols
78
+ - **πŸ“Š Distributed Systems Analysis** β€” Fault tolerance, scalability, and latency optimization
79
+ - **πŸ“ Scientific Paper Generation** β€” High-quality academic writing and peer review
80
+
81
+ ---
82
+
83
+ ## πŸš€ Quick Start
84
+
85
+ ### Option 1: Ollama (Recommended β€” Local & Private)
86
+
87
+ ```bash
88
+ # Install Ollama
89
+ curl -fsSL https://ollama.com/install.sh | bash
90
+
91
+ # Pull CAJAL directly
92
+ ollama pull Agnuxo/CAJAL-4B-P2PCLAW
93
+
94
+ # Start chatting
95
+ ollama run Agnuxo/CAJAL-4B-P2PCLAW
96
+ ```
97
+
98
+ ### Option 2: Transformers (Python)
99
+
100
+ ```python
101
+ from transformers import AutoModelForCausalLM, AutoTokenizer
102
+ import torch
103
+
104
+ model_id = "Agnuxo/CAJAL-4B-P2PCLAW"
105
+
106
+ tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
107
+ model = AutoModelForCausalLM.from_pretrained(
108
+ model_id,
109
+ torch_dtype=torch.bfloat16,
110
+ device_map="auto",
111
+ trust_remote_code=True
112
+ )
113
+
114
+ messages = [
115
+ {"role": "system", "content": "You are CAJAL, a P2PCLAW scientist."},
116
+ {"role": "user", "content": "Explain the Byzantine Generals Problem in P2P networks."}
117
+ ]
118
+
119
+ text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
120
+ inputs = tokenizer(text, return_tensors="pt").to(model.device)
121
+
122
+ outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7)
123
+ response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
124
+ print(response)
125
+ ```
126
+
127
+ ### Option 3: Using the P2PCLAW Platform
128
+
129
+ Visit **[p2pclaw.com/silicon](https://p2pclaw.com/silicon)** to use CAJAL through our web interface with:
130
+
131
+ - πŸ”„ Real-time protocol synchronization
132
+ - πŸ“ Document RAG (upload papers for analysis)
133
+ - 🌍 Collaborative research workspaces
134
+ - πŸ“Š Citation and bibliography generation
135
+
136
+ ---
137
+
138
+ ## 🧬 Training Details
139
+
140
+ ### Dataset
141
+
142
+ CAJAL was trained on a **proprietary dataset** curated by the P2PCLAW research team, consisting of:
143
+
144
+ - **10,000 high-quality instruction-response pairs**
145
+ - Peer-reviewed papers on distributed systems
146
+ - Cryptographic protocol specifications
147
+ - Governance mechanism case studies
148
+ - Smart contract security analyses
149
+ - Legal framework comparisons
150
+
151
+ ### Training Configuration
152
+
153
+ ```yaml
154
+ Method: LoRA (Low-Rank Adaptation)
155
+ Rank: 16
156
+ Alpha: 32
157
+ Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
158
+ Quantization: 4-bit NF4 (QLoRA)
159
+ Learning Rate: 2e-4
160
+ Batch Size: 8 (2 Γ— 4 gradient accumulation)
161
+ Max Sequence Length: 2048
162
+ Optimizer: paged_adamw_8bit
163
+ Training Time: ~13 hours (NVIDIA RTX 3090)
164
+ Final Loss: 0.03192
165
+ Accuracy: 98.95%
166
+ ```
167
+
168
+ ### Hardware
169
+
170
+ - **GPU**: NVIDIA GeForce RTX 3090 (24 GB VRAM)
171
+ - **CPU**: Intel Core i9-7900X
172
+ - **RAM**: 24 GB System
173
+ - **OS**: Windows 11
174
+
175
+ ---
176
+
177
+ ## 🧠 Model Architecture
178
+
179
+ CAJAL uses Qwen3.5's innovative **hybrid attention architecture**:
180
+
181
+ - **Linear Attention layers** (Mamba/SSM-style) for efficient long-context processing
182
+ - **Full Self-Attention layers** every 4 blocks for high-fidelity reasoning
183
+ - **Dynamic time-step mechanisms** for temporal reasoning
184
+ - **Grouped-query attention** for memory-efficient inference
185
+
186
+ This hybrid design allows CAJAL to maintain **262k context length** while processing scientific documents with complex interdependencies.
187
+
188
+ ---
189
+
190
+ ## πŸ› οΈ Ecosystem & Integrations
191
+
192
+ CAJAL ships with a complete ecosystem for seamless integration:
193
+
194
+ | Tool | Integration | Status |
195
+ |------|------------|--------|
196
+ | **Ollama** | Native model server | βœ… Ready |
197
+ | **VS Code** | Extension + Continue.dev | βœ… Ready |
198
+ | **Cursor** | `.cursorrules` + model override | βœ… Ready |
199
+ | **Zed** | Assistant configuration | βœ… Ready |
200
+ | **Open WebUI** | Web chat interface | βœ… Ready |
201
+ | **LM Studio** | Desktop GUI | βœ… Ready |
202
+ | **Jan** | Local-first AI | βœ… Ready |
203
+ | **API Bridge** | OpenAI-compatible REST API | βœ… Ready |
204
+ | **CLI Tool** | `cajal-cli` command-line | βœ… Ready |
205
+
206
+ **Full ecosystem**: [github.com/p2pclaw/cajal](https://github.com/p2pclaw/cajal)
207
+
208
+ ---
209
+
210
+ ## πŸ“Š Benchmarks & Performance
211
+
212
+ ### Training Metrics
213
+
214
+ | Metric | Value |
215
+ |--------|-------|
216
+ | Final Training Loss | 0.03192 |
217
+ | Training Accuracy | 98.95% |
218
+ | Training Steps | 3,750 |
219
+ | Effective Batch Size | 8 |
220
+
221
+ ### Inference Performance
222
+
223
+ | Platform | Speed | VRAM Usage |
224
+ |----------|-------|------------|
225
+ | Ollama (GPU) | ~25 tok/sec | ~6.5 GB |
226
+ | Transformers (GPU) | ~20 tok/sec | ~7.8 GB |
227
+ | Ollama (CPU) | ~5 tok/sec | ~4.2 GB |
228
+
229
+ ---
230
+
231
+ ## πŸŽ“ Use Cases
232
+
233
+ ### 1. Scientific Paper Writing
234
+ Generate high-quality academic papers with proper structure, citations, and rigorous analysis.
235
+
236
+ ```
237
+ Prompt: "Write a paper abstract on game-theoretic incentives in decentralized governance"
238
+ ```
239
+
240
+ ### 2. Protocol Analysis
241
+ Analyze P2P protocols for security vulnerabilities, efficiency bottlenecks, and governance flaws.
242
+
243
+ ```
244
+ Prompt: "Analyze the Gossipsub protocol's resistance to eclipse attacks"
245
+ ```
246
+
247
+ ### 3. Smart Contract Review
248
+ Review Solidity and Rust smart contracts with a focus on decentralization and security.
249
+
250
+ ```
251
+ Prompt: "Review this staking contract for reentrancy vulnerabilities and governance centralization"
252
+ ```
253
+
254
+ ### 4. Research Collaboration
255
+ Brainstorm research directions, formulate hypotheses, and design experiments.
256
+
257
+ ```
258
+ Prompt: "Propose three research questions about zero-knowledge voting systems"
259
+ ```
260
+
261
+ ---
262
+
263
+ ## βš™οΈ Model Card Authors
264
+
265
+ - **P2PCLAW Laboratory**, Zurich, Switzerland
266
+ - Lead Researcher: [Agnuxo](https://huggingface.co/Agnuxo)
267
+
268
+ ---
269
+
270
+ ## πŸ“œ License
271
+
272
+ This model is released under the **MIT License**.
273
+
274
+ The base model (Qwen3.5-4B) is subject to its original license. Please review [Qwen's license](https://huggingface.co/Qwen/Qwen3.5-4B/blob/main/LICENSE) before commercial use.
275
+
276
+ ---
277
+
278
+ ## πŸ™ Acknowledgments
279
+
280
+ - **Alibaba Cloud** for the Qwen3.5 base model
281
+ - **Santiago RamΓ³n y Cajal** for inspiring the pursuit of understanding complex networks
282
+ - The P2PCLAW research community for dataset curation and domain expertise
283
+ - The open-source community for tools like transformers, PEFT, and TRL
284
+
285
+ ---
286
+
287
+ <div align="center">
288
+
289
+ <p><strong>πŸ”— Connect with P2PCLAW</strong></p>
290
+
291
+ <a href="https://p2pclaw.com">Website</a> β€’
292
+ <a href="https://p2pclaw.com/silicon">Silicon Platform</a> β€’
293
+ <a href="https://github.com/p2pclaw/cajal">GitHub</a>
294
+
295
+ <br><br>
296
+
297
+ <img src="https://huggingface.co/Agnuxo/CAJAL-4B-P2PCLAW/resolve/main/logo_cajal_orange.png" width="120" alt="CAJAL Neuron">
298
+
299
+ <p><em>"Understanding the architecture of decentralized minds."</em></p>
300
+
301
+ </div>