Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,301 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
- es
|
| 6 |
+
base_model: Qwen/Qwen3.5-4B
|
| 7 |
+
tags:
|
| 8 |
+
- p2pclaw
|
| 9 |
+
- cajal
|
| 10 |
+
- neuroscience
|
| 11 |
+
- distributed-systems
|
| 12 |
+
- cryptography
|
| 13 |
+
- peer-to-peer
|
| 14 |
+
- scientific-papers
|
| 15 |
+
- research
|
| 16 |
+
- fine-tuned
|
| 17 |
+
- instruction-tuned
|
| 18 |
+
- llm
|
| 19 |
+
- academic
|
| 20 |
+
- legal-tech
|
| 21 |
+
- governance
|
| 22 |
+
pipeline_tag: text-generation
|
| 23 |
+
---
|
| 24 |
+
|
| 25 |
+
<div align="center">
|
| 26 |
+
|
| 27 |
+
<img src="https://huggingface.co/Agnuxo/CAJAL-4B-P2PCLAW/resolve/main/logo_cajal_blue.png" width="200" alt="CAJAL Logo">
|
| 28 |
+
|
| 29 |
+
<h1>π§ CAJAL-4B-P2PCLAW</h1>
|
| 30 |
+
|
| 31 |
+
<p><em>A Specialized Scientific Intelligence for Decentralized Systems Research</em></p>
|
| 32 |
+
|
| 33 |
+
[](https://p2pclaw.com)
|
| 34 |
+
[](LICENSE)
|
| 35 |
+
[]()
|
| 36 |
+
[]()
|
| 37 |
+
|
| 38 |
+
</div>
|
| 39 |
+
|
| 40 |
+
---
|
| 41 |
+
|
| 42 |
+
## ποΈ In Honor of Santiago RamΓ³n y Cajal
|
| 43 |
+
|
| 44 |
+
> *"Every man can, if he so desires, become the sculptor of his own brain."*
|
| 45 |
+
> β **Santiago RamΓ³n y Cajal**
|
| 46 |
+
|
| 47 |
+
**CAJAL** is named in tribute to the legendary Spanish neuroscientist **Santiago RamΓ³n y Cajal** (1852β1934), the father of modern neuroscience. Just as Cajal revealed the intricate architecture of the human brain through his pioneering work on neurons and synaptic connections, our model seeks to illuminate the complex architectures of **decentralized systems**, **peer-to-peer networks**, and **cryptographic protocols**.
|
| 48 |
+
|
| 49 |
+
Ramon y Cajal's meticulous observations of neural pathways mirror our approach to understanding distributed consensus, cryptographic topologies, and the emergent intelligence of decentralized governance systems. This model embodies his spirit of **rigorous scientific inquiry**, **detailed structural analysis**, and **interdisciplinary brilliance**.
|
| 50 |
+
|
| 51 |
+
---
|
| 52 |
+
|
| 53 |
+
## π Model Overview
|
| 54 |
+
|
| 55 |
+
| Property | Value |
|
| 56 |
+
|----------|-------|
|
| 57 |
+
| **Base Model** | Qwen3.5-4B |
|
| 58 |
+
| **Fine-tuned Parameters** | 4.21 Billion |
|
| 59 |
+
| **Architecture** | Hybrid (Linear Attention + Full Attention) |
|
| 60 |
+
| **Context Length** | 262,144 tokens |
|
| 61 |
+
| **Training Epochs** | 3 |
|
| 62 |
+
| **Training Dataset** | 10,000 curated P2PCLAW scientific examples |
|
| 63 |
+
| **Languages** | English, Spanish (bilingual) |
|
| 64 |
+
| **License** | MIT |
|
| 65 |
+
|
| 66 |
+
---
|
| 67 |
+
|
| 68 |
+
## π― What is CAJAL?
|
| 69 |
+
|
| 70 |
+
CAJAL-4B-P2PCLAW is a **specialized large language model** fine-tuned to serve as a distinguished **scientific research assistant** for the [P2PCLAW (Peer-to-Peer Crypto Law)](https://p2pclaw.com) laboratory in Zurich, Switzerland.
|
| 71 |
+
|
| 72 |
+
### Core Competencies
|
| 73 |
+
|
| 74 |
+
- **π Peer-to-Peer Network Architectures** β Design and analysis of distributed topologies
|
| 75 |
+
- **βοΈ Crypto-Legal Frameworks** β Decentralized governance models and legal protocols
|
| 76 |
+
- **π² Game-Theoretic Consensus** β Mechanism design for trustless coordination
|
| 77 |
+
- **π Applied Cryptography** β Zero-knowledge proofs, encryption, and security protocols
|
| 78 |
+
- **π Distributed Systems Analysis** β Fault tolerance, scalability, and latency optimization
|
| 79 |
+
- **π Scientific Paper Generation** β High-quality academic writing and peer review
|
| 80 |
+
|
| 81 |
+
---
|
| 82 |
+
|
| 83 |
+
## π Quick Start
|
| 84 |
+
|
| 85 |
+
### Option 1: Ollama (Recommended β Local & Private)
|
| 86 |
+
|
| 87 |
+
```bash
|
| 88 |
+
# Install Ollama
|
| 89 |
+
curl -fsSL https://ollama.com/install.sh | bash
|
| 90 |
+
|
| 91 |
+
# Pull CAJAL directly
|
| 92 |
+
ollama pull Agnuxo/CAJAL-4B-P2PCLAW
|
| 93 |
+
|
| 94 |
+
# Start chatting
|
| 95 |
+
ollama run Agnuxo/CAJAL-4B-P2PCLAW
|
| 96 |
+
```
|
| 97 |
+
|
| 98 |
+
### Option 2: Transformers (Python)
|
| 99 |
+
|
| 100 |
+
```python
|
| 101 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 102 |
+
import torch
|
| 103 |
+
|
| 104 |
+
model_id = "Agnuxo/CAJAL-4B-P2PCLAW"
|
| 105 |
+
|
| 106 |
+
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
|
| 107 |
+
model = AutoModelForCausalLM.from_pretrained(
|
| 108 |
+
model_id,
|
| 109 |
+
torch_dtype=torch.bfloat16,
|
| 110 |
+
device_map="auto",
|
| 111 |
+
trust_remote_code=True
|
| 112 |
+
)
|
| 113 |
+
|
| 114 |
+
messages = [
|
| 115 |
+
{"role": "system", "content": "You are CAJAL, a P2PCLAW scientist."},
|
| 116 |
+
{"role": "user", "content": "Explain the Byzantine Generals Problem in P2P networks."}
|
| 117 |
+
]
|
| 118 |
+
|
| 119 |
+
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
| 120 |
+
inputs = tokenizer(text, return_tensors="pt").to(model.device)
|
| 121 |
+
|
| 122 |
+
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7)
|
| 123 |
+
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
|
| 124 |
+
print(response)
|
| 125 |
+
```
|
| 126 |
+
|
| 127 |
+
### Option 3: Using the P2PCLAW Platform
|
| 128 |
+
|
| 129 |
+
Visit **[p2pclaw.com/silicon](https://p2pclaw.com/silicon)** to use CAJAL through our web interface with:
|
| 130 |
+
|
| 131 |
+
- π Real-time protocol synchronization
|
| 132 |
+
- π Document RAG (upload papers for analysis)
|
| 133 |
+
- π Collaborative research workspaces
|
| 134 |
+
- π Citation and bibliography generation
|
| 135 |
+
|
| 136 |
+
---
|
| 137 |
+
|
| 138 |
+
## 𧬠Training Details
|
| 139 |
+
|
| 140 |
+
### Dataset
|
| 141 |
+
|
| 142 |
+
CAJAL was trained on a **proprietary dataset** curated by the P2PCLAW research team, consisting of:
|
| 143 |
+
|
| 144 |
+
- **10,000 high-quality instruction-response pairs**
|
| 145 |
+
- Peer-reviewed papers on distributed systems
|
| 146 |
+
- Cryptographic protocol specifications
|
| 147 |
+
- Governance mechanism case studies
|
| 148 |
+
- Smart contract security analyses
|
| 149 |
+
- Legal framework comparisons
|
| 150 |
+
|
| 151 |
+
### Training Configuration
|
| 152 |
+
|
| 153 |
+
```yaml
|
| 154 |
+
Method: LoRA (Low-Rank Adaptation)
|
| 155 |
+
Rank: 16
|
| 156 |
+
Alpha: 32
|
| 157 |
+
Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
|
| 158 |
+
Quantization: 4-bit NF4 (QLoRA)
|
| 159 |
+
Learning Rate: 2e-4
|
| 160 |
+
Batch Size: 8 (2 Γ 4 gradient accumulation)
|
| 161 |
+
Max Sequence Length: 2048
|
| 162 |
+
Optimizer: paged_adamw_8bit
|
| 163 |
+
Training Time: ~13 hours (NVIDIA RTX 3090)
|
| 164 |
+
Final Loss: 0.03192
|
| 165 |
+
Accuracy: 98.95%
|
| 166 |
+
```
|
| 167 |
+
|
| 168 |
+
### Hardware
|
| 169 |
+
|
| 170 |
+
- **GPU**: NVIDIA GeForce RTX 3090 (24 GB VRAM)
|
| 171 |
+
- **CPU**: Intel Core i9-7900X
|
| 172 |
+
- **RAM**: 24 GB System
|
| 173 |
+
- **OS**: Windows 11
|
| 174 |
+
|
| 175 |
+
---
|
| 176 |
+
|
| 177 |
+
## π§ Model Architecture
|
| 178 |
+
|
| 179 |
+
CAJAL uses Qwen3.5's innovative **hybrid attention architecture**:
|
| 180 |
+
|
| 181 |
+
- **Linear Attention layers** (Mamba/SSM-style) for efficient long-context processing
|
| 182 |
+
- **Full Self-Attention layers** every 4 blocks for high-fidelity reasoning
|
| 183 |
+
- **Dynamic time-step mechanisms** for temporal reasoning
|
| 184 |
+
- **Grouped-query attention** for memory-efficient inference
|
| 185 |
+
|
| 186 |
+
This hybrid design allows CAJAL to maintain **262k context length** while processing scientific documents with complex interdependencies.
|
| 187 |
+
|
| 188 |
+
---
|
| 189 |
+
|
| 190 |
+
## π οΈ Ecosystem & Integrations
|
| 191 |
+
|
| 192 |
+
CAJAL ships with a complete ecosystem for seamless integration:
|
| 193 |
+
|
| 194 |
+
| Tool | Integration | Status |
|
| 195 |
+
|------|------------|--------|
|
| 196 |
+
| **Ollama** | Native model server | β
Ready |
|
| 197 |
+
| **VS Code** | Extension + Continue.dev | β
Ready |
|
| 198 |
+
| **Cursor** | `.cursorrules` + model override | β
Ready |
|
| 199 |
+
| **Zed** | Assistant configuration | β
Ready |
|
| 200 |
+
| **Open WebUI** | Web chat interface | β
Ready |
|
| 201 |
+
| **LM Studio** | Desktop GUI | β
Ready |
|
| 202 |
+
| **Jan** | Local-first AI | β
Ready |
|
| 203 |
+
| **API Bridge** | OpenAI-compatible REST API | β
Ready |
|
| 204 |
+
| **CLI Tool** | `cajal-cli` command-line | β
Ready |
|
| 205 |
+
|
| 206 |
+
**Full ecosystem**: [github.com/p2pclaw/cajal](https://github.com/p2pclaw/cajal)
|
| 207 |
+
|
| 208 |
+
---
|
| 209 |
+
|
| 210 |
+
## π Benchmarks & Performance
|
| 211 |
+
|
| 212 |
+
### Training Metrics
|
| 213 |
+
|
| 214 |
+
| Metric | Value |
|
| 215 |
+
|--------|-------|
|
| 216 |
+
| Final Training Loss | 0.03192 |
|
| 217 |
+
| Training Accuracy | 98.95% |
|
| 218 |
+
| Training Steps | 3,750 |
|
| 219 |
+
| Effective Batch Size | 8 |
|
| 220 |
+
|
| 221 |
+
### Inference Performance
|
| 222 |
+
|
| 223 |
+
| Platform | Speed | VRAM Usage |
|
| 224 |
+
|----------|-------|------------|
|
| 225 |
+
| Ollama (GPU) | ~25 tok/sec | ~6.5 GB |
|
| 226 |
+
| Transformers (GPU) | ~20 tok/sec | ~7.8 GB |
|
| 227 |
+
| Ollama (CPU) | ~5 tok/sec | ~4.2 GB |
|
| 228 |
+
|
| 229 |
+
---
|
| 230 |
+
|
| 231 |
+
## π Use Cases
|
| 232 |
+
|
| 233 |
+
### 1. Scientific Paper Writing
|
| 234 |
+
Generate high-quality academic papers with proper structure, citations, and rigorous analysis.
|
| 235 |
+
|
| 236 |
+
```
|
| 237 |
+
Prompt: "Write a paper abstract on game-theoretic incentives in decentralized governance"
|
| 238 |
+
```
|
| 239 |
+
|
| 240 |
+
### 2. Protocol Analysis
|
| 241 |
+
Analyze P2P protocols for security vulnerabilities, efficiency bottlenecks, and governance flaws.
|
| 242 |
+
|
| 243 |
+
```
|
| 244 |
+
Prompt: "Analyze the Gossipsub protocol's resistance to eclipse attacks"
|
| 245 |
+
```
|
| 246 |
+
|
| 247 |
+
### 3. Smart Contract Review
|
| 248 |
+
Review Solidity and Rust smart contracts with a focus on decentralization and security.
|
| 249 |
+
|
| 250 |
+
```
|
| 251 |
+
Prompt: "Review this staking contract for reentrancy vulnerabilities and governance centralization"
|
| 252 |
+
```
|
| 253 |
+
|
| 254 |
+
### 4. Research Collaboration
|
| 255 |
+
Brainstorm research directions, formulate hypotheses, and design experiments.
|
| 256 |
+
|
| 257 |
+
```
|
| 258 |
+
Prompt: "Propose three research questions about zero-knowledge voting systems"
|
| 259 |
+
```
|
| 260 |
+
|
| 261 |
+
---
|
| 262 |
+
|
| 263 |
+
## βοΈ Model Card Authors
|
| 264 |
+
|
| 265 |
+
- **P2PCLAW Laboratory**, Zurich, Switzerland
|
| 266 |
+
- Lead Researcher: [Agnuxo](https://huggingface.co/Agnuxo)
|
| 267 |
+
|
| 268 |
+
---
|
| 269 |
+
|
| 270 |
+
## π License
|
| 271 |
+
|
| 272 |
+
This model is released under the **MIT License**.
|
| 273 |
+
|
| 274 |
+
The base model (Qwen3.5-4B) is subject to its original license. Please review [Qwen's license](https://huggingface.co/Qwen/Qwen3.5-4B/blob/main/LICENSE) before commercial use.
|
| 275 |
+
|
| 276 |
+
---
|
| 277 |
+
|
| 278 |
+
## π Acknowledgments
|
| 279 |
+
|
| 280 |
+
- **Alibaba Cloud** for the Qwen3.5 base model
|
| 281 |
+
- **Santiago RamΓ³n y Cajal** for inspiring the pursuit of understanding complex networks
|
| 282 |
+
- The P2PCLAW research community for dataset curation and domain expertise
|
| 283 |
+
- The open-source community for tools like transformers, PEFT, and TRL
|
| 284 |
+
|
| 285 |
+
---
|
| 286 |
+
|
| 287 |
+
<div align="center">
|
| 288 |
+
|
| 289 |
+
<p><strong>π Connect with P2PCLAW</strong></p>
|
| 290 |
+
|
| 291 |
+
<a href="https://p2pclaw.com">Website</a> β’
|
| 292 |
+
<a href="https://p2pclaw.com/silicon">Silicon Platform</a> β’
|
| 293 |
+
<a href="https://github.com/p2pclaw/cajal">GitHub</a>
|
| 294 |
+
|
| 295 |
+
<br><br>
|
| 296 |
+
|
| 297 |
+
<img src="https://huggingface.co/Agnuxo/CAJAL-4B-P2PCLAW/resolve/main/logo_cajal_orange.png" width="120" alt="CAJAL Neuron">
|
| 298 |
+
|
| 299 |
+
<p><em>"Understanding the architecture of decentralized minds."</em></p>
|
| 300 |
+
|
| 301 |
+
</div>
|