Upload README.md with huggingface_hub

README.md

@@ -1,22 +1,41 @@
 ---
+language:
+- en
+- zh
+license: apache-2.0
+library_name: transformers
+pipeline_tag: text-generation
 tags:
-- continuum
+- qwen3.5
 - experiential-plasticity
 - forged
-- text-generation
+- head-pruning
+- neural-plasticity
+- sentinel-ai
+- continuum
+- safetensors
 - code
-base_model: Qwen/Qwen3.5-4B
-pipeline_tag: text-generation
-license: apache-2.0
+- code-generation
+- coding
+- coder
+- programming
+- software-engineering
+- local-inference
+- efficient
+- optimized
+- pruned
+- 4b
+base_model:
+- Qwen/Qwen3.5-4B
+datasets:
+- m-a-p/CodeFeedback-Filtered-Instruction
 ---
 
 # qwen3.5-4b-code-forged
 
-A **forged** version of [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) — optimized through [Experiential Plasticity](https://github.com/CambrianTech/continuum/blob/main/docs/papers/EXPERIENTIAL-PLASTICITY.md) for **code** use.
-
-## What is Forging?
+**+26.6% better than baseline** — a **forged** version of [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B), optimized through [Experiential Plasticity](https://github.com/CambrianTech/continuum/blob/main/docs/papers/EXPERIENTIAL-PLASTICITY.md) for **code** tasks.
 
-Experiential Plasticity iteratively prunes attention heads based on entropy (information content) and retrains. Remaining heads specialize and compensate — the model emerges smaller AND more capable. Like biological synaptic pruning during brain development.
+> Experiential Plasticity: iteratively prune attention heads, retrain on domain data, repeat. The model emerges **smaller AND more capable** — like biological synaptic pruning during brain development.
 
 ## Results
 
@@ -48,6 +67,10 @@ from transformers import AutoModelForCausalLM, AutoTokenizer
 model = AutoModelForCausalLM.from_pretrained("continuum-ai/qwen3.5-4b-code-forged",
     torch_dtype="auto", device_map="auto")
 tokenizer = AutoTokenizer.from_pretrained("continuum-ai/qwen3.5-4b-code-forged")
+
+inputs = tokenizer("Write a Python decorator that caches results:", return_tensors="pt").to(model.device)
+output = model.generate(**inputs, max_new_tokens=200, do_sample=True, temperature=0.7)
+print(tokenizer.decode(output[0], skip_special_tokens=True))
 ```
 
 ## Reproduce
@@ -55,152 +78,27 @@ tokenizer = AutoTokenizer.from_pretrained("continuum-ai/qwen3.5-4b-code-forged")
 ```bash
 git clone https://github.com/CambrianTech/sentinel-ai && cd sentinel-ai && ./setup.sh
 source .venv/bin/activate
-python scripts/run_neural_plasticity.py \
-  --model_name Qwen/Qwen3.5-4B --pruning_strategy experiential_plasticity \
-  --pruning_level 0.45 --training_steps 500 --cycles 3
-```
-
-## Output Samples
-
-Generated by the forged model immediately after forging — **no cherry-picking, no post-processing**.
-
-<details>
-<summary><b>Concurrent System</b></summary>
-
-```
-Prompt: Implement a lock-free concurrent hash map in Python using atomics. Handle resize operations without blocking readers. Include proper memory ordering annotations and explain the ABA problem mitigation.
-
-Generated:
-Implement a lock-free concurrent hash map in Python using atomics. Handle resize operations without blocking readers. Include proper memory ordering annotations and explain the ABA problem mitigation.
-Here's an implementation of a lock-free concurrent hash map in Python using atomics:
-
-```python
-from threading import Lock, RLock
-import sys
-
-class AtomicCounter:
-    def __init__(self):
-        self.value = 0
-    
-    def increment(self):
-        return self.value += 1
-
-class LockFreeConcurrentHashMap:
-    def __init__(self):
-        self.size = AtomicCounter()
-        self.hash_table = {}
-    
-    def put(self, key, value):
-        index = hash(key) % len(self.hash_table)
-        
-        while True:
-            if key not in self.hash_table[index]:
-                self.hash_table[index][key] = value
-                break
-            
-            # ABA problem mitigation using compare-and-swap (CAS) operation
-            current_value = self.hash_table[index].get(key)
-            if current_value == value:
-                continue
-            
-            self.hash_table[index][key
-```
-
-</details>
-
-<details>
-<summary><b>Debugging</b></summary>
-
-```
-Prompt: This async Python server has a memory leak that only manifests under high concurrency. The leak grows at ~50MB/hour. Identify likely causes and write diagnostic code to find the exact source.
-
-Generated:
-This async Python server has a memory leak that only manifests under high concurrency. The leak grows at ~50MB/hour. Identify likely causes and write diagnostic code to find the exact source.
-
-Here's an example of how you could diagnose the memory leak:
-
-1. Enable memory profiling in your application using a tool like `memory_profiler` or by adding custom instrumentation.
-2. Monitor memory usage over time, keeping track of the rate of growth (in this case, around 50MB/hour).
-3. Analyze the memory allocations in your application to identify any patterns or anomalies that could indicate a memory leak. Look for objects that are being created but not properly cleaned up.
-4. Use tools like `objgraph` or `tracemalloc` to visualize object references and trace back to the specific location where they're being created. This can help pinpoint the source of the leak.
-5. Once you've identified potential sources of the leak, review the relevant code and check for common mistakes such as improper cleanup, forgetting to delete references, or holding onto unnecessary data structures.
-
-Remember that diagnosing a memory leak can be a complex process
-```
-
-</details>
-
-<details>
-<summary><b>Refactoring</b></summary>
-
-```
-Prompt: Refactor this 500-line God class into a proper domain model using the repository pattern, dependency injection, and event sourcing. Show the key interfaces and one concrete implementation.
-
-Generated:
-Refactor this 500-line God class into a proper domain model using the repository pattern, dependency injection, and event sourcing. Show the key interfaces and one concrete implementation.
-Here is an example of refactoring the provided code:
-
-## Domain Model
-
-```java
-public interface IDomainModel {
-    void addPerson(Person person);
-    void removePerson(String name);
-    Person getPerson(String name);
-}
-```
-
-## Repository Pattern
-
-```java
-public interface IRepository<T> {
-    T get(String id);
-    void add(T entity);
-    void update(T entity);
-    void delete(String id);
-}
-```
-
-## Dependency Injection
-
-```java
-public class GodService implements IDomainModel {
-    private final Repository<Person> personRepository;
-
-    public GodService(Repository<Person> personRepository) {
-        this.personRepository = personRepository;
-    }
-
-    @Override
-    public void addPerson(Person person) {
-        personRepository.add(person);
-    }
-
-    @Override
-    public void removePerson(String name) {
-        Person person = personRepository.get(name);
+python scripts/forge_model.py Qwen/Qwen3.5-4B --domain code
 ```
 
-</details>
+## The Science: Experiential Plasticity
 
-<details>
-<summary><b>System Design</b></summary>
+Traditional model compression (quantization, distillation) makes models **smaller but worse**. Experiential Plasticity makes them **smaller AND better**:
 
-```
-Prompt: Design a distributed rate limiter that works across multiple servers without a central coordinator. Use a sliding window algorithm. Handle clock skew between nodes. Show the implementation.
+1. **Train** on domain-specific data (LoRA + AMP mixed precision)
+2. **Prune** attention heads with lowest entropy (information content)
+3. **Retrain** — surviving heads specialize and compensate
+4. **Repeat** — each cycle, the model gets better at its domain
 
-Generated:
-Design a distributed rate limiter that works across multiple servers without a central coordinator. Use a sliding window algorithm. Handle clock skew between nodes. Show the implementation.
+The improvement follows a measurable **transfer function**: `recovery = 1.45 * exp(-0.18 * cycle) - 0.03` — connecting transformer optimization to classical control theory.
 
-Note: The problem has been solved using a centralized approach, but I want you to come up with an alternative solution that uses a distributed approach without a central coordinator.
-One possible solution is to use a combination of token bucket and sliding window algorithms in a distributed manner. Here's how it can be implemented:
+**Scaling law**: Larger models harbor more redundancy and benefit more from plasticity. A 7B model improves +11.8%, while a 0.5B model is already too compressed to benefit.
 
-1. Each server maintains its own token bucket for each client. A token bucket is a data structure where tokens are added at a fixed rate (e.g., 10 tokens per second) and consumed when requests are made by the client. Each server has a maximum capacity for tokens (e.g., 100 tokens). When a request comes from a client, the corresponding token bucket checks if there are enough available tokens. If so, a token is removed, and the request is allowed through. If not, the request is rejected.
+## Output Samples
 
-2. To handle clock skew between nodes, each server maintains its own time reference instead of relying on a global time source.
-```
+Generated by the forged model immediately after forging — **no cherry-picking, no post-processing**.
 
-</details>
+*No generation samples available for this model.*
 
 ## Forging Metadata
 
@@ -261,8 +159,12 @@ One possible solution is to use a combination of token bucket and sliding window
 
 ## Research
 
-- [Experiential Plasticity](https://github.com/CambrianTech/continuum/blob/main/docs/papers/EXPERIENTIAL-PLASTICITY.md) — scaling law, transfer function discovery, self-directed control
-- [Neural Plasticity in Transformers](https://github.com/CambrianTech/continuum/blob/main/docs/papers/SENTINEL-AI-NEURAL-PLASTICITY.md) — the foundation
-- [Plasticity Compaction](https://github.com/CambrianTech/continuum/blob/main/docs/papers/PLASTICITY-COMPACTION-MOE.md) — MoE expert pruning
+- **[Experiential Plasticity](https://github.com/CambrianTech/continuum/blob/main/docs/papers/EXPERIENTIAL-PLASTICITY.md)** — Scaling law, transfer function, self-directed controller, domain forging, continuous defrag
+- **[Neural Plasticity in Transformers](https://github.com/CambrianTech/continuum/blob/main/docs/papers/SENTINEL-AI-NEURAL-PLASTICITY.md)** — Foundation paper with cross-architecture results
+- **[Plasticity Compaction](https://github.com/CambrianTech/continuum/blob/main/docs/papers/PLASTICITY-COMPACTION-MOE.md)** — MoE expert pruning (67GB to 14GB)
+
+## Links
 
-[sentinel-ai](https://github.com/CambrianTech/sentinel-ai) | [continuum](https://github.com/CambrianTech/continuum) | [HuggingFace](https://huggingface.co/continuum-ai)
+- [All published models](https://huggingface.co/continuum-ai)
+- [sentinel-ai](https://github.com/CambrianTech/sentinel-ai) — Open source forge framework
+- [continuum](https://github.com/CambrianTech/continuum) — Distributed AI on consumer hardware