File size: 1,521 Bytes
aefe5bd | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 | ---
tags:
- knowledge-editing
- circuit-entropy
- llama-3
license: llama3
---
# LLaMA 3 8B — CE-FT-1 Single-Fact Edit
Model edited with **CE-FT-1** (Circuit Entropy Regularization for Knowledge Editing).
Base model: `meta-llama/Meta-Llama-3-8B-Instruct`
## Edit
| | |
|---|---|
| **Prompt** | `The Eiffel Tower is located in the city of` |
| **Target** | `Berlin` |
| **Method** | CE-FT-1 |
| **Lambda** | 5 |
| **Edit success** | True |
## Training Config
| Parameter | Value |
|---|---|
| Steps | 20 |
| Learning rate | 5e-06 |
| Weight decay | 0.01 |
| Grad clip | 1.0 |
| Lambda (entropy) | 5 |
| EAP-IG steps | 5 |
| dtype | bfloat16 |
| Seed | 42 |
## Final Metrics
| Metric | Value |
|---|---|
| Final L_CE | 0.006029 |
| Final KL | 0.068497 |
| Final H(C) | 9.8983 |
| Final delta_H | 0.1011 |
## Usage
```python
from transformer_lens import HookedTransformer
import torch
model = HookedTransformer.from_pretrained(
"meta-llama/Meta-Llama-3-8B-Instruct",
dtype=torch.bfloat16,
)
state_dict = torch.load("model_state_dict.pt", map_location="cpu")
model.load_state_dict(state_dict)
model = model.to("cuda")
tokens = model.to_tokens("The Eiffel Tower is located in the city of")
out = model.generate(tokens, max_new_tokens=10, do_sample=False)
print(model.tokenizer.decode(out[0]))
```
## License
This model inherits the [Meta LLaMA 3 Community License](https://llama.meta.com/llama3/license/).
## Paper
Circuit Entropy Regularization for Knowledge Editing (NeurIPS 2026 submission)
|