README.md

---
tags:
  - knowledge-editing
  - circuit-entropy
  - llama-3
license: llama3
---

# LLaMA 3 8B — CE-FT-1 Single-Fact Edit

Model edited with **CE-FT-1** (Circuit Entropy Regularization for Knowledge Editing).

Base model: `meta-llama/Meta-Llama-3-8B-Instruct`

## Edit

| | |
|---|---|
| **Prompt** | `The Eiffel Tower is located in the city of` |
| **Target** | `Berlin` |
| **Method** | CE-FT-1 |
| **Lambda** | 5 |
| **Edit success** | True |

## Training Config

| Parameter | Value |
|---|---|
| Steps | 20 |
| Learning rate | 5e-06 |
| Weight decay | 0.01 |
| Grad clip | 1.0 |
| Lambda (entropy) | 5 |
| EAP-IG steps | 5 |
| dtype | bfloat16 |
| Seed | 42 |

## Final Metrics

| Metric | Value |
|---|---|
| Final L_CE | 0.006029 |
| Final KL | 0.068497 |
| Final H(C) | 9.8983 |
| Final delta_H | 0.1011 |

## Usage

```python
from transformer_lens import HookedTransformer
import torch

model = HookedTransformer.from_pretrained(
    "meta-llama/Meta-Llama-3-8B-Instruct",
    dtype=torch.bfloat16,
)
state_dict = torch.load("model_state_dict.pt", map_location="cpu")
model.load_state_dict(state_dict)
model = model.to("cuda")

tokens = model.to_tokens("The Eiffel Tower is located in the city of")
out = model.generate(tokens, max_new_tokens=10, do_sample=False)
print(model.tokenizer.decode(out[0]))
```

## License

This model inherits the [Meta LLaMA 3 Community License](https://llama.meta.com/llama3/license/).

## Paper

Circuit Entropy Regularization for Knowledge Editing (NeurIPS 2026 submission)