Instructions to use elliottower2/gpt2-rti-zero-ablated with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use elliottower2/gpt2-rti-zero-ablated with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="elliottower2/gpt2-rti-zero-ablated")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("elliottower2/gpt2-rti-zero-ablated")
model = AutoModelForCausalLM.from_pretrained("elliottower2/gpt2-rti-zero-ablated")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use elliottower2/gpt2-rti-zero-ablated with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "elliottower2/gpt2-rti-zero-ablated"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "elliottower2/gpt2-rti-zero-ablated",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/elliottower2/gpt2-rti-zero-ablated

SGLang

How to use elliottower2/gpt2-rti-zero-ablated with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "elliottower2/gpt2-rti-zero-ablated" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "elliottower2/gpt2-rti-zero-ablated",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "elliottower2/gpt2-rti-zero-ablated" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "elliottower2/gpt2-rti-zero-ablated",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use elliottower2/gpt2-rti-zero-ablated with Docker Model Runner:
```
docker model run hf.co/elliottower2/gpt2-rti-zero-ablated
```

gpt2-rti-zero-ablated / README.md

elliottower2

Upload folder using huggingface_hub

7e26c0e verified 3 days ago

preview code

raw

history blame contribute delete

1.79 kB

	---
	license: mit
	library_name: transformers
	tags:
	- gpt2
	- onnx
	- mechanistic-interpretability
	- circuit-ablation
	- rti-circuit
	base_model: openai-community/gpt2
	---

	# GPT-2 with RTI Circuit Zero-Ablated

	GPT-2 (124M) with the 15-head Repeated Token Identification (RTI) circuit removed via zero ablation.

	## What was ablated

	The RTI circuit consists of 15 attention heads across 4 functional tiers:

	\| Tier \| Heads \| Function \|
	\|------\|-------\|----------\|
	\| Backbone \| 0.8, 0.9, 0.11 \| Broad token matching via positional/frequency features \|
	\| Detector \| 4.11 \| Repeated-token detection gate \|
	\| Copier \| 4.0, 5.6, 5.7, 7.0, 8.4, 8.7, 9.3, 9.10 \| Copy repeated token identity to output \|
	\| Readout \| 10.11, 11.9, 11.11 \| Route copied information to final logits \|

	## Ablation method

	Zero ablation: For each circuit head, the corresponding columns of `c_proj.weight` (the output projection W_O) were set to zero. This prevents the head from writing anything to the residual stream, effectively removing its contribution.

	## Effect

	The ablated model loses the ability to predict repeated tokens. For example:
	- Normal GPT-2: "The cat sat on the mat. The cat" → " was a little bit older than me, but I"
	- Zero-ablated: "The cat sat on the mat. The cat" → " sat on the mat. The cat sat on the"

	## Usage with Transformers.js

	```javascript
	import { AutoModelForCausalLM, AutoTokenizer } from '@huggingface/transformers';

	const model = await AutoModelForCausalLM.from_pretrained('elliottower2/gpt2-rti-zero-ablated', {
	dtype: 'fp32',
	});
	const tokenizer = await AutoTokenizer.from_pretrained('elliottower2/gpt2-rti-zero-ablated');
	```

	## Citation

	Part of the factorization-circuits project studying weight-space circuit discovery in transformers.