---
base_model: microsoft/Fara-7B
library_name: transformers
license: other
pipeline_tag: text-generation
tags:
  - abliteration
  - refusal-removal
  - uncensored
  - research
  - qwen2_5_vl
  - orthogonalization
---

# Fara-7B Abliterated v2

A refusal-direction-orthogonalized variant of `microsoft/Fara-7B` (Qwen2.5-VL based).

Built using:
- https://github.com/HOLYKEYZ/model-unfetter

## Method

Using harmful + harmless probe sets, residual-stream activations were extracted across layers 0–27 to identify the strongest refusal direction.

Best layer:
- 13

Orthogonalization was applied in fp32 to:
- `embed_tokens`
- every `self_attn.o_proj`
- every `mlp.down_proj`

Total modified tensors:
- 57

Formula:

```python
W ← W - r rᵀ W
```

## Results

Held-out harmful evaluation set:
- Original Fara-7B: 5/160 compliance (~3.1%)
- Abliterated v2: 158/160 compliance (~98.75%)

Held-out refusal probe:
- Before: 155/160 refusals
- After: 2/160 refusals

## Notes

- fp32 surgery used to avoid precision issues from v1
- edits applied only to the language tower
- held-out evaluation set was separate from the layer-selection probe set

Research artifact only. Use responsibly and follow upstream Fara/Qwen license terms.