| --- |
| base_model: Qwen/Qwen3.6-27B |
| library_name: peft |
| pipeline_tag: text-generation |
| license: mit |
| tags: |
| - activation-oracles |
| - interpretability |
| - lora |
| - peft |
| - self-introspection |
| --- |
| |
| # Activation Oracle for Qwen3.6-27B |
|
|
| This is a PEFT LoRA adapter for `Qwen/Qwen3.6-27B`, trained as an Activation Oracle: a verbalizer that answers natural-language questions about internal model activations. |
|
|
| The adapter is intended for use with the Activation Oracles codebase and demo workflow, where target-model activations are injected into the verbalizer via activation steering hooks. |
|
|
| ## Details |
|
|
| - Base model: `Qwen/Qwen3.6-27B` |
| - Adapter type: LoRA |
| - LoRA rank: 64 |
| - LoRA alpha: 128 |
| - LoRA dropout: 0.05 |
| - Training mixture: LatentQA, binary classification tasks, and Past Lens/self-supervised context prediction |
| - Activation layers: 25%, 50%, and 75% depth of the target model |
| - Hook layer: 1 |
|
|
| ## Usage |
|
|
| See the project repository for end-to-end inference code: |
|
|
| - GitHub: https://github.com/federicotorrielli/activation_oracles_qwen36 |
|
|
| Basic adapter loading: |
|
|
| ```python |
| from peft import PeftModel |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.6-27B") |
| tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.6-27B") |
| model = PeftModel.from_pretrained(base_model, "EvilScript/activation-oracle-qwen3.6-27B") |
| ``` |
|
|
| Loading the adapter alone does not perform activation-oracle inference; the activation collection and steering-hook path is implemented in the repository. |
|
|
