---
license: mit
pipeline_tag: text-generation
library_name: transformers
tags:
- enhancer-generation
- jamba
---

# EnhancAR Sorted
EnhancAR Sorted is an autoregressive generative model of enhancer homology families, trained on 233,158,475 enhancers extracted from 241 vertebrate genomes. By "unrolling" homology families (enhancer sequences are sorted into sets of homology sequences, and input data is sequences concatenated to each other with a separator token delimiting different sequences), EnhancAR-Sorted learns to generate new sequences that conserve the function of prompt sequences. We demonstrate that this can be used to design new enhancers "by example", which is particularly useful when the function of enhancers is not known a priori.

## Model Details

- **Developed by:** Kevin K. Yang, Alex J. Lee, Andrew G. Duncan, Micaela E Consens, Lorin Crawford, Jennifer A. Mitchell, and Alan M. Moses
- **Model type:** Hybrid state-space-model transformer architecture with mixture-of-experts
- **License:** MIT

## Model Sources

- **Repository:** https://github.com/microsoft/enhancar
- **Data:** https://zenodo.org/records/19052620

## Uses

EnhancAR-sorted is a generative enhancer model that can be used for:
* Unconditional generation of enhancer sequences
* Conditional design of shortened enhancer sequences

## Getting Started with EnhancAR-Sorted

**Requirements**:
* PyTorch: 2.7.1
* CUDA: 12.8 and above
  
```py
pip install transformers==4.48.2 huggingface_hub
```

**Example generation**:

The following shows an example for generating an unconditional sequence. To generate a sequence conditionally, replace the input_sequence with a string of the form:
{homolog A/homolog B/.../homolog N}.

For a full example, please see the [Google Colab]()

```py
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, AutoConfig

config = AutoConfig.from_pretrained("aduncan94/EnhancAR-Sorted", trust_remote_code=True)
config.use_mamba_kernels = False
model = AutoModelForCausalLM.from_pretrained("aduncan94/EnhancAR-Sorted", trust_remote_code=True, config=config).to("cuda")
tokenizer = AutoTokenizer.from_pretrained("aduncan94/EnhancAR-Sorted", trust_remote_code=True)

input_sequence = "{"
inputs = tokenizer(input_sequence, return_tensors="pt", return_token_type_ids=False, add_special_tokens=False)

outputs = generate_sequence(inputs, model, tokenizer)
output = tokenizer.batch_decode(outputs, skip_special_tokens=False)[0]

print(output)
```

## Technical Specifications

### Compute Infrastructure
* The 170M-parameter model was trained on 8 MI300X GPUs

## Citation

If you use the code, models, or results, please cite our [preprint](https://www.biorxiv.org/content/10.64898/2026.04.13.718170v1)