aduncan94/EnhancAR-Sorted

EnhancAR Sorted

EnhancAR Sorted is an autoregressive generative model of enhancer homology families, trained on 233,158,475 enhancers extracted from 241 vertebrate genomes. By "unrolling" homology families (enhancer sequences are sorted into sets of homology sequences, and input data is sequences concatenated to each other with a separator token delimiting different sequences), EnhancAR-Sorted learns to generate new sequences that conserve the function of prompt sequences. We demonstrate that this can be used to design new enhancers "by example", which is particularly useful when the function of enhancers is not known a priori.

Model Details

• Developed by: Kevin K. Yang, Alex J. Lee, Andrew G. Duncan, Micaela E Consens, Lorin Crawford, Jennifer A. Mitchell, and Alan M. Moses
• Model type: Hybrid state-space-model transformer architecture with mixture-of-experts
• License: MIT

Model Sources

• Repository: https://github.com/microsoft/enhancar
• Data: https://zenodo.org/records/19052620

Uses

EnhancAR-sorted is a generative enhancer model that can be used for:

• Unconditional generation of enhancer sequences
• Conditional design of shortened enhancer sequences

Getting Started with EnhancAR-Sorted

Requirements:

• PyTorch: 2.7.1
• CUDA: 12.8 and above

pip install transformers==4.48.2 huggingface_hub

Example generation:

The following shows an example for generating an unconditional sequence. To generate a sequence conditionally, replace the input_sequence with a string of the form: {homolog A/homolog B/.../homolog N}.

For a full example, please see the Google Colab

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, AutoConfig

config = AutoConfig.from_pretrained("aduncan94/EnhancAR-Sorted", trust_remote_code=True)
config.use_mamba_kernels = False
model = AutoModelForCausalLM.from_pretrained("aduncan94/EnhancAR-Sorted", trust_remote_code=True, config=config).to("cuda")
tokenizer = AutoTokenizer.from_pretrained("aduncan94/EnhancAR-Sorted", trust_remote_code=True)

input_sequence = "{"
inputs = tokenizer(input_sequence, return_tensors="pt", return_token_type_ids=False, add_special_tokens=False)

outputs = generate_sequence(inputs, model, tokenizer)
output = tokenizer.batch_decode(outputs, skip_special_tokens=False)[0]

print(output)

Technical Specifications

Compute Infrastructure

• The 170M-parameter model was trained on 8 MI300X GPUs

Citation

If you use the code, models, or results, please cite our preprint