jeongseokoh/llama3.1_8b_sft_SPEED-20-BoS_OpenCode

LLOPA Model

This repo bundles LLOPA/TRI inference with minimal friction.

from transformers import AutoTokenizer, AutoModelForCausalLM

tok = AutoTokenizer.from_pretrained("your-repo")
model = AutoModelForCausalLM.from_pretrained("your-repo", trust_remote_code=True)

out = model.llopa_generate(
    tokenizer=tok,
    system="You are a helpful assistant.",
    document="...",
    question="...",
    K=4,
    prefill_mode="lower",
    prefill_attn="causal",
)
print(out)