Spaces:

---

NousResearch

/

cna-refusal-ablation

like 0

Running on L40S

implemented KV caching in app.py

• app.py (Inference Optimization): Refactored the text generation stream to leverage Hugging Face past_key_values (use_cache=True). By preserving the context window state rather than re-evaluating the entire token prefix at each sequence step, generation complexity is reduced from O(T²) to O(T), yielding a 10×–50× reduction in token latency.