README.md

---
title: REPOMIND
emoji: 🧠
colorFrom: indigo
colorTo: red
sdk: gradio
sdk_version: 6.14.0
python_version: '3.13'
app_file: app.py
pinned: false
license: mit
short_description: Repo-scale coding agent — 256K context on a single MI300X
tags:
  - amd-hackathon-2026
  - amd-developer-hackathon
  - agents
  - coding-agent
  - long-context
  - rocm
  - mi300x
  - qwen3-coder
  - vllm
---

# REPOMIND

> Open-source repo-scale coding agent for self-hosted use. Designed to ingest an entire git repo (256K tokens, FP8) and reason across it on a single AMD MI300X — what NVIDIA H100 80GB cannot accommodate by VRAM accounting (~143GB total > 80GB).

**Built for the [AMD Developer Hackathon 2026](https://lablab.ai/ai-hackathons/amd-developer)** · MIT License · [GitHub source](https://github.com/SRKRZ23/repomind)

## Why MI300X?

- Qwen3-Coder-Next-FP8 weights ≈ 80 GB
- 256K KV cache @ FP8 ≈ 38 GB
- activations ≈ 25 GB → **~143 GB total on a single GPU**
- NVIDIA H100 80GB cannot accommodate this configuration on a single card by VRAM accounting (~143 GB > 80 GB). AMD MI300X 192 GB has the headroom.

This is a memory-architecture story, not a CUDA-vs-ROCm one.

## Stack

- **Model**: `Qwen/Qwen3-Coder-Next-FP8` — 80B params, 3B active (MoE)
- **Inference**: vLLM ROCm 7 with `qwen3_coder` tool-call parser
- **Agent loop**: SC-TIR style (PLAN → CALL TOOL → OBSERVE → THINK → ANSWER)
- **Tools**: `read_file` · `grep_codebase` · `execute_code` (sandboxed) · `run_tests` · `git_log`

## Status — verified on real MI300X (2026-05-05 / 2026-05-06)

Full stress test on a single AMD MI300X x1 (AMD Developer Cloud, $1.99/hr, vLLM 0.17.1 + ROCm 7.2 Quick Start image). **2 sessions, 124 min total, ~$4.12.**

**Memory budget — Qwen3-Coder-Next-FP8 + 256K context, FP8 KV cache:**
- ✅ Model weights in VRAM: **77.29 GiB**
- ✅ Available KV cache: **94.58 GiB** (2,065,744 tokens)
- ✅ VRAM peak: **176 GiB / 191.7 GiB** (92% utilization)
- ✅ `--max-model-len 262144` started, `Application startup complete`
- ✅ `/v1/models` returns `max_model_len: 262144`

**Concurrency stress (24 cells, default Triton attention, all 144 outputs clean):**
- ✅ **31/31 success at 8K, 16K, 32K, AND 64K** — every realistic-developer context
- ✅ **25/31 at 128K**, **6-8 at 256K** within a 15-minute window (compute-bound, honest ceiling)
- ✅ Aggregate throughput at N=31: 78.5 tok/s @ 8K · 31.4 @ 16K · 12.1 @ 32K · 3.6 @ 64K

**Long-context coherence — needle-in-haystack at 200K:**
- ✅ **3/3 positions passed** (early, middle, late) — model recovers embedded sentinel function and constant
- ✅ This proves 256K window is *usable*, not just *allocated*

**End-to-end repo ingestion — 9/9 questions answered correctly:**
- ✅ REPOMIND self (68K tokens, 68 files) — 3/3
- ✅ pallets/flask (408K total → fitted 180K) — 3/3
- ✅ **pytorch/vision (1.3M tokens, 581 files, 6,799 chunks → fitted 180K) — 3/3** with correct file path citations

**Tuning attempt — measured regression worth reporting:**
- ⚠️ Tried `--attention-backend ROCM_AITER_FA` (AMD's hand-tuned MI300X kernels)
- Throughput **2-4× higher** under AITER, TTFT 2.8× faster at 64K
- BUT output **degenerates to repeating-punctuation gibberish** in 137/144 cells under FP8 KV cache
- Default Triton stays the production-safe choice; filed for AMD upstream investigation

**Cost — at AMD Cloud $1.99/hr:**
- ✅ ~$45.75 / 1M completion tokens (aggregate at 32K, N=31)
- ✅ 14.5 active continuous queriers per MI300X, or 70–140 dev seats for typical bursty engineering teams
- ✅ Owned MI300X ($18K) breaks even vs Cursor in 3–6 months at team-of-100 usage

This Space currently runs CPU-basic with the **mock LLM backend** because keeping a paid MI300X droplet up 24/7 for sporadic visitors is uneconomical. **Final demo wires to a live MI300X endpoint** during the judging window.

Full evidence pack (7 JSON results + 5 PNG plots + e2e prompts/answers + 2× rocm-smi snapshots + run logs) is in the repo:
[github.com/SRKRZ23/repomind/tree/main/benchmarks/2026-05-05-mi300x-stress-test](https://github.com/SRKRZ23/repomind/tree/main/benchmarks/2026-05-05-mi300x-stress-test)
Extended PHASE 1+2 narrative (24-cell matrix + AITER A/B): [extended/SUMMARY.md](https://github.com/SRKRZ23/repomind/tree/main/benchmarks/2026-05-05-mi300x-stress-test/extended).

If the MI300X memory-architecture pitch resonates, **a like on this Space helps us with the Hugging Face Special Prize judging** 🤗

## Author

**Sardor Razikov** — Independent ML Engineer · Tashkent 🇺🇿
- Kaggle SPR 2026 #7/371 (Top 1.9%) · S6E3 #23/4,142 · AIMO3 39/50 (XTX $2.2M)
- [Epistemic Curie Benchmark on Zenodo](https://doi.org/10.5281/zenodo.19791329)
- [GitHub](https://github.com/SRKRZ23/repomind) · [LinkedIn](https://www.linkedin.com/in/sardor-razikov-569a5327b) · [X / Twitter](https://x.com/SardorRazi99093)
- Email: razikovsardor1@gmail.com · razikovs777@gmail.com