---
title: RPC-Bench Leaderboard
emoji: 📊
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 4.44.1
python_version: 3.12
app_file: app.py
pinned: false
license: mit
---

<p align="center">
    🌐 <a href="https://rpc-bench.github.io/" target="_blank">Project Page</a> •
    💻 <a href="https://github.com/RPC-Bench/PRC-Bench" target="_blank">GitHub</a> •
    📖 <a href="https://arxiv.org/abs/2601.14289" target="_blank">Paper</a> •
    🤗 <a href="https://huggingface.co" target="_blank">Hugging Face</a> •
    🧭 <a href="https://community.modelscope.cn/" target="_blank">ModelScope</a>
</p>

# RPC-Bench Leaderboard

RPC-Bench is a benchmark for research paper comprehension. This Space provides two functions:

- a public leaderboard for published submissions
- a submission entry for uploading new evaluation files

## Expected repository layout

The Space is designed to work with a separate submission dataset repository.

```text
space/
├── app.py
├── constants.py
├── eval.py
├── requirements.txt
└── benchmark/
    ├── dev.json
    └── test.json
```

If `benchmark/dev.json` and `benchmark/test.json` are not bundled in the Space repo, set `RPC_BENCH_GOLD_DIR` or `RPC_BENCH_GOLD_PATH` through Space secrets / variables.

The static leaderboard seed is stored in `leaderboard_seed.csv`. `index.html` is only used locally to generate that CSV and should not be uploaded to the Space repository.

## Submission format

Uploaded files should be JSONL with one answer per line:

```json
{"id":"...", "part_idx":1, "question":"...", "gen_answer":"...", "category":"..."}
```

## Required environment variables

- `HF_TOKEN`: token for cloning and pushing the submission repository
- `SUBMISSION_REPO_ID`: dataset repo used to store leaderboard results
- `RPC_BENCH_GOLD_DIR`: optional directory containing `dev.json` and `test.json`
- `OPENAI_API_KEY`: optional, required if you want the Space to run LLM-based judging inline
- `OPENAI_BASE_URL`: optional, for OpenAI-compatible endpoints

The Space can still accept uploads when the judge variables are missing, but evaluation will be marked as pending.