Move own-solver/ARC-GEN-README.md to own-solver/
Browse files- own-solver/ARC-GEN-README.md +79 -0
own-solver/ARC-GEN-README.md
ADDED
|
@@ -0,0 +1,79 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
<p align="center">
|
| 2 |
+
<img src="misc/images/arc-gen-logo.jpg">
|
| 3 |
+
</p>
|
| 4 |
+
|
| 5 |
+
This repository contains the source code for *ARC-GEN*, a mimetic procedural benchmark generator for the Abstraction and Reasoning Corpus.
|
| 6 |
+
|
| 7 |
+
For a more in-depth description of this work, see the [corresponding paper on arxiv](https://arxiv.org/abs/2511.00162).
|
| 8 |
+
|
| 9 |
+
## News
|
| 10 |
+
|
| 11 |
+
* `2026-04-04`: ARC-GEN to be used as the official benchmark generator in the [2026 NeuroGolf Championship](https://www.kaggle.com/competitions/neurogolf-2026) featured at [IJCAI-ECAI 2026](https://2026.ijcai.org/).
|
| 12 |
+
* `2026-03-25`: ARC-GEN now supports 500 additional tasks from [ARC-AGI-2](https://arcprize.org/arc-agi/2).
|
| 13 |
+
* `2025-10-31`: An ARC-GEN overview is now available on [arxiv](https://arxiv.org/abs/2511.00162).
|
| 14 |
+
* `2025-07-31`: ARC-GEN to be used as the official benchmark generator in the [2025 Google Code Golf Championship](https://www.kaggle.com/competitions/google-code-golf-2025) featured at [NeurIPS 2025](https://neurips.cc/Conferences/2025).
|
| 15 |
+
* `2025-05-15`: The initial ARC-GEN repository committed to GitHub.
|
| 16 |
+
|
| 17 |
+
## Installation
|
| 18 |
+
|
| 19 |
+
```
|
| 20 |
+
$ git clone --recurse-submodules https://github.com/google/ARC-GEN.git && cd ARC-GEN
|
| 21 |
+
```
|
| 22 |
+
|
| 23 |
+
## Usage
|
| 24 |
+
|
| 25 |
+
For **benchmark generation**, use the `generate` command with two arguments: the task ID, and the desired number of example pairs.
|
| 26 |
+
|
| 27 |
+
```
|
| 28 |
+
$ python3 arc_gen.py generate 1e0a9b12 1000
|
| 29 |
+
[{'input': [[4, 0, 0, 0], [0, 0, 0, 0], [4, 0, 8, 0], [0, 3, 8, 0]], 'output': ...
|
| 30 |
+
```
|
| 31 |
+
|
| 32 |
+
For **validation** (i.e., to ensure that the ARC-GEN generators can collectively reproduce the original [ARC-AGI-1](https://github.com/fchollet/ARC-AGI) benchmark suite), use the `validate` command:
|
| 33 |
+
|
| 34 |
+
```
|
| 35 |
+
$ python3 arc_gen.py validate
|
| 36 |
+
A total of 400 generators passed.
|
| 37 |
+
A total of 0 generators failed.
|
| 38 |
+
```
|
| 39 |
+
|
| 40 |
+
For an example of customized **variations**, refer to [arc_gen_variations.py](https://github.com/google/ARC-GEN/blob/main/arc_gen_variations.py), which produces two variations on [Task #125](https://arcprize.org/play?task=543a7ed5):
|
| 41 |
+
|
| 42 |
+
```
|
| 43 |
+
generator, _ = task_list.task_list().get("543a7ed5")
|
| 44 |
+
examples = []
|
| 45 |
+
# Two examples of a "large" variation on Task #125.
|
| 46 |
+
examples.extend([generator(boxes=8, size=28) for _ in range(2)])
|
| 47 |
+
# Two examples of a "large + inverted" variation on Task #125.
|
| 48 |
+
common.set_colors([0, 1, 2, 6, 8, 5, 3, 7, 4, 9])
|
| 49 |
+
examples.extend([generator(boxes=8, size=28) for _ in range(2)])
|
| 50 |
+
```
|
| 51 |
+
|
| 52 |
+
## The ARC-GEN-100K Dataset
|
| 53 |
+
|
| 54 |
+
For those seeking a pre-generated dataset of sample pairs, the link below provides a static benchmark suite containing 100,000 examples produced by ARC-GEN (covering all four-hundred tasks):
|
| 55 |
+
|
| 56 |
+
<p align="center">
|
| 57 |
+
https://www.kaggle.com/datasets/arcgen100k/the-arc-gen-100k-dataset
|
| 58 |
+
<br><br>
|
| 59 |
+
<img src="misc/images/arc-gen-gallery-faded.png">
|
| 60 |
+
</p>
|
| 61 |
+
|
| 62 |
+
## How to Cite?
|
| 63 |
+
|
| 64 |
+
```
|
| 65 |
+
@misc{Moffitt2025,
|
| 66 |
+
title={{ARC-GEN: A Mimetic Procedural Benchmark Generator for the Abstraction and Reasoning Corpus}},
|
| 67 |
+
author={Michael D. Moffitt},
|
| 68 |
+
year={2025},
|
| 69 |
+
eprint={2511.00162},
|
| 70 |
+
archivePrefix={arXiv},
|
| 71 |
+
primaryClass={cs.AI},
|
| 72 |
+
url={https://arxiv.org/abs/2511.00162},
|
| 73 |
+
}
|
| 74 |
+
```
|
| 75 |
+
|
| 76 |
+
## Other Resouces
|
| 77 |
+
|
| 78 |
+
* [RE-ARC: Reverse-Engineering the Abstraction and Reasoning Corpus](https://github.com/michaelhodel/re-arc) by Michael Hodel
|
| 79 |
+
* [Bootstrapping ARC: Synthetic Problem Generation for ARC Visual Reasoning Tasks](https://github.com/xu3kev/BARC) by Wen-Ding Li and others
|