rogermt commited on
Commit
b311dde
·
verified ·
1 Parent(s): 72354fe

Move own-solver/ARC-GEN-README.md to own-solver/

Browse files
Files changed (1) hide show
  1. own-solver/ARC-GEN-README.md +79 -0
own-solver/ARC-GEN-README.md ADDED
@@ -0,0 +1,79 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <p align="center">
2
+ <img src="misc/images/arc-gen-logo.jpg">
3
+ </p>
4
+
5
+ This repository contains the source code for *ARC-GEN*, a mimetic procedural benchmark generator for the Abstraction and Reasoning Corpus.
6
+
7
+ For a more in-depth description of this work, see the [corresponding paper on arxiv](https://arxiv.org/abs/2511.00162).
8
+
9
+ ## News
10
+
11
+ * `2026-04-04`: ARC-GEN to be used as the official benchmark generator in the [2026 NeuroGolf Championship](https://www.kaggle.com/competitions/neurogolf-2026) featured at [IJCAI-ECAI 2026](https://2026.ijcai.org/).
12
+ * `2026-03-25`: ARC-GEN now supports 500 additional tasks from [ARC-AGI-2](https://arcprize.org/arc-agi/2).
13
+ * `2025-10-31`: An ARC-GEN overview is now available on [arxiv](https://arxiv.org/abs/2511.00162).
14
+ * `2025-07-31`: ARC-GEN to be used as the official benchmark generator in the [2025 Google Code Golf Championship](https://www.kaggle.com/competitions/google-code-golf-2025) featured at [NeurIPS 2025](https://neurips.cc/Conferences/2025).
15
+ * `2025-05-15`: The initial ARC-GEN repository committed to GitHub.
16
+
17
+ ## Installation
18
+
19
+ ```
20
+ $ git clone --recurse-submodules https://github.com/google/ARC-GEN.git && cd ARC-GEN
21
+ ```
22
+
23
+ ## Usage
24
+
25
+ For **benchmark generation**, use the `generate` command with two arguments: the task ID, and the desired number of example pairs.
26
+
27
+ ```
28
+ $ python3 arc_gen.py generate 1e0a9b12 1000
29
+ [{'input': [[4, 0, 0, 0], [0, 0, 0, 0], [4, 0, 8, 0], [0, 3, 8, 0]], 'output': ...
30
+ ```
31
+
32
+ For **validation** (i.e., to ensure that the ARC-GEN generators can collectively reproduce the original [ARC-AGI-1](https://github.com/fchollet/ARC-AGI) benchmark suite), use the `validate` command:
33
+
34
+ ```
35
+ $ python3 arc_gen.py validate
36
+ A total of 400 generators passed.
37
+ A total of 0 generators failed.
38
+ ```
39
+
40
+ For an example of customized **variations**, refer to [arc_gen_variations.py](https://github.com/google/ARC-GEN/blob/main/arc_gen_variations.py), which produces two variations on [Task #125](https://arcprize.org/play?task=543a7ed5):
41
+
42
+ ```
43
+ generator, _ = task_list.task_list().get("543a7ed5")
44
+ examples = []
45
+ # Two examples of a "large" variation on Task #125.
46
+ examples.extend([generator(boxes=8, size=28) for _ in range(2)])
47
+ # Two examples of a "large + inverted" variation on Task #125.
48
+ common.set_colors([0, 1, 2, 6, 8, 5, 3, 7, 4, 9])
49
+ examples.extend([generator(boxes=8, size=28) for _ in range(2)])
50
+ ```
51
+
52
+ ## The ARC-GEN-100K Dataset
53
+
54
+ For those seeking a pre-generated dataset of sample pairs, the link below provides a static benchmark suite containing 100,000 examples produced by ARC-GEN (covering all four-hundred tasks):
55
+
56
+ <p align="center">
57
+ https://www.kaggle.com/datasets/arcgen100k/the-arc-gen-100k-dataset
58
+ <br><br>
59
+ <img src="misc/images/arc-gen-gallery-faded.png">
60
+ </p>
61
+
62
+ ## How to Cite?
63
+
64
+ ```
65
+ @misc{Moffitt2025,
66
+ title={{ARC-GEN: A Mimetic Procedural Benchmark Generator for the Abstraction and Reasoning Corpus}},
67
+ author={Michael D. Moffitt},
68
+ year={2025},
69
+ eprint={2511.00162},
70
+ archivePrefix={arXiv},
71
+ primaryClass={cs.AI},
72
+ url={https://arxiv.org/abs/2511.00162},
73
+ }
74
+ ```
75
+
76
+ ## Other Resouces
77
+
78
+ * [RE-ARC: Reverse-Engineering the Abstraction and Reasoning Corpus](https://github.com/michaelhodel/re-arc) by Michael Hodel
79
+ * [Bootstrapping ARC: Synthetic Problem Generation for ARC Visual Reasoning Tasks](https://github.com/xu3kev/BARC) by Wen-Ding Li and others