Update TODO.md
Browse files
TODO.md
CHANGED
|
@@ -0,0 +1,29 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Here's what I need from you and what the situation looks like:
|
| 2 |
+
|
| 3 |
+
Current Status: 306/400 ✅ (was 294)
|
| 4 |
+
What the model_runner.py code IS
|
| 5 |
+
That's the 2024 ARC Prize winning LLM infrastructure — it fine-tunes a language model to generate ARC solutions as text. It's a completely different
|
| 6 |
+
competition format (text answers,
|
| 7 |
+
not ONNX models). It won't help us directly.
|
| 8 |
+
|
| 9 |
+
What WOULD help — things you can search for:
|
| 10 |
+
1. ARC-GEN additional training data The ARC-GEN paper generates extra input/output examples for each task. More examples → our lstsq conv solver gets more
|
| 11 |
+
|
| 12 |
+
2. data → better fitting → more tasks solved. Search Kaggle/GitHub for "arc-gen" data files or the generator code.
|
| 13 |
+
|
| 14 |
+
3. NeuroGolf / Code Golf Championship notebooks Search Kaggle for neurogolf 2026 or code golf championship ARC ONNX. Other competitors' approaches would be gold.
|
| 15 |
+
|
| 16 |
+
4. CompressARC code (paper) This 76K param approach trains a tiny network PER TASK via gradient descent. If we could adapt it to produce opset-10 ONNX models,
|
| 17 |
+
|
| 18 |
+
5. it could solve many of our 94 remaining tasks. Search GitHub for "CompressARC" or the authors' repos.
|
| 19 |
+
|
| 20 |
+
What's actually blocking us (94 unsolved tasks)
|
| 21 |
+
Category Count Example Why unsolved
|
| 22 |
+
Variable diff-shape (output smaller) ~60 Extract subregion from grid Output shape depends on input content — can't build static ONNX
|
| 23 |
+
Variable diff-shape (output larger) ~17 Tile/upscale by variable factor Same problem
|
| 24 |
+
Same-shape but complex ~10 Multi-step reasoning, flood fill Conv can't learn non-local/algorithmic patterns
|
| 25 |
+
Fixed diff-shape (output larger) ~7 Input-driven block placement Output depends on input VALUES, not just positions
|
| 26 |
+
The fundamental challenge: 94 tasks require reasoning that depends on input content (not just a fixed pixel remapping or local conv pattern).
|
| 27 |
+
Our current ONNX opset 10 toolkit (Conv, Gather, ArgMax, etc.) can only express fixed mappings.
|
| 28 |
+
We'd need to find tasks where the mapping IS fixed but our solver just hasn't found it yet — likely by adding more training examples
|
| 29 |
+
(ARC-GEN) or trying bigger conv kernels with more time budget.
|