Commit ·
4dae5fa
1
Parent(s): bb09849
Archive push logs; bump rebuild so on_hf cache refreshes with 299 live stubs
Browse files- logs/README.md +39 -0
- logs/push_batch_initial_10.log +11 -0
- logs/push_bulk_parallel_day1.log +0 -0
logs/README.md
ADDED
|
@@ -0,0 +1,39 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Push logs — chronological
|
| 2 |
+
|
| 3 |
+
Persisted output of every metadata-stub push run. Each file is raw stdout
|
| 4 |
+
from `scripts/push_metadata_stubs.py`, kept verbatim so we have a paper
|
| 5 |
+
trail of which slugs went up when and what errored.
|
| 6 |
+
|
| 7 |
+
## Session: 2026-04-19
|
| 8 |
+
|
| 9 |
+
### `push_batch_initial_10.log`
|
| 10 |
+
|
| 11 |
+
First sanity run. 10 diverse stubs pushed to validate the renderer on a
|
| 12 |
+
variety of sources (NEMAR HBN releases, OpenNeuro, MEG, iEEG, various
|
| 13 |
+
pathologies).
|
| 14 |
+
|
| 15 |
+
Slugs: `ds000117`, `ds000246`, `ds000247`, `EEG2025r1`, `ds003800`,
|
| 16 |
+
`ds002799`, `EEG2025r10`, `ds004551`, `ds004598`, `ds003061`.
|
| 17 |
+
|
| 18 |
+
Pushed: **10**. Failures: 0. Wall clock: ~3m13s (serial, ~20 s/push).
|
| 19 |
+
|
| 20 |
+
### `push_bulk_parallel_day1.log`
|
| 21 |
+
|
| 22 |
+
Option C: every remaining slug, parallelized with 12 workers.
|
| 23 |
+
|
| 24 |
+
- Attempted: 670 (736 total − 10 initial − 56 already on HF from prior runs).
|
| 25 |
+
- **Pushed: 233** (18:02:37 → 18:09:02, ~1.7 s/push wall clock × 12 threads).
|
| 26 |
+
- Hit HF's org-level **rate limit of 300 dataset repo creations per day**.
|
| 27 |
+
- Remaining: ~437 (queued for tomorrow).
|
| 28 |
+
|
| 29 |
+
Script is idempotent — resume with the same command:
|
| 30 |
+
|
| 31 |
+
```bash
|
| 32 |
+
python scripts/push_metadata_stubs.py --all --skip-existing --workers 12
|
| 33 |
+
```
|
| 34 |
+
|
| 35 |
+
## Totals after day 1
|
| 36 |
+
|
| 37 |
+
- `EEGDash/*` dataset repos live: **299** (plus the `catalog` Space).
|
| 38 |
+
- Remaining to push: **~437**.
|
| 39 |
+
- HF rate-limit window resets 24 h after the first repo creation of the day.
|
logs/push_batch_initial_10.log
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
2026-04-19 17:32:52,783 INFO push_metadata_stubs — pushed EEGDash/eeg2025r1
|
| 2 |
+
2026-04-19 17:33:07,209 INFO push_metadata_stubs — pushed EEGDash/eeg2025r10
|
| 3 |
+
2026-04-19 17:33:28,586 INFO push_metadata_stubs — pushed EEGDash/ds000117
|
| 4 |
+
2026-04-19 17:33:53,048 INFO push_metadata_stubs — pushed EEGDash/ds000246
|
| 5 |
+
2026-04-19 17:34:14,632 INFO push_metadata_stubs — pushed EEGDash/ds000247
|
| 6 |
+
2026-04-19 17:34:37,823 INFO push_metadata_stubs — pushed EEGDash/ds002799
|
| 7 |
+
2026-04-19 17:34:59,210 INFO push_metadata_stubs — pushed EEGDash/ds003061
|
| 8 |
+
2026-04-19 17:35:22,699 INFO push_metadata_stubs — pushed EEGDash/ds003800
|
| 9 |
+
2026-04-19 17:35:44,064 INFO push_metadata_stubs — pushed EEGDash/ds004551
|
| 10 |
+
2026-04-19 17:36:05,691 INFO push_metadata_stubs — pushed EEGDash/ds004598
|
| 11 |
+
2026-04-19 17:36:05,944 INFO push_metadata_stubs — done — 10 stubs processed
|
logs/push_bulk_parallel_day1.log
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|