Upload docs/HUGGINGFACE_UPLOAD.md with huggingface_hub
Browse files- docs/HUGGINGFACE_UPLOAD.md +48 -0
docs/HUGGINGFACE_UPLOAD.md
ADDED
|
@@ -0,0 +1,48 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Hugging Face Artifact Upload
|
| 2 |
+
|
| 3 |
+
Large DecodeShare artifacts should live outside Git history. The intended
|
| 4 |
+
repository is:
|
| 5 |
+
|
| 6 |
+
```text
|
| 7 |
+
Zishan-Shao/decodeshare
|
| 8 |
+
```
|
| 9 |
+
|
| 10 |
+
The file `docs/artifact_manifest.tsv` lists large local files and suggested
|
| 11 |
+
paths under `artifacts/` in the Hugging Face repository.
|
| 12 |
+
|
| 13 |
+
## Suggested upload pattern
|
| 14 |
+
|
| 15 |
+
Install and authenticate the Hugging Face CLI:
|
| 16 |
+
|
| 17 |
+
```bash
|
| 18 |
+
pip install -U huggingface_hub[hf_transfer]
|
| 19 |
+
hf auth login
|
| 20 |
+
```
|
| 21 |
+
|
| 22 |
+
For a full upload from the original workspace:
|
| 23 |
+
|
| 24 |
+
```bash
|
| 25 |
+
cd
|
| 26 |
+
hf upload Zishan-Shao/decodeshare Hype1/results/acts artifacts/Hype1/results/acts
|
| 27 |
+
hf upload Zishan-Shao/decodeshare downstream/outputs artifacts/downstream/outputs
|
| 28 |
+
hf upload Zishan-Shao/decodeshare patch_back/results artifacts/patch_back/results
|
| 29 |
+
```
|
| 30 |
+
|
| 31 |
+
For a smaller first release, upload only the most reusable artifacts:
|
| 32 |
+
|
| 33 |
+
```bash
|
| 34 |
+
hf upload Zishan-Shao/decodeshare Hype1/results/acts artifacts/Hype1/results/acts
|
| 35 |
+
hf upload Zishan-Shao/decodeshare patch_back/results artifacts/patch_back/results
|
| 36 |
+
```
|
| 37 |
+
|
| 38 |
+
## Notes
|
| 39 |
+
|
| 40 |
+
- The current GitHub branch excludes `.npy`, `.npz`, `.pt`, `.bin`, and related
|
| 41 |
+
large binary formats.
|
| 42 |
+
- The largest local files are downstream `.pt` profiling and whitening outputs;
|
| 43 |
+
decide whether those are necessary before uploading the full manifest.
|
| 44 |
+
- The manifest is intentionally broad and includes local experimental archives.
|
| 45 |
+
Inspect it before uploading everything.
|
| 46 |
+
- If you want this to be a dataset repository rather than a model repository,
|
| 47 |
+
create or switch to a Hugging Face Dataset repo and add `--repo-type dataset`
|
| 48 |
+
to the `hf upload` commands.
|