vectorplasticity commited on
Commit
a6d556d
Β·
verified Β·
1 Parent(s): a518f00

Add comprehensive README.md with documentation

Browse files
Files changed (1) hide show
  1. README.md +89 -5
README.md CHANGED
@@ -1,10 +1,94 @@
1
  ---
2
  title: Universal Model Trainer
3
- emoji: πŸ“‰
4
- colorFrom: indigo
5
- colorTo: pink
6
  sdk: docker
7
- pinned: false
 
 
 
 
 
 
 
 
8
  ---
9
 
10
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  title: Universal Model Trainer
3
+ emoji: πŸš€
4
+ colorFrom: blue
5
+ colorTo: purple
6
  sdk: docker
7
+ pinned: true
8
+ license: mit
9
+ short_description: Universal ML training dashboard for HuggingFace
10
+ tag:
11
+ - training
12
+ - machine-learning
13
+ - transformers
14
+ - pytorch
15
+ - docker
16
  ---
17
 
18
+ # πŸš€ Universal Model Trainer
19
+
20
+ A comprehensive, production-ready dashboard for training machine learning models on the HuggingFace ecosystem. Supports multiple task types, datasets, and model architectures with full user control.
21
+
22
+ ## ✨ Features
23
+
24
+ ### Supported Task Types
25
+ - **Causal Language Modeling** - Text generation (GPT-style)
26
+ - **Masked Language Modeling** - BERT-style pretraining
27
+ - **Sequence-to-Sequence** - Translation, summarization
28
+ - **Token Classification** - NER, POS tagging
29
+ - **Text Classification** - Sentiment, topic classification
30
+ - **Question Answering** - Extractive QA
31
+ - **Image Classification** - Vision tasks
32
+ - **Audio Classification** - Sound classification
33
+
34
+ ### Training Features
35
+ - **PEFT/LoRA Support** - Memory-efficient fine-tuning
36
+ - **DeepSpeed Integration** - Distributed training
37
+ - **Quantization Options** - 4-bit, 8-bit training
38
+ - **Custom Hyperparameters** - Full control over training config
39
+ - **Real-time Progress** - Live training metrics
40
+ - **Job Queue System** - Manage multiple training jobs
41
+ - **Model Versioning** - Track experiment history
42
+
43
+ ### Dataset Support
44
+ - **HuggingFace Hub** - 100,000+ datasets
45
+ - **Custom Upload** - CSV, JSON, JSONL support
46
+ - **Dataset Preprocessing** - Automatic tokenization
47
+
48
+ ## πŸ”§ API Endpoints
49
+
50
+ ### Training
51
+ ```
52
+ POST /api/train/start - Start a new training job
53
+ GET /api/train/status - Get job status
54
+ POST /api/train/stop - Stop running job
55
+ GET /api/train/history - View training history
56
+ ```
57
+
58
+ ### Models & Datasets
59
+ ```
60
+ GET /api/models/search - Search HuggingFace models
61
+ GET /api/datasets/search - Search HuggingFace datasets
62
+ GET /api/models/info - Get model info
63
+ ```
64
+
65
+ ### System
66
+ ```
67
+ GET /api/system/status - System health and resources
68
+ GET /api/system/gpus - Available GPU info
69
+ ```
70
+
71
+ ## πŸ—οΈ Architecture
72
+
73
+ - **FastAPI Backend** - Modern async Python web framework
74
+ - **Redis Queue** - Background job processing
75
+ - **SQLite/PostgreSQL** - Job and experiment persistence
76
+ - **HuggingFace Hub** - Model and dataset hosting
77
+
78
+ ## πŸ“ Configuration
79
+
80
+ Set the following environment variables:
81
+ - `HF_TOKEN` - Your HuggingFace API token (required for pushing models)
82
+ - `WANDB_API_KEY` - Weights & Biases API key (optional, for experiment tracking)
83
+
84
+ ## πŸ“š Documentation
85
+
86
+ Full API documentation available at `/docs` when the Space is running.
87
+
88
+ ## 🀝 Contributing
89
+
90
+ Contributions welcome! See issues for planned features.
91
+
92
+ ## πŸ“„ License
93
+
94
+ MIT License - See LICENSE file for details.