gorold commited on
Commit
d6727ec
·
verified ·
1 Parent(s): d706d88

Restore README.md (revert accidental overwrite from PyTorchModelHubMixin push)

Browse files
Files changed (1) hide show
  1. README.md +129 -5
README.md CHANGED
@@ -1,10 +1,134 @@
1
  ---
2
  tags:
3
- - model_hub_mixin
 
 
 
 
 
 
 
4
  - pytorch_model_hub_mixin
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5
  ---
 
6
 
7
- This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration:
8
- - Code: [More Information Needed]
9
- - Paper: [More Information Needed]
10
- - Docs: [More Information Needed]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  tags:
3
+ - time-series-forecasting
4
+ - foundation-models
5
+ - pretrained-models
6
+ - time-series
7
+ - timeseries
8
+ - forecasting
9
+ - observability
10
+ - safetensors
11
  - pytorch_model_hub_mixin
12
+ license: apache-2.0
13
+ pipeline_tag: time-series-forecasting
14
+ thumbnail: https://corp.dd-static.net/img/about/presskit/kit/press_kit.png
15
+ model-index:
16
+ - name: Toto-2.0-313m
17
+ results:
18
+ - task:
19
+ type: time-series-forecasting
20
+ dataset:
21
+ name: BOOM
22
+ type: BOOM
23
+ metrics:
24
+ - name: CRPS
25
+ type: CRPS
26
+ value: 0.351
27
+ - name: MASE
28
+ type: MASE
29
+ value: 0.585
30
+ source:
31
+ name: BOOM 💥 Observability Time-Series Forecasting Leaderboard
32
+ url: https://huggingface.co/spaces/Datadog/BOOM
33
+ - task:
34
+ type: time-series-forecasting
35
+ dataset:
36
+ name: GIFT-Eval
37
+ type: GIFT-Eval
38
+ metrics:
39
+ - name: CRPS
40
+ type: CRPS
41
+ value: 0.481
42
+ - name: MASE
43
+ type: MASE
44
+ value: 0.703
45
+ source:
46
+ name: GIFT-Eval Time Series Forecasting Leaderboard
47
+ url: https://huggingface.co/spaces/Salesforce/GIFT-Eval
48
+
49
  ---
50
+ # Toto-2.0-313m
51
 
52
+ Toto (**T**ime Series **O**ptimized **T**ransformer for [**O**bservability](https://www.datadoghq.com/knowledge-center/observability/)) is a family of time series foundation models for multivariate forecasting developed by [Datadog](https://www.datadoghq.com/). **Toto 2.0** is the current generation, featuring u-μP-scaled transformers ranging from 4M to 2.5B parameters.
53
+
54
+ ---
55
+
56
+ ## ✨ Key Features
57
+
58
+ - **Zero-Shot Forecasting**: Forecast without fine-tuning on your specific time series.
59
+ - **Multi-Variate Support**: Efficiently process multiple variables using alternating time/variate attention.
60
+ - **Probabilistic Predictions**: Generate point forecasts and uncertainty estimates via a quantile output head.
61
+ - **Decoder-Only Architecture**: Support for variable prediction horizons and context lengths.
62
+ - **u-μP Scaling**: Stable training transfer across all model sizes.
63
+
64
+ <div style="width: 100%; margin: auto; padding: 1rem;">
65
+ <img src="figures/architecture.png" alt="Toto 2.0 architecture" style="width: 100%; height: auto;" />
66
+ <em style="display: block; margin-top: 0.5rem; text-align: center;">
67
+ Overview of the Toto 2.0 architecture.
68
+ </em>
69
+ </div>
70
+
71
+ ---
72
+
73
+ ## ⚡ Quick Start
74
+
75
+ Inference code is available on [GitHub](https://github.com/DataDog/toto).
76
+
77
+ ### Installation
78
+
79
+ ```bash
80
+ pip install "toto-2 @ git+https://github.com/DataDog/toto.git#subdirectory=toto2"
81
+ ```
82
+
83
+ ### Inference Example
84
+
85
+ ```python
86
+ import torch
87
+ from toto2 import Toto2Model
88
+
89
+ model = Toto2Model.from_pretrained("Datadog/Toto-2.0-313m")
90
+ model = model.to("cuda").eval()
91
+
92
+ # (batch, n_variates, time_steps)
93
+ target = torch.randn(1, 1, 512, device="cuda")
94
+ target_mask = torch.ones_like(target, dtype=torch.bool)
95
+ series_ids = torch.zeros(1, 1, dtype=torch.long, device="cuda")
96
+
97
+ # Returns quantiles of shape (9, batch, n_variates, horizon)
98
+ # Quantile levels: [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]
99
+ quantiles = model.forecast(
100
+ {"target": target, "target_mask": target_mask, "series_ids": series_ids},
101
+ horizon=96,
102
+ )
103
+ ```
104
+
105
+ For more examples, see the [Quick Start notebook](https://github.com/DataDog/toto/blob/main/toto2/notebooks/quick_start.ipynb) and [GluonTS integration notebook](https://github.com/DataDog/toto/blob/main/toto2/notebooks/gluonts_integration.ipynb).
106
+
107
+ ---
108
+
109
+ ## 💾 Available Checkpoints
110
+
111
+ | Checkpoint | Parameters |
112
+ |---|---|
113
+ | [Toto-2.0-4m](https://huggingface.co/Datadog/Toto-2.0-4m) | 4M |
114
+ | [Toto-2.0-22m](https://huggingface.co/Datadog/Toto-2.0-22m) | 22M |
115
+ | [Toto-2.0-313m](https://huggingface.co/Datadog/Toto-2.0-313m) | 313M |
116
+ | [Toto-2.0-1B](https://huggingface.co/Datadog/Toto-2.0-1B) | 1B |
117
+ | [Toto-2.0-2.5B](https://huggingface.co/Datadog/Toto-2.0-2.5B) | 2.5B |
118
+
119
+ ---
120
+
121
+ ## 🔗 Additional Resources
122
+
123
+ - **[Blog Post](https://www.datadoghq.com/blog/ai/toto-2/)**
124
+ - **[GitHub Repository](https://github.com/DataDog/toto)**
125
+ - **[BOOM Dataset](https://huggingface.co/datasets/Datadog/BOOM)**
126
+ - **[Toto 1.0 Weights](https://huggingface.co/Datadog/Toto-Open-Base-1.0)**
127
+
128
+ ---
129
+
130
+ ## 📖 Citation
131
+
132
+ ```bibtex
133
+ (citation coming soon)
134
+ ```