File size: 4,055 Bytes
8127dec 025dc59 8127dec 025dc59 c5b7356 8127dec 025dc59 8127dec 025dc59 c5b7356 025dc59 c5b7356 025dc59 c5b7356 025dc59 c5b7356 025dc59 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 | ---
tags:
- time-series-forecasting
- foundation-models
- pretrained-models
- time-series
- timeseries
- forecasting
- observability
- safetensors
- pytorch_model_hub_mixin
license: apache-2.0
pipeline_tag: time-series-forecasting
thumbnail:
results:
- task: time-series-forecasting
dataset:
name: GIFT-Eval
metrics:
- name: MASE
type: mase
value: 0.757
- name: CRPS
type: brier_score
value: 0.524
source:
name: GIFT-Eval Time Series Forecasting Leaderboard
url: https://huggingface.co/spaces/Salesforce/GIFT-Eval
- task: time-series-forecasting
dataset:
name: BOOM
metrics:
- name: MASE
type: mase
value: 0.624
- name: CRPS
type: brier_score
value: 0.717
source:
name: BOOM 💥 Observability Time-Series Forecasting Leaderboard
url: https://huggingface.co/spaces/Datadog/BOOM
---
# Toto-2.0-4m
Toto (**T**ime Series **O**ptimized **T**ransformer for [**O**bservability](https://www.datadoghq.com/knowledge-center/observability/)) is a family of time series foundation models for multivariate forecasting developed by [Datadog](https://www.datadoghq.com/). **Toto 2.0** is the current generation, featuring u-μP-scaled transformers ranging from 4M to 2.5B parameters.
---
## ✨ Key Features
- **Zero-Shot Forecasting**: Forecast without fine-tuning on your specific time series.
- **Multi-Variate Support**: Efficiently process multiple variables using alternating time/variate attention.
- **Probabilistic Predictions**: Generate point forecasts and uncertainty estimates via a quantile output head.
- **Decoder-Only Architecture**: Support for variable prediction horizons and context lengths.
- **u-μP Scaling**: Stable training transfer across all model sizes.
<div style="width: 100%; margin: auto; padding: 1rem;">
<img src="figures/architecture.png" alt="Toto 2.0 architecture" style="width: 100%; height: auto;" />
<em style="display: block; margin-top: 0.5rem; text-align: center;">
Overview of the Toto 2.0 architecture.
</em>
</div>
---
## ⚡ Quick Start
Inference code is available on [GitHub](https://github.com/DataDog/toto).
### Installation
```bash
pip install "toto-2 @ git+https://github.com/DataDog/toto.git#subdirectory=toto2"
```
### Inference Example
```python
import torch
from toto2 import Toto2Model
model = Toto2Model.from_pretrained("Datadog/Toto-2.0-22m")
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device).eval()
# (batch, n_variates, time_steps)
target = torch.randn(1, 1, 512, device=device)
target_mask = torch.ones_like(target, dtype=torch.bool)
series_ids = torch.zeros(1, 1, dtype=torch.long, device=device)
# Returns quantiles of shape (9, batch, n_variates, horizon)
# Quantile levels: [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]
quantiles = model.forecast(
{"target": target, "target_mask": target_mask, "series_ids": series_ids},
horizon=96,
decode_block_size=768,
has_missing_values=False,
)
```
For more examples, see the [Quick Start notebook](https://github.com/DataDog/toto/blob/main/toto2/notebooks/quick_start.ipynb) and [GluonTS integration notebook](https://github.com/DataDog/toto/blob/main/toto2/notebooks/gluonts_integration.ipynb).
---
## 💾 Available Checkpoints
| Checkpoint | Parameters |
|---|---|
| [Toto-2.0-4m](https://huggingface.co/Datadog/Toto-2.0-4m) | 4M |
| [Toto-2.0-22m](https://huggingface.co/Datadog/Toto-2.0-22m) | 22M |
| [Toto-2.0-313m](https://huggingface.co/Datadog/Toto-2.0-313m) | 313M |
| [Toto-2.0-1B](https://huggingface.co/Datadog/Toto-2.0-1B) | 1B |
| [Toto-2.0-2.5B](https://huggingface.co/Datadog/Toto-2.0-2.5B) | 2.5B |
---
## 🔗 Additional Resources
- **[GitHub Repository](https://github.com/DataDog/toto)**
- **[BOOM Dataset](https://huggingface.co/datasets/Datadog/BOOM)**
- **[Toto 1.0 Weights](https://huggingface.co/Datadog/Toto-Open-Base-1.0)**
---
## 📖 Citation
```bibtex
(citation coming soon)
```
|