Instructions to use kyLELEng/market-gpt-return-token with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use kyLELEng/market-gpt-return-token with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="kyLELEng/market-gpt-return-token")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("kyLELEng/market-gpt-return-token")
model = AutoModelForCausalLM.from_pretrained("kyLELEng/market-gpt-return-token")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use kyLELEng/market-gpt-return-token with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "kyLELEng/market-gpt-return-token"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kyLELEng/market-gpt-return-token",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/kyLELEng/market-gpt-return-token

SGLang

How to use kyLELEng/market-gpt-return-token with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "kyLELEng/market-gpt-return-token" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kyLELEng/market-gpt-return-token",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "kyLELEng/market-gpt-return-token" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kyLELEng/market-gpt-return-token",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use kyLELEng/market-gpt-return-token with Docker Model Runner:
```
docker model run hf.co/kyLELEng/market-gpt-return-token
```

kyLELEng commited on 17 days ago

Commit

c011bf2

verified ·

1 Parent(s): 8fc55f9

Train MarketGPT return-token model

Browse files

Files changed (13) hide show

README.md +496 -0
asset_list.json +14 -0
bucket_edges.json +104 -0
config.json +35 -0
cross_sectional_corr_generated.csv +13 -0
cross_sectional_corr_real.csv +13 -0
distribution_comparison.csv +17 -0
generation_config.json +7 -0
market_vocab.json +27 -0
metrics.json +428 -0
model.safetensors +3 -0
preprocess_config.json +9 -0
training_config.json +58 -0

README.md ADDED Viewed

	@@ -0,0 +1,496 @@

+---
+library_name: transformers
+tags:
+- gpt2
+- causal-lm
+- time-series
+- finance
+- return-tokenization
+- probabilistic-generation
+datasets:
+- siddharthmb/stocks-ohlcv
+---
+# MarketGPT Return-token Model
+This is a GPT-style causal language model trained from scratch on volatility-normalized return tokens.
+Instead of directly regressing future continuous returns, the pipeline converts each asset return into a discrete bucket token:
+```text
+past return tokens -> future return tokens
+```
+The model can sample multiple future return paths with constrained generation.
+## Data
+- Data source: `siddharthmb/stocks-ohlcv` when `use_synthetic=false`; synthetic heavy-tailed market panel when `use_synthetic=true`
+- Assets: `AAPL, MSFT, AMZN, GOOGL, NVDA, TSLA, AMD, INTC, JPM, BAC, V, MA`
+- Return: `log(price_t / price_t-1)`
+- Normalization: return divided by EWM volatility, span `60`
+- Buckets: `101` `quantile` buckets over clipped normalized returns
+- Split: chronological train / validation / test
+- Leakage control: quantile bucket edges are fitted only on the train period when `bucket_method=quantile`
+## Vocabulary
+```text
+PAD = 0
+BOS = 1
+EOS = 2
+STEP = 3
+asset_bucket_token = 4 + asset_id * num_buckets + bucket_id
+```
+- vocab size: `1216`
+- sequence length: `2082`
+## Model
+```python
+GPT2Config(
+    vocab_size=1216,
+    n_positions=2082,
+    n_embd=384,
+    n_layer=6,
+    n_head=6,
+)
+```
+## Metrics
+```json
+{
+  "train": {
+    "train_runtime": 55.4346,
+    "train_samples_per_second": 260.487,
+    "train_steps_per_second": 8.208,
+    "total_flos": 1920653130792960.0,
+    "train_loss": 3.8263048025277944,
+    "epoch": 5.0
+  },
+  "validation": {
+    "eval_loss": 5.103570461273193,
+    "eval_runtime": 0.5501,
+    "eval_samples_per_second": 656.204,
+    "eval_steps_per_second": 165.414,
+    "epoch": 5.0
+  },
+  "test": {
+    "test_loss": 5.131337642669678,
+    "test_runtime": 0.59,
+    "test_samples_per_second": 613.581,
+    "test_steps_per_second": 154.243,
+    "epoch": 5.0
+  },
+  "generation": {
+    "num_contexts": 12,
+    "num_sample_paths": 24,
+    "directional_accuracy_median_path": 0.5219184027777778,
+    "generated_degeneration_ratio_abs_lt_1e-5": 0.0,
+    "true_degeneration_ratio_abs_lt_1e-5": 0.0,
+    "path_diversity": 0.03765995055437088,
+    "distribution_generated": {
+      "mean": 0.004333046730607748,
+      "std": 0.08561652898788452,
+      "skew": -0.5086434483528137,
+      "kurtosis": 56.622493743896484,
+      "q01": -0.2753317952156067,
+      "q05": -0.037811942398548126,
+      "q95": 0.05145534127950668,
+      "q99": 0.33806970715522766
+    },
+    "distribution_true": {
+      "mean": -0.002802362898364663,
+      "std": 0.030968399718403816,
+      "skew": -2.6303722858428955,
+      "kurtosis": 23.258527755737305,
+      "q01": -0.09194755554199219,
+      "q05": -0.05170159786939621,
+      "q95": 0.04157276824116707,
+      "q99": 0.06143633648753166
+    },
+    "cross_sectional_corr_mae": 0.24682527780532837,
+    "cross_sectional_corr_real_matrix": [
+      [
+        1.0,
+        0.6331244707107544,
+        0.51563560962677,
+        0.7108922600746155,
+        0.5545589923858643,
+        0.5535476207733154,
+        0.5244247317314148,
+        0.21973548829555511,
+        0.3254808187484741,
+        0.296897292137146,
+        0.6087028384208679,
+        0.5639088153839111
+      ],
+      [
+        0.6331244707107544,
+        1.0,
+        0.6599540114402771,
+        0.7749960422515869,
+        0.5495731830596924,
+        0.6230116486549377,
+        0.46573472023010254,
+        0.48720842599868774,
+        0.45291945338249207,
+        0.311764121055603,
+        0.6260378360748291,
+        0.47891733050346375
+      ],
+      [
+        0.51563560962677,
+        0.6599540114402771,
+        1.0,
+        0.6038300395011902,
+        0.5262182950973511,
+        0.5929338335990906,
+        0.42031070590019226,
+        0.8166924715042114,
+        0.5714275240898132,
+        0.5298638343811035,
+        0.4197486639022827,
+        0.4404708445072174
+      ],
+      [
+        0.7108922600746155,
+        0.7749960422515869,
+        0.6038300395011902,
+        1.0,
+        0.5228630900382996,
+        0.6676064133644104,
+        0.5059135556221008,
+        0.45712900161743164,
+        0.39244410395622253,
+        0.2839842736721039,
+        0.5785690546035767,
+        0.5194220542907715
+      ],
+      [
+        0.5545589923858643,
+        0.5495731830596924,
+        0.5262182950973511,
+        0.5228630900382996,
+        1.0,
+        0.7413638830184937,
+        0.6974919438362122,
+        0.4451492726802826,
+        0.18301637470722198,
+        0.11316652595996857,
+        0.3873818814754486,
+        0.4405401945114136
+      ],
+      [
+        0.5535476207733154,
+        0.6230116486549377,
+        0.5929338335990906,
+        0.6676064133644104,
+        0.7413638830184937,
+        1.0,
+        0.626915693283081,
+        0.5260854959487915,
+        0.2714369297027588,
+        0.18484577536582947,
+        0.5772184729576111,
+        0.4420454204082489
+      ],
+      [
+        0.5244247317314148,
+        0.46573472023010254,
+        0.42031070590019226,
+        0.5059135556221008,
+        0.6974919438362122,
+        0.626915693283081,
+        1.0,
+        0.3260938227176666,
+        0.11072525382041931,
+        0.11624810099601746,
+        0.23918883502483368,
+        0.18416988849639893
+      ],
+      [
+        0.21973548829555511,
+        0.48720842599868774,
+        0.8166924715042114,
+        0.45712900161743164,
+        0.4451492726802826,
+        0.5260854959487915,
+        0.3260938227176666,
+        1.0,
+        0.587582528591156,
+        0.5751133561134338,
+        0.24537551403045654,
+        0.28593382239341736
+      ],
+      [
+        0.3254808187484741,
+        0.45291945338249207,
+        0.5714275240898132,
+        0.39244410395622253,
+        0.18301637470722198,
+        0.2714369297027588,
+        0.11072525382041931,
+        0.587582528591156,
+        1.0,
+        0.7873843312263489,
+        0.39742162823677063,
+        0.378302663564682
+      ],
+      [
+        0.296897292137146,
+        0.311764121055603,
+        0.5298638343811035,
+        0.2839842736721039,
+        0.11316652595996857,
+        0.18484577536582947,
+        0.11624810099601746,
+        0.5751133561134338,
+        0.7873843312263489,
+        1.0,
+        0.16922225058078766,
+        0.11870864033699036
+      ],
+      [
+        0.6087028384208679,
+        0.6260378360748291,
+        0.4197486639022827,
+        0.5785690546035767,
+        0.3873818814754486,
+        0.5772184729576111,
+        0.23918883502483368,
+        0.24537551403045654,
+        0.39742162823677063,
+        0.16922225058078766,
+        1.0,
+        0.8636203408241272
+      ],
+      [
+        0.5639088153839111,
+        0.47891733050346375,
+        0.4404708445072174,
+        0.5194220542907715,
+        0.4405401945114136,
+        0.4420454204082489,
+        0.18416988849639893,
+        0.28593382239341736,
+        0.378302663564682,
+        0.11870864033699036,
+        0.8636203408241272,
+        1.0
+      ]
+    ],
+    "cross_sectional_corr_generated_matrix": [
+      [
+        1.0,
+        0.6723060011863708,
+        0.5211983919143677,
+        0.42777976393699646,
+        0.3105679154396057,
+        0.15275193750858307,
+        0.09020064026117325,
+        0.05684993788599968,
+        0.027994271367788315,
+        0.021539758890867233,
+        0.0018611304694786668,
+        0.00014955938968341798
+      ],
+      [
+        0.6723060011863708,
+        1.0,
+        0.740753710269928,
+        0.5988911390304565,
+        0.4051375389099121,
+        0.19946546852588654,
+        0.12246865034103394,
+        0.07496435195207596,
+        0.041361626237630844,
+        0.03579115495085716,
+        0.0038796490989625454,
+        0.006104168947786093
+      ],
+      [
+        0.5211983919143677,
+        0.740753710269928,
+        1.0,
+        0.7721667885780334,
+        0.5105940699577332,
+        0.25858888030052185,
+        0.16144169867038727,
+        0.09339627623558044,
+        0.051841165870428085,
+        0.04847245290875435,
+        0.015083085745573044,
+        0.013859635218977928
+      ],
+      [
+        0.42777976393699646,
+        0.5988911390304565,
+        0.7721667885780334,
+        1.0,
+        0.635029137134552,
+        0.3101363778114319,
+        0.20397040247917175,
+        0.12052255123853683,
+        0.07821977138519287,
+        0.07283712923526764,
+        0.04090026021003723,
+        0.03736479952931404
+      ],
+      [
+        0.3105679154396057,
+        0.4051375389099121,
+        0.5105940699577332,
+        0.635029137134552,
+        1.0,
+        0.44570595026016235,
+        0.278595894575119,
+        0.16786691546440125,
+        0.11481305956840515,
+        0.10485804080963135,
+        0.06077614799141884,
+        0.057598281651735306
+      ],
+      [
+        0.15275193750858307,
+        0.19946546852588654,
+        0.25858888030052185,
+        0.3101363778114319,
+        0.44570595026016235,
+        1.0,
+        0.4601401388645172,
+        0.2706546485424042,
+        0.18775667250156403,
+        0.17438025772571564,
+        0.11574845016002655,
+        0.10984478890895844
+      ],
+      [
+        0.09020064026117325,
+        0.12246865034103394,
+        0.16144169867038727,
+        0.20397040247917175,
+        0.278595894575119,
+        0.4601401388645172,
+        1.0,
+        0.5125625729560852,
+        0.34511199593544006,
+        0.31734904646873474,
+        0.19516591727733612,
+        0.18664193153381348
+      ],
+      [
+        0.05684993788599968,
+        0.07496435195207596,
+        0.09339627623558044,
+        0.12052255123853683,
+        0.16786691546440125,
+        0.2706546485424042,
+        0.5125625729560852,
+        1.0,
+        0.588446855545044,
+        0.5465589761734009,
+        0.312430739402771,
+        0.29470688104629517
+      ],
+      [
+        0.027994271367788315,
+        0.041361626237630844,
+        0.051841165870428085,
+        0.07821977138519287,
+        0.11481305956840515,
+        0.18775667250156403,
+        0.34511199593544006,
+        0.588446855545044,
+        1.0,
+        0.9245684146881104,
+        0.5329838991165161,
+        0.49864715337753296
+      ],
+      [
+        0.021539758890867233,
+        0.03579115495085716,
+        0.04847245290875435,
+        0.07283712923526764,
+        0.10485804080963135,
+        0.17438025772571564,
+        0.31734904646873474,
+        0.5465589761734009,
+        0.9245684146881104,
+        1.0,
+        0.5616907477378845,
+        0.5243045091629028
+      ],
+      [
+        0.0018611304694786668,
+        0.0038796490989625454,
+        0.015083085745573044,
+        0.04090026021003723,
+        0.06077614799141884,
+        0.11574845016002655,
+        0.19516591727733612,
+        0.312430739402771,
+        0.5329838991165161,
+        0.5616907477378845,
+        1.0,
+        0.920161783695221
+      ],
+      [
+        0.00014955938968341798,
+        0.006104168947786093,
+        0.013859635218977928,
+        0.03736479952931404,
+        0.057598281651735306,
+        0.10984478890895844,
+        0.18664193153381348,
+        0.29470688104629517,
+        0.49864715337753296,
+        0.5243045091629028,
+        0.920161783695221,
+        1.0
+      ]
+    ],
+    "generated_abs_return_autocorr_lag1": 0.019996171948218167,
+    "true_abs_return_autocorr_lag1": 0.2633381616777056
+  },
+  "data": {
+    "tickers": [
+      "AAPL",
+      "MSFT",
+      "AMZN",
+      "GOOGL",
+      "NVDA",
+      "TSLA",
+      "AMD",
+      "INTC",
+      "JPM",
+      "BAC",
+      "V",
+      "MA"
+    ],
+    "price_shape": [
+      3790,
+      12
+    ],
+    "return_shape": [
+      3770,
+      12
+    ],
+    "bucket_method": "quantile",
+    "num_samples": 3611,
+    "train_samples": 2888,
+    "validation_samples": 361,
+    "test_samples": 362,
+    "sequence_length": 2082,
+    "vocab_size": 1216,
+    "num_parameters": 11913984,
+    "data_source": "siddharthmb/stocks-ohlcv"
+  }
+}
+```
+## Intended Use
+Research and experimentation with market return sequence generation. This model is not a trading system and is not investment advice.

asset_list.json ADDED Viewed

	@@ -0,0 +1,14 @@

+[
+  "AAPL",
+  "MSFT",
+  "AMZN",
+  "GOOGL",
+  "NVDA",
+  "TSLA",
+  "AMD",
+  "INTC",
+  "JPM",
+  "BAC",
+  "V",
+  "MA"
+]

bucket_edges.json ADDED Viewed

	@@ -0,0 +1,104 @@

+[
+  -5.0,
+  -2.5640621185302734,
+  -2.0861096382141113,
+  -1.8310669660568237,
+  -1.6588560342788696,
+  -1.5273399353027344,
+  -1.412660837173462,
+  -1.3142497539520264,
+  -1.2324836254119873,
+  -1.1628164052963257,
+  -1.0965759754180908,
+  -1.0370841026306152,
+  -0.9799642562866211,
+  -0.9281984567642212,
+  -0.8819820284843445,
+  -0.8356001377105713,
+  -0.7926501631736755,
+  -0.755145788192749,
+  -0.7170299887657166,
+  -0.678482711315155,
+  -0.6437629461288452,
+  -0.6132894158363342,
+  -0.5807541012763977,
+  -0.5498068332672119,
+  -0.520163893699646,
+  -0.48915788531303406,
+  -0.46171873807907104,
+  -0.4333514869213104,
+  -0.4081476032733917,
+  -0.38186678290367126,
+  -0.3573564887046814,
+  -0.3329995274543762,
+  -0.3090900182723999,
+  -0.2860213816165924,
+  -0.2642545998096466,
+  -0.24296307563781738,
+  -0.22225084900856018,
+  -0.20184440910816193,
+  -0.18016014993190765,
+  -0.16107362508773804,
+  -0.14016427099704742,
+  -0.12129930406808853,
+  -0.10306043177843094,
+  -0.0838770791888237,
+  -0.0651112049818039,
+  -0.048510488122701645,
+  -0.03162214905023575,
+  -0.012857679277658463,
+  0.0,
+  0.003500344231724739,
+  0.024453746154904366,
+  0.042099423706531525,
+  0.05944214388728142,
+  0.07699255645275116,
+  0.09591319411993027,
+  0.11562864482402802,
+  0.134685680270195,
+  0.15423625707626343,
+  0.175302654504776,
+  0.1944105327129364,
+  0.21569515764713287,
+  0.23659121990203857,
+  0.25711652636528015,
+  0.27793654799461365,
+  0.29974064230918884,
+  0.32058408856391907,
+  0.3441528081893921,
+  0.3674743175506592,
+  0.3901389539241791,
+  0.41622453927993774,
+  0.439811646938324,
+  0.46468427777290344,
+  0.48801037669181824,
+  0.5133446455001831,
+  0.5387183427810669,
+  0.5662073493003845,
+  0.5953604578971863,
+  0.6257803440093994,
+  0.6574458479881287,
+  0.6881090998649597,
+  0.7192966938018799,
+  0.7515752911567688,
+  0.7850026488304138,
+  0.8196272850036621,
+  0.856989324092865,
+  0.8949403762817383,
+  0.9338142275810242,
+  0.9723265767097473,
+  1.016391396522522,
+  1.0668745040893555,
+  1.1130402088165283,
+  1.169913411140442,
+  1.2257028818130493,
+  1.2931857109069824,
+  1.3646200895309448,
+  1.4440101385116577,
+  1.539734125137329,
+  1.6749216318130493,
+  1.8327667713165283,
+  2.051732301712036,
+  2.4239859580993652,
+  5.0
+]

config.json ADDED Viewed

	@@ -0,0 +1,35 @@

+{
+  "activation_function": "gelu_new",
+  "add_cross_attention": false,
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.1,
+  "bos_token_id": 1,
+  "dtype": "float32",
+  "embd_pdrop": 0.1,
+  "eos_token_id": 2,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_ctx": 2082,
+  "n_embd": 384,
+  "n_head": 6,
+  "n_inner": null,
+  "n_layer": 6,
+  "n_positions": 2082,
+  "pad_token_id": 0,
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.1,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.6.2",
+  "use_cache": false,
+  "vocab_size": 1216
+}

cross_sectional_corr_generated.csv ADDED Viewed

	@@ -0,0 +1,13 @@

+,AAPL,MSFT,AMZN,GOOGL,NVDA,TSLA,AMD,INTC,JPM,BAC,V,MA
+AAPL,1.0,0.6723060011863708,0.5211983919143677,0.42777976393699646,0.3105679154396057,0.15275193750858307,0.09020064026117325,0.05684993788599968,0.027994271367788315,0.021539758890867233,0.0018611304694786668,0.00014955938968341798
+MSFT,0.6723060011863708,1.0,0.740753710269928,0.5988911390304565,0.4051375389099121,0.19946546852588654,0.12246865034103394,0.07496435195207596,0.041361626237630844,0.03579115495085716,0.0038796490989625454,0.006104168947786093
+AMZN,0.5211983919143677,0.740753710269928,1.0,0.7721667885780334,0.5105940699577332,0.25858888030052185,0.16144169867038727,0.09339627623558044,0.051841165870428085,0.04847245290875435,0.015083085745573044,0.013859635218977928
+GOOGL,0.42777976393699646,0.5988911390304565,0.7721667885780334,1.0,0.635029137134552,0.3101363778114319,0.20397040247917175,0.12052255123853683,0.07821977138519287,0.07283712923526764,0.04090026021003723,0.03736479952931404
+NVDA,0.3105679154396057,0.4051375389099121,0.5105940699577332,0.635029137134552,1.0,0.44570595026016235,0.278595894575119,0.16786691546440125,0.11481305956840515,0.10485804080963135,0.06077614799141884,0.057598281651735306
+TSLA,0.15275193750858307,0.19946546852588654,0.25858888030052185,0.3101363778114319,0.44570595026016235,1.0,0.4601401388645172,0.2706546485424042,0.18775667250156403,0.17438025772571564,0.11574845016002655,0.10984478890895844
+AMD,0.09020064026117325,0.12246865034103394,0.16144169867038727,0.20397040247917175,0.278595894575119,0.4601401388645172,1.0,0.5125625729560852,0.34511199593544006,0.31734904646873474,0.19516591727733612,0.18664193153381348
+INTC,0.05684993788599968,0.07496435195207596,0.09339627623558044,0.12052255123853683,0.16786691546440125,0.2706546485424042,0.5125625729560852,1.0,0.588446855545044,0.5465589761734009,0.312430739402771,0.29470688104629517
+JPM,0.027994271367788315,0.041361626237630844,0.051841165870428085,0.07821977138519287,0.11481305956840515,0.18775667250156403,0.34511199593544006,0.588446855545044,1.0,0.9245684146881104,0.5329838991165161,0.49864715337753296
+BAC,0.021539758890867233,0.03579115495085716,0.04847245290875435,0.07283712923526764,0.10485804080963135,0.17438025772571564,0.31734904646873474,0.5465589761734009,0.9245684146881104,1.0,0.5616907477378845,0.5243045091629028
+V,0.0018611304694786668,0.0038796490989625454,0.015083085745573044,0.04090026021003723,0.06077614799141884,0.11574845016002655,0.19516591727733612,0.312430739402771,0.5329838991165161,0.5616907477378845,1.0,0.920161783695221
+MA,0.00014955938968341798,0.006104168947786093,0.013859635218977928,0.03736479952931404,0.057598281651735306,0.10984478890895844,0.18664193153381348,0.29470688104629517,0.49864715337753296,0.5243045091629028,0.920161783695221,1.0

cross_sectional_corr_real.csv ADDED Viewed

	@@ -0,0 +1,13 @@

+,AAPL,MSFT,AMZN,GOOGL,NVDA,TSLA,AMD,INTC,JPM,BAC,V,MA
+AAPL,1.0,0.6331244707107544,0.51563560962677,0.7108922600746155,0.5545589923858643,0.5535476207733154,0.5244247317314148,0.21973548829555511,0.3254808187484741,0.296897292137146,0.6087028384208679,0.5639088153839111
+MSFT,0.6331244707107544,1.0,0.6599540114402771,0.7749960422515869,0.5495731830596924,0.6230116486549377,0.46573472023010254,0.48720842599868774,0.45291945338249207,0.311764121055603,0.6260378360748291,0.47891733050346375
+AMZN,0.51563560962677,0.6599540114402771,1.0,0.6038300395011902,0.5262182950973511,0.5929338335990906,0.42031070590019226,0.8166924715042114,0.5714275240898132,0.5298638343811035,0.4197486639022827,0.4404708445072174
+GOOGL,0.7108922600746155,0.7749960422515869,0.6038300395011902,1.0,0.5228630900382996,0.6676064133644104,0.5059135556221008,0.45712900161743164,0.39244410395622253,0.2839842736721039,0.5785690546035767,0.5194220542907715
+NVDA,0.5545589923858643,0.5495731830596924,0.5262182950973511,0.5228630900382996,1.0,0.7413638830184937,0.6974919438362122,0.4451492726802826,0.18301637470722198,0.11316652595996857,0.3873818814754486,0.4405401945114136
+TSLA,0.5535476207733154,0.6230116486549377,0.5929338335990906,0.6676064133644104,0.7413638830184937,1.0,0.626915693283081,0.5260854959487915,0.2714369297027588,0.18484577536582947,0.5772184729576111,0.4420454204082489
+AMD,0.5244247317314148,0.46573472023010254,0.42031070590019226,0.5059135556221008,0.6974919438362122,0.626915693283081,1.0,0.3260938227176666,0.11072525382041931,0.11624810099601746,0.23918883502483368,0.18416988849639893
+INTC,0.21973548829555511,0.48720842599868774,0.8166924715042114,0.45712900161743164,0.4451492726802826,0.5260854959487915,0.3260938227176666,1.0,0.587582528591156,0.5751133561134338,0.24537551403045654,0.28593382239341736
+JPM,0.3254808187484741,0.45291945338249207,0.5714275240898132,0.39244410395622253,0.18301637470722198,0.2714369297027588,0.11072525382041931,0.587582528591156,1.0,0.7873843312263489,0.39742162823677063,0.378302663564682
+BAC,0.296897292137146,0.311764121055603,0.5298638343811035,0.2839842736721039,0.11316652595996857,0.18484577536582947,0.11624810099601746,0.5751133561134338,0.7873843312263489,1.0,0.16922225058078766,0.11870864033699036
+V,0.6087028384208679,0.6260378360748291,0.4197486639022827,0.5785690546035767,0.3873818814754486,0.5772184729576111,0.23918883502483368,0.24537551403045654,0.39742162823677063,0.16922225058078766,1.0,0.8636203408241272
+MA,0.5639088153839111,0.47891733050346375,0.4404708445072174,0.5194220542907715,0.4405401945114136,0.4420454204082489,0.18416988849639893,0.28593382239341736,0.378302663564682,0.11870864033699036,0.8636203408241272,1.0

distribution_comparison.csv ADDED Viewed

	@@ -0,0 +1,17 @@

+series,metric,value
+generated,mean,0.004333046730607748
+generated,std,0.08561652898788452
+generated,skew,-0.5086434483528137
+generated,kurtosis,56.622493743896484
+generated,q01,-0.2753317952156067
+generated,q05,-0.037811942398548126
+generated,q95,0.05145534127950668
+generated,q99,0.33806970715522766
+true,mean,-0.002802362898364663
+true,std,0.030968399718403816
+true,skew,-2.6303722858428955
+true,kurtosis,23.258527755737305
+true,q01,-0.09194755554199219
+true,q05,-0.05170159786939621
+true,q95,0.04157276824116707
+true,q99,0.06143633648753166

generation_config.json ADDED Viewed

	@@ -0,0 +1,7 @@

+{
+  "do_sample": true,
+  "temperature": 1.0,
+  "top_p": 0.95,
+  "constrained_generation": true,
+  "max_new_tokens": 416
+}

market_vocab.json ADDED Viewed

	@@ -0,0 +1,27 @@

+{
+  "special": {
+    "PAD": 0,
+    "BOS": 1,
+    "EOS": 2,
+    "STEP": 3
+  },
+  "offset": 4,
+  "num_assets": 12,
+  "num_buckets": 101,
+  "vocab_size": 1216,
+  "token_formula": "token_id = offset + asset_id * num_buckets + bucket_id",
+  "asset_list": [
+    "AAPL",
+    "MSFT",
+    "AMZN",
+    "GOOGL",
+    "NVDA",
+    "TSLA",
+    "AMD",
+    "INTC",
+    "JPM",
+    "BAC",
+    "V",
+    "MA"
+  ]
+}

metrics.json ADDED Viewed

	@@ -0,0 +1,428 @@

+{
+  "train": {
+    "train_runtime": 55.4346,
+    "train_samples_per_second": 260.487,
+    "train_steps_per_second": 8.208,
+    "total_flos": 1920653130792960.0,
+    "train_loss": 3.8263048025277944,
+    "epoch": 5.0
+  },
+  "validation": {
+    "eval_loss": 5.103570461273193,
+    "eval_runtime": 0.5501,
+    "eval_samples_per_second": 656.204,
+    "eval_steps_per_second": 165.414,
+    "epoch": 5.0
+  },
+  "test": {
+    "test_loss": 5.131337642669678,
+    "test_runtime": 0.59,
+    "test_samples_per_second": 613.581,
+    "test_steps_per_second": 154.243,
+    "epoch": 5.0
+  },
+  "generation": {
+    "num_contexts": 12,
+    "num_sample_paths": 24,
+    "directional_accuracy_median_path": 0.5219184027777778,
+    "generated_degeneration_ratio_abs_lt_1e-5": 0.0,
+    "true_degeneration_ratio_abs_lt_1e-5": 0.0,
+    "path_diversity": 0.03765995055437088,
+    "distribution_generated": {
+      "mean": 0.004333046730607748,
+      "std": 0.08561652898788452,
+      "skew": -0.5086434483528137,
+      "kurtosis": 56.622493743896484,
+      "q01": -0.2753317952156067,
+      "q05": -0.037811942398548126,
+      "q95": 0.05145534127950668,
+      "q99": 0.33806970715522766
+    },
+    "distribution_true": {
+      "mean": -0.002802362898364663,
+      "std": 0.030968399718403816,
+      "skew": -2.6303722858428955,
+      "kurtosis": 23.258527755737305,
+      "q01": -0.09194755554199219,
+      "q05": -0.05170159786939621,
+      "q95": 0.04157276824116707,
+      "q99": 0.06143633648753166
+    },
+    "cross_sectional_corr_mae": 0.24682527780532837,
+    "cross_sectional_corr_real_matrix": [
+      [
+        1.0,
+        0.6331244707107544,
+        0.51563560962677,
+        0.7108922600746155,
+        0.5545589923858643,
+        0.5535476207733154,
+        0.5244247317314148,
+        0.21973548829555511,
+        0.3254808187484741,
+        0.296897292137146,
+        0.6087028384208679,
+        0.5639088153839111
+      ],
+      [
+        0.6331244707107544,
+        1.0,
+        0.6599540114402771,
+        0.7749960422515869,
+        0.5495731830596924,
+        0.6230116486549377,
+        0.46573472023010254,
+        0.48720842599868774,
+        0.45291945338249207,
+        0.311764121055603,
+        0.6260378360748291,
+        0.47891733050346375
+      ],
+      [
+        0.51563560962677,
+        0.6599540114402771,
+        1.0,
+        0.6038300395011902,
+        0.5262182950973511,
+        0.5929338335990906,
+        0.42031070590019226,
+        0.8166924715042114,
+        0.5714275240898132,
+        0.5298638343811035,
+        0.4197486639022827,
+        0.4404708445072174
+      ],
+      [
+        0.7108922600746155,
+        0.7749960422515869,
+        0.6038300395011902,
+        1.0,
+        0.5228630900382996,
+        0.6676064133644104,
+        0.5059135556221008,
+        0.45712900161743164,
+        0.39244410395622253,
+        0.2839842736721039,
+        0.5785690546035767,
+        0.5194220542907715
+      ],
+      [
+        0.5545589923858643,
+        0.5495731830596924,
+        0.5262182950973511,
+        0.5228630900382996,
+        1.0,
+        0.7413638830184937,
+        0.6974919438362122,
+        0.4451492726802826,
+        0.18301637470722198,
+        0.11316652595996857,
+        0.3873818814754486,
+        0.4405401945114136
+      ],
+      [
+        0.5535476207733154,
+        0.6230116486549377,
+        0.5929338335990906,
+        0.6676064133644104,
+        0.7413638830184937,
+        1.0,
+        0.626915693283081,
+        0.5260854959487915,
+        0.2714369297027588,
+        0.18484577536582947,
+        0.5772184729576111,
+        0.4420454204082489
+      ],
+      [
+        0.5244247317314148,
+        0.46573472023010254,
+        0.42031070590019226,
+        0.5059135556221008,
+        0.6974919438362122,
+        0.626915693283081,
+        1.0,
+        0.3260938227176666,
+        0.11072525382041931,
+        0.11624810099601746,
+        0.23918883502483368,
+        0.18416988849639893
+      ],
+      [
+        0.21973548829555511,
+        0.48720842599868774,
+        0.8166924715042114,
+        0.45712900161743164,
+        0.4451492726802826,
+        0.5260854959487915,
+        0.3260938227176666,
+        1.0,
+        0.587582528591156,
+        0.5751133561134338,
+        0.24537551403045654,
+        0.28593382239341736
+      ],
+      [
+        0.3254808187484741,
+        0.45291945338249207,
+        0.5714275240898132,
+        0.39244410395622253,
+        0.18301637470722198,
+        0.2714369297027588,
+        0.11072525382041931,
+        0.587582528591156,
+        1.0,
+        0.7873843312263489,
+        0.39742162823677063,
+        0.378302663564682
+      ],
+      [
+        0.296897292137146,
+        0.311764121055603,
+        0.5298638343811035,
+        0.2839842736721039,
+        0.11316652595996857,
+        0.18484577536582947,
+        0.11624810099601746,
+        0.5751133561134338,
+        0.7873843312263489,
+        1.0,
+        0.16922225058078766,
+        0.11870864033699036
+      ],
+      [
+        0.6087028384208679,
+        0.6260378360748291,
+        0.4197486639022827,
+        0.5785690546035767,
+        0.3873818814754486,
+        0.5772184729576111,
+        0.23918883502483368,
+        0.24537551403045654,
+        0.39742162823677063,
+        0.16922225058078766,
+        1.0,
+        0.8636203408241272
+      ],
+      [
+        0.5639088153839111,
+        0.47891733050346375,
+        0.4404708445072174,
+        0.5194220542907715,
+        0.4405401945114136,
+        0.4420454204082489,
+        0.18416988849639893,
+        0.28593382239341736,
+        0.378302663564682,
+        0.11870864033699036,
+        0.8636203408241272,
+        1.0
+      ]
+    ],
+    "cross_sectional_corr_generated_matrix": [
+      [
+        1.0,
+        0.6723060011863708,
+        0.5211983919143677,
+        0.42777976393699646,
+        0.3105679154396057,
+        0.15275193750858307,
+        0.09020064026117325,
+        0.05684993788599968,
+        0.027994271367788315,
+        0.021539758890867233,
+        0.0018611304694786668,
+        0.00014955938968341798
+      ],
+      [
+        0.6723060011863708,
+        1.0,
+        0.740753710269928,
+        0.5988911390304565,
+        0.4051375389099121,
+        0.19946546852588654,
+        0.12246865034103394,
+        0.07496435195207596,
+        0.041361626237630844,
+        0.03579115495085716,
+        0.0038796490989625454,
+        0.006104168947786093
+      ],
+      [
+        0.5211983919143677,
+        0.740753710269928,
+        1.0,
+        0.7721667885780334,
+        0.5105940699577332,
+        0.25858888030052185,
+        0.16144169867038727,
+        0.09339627623558044,
+        0.051841165870428085,
+        0.04847245290875435,
+        0.015083085745573044,
+        0.013859635218977928
+      ],
+      [
+        0.42777976393699646,
+        0.5988911390304565,
+        0.7721667885780334,
+        1.0,
+        0.635029137134552,
+        0.3101363778114319,
+        0.20397040247917175,
+        0.12052255123853683,
+        0.07821977138519287,
+        0.07283712923526764,
+        0.04090026021003723,
+        0.03736479952931404
+      ],
+      [
+        0.3105679154396057,
+        0.4051375389099121,
+        0.5105940699577332,
+        0.635029137134552,
+        1.0,
+        0.44570595026016235,
+        0.278595894575119,
+        0.16786691546440125,
+        0.11481305956840515,
+        0.10485804080963135,
+        0.06077614799141884,
+        0.057598281651735306
+      ],
+      [
+        0.15275193750858307,
+        0.19946546852588654,
+        0.25858888030052185,
+        0.3101363778114319,
+        0.44570595026016235,
+        1.0,
+        0.4601401388645172,
+        0.2706546485424042,
+        0.18775667250156403,
+        0.17438025772571564,
+        0.11574845016002655,
+        0.10984478890895844
+      ],
+      [
+        0.09020064026117325,
+        0.12246865034103394,
+        0.16144169867038727,
+        0.20397040247917175,
+        0.278595894575119,
+        0.4601401388645172,
+        1.0,
+        0.5125625729560852,
+        0.34511199593544006,
+        0.31734904646873474,
+        0.19516591727733612,
+        0.18664193153381348
+      ],
+      [
+        0.05684993788599968,
+        0.07496435195207596,
+        0.09339627623558044,
+        0.12052255123853683,
+        0.16786691546440125,
+        0.2706546485424042,
+        0.5125625729560852,
+        1.0,
+        0.588446855545044,
+        0.5465589761734009,
+        0.312430739402771,
+        0.29470688104629517
+      ],
+      [
+        0.027994271367788315,
+        0.041361626237630844,
+        0.051841165870428085,
+        0.07821977138519287,
+        0.11481305956840515,
+        0.18775667250156403,
+        0.34511199593544006,
+        0.588446855545044,
+        1.0,
+        0.9245684146881104,
+        0.5329838991165161,
+        0.49864715337753296
+      ],
+      [
+        0.021539758890867233,
+        0.03579115495085716,
+        0.04847245290875435,
+        0.07283712923526764,
+        0.10485804080963135,
+        0.17438025772571564,
+        0.31734904646873474,
+        0.5465589761734009,
+        0.9245684146881104,
+        1.0,
+        0.5616907477378845,
+        0.5243045091629028
+      ],
+      [
+        0.0018611304694786668,
+        0.0038796490989625454,
+        0.015083085745573044,
+        0.04090026021003723,
+        0.06077614799141884,
+        0.11574845016002655,
+        0.19516591727733612,
+        0.312430739402771,
+        0.5329838991165161,
+        0.5616907477378845,
+        1.0,
+        0.920161783695221
+      ],
+      [
+        0.00014955938968341798,
+        0.006104168947786093,
+        0.013859635218977928,
+        0.03736479952931404,
+        0.057598281651735306,
+        0.10984478890895844,
+        0.18664193153381348,
+        0.29470688104629517,
+        0.49864715337753296,
+        0.5243045091629028,
+        0.920161783695221,
+        1.0
+      ]
+    ],
+    "generated_abs_return_autocorr_lag1": 0.019996171948218167,
+    "true_abs_return_autocorr_lag1": 0.2633381616777056
+  },
+  "data": {
+    "tickers": [
+      "AAPL",
+      "MSFT",
+      "AMZN",
+      "GOOGL",
+      "NVDA",
+      "TSLA",
+      "AMD",
+      "INTC",
+      "JPM",
+      "BAC",
+      "V",
+      "MA"
+    ],
+    "price_shape": [
+      3790,
+      12
+    ],
+    "return_shape": [
+      3770,
+      12
+    ],
+    "bucket_method": "quantile",
+    "num_samples": 3611,
+    "train_samples": 2888,
+    "validation_samples": 361,
+    "test_samples": 362,
+    "sequence_length": 2082,
+    "vocab_size": 1216,
+    "num_parameters": 11913984,
+    "data_source": "siddharthmb/stocks-ohlcv"
+  }
+}

model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:95b41406ef41a541af34ce81fad31bbb9b77081d41b33e43ed4c6b200aba29d2
+size 47663480

preprocess_config.json ADDED Viewed

	@@ -0,0 +1,9 @@

+{
+  "return": "log(price_t / price_t-1)",
+  "volatility_normalization": "return / ewm_volatility",
+  "vol_span": 60,
+  "bucket_method": "quantile",
+  "bucket_edge_fit_period": "train_period_only",
+  "num_buckets": 101,
+  "clip_value": 5.0
+}

training_config.json ADDED Viewed

	@@ -0,0 +1,58 @@

+{
+  "model_repo_id": "kyLELEng/market-gpt-return-token-v2",
+  "dataset_id": "siddharthmb/stocks-ohlcv",
+  "dataset_file": "ohlcv.csv",
+  "use_synthetic": false,
+  "synthetic_steps": 900,
+  "tickers": [
+    "AAPL",
+    "MSFT",
+    "AMZN",
+    "GOOGL",
+    "NVDA",
+    "TSLA",
+    "AMD",
+    "INTC",
+    "JPM",
+    "BAC",
+    "V",
+    "MA"
+  ],
+  "num_assets": 12,
+  "num_buckets": 101,
+  "bucket_method": "quantile",
+  "clip_value": 5.0,
+  "vol_span": 60,
+  "context_steps": 128,
+  "prediction_steps": 32,
+  "window_stride": 1,
+  "train_fraction": 0.8,
+  "validation_fraction": 0.1,
+  "n_embd": 384,
+  "n_layer": 6,
+  "n_head": 6,
+  "dropout": 0.1,
+  "forecast_only_loss": true,
+  "per_device_train_batch_size": 4,
+  "per_device_eval_batch_size": 4,
+  "gradient_accumulation_steps": 8,
+  "learning_rate": 0.0003,
+  "weight_decay": 0.1,
+  "warmup_ratio": 0.05,
+  "num_train_epochs": 5.0,
+  "max_steps": -1,
+  "logging_steps": 20,
+  "eval_strategy": "epoch",
+  "save_strategy": "no",
+  "bf16": true,
+  "fp16": false,
+  "random_seed": 7,
+  "num_eval_contexts": 12,
+  "num_sample_paths": 24,
+  "generation_temperature": 1.0,
+  "generation_top_p": 0.95,
+  "max_csv_chunks": null,
+  "csv_chunksize": 500000,
+  "min_rows_per_ticker": 1000,
+  "output_dir": "market-gpt-return-token"
+}