Hugging Face's logo

chandar-lab
/
copep-checkpoints

Transformers
Safetensors
protein
biology
protein-language-model
continual-learning
Model card Files
xet
 Community
dapatil211 commited on
Commit
021848e
·
1 Parent(s): bbf4ad5

Upload 90 AMPLIFY continual-learning checkpoints

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +61 -0
  2. continual/task_1/config.json +26 -0
  3. continual/task_1/model.safetensors +3 -0
  4. continual/task_2/config.json +26 -0
  5. continual/task_2/model.safetensors +3 -0
  6. continual/task_3/config.json +26 -0
  7. continual/task_3/model.safetensors +3 -0
  8. continual/task_4/config.json +26 -0
  9. continual/task_4/model.safetensors +3 -0
  10. continual/task_5/config.json +26 -0
  11. continual/task_5/model.safetensors +3 -0
  12. continual/task_6/config.json +26 -0
  13. continual/task_6/model.safetensors +3 -0
  14. continual/task_7/config.json +26 -0
  15. continual/task_7/model.safetensors +3 -0
  16. continual/task_8/config.json +26 -0
  17. continual/task_8/model.safetensors +3 -0
  18. continual/task_9/config.json +26 -0
  19. continual/task_9/model.safetensors +3 -0
  20. gradient_ascent/task_1/config.json +26 -0
  21. gradient_ascent/task_1/model.safetensors +3 -0
  22. gradient_ascent/task_2/config.json +26 -0
  23. gradient_ascent/task_2/model.safetensors +3 -0
  24. gradient_ascent/task_3/config.json +26 -0
  25. gradient_ascent/task_3/model.safetensors +3 -0
  26. gradient_ascent/task_4/config.json +26 -0
  27. gradient_ascent/task_4/model.safetensors +3 -0
  28. gradient_ascent/task_5/config.json +26 -0
  29. gradient_ascent/task_5/model.safetensors +3 -0
  30. gradient_ascent/task_6/config.json +26 -0
  31. gradient_ascent/task_6/model.safetensors +3 -0
  32. gradient_ascent/task_7/config.json +26 -0
  33. gradient_ascent/task_7/model.safetensors +3 -0
  34. gradient_ascent/task_8/config.json +26 -0
  35. gradient_ascent/task_8/model.safetensors +3 -0
  36. gradient_ascent/task_9/config.json +26 -0
  37. gradient_ascent/task_9/model.safetensors +3 -0
  38. hare_tortoise/task_0/config.json +26 -0
  39. hare_tortoise/task_0/model.safetensors +3 -0
  40. hare_tortoise/task_1/config.json +26 -0
  41. hare_tortoise/task_1/model.safetensors +3 -0
  42. hare_tortoise/task_2/config.json +26 -0
  43. hare_tortoise/task_2/model.safetensors +3 -0
  44. hare_tortoise/task_3/config.json +26 -0
  45. hare_tortoise/task_3/model.safetensors +3 -0
  46. hare_tortoise/task_4/config.json +26 -0
  47. hare_tortoise/task_4/model.safetensors +3 -0
  48. hare_tortoise/task_5/config.json +26 -0
  49. hare_tortoise/task_5/model.safetensors +3 -0
  50. hare_tortoise/task_6/config.json +26 -0
README.md ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - protein
5
+ - biology
6
+ - protein-language-model
7
+ - continual-learning
8
+ library_name: transformers
9
+ ---
10
+
11
+ # CoPeP Continual Learning Checkpoints
12
+
13
+ This repository contains **90 checkpoints** from continual
14
+ learning experiments with the [AMPLIFY](https://huggingface.co/chandar-lab/AMPLIFY_120M)
15
+ protein language model (120M parameters).
16
+
17
+ ## Loading a checkpoint
18
+
19
+ ```python
20
+ from transformers import AutoModel
21
+
22
+ model = AutoModel.from_pretrained(
23
+ "chandar-lab/copep-checkpoints",
24
+ subfolder="replay/task_5",
25
+ trust_remote_code=True,
26
+ )
27
+ ```
28
+
29
+ ## Available checkpoints
30
+
31
+ | Method | Tasks |
32
+ |--------|-------|
33
+ | `continual` | task_0, task_1, task_2, task_3, task_4, task_5, task_6, task_7, task_8, task_9 |
34
+ | `gradient_ascent` | task_0, task_1, task_2, task_3, task_4, task_5, task_6, task_7, task_8, task_9 |
35
+ | `hare_tortoise` | task_0, task_1, task_2, task_3, task_4, task_5, task_6, task_7, task_8, task_9 |
36
+ | `joint` | task_0, task_1, task_2, task_3, task_4, task_5, task_6, task_7, task_8, task_9 |
37
+ | `match` | task_0, task_1, task_2, task_3, task_4, task_5, task_6, task_7, task_8, task_9 |
38
+ | `random_labels` | task_0, task_1, task_2, task_3, task_4, task_5, task_6, task_7, task_8, task_9 |
39
+ | `replay` | task_0, task_1, task_2, task_3, task_4, task_5, task_6, task_7, task_8, task_9 |
40
+ | `shrink_perturb` | task_0, task_1, task_2, task_3, task_4, task_5, task_6, task_7, task_8, task_9 |
41
+ | `single_year` | task_0, task_1, task_2, task_3, task_4, task_5, task_6, task_7, task_8, task_9 |
42
+
43
+
44
+ Each `task_N` subfolder contains a `config.json` and `model.safetensors`.
45
+
46
+ ### Task mapping
47
+
48
+ - **task_0** : pre-2004 (base model)
49
+ - **task_1** – **task_9** : successive temporal splits of UniRef data
50
+
51
+ For methods that start from task_1 (continual, gradient_ascent, match,
52
+ random_labels, replay, shrink_perturb), `task_0` is the same checkpoint as
53
+ `single_year/task_0` (the base pre-trained model).
54
+
55
+ ## Model architecture
56
+
57
+ - **Architecture:** Transformer encoder with RoPE + SwiGLU
58
+ - **Parameters:** ~120M
59
+ - **Config:** hidden_size=640, num_hidden_layers=24, num_attention_heads=10, intermediate_size=2560
60
+ - **Vocab size:** 32 (amino acid tokens + special tokens)
61
+ - **Max length:** 512 (training), 50000 (inference with RoPE extrapolation)
continual/task_1/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
continual/task_1/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5854b33a5d05a4fb2db7eaaa8b333340719dfa9c2f8a728da5340e2a13d11f20
3
+ size 473147704
continual/task_2/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
continual/task_2/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:51f93bcdbe66289adb006b0ebbcf5968ece93a3f8fc5a4b479745dad1d323b16
3
+ size 473147704
continual/task_3/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
continual/task_3/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:45d14cd6b757ef21f7f63569f106fc9958d740dd6aacbb9b127e523feebe825c
3
+ size 473147704
continual/task_4/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
continual/task_4/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:df6c39c0c5beb60a3ba0f8e53e9068ed9a09445e235cdc2c340ae29c70ca0455
3
+ size 473147704
continual/task_5/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
continual/task_5/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:893b88e2378e9ff0343aa06c727980719db4d8a5a1adc126da6a21c34fea0dae
3
+ size 473147704
continual/task_6/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
continual/task_6/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3acb066e6e48c67d9ee8c04fe274407313859091aa858135adb0d8641da262bb
3
+ size 473147704
continual/task_7/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
continual/task_7/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7eb9c779ded16143d9a0e8ce2ef1b81d2863753ecc6223d4190317864b4d710d
3
+ size 473147704
continual/task_8/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
continual/task_8/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1a5777d89471265d3897f8fe389628bf28cfedb86ee9ab0b37f8857819579447
3
+ size 473147704
continual/task_9/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
continual/task_9/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ed8300d772972c5c84012ccb5851c69374f6d723919f0caa9c24e5581198e484
3
+ size 473147704
gradient_ascent/task_1/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
gradient_ascent/task_1/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3e65f4b8bb3660a233fb41897e43681d2d75e7528cc013889e76ac42d0bda3af
3
+ size 473147704
gradient_ascent/task_2/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
gradient_ascent/task_2/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d0ac712877b66ab78c6e1f0ef284b6960a7b79f52e7e1cc09c12165781fb8040
3
+ size 473147704
gradient_ascent/task_3/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
gradient_ascent/task_3/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:56f0e79a3a921a1cddb916f4ee516d5e0bbe04a80e843cda2a2e65f7a41aca40
3
+ size 473147704
gradient_ascent/task_4/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
gradient_ascent/task_4/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fc3bed16f96d632a411ff9cbbb01776e1ddd2e57febca574cacd20732c537bac
3
+ size 473147704
gradient_ascent/task_5/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
gradient_ascent/task_5/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0507cb1bc4dd762a6bf77faf75f8da4a6354a2200fea627aeeeb48abc485ad8e
3
+ size 473147704
gradient_ascent/task_6/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
gradient_ascent/task_6/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d9c8139356df4bb8fb11561172229eb52642214af11e637ac343d3d6b9c4af9f
3
+ size 473147704
gradient_ascent/task_7/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
gradient_ascent/task_7/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e305e7ea855ca38fc8e17db10e2c6aece6538e6abb34d8171a5ebe4f723c6967
3
+ size 473147704
gradient_ascent/task_8/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
gradient_ascent/task_8/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b4a75a716073505887967efddb9085c6a0fa205a2c6cfd4d9e78ba0f476003d3
3
+ size 473147704
gradient_ascent/task_9/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
gradient_ascent/task_9/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1046bea5e85e820bc33258ed050455b238132add873430bd8f5e42cb6ed9cee2
3
+ size 473147704
hare_tortoise/task_0/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
hare_tortoise/task_0/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:30c4c1d1b97fa3e741c8df173ae5da20e0ac381697667bb31a9f7d4de5bcf81d
3
+ size 473147704
hare_tortoise/task_1/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
hare_tortoise/task_1/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a182938c906d4c790522d4229b38f2e25c576f33bd49a1ea069cce8e6f40af23
3
+ size 473147704
hare_tortoise/task_2/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
hare_tortoise/task_2/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cc53f640248c639bdd9a30f23586b1192bc19ecd05cce3f53f8abad240a86123
3
+ size 473147704
hare_tortoise/task_3/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
hare_tortoise/task_3/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8a578598a41ad59e5c21b21be087c2a29501ffe7dc72299271afb1755c407485
3
+ size 473147704
hare_tortoise/task_4/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
hare_tortoise/task_4/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7d1ce4581681fd21008543603c7f1ed23d741cf6a626f42a4f0deb0086371780
3
+ size 473147704
hare_tortoise/task_5/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }
hare_tortoise/task_5/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:da4beeab93aa963f4c99dc2aa2fbf19051018b1c3cd1f6e443bf0798338f520a
3
+ size 473147704
hare_tortoise/task_6/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "AMPLIFY"
4
+ ],
5
+ "auto_map": {
6
+ "AutoConfig": "modeling_amplify.AMPLIFYConfig",
7
+ "AutoModel": "modeling_amplify.AMPLIFY",
8
+ "AutoModelForMaskedLM": "modeling_amplify.AMPLIFY"
9
+ },
10
+ "base_scale": 0.03227486121839514,
11
+ "decoder_init_range": 0.02,
12
+ "dtype": "float32",
13
+ "embedding_init_range": 0.02,
14
+ "hidden_size": 640,
15
+ "intermediate_size": 2560,
16
+ "max_length": 512,
17
+ "max_protein_length": 50000,
18
+ "model_type": "AMPLIFY",
19
+ "norm_eps": 1e-05,
20
+ "normalized_transformer": false,
21
+ "num_attention_heads": 10,
22
+ "num_hidden_layers": 24,
23
+ "pad_token_id": 0,
24
+ "transformers_version": "4.57.0",
25
+ "vocab_size": 32
26
+ }