Upload folder using huggingface_hub

README.md

@@ -0,0 +1,21 @@
+
+# LLaMA3
+
+custom Llama-based model trained on tiny shakespeare text using JAX.
+
+## Model Details
+- **Architecture**: Llama-based Transformer
+- **Parameters**: ~31.6M
+- **Training Framework**: JAX
+- **Tokenization**: GPT-2 Encoding (tiktoken)
+
+## How to use
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model = AutoModelForCausalLM.from_pretrained("parkneurals/LLaMA3")
+# Note: You will need a GPT-2 compatible tokenizer to run this
+```
+
+## Dataset
+Trained on the classic Tiny Shakespeare dataset.

config.json

@@ -0,0 +1,16 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "model_type": "llama",
+  "hidden_size": 256,
+  "intermediate_size": 1024,
+  "num_attention_heads": 8,
+  "num_hidden_layers": 6,
+  "num_key_value_heads": 4,
+  "vocab_size": 50257,
+  "rms_norm_eps": 1e-06,
+  "max_position_embeddings": 512,
+  "torch_dtype": "float32",
+  "transformers_version": "4.31.0"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:265298accb76bcbf89eaa3418b001117f96cf8e32462a50474f4d336a1b04fab
+size 110798056