| --- |
| language: |
| - ko |
| base_model: |
| - EleutherAI/polyglot-ko-1.3b |
| pipeline_tag: text-generation |
| tags: |
| - ingredient |
| - chatbot |
| - review |
| - usage |
| license: apache-2.0 |
| --- |
| |
| ## 🐥 Base Model |
|
|
| **EleutherAI/polyglot-ko-1.3b** |
|
|
| This model is based on the Korean version of Polyglot-1.3B, an open-source language model released by EleutherAI. |
| It is pre-trained on a large-scale Korean corpus and designed for general-purpose Korean language understanding and generation tasks. |
|
|
| --- |
|
|
| ## Training Procedure |
|
|
| ### Training Hyperparameters |
|
|
| The following hyperparameters were used during training: |
|
|
| - `output_dir`: `./qlora_model_eleutherai` |
| - `per_device_train_batch_size`: `2` |
| - `gradient_accumulation_steps`: `4` |
| - `total_batch_size`: `8 (2 x 4)` |
| - `learning_rate`: `2e-5` |
| - `num_train_epochs`: `2` |
| - `fp16`: `True` |
| - `logging_dir`: `./logs` |
| - `logging_steps`: `5` |
| - `save_steps`: `100` |
| - `save_total_limit`: `1` |
| - `load_best_model_at_end`: `True` |
| - `metric_for_best_model`: `loss` |
