diff --git "a/README.md" "b/README.md" new file mode 100644--- /dev/null +++ "b/README.md" @@ -0,0 +1,515 @@ +--- +tags: +- setfit +- sentence-transformers +- text-classification +- generated_from_setfit_trainer +widget: +- text: '상품명 : 애견 고양이 펫밀크 반려동물 우유 관절 피모 장건강 10개입 매핑_카테고리1 : (#M)생활/건강>반려동물>고양이 사료>분유/우유 + 매핑_카테고리2 : MP > traverse > Naverstore > 반려동물용품 > 고양이용품 > 사료 옵션명 : ' +- text: '매핑_카테고리2 : MP > Naverstore > 반려동물용품 > 강아지용품 > 건강/관리용품 > 유산균 옵션명 : ▶사은품04_심장 + 30p 상품명 : 종근당 라비벳 강아지 유산균 장건강 심장 30p 매핑_카테고리1 : (#M)생활/건강>반려동물>강아지 건강/관리용품>강아지유산균 ' +- text: '매핑_카테고리2 : Naverstore > marskorea브랜드스토어 > 강아지 사료 상품명 : 시저 홀썸볼 닭 당근 보리 콩 85g + 강아지사료 옵션명 : 매핑_카테고리1 : (#M)홈>강아지 사료>시저 홀썸볼 ' +- text: '매핑_카테고리2 : Naverstore > 반려동물용품 > 고양이용품 > 사료 > 건식 옵션명 : 1EA 매핑_카테고리1 : (#M)홈>생활/건강>반려동물>고양이 + 사료>건식사료 상품명 : 피부모질관리 전연령묘용 고양이사료 5kg 반려식품 ' +- text: '매핑_카테고리2 : Naverstore > 반려동물용품 > 고양이용품 > 사료 > 동결건조 상품명 : [하프클럽/친절한도그씨]생식본능 + LID 토끼 캣 2kg 고양이사료 옵션명 : 매핑_카테고리1 : 생활/건강>반려동물>고양이 사료>동결건조 사료 ' +metrics: +- accuracy +pipeline_tag: text-classification +library_name: setfit +inference: true +base_model: intfloat/multilingual-e5-base +model-index: +- name: SetFit with intfloat/multilingual-e5-base + results: + - task: + type: text-classification + name: Text Classification + dataset: + name: Unknown + type: unknown + split: test + metrics: + - type: accuracy + value: 0.7847983933261239 + name: Accuracy +--- + +# SetFit with intfloat/multilingual-e5-base + +This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [intfloat/multilingual-e5-base](https://huggingface.co/intfloat/multilingual-e5-base) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification. + +The model has been trained using an efficient few-shot learning technique that involves: + +1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning. +2. Training a classification head with features from the fine-tuned Sentence Transformer. + +## Model Details + +### Model Description +- **Model Type:** SetFit +- **Sentence Transformer body:** [intfloat/multilingual-e5-base](https://huggingface.co/intfloat/multilingual-e5-base) +- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance +- **Maximum Sequence Length:** 512 tokens +- **Number of Classes:** 124 classes + + + + +### Model Sources + +- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit) +- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055) +- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit) + +### Model Labels +| Label | Examples | +|:------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| 33 | | +| 42 | | +| 99 | | +| 94 | | +| 98 | | +| 30 | | +| 122 | | +| 13 | | +| 50 | | +| 82 | | +| 120 | | +| 63 | | +| 53 | | +| 55 | | +| 97 | | +| 58 | | +| 80 | | +| 46 | | +| 34 | | +| 81 | | +| 105 | | +| 0 | | +| 111 | | +| 47 | | +| 1 | | +| 85 | | +| 40 | | +| 96 | | +| 101 | | +| 67 | | +| 107 | | +| 112 | | +| 49 | | +| 95 | | +| 4 | | +| 60 | | +| 75 | | +| 118 | | +| 108 | | +| 65 | | +| 41 | | +| 115 | | +| 52 | | +| 93 | | +| 22 | | +| 35 | | +| 114 | | +| 48 | | +| 6 | | +| 78 | | +| 26 | | +| 14 | | +| 29 | | +| 59 | | +| 102 | | +| 51 | | +| 79 | | +| 28 | | +| 24 | | +| 66 | | +| 72 | | +| 121 | | +| 62 | | +| 45 | | +| 83 | | +| 77 | | +| 36 | | +| 3 | | +| 5 | | +| 113 | | +| 7 | | +| 87 | | +| 91 | | +| 71 | | +| 89 | | +| 27 | | +| 43 | | +| 117 | | +| 110 | | +| 8 | | +| 103 | | +| 38 | | +| 69 | | +| 39 | | +| 31 | | +| 88 | | +| 57 | | +| 84 | | +| 11 | | +| 109 | | +| 21 | | +| 9 | | +| 10 | | +| 76 | | +| 90 | | +| 37 | | +| 54 | | +| 61 | | +| 56 | | +| 18 | | +| 119 | | +| 25 | | +| 73 | | +| 23 | | +| 106 | | +| 116 | | +| 100 | | +| 86 | | +| 74 | | +| 104 | | +| 32 | | +| 2 | | +| 16 | | +| 64 | | +| 19 | | +| 70 | | +| 44 | | +| 15 | | +| 92 | | +| 123 | | +| 17 | | +| 68 | | +| 20 | | +| 12 | | + +## Evaluation + +### Metrics +| Label | Accuracy | +|:--------|:---------| +| **all** | 0.7848 | + +## Uses + +### Direct Use for Inference + +First install the SetFit library: + +```bash +pip install setfit +``` + +Then you can load this model and run inference. + +```python +from setfit import SetFitModel + +# Download from the 🤗 Hub +model = SetFitModel.from_pretrained("mini1013/master_item_ps_flat") +# Run inference +preds = model("매핑_카테고리2 : Naverstore > marskorea브랜드스토어 > 강아지 사료 상품명 : 시저 홀썸볼 닭 당근 보리 콩 85g 강아지사료 옵션명 : 매핑_카테고리1 : (#M)홈>강아지 사료>시저 홀썸볼 ") +``` + + + + + + + + + +## Training Details + +### Training Set Metrics +| Training set | Min | Median | Max | +|:-------------|:----|:--------|:----| +| Word count | 7 | 26.5414 | 50 | + +| Label | Training Sample Count | +|:------|:----------------------| +| 0 | 200 | +| 1 | 200 | +| 2 | 162 | +| 3 | 200 | +| 4 | 200 | +| 5 | 200 | +| 6 | 200 | +| 7 | 200 | +| 8 | 77 | +| 9 | 200 | +| 10 | 200 | +| 11 | 53 | +| 12 | 19 | +| 13 | 200 | +| 14 | 200 | +| 15 | 166 | +| 16 | 81 | +| 17 | 93 | +| 18 | 200 | +| 19 | 135 | +| 20 | 26 | +| 21 | 200 | +| 22 | 200 | +| 23 | 200 | +| 24 | 200 | +| 25 | 141 | +| 26 | 200 | +| 27 | 200 | +| 28 | 200 | +| 29 | 200 | +| 30 | 200 | +| 31 | 200 | +| 32 | 50 | +| 33 | 200 | +| 34 | 200 | +| 35 | 200 | +| 36 | 200 | +| 37 | 199 | +| 38 | 96 | +| 39 | 200 | +| 40 | 200 | +| 41 | 200 | +| 42 | 93 | +| 43 | 200 | +| 44 | 116 | +| 45 | 200 | +| 46 | 200 | +| 47 | 138 | +| 48 | 200 | +| 49 | 184 | +| 50 | 123 | +| 51 | 151 | +| 52 | 200 | +| 53 | 200 | +| 54 | 46 | +| 55 | 200 | +| 56 | 200 | +| 57 | 200 | +| 58 | 200 | +| 59 | 200 | +| 60 | 199 | +| 61 | 200 | +| 62 | 200 | +| 63 | 200 | +| 64 | 200 | +| 65 | 195 | +| 66 | 132 | +| 67 | 35 | +| 68 | 10 | +| 69 | 194 | +| 70 | 200 | +| 71 | 200 | +| 72 | 200 | +| 73 | 200 | +| 74 | 200 | +| 75 | 200 | +| 76 | 200 | +| 77 | 200 | +| 78 | 200 | +| 79 | 200 | +| 80 | 200 | +| 81 | 200 | +| 82 | 200 | +| 83 | 200 | +| 84 | 79 | +| 85 | 200 | +| 86 | 200 | +| 87 | 152 | +| 88 | 200 | +| 89 | 200 | +| 90 | 200 | +| 91 | 198 | +| 92 | 76 | +| 93 | 200 | +| 94 | 200 | +| 95 | 200 | +| 96 | 200 | +| 97 | 195 | +| 98 | 200 | +| 99 | 200 | +| 100 | 200 | +| 101 | 196 | +| 102 | 200 | +| 103 | 200 | +| 104 | 200 | +| 105 | 200 | +| 106 | 200 | +| 107 | 200 | +| 108 | 200 | +| 109 | 200 | +| 110 | 200 | +| 111 | 200 | +| 112 | 200 | +| 113 | 200 | +| 114 | 200 | +| 115 | 200 | +| 116 | 200 | +| 117 | 200 | +| 118 | 200 | +| 119 | 200 | +| 120 | 200 | +| 121 | 200 | +| 122 | 200 | +| 123 | 14 | + +### Training Hyperparameters +- batch_size: (64, 64) +- num_epochs: (10, 10) +- max_steps: -1 +- sampling_strategy: oversampling +- num_iterations: 20 +- body_learning_rate: (2e-05, 1e-05) +- head_learning_rate: 0.01 +- loss: BatchAllTripletLoss +- distance_metric: cosine_distance +- margin: 0.25 +- end_to_end: False +- use_amp: True +- warmup_proportion: 0.1 +- l2_weight: 0.01 +- seed: 42 +- eval_max_steps: -1 +- load_best_model_at_end: False + +### Training Results +| Epoch | Step | Training Loss | Validation Loss | +|:------:|:----:|:-------------:|:---------------:| +| 0.0029 | 1 | 0.2325 | - | +| 0.1449 | 50 | 0.195 | - | +| 0.2899 | 100 | 0.1328 | - | +| 0.4348 | 150 | 0.1576 | - | +| 0.5797 | 200 | 0.1665 | - | +| 0.7246 | 250 | 0.162 | - | +| 0.8696 | 300 | 0.1705 | - | +| 1.0145 | 350 | 0.1518 | - | +| 1.1594 | 400 | 0.1588 | - | +| 1.3043 | 450 | 0.1696 | - | +| 1.4493 | 500 | 0.1581 | - | +| 1.5942 | 550 | 0.1519 | - | +| 1.7391 | 600 | 0.159 | - | +| 1.8841 | 650 | 0.1559 | - | +| 2.0290 | 700 | 0.1594 | - | +| 2.1739 | 750 | 0.1447 | - | +| 2.3188 | 800 | 0.1673 | - | +| 2.4638 | 850 | 0.1685 | - | +| 2.6087 | 900 | 0.1639 | - | +| 2.7536 | 950 | 0.1608 | - | +| 2.8986 | 1000 | 0.1589 | - | +| 3.0435 | 1050 | 0.1474 | - | +| 3.1884 | 1100 | 0.1548 | - | +| 3.3333 | 1150 | 0.1502 | - | +| 3.4783 | 1200 | 0.1443 | - | +| 3.6232 | 1250 | 0.1715 | - | +| 3.7681 | 1300 | 0.1614 | - | +| 3.9130 | 1350 | 0.1581 | - | +| 4.0580 | 1400 | 0.151 | - | +| 4.2029 | 1450 | 0.1542 | - | +| 4.3478 | 1500 | 0.1561 | - | +| 4.4928 | 1550 | 0.1515 | - | +| 4.6377 | 1600 | 0.1612 | - | +| 4.7826 | 1650 | 0.1555 | - | +| 4.9275 | 1700 | 0.1565 | - | +| 5.0725 | 1750 | 0.1442 | - | +| 5.2174 | 1800 | 0.1473 | - | +| 5.3623 | 1850 | 0.1452 | - | +| 5.5072 | 1900 | 0.1432 | - | +| 5.6522 | 1950 | 0.1487 | - | +| 5.7971 | 2000 | 0.1527 | - | +| 5.9420 | 2050 | 0.1555 | - | +| 6.0870 | 2100 | 0.139 | - | +| 6.2319 | 2150 | 0.1396 | - | +| 6.3768 | 2200 | 0.1493 | - | +| 6.5217 | 2250 | 0.144 | - | +| 6.6667 | 2300 | 0.1527 | - | +| 6.8116 | 2350 | 0.1469 | - | +| 6.9565 | 2400 | 0.1452 | - | +| 7.1014 | 2450 | 0.1503 | - | +| 7.2464 | 2500 | 0.1417 | - | +| 7.3913 | 2550 | 0.1519 | - | +| 7.5362 | 2600 | 0.1468 | - | +| 7.6812 | 2650 | 0.146 | - | +| 7.8261 | 2700 | 0.1425 | - | +| 7.9710 | 2750 | 0.1541 | - | +| 8.1159 | 2800 | 0.1444 | - | +| 8.2609 | 2850 | 0.1394 | - | +| 8.4058 | 2900 | 0.1541 | - | +| 8.5507 | 2950 | 0.1494 | - | +| 8.6957 | 3000 | 0.1427 | - | +| 8.8406 | 3050 | 0.1555 | - | +| 8.9855 | 3100 | 0.1409 | - | +| 9.1304 | 3150 | 0.1402 | - | +| 9.2754 | 3200 | 0.154 | - | +| 9.4203 | 3250 | 0.1421 | - | +| 9.5652 | 3300 | 0.1488 | - | +| 9.7101 | 3350 | 0.1502 | - | +| 9.8551 | 3400 | 0.1354 | - | +| 10.0 | 3450 | 0.1395 | - | + +### Framework Versions +- Python: 3.10.12 +- SetFit: 1.1.2 +- Sentence Transformers: 4.1.0 +- Transformers: 4.52.1 +- PyTorch: 2.7.0+cu126 +- Datasets: 3.6.0 +- Tokenizers: 0.21.1 + +## Citation + +### BibTeX +```bibtex +@article{https://doi.org/10.48550/arxiv.2209.11055, + doi = {10.48550/ARXIV.2209.11055}, + url = {https://arxiv.org/abs/2209.11055}, + author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren}, + keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences}, + title = {Efficient Few-Shot Learning Without Prompts}, + publisher = {arXiv}, + year = {2022}, + copyright = {Creative Commons Attribution 4.0 International} +} +``` + + + + + + \ No newline at end of file