SetFit Polarity Model with firqaaa/indo-setfit-absa-bert-base-restaurants-polarity

This is a SetFit model that can be used for Aspect Based Sentiment Analysis (ABSA). This SetFit model uses firqaaa/indo-setfit-absa-bert-base-restaurants-polarity as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification. In particular, this model is in charge of classifying aspect polarities.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

This model was trained within the context of a larger system for ABSA, which looks like so:

Use a spaCy model to select possible aspect span candidates.
Use a SetFit model to filter these possible aspect span candidates.
Use this SetFit model to classify the filtered aspect span candidates.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: firqaaa/indo-setfit-absa-bert-base-restaurants-polarity
Classification head: a LogisticRegression instance
spaCy Model: id_core_news_trf
SetFitABSA Aspect Model: Funnyworld1412/ABSA_review_game_genshin_impact-aspect
SetFitABSA Polarity Model: Funnyworld1412/ABSA_review_game_genshin_impact-polarity
Maximum Sequence Length: 8192 tokens
Number of Classes: 2 classes

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
negatif	'kebanyakan npc teyvat story utama mc dilupain:saranku developer menciptakan story menarik kehilangan player player yg bertahan repetitif monoton update size gede doang yg isinya chest itupun sampah puzzle yg rumit chest nya sampah story kebanyakan npc teyvat story utama mc dilupain gak difokusin map kalo udah kosong ya nyampah bikin size gede doang main 3 monoton perkembangan buruk' 'tolong ditambah reward gachanya player kesulitan:tolong ditambah reward gachanya player kesulitan primo quest eksplorasi 100 dasar developer kapitalis game monoton ramah player kekurangan bahan gacha karakter' 'gitu aja sampek event selesai primogemnya 10:cuman saran pelit biar player gak kabur game sebelah hadiah event quest perbaiki udah nunggu event hadiah cuman gitu gitu aja sampek event selesai primogemnya 10 pull gacha gak tingakat kesulitan beda hadiah main kabur kalok pelit 1 jariang mohon perbaiki server indonya trimaksih'
positif	'gameplay nya menarik story:gameplay nya menarik story questnya bikin boring menyelesaikan quest kepala frustasi karna dialog gak ngotak panjangnya gak skip developer pelit hadiah senang bermain game karna kikirannya puzzle nya questnya otak pusing developer respon bug tunggu viral bug nya benerin' 'selebihnya bagus cerita story sound effect tampilan:tolong pelit hoyoverse pemain pemain yg yg main karna pemain suka gratisan ntah artefak primoge character item karna jujur pemain puas gamenya upgrade character kumpulan item yg kebanyakan susah pemain kekurangan game selebihnya bagus cerita story sound effect tampilan didalam game yg lumayan bagus' 'bagus cerita story sound effect tampilan didalam game:tolong pelit hoyoverse pemain pemain yg yg main karna pemain suka gratisan ntah artefak primoge character item karna jujur pemain puas gamenya upgrade character kumpulan item yg kebanyakan susah pemain kekurangan game selebihnya bagus cerita story sound effect tampilan didalam game yg lumayan bagus'

Label

Examples

negatif

'kebanyakan npc teyvat story utama mc dilupain:saranku developer menciptakan story menarik kehilangan player player yg bertahan repetitif monoton update size gede doang yg isinya chest itupun sampah puzzle yg rumit chest nya sampah story kebanyakan npc teyvat story utama mc dilupain gak difokusin map kalo udah kosong ya nyampah bikin size gede doang main 3 monoton perkembangan buruk'
'tolong ditambah reward gachanya player kesulitan:tolong ditambah reward gachanya player kesulitan primo quest eksplorasi 100 dasar developer kapitalis game monoton ramah player kekurangan bahan gacha karakter'
'gitu aja sampek event selesai primogemnya 10:cuman saran pelit biar player gak kabur game sebelah hadiah event quest perbaiki udah nunggu event hadiah cuman gitu gitu aja sampek event selesai primogemnya 10 pull gacha gak tingakat kesulitan beda hadiah main kabur kalok pelit 1 jariang mohon perbaiki server indonya trimaksih'

positif

'gameplay nya menarik story:gameplay nya menarik story questnya bikin boring menyelesaikan quest kepala frustasi karna dialog gak ngotak panjangnya gak skip developer pelit hadiah senang bermain game karna kikirannya puzzle nya questnya otak pusing developer respon bug tunggu viral bug nya benerin'
'selebihnya bagus cerita story sound effect tampilan:tolong pelit hoyoverse pemain pemain yg yg main karna pemain suka gratisan ntah artefak primoge character item karna jujur pemain puas gamenya upgrade character kumpulan item yg kebanyakan susah pemain kekurangan game selebihnya bagus cerita story sound effect tampilan didalam game yg lumayan bagus'
'bagus cerita story sound effect tampilan didalam game:tolong pelit hoyoverse pemain pemain yg yg main karna pemain suka gratisan ntah artefak primoge character item karna jujur pemain puas gamenya upgrade character kumpulan item yg kebanyakan susah pemain kekurangan game selebihnya bagus cerita story sound effect tampilan didalam game yg lumayan bagus'

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import AbsaModel

# Download from the 🤗 Hub
model = AbsaModel.from_pretrained(
    "Funnyworld1412/ABSA_review_game_genshin_impact-aspect",
    "Funnyworld1412/ABSA_review_game_genshin_impact-polarity",
)
# Run inference
preds = model("The food was great, but the venue is just way too busy.")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	7	31.0185	70

Label	Training Sample Count
konflik	0
negatif	208
netral	0
positif	116

Training Hyperparameters

batch_size: (4, 4)
num_epochs: (1, 1)
max_steps: -1
sampling_strategy: oversampling
num_iterations: 10
body_learning_rate: (2e-05, 1e-05)
head_learning_rate: 0.01
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
seed: 42
eval_max_steps: -1
load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0006	1	0.2317	-
0.0309	50	0.0253	-
0.0617	100	0.0008	-
0.0926	150	0.4789	-
0.1235	200	0.0215	-
0.1543	250	0.0012	-
0.1852	300	0.1843	-
0.2160	350	0.0014	-
0.2469	400	0.0013	-
0.2778	450	0.0012	-
0.3086	500	0.0016	-
0.3395	550	0.0004	-
0.3704	600	0.0006	-
0.4012	650	0.0017	-
0.4321	700	0.0012	-
0.4630	750	0.0005	-
0.4938	800	0.0003	-
0.5247	850	0.0004	-
0.5556	900	0.0006	-
0.5864	950	0.2368	-
0.6173	1000	0.0003	-
0.6481	1050	0.0005	-
0.6790	1100	0.0006	-
0.7099	1150	0.0008	-
0.7407	1200	0.0924	-
0.7716	1250	0.0003	-
0.8025	1300	0.0003	-
0.8333	1350	0.0003	-
0.8642	1400	0.0006	-
0.8951	1450	0.0005	-
0.9259	1500	0.0004	-
0.9568	1550	0.0003	-
0.9877	1600	0.0002	-
1.0	1620	-	0.1328

Framework Versions

Python: 3.10.13
SetFit: 1.0.3
Sentence Transformers: 3.0.1
spaCy: 3.7.5
Transformers: 4.36.2
PyTorch: 2.1.2
Datasets: 2.19.2
Tokenizers: 0.15.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}