tonnnnnnnnnnnnn/triplet-loss-v1

SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2 on the sample_prep_triplet dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • sample_prep_triplet

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("tonnnnnnnnnnnnn/triplet-loss-v1")
# Run inference
sentences = [
    'A spin-off about Juda and Daato raising their kids. The trickster twins take the center stage.',
    'What is the name of the manga with the description "During the early years of the hottest MMORPG called Arena, the world’s top ranker Meleegod vanished after setting every possible record in the virtual world. In the real world, this deathly melee damage dealer was the happy-go-lucky Kang Hyeonu, who simply deleted his character and quit his hobby to finish mandatory military service and inherit his father’s company. This rosy future dissolves within two years when a business rival makes his father go bankrupt. Hyeonu struggles to make ends meet with a string of part-time jobs, but his best friend has a better idea. He urges Hyeonu to conquer the Arena once again and earn a cushy living as a gaming celebrity. Vowing to strike it rich and avenge his father, the former Meleegod returns to the Arena as a newbie. Only this time, he’s out for blood!"?',
    'What manga is about Juda and Daato raising their kids?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000, -0.5077,  0.9839],
#         [-0.5077,  1.0000, -0.5727],
#         [ 0.9839, -0.5727,  1.0000]])

Evaluation

Metrics

Triplet

• Datasets: manga-dev, wikipedia-sections-test, wikipedia-sections-test, wikipedia-sections-test and wikipedia-sections-test
• Evaluated with TripletEvaluator

Metric	manga-dev	wikipedia-sections-test
cosine_accuracy	0.0133	0.0133

Training Details

Training Dataset

sample_prep_triplet

• Dataset: sample_prep_triplet at 5bea00c
• Size: 8,100 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 7 tokens mean: 100.77 tokens max: 246 tokens	min: 7 tokens mean: 35.69 tokens max: 242 tokens	min: 7 tokens mean: 44.12 tokens max: 242 tokens

• Samples:

  anchor	positive	negative
  Spunky mercenary mage Priscilla Mortia sets out on her final mission: to reclaim the last 15 years of her tragic life. Using a mana stone called the Dragon Heart, she travels back in time to when her family ruled over the land. While Priscilla’s goal is to reawaken her magical powers by obtaining the Dragon Heart in the past, she must survive the sinister plots of her wicked stepmother and her elderly fiance. To aid in her quest she seeks out Kian Lustin, a renowned swordsman who once captured her heart. But why does their fateful first meeting seem strangely familiar? Can Priscilla finally gain Kian's allegiance and claim her rightful seat as the heir to House Mortia?	What is the name of a manga about a group of boys who encounter a mysterious village full of youkai?	What is the name of the manga with the description "Spunky mercenary mage Priscilla Mortia sets out on her final mission: to reclaim the last 15 years of her tragic life. Using a mana stone called the Dragon Heart, she travels back in time to when her family ruled over the land. While Priscilla’s goal is to reawaken her magical powers by obtaining the Dragon Heart in the past, she must survive the sinister plots of her wicked stepmother and her elderly fiance. To aid in her quest she seeks out Kian Lustin, a renowned swordsman who once captured her heart. But why does their fateful first meeting seem strangely familiar? Can Priscilla finally gain Kian's allegiance and claim her rightful seat as the heir to House Mortia?"
  Rayne woke up with no memories of the apocalypse. Samael is just... a delinquent, who happened to find Rayne at a rooftop. They both ended up making a deal in order to survive in a world where cannibalistic monsters roam.	What is the legendary road of immortality?	What manga has a protagonist named Rayne who wakes up with no memories of the apocalypse and a delinquent named Samael?
  A flawless love story of the flawed. Stuttering lady Maximilian is forced into a marriage with Sir Riftan, but he leaves on a campaign after their wedding night. 3 years later, he triumphantly returns, ready to cherish her. As life with her husband finally begins, she only has one question — does she deserve this love and happiness?	What is the name of a manga about a 24-year-old woman who has a one-night stand with her landlady and develops a relationship with her?	What manga is about longing for love and marriage?

• Loss: TripletLoss with these parameters:

  {
      "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
      "triplet_margin": 5
  }
  

Evaluation Dataset

sample_prep_triplet

• Dataset: sample_prep_triplet at 5bea00c
• Size: 900 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 900 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 7 tokens mean: 99.81 tokens max: 256 tokens	min: 7 tokens mean: 35.44 tokens max: 234 tokens	min: 7 tokens mean: 43.91 tokens max: 228 tokens

• Samples:

  anchor	positive	negative
  So-Mang Park wants to have a relationship at least once in her teens. But true love leading to a dreamy romantic marriage is just a crazy delusion that'll never happen. In reality, So-mang Park and the word "date" just don't belong together. Then one day a ray of hope shines upon her. Is this what flirting is? This is a story of a cheerful yet delusional girl's quest for romance.	What is the name of the manga set in a war-ravaged island with ex-yakuza Kazuki Taizou?	What is a romantic story about a young couple's first love?
  As the most popular member of super idol group FLY, Yusen was soaring high until a joke goes horribly awry. Every day he’s now top of the news for some new social media disaster. Yelling at flight attendants, beating children, cursing on live TV, he tries his best but haters gonna hate and the news gonna fake. Even as the news misinterprets his every move, he refuses to quit for those fans who stand by him no matter what. Top Headline: Super Idol Yusen is off his rocker or is the pop star just misunderstood?	What is the name of the manga with the following description?	What is the latest news about Yusen from FLY?
  I reincarnated as the Villainess In a novel I had read, Jubellian Eloy Floyen. I was doomed to die, but no! With the memories of my previous life, I will avoid my death flag! “I will not marry a man who has no honor, no family, no property, and no Power!” As I declared my will to live a glorious and glamorous life, my Dad decided to make the crown prince my fiancé?! Is he Mad?! I’ll just treat the engagement contract as a last measure. “Why are you avoiding your engagement with the Crown Prince?” “Because it scares me! He kills people who don’t obey him right away!” A few days later the Crown Prince sent me a letter. “Don’t worry, I won’t kill you.” Oh no….did I already do something worth dying for?	What is the name of a manga about an alien from Chlorodium #2 who gets used to eating human food?	What is the name of the manga with the description "I reincarnated as the Villainess In a novel I had read, Jubellian Eloy Floyen. I was doomed to die, but no! With the memories of my previous life, I will avoid my death flag! “I will not marry a man who has no honor, no family, no property, and no Power!” As I declared my will to live a glorious and glamorous life, my Dad decided to make the crown prince my fiancé?! Is he Mad?! I’ll just treat the engagement contract as a last measure. “Why are you avoiding your engagement with the Crown Prince?” “Because it scares me! He kills people who don’t obey him right away!” A few days later the Crown Prince sent me a letter. “Don’t worry, I won’t kill you.” Oh no….did I already do something worth dying for?"

• Loss: TripletLoss with these parameters:

  {
      "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
      "triplet_margin": 5
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• num_train_epochs: 4
• warmup_ratio: 0.1
• fp16: True

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 4
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss	manga-dev_cosine_accuracy	wikipedia-sections-test_cosine_accuracy
0.7874	100	4.3192	4.0202	0.0178	-
1.5748	200	4.0382	3.9409	0.0211	-
2.3622	300	3.9501	3.9073	0.0178	-
3.1496	400	3.9062	3.8886	0.0144	-
3.9370	500	3.881	3.8774	0.0133	-
-1	-1	-	-	-	0.0133

Framework Versions

• Python: 3.10.0
• Sentence Transformers: 5.1.2
• Transformers: 4.57.1
• PyTorch: 2.5.1+cu121
• Accelerate: 1.11.0
• Datasets: 4.4.1
• Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}