SentenceTransformer based on google/embeddinggemma-300m

This is a sentence-transformers model finetuned from google/embeddinggemma-300m. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: google/embeddinggemma-300m
Maximum Sequence Length: 256 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ahmedHamdi/narrative-similarity-fr-en-gemma")
# Run inference
sentences = [
    "In a timeless Los Angeles, Henry Chinaski (Mickey Rourke), a writer without any real passion, drinks, writes poems, and even manages to pick up a girl, provided she's even more drunk than he is. He spends most of his time at the Golden Horn, a Los Angeles bar, where he regularly gets into fights with Eddie (Frank Stallone), a waiter as muscular as he is arrogant. The patrons even bet on their brawls. Amidst all this chaos, Henry meets Wanda (Faye Dunaway), also an alcoholic and in the process of getting a divorce, with whom he moves in. The two outsiders try to lead a normal life, but they become brutally attached to each other in a maelstrom of emotions mixing love, intoxication, violence, and madness. Their job searches remain fruitless. Soon, Henry has an affair with a wealthy publisher, Tully Sorenson (Alice Krige), who has published one of his works and offers to become his patron. While Henry could have been published and Wanda could have rid herself of men and demons, the couple resigns themselves to abandonment and chaos.",
    " Henry Chinaski is a destitute alcoholic who lives in a rundown apartment in Los Angeles and works menial jobs when he can find them. An intelligent man and keenly aware of his circumstance, he finds solace in expressing his feelings and perceptions of the world through writing poetry and short stories which he submits to magazines and papers for a few extra dollars. Henry frequents a bar called The Golden Horn, where he drinks, hangs out with other down and out alcoholics, and gets into altercations with the tough guy bartender he hates, Eddie. One night, Henry gets into a fight with Eddie and loses. The next day, he delivers sandwiches to two patrons in the bar. Realizing he needs to eat before fighting (to gain energy) to win, he grabs the sandwich out of one of the patrons' hands and eats it, disgusting the patron and angering the bar owner, Jim, one of Henry's best friends. Jim tells Henry to go lie down in his apartment for a few hours. After an afternoon nap, Henry steals bread, bologna, and wine from another apartment to eat and drink in preparation to fight Eddie. Henry returns to The Golden Horn later that evening, and antagonizes Eddie until he finally challenges him to another fight. Henry jumps over the bar and drinks beer out of the tap, infuriating Eddie. Jim puts a wager on Henry to beat Eddie. Henry wins the fight, and is then kicked out of the bar. Jim wins his bet, which he then gives to Henry so that he can buy a couple of drinks for the night. Henry then staggers on to another establishment down the street called the Kenmore, where he continues his imbibement. There, he meets Wanda, a fellow alcoholic and a kept woman. Wanda is initially annoyed with Henry, telling him she hates people, but is intrigued by his sarcastic and witty responses. The two buy liquor at a store across the street, and Wanda then steals corn from a cornfield, attracting the attention of the police. The two run to her apartment, evading them. Wanda boils the corn but discovers it is green and inedible, and then freaks out, saying that nothing in her life ever works out. Henry comforts her. The next day, Henry sets out to get a job to finance their booze-soaked life together. Henry then takes Wanda back to The Golden Horn to meet Jim as well as to cash an income tax rebate he received working six months in a toy factory. However, things become acrimonious between Henry and Wanda when Henry discovers that Wanda has slept with Eddie. After he chastises her for it, Wanda beats Henry with her purse, knocking him unconscious. Later, a detective following Henry sees him covered in blood and calls 911. Two paramedics arrive and are unfazed by Henry's being covered in blood, telling him not to waste their time. Wanda returns later, and the two apologize to one another. Later that night, Wanda claims to be dying in bed, seeing angels. Henry calls 911 and the same two paramedics arrive, much to his surprise, and they claim Wanda is just drunk and too fat. After Wanda leaves to look for a job the next morning, Henry is tracked down by Tully Sorenson, a wealthy female book publisher, who has been impressed with his writing and is interested in publishing some of his work. She finds him through the detective she has hired. Knowing Henry is destitute, Tully pays him an advance of $500. Henry then breaks into another apartment after hearing a man abusing his wife. After the man threatens to cut his wife's throat, he and Henry get into an altercation which results in the man being stabbed. Henry scrambles out of the apartment building and goes for a drive in L.A. with Tully. At one point, he rams a car where a man and woman are making out while the light is green. Tully tells him he was immature and reckless in his response. She then takes him back to her home where, after drinks, the two sleep together. At first, Henry is impressed with the promise of wealth and security, including an endless supply of booze that working for Tully could provide. However, he begins to realize that he is uncomfortable being involved with Tully, romantically or professionally, because of class differences, telling her that she is trapped in a cage with golden bars. Henry determines he must leave, that returning to his life of destitution and alcoholism is the only truth he knows. After leaving Tully's house, Henry returns to his usual bar and to Wanda. Henry buys drinks for all of his friends at the bar. Eddie suspects Henry has no money and is itching for a fight, so he tells Henry that he owes him $40 for the drinks. To Eddie's surprise, Henry pays with some of the advance he received from Tully and sarcastically leaves a tip for Eddie, saying Buy a drink on me. Tully heads out to see if she can change his mind, and finds him at the bar where a drunken, jealous Wanda proceeds to beat her up. When Henry does not intercede, Tully realizes that Henry does not care about her and does not want her help. So she leaves the bar and gives up on publishing his work, realizing that her pursuit of him was futile. Eddie calls Henry out, and they go out behind the bar for another fight. As Henry and the other barflies follow Eddie out the door, the camera pans out to the front of the bar to the sound of punches and the crowd cheering the two men.",
    "The film is set in the 17th century during the Swedish invasion of the Polish–Lithuanian Commonwealth in the years 1655 to 1658, known as The Deluge, which was eventually thwarted by Polish-Lithuanian forces. However, a quarter of the Polish-Lithuanian population died from the war and plague, and the country's economy was devastated.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.8423, -0.2107],
#         [ 0.8423,  1.0000, -0.0998],
#         [-0.2107, -0.0998,  1.0000]])

Training Details

Training Dataset

Unnamed Dataset

Size: 17,628 training samples
Columns: sentence_0 and sentence_1
Approximate statistics based on the first 1000 samples:
sentence_0 sentence_1
type string string
details
min: 7 tokens
mean: 128.37 tokens
max: 256 tokens

min: 3 tokens
mean: 229.91 tokens
max: 256 tokens

	sentence_0	sentence_1
type	string	string
details	min: 7 tokens mean: 128.37 tokens max: 256 tokens	min: 3 tokens mean: 229.91 tokens max: 256 tokens

Samples:

sentence_0	sentence_1
`Birdee, a devoted mother and wife, sees her life crumble when she discovers her husband's infidelity. She then returns to her mother's home and learns to overcome her pain.`	Birdee Pruitt is a Chicago housewife who is invited onto the national Toni Post talk show under the pretense of getting a free makeover. Instead, she is ambushed with the revelation that her husband Bill has been having an affair with her best friend Connie. Humiliated, Birdee and her precocious daughter Bernice move back to Birdee's hometown of Smithville, Texas, with her eccentric mother Ramona and her young, imaginative nephew Travis. As Birdee and Bernice leave Chicago, Birdee gives Bernice a letter from Bill, telling her how much he misses her. Birdee struggles to make a new life as a working single mother and deals with the growing attraction between herself and a former high school classmate, Justin Matisse, who Ramona hopes that Birdee will get together with. She also tries to rebuild her relationship with her estranged mother, her father (who has Alzheimer's disease), and Bernice. Bernice desperately wants to be with her father and attempts to sabotage the romantic overtures ...
Southern United States. Alice Cable is appointed to work with Alec Holland after the death of her predecessor. In his laboratory deep in the swamps, Alec works with his sister Linda. They accidentally discover a plant cell that could eradicate world hunger. Anton Arcane and his men break into the laboratory, steal his research, and kidnap Alice. Alec Holland is left for dead. Burned and affected by the product he invented, Alec transforms into a repulsive creature consumed by a thirst for revenge. Alice will try to find him with the help of a young boy, Jude.	After a scientist is mysteriously killed while assisting a top-secret bioengineering project, government worker Alice Cable arrives at the bogs to serve as his replacement. Alice immediately notices that one of the team's swamp sensors has malfunctioned, Harry Ritter reveals that her predecessor was attempting to repair it when he was killed. Charlie tells Ritter a rumor about an evil paramilitary leader named Anton Arcane, who intends to hijack their operation. Alice introduces herself to Dr. Linda Holland and her brother, lead scientist Dr. Alec Holland, who takes her on a tour and encourages her to admire the beauty of the swamps. After noting the disappearance of one of their workers, the group hears a loud bang and returns to the laboratory, where Linda shows off her recent breakthrough: a glowing, plant-based concoction with explosive properties. Sometime later, Alec notices that droplets of Linda's formula spawned rapid plant growth on the surfaces they touched. Suddenly, a grou...
`Aikido specialist Nicolo "Nico" Toscani was recruited by the CIA in 1969. However, a disagreement with one of his superiors, Kurt Zagon, led him to resign. A few years later, Nico is now a sergeant with the Chicago police. Headstrong, he doesn't always follow orders. When he arrests a drug dealer, the authorities release him. Nico then sets out to challenge the system and uncover a vast conspiracy.`	Sergeant Nico Toscani, who traces his roots to Palermo, Sicily, is a detective in the Chicago Police Department's vice squad. At an early age he had become interested in martial arts, and moved to Japan to study them. In 1969, Toscani was recruited to join the CIA by Nelson Fox and was involved in covert operations on the Vietnamese-Cambodian border during the Vietnam War. There, he became disgusted with DCI Kurt Zagon, who tortured prisoners. A stand-off occurred when Toscani tried to stop a torture session, and he left the CIA. Toscani returned to Chicago, joined the police department, and got married. Toscani and his new partner Detective Delores Jacks Jackson are investigating a drug ring, and after busting two of the dealers, including Salvadorian drug lord Tony Salvano, Toscani finds C-4 explosive. Shortly afterward, the men that Toscani and Jackson arrested are released at the request of federal officials, and Toscani is ordered to stand down. Later, the priest of Toscani's pari...

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false
}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 4
per_device_eval_batch_size: 4
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: no
prediction_loss_only: True
per_device_train_batch_size: 4
per_device_eval_batch_size: 4
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 3
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.1135	500	0.0147
0.2269	1000	0.0331
0.3404	1500	0.0635
0.4538	2000	0.0954
0.5673	2500	0.1016
0.6807	3000	0.0799
0.7942	3500	0.0845
0.9076	4000	0.0914
1.0211	4500	0.0697
1.1346	5000	0.0675
1.2480	5500	0.0363
1.3615	6000	0.0401
1.4749	6500	0.0434
1.5884	7000	0.0349
1.7018	7500	0.0486
1.8153	8000	0.0317
1.9287	8500	0.0398
2.0422	9000	0.0113
2.1557	9500	0.0152
2.2691	10000	0.0107
2.3826	10500	0.0076
2.4960	11000	0.0111
2.6095	11500	0.0087
2.7229	12000	0.0068
2.8364	12500	0.003
2.9499	13000	0.0044

Framework Versions

Python: 3.9.18
Sentence Transformers: 5.1.2
Transformers: 4.57.6
PyTorch: 2.8.0+cu128
Accelerate: 1.10.1
Datasets: 4.5.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}