SentenceTransformer based on google/embeddinggemma-300m

This is a sentence-transformers model finetuned from google/embeddinggemma-300m. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: google/embeddinggemma-300m
Maximum Sequence Length: 256 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ahmedHamdi/narrative-similarity-all-en-gemma")
# Run inference
sentences = [
    "A family—mother, father, and daughter—is traveling on a highway in the United States. The father, who is in rehab for alcoholism, is driving the car. They are returning from a disastrous Thanksgiving lunch with Joanne's parents. The lunch had been awkward, as they began discussing topics that made Ray uncomfortable, such as the death of his ex-wife, who died in an accident eight years prior. During the drive, they had to stop for a bathroom break. In a moment of inattention from the father, the little girl fell into a ditch. A dog frightened her and cornered her. Her father, Ray, tried to throw a rock at the dog to distract it, but it was too late. He went to retrieve her and also fell into the ditch. Peri fractured her right arm, and her father hit his head. The daughter was hospitalized for treatment, and her father collapsed shortly afterward from exhaustion. Upon waking, he realizes that no one can corroborate or confirm his family's hospital admission or medical history in the patient records, leading him to suspect a sinister conspiracy. The conclusion they reached was that it had all been a product of his imagination; Peri had died the moment she fell into the ditch. When the woman saw what had happened to Peri, she pushed her violently and ended up bleeding to death as well. After this, his mind went blank, and he constructed his own narrative. The people around him saw the truth, but he constantly judged them and believed they were lying to him.",
    " While driving back from a Thanksgiving visit to Joanne's parents, she and her husband, Ray, argue about the state of their relationship. After returning from a restroom, their young daughter, Peri, loses her compact mirror, prompting the parents to look for it. While Ray is distracted, Peri gets startled by a dog on an abandoned construction site. Ray throws a rock to scare the dog, startling Peri, who falls into the pit. Ray, attempting to grab her, hits his head, watching as a distressed Joanne climbs down, panicking over Peri. After his head clears, he gets to a hospital to have Peri's injured arm examined. During the long admissions process, Ray and Joanne deny a request for Peri to be registered as an organ donor. Peri is examined by a doctor to be fractured; Dr. Berthram recommends a CAT scan in case of a head injury. Joanne accompanies her to the scanning facility in the lower basement as Ray falls asleep at the waiting area. Ray awakens hours later with a sutured forehead and asks the hospital staff if he can see his wife and daughter, only to find that there's no such record. The only nurse present from the earlier shift says that Ray came alone and was treated for a head wound, distressing him. He gets pepper-sprayed and restrained by security, who sedate him and lock him in a room. He takes several adrenaline autoinjectors in a cabinet to counteract the sedative and escape to the parking lot. He waves down two police officers, who agree to investigate. A doctor directs the two officers and Ray to the imaging lab on the third floor, not the basement. Ray becomes agitated as the nurse recalled what really happened with him. Dr. Berthram claims that Ray stated his intent to wait for his deceased first wife, confusing him. He asserts that Abby is dead, reminding Dr. Berthram of the conversation that Peri had with him. The security officer reviews footage from a delayed security system; seeing no sign of Joanne and Peri. Ray, in a rage, remembers the cashier of the gas station as a witness, and the officers agree to track her down. Ray finds Peri's bloodstained scarf. Dr. Jacobs questions Ray about the car accident that his file says he was treated for; it also mentions alcoholism, which Ray counters by citing he's been sober for eight years, though further questions the possibility that his alcoholism started this delusion, which he denies. Dr. Jacobs suggests that Ray accompany her and the police to the construction site, noting to Berthram that Joanne and Peri might be there, only for Ray to accuse the staff and police of covering up their disappearance. A K-9 unit finds Peri's shattered compact mirror and a large pool of blood. Accusations continue from Ray when the police officers express disbelief. Dr. Jones asks him to recount what led to the accident— he fabricates memories of the incident, leading the police officers to arrest Ray for murder. Ray, after seeing an opportunity for distraction, holds Dr. Jacobs at gunpoint, demanding her hospital keycard. After disarming and locking the officers in the bathroom, Ray drives to the hospital, remembering the car crash that killed a pregnant Abby. He takes to the elevator disguised as a doctor, pulling a security guard in and choking him. At the basement, Ray sees nurses carrying human organs, and corpses that have been disembowelled. He finds Peri surrounded by surgeons preparing to harvest her organs, with a sedated Joanne nearby. Ray breaks them out of the hospital despite surgeons' protest, reaching the car, getting to the road back home.",
    "Alcoholic Rayburn Swanson is the owner of the Gwen Swanson Sanctuary, named after his missing daughter. The sanctuary is an act of penance, both because Gwen disapproved of his trapper lifestyle and because she had disappeared from his car five years earlier while he went into a store to buy whiskey. Rayburn keeps hunters out by using surveillance cameras. Sheriff Alice Gustafson is the sister of a troubled young man, Brooks. While examining the body of a young girl found by a lake, Alice notices a scar on the girl's throat, along with the point of a spear-head (marked with an “MB”) buried in a tree. After learning about the body, Rayburn arrives to make a positive ID, but it is not Gwen. Alice learns that the spear was used with a rare weapon known as an atlatl. Rayburn sees a man in a ghillie suit stalking the woods; when Rayburn attempts to confront the man, he is wounded by a thrown spear. While fleeing, he comes across a black pickup truck with the license plates removed. He marks the truck by scratching a small 'x' on the fender. After returning home and stitching his wound, he reviews his surveillance footage and observes a girl being hunted in the sanctuary by the same man. Rayburn returns to the sanctuary and rescues the girl, Molly, then keeps her safe in an unused spike trap overnight. She was rendered mute, having a throat scar similar to the one on the dead girl. They make it back to the cabin the next evening but are ambushed by the hunter. The hunter injures Rayburn and stabs Molly with a spear. Alice arrives and, after seeing that Molly has been stabbed, immediately suspects Rayburn, until the hunter appears behind her. Still disguised, his mannerisms cause her to believe it is her brother. Alice shoots Rayburn, wounding him, and causing him to flee so that her brother can escape. Alice searches for Rayburn and steps on a bear trap he had set, injuring her ankle. He escapes in her patrol car and calls an ambulance for Molly; Alice calls for Rayburn's arrest. He flees to the home of his former wife, Debbie, and her husband, Karl Blackhawk, Sheriff of the local Indian tribal police. They take him to the Factory, an abandoned mill where the homeless stay, and call Dr. Jon Boone to treat him. Brooks coincidentally arrives and is arrested by Karl, although he has an alibi to establish his innocence. Alice, realizing her moral failure, promises Molly, now hospitalized, that she will catch the killer. Rayburn decides to clean up and throw out his liquor, except for the still unopened bottle he bought on the day Gwen had gone missing. He remembers the marked truck and locates it. After breaking into a house, he finds the same weapons and clothes worn by the killer, along with a trove of missing posters, including one for his daughter. After finding a still-living girl under a sheet with her throat surgically cut open, the killer appears and captures Rayburn by knocking him unconscious. Alice notices issues with the autopsy report by Dr. Boone. While waiting for him in his office, she sees a photo of the doctor with his daughter Melissa wearing a necklace with the initials MB. Alice realizes that Boone is the killer and immediately calls for backup to raid Boone's home; the same home Rayburn had found. Boone explains to Rayburn that while grieving his own daughter, who had been killed by a drunk driver, he began kidnapping and killing teen girls to save them from bad parents, including Rayburn. Boone takes Rayburn to the wildlife sanctuary and forces him to flee so that he can be hunted. Alice sees Rayburn on the surveillance. Rayburn overpowers Boone and is beating him severely as Alice arrives; she attempts to stop Rayburn, but he ignores her and throws Boone into one of the spike traps, mortally wounding him. Alice looks on as Rayburn closes the doctor in, telling her that they are even. She later sits in a patrol car, listening to the radio about the ongoing manhunt for Dr Boone. Rayburn and his ex-wife have a funeral for Gwen, which Molly attends, providing him closure. He then empties his last bottle of whisky into the lake in memory of his daughter.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.8104, 0.0089],
#         [0.8104, 1.0000, 0.1449],
#         [0.0089, 0.1449, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

Size: 28,380 training samples
Columns: sentence_0 and sentence_1
Approximate statistics based on the first 1000 samples:
sentence_0 sentence_1
type string string
details
min: 10 tokens
mean: 128.41 tokens
max: 256 tokens

min: 3 tokens
mean: 233.02 tokens
max: 256 tokens

	sentence_0	sentence_1
type	string	string
details	min: 10 tokens mean: 128.41 tokens max: 256 tokens	min: 3 tokens mean: 233.02 tokens max: 256 tokens

Samples:

sentence_0	sentence_1
Alex (Emilio Estevez) is a race car driver. After an accident, he wakes up in 2009. His city has become a jungle where only businessmen and bounty hunters survive. Thanks to technology, it's now possible to transfer one's mind into another body, thus extending one's life. The fact that the world of 2009 is extremely toxic forces those who want to change bodies to "harvest" healthy humans from the past; all that's needed is to know the exact time and day of the subject's death and transfer them to the future. Alex died on a racetrack while the scene was being filmed, making him an ideal candidate, a "FREEJACK." Alex manages to escape just before being lobotomized to steal his body, but a ruthless bounty hunter, Victor Vacendak (Mick Jagger), is determined to track him down and catch him, or he'll lose his job.	In 2009, the super-wealthy achieve immortality by hiring bonejackers, mercenaries equipped with time travel devices, to snatch people from the past, just prior to the moment of their deaths, for use as substitute bodies. Those who escape are known as freejacks and are considered less than human under the law. In this dystopian future, most people suffer from poor physical health as a result of rampant drug use and environmental pollution, making them unattractive as replacement bodies. Alex Furlong is a Formula One racer who is about to die in a spectacular 1991 crash when a time machine snatches him from the cockpit and into 21st century New York City, now a futuristic dystopia populated by scavengers and killers. When Furlong's captors are ambushed by a hit squad, Furlong escapes from Victor Vacendak, a hardened mercenary who has snatched him on behalf of the powerful McCandless Corporation. Alex's former fiancée Julie Redlund is now an executive at McCandless, handling high-stakes m...
During the Shah's regime, when Mohsen Makhmalbaf was a 17-year-old guerrilla fighter, along with a girl who was also a guerrilla fighter, they attacked a policeman to disarm him. The policeman and Makhmalbaf injured each other, and the girl disappeared. 20 years later, when Makhmalbaf is a 37-year-old director and gives a newspaper supplement for casting calls for Salaam Cinema, the former policeman goes to Makhmalbaf's house and sends a message to Makhmalbaf through his youngest daughter, Hana, asking for a role in the film. Makhmalbaf decides to make a film about the disarmament that happened 20 years prior to today's screening. He chooses a 17-year-old boy to play his role as a young man. Then, each of them follows their own youth with a separate camera to accurately portray how the incident occurred 20 years ago. Both arrive at the disarmament site from two different locations." Different directions emerge, but the 17-year-old, who plays the role of Makhmalbaf's youth, is unwilling...	`The film is a semi-autobiographical account of Makhmalbaf's experience as a teenager when, as a seventeen-year-old, he stabbed a policeman at a protest rally and was jailed. Two decades later, Makhmalbaf made the decision to track down the policeman whom he had injured in an attempt to make amends. A Moment of Innocence is a dramatization of that real event.`
In the year 2154, Jake Sully, a paraplegic former Marine, agrees to participate in the Avatar program to replace his deceased twin brother, Tom Sully. He is sent to Pandora, one of the moons of Polyphemus, a gas giant orbiting Alpha Centauri A. Alpha Centauri A is the primary star of the Alpha Centauri star system, located in the solar system. Covered in lush jungle, Pandora is populated by flora and fauna as magnificent as they are formidable. Its atmosphere is distinguished by a high concentration of sulfuric acid, deadly to humans. Furthermore, the planet is inhabited by the Na'vi, an indigenous humanoid species whom they consider primitive and hostile. Yet, they are characterized by a way of life in complete harmony with nature. Physically, the Na'vi can reach three meters in height. They have blue-green skin and a long tail similar to that of a lion. Long, light-colored filaments, extending from the nape of their necks and protected by a braid they weave around them, are an organ ...	In 2154, the natural resources of the Earth have been depleted and is suffering ecocide. The Resources Development Administration (RDA) mines the valuable mineral unobtanium on Pandora, a moon in the Alpha Centauri star system. Pandora, whose atmosphere is inhospitable to humans, is inhabited by the Na'vi, , blue-skinned, sapient humanoids that live in harmony with nature. To explore Pandora, genetically matched human scientists use Na'vi-human hybrids called avatars. Paraplegic Marine Jake Sully is sent to Pandora to replace his deceased identical twin, who had signed up to be an operator. Avatar Program head Dr. Grace Augustine considers Sully inadequate but accepts him as a bodyguard. While escorting the avatars of Grace and Dr. Norm Spellman, Jake's avatar is attacked by Pandoran wildlife, and he flees into the forest, where he is rescued by female Na'vi Neytiri. Suspicious of Jake, she takes him to her clan. Neytiri's mother, Mo'at, the clan's spiritual leader, orders her daughte...

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false
}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 4
per_device_eval_batch_size: 4
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: no
prediction_loss_only: True
per_device_train_batch_size: 4
per_device_eval_batch_size: 4
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 3
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.0705	500	0.0797
0.1409	1000	0.0918
0.2114	1500	0.1027
0.2819	2000	0.1456
0.3524	2500	0.1759
0.4228	3000	0.1899
0.4933	3500	0.2332
0.5638	4000	0.2435
0.6342	4500	0.2031
0.7047	5000	0.1814
0.7752	5500	0.1817
0.8457	6000	0.2196
0.9161	6500	0.2048
0.9866	7000	0.2011
1.0571	7500	0.1463
1.1276	8000	0.1511
1.1980	8500	0.1332
1.2685	9000	0.1413
1.3390	9500	0.1361
1.4094	10000	0.1525
1.4799	10500	0.146
1.5504	11000	0.1527
1.6209	11500	0.1264
1.6913	12000	0.1094
1.7618	12500	0.1556
1.8323	13000	0.1191
1.9027	13500	0.144
1.9732	14000	0.1393
2.0437	14500	0.0856
2.1142	15000	0.0749
2.1846	15500	0.0485
2.2551	16000	0.0727
2.3256	16500	0.0583
2.3961	17000	0.0606
2.4665	17500	0.0684
2.5370	18000	0.0739
2.6075	18500	0.0472
2.6779	19000	0.0592
2.7484	19500	0.0621
2.8189	20000	0.0648
2.8894	20500	0.0586
2.9598	21000	0.0613

Framework Versions

Python: 3.9.18
Sentence Transformers: 5.1.2
Transformers: 4.57.6
PyTorch: 2.8.0+cu128
Accelerate: 1.10.1
Datasets: 4.5.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}