SentenceTransformer based on google/embeddinggemma-300m

This is a sentence-transformers model finetuned from google/embeddinggemma-300m. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: google/embeddinggemma-300m
Maximum Sequence Length: 256 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ahmedHamdi/narrative-similarity-pt-en-gemma")
# Run inference
sentences = [
    'Walter Mitty (Ben Stiller) is a negative asset manager at Life magazine who frequently daydreams about fantastical adventures and has a crush on his colleague Cheryl (Kristen Wiig). Photojournalist Sean O\'Connell (Sean Penn), who works closely with Mitty on his statements, sends him a package containing his latest negatives and a wallet as a gift in appreciation of Mitty\'s excellent work. In addition, the package contains a special photograph, negative 25, which he says captures the Quintessence of Life and should be used for the cover of the magazine\'s final print edition once it goes online. Unfortunately, this particular negative is missing from the package, and Mitty is forced to stop the detestable corporate transition manager Ted Hendricks (Adam Scott) to deal with the mess. Using the other negatives as clues, Mitty discovers that O\'Connell is in Greenland and flies there to try and find him. Arriving in Greenland, Mitty goes Mitty goes to a bar asking about O\'Connell. The bartender explains that O\'Connell has already left on a ship, and to find him, Mitty needs to go in the postal helicopter, and the pilot is drunk singing a karaoke version of "Don\'t You Want Me" in the bar. Mitty refuses to fly with a drunk pilot, but imagines Cheryl singing "Space Oddity" to him, gains newfound confidence, and jumps aboard the helicopter. Arriving at the ship, Mitty discovers that the helicopter cannot land on the ship. Mitty has misunderstood, thinking that the pilot had to jump onto the ship, where there was a boat approaching to pick him up, but then dived into icy shark-infested waters before he could be brought aboard. In doing so, Mitty realizes this whole adventure is actually happening, contrary to his daydreams. The sailors remember O\'Connell, and even offer Mitty the cake he left behind, and Mitty discovers another clue in the wrapping paper. His journey continues to Iceland, where O\'Connell is. Mitty went to photograph the Eyjafjallajökull volcano. The eruption prevents Mitty from finding O\'Connell, and he is forced to interrupt the search after receiving a text message telling him to return to New York immediately. For his failure, his first professional defeat in his long career with the magazine, Mitty is fired and is further disheartened to discover that Cheryl, who was leaving earlier, seems to have reconciled with her ex-husband. Mitty returns home, completely disheartened, and throws away his wallet when he visits his mother (Shirley MacLaine), and to his surprise, she mentions having found O\'Connell. She had said so before, but at the same time, dreaming, he couldn\'t hear her. He then finds a new lead to continue hunting O\'Connell. Mitty eventually tracks O\'Connell down in the Himalayas, attempting to photograph a rare snow leopard, and asks about the missing negative. O\'Connell explains that the message about taking a closer look was literal: the negative. It was in the wallet. Mitty joins him in a football game with some locals. Mitty flies to Los Angeles, but is detained by airport security, and calls the only person he knows in Los Angeles; Todd Maher (Patton Oswalt), a representative from eHarmony who has kept in touch through his adventures. Mitty returns home and helps his mother sell her piano; he mentions that he no longer has a wallet. His mother says that she always keeps her skills and she produces the wallet he had thrown away. Without looking at the photo, an emboldened Mitty hands it over to Life magazine, tells the management that it was the photograph Sean O\'Connell wanted for the final issue, and reprimands Hendricks for disrespecting the team that made the magazine so honored. Walter reunites with Cheryl when they receive their compensation packages and discovers that Cheryl\'s ex-husband was only at her house in order to repair the refrigerator. Excited by the turn of events, Mitty and Cheryl see the final issue of the magazine on the newsstand. The newspaper, with its cover greeting the staff, was revealed to show Mitty sitting in front of the Life building, holding a sheet of negatives up to the light and examining them with an eyepiece.',
    " Walter Mitty is a negative assets manager at Life magazine living alone in New York City. He chronically daydreams and has a secret crush on Cheryl Melhoff, a coworker. Walter attempts to contact Cheryl via eHarmony but eHarmony customer service agent Todd Mahar explains that Walter's account is not fully filled out: the been there and done that sections are blank. Walter works with legendary photojournalist Sean O'Connell, although they have never met in person. At work later that day, Walter receives a negative roll from Sean with a wallet as a gift in appreciation of Walter's work over the years. A letter from Sean explains that he believes negative #25 captures the quintessence of life and should be used for the cover of the magazine's final print issue before it becomes exclusively digital, yet the negative is missing from the roll. Ted Hendricks, the obnoxious manager of the magazine's transition, has also heard Sean's recommendation. When he asks about #25, Walter stalls and lies, worried about being fired. Walter then asks Cheryl for help, and she suggests that Walter use the other negatives as clues to Sean's location. He and Hernando, his understudy, are stumped by a negative depicting a curve, until they notice another containing the name of a ship registered in Greenland. Walter reluctantly takes a plane there. A bartender in Nuuk explains that Sean left on a ship. To find him, Mitty would need to go on the postal helicopter, and the pilot is drunk. Mitty recognizes the pilot's thumb from one of the negatives, and after hesitating, joins the pilot on a trip to bring supplies to the ship. Walter accidentally jumps into ice-cold, shark-infested waters, losing the ship's supplies and preventing radio communication when he finally comes aboard. There, Walter learns that Sean departed the ship a few days earlier and deduces from notes on wrapping paper for a clementine cake Sean left behind that he is heading to Iceland to photograph the volcano Eyjafjallajökull. Walter bikes, skateboards, and runs through the Icelandic countryside to find Sean, but just misses him as the volcano erupts. Dejected, he returns home. Ted assumes that Walter misplaced the negative and fires him. He tries to visit Cheryl, but spots her ex-husband and leaves before talking to her. Walter visits his mother and throws away the wallet from Sean. To his surprise, Walter recognizes the curve of the piano in his mother's house while looking at the last negative. When asked, she tells Walter that she met Sean and baked him the clementine cake. She had told Walter earlier, but he was daydreaming. Walter figures out from the notes that Sean is in the Afghan Himalayas and finds him photographing a rare snow leopard. When asked about the negative, Sean explains that, attempting to be playful, he had placed the negative in the wallet. He decides not to tell Walter what the picture actually depicts. When Walter returns to America, the airport security at Los Angeles detains him for arriving from Afghanistan. To verify his identity, Walter calls the only person he knows in Los Angeles: Todd, who has kept in contact during Mitty's travels. The two chat and Todd expresses admiration for how adventurous Walter appears. Walter receives the wallet from his mother, who had retrieved it from the trash, and finally obtains the negative but chooses not to see it. Emboldened, he delivers it to Lifes offices and berates Hendricks for disrespecting the staff that made the magazine so honored. Walter reunites with Cheryl, and thanks her for inspiring him on his journey. Cheryl eagerly asks about his adventures and tells him that her ex-husband had only been at her house to help with repairs. Walking along the street, they see the final print issue on sale at a newsstand, and on its cover, they see the photograph from #25 for the first time: it shows Walter sitting outside of the Life building, examining a contact sheet; the magazine is dedicated to Lifes staff, and Sean's note actually referred to quintessence of Life. Walter and Cheryl continue their walk holding hands.",
    "In Europe, American high school students and crimefighters Kim Possible and Ron Stoppable have thwarted a world dominating scheme of Professor Dementor and rescued Dr. Glopman, whom Dementor had kidnapped. As Kim Possible and Ron Stoppable start their first day of school, they meet a new student named Athena and take her on a mission to stop the plot of the evil Dr. Drakken. Athena (a skilled bōjutsuka) defeats Dr. Drakken's henchwoman Shego, making her the topic of conversation at Middleton High School despite Kim's jealousy. When the school honors Athena and her good deed, Shego and Dr. Drakken force their way into the ceremony. Kim tries to defeat Shego and her army of henchwomen but falls to the ground, leaving Athena vulnerable to kidnap. Athena is kidnapped, and Kim is laughed at. After Kim talks with her family and Ron, they decide to rescue Athena from Drakken's and Shego's lair. At the lair, Kim discovers Athena is part of Drakken's and Shego's plan to steal Kim's motivational essence and transfer it into Drakken with his brain-modulating device. Athena is revealed to be an android built by Drakken, specifically for this. As Kim short-circuits the transfer machine, Dr. Drakken is de-aged into a pre-adolescent version of himself. Despite Kim's insistence that she save herself, Athena stays behind to turn off the now-unstable machine. The lair explodes and Athena is presumed killed, but it is revealed that she has survived the explosion. Kim and Ron take her home to be repaired and programmed to be a hero alongside themselves. During the credits, Dr. Drakken, posing as a gifted student with Shego posing as his mother, enrolls at Kim's high school and begins his plan to defeat her.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.8464, -0.0673],
#         [ 0.8464,  1.0000, -0.0590],
#         [-0.0673, -0.0590,  1.0000]])

Training Details

Training Dataset

Unnamed Dataset

Size: 7,104 training samples
Columns: sentence_0 and sentence_1
Approximate statistics based on the first 1000 samples:
sentence_0 sentence_1
type string string
details
min: 11 tokens
mean: 123.93 tokens
max: 256 tokens

min: 3 tokens
mean: 238.68 tokens
max: 256 tokens

	sentence_0	sentence_1
type	string	string
details	min: 11 tokens mean: 123.93 tokens max: 256 tokens	min: 3 tokens mean: 238.68 tokens max: 256 tokens

Samples:

sentence_0	sentence_1
John Singer, a deaf man who is also unable to speak, rents a room in the Kelly family's house, where he forms a difficult friendship with Mick, a teenager full of typical teenage problems. John also tries to help the people he interacts with, such as his Greek friend Antonapoulos, who is deaf and mentally disabled, the alcoholic Blount, Dr. Copeland, who is dying of cancer, and Portia, Copeland's daughter, who has a higher education but works as a maid and is married to a farm laborer. Main awards	John Singer is a deaf-mute who works as a silver engraver in a southern US town. His only friend is a mentally disabled mute, Spiros Antonapoulos, who continually gets into trouble with the law, since he does not know any better. When Spiros is committed to a mental institution by his cousin, who is his guardian, John offers to become Spiros' guardian, but he is told that Spiros will have to go to the institution until this has been arranged. John decides to move to a town near the institution in order to be near his friend. He finds work there and rents a room in the home of Mr. and Mrs. Kelly, who are having financial difficulties as a result of Mr. Kelly's recent hip injury. Because the Kellys' teenage daughter, Margaret ('Mick'), resents having to give up her room to him, John tries to win her friendship. He also tries to become friends with Jake Blount, a semi-alcoholic drifter, and Dr. Copeland, an embittered African American physician who is secretly dying of lung cancer. John ...
`Jimmy Bobo (Sylvester Stallone) is a hitman and Taylor Kwon (Sun Kang) is a New York City police detective, both belonging to the opposite side of the law. However, two ruthless murders bring the pair together on a dangerous journey through the streets of New Orleans and the backrooms of Washington, seeking revenge for the deaths of their respective partners.`	In New Orleans, hitman Jimmy Bobo and his partner Louis Blanchard kill a corrupt former WDCPD policeman, Hank Greely. Decompressing at a bar, Blanchard is murdered by another hitman, Keegan, whose attempt to also kill Bobo fails. WDCPD Detective Taylor Kwon arrives to investigate Greely's death. Meanwhile, Keegan meets with his employer, Robert Morel, and Morel's lawyer Marcus Baptiste. Baptiste reveals that Greely tried to blackmail Morel, and provided local mobster Baby Jack with a file detailing Morel's illegal operations. Keegan kills Baby Jack and his men and retrieves the file. Kwon meets Bobo but is attacked by corrupt cops ordered by Morel to prevent Kwon from investigating Greely. He is shot before Bobo rescues him. Using Kwon's intel, they go to a massage parlor, where Bobo interrogates Ronnie Earl, the middleman who hired Bobo and Blanchard on Morel's behalf. Ronnie tries to kill Bobo, but Bobo kills him after a struggle. When his gun jams, nearly causing his death, Bobo ang...
`Elizabeth Bennet is a beautiful young woman and an expert in martial arts and weapons handling. When a plague begins to transform people into cannibalistic zombies, Elizabeth will have to use all her skills to fight this evil alongside the proud Mr. Darcy.`	Early 19th century England is besieged by zombies; the Bennet sisters—Elizabeth, Jane, Kitty, Lydia, and Mary—have all been trained in the art of weaponry and martial arts in China at their father's behest so they can defend themselves. Mrs. Bennet only wants to see her daughters married off to wealthy suitors. The Bennets attend a country dance also attended by newcomers Colonel Darcy, his good friend the amiable Charles Bingley and Bingley's snobbish sister Caroline. There, the young and handsome Bingley falls for lovely, sweet natured Jane. Charles Bingley has inherited £100,000 (£ million today)—attracting Mrs. Bennet's attention as a desirable suitor for her daughter. When zombies attack the ball, the Bennet sisters fight them off, and Colonel Darcy, a skilled zombie killer who was trained in Japanese martial arts – with property that pays him £10,000 annually (£ today) – is attracted to Elizabeth, although his behavior is outwardly aloof. During the ball, Darcy is disgusted to ov...

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false
}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 4
per_device_eval_batch_size: 4
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: no
prediction_loss_only: True
per_device_train_batch_size: 4
per_device_eval_batch_size: 4
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 3
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.2815	500	0.0229
0.5631	1000	0.08
0.8446	1500	0.0731
1.1261	2000	0.0426
1.4077	2500	0.0211
1.6892	3000	0.0222
1.9707	3500	0.0143
2.2523	4000	0.006
2.5338	4500	0.0015
2.8153	5000	0.0015

Framework Versions

Python: 3.9.21
Sentence Transformers: 5.1.2
Transformers: 4.57.6
PyTorch: 2.8.0+cu128
Accelerate: 1.10.1
Datasets: 4.5.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}