SentenceTransformer based on google/embeddinggemma-300m

This is a sentence-transformers model finetuned from google/embeddinggemma-300m. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: google/embeddinggemma-300m
Maximum Sequence Length: 256 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ahmedHamdi/IR-es-en-gemma_masked")
# Run inference
sentences = [
    'Fifteen African American men board a bus in GPE bound for GPE, GPE, where they plan to attend the Million Man March. Aside from their race, destination, and gender, the men have nothing in common: PERSON is the trip organizer; Xavier is an aspiring filmmaker hoping to make a documentary about the March; PERSON is a vain but charismatic and openly sexist actor; PERSON and PERSON are a gay couple; PERSON is a biracial police officer; GPE is a former gang member turned devout Muslim who has avoided prosecution for the murders he committed; and PERSON is a petty criminal who has been allowed to violate his probation to attend the march on the condition that he remains handcuffed to his father.',
    "In 1973, a squad of nine ORG soldiers convene in a local bayou for weekend maneuvers. New to the squad is ORG, a cynical transfer from ORG. He soon becomes disgusted with the arrogant behavior and attitudes of the men. A happily-married chemical engineer in his civilian life, ORG wants no part of a date with prostitutes which ORG has arranged for himself and their squad-mates. Nevertheless, he hits it off with the amiable PERSON, and both find themselves to be the most level-headed soldiers in their squad. The nine soldiers set out on patrol and soon get disoriented in the remote swampland. They come across a seemingly-abandoned campsite with several canoe-type boats called pirogues. To continue onward, the PERSON need the pirogues to get across the flooded swamp. The squad's leader, Staff PERSON, a Vietnam War veteran, tells one of his men to leave a note explaining the situation, and orders the soldiers into three of the four pirogues. As they set out across the bayou, a group of ORG hunter-trappers return and yell at the soldiers for having taken their boats. In response, ORG fires blanks from his M-60 machine gun at the ORG, thinking it amusing when the ORG take cover. The ORG return fire with live ammunition, killing Poole and sending the squad into a panic as they make their way toward cover. Sgt. Casper – the strict, inexperienced, and unpopular second-in-command – orders the squad to continue their mission. They discover that ORG. ORG has brought along a box of live ammunition for hunting purposes. ORG is forced at knife-point by ORG to give up the live ammo, and GPE divides it evenly among the soldiers, in order to bolster their chances of defense. They reach the shack of a one-armed ORG hunter-trapper, who speaks only French. GPE has him arrested as a prisoner of war. The emotionally unstable ORG. Bowden uses gasoline to ignite some ORG inside the shack, blowing it up, along with numerous provisions of food, fresh water, guns, and ammunition. The soldiers feel increasingly threatened. Hearing the barking of dogs, the PERSON presume they're about to be rescued. The dogs, however, belong to the ORG, who are now stalking the soldiers because of ORG's actions. The PERSON fend off the attacking dogs, only to find that lethal boobytraps have been set for them. Pvt. GPE is killed when he trips a spear-bed built into a spring-released cradle-frame, impaling him. The squad camps for the night. Overnight, PERSON begins to have a serious mental breakdown, refusing to speak to anyone or move. The group decides to tie him up for the night. The following morning, ORG tortures the one-armed Cajun by dunking his head in a fetid pond. ORG discovers this and tries to stop it. Both soldiers get in a fight to the death with bayonets. As the men fight, ORG gains the upper hand. To his surprise, the one-armed Cajun screams in English for ORG to kill ORG, which he does. After GPE is killed, the one-armed ORG escapes into the swamp.  PERSON, who was close friends with ORG, threatens to kill ORG. He is placated by GPE’s promise to have ORG court-martialed, but by this point the rest of the squad has mutinied against GPE and move forward with PERSON as their new leader. After thinking they have spotted the vengeful ORG, ORG opens fire and apparently kills one of them. As a helicopter passes overhead, the soldiers frantically try to signal it. Rushing headlong into the swamp after the chopper as it moves onward, GPE sinks to his death in quicksand. The two factions of the former ORG split up to search for GPE. Casper throws a homemade grenade at the ORG to no avail, then fixes his bayonet to his rifle and charges at them. He is shot several times and killed before he can reach them, while a broken and weeping ORG is shot from a distance in the heart and also killed. PERSON, ORG, and the now-catatonic GPE camp for the night and discuss their lives, agreeing they will fight for their own survival and each others. The following morning GPE and PERSON are jolted awake by a freight train, and then realize that GPE is missing. They move to the train tracks and find PERSON hanging dead from a bridge. The one-armed ORG appears on the tracks overhead. In perfect accented English, he warns PERSON and ORG to leave the ORG' territory while they still can. He gives them directions on how to escape the bayou, apparently in recognition of PERSON treating him humanely and ORG outright saving him from being murdered by GPE.  Following the one-armed ORG's advice, PERSON and ORG follow a dirt road and end up hitching a ride with a friendly Cajun couple. They are driven to a pig roast at a nearby LOC village. As PERSON happily mixes with the villagers, a wary ORG sees the arrival of the three remaining hunter-trappers who massacred their squad. One of ORG's would-be-killers chases him into a shed and wounds him in the arm. PERSON suddenly shows up, firing blanks at the LOC as a distraction, giving ORG the chance to stab him in the groin. The other two ORG arrive, and PERSON runs, leading them away from the injured ORG. PERSON, hiding around a corner, hits one of the Cajuns in the face with the butt of his M-16, knocking him unconscious. The remaining ORG gives chase, but as he is about to shoot PERSON, ORG grabs him from behind. This gives PERSON the opportunity to stab him to death with his bayonet. Leaving behind the village, PERSON and ORG slip away unseen into the swamp. As the duo moves into the swamp, another helicopter arrives overhead and seems to stay in the vicinity. They get back to the dirt road just in time to see a ORG truck drive towards them. They look at each other, knowing they are finally safe.",
    "A prologue set before The 13 Ghosts of ORG depicts a young PERSON and his partner PERSON hunting down and successfully sealing the final ghost into ORG. Mortifer, however, is attacked and seemingly killed by minion ghosts as a tearful ORG makes his escape. In the opening credits, PERSON recaps the events of the series in which ORG eventually set the ghosts free once more but would catch all but one with the help of his friends PERSON and PERSON. In the present day at Old PERSON, ORG is forced to retire from solving mysteries after catching a teenager-fearing PERSON and largely due to the ungrateful Sheriff telling them they could be arrested for criminal negligence. In addition, Old PERSON owner Mrs. PERSON was handcuffed outside for trying to stop a deal with PERSON. The following day, they hold a garage sale and sell items from their old mysteries, even ORG, leaving PERSON depressed. They then discover the crystal ball that PERSON, PERSON, and ORG used to contact PERSON during their ghost recapturing adventures. At that moment, their old friend ORG contacts the gang about the final ghost. PERSON fills PERSON and a skeptical Velma in, and unveils the red van dubbed ORG, and the gang start on their journey to meet ORG. On their way to his estate, they are chased by a phantom car but PERSON manages to skillfully outmaneuver it. Seeing PERSON take charge only furthers PERSON's existential crisis as he feels he is without a role. The gang meet PERSON, but are all attacked by the thirteenth ghost: a hulking, winged demon-like creature named PERSON. PERSON reveals that he sent the chest to ORG residence in the Himalayas and the gang travels there using a flying boat. Upon arrival, they spot PERSON who inquired about the crystal ball at the garage sale. PERSON, PERSON, Scooby, and PERSON tail him but are attacked by the phantom car again which uses its revving engine to create an avalanche. Vincent reveals he is unable to use his magic and the four crash into a temple. Meanwhile, ORG search a seemingly abandoned post office and unknowingly meet an adult ORG who tries to sell them ORG merchandise and ghost-catching equipment. They leave but are snatched up by something even ORG cannot explain and she comes to the conclusion that it must have been real ghosts. PERSON had been waiting for ORG and chases the four when they find the chest. PERSON breaks his crystal ball and somehow transports the kids outside the temple where they meet Velma and PERSON. Fred cheers for his despondent team and a restored PERSON conducts a plan. Velma visits ORG store and takes everything. She, PERSON, and ORG are once again pursued by the phantom car but are able to evade capture with ORG upgrades installed. PERSON, PERSON, and ORG infiltrate the castle to help PERSON and reunite with ORG, GPE, and PERSON. They find ORG defeated. He reveals that PERSON is the demon form of his magically powerful ancestor PERSON and thus blames the impending doom on himself. PERSON opens ORG, something that Velma points out as being impossible, as the Chest can only be opened by the living. PERSON tries to run but is intercepted by PERSON and revealed as PERSON, who reveals that he faked his death by casting an illusion. PERSON wanted to sell the Chest to PERSON for $10 million, but PERSON reveals himself as an international agent named PERSON and arrests PERSON. However, he begins to escape in the phantom car, only to be recaptured. A visage of the real PERSON and PERSON briefly appears. Velma deduces from a tome that the real PERSON was actually watching over PERSON and has achieved redemption. However, she fabricated this story to give ORG closure. She debunks the first twelve ghosts as hallucinations due to the Himalayan thin air and teases opening the real ORG. However, upon seeing the gang's fearful faces, she decides not to. PERSON decides the gang should continue solving mysteries.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.5560, 0.0329],
#         [0.5560, 1.0000, 0.0574],
#         [0.0329, 0.0574, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

Size: 2,340 training samples
Columns: sentence_0 and sentence_1
Approximate statistics based on the first 1000 samples:
sentence_0 sentence_1
type string string
details
min: 5 tokens
mean: 131.46 tokens
max: 256 tokens

min: 12 tokens
mean: 221.57 tokens
max: 256 tokens

	sentence_0	sentence_1
type	string	string
details	min: 5 tokens mean: 131.46 tokens max: 256 tokens	min: 12 tokens mean: 221.57 tokens max: 256 tokens

Samples:

sentence_0	sentence_1
PERSON, a woman who frequents clandestine casinos and flaunts her elegance to conceal her poverty while waiting for a wealthy man to appear and take care of her, chats with the old man begging at the entrance of a casino. Shortly after, a rich and mysterious suitor offers her a life of luxury, albeit one far removed from social conventions. He doesn't want to marry or tell her anything about his life; in return, he offers her luxuries and everything she's ever dreamed of. But she also meets the son of a very important textile businessman who becomes obsessed with her and begins to pursue her. He proposes they run away together and even steals to finance their escape to GPE. But on the day they are supposed to leave, PERSON doesn't leave the house at the usual time, and PERSON becomes frightened. The beggar and PERSON turn out to be the same person. She decides not to leave PERSON and instead goes to the church where the beggar is asking for alms. They enter and get married.	PERSON, a woman gambler who frequents underground casinos, who flaunts her elegance to hide her poverty while waiting for a wealthy man to appear and take care of her, talks to an old man begging at the door of a casino. Shortly after, a rich and mysterious man offers her a life of luxury, albeit far from social conventions. He does not want to marry or tell her anything about his life; In return, he offers her luxuries and everything she has dreamed of all her life. She also meet the son of a very important textile businessman, who becomes obsessed with her and begins to persecute her. He proposes that they run away together, and even steals to finance her escape to GPE. The day they were supposed to leave, the husband doesn't leave the house at the usual time and PERSON gets scared. The beggar and the husband turn out to be the same person. She decides not to leave him and instead goes to the church, where the beggar begs for alms. They both get married.
`PERSON is serving a long sentence for murder. He had long since given up on life until a fellow inmate gives him a packet of seeds, which he reluctantly accepts. To everyone's surprise, flowers bloom in the barren prison soil, restoring not only his joy for life but also offering him the chance to be free again.`	When PERSON, a convicted murderer, is placed in an experimental programme to finish off his prison sentence, all he wants is peace and quiet. After his wise, elderly roommate Fergus, imprisoned for killing three wives, introduces him to gardening, PERSON uncovers a talent and passion for plants. When he accidentally raises a patch of double-violets, the warden assigns him to cultivate a garden, with other prisoners as his assistants. Teaming up with his fellow inmates, PERSON gets the attention of celebrated gardener ORG. Soon, the unexpected gardeners are preparing to compete for ORG. When PERSON meets GPE's beautiful daughter PERSON, he discovers another reason to fight for his freedom: true love.
`PERSON is a young father and husband who adores his family, but also his job, since that's where he spends most of his time. One afternoon, when he gets home from work, his wife PERSON confronts him and leaves him to take care of their six-year-old son. PERSON must learn to be a father while also trying not to neglect his career. But just when he's settled into his new life and begins to feel fulfilled as a father, PERSON returns. And she wants her son back.`	The film is set in ORG in LOC on LOC of GPE. Major PERSON Pollock (PERSON) tries but fails to hide an article about himself in GPE. His attempt to keep the article from the eyes of the other guests at the residential hotel only succeeds in heightening their awareness of it, particularly the strict ORG (Gladys Cooper) and the more relaxed and compassionate PERSON (PERSON). The two women read in the paper that Major Pollock has pled guilty to sexually harassing several young women in a theatre. However the filed complaints are, in themselves, questionable. PERSON wants Major Pollock expelled from the hotel and holds a meeting with other long-term residents to decide the issue before presenting it to the manager, Miss PERSON (PERSON). PERSON leads the meeting arguing for the ORG's expulsion, and despite the opposing views of the other residents, she informs Miss PERSON that the Major must leave. PERSON (PERSON) and PERSON (PERSON), who were formerly married to one another, meet outside. ...

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false
}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 4
per_device_eval_batch_size: 4
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: no
prediction_loss_only: True
per_device_train_batch_size: 4
per_device_eval_batch_size: 4
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 3
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.8547	500	0.5841
1.7094	1000	0.3365
2.5641	1500	0.0972

Framework Versions

Python: 3.9.18
Sentence Transformers: 5.1.2
Transformers: 4.57.6
PyTorch: 2.8.0+cu128
Accelerate: 1.10.1
Datasets: 4.5.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}