SentenceTransformer based on google/embeddinggemma-300m

This is a sentence-transformers model finetuned from google/embeddinggemma-300m. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: google/embeddinggemma-300m
Maximum Sequence Length: 256 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ahmedHamdi/IR-pt-en-gemma_masked")
# Run inference
sentences = [
    "The central story is the conflict between a group of poor miners and a powerful city man. It begins with a stranger defending a miner from a gang of thugs. To the miner's surprise, he is wearing a religious collar. The miner then invites this preacher to dinner at his home. As the film progresses, the preacher integrates into the poor community until the time comes for a confrontation with the forces of the powerful capitalist.",
    "In LOC, outside GPE, GPE in GPE, mining baron ORG is waging a war of intimidation against independent prospectors and their families, including PERSON who is courting PERSON. PERSON's teenaged daughter, PERSON, desperate for deliverance from ORG after a gang of his men attack the mining camp and kill her dog, prays for a miracle. Shortly afterward, a man atop a pale horse rides into LOC. When PERSON heads to town to pick up supplies, four of ORG's men beat him with axe handles before the stranger fights them off with his own axe handle. Hull then invites his rescuer to dinner and, while the stranger is washing, notices what appears to be six bullet wounds in his back. When the stranger arrives at the dining table, he is wearing a clerical collar and is thereafter referred to as PERSON. ORG son, PERSON, attempts to scare off the PERSON with a gigantic workman named PERSON. PERSON, however, disables ORG with a sledgehammer blow to the groin. Coy returns from GPE, learns of the PERSON's arrival, and unsuccessfully attempts to bribe and then threaten him. At PERSON's suggestion, ORG then offers the miners $1,000 per claim provided they evacuate within 24 hours. ORG says he plans to hire the services of a corrupt marshal named PERSON to clear them out if they refuse. The miners ultimately reject ORG's offer, despite PERSON warning them about GPE. PERSON expresses her love for PERSON, but he gently rebuffs her. Megan angrily assumes PERSON is really in love with PERSON. ORG's men dam the creek forcing the miners to mine a dry bed. Megan rides into ORG's camp, where PERSON shows her the blasting operation before he attempts to rape her. Preacher arrives on horseback, disarms PERSON and shoots him through the hand. Stockburn and his deputies arrive in ORG. Coy gives him a rough description of the PERSON, which startles ORG, because the man PERSON is describing is dead. PERSON, one of the miners and ORG's former partner, discovers a large gold nugget in the dry creek bed and rides into town with his teenage sons, where he yells drunken abuse at ORG from the street. Stockburn and his deputies gun him down, and ORG sends a message that he wants PERSON to meet him in town the next morning. PERSON begs PERSON not to go, and tells him she will marry PERSON, despite her feelings for PERSON. The following day, PERSON and PERSON blow up ORG's mining site with dynamite. To stop Hull from following him, PERSON scares off PERSON's horse and rides into town alone. In the gunfight that follows, he kills all but the two of ORG's men who run away, and then, one by one, all six of ORG's deputies. In the final shootout, ORG recognizes PERSON in disbelief before he is shot in the chest five times, leaving five exitwounds in the back similar to the wounds on PERSON's back. Finally, PERSON kills him with a shot to the head. ORG, watching from his office, aims a rifle at the PERSON but is shot dead by Hull. Preacher nods at Hull and rides off toward the snow-capped mountains. Megan arrives and, seeing PERSON riding away, shouts her love and thanks to him.",
    "In the mid-1970's, fake psychic ORG (PERSON) and her boyfriend PERSON (PERSON) attempt to locate the nephew of wealthy, guilt-ridden, elderly ORG (PERSON). ORG's recently-deceased sister gave the baby boy up for adoption, but GPE now wants to make him her heir and will pay ORG $10,000 to find him. Julia knows almost nothing about the infant. During his investigation PERSON discovers that the boy was given the name PERSON, and is thought to have died while still young. However George tracks down a man, PERSON (PERSON), who paid for PERSON's tombstone years after his supposed death, and PERSON comes to think the grave is empty. George and ORG bicker frequently, but he is as good an investigator as she is a psychic, and their relationship is solid. Meanwhile, it has been revealed to the viewers that Shoebridge murdered his adoptive parents and faked his own death, and is now a successful jeweler in GPE known as PERSON (PERSON). He and his live-in girlfriend PERSON (Karen Black) kidnap millionaires and dignitaries, confining them in a secure room in the cellar of their home, and return them in exchange for ransoms in the form of valuable gemstones. PERSON conceals the latest ransom, a large diamond, in plain sight within a crystal chandelier hanging above the home's main staircase. When PERSON learns that PERSON is investigating him, he enlists PERSON (the two had murdered PERSON's adoptive parents long ago) to kill ORG and PERSON. PERSON initially refuses to help, then contacts ORG and PERSON, telling them to meet him at a café on a mountain road. He cuts the brakeline of ORG's car, but they manage to survive their dangerous high-speed descent. Maloney tries to run them over, but dies in a fiery explosion when he swerves to avoid an oncoming car and his car goes over the edge. At ORG's funeral, his wife (PERSON) tearfully confesses, under pressure of PERSON's questioning, that PERSON's name is now PERSON. PERSON must go to work driving his taxi for an evening shift, so ORG tracks down various A. Adamsons in GPE, eventually reaching the jewelry store as it closes for the day. PERSON's assistant Mrs. PERSON (ORG) offers to let ORG leave a note. ORG tricks Mrs. PERSON into giving her his home address. PERSON and PERSON are bundling their latest kidnap victim, ORG (PERSON), into their car when ORG rings their doorbell. They attempt to drive out of their garage, but ORG's car blocks their way. She tells PERSON that his aunt wants to make him her heir, and for a moment everyone seems delighted with developments. Then ORG sees the unconscious bishop, and she is abducted by the couple. PERSON drugs her and leaves her in the cellar, to deal with after they exchange the bishop for ransom. Searching for ORG, PERSON finds her car outside GPE and ORG's house. When no one answers the door, he breaks in and searches for her. He finds her handbag with blood stains on it, and indications of a struggle. When PERSON and PERSON return home PERSON hides upstairs. He overhears PERSON telling PERSON about his plan to kill ORG and make her death seem a suicide. George manages to talk to ORG, who is faking unconsciousness in the cellar (left open by PERSON when he went to check on her) and they come up with a plan. PERSON and ORG enter to carry ORG out to the car, but she knocks them down and runs out and PERSON locks the kidnappers in. ORG then goes into what appears to be a genuine trance. She walks out of the basement and climbs halfway up the main staircase, stops, and points at the huge diamond hidden in the chandelier. ORG then wakes and asks PERSON what she is doing there. He excitedly tells her that she is indeed a real psychic. He calls the police to collect the reward for capturing the kidnappers and finding the jewels. A smiling ORG winks at the camera.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.8267, -0.2054],
#         [ 0.8267,  1.0000, -0.0620],
#         [-0.2054, -0.0620,  1.0000]])

Training Details

Training Dataset

Unnamed Dataset

Size: 5,804 training samples
Columns: sentence_0 and sentence_1
Approximate statistics based on the first 1000 samples:
sentence_0 sentence_1
type string string
details
min: 11 tokens
mean: 116.74 tokens
max: 256 tokens

min: 15 tokens
mean: 234.66 tokens
max: 256 tokens

	sentence_0	sentence_1
type	string	string
details	min: 11 tokens mean: 116.74 tokens max: 256 tokens	min: 15 tokens mean: 234.66 tokens max: 256 tokens

Samples:

sentence_0	sentence_1
`An American ballet student arrives at the prestigious German academy that accepted her to begin her studies, but soon realizes that the school is a front for a world of murder and witchcraft.`	PERSON, a young American ballet student, arrives in GPE, GPE, GPE during a torrential downpour to study at ORG, a prestigious German dance school. She sees another student, PERSON, flee the school in terror. Suzy is refused entry to the school and forced to stay in town overnight. PERSON takes refuge at a friend's apartment and tells her that something sinister happened at the school. PERSON is ambushed by a shadowy figure who stabs her repeatedly and drags her to the roof of the apartment building before hanging her with a noose by throwing her through the building's skylight. PERSON's friend is also killed after being impaled by a falling giant shard of glass while trying to alert other tenants to the murder. Suzy returns to the school the next morning, where she meets PERSON, the head instructor, and ORG, the deputy headmistress. Tanner introduces PERSON, one of the school's servants. She also meets classmates PERSON and PERSON, her new roommate. Suzy experiences an unsettling encou...
PERSON (PERSON) is an executive who has grown accustomed to being alone, but doesn't particularly like it. Abandoned by her biological parents, she spent most of her childhood being raised by ORG), an adoptive father who never truly loved PERSON after her adoptive mother died. One day, PERSON learns that ORG has died. Alone in her apartment, after trying to feed her now-dead pet fish, she breaks down and cries uncontrollably. The next day at work, PERSON receives an unexpected flower delivery from a secret admirer. Intrigued, she presses the delivery man for information about who might have sent the flowers. He says the sender wishes to remain anonymous. PERSON asks her friends for names and visits the florist without success. After getting to know each other better, he confesses to sending them. PERSON (PERSON) runs a flower shop and often takes long walks around the neighborhood at night, trying to let go of memories of his deceased wife and son. He saw PERSON crying at her window an...	PERSON (PERSON) is a business executive who has gotten used to being alone but doesn't like it very much. She was abandoned by her birth parents and then spent most of her childhood being raised by ORG), a foster father who never really loved PERSON after her adopted mother died. One day, PERSON gets word that ORG has died. Alone in her apartment, after attempting to feed her now dead pet fish, she breaks down and cries uncontrollably. The next day at work, PERSON gets an unexpected delivery of flowers from a secret admirer. Puzzled, she presses the delivery man for information on who might have sent her the flowers. He says the sender wants to remain anonymous. PERSON asks her friends for names and visits the flower shop to no avail. After getting to know each other better, the florist confesses that he sent them. PERSON (PERSON) runs a flower shop and often takes long walks through the neighborhood at night, trying to lose memories of his deceased wife and child. He saw PERSON crying...
`The story focuses on the relationship between two teenage friends, PERSON (Thora Birch) and GPE (PERSON), who have just finished high school and feel insecure about the future. While conflicts arise that test their friendship, ORG meets PERSON (PERSON), a lonely and shy record collector, through an advertisement and realizes she has a lot in common with him.`	Best friends PERSON and GPE face the summer after their high school graduation, with no plans for their future, other than to find jobs and live together. The girls are cynical social outcasts, but GPE is more popular with boys than Enid. ORG's diploma is withheld on the condition that she attend a remedial art class. Even though she is a talented artist, her art teacher, PERSON, believes that art must be socially meaningful and dismisses ORG's sketches as nothing more than light entertainment. The girls see a personal ad in which a lonely, middle-aged man named PERSON asks a woman he met recently to contact him. ORG makes a prank phone call to GPE, pretending to be the woman and inviting him to meet her at a diner. The two girls and their friend, PERSON, secretly watch PERSON at the diner and make fun of him. ORG soon begins to feel sympathy for PERSON, and they follow him to his apartment building. Later they find him selling vintage records in a garage sale. ORG buys an old blues al...

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false
}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 4
per_device_eval_batch_size: 4
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: no
prediction_loss_only: True
per_device_train_batch_size: 4
per_device_eval_batch_size: 4
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 3
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.3446	500	0.0573
0.6892	1000	0.111
1.0338	1500	0.1244
1.3784	2000	0.0655
1.7229	2500	0.0642
2.0675	3000	0.0371
2.4121	3500	0.0168
2.7567	4000	0.0096

Framework Versions

Python: 3.9.21
Sentence Transformers: 5.1.2
Transformers: 4.57.6
PyTorch: 2.8.0+cu128
Accelerate: 1.10.1
Datasets: 4.5.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}