SentenceTransformer based on google/embeddinggemma-300m

This is a sentence-transformers model finetuned from google/embeddinggemma-300m. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: google/embeddinggemma-300m
Maximum Sequence Length: 256 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ahmedHamdi/IR-all-en-gemma_masked")
# Run inference
sentences = [
    "Wolfhound, or PERSON, is the last of the Gray Dog clan, slaughtered when he was still very young. After a hard-fought battle to earn his freedom from the mines where he was forced to work as a slave, he becomes a formidable warrior. With his only friend, a bat, he searches for ORG, who killed his parents. He travels to the castle of ORG, or ORG, who participated in the destruction of his village, and kills him. While fighting ORG, he discovers two prisoners: the scholar ORG and a slave, PERSON. After freeing them, he accompanies them to the city of ORG, whose inhabitants are in utter disarray: they are preparing for an imminent attack by the dark druid GPE. Therefore, the king of the city has decided, in order to save his city, to give his daughter, Princess Elen, in marriage to the warrior PERSON, who has promised to protect ORG. Princess Elen fears the long and perilous journey to her future husband's land, and PERSON asks to be her bodyguard. She accepts, and he accompanies her. During the journey, Wolfhound witnesses several mysterious events, while the true purpose of the trip is gradually revealed...",
    "The setting of the film is a high fantasy Dark Ages Europe, in which desperate and bloodthirsty warlords fight brutal battles in their eternal quest for overlordship. Yet their swords, shields, lances, spears and arrows are all brittle, prone to wear and tear, and dull, protracting their campaigns against each other without end. Word spreads of a man in one of the Northern tribes: an adept blacksmith capable of crafting far hardier, stronger, sharper, and more durable weapons than any other known to exist, with aid of a mystical element. The warlords search for the enigmatic master of weapons to no avail. A former druid chieftain named GPE, jaded by years of warfare and believing the Radiant Gods have abandoned him, learns of the identity and whereabouts of the blacksmith and his coveted weapons, more valuable than any gold in such a time. With this knowledge, GPE's heart is hardened to mercy and replaced with greed and bloodlust. Bringing even the feared warlord, ORG, under his spell, GPE massacres an entire people, the Clan of the Grey Hounds, including their blacksmith, and takes his legendary weapons. The only survivor of the tribe, the ORG's fair-haired son, witnesses the slaughter and bites at GPE's tattooed hand. Instead of killing the boy, GPE orders the boy to be sent into the savage mines of LOC and to live a life of hard labor there, in conditions which will no doubt kill the boy anyway. Against all odds, the boy cheats death through sheer determination and will, his strength becomes powerful through many years of hard labor, scarred by violence, and his prowess honed by a hunger for revenge. For many years his only companions were disheartened, tortured slaves and a strangely intelligent bat, PERSON, who is destined to become his constant companion. After killing his overseers with weapons created using his father's secret techniques and escaping the mines, he becomes a wanderer known as Wolfhound: a great and fearless warrior, as well as a true master of weapons like his father before him. After making the long journey to ORG castle, GPE finally conquers his archenemy. He also frees two prisoners, the sage and healer ORG and slave-girl GPE. Wolfhound accompanies them to their home city of ORG, which has been thrown into turmoil. The power-crazed forces under GPE are poised to attack the city with their superior weapons at any moment. With enough bloodshed, and the right ritual involving royal blood, GPE believes he can resurrect the ancient ORG from ORG and draw power from her. ORG, to save the city from destruction, is giving his daughter away in marriage to GPE, a young warrior-prince who promises to protect ORG - and who also happens to be the son of the late warlord, ORG. Princess Elen must travel to the land of her husband-to-be, and asks PERSON to be her guard in this dangerous journey. Wolfhound agrees to serve the princess and is caught up in a whirlwind of mysterious events, as the true purpose of the journey is gradually revealed, a secret which could plunge the world into an eternal living nightmare...",
    " PERSON, a twelve-year-old horse-crazy girl, lives with her family in GPE, a small village in GPE, GPE. After winning a spirited gelding in a raffle, she dreams of training him for the Grand National steeplechase. Penniless young drifter PERSON, who discovered Mrs. PERSON's name and address among his late father's effects, arrives at the PERSON farm. Hoping to profit from the association, ORG accepts an invitation to dinner and a night's lodging at the PERSON' home. PERSON is unwilling to allow Mi to trade on his father's good name and remains vague about their connection. Nevertheless, she convinces her husband to hire Mi as a store helper, over his better judgment. It is eventually revealed that ORG's career as a steeplechase jockey ended in a collision that resulted in another jockey's death. The accident left Mi fearing riding and hating horses. Velvet calls her horse The Pie because his previous owner called the troublesome gelding a pirate. Seeing the ORG's natural talent, PERSON pleads with Mi to train him for the Grand National. Mi believes it a fool's errand, not because the horse lacks the ability, but because they are unable to finance the effort. He makes his case to Mrs. PERSON, but she consents to ORG's desire to train the horse. To cover the entrance fee and other costs, Mrs. PERSON gives ORG the prize money she won for swimming across the English Channel. Velvet and Mi train the Pie and enter him into the race. Mi and ORG travel to the Grand National. Mi hires a professional jockey, but the night before the race, ORG senses he lacks faith in the ORG and will lose. ORG dismisses the jockey, leaving them without a rider. That night, Mi overcomes his fear of riding and intends to race the Pie himself only to discover ORG wearing the jockey silks and intending to ride. Knowing the dangers, Mi attempts to dissuade PERSON, who is determined to ride. As the race unfolds, ORG and the Pie clear all hurdles and win the race. Elated but exhausted, ORG falls off her mount just after the finish. However, PERSON and Pie are disqualified for violating the rule requiring the winning jockey not to dismount before reaching the enclosure.   When it is discovered that the jockey is a girl, ORG becomes a media sensation and receives lucrative offers to travel to GPE and be filmed with the Pie. To her father's disappointment, ORG tearfully declines all offers, claiming that the ORG would not understand the intense scrutiny. PERSON says that she raced the Pie at the Grand National because he deserved a chance for greatness. ORG chooses a normal life for herself and her horse.  Sometime later, Mi, ready to resume his old life, takes his leave without bidding ORG goodbye. ORG is heartbroken, but PERSON says it was time for him to resume his old life. She gives ORG permission to tell Mi that his father coached PERSON to swim ORG. Astride the Pie, Velvet catches Mi at the top of a hill against a sunset sky, where she tells him about his father.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.6661, 0.0510],
#         [0.6661, 1.0000, 0.0435],
#         [0.0510, 0.0435, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

Size: 24,420 training samples
Columns: sentence_0 and sentence_1
Approximate statistics based on the first 1000 samples:
sentence_0 sentence_1
type string string
details
min: 6 tokens
mean: 122.02 tokens
max: 256 tokens

min: 19 tokens
mean: 231.02 tokens
max: 256 tokens

	sentence_0	sentence_1
type	string	string
details	min: 6 tokens mean: 122.02 tokens max: 256 tokens	min: 19 tokens mean: 231.02 tokens max: 256 tokens

Samples:

sentence_0	sentence_1
Jan, a young woman of legal age, lives alone with her father, PERSON, a prominent lawyer struggling with alcoholism. Father and daughter are extremely close. PERSON has raised his daughter with a strong sense of freedom, believing she should learn from her experiences and experiences. This attitude puts PERSON and Jan at odds with the rest of their family. PERSON is tasked with defending PERSON, a gangster accused of murder who has little chance of escaping the gallows. Behind the scenes of the trial, ORG is introduced to Jan, and the two are immediately drawn to each other. As the trial concludes, Jan visits her grandmother, who is celebrating her 100th birthday. During the reception, Dwight announces his engagement to Jan. PERSON, who has successfully exonerated ORG, joins her later, accompanied by ORG himself. The family then makes it clear to ORG that he is not welcome. This attitude shocks Jan, who leaves with him. Jan becomes ORG's mistress and eventually tells her father, who di...	Defense lawyer PERSON successfully defends known gangster PERSON from a murder charge, despite his knowledge of ORG other illegal activities. His upper-class family has all but disowned him and his daughter PERSON, due to GPE's alcoholism and PERSON's free spirited willfulness. Jan is engaged to clean-cut PERSON, but their relationship is threatened when she meets PERSON and becomes enamored of him and his exciting life. As PERSON continues to slip deeper into alcoholism, Jan breaks her engagement with PERSON and begins a clandestine affair with ORG, which grows into love. This comes to a head when ORG asks a drunken PERSON if he can marry Jan; GPE, offended by the request, angrily refuses, and when he discovers Jan in ORG boudoir, takes her home. They have an argument over their respective vices, and Jan proposes a deal: she will never see PERSON again if GPE will give up drinking. Despite knowing he cannot keep his promise, PERSON, and the two of them leave for a cleansing camping ho...
`PERSON, a printing company owner, tells his banker, PERSON, that if he can't collect a seven-year-old invoice from PERSON Perrin within a week, he's ruined. PERSON also tells PERSON that if he doesn't receive the invoice from GPE within the week, he's also ruined. The two young men are evicted from GPE's cement factory. PERSON goes to a store owned by GPE, prepared to stay there until he receives payment.`	As described in a film magazine review, PERSON, owner of a printing business, tells his credit manager PERSON that if he cannot collect a seven-year-old bill for $25.11 against PERSON in a week, he is through. PERSON also tells PERSON that if he does not get ORG's account within a week, he is through, too. Both young men are thrown out of GPE's cement yard. PERSON goes to a modiste shop owned by GPE, ready to park there until he receives payment. He makes various attempts to see GPE, who finally beats him up and wrecks the shop. When ORG wins the love of GPE's daughter and, with her connivance, finally secures the payment of the bill, Perrin capitulates and offers him a job.
Overview In GPE, a new villain is terrorizing the city with his cold: Mister PERSON. His real name is ORG, and this former scientist was injured in an accident during his cryogenics research. He can no longer withstand temperatures above 0 degrees Celsius. His goal is to take the city hostage in exchange for funds to finance his research into a cure for the disease, a rare illness contracted by his wife. While ORG and PERSON try to thwart PERSON's plans, in LOC, Dr. PERSON becomes ORG Ivy and sets out to eradicate humanity to allow the development of genetically modified carnivorous plants. Assisted by PERSON, a super-fighter powered by tropical flower venom, she arrives in PERSON to carry out her plan and allies herself with ORG. The two superheroes must fight this evil duo, while overcoming their disagreements and the rivalries created by Ivy's chemical manipulations. They are joined in their fight by Batgirl.	Batman and his partner, PERSON, encounter a new foe, Mr. PERSON, who has left a string of diamond robberies in his wake. During a confrontation in the natural history museum, ORG steals a bigger diamond and flees, freezing PERSON and leaving ORG unable to pursue him. Later, ORG and PERSON learn that PERSON was originally Dr. PERSON, a scientist working to develop a cure for ORG's Syndrome, hoping to heal his terminally ill wife, PERSON. After a lab accident, Fries was rendered unable to live at average temperatures and forced to wear a cryogenic suit powered by diamonds for survival. At a PERSON lab in GPE, botanist Dr. PERSON is working under the deranged Dr. PERSON, who has turned her research on plants into the supersoldier drug GPE. After witnessing Woodrue use the formula to turn serial killer PERSON into the hulking PERSON, she threatens to expose PERSON's experiments. Woodrue attempts to kill her by overturning a shelf of various toxins; instead, ORG is mutated by the toxins in...

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false
}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 4
per_device_eval_batch_size: 4
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: no
prediction_loss_only: True
per_device_train_batch_size: 4
per_device_eval_batch_size: 4
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 3
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.0819	500	0.1206
0.1638	1000	0.0995
0.2457	1500	0.1492
0.3276	2000	0.2053
0.4095	2500	0.2165
0.4914	3000	0.2356
0.5733	3500	0.2614
0.6552	4000	0.2736
0.7371	4500	0.2743
0.8190	5000	0.2452
0.9009	5500	0.2576
0.9828	6000	0.2572
1.0647	6500	0.2198
1.1466	7000	0.1871
1.2285	7500	0.1996
1.3104	8000	0.1683
1.3923	8500	0.1731
1.4742	9000	0.153
1.5561	9500	0.1626
1.6380	10000	0.1818
1.7199	10500	0.1203
1.8018	11000	0.1466
1.8837	11500	0.1335
1.9656	12000	0.1336
2.0475	12500	0.1032
2.1294	13000	0.085
2.2113	13500	0.0738
2.2932	14000	0.0997
2.3751	14500	0.0709
2.4570	15000	0.0903
2.5389	15500	0.0646
2.6208	16000	0.0838
2.7027	16500	0.0994
2.7846	17000	0.0793
2.8665	17500	0.1048
2.9484	18000	0.0595

Framework Versions

Python: 3.9.21
Sentence Transformers: 5.1.2
Transformers: 4.57.6
PyTorch: 2.8.0+cu128
Accelerate: 1.10.1
Datasets: 4.5.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}