SentenceTransformer based on google/embeddinggemma-300m

This is a sentence-transformers model finetuned from google/embeddinggemma-300m. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: google/embeddinggemma-300m
Maximum Sequence Length: 256 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ahmedHamdi/IR-es-en-gemma")
# Run inference
sentences = [
    'A group of people find wolves dead from eating sand in the central city, an abandoned village on the shores of the Aral Sea. Upon arriving, the group discovers swarms of dogs abandoned by the village; in revenge for the brutality of the people, the dogs begin to attack. Overcome by fear and madness, the people begin to destroy each other.',
    " Born on a battlefield, Conan is the son of Corin, barbarian chieftain. As a child, he proves himself a skilled but violent warrior to his father. One day, Conan's village is attacked by Khalar Zym, a warlord who wishes to reunite the Mask of Acheron to revive his dead wife Maliva, an evil sorceress, and conquer Hyboria. The mask, crafted by sorcerers and used to subjugate the world, was broken into pieces and scattered among the barbarian tribes to prevent unification. After locating Corin's tribe's piece and slaughtering the entire village, Zym leaves Conan the only survivor. Years later, Conan, now an adult, is a pirate and hunts Zym, seeking revenge for his father's death. In the city of Messantia, he meets Ela-Shan, a thief being chased by Lucius, one of Zym's soldiers that Conan recognizes from his village's slaughter. He allows himself to be captured alongside Ela-Shan to confront Lucius. Torturing him, Conan learns that Zym seeks the female pureblood descendant of the sorcerers of Acheron to unleash the mask's power. Conan frees Lucius' prisoners, who kill Lucius. Zym and his daughter, the sorceress Marique, attack a monastery to find the pureblood descendant. Fassir, a monk, tells his student, Tamara, to flee, knowing she is whom Zym seeks. Conan rescues Tamara and captures Zym's righthand man, Remo. After learning Tamara's identity, Conan catapults Remo into Zym's camp, killing him and sending Zym a message. Zym and Marique confront Conan and Zym finally recognizes him as the boy he left alive. Amidst their battle, Marique poisons Conan with a boomerang sword and he is rescued by Tamara. They jump into the sea and return to Conan's ship, where he recovers. The boat is attacked by Zym's men but Conan and his gang defeat them. Conan departs for Zym's kingdom. His friend Artus sends Tamara after him, and the two spend the night together. The next day, as she is returning to the boat, Tamara is captured by Zym's men. Conan asks Ela-Shan for help breaking into Zym's castle. Zym uses Tamara's blood to mend the mask, planning to use her body as a vessel for his wife's soul. After confronting a tentacled monster that guards Zym's dungeons, Conan frees Tamara and battles Zym, eventually reclaiming the sword Marique had stolen from his father. He also cuts off Marique's hand and she is fatally impaled. Conan and Tamara become trapped on a bridge as Zym uses the mask's power to call forth his wife. Her spirit begins to possess Tamara's body. Conan destroys the bridge, causing Zym to fall into the lava below as he and Tamara escape. Conan returns Tamara to her birthplace, then travels to his old village. He recalls the memory of his father, content that he has avenged his death and recovered his father's sword, thereby restoring his honor.",
    "Not long after the invasion and occupation of Earth by a race of powerful robots wanting human knowledge and ingenuity, humans are confined to their homes. Leaving without permission would be to risk their lives. Monitored by the electronic implants in their necks, the robot sentries are able to track the movement of humans in order to control them. If any person attempts to exit their home, they are given warnings by the robot sentries to return to their home. If he or she does not comply within ten seconds, they are killed. At the beginning of the film, a teen going by the name of Sean Flynn (Callan McAuliffe), is seeking his father, who went missing not long after the robots invaded, sending out hand-drawn lost posters hidden in tennis balls and fruit. Later, Nathan, a friend of Sean's, is seen attempting to repair his PlayStation, when Connor, a young boy, accidentally shocks Nathan, while a girl named Alexandra watches. The group discovers that Nathan's implant has been turned off by the electrical shock, and then perform the same procedure on each other to stay outside without being tracked down. The group enters a local museum before Sean suggests that they go look for his father, Danny (Steven Mackintosh) at the school, where the files on all the people are kept. They discover that Danny is still alive, having been moved to a hotel, but are then caught and brought to a room with a deep scanner after their implants reboot. Here, Robin Smythe (Ben Kingsley) asks them how they turned off their implants. When they refuse to answer, Sean's uncle is brought in, and receives a black implant, before being subjected to a deep scan, a painful process that searches through all of a person's mental faculties before rendering them unable to eat, causing them to die in a few days. When the children still refuse to answer, Sean is also given a black implant before being subjected to a deep scan. In the midst of it, Sean insults Smythe, causing him to accidentally interrupt the deep scan, allowing Sean to escape the deep scanner alive. A few seconds later, Connor, who had been left outside, bursts in with a makeshift fireworks launcher and frees the other children. The children hide in a bowling alley, where they turn off their implants once again before running to the hotel. When a large robot walks by, the four children hide next to a doorway, where Sean inadvertently controls the robot. The children then meet Monique, a woman who wants to know how to turn off the implants. In exchange, the boys meet Swanny, who has had his implant removed by a watchmaker, and also tells the children to go to a Stone Circle. However, an announcement reveals that Sean's mother, Kate, has been taken a prisoner in the area headquarters, a castle. With Monique's help, the children successfully get to the castle. However, they are caught by a large robot. Sean then discovers that he can take control of the robots because of his black implant, after discovering that the large robot responds to his movement. Meanwhile, Smythe is speaking with Kate about how he and she could live together before an alarm goes off. Smythe leaves to go see what has happened. A few moments later, a young guard is tricked into giving the keys to the door to Kate. Meanwhile, outside, Sean appears to have been caught by the large robot, with Smythe scolding him. Sean then turns the large robot's weapon on Smythe and his team, forcing them to drop their weapons, which are picked up by the other children. Kate suddenly dashes by on a horse, causing Smythe and his team to follow. This opens the line of fire for two clankers, insectoid robots with a top-mounted weapon, which destroy the large robot, forcing Sean to hide behind it. One of the clankers then jumps down and prepares to fire at Sean, before Sean takes control of the robot and uses it to destroy the other, before commanding the clanker to deactivate. The children then track down Kate, before heading to the Stone Circle, deciphering a message written in graffiti to find the location of a human camp, an old tin mine. Meanwhile, Smythe is told by Mediator 452, a recurring character in the film, that a large number of deep scanners are arriving and that Smythe will be the first to be scanned if Sean is not captured by the time that they arrive. Sean and his friends are seen arriving at the human camp, where Sean is reunited with his father. The small group has their implants removed. The next day, the robots descend on the community seeking Sean, who is quickly discovered to be missing. Sean is seen reinstalling his black implant, before mentally controlling a robot craft that rams and destroys the cube, the local robot mothership, halting the invasion. He narrowly escapes the impact. He interfaces with a damaged Mediator, a robot in human form, and mentally sends commands to end the invasion of the Earth, destroying the robots and their craft. The film ends with jubilation as the local population celebrates in town. In the closing scene, Sean looks up at the stars.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.4836, 0.1713],
#         [0.4836, 1.0000, 0.2493],
#         [0.1713, 0.2493, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

Size: 2,340 training samples
Columns: sentence_0 and sentence_1
Approximate statistics based on the first 1000 samples:
sentence_0 sentence_1
type string string
details
min: 11 tokens
mean: 135.12 tokens
max: 256 tokens

min: 11 tokens
mean: 225.08 tokens
max: 256 tokens

	sentence_0	sentence_1
type	string	string
details	min: 11 tokens mean: 135.12 tokens max: 256 tokens	min: 11 tokens mean: 225.08 tokens max: 256 tokens

Samples:

sentence_0	sentence_1
Ted Stryker is a Canadian ex-military pilot who blames himself for the deaths of his squadron members during World War II. Despite the passage of time, this guilt continues to haunt him, causing him to have a fear of flying and of piloting again. This leads his wife, Elena, to leave him one day. Despite his phobia, Ted arrives at the airport in time to buy a ticket for the same flight to Vancouver as his wife. During the flight, some of the passengers and crew—including the pilots—suffer a bacterial infection from spoiled food. It is then discovered that Ted is the only passenger capable of taking control of the aircraft.	During the closing days of the Second World War, six members of the Royal Canadian Air Force fighter squadron led by pilot Ted Stryker are killed because of a command decision he made. Years later, in civilian life in Winnipeg, Manitoba, a guilt-stricken Stryker goes through many jobs, and his marriage is in trouble. Stryker finds a note at home: his wife Ellen has taken their young son Joey and is leaving him, flying to Vancouver. He rushes to Winnipeg Airport to board the same flight, Cross-Canada Air Lines Flight 714. He asks his wife for one last chance, but Ellen says that she can no longer love a man she does not respect. Stewardess Janet Turner begins the meal service, offering meat or fish as the choices. When a number of passengers and the co-pilot begin feeling sick, a doctor aboard determines that the fish is the cause. The pilot also becomes seriously ill and cannot fly the airplane. Before he passes out, he turns on the autopilot. The stewardess determines that Stryker is...
`A day in the life of a typical farming family in central Chile. The grandmother sells cheese by the roadside, the grandfather works in the fields, the daughter is a cook at a guesthouse, and the grandson goes to school. In four separate sections, we follow them through their small joys and sorrows, throughout a day that unabashedly reveals a changing Chile, one that few truly understand.`	In 1969, at the time of the Apollo 11 mission, American soldier Dalton Joiner, fighting in the Vietnam War, uses his time of R&R supposed to be spent in Hong Kong, to fly back to the United States to re-capture the heart of his girlfriend Jane. Fellow soldier Mickey Wright accompanies him. Jane now calls herself Juniper, and is a member of a group of anti-war activists. Joiner and Wright pretend they are AWOL, and are admired by the group for that, until it is revealed that they plan to return in time. Juniper breaks up (again) with Joiner, which makes him decide to flee to Canada. Wright falls in love with Candace, but returns to Vietnam.
`Sharon Curley is 20 years old and the eldest daughter of an Irish family. She has become pregnant and refuses to reveal to anyone who the father of her unborn child is, which soon makes her the subject of neighborhood gossip.`	Soon after a wild night at the pub, twenty-year-old Sharon Curley finds herself expecting a little snapper by a man she loathes. Her refusal to name the father sets in motion a family drama involving her three brothers, two sisters, and her parents, along with her employers and all her friends. Kellegher, playing the role as a coarse, earthy, yet remarkably sensible young woman soon discovers who her friends really are, as some people tease and torment her, some make remarks to her siblings, some force her father to take direct action in her defence, and all spread gossip. She decides to keep the baby (snapper) and her family, each in their own way, eventually decides to support her. Her father particularly studies up on childbirth and female anatomy (with gratifying results for his wife as a bonus). Des Curley, Sharon's father, shows the whole world in his face, his emotions ranging from outrage toward Sharon for embarrassing the family to tender concern as her time draws near. As the...

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false
}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 4
per_device_eval_batch_size: 4
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: no
prediction_loss_only: True
per_device_train_batch_size: 4
per_device_eval_batch_size: 4
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 3
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.8547	500	0.5078
1.7094	1000	0.2785
2.5641	1500	0.0663

Framework Versions

Python: 3.9.18
Sentence Transformers: 5.1.2
Transformers: 4.57.6
PyTorch: 2.8.0+cu128
Accelerate: 1.10.1
Datasets: 4.5.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}