SentenceTransformer based on google/embeddinggemma-300m

This is a sentence-transformers model finetuned from google/embeddinggemma-300m. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: google/embeddinggemma-300m
Maximum Sequence Length: 256 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ahmedHamdi/narrative-similarity-es-en-gemma-masked-NE")
# Run inference
sentences = [
    'It tells the story of how government agencies, along with oil companies, exploited the newly discovered oil in the historic city of Oloibiri.',
    'When crude oil was discovered in Oloibiri in 1956, an educated ORG (Olu Jacobs) could not influence his kin to demand that they have control over how the oil company extracts the oil, and ensure that their people are employed in decent positions in the companies because they were satisfied with the minute promises of the ORG company- a fictitious representation of ORG. After many years, he is now tormented by his past- the catastrophe that befell the people of Oloibiri. However, GPE, aka PERSON (PERSON), a graduate of geology and a former employee of ORG, and his gang have become vigilantes in the oil-rich region and proving the nemesis of white companies and locals looking to further exploit the area. Gunpowder kills his former partner, PERSON, whom he accuses of greed. However, things get pacier when news arrives that there’s been an approval for PERSON to extract oil from ORG, a nearby community- a deal worth $300 million in revenue each year. But ORG had infiltrated PERSON and has a spy working as a personal assistant to the founder and director of the organisation, PERSON (PERSON). The spy, PERSON, ORG) helps Gunpowder deliver a mail to PERSON in a bid to dissuade PERSON from continuing with the plan. PERSON finds pictures of malnourished Nigerian children and those affected by crude oil poisoning inside the envelope. Nevertheless, PERSON has good intentions for the communities and wants to help solve their problems, unlike other oil companies. He decides to visit GPE to see things for himself with the help of their Nigerian representative, PERSON (TK Bello). Cyril seldom cares about the state of the communities and is only interested in enriching his pocket. PERSON arrives in GPE but his convoy including ORG, and a group of soldiers, is ambushed by heavily armed Gunpowder and his men. PERSON initially escapes with the help of a soldier and is found by ORG’s grandson’s love interest, a doctor who takes PERSON to ORG’s house. Timipre threatens to take PERSON to Gunpowder but the lovers manage to convince him to help PERSON. He eventually agrees and connives with ORG’s mother to find a haven for PERSON. Meanwhile security personnel were already aware of the attempted kidnap. PERSON and PERSON, ORG accomplices in the GPE have kidnapped PERSON’s family and his second in command (Mr PERSON), ransoming him to initiate a cancellation of ORG (ORG).\xa0 However, an exasperated Gunpowder eventually captures ORG, his grandson, the doctor and PERSON. The ORG is now cancelled but Gunpowder reneges on his agreement with ORG to release GPE when the ORG is cancelled, insisting that PERSON and the likes of PERSON must pay with their blood. Infuriated by Gunpowder’s betrayal and blood lust, PERSON reports himself to the police and reveals PERSON hideout. In the meantime, ORG, his grandson, the doctor, and PERSON were escaping captive after ORG kills one of ORG boys. Heavily armed soldiers then attack PERSON hideout, outnumbering and overpowering them. Gunpowder is shut down from his watchtower and as he falls, a glass cup falls and shatters from the hands of his mother, a sign that his son had died. Then we see a younger version of him in front of an altar with a Rosary, saying, “Father please bless my Rosary.” Upon landing, his body is soaked in a pool of crude oil pouring from the top. The movie ends with a newspaper report that ORG has become a representative of the community and will be working together with PERSON in the best interest of the locals.',
    "This story takes place at LOC School,(presumably somewhere in GPE judging by the landscape) where PERSON is a shy 16-year-old student who has had a secret romantic interest in PERSON for two years. She is surprised when he starts to show a romantic interest in her, considering he is a senior wrestler, and the most popular guy in school. ORG two best friends, ORG and PERSON,  accompany her to an after school wrestling match where PERSON and Stacy first lock eyes with each other and ORG encourages PERSON to go for it. One night ORG and Stacy hang out at a local pizza parlor where ORG gets to meet PERSON face to face and ORG gets to meet his friend PERSON, another wrestler on the school team who in turn notices ORG. PERSON approaches them at their table at the restaurant and invites ORG to one of his wrestling practices after school and they hit it off. PERSON then invites PERSON and her friends along with his friends to the local hangout, a lake for a late night lake party and bonfire. They sneak off and have an intimate talk and they both find out that their fathers had abandoned them. They then decide to start dating. That same night ORG starts a relationship with PERSON. In a side story we meet ORG’s mom, PERSON, who is a working single mom dating a man named PERSON whom Stacy deeply dislikes because she believes him to be emotionally abusive towards her mom. She often urges her mom to leave him and find someone who treats her better. After a few weeks of dating, PERSON starts showing possessive behavior. (Some instances include an episode in the school library when friends try to join in a private study session between PERSON and PERSON and he angrily chastises Stacy for inviting them to sit with them, and angrily refusing to allow ORG to meet his mom just because he doesn’t want it then giving Stacy a list of controlling rules like she needs to be home at night when he calls). He is also jealous whenever another guy approaches or talks about ORG, and he does not want her to hang out with anyone when they are together. At first, she does not suspect that there is anything wrong, because he immediately apologizes after getting mad at her, tells her that he loves her, and gives her gifts to show his remorse, like a beautiful gold bracelet and roses (because PERSON told him early on she likes getting flowers from a guy the night they met in the pizza parlor). One day ORG and ORG go shopping and ORG reveals bruises on her back in the fitting room, concerning ORG. Stacy reveals to ORG that the bruises came from one night out at the lake when PERSON and PERSON shared their first sexual experience, hence Stacy losing her virginity. Stacy promises after the fact that PERSON and her will be together forever which she reveals to ORG.  At school, boys start to notice ORG after ORG convinces her to wear a mid-thigh length skirt she bought to wear to impress PERSON. In the boys' locker room, one of the boys gives Stacy a compliment, and this outrages PERSON. After this, he seeks out ORG in the school halls wearing the skirt and furiously tells her she looks like a slut, demands that she wear proper clothes, and forces her to change into a pair of his track pants in the bathroom. She tries to explain that she was only dressing up for him, but he becomes more furious and throws and slams her into the wall in the gym during wrestling practice which worries the coach and other wrestlers who notice what happened. The coach questions this, concerned he may have to report the incident but ORG defends PERSON by saying she “slipped”.Back at home, PERSON worries that ORG is spending too much time with PERSON, but PERSON assures her that he loves her and vice versa. Meanwhile, PERSON and PERSON also grow concerned about ORG's relationship with him. Stacy won’t join in helping them decorate the school gym for a dance because PERSON has limited ORG’s time with all her friends. PERSON soon learns from PERSON's cousin, PERSON, that his last girlfriend PERSON changed schools after being hit by him. ORG’s worry continues to grow when she notices that ORG has several more unexplained bruises on her arm. She talks to her about the rumor she heard, but PERSON, frustrated, shuts her down and assures her that she can take care of herself. By accident, while out shopping after school, ORG finally meets PERSON's mother; PERSON catches them talking to each other and becomes furious and violent slamming ORG into a truck and telling her that she needs to mind her business and not talk to his mom. Upset, Stacy  refuses to see PERSON again, and later that night takes out her anger on her mom PERSON, calling her irresponsible because of her own emotionally abusive relationship with PERSON saying she’d rather have a mom than a “friend” who goes out all the time with her boyfriend. PERSON later wins ORG’s trust back by sneaking into her house after PERSON goes out for the night with PERSON. PERSON tells PERSON about the alcoholism that runs in his family. His father was an abusive alcoholic and his mother would not leave him. He left on his own. PERSON manages to coax PERSON back into a relationship by appealing to her about her own issues with her mom and relating to her with stories of his estranged dad. andThey later attend the big 1950s themed sock hop school dance where they party and dance with friends. They are happy for a while, but trouble begins again when he sees her talking to another boy at the dance. He pulls her outside into the parking lot and slaps her, all of which is witnessed by PERSON and PERSON, who are nearby.  PERSON quickly confronts PERSON about how badly PERSON is treating her, but she defends him, saying he has been through a lot. When PERSON says that PERSON doesn't love her, she angrily says, Then you don't know what love is! before taking off with him in his car. PERSON drops PERSON off at home and outside ORG hears her mom arguing with PERSON. Stacy stays at ORG’s house afterwards and while there ORG helps her see the truth about PERSON and how he is not treating her right. Also while there, PERSON calls ORG and tells her that she ended things with PERSON which further helps Stacy see the light, helping Stacy confirm her choice to end it with PERSON because she sees her own mom trying to end her dysfunctional dating cycle, too. Stacy ends her relationship with PERSON at school. When he does not take it well, she offers for them to just be friends, telling him that she needs to focus on her other friends and her studies. Stacy gets invited to a bowling birthday party next Saturday night with ORG and all her other friends to forget PERSON but another mutual friend, PERSON (who has a secret crush on PERSON) calls him from a pay phone trying to make plans with him. He asks to talk to ORG and he offers to pick her up and have her hang out at his place with him and his other friend PERSON. He invites PERSON too. While at his place he accuses PERSON of being out with another guy at the party and slaps her again despite the fact that they are no longer dating. In private he kisses PERSON and tells her if anyone asks it was just him and ORG there and that they fought and she ran out. Later that night, PERSON, accompanied by PERSON, convinces PERSON to get in his car for a ride. He drives them to the lake and walks off with her to be alone while PERSON waits in the car. The next day, it is reported that she is missing.  PERSON calls ORG looking for ORG and ORG in turn asks PERSON what happened. PERSON claimed that PERSON and him had a fight and she ran out. They go out searching for her. Most people guess that ORG hitchhiked and was murdered, some think she might be staying somewhere after a fight with PERSON.TV news coverage of ORG’s disappearance links it to a classmate but PERSON suspects that PERSON has something to do with it especially after she finds PERSON's purse in his room. PERSON convinces her that he is very concerned about ORG’s disappearance and he enlists everyone to help look for her. PERSON is also convinced that PERSON is responsible and asks PERSON, who was at PERSON's house on the night of ORG's disappearance, for information. Because of PERSON exploiting her attraction to him PERSON insists that all she knows is ORG ran out of the house after arguing with PERSON, which frustrates PERSON. Accompanied by PERSON, PERSON goes to the police, informing PERSON and PERSON about how abusive ORG's relationship with PERSON was. PERSON eventually talks to them as well, admitting that PERSON and PERSON took PERSON to the lake. Detective PERSON gets an anonymous tip from a phone call from PERSON telling him to “go out by the lake”. Vince hangs up before the detective can identify him. He goes by the lake and finds a flower washed up by the shore and keeps it as evidence. PERSON and PERSON do get arrested and PERSON blames PERSON's disappearance on PERSON. When confronted by this lie, he admits that PERSON was the one last seen with her, and later reveals that when he came back alone, he said that if he can't have her, no one's gonna. PERSON is free to go, since he told PERSON the truth, that he didn't know if PERSON killed PERSON. Detective PERSON is then convinced PERSON did so when she ended their relationship and refused to get back together. When he confronts him, showing the flower from the lake, he realizes he is cornered and shortly admits that he slit her throat when she refused to get back together with him and then disposed of her murdered body in the lake. ORG's body is soon found in the lake, wrapped in a trash bag bound with duct tape,and tied down with cinder blocks. PERSON and PERSON grieve together as the coroner takes her away. A court trial soon follows with the district attorney asking everyone if they ever saw PERSON hit Stacy. PERSON says she did, but that it was because ORG wouldn't listen to him. PERSON and other classmates also admit that they witnessed the abuse, but figured she would leave him eventually. Val, PERSON, PERSON, and PERSON expose the abusive ways that PERSON treated Stacy, with PERSON saying how she desperately wanted to oust him, but was afraid of losing ORG's friendship if she were to say what he was doing to her. He is eventually found guilty and sentenced to life in prison without the possibility of parole. PERSON breaks down in tears of relief from the verdict and PERSON’s mom tearfully hugs her son. Before court is adjourned, the judge addresses everyone in the room (particularly the younger audience) to tell someone next time they see a friend being abused instead of standing by and doing nothing, implying how it could have saved ORG's life.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.7799, -0.0668],
#         [ 0.7799,  1.0000, -0.0696],
#         [-0.0668, -0.0696,  1.0000]])

Training Details

Training Dataset

Unnamed Dataset

Size: 3,644 training samples
Columns: sentence_0 and sentence_1
Approximate statistics based on the first 1000 samples:
sentence_0 sentence_1
type string string
details
min: 10 tokens
mean: 133.34 tokens
max: 256 tokens

min: 3 tokens
mean: 231.22 tokens
max: 256 tokens

	sentence_0	sentence_1
type	string	string
details	min: 10 tokens mean: 133.34 tokens max: 256 tokens	min: 3 tokens mean: 231.22 tokens max: 256 tokens

Samples:

sentence_0	sentence_1
`A crime boss tries to escape with loot belonging to another thief.`	`To get back in the good graces of her murderous boss (PERSON), a seductive thief (PERSON) recruits an ex-lover (PERSON) to steal rare and valuable gems.`
`Now it's ORG's (PERSON) turn to have to deal with his wife's daughter's father struggling to keep the girl and kick him out of the family, PERSON and PERSON (PERSON and PERSON) during the holidays, along with ORG's conflict with his children's biological father, PERSON (PERSON).`	Finally friends, PERSON and PERSON co-parent ORG's two children, PERSON and PERSON, who take turns at each father's house. ORG has remarried to PERSON, a doctor/novelist, and is a stepfather to her daughter PERSON. The men decide to have a joint Christmas rather than separate ones, after PERSON embarrassingly admits she would prefer one together Christmas. ORG's tough fighter pilot/PERSON astronaut father PERSON and PERSON's overbearing and cheerful father, PERSON, unexpectedly arrive for Christmas. PERSON and GPE warmly embrace PERSON, as he is very present in their lives, while PERSON hasn't seen them since they were toddlers. From his envy of the affection the children show PERSON, PERSON rents a large cabin through ORG, to house them all through the holiday week. PERSON and ORG's co-parenting is put to the test by PERSON's meddling. ORG's beef with PERSON mainly stems from the fact that he was rarely there for him as a child. Especially painful was when, during a youth PERSON reci...
`In August 1968, actress PERSON (PERSON) tells a reporter about a nightmare in which she and her former boyfriend, PERSON (PERSON), were murdered in their home at 10050 PERSON in GPE. A year later, PERSON, now pregnant, returns home after spending six months in LOC filming her latest movie. Despite her joy at being reunited with her friends PERSON, PERSON (PERSON), and ORG (Pawel Szajda), a series of strange events begins to make her feel unsafe in the house she loves so much.`	In August 1968, actress PERSON describes to a journalist a nightmare she had, in which she and PERSON, her friend and former lover, had their throats slashed. One year later, a very pregnant PERSON returns home after spending six months in LOC filming The Thirteen Chairs. She is initially happy to be reunited with PERSON, as well as friend/house-sitter PERSON and PERSON's boyfriend PERSON. PERSON's husband , PERSON, has stayed behind in LOC to complete the script for his next film The Day of the Dolphin, but has assured PERSON he will be home in time for the birth of their baby. That night, PERSON tells PERSON that she believes Roman is having an affair; PERSON tells her it doesn't have to be this way, but PERSON is determined to save her marriage for the sake of the baby. Later, PERSON becomes angered upon finding that a new caretaker, PERSON, is living in a trailer on the property without her knowledge and asks what happened to the previous caretaker, PERSON. She is told PERSON hasn'...

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false
}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 4
per_device_eval_batch_size: 4
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: no
prediction_loss_only: True
per_device_train_batch_size: 4
per_device_eval_batch_size: 4
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 3
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.5488	500	0.6079
1.0977	1000	0.68
1.6465	1500	0.4051
2.1954	2000	0.2539
2.7442	2500	0.08

Framework Versions

Python: 3.9.21
Sentence Transformers: 5.1.2
Transformers: 4.57.6
PyTorch: 2.8.0+cu128
Accelerate: 1.10.1
Datasets: 4.5.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}