ahmedHamdi/IR-all-en-gemma

SentenceTransformer based on google/embeddinggemma-300m

This is a sentence-transformers model finetuned from google/embeddinggemma-300m. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: google/embeddinggemma-300m
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ahmedHamdi/IR-all-en-gemma")
# Run inference
sentences = [
    "Johnny Kelly (Gig Young) is a Chicago police officer. Lacking passion for his job, he's simply followed in his father's footsteps, also a police officer, and working at the same precinct. He lives with a wife he no longer loves and dreams of leaving for California with his mistress, Sally Angel Face Connors (Mala Powers), a mediocre cabaret singer. She pressures him to leave the police force and his wife, or she'll resume her affair with Greg Warren (Wally Cassell), a former actor who now works as a mime outside the cabaret. One evening, Kelly announces his departure from the police. He knows this is his last night. To convince him to stay, his father assigns him an old friend from the precinct, Sergeant Joe (Chill Wills). They are called to the home of a crooked lawyer, Penrod Biddel (Edward Arnold). Biddel claims that a man working for him is going to rob him. This man is Hayes Stewart (William Talman), a former magician turned burglar. Kelly agrees, and a long wait begins.",
    "The story begins with a voice-over by the Voice of Chicago introducing the world and the main characters of the film. There is Sally Angel Face Connors (Mala Powers), an exotic dancer in a nightclub; Gregg Warren (Wally Cassell), a former actor working as a performance artist in the nightclub window, a Mechanical Man; Johnny Kelly (Gig Young) a cop having an affair with Angel Face and struggling with his conscience whether to leave his wife; Penrod Biddel  (Edward Arnold), a successful and smooth crooked attorney; Hayes Stewart (William Talman), a magician who has turned to making a career as a pickpocket and then as a thief. Officer Johnny Kelly is disillusioned with his job which he took to please his father, and writes a letter of resignation which he intends to hand in at the end of his shift. He calls Penrod Biddel to accept an employment offer that the lawyer had made, and agrees to meet him later that evening. Johnny's wife Kathy Kelly (Paula Raymond) discovers Johnny's plan to quit his job and calls his father Sgt. John Kelly Sr. (Otto Hulett) to ask him to intervene; Kelly Sr. is concerned for his son's happiness and has a talk with Johnny. On this night, Johnny's regular partner calls in sick and his replacement is Sgt. Joe (Chill Wills) who doesn't give his surname and whose voice can be recognized as the introductory Voice of Chicago. As they begin the nightly shift, Sgt. Joe has plenty of homespun advice for Johnny, whose negative energy casts a pall on their working environment. As the night progresses, Johnny visits Angel Face to re-affirm their plans to go away. He also meets with the lawyer Penrod Biddel who asks Johnny to pick up Hayes Stewart and leave him across the state line for the Indiana Police to arrest and incarcerate. Johnny says 'no' to this but changes his mind when Biddel tells him that Johnny's brother 'Stubby' (Ron Hagerthy) is associating with the criminal and will surely get into trouble unless Johnny steps in and does what Biddel wants. The lawyer pays Johnny $5,000 for the job, but it goes wrong and Stewart gets away, knowing Biddel is out to get him. Johnny and Sgt. Joe answer a call for a woman having a baby and Johnny performs the delivery; they also answer a call for an illegal gambling game on the street, arresting the ringleader and giving the money back to the men who have been hoodwinked. After each call, Sgt. Joe lays out another bit of wisdom that seemingly works to influence Johnny to re-evaluate his life. Hayes Stewart has obtained incriminating evidence on Penrod Biddel and is having an affair with Biddel’s wife, Lydia (Marie Windsor). Stewart shoots the lawyer and he and Lydia escape, running to the nightclub where Angel Face dances. Having discovered Biddel's agreement with Johnny to take him out of the state, Stewart calls the police asking to meet with Officer Kelly. Johnny's father takes the call and, mistaking the father for the son, Stewart shoots Kelly Sr., who dies in his son's arms. Stewart takes Lydia and Stubby and escapes but can't get far because of the police closing in. He shoots Lydia dead in front of the nightclub window where the Mechanical Man is performing. He stays close by, unsure if the Mechanical Man is a real man or an automaton; if he is real he intends to kill him. Gregg Warren, the Mechanical Man, is in love with Angel Face, and thinks he can bait the killer for Johnny. This sense of honor, and his father's murder, makes clear to Johnny what he really holds valuable in his life. As Gregg Warren performs in the window, Angel Face falsely declares her love for him, causing the Mechanical Man to shed a tear. Hayes Stewart sees that he is indeed a real man and shoots, revealing his hideout. Stewart pistol-whips Stubby. Johnny Kelly goes after Stewart and, after a chase along an elevated railroad track and a fight, Stewart is electrocuted on the live rail and falls to his death. Johnny considers what has happened this eventful night, re-evaluates his priorities in life, and is reconciled with his wife. The mysterious Sgt. Joe has disappeared.",
    ' This is set in 1958 during an open referendum for the adoption of the Constitution of the Fifth French Republic. It is about the daily life of Irene, a communist activist who was rescued from concentration camps by the Red Army when she was young, and who is married to Bernard, a small Gaullist shopkeeper. Their political differences undermine the couple. In a historical moment, the Alexandrov Ensemble come to give a performance in France, allowing Irene to meet veterans of the Battle of Stalingrad in the sight of her jealous husband. She will do anything to convince him to vote no in the referendum.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.8008, -0.1421],
#         [ 0.8008,  1.0000, -0.1480],
#         [-0.1421, -0.1480,  1.0000]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 24,420 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 10 tokens mean: 123.24 tokens max: 256 tokens	min: 11 tokens mean: 230.24 tokens max: 256 tokens

• Samples:

  sentence_0	sentence_1
  After learning that her astrophysicist father is being held captive on a distant planet amidst a universe-spanning evil, Meg Murry works with her highly intelligent younger brother, her classmate, and three astral travelers to save him.	In the late 1880s, a British couple and their infant son are shipwrecked in Equatorial Africa. The adults build a treehouse but are killed by Sabor, a villainous leopard. Kala and Kerchak, the leaders of a troop of gorillas, lose their infant son to Sabor as well. Kala finds the human infant and after saving him from Sabor, she adopts the human infant, despite Kerchak's objections. Years later, the adopted human, named Tarzan, begins to befriend other animals, including Kala's niece Terk and the paranoid elephant Tantor. Tarzan finds himself treated differently because of his different physique, so he makes valiant efforts to improve himself. As an adult, Tarzan manages to kill Sabor with a spear, gaining Kerchak's reluctant approval. Meanwhile, a team of human explorers from England, consisting of Professor Archimedes Q. Porter, his daughter Jane, and their hunter escort Clayton, are looking to study the gorillas. Jane accidentally becomes separated from the group and is chased by a ...
  China, Tang Dynasty. Returning to the Forbidden City after a long absence, the emperor discovers a plot brewing within the very heart of his palace. The conspirators' dangerous alliances and manipulations have only one goal: to seize power in the world's greatest kingdom. The betrayal will come from within: a rebellion led by the empress herself. Plot Details: The plot is based on Cao Yu's 1934 drama, The Storm (雷雨, Hanyu pinyin: Lei Yu), but is transposed to the imperial court of the post-Tang Dynasty in 928, during the Five Dynasties and Ten Kingdoms period. Emperor Ping is a man of humble origins and great ambition, who ascended the throne from the modest rank of captain. He abandoned his first wife to marry the Princess of Liang, now his empress, Phoenix (Fenghuang in Chinese). The emperor dislikes his second wife, attributing her sour mood to illness and forcing her to take a remedy he concocted himself every two hours for the past ten years. She and Crown Prince Wan, the empe...	On the eve of the Double Ninth Festival, the Emperor (Chow Yun-fat) and the second of his three sons, Prince Jai (Jay Chou), return from their military campaign to Nanjing so they can celebrate the holiday with their family. However, the Empress has been in an affair with the first son, Crown Prince Wan (Liu Ye), who was born of the Emperor's previous wife. At the same time, Crown Prince Wan has been in an affair with Jiang Chan (Li Man), daughter of the Imperial Doctor, and is keen on rejecting the throne so that he may run away with her. Prince Jai notices the declining health of his mother, and is confused by her sudden interest in chrysanthemums, the golden flowers. The Empress explains that the tea she drinks has been poisoned for some time by the Emperor, but that she is planning a rebellion to overthrow him. After some initial hesitation, Prince Jai agrees to be the leader of the rebellion. The Empress hires a mysterious woman to discover the type of poison which she is sufferi...
  Long unemployed, Adam White, a skilled reporter, accepts a job as the advice columnist for a second-rate newspaper. An honest but anxious man, he forces himself to do a good job he dislikes. Too good, in fact. He makes the mistake of becoming personally involved in the private life of a female reader and clashes repeatedly with his cynical editor-in-chief, who constantly seeks to humiliate him.	The story opens on a small-town street. A man throws a bundle of papers onto the sidewalk from the back of a truck labeled Chronicle. Adam White is sitting in a bar when a woman offers him a drink. He refuses, explaining that alcohol seems to be poisonous to him. After talking with her for a while, he learns she is married to William Shrike, Editor-in-Chief of the Chronicle, where Adam is hoping to work. The editor shows up to meet his wife only to find her talking with Adam. When Shrike asks how Adam found him, Adam explains: I heard there was a bar where newspaper people hang out. I came here since it is the closest to the Chronicle, the only paper in town. Florence Shrike says Adam can write, and he deserves the chance to prove it. Shrike retorts: OK, so write! Adam hems and haws momentarily, but then delivers the following story: The Chronicle is pleased to announce the addition of a new member to our staff. He met the Editor in Chief, who went so far as to insult his own wife in a...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 4
• per_device_eval_batch_size: 4
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 4
• per_device_eval_batch_size: 4
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.0819	500	0.0576
0.1638	1000	0.0734
0.2457	1500	0.0777
0.3276	2000	0.1451
0.4095	2500	0.1628
0.4914	3000	0.1406
0.5733	3500	0.1331
0.6552	4000	0.2098
0.7371	4500	0.1578
0.8190	5000	0.1475
0.9009	5500	0.1605
0.9828	6000	0.1393
1.0647	6500	0.1389
1.1466	7000	0.1146
1.2285	7500	0.091
1.3104	8000	0.1137
1.3923	8500	0.1032
1.4742	9000	0.1012
1.5561	9500	0.0994
1.6380	10000	0.096
1.7199	10500	0.0748
1.8018	11000	0.099
1.8837	11500	0.0794
1.9656	12000	0.1126
2.0475	12500	0.0698
2.1294	13000	0.0354
2.2113	13500	0.0489
2.2932	14000	0.0553
2.3751	14500	0.0344
2.4570	15000	0.039
2.5389	15500	0.0369
2.6208	16000	0.0369
2.7027	16500	0.0346
2.7846	17000	0.0256
2.8665	17500	0.0234
2.9484	18000	0.0258

Framework Versions

• Python: 3.9.18
• Sentence Transformers: 5.1.2
• Transformers: 4.57.6
• PyTorch: 2.8.0+cu128
• Accelerate: 1.10.1
• Datasets: 4.5.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}