ahmedHamdi/narrative-similarity-pt-en-gemma-masked-NE

SentenceTransformer based on google/embeddinggemma-300m

This is a sentence-transformers model finetuned from google/embeddinggemma-300m. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: google/embeddinggemma-300m
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ahmedHamdi/narrative-similarity-pt-en-gemma-masked-NE")
# Run inference
sentences = [
    'An idealistic reporter, PERSON, and his younger brother, PERSON, investigate the events surrounding a murder in order to exonerate a man on death row, PERSON.',
    "PERSON, the former maid of the PERSON family, recounts the events of the summer of 1969 when PERSON, an idealistic reporter, returned to his hometown of GPE, GPE, to investigate the events surrounding the 1965 murder of a violent local sheriff. PERSON, a swamp-dwelling alligator hunter and small-time criminal, is on death row for the murder. Ward and his colleague, Englishman PERSON, investigative reporters with ORG, plan to help exonerate him. Some evidence against PERSON was lost, which ORG and PERSON plan to expose as redneck injustice. PERSON had fallen in love with PERSON, though they have not met, only exchanging correspondence. She arrives in GPE, determined to prove his innocence so they could marry. GPE requested the help of ORG and PERSON who then hired PERSON, ORG's younger brother, as their driver. Ward has mixed feelings about returning home to his estranged father, ORG, who runs PERSON's local newspaper. The brothers dislike their divorced father's latest girlfriend, PERSON. PERSON works as a paperboy for his father's business after having been expelled from college, ending his collegiate swimming career. His only real friend is PERSON, who helped ORG raise him after their mother left.PERSON proclaims his love to GPE, giving her his mother’s  ring, but she repeats that she is devoted to PERSON. PERSON is initially hostile to the reporters. Contrary to the romantic portrayal he had painted of himself in his letters to GPE, he reveals himself to be racist, sexist, and crude. After PERSON tells them his alibi, the PERSON travel to meet PERSON Uncle Tyree. Tyree, who lives in pitiful conditions in the middle of the swamp, is initially reluctant to admit his own crime to save his nephew's life, but finally admits that they were away the night of the murder, stealing sod from a golf course in GPE. Yardley and GPE visit the golf course to verify the story. Yardley claims to have found the developer who bought the stolen sod, saying that the man requested anonymity, so refuses to disclose his name even to ORG. Yardley goes back to GPE to start writing the article. Suspicious of PERSON's motives, ORG decides to check GPE himself, with PERSON and GPE in tow. On the way Ward gets drunk at a bar and approaches a black man. During the night, GPE and PERSON hear alarming sounds and run from their rooms. They find the black man, with several others, viciously beating ORG. Ward is admitted to hospital. PERSON goes to GPE, hoping to convince PERSON not to publish the unproven facts with ORG’s name as co-reporter. During the confrontation, PERSON reveals he's actually an American pretending to be English to escape discrimination. He also reveals he had given Ward sexual favors in the past, which was the beginning of ORG's self-loathing infatuation with black men. PERSON does not resent Ward for being a homosexual, but for keeping it a secret from him. The article is published and PERSON leaves for GPE with a book deal. PERSON obtains a pardon, and GPE goes to live with him in the swamp. Months later she is unhappy with his abusive behavior and sends a letter to PERSON telling him she made a mistake and plans to reunite with him at his father's wedding. PERSON does not receive the letter until the wedding reception, smuggled to him by PERSON, who was fired from the PERSON household and knew that PERSON did not intend to pass the letter to PERSON. GPE is not at the wedding. PERSON leaves to rescue her, joined by ORG, who has revealed that the anonymous developer does not exist, undercutting PERSON alibi. PERSON and Ward find that PERSON has killed GPE rather than let her attend the wedding. A fight ensues and PERSON kills Ward, while PERSON dives into the swamp and evades him all night. The next morning he retrieves his loved ones’ bodies and boats away. PERSON concludes by recounting that PERSON was convicted for the murders of ORG and GPE and sent to the electric chair. PERSON later saw his mother at ORG's funeral. He would never get over GPE.",
    " Young mother PERSON puts her daughter to bed while her husband PERSON watches golf on television. After the child is asleep, she goes downstairs and announces she's leaving him. He runs upstairs to the bedroom and holds their daughter out the window, threatening to drop her if PERSON leaves. Years later, PERSON and PERSON return home from church for a quiet afternoon. He watches TV and drinks in an identical pose. Late that night a call wakes them. PERSON picks up the phone and hears a woman's voice. PERSON reports to work at a prison, where he is a parole officer. He is called into the warden's office. His upcoming retirement is brought up. PERSON requests that he keep all of his inmates until he leaves, in order to see them through until review. PERSON has a new case in his office, named PERSON. The inmate insists that he likes to be called ORG. ORG asks PERSON if he can help him get out early. They attempt to talk about each other's wives. PERSON explains that he does not want to discuss his wife, and that they are there to talk about his case. Stone later phones his wife, ORG, from prison. Stone and PERSON hold several more meetings. Stone tells him he deserves to be free. That night ORG leaves a message on PERSON and PERSON's answering machine. She shows up at the prison the next day to meet PERSON. Lucetta phones PERSON again and they meet for lunch. They end up at ORG's home. After a few drinks, he sleeps with ORG. At prison, two guards arrive to escort ORG to the infirmary. While waiting for someone to see him, he witnesses another inmate being brutally murdered. PERSON soon goes to see ORG several more times for sex. PERSON tells her no-one can know about their relationship. One day, he tells ORG that he sent the report recommending early release. The next morning, PERSON asks the warden for ORG's report. The warden informs him ORG's parole hearing is in an hour and no changes can be made. PERSON does not stay for the hearing. ORG is informed that he will be released. Stone tells PERSON that he knows about the relationship between him and ORG. That night, PERSON goes home with paranoia and is awoken by a fire in his home. Later, PERSON, her daughter and her granddaughter look through photo albums. PERSON is now retired trying to determine his future.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.7857, 0.0596],
#         [0.7857, 1.0000, 0.0917],
#         [0.0596, 0.0917, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 7,104 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 12 tokens mean: 119.86 tokens max: 256 tokens	min: 3 tokens mean: 236.51 tokens max: 256 tokens

• Samples:

  sentence_0	sentence_1
  The story takes place on the planet PERSON, where two kings decide to unite their children, Colwyn and ORG, in marriage to strengthen the ties between their two nations and combat the Assassin invasion. The wedding ceremony is interrupted by the LOC, servants of the Beast, who kidnap Princess PERSON and take her to their master, who intends to marry her off, leaving Colwyn for dead. Colwyn receives help from Ynyr (ORG), a wise old man who guides him to ORG, the PERSON's lair. However, ORG teleports to a new location every morning, forcing the protagonist to follow it. Along the way, the hero encounters various characters such as: PERSON (PERSON), the cyclops; PERSON (PERSON), the mage; and a group of fugitive mercenaries led by PERSON (PERSON).	A narrator tells of a prophecy that a king and queen who will rule their world, and then their son will rule the galaxy. The planet PERSON is invaded by an entity known as the Beast and his army of ORG, who travel the galaxy in a mountain-like spaceship called the Black Fortress. Prince Colwyn and Princess PERSON plan to marry in the hope that their two kingdoms' combined forces can defeat the PERSON's army. However, the Slayers attack before the wedding is completed, devastating the native ORG armies, wounding Colwyn, and kidnapping ORG. Colwyn is nursed back to health by Ynyr, the Old One. Ynyr tells Colwyn that the Beast can be defeated with the ORG, an ancient, magical, five-pointed weapon resembling a large throwing star. Colwyn retrieves the Glaive from a mountain cave, and sets out to find the Black Fortress, which teleports to a new location every sunrise. As they travel, Colwyn and Ynyr are joined by the magician ORG and a band of nine thieves and fighters: PERSON, PERSON, ...
  PERSON (PERSON), a lieutenant in the military, along with his captain, PERSON (PERSON), goes to the Great Smoky Mountains to visit his distant maternal relatives and convince them to sell their land, which will become a missile site. During his visit, PERSON encounters his cousin, PERSON (PERSON), who is his blonde doppelganger. He also meets his two cousins, GPE (PERSON) and PERSON (PERSON), who both become interested in PERSON. In the end, PERSON ends up with GPE and introduces Selena to his best friend. Meanwhile, PERSON meets PERSON (PERSON), a beautiful but irritable ORG. PERSON tries to convince Pappy Tatum to sell one side of the mountain to the government, on the condition that they don't interfere with his business on the other side.	ORG has run into a dead end trying to negotiate the lease of mountaintop land owned by PERSON, in the Great Smoky Mountains of GPE, for use as an ICBM missile base. ORG General PERSON gives Captain PERSON seven days to secure the lease, or face permanent assignment to GPE. After a quick computer search of military records, ORG requests that ORG pilot Second Lt. PERSON, born elsewhere in the Great Smoky Mountains, be assigned as his number two. When they arrive in GPE with a small platoon, dark-haired PERSON is surprised to meet his look-alike third cousin PERSON, a blond hillbilly. PERSON also meets his two beautiful country cousins, GPE and GPE, who compete to win his affections. PERSON eventually chooses GPE and pairs off GPE with his friend, Master PERSON. PERSON. PERSON, on the other hand, falls for ORG, a beautiful but fiery soldier. There are also a group of 13 mountain maidens called the ORG who create havoc when they set their sights on the marriage-eligible soldiers. PERSON pe...
  A student at ORG is confronted with the supernatural powers of the headmistress.	PERSON, a difficult young girl, is sent to the mysterious ORG after her delinquent behavior becomes too much for her school to handle. When she arrives at ORG, PERSON meets eccentric headmistress Madame PERSON and the school's only other students, four teenage girls with similar behavioral problems (ORG, GPE, ORG, and GPE). Technology is rarely used, and the girls can only phone their families in the domineering presence of the headmistress. The girls attend a variety of creative and intellectual classes, which begins to draw out unknown talents in them. ORG is the first to show a troubling obsession with her work, losing sleep, refusing to eat, and going into odd trances while creating amazing works of art. PERSON writes beautiful poetry and stories that deeply disturb her. PERSON and GPE slowly begin to experience the same bad side effects, saying it was like someone else was using their bodies. Only the belligerent ORG shows no progress, much to Madame PERSON's annoyance. During art...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 4
• per_device_eval_batch_size: 4
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 4
• per_device_eval_batch_size: 4
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.2815	500	0.0559
0.5631	1000	0.1596
0.8446	1500	0.1619
1.1261	2000	0.1088
1.4077	2500	0.0872
1.6892	3000	0.0662
1.9707	3500	0.0583
2.2523	4000	0.0221
2.5338	4500	0.0279
2.8153	5000	0.0176

Framework Versions

• Python: 3.9.21
• Sentence Transformers: 5.1.2
• Transformers: 4.57.6
• PyTorch: 2.8.0+cu128
• Accelerate: 1.10.1
• Datasets: 4.5.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}