ahmedHamdi/narrative-similarity-all-en-gemma-masked-NE

SentenceTransformer based on google/embeddinggemma-300m

This is a sentence-transformers model finetuned from google/embeddinggemma-300m. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: google/embeddinggemma-300m
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ahmedHamdi/narrative-similarity-all-en-gemma-masked-NE")
# Run inference
sentences = [
    "PERSON was traumatized after the Vietnam War. His disability ruined his professional and personal life. His friend PERSON witnesses a murder and believes he recognizes the killer. But the killer is a suspect. The two friends decide to investigate. During an American parade, PERSON recognizes the killer by the dark glasses and the strange hat. It's Mr. PERSON, who runs a very large oil company and is very powerful because he buys everyone off and is very well protected at the head of his empire. PERSON suggests that PERSON write him a letter directly, telling him that he recognized him and offering to forget about it if he receives money from him. He will then report him to the police, returning the money obtained from PERSON through blackmail. PERSON doesn't reply, but he wants revenge and to kill PERSON, the crippled veteran, and PERSON, his friend who identified him at night on the road as the perpetrator of the brutal and sexual murder of a teenage girl. Determined to kill PERSON, PERSON and PERSON break into his house and eventually catch up with him during a private party. PERSON is killed, and PERSON ultimately kills PERSON in his office.",
    "One rainy night in GPE, GPE, PERSON car breaks down on a side road. He sees a large car draw up a little way behind him. A man throws something into a garbage can. At first, Bone thinks nothing of it and proceeds to meet his friend, GPE veteran PERSON. The following day, a young girl who has been brutally murdered is found in the garbage can, and Bone becomes a suspect. When Bone sees who he thinks is the same man in the Santa Barbara Founder's Day Paradelocal tycoon PERSON begins to take an interest in the case. His interest soon becomes a conspiracy theory that develops into an investigation with his skeptical friend and the dead girl's sister, along for the ride. After Cutter attempts to blackmail PERSON as a way of making PERSON incriminate himself, ORG's house mysteriously burns down with his wife, PERSON, inside. Convinced that PERSON had been trying to silence Bone, Cutter begins researching PERSON. He steals an invitation to a party at ORG house and gets Bone to drive him. When Cutter tells Bone he plans to kill PERSON, Bone attempts to leave the party but is blocked by other parked cars and instead goes after Cutter to convince him not to kill PERSON. After being chased by security, Bone winds up in PERSON's office. After a brief conversation in which PERSON assumes that ORG war experience has made him paranoid, Cutter suddenly crashes through the window after stealing one of PERSON's horses. As Cutter is dying from injuries from the broken glass, Bone asserts that PERSON killed the girl. PERSON states What if it were? Bone steadies ORG's gun in ORG hand and fires the pistol as the film cuts to black.",
    "PERSON is a struggling medical student who moonlights as a stand-up comedian. It quickly becomes evident that he is lousy at the former and excels at the latter. And yet, when he is given a chance at the big time, he cracks under the pressure. ORG is a dedicated housewife who yearns to be a comic. She has the raw talent but does not have the command of craft that PERSON possesses. At first, he doesn't give ORG the time of day. PERSON is derailed by the unexpected appearance of his father and brother, both medical professionals. ORG's unfailing support wins PERSON's affections and he teaches her the fundamentals of stand-up comedy. ORG has spent her cookie jar money to buy jokes. PERSON advises her to connect with the audience to unveil the honest humor in her life as a wife and mother. ORG discovers her natural gift of making people laugh. An uneasy friendship develops between the two as they share the personal conflicts they must resolve: PERSON's desire to make it big vs. his inability to do so and ORG's love of comedy vs. her love for her family. PERSON, beginning to appear emotionally unstable, develops a romantic attraction to ORG–to her dismay. ORG struggles to remain loyal to her family and her friend, while maintaining her conviction and love of comedy. Steven mimes a painful rendition of PERSON famous dance routine from PERSON' in the Rain. The film culminates in a competition at the Gas Station comedy club where ORG, PERSON and other aspiring comedians have been performing. A judges panel of television executives promise the winner a prime time opportunity and possible stardom. As they compete on stage, the characters also grapple with conflicts among their desires for success on stage, their loyalties to one another, and the expectations of their families. Pending the judges' final tally, with a note of support from her husband in her hand, and hearing PERSON has only two of the five judges' votes, ORG withdraws in case the winner is me and persists in leaving when the club owner reveals she was in fact the winner. She leaves with her husband who, after watching his wife do stand-up for the first time, is won over and begins suggesting ideas for her next set. The pair walk away arm in arm reminiscing about the funny and endearing sayings of their children. Inside, PERSON is declared the winner of the show, which reflects ORG's judgment and that of their competing fellow comics.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.6310, -0.0426],
#         [ 0.6310,  1.0000, -0.0676],
#         [-0.0426, -0.0676,  1.0000]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 28,380 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 13 tokens mean: 124.89 tokens max: 256 tokens	min: 3 tokens mean: 228.69 tokens max: 256 tokens

• Samples:

  sentence_0	sentence_1
  PERSON comes to GPE to avenge his daughter, PERSON, whose family was murdered by hitmen hired by a local triad boss. Lost in an unfamiliar city, he encounters three hitmen returning to his hotel to carry out their contract by assassinating PERSON's unfaithful lover. Costello, witnessing the scene, keeps their identities secret in exchange for their commitment to find the man who hired him and his daughter's killers. He gives them everything he owns as payment. The three men (PERSON, ORG, and PERSON) agree and set out to find the killers. They manage to identify them and easily locate them in GPE. Costello, whose passport lists him as a restaurateur, reveals himself at this point to be a former gangster, retired from the business but still very much in the know. He leaves with his three men for GPE to execute the assassins. Following an initial pitched battle between PERSON's group and the killers of his daughter's family, the truth comes to light: the assassins were hired by PERSON him...	In GPE, three men break into a house, shoot PERSON (Sylvie PERSON) and kill her husband and two children. PERSON's father, PERSON (PERSON), arrives to visit his daughter who is now suffering from serious injuries. She is nevertheless able to tell her father that there were three shooters and that she shot one of the killers in the ear. At a hotel, Costello meets PERSON (PERSON), PERSON (PERSON) and PERSON (PERSON), a trio of hitmen who are hired to murder the unfaithful wife of ORG crime boss PERSON (PERSON). After overhearing the murder in a hotel room, Costello reaches an unspoken agreement to walk away. Costello later tracks down PERSON's syndicate, hands them a stack of euros and his watch, and asks them to help him avenge his daughter's family's deaths. Before doing so though, he takes a ORG picture of each of the hitmen, and writes their names on the pictures so that he will not forget who they are. The four go to the apartment where the shooting occurred and work out what happe...
  PERSON and PERSON both aspire to be writers. But while PERSON's manuscript is rejected, PERSON's is published, and the young man becomes an overnight sensation and a prominent figure in Norwegian literature. Six months later, PERSON and his friends pick up PERSON from the psychiatric hospital. PERSON wants nothing more to do with literature, but PERSON hasn't given up and tries to convince him to start writing again.	Best friends PERSON and PERSON, both 23, have written books and fantasize that their novels will become cult hits. However when they both submit manuscripts PERSON's is rejected. PERSON's, on the other hand, is immediately accepted and a year later he becomes a star of the Norwegian literary scene. Six months later, PERSON and his friends pick up PERSON at a psychiatric hospital to bring him home after treatment following a suicide attempt. It is revealed that PERSON is suffering from psychosis which doctors believe was triggered by his whirlwind romance with PERSON, a girl he met and fell in love with at an underground punk show. Still unpublished, PERSON hasn't given up his dream, and continues to submit while PERSON shies away from all mention of writing. Instead PERSON tries to reunite with PERSON, who on the advice of his psychiatrists hasn't seen PERSON in seven months. PERSON's revised novel finally is accepted by a publisher. He struggles to assert himself during the editing pr...
  At an elite arts academy, a shy music student begins to overshadow her more talented twin when she discovers a notebook belonging to a deceased classmate.	A girl is playing the violin, until a grandfather clock chimes, and she jumps off the balcony. PERSON and PERSON are twin sisters attending their last year at ORG, a prestigious boarding school for the performing arts. Both sisters are classical pianists; however, ORG, a prodigy, has already been accepted to ORG while PERSON, overshadowed by her sister, has decided to take a gap year. Another student, GPE, dies by suicide, as seen earlier. The staff announce that GPE will be replaced at the senior school showcase and auditions will be held to see who will play in her place. PERSON discovers PERSON's theory notebook, having fallen off a shelf. Both sisters decide to audition, but at the last minute, PERSON decides to audition using the same piece as ORG, Piano Concerto No. 2. After finishing practicing, PERSON hears violin music from GPE's room that stops as soon as she enters, and finds a sun symbol etched into the wall behind a curtain, that matches the one on the notebook. She decide...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 4
• per_device_eval_batch_size: 4
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 4
• per_device_eval_batch_size: 4
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.0705	500	0.1075
0.1409	1000	0.1394
0.2114	1500	0.1439
0.2819	2000	0.1868
0.3524	2500	0.2972
0.4228	3000	0.3115
0.4933	3500	0.3378
0.5638	4000	0.2573
0.6342	4500	0.2849
0.7047	5000	0.3128
0.7752	5500	0.3144
0.8457	6000	0.314
0.9161	6500	0.3026
0.9866	7000	0.2774
1.0571	7500	0.2294
1.1276	8000	0.2293
1.1980	8500	0.2075
1.2685	9000	0.2702
1.3390	9500	0.2181
1.4094	10000	0.2351
1.4799	10500	0.2265
1.5504	11000	0.2209
1.6209	11500	0.1903
1.6913	12000	0.1809
1.7618	12500	0.2012
1.8323	13000	0.2169
1.9027	13500	0.1615
1.9732	14000	0.2119
2.0437	14500	0.1154
2.1142	15000	0.1261
2.1846	15500	0.129
2.2551	16000	0.1255
2.3256	16500	0.0806
2.3961	17000	0.1446
2.4665	17500	0.1237
2.5370	18000	0.0924
2.6075	18500	0.1076
2.6779	19000	0.113
2.7484	19500	0.0807
2.8189	20000	0.101
2.8894	20500	0.106
2.9598	21000	0.0987

Framework Versions

• Python: 3.9.21
• Sentence Transformers: 5.1.2
• Transformers: 4.57.6
• PyTorch: 2.8.0+cu128
• Accelerate: 1.10.1
• Datasets: 4.5.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}