sepidnes/obesity-blog-ft-16c9c0d6-896a-4bae-84fa-a1aae1e1bb35

SentenceTransformer based on Snowflake/snowflake-arctic-embed-m

This is a sentence-transformers model finetuned from Snowflake/snowflake-arctic-embed-m. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Snowflake/snowflake-arctic-embed-m
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sepidnes/obesity-blog-ft-16c9c0d6-896a-4bae-84fa-a1aae1e1bb35")
# Run inference
sentences = [
    'What is the medical definition of obesity as described in the context?',
    'Definition of ObesityUnderstanding What It Truly Means To Be Obese\nThe best definition of obesity used by\ndoctors all over the world is "the accumulation of excessive body fat to\nan extent that it adversely affects the health and well being of the\nindividual". The presence of excessive body fat or being overweight or\nobese is usually determined by measuring the amount of weight you carry for\nyour height. This is called a measure of your BMI or body mass index. But what\nexactly does this mean to you as an individual? What are the grades or degrees\nof obesity and what treatment options exist for each grades or stages of\nobesity? Let’s find out here.',
    'Ugrankar saw that during times of plenty, a protein machine could fill these small membrane-wrapped fat droplets with lots of triacylglycerol, a high-energy, durable form of fat storage. Their ready access at the surface of the adipocyte allows the fly to balance lipid storage locally with energy release into its blood in times of famine.\nUgrankar’s adeptness at the microscope resulted in this beautiful photo, which was earlier featured in the American Society for Cell Biology’s Green Fluorescent Protein Image and Video Contest. But her work and that of many others help to open a vital window into nutrition science and many critical mechanistic questions about the causes of obesity, insulin resistance, hyperglycemia, and even reduced lifespan.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.8214
cosine_accuracy@3	0.9643
cosine_accuracy@5	0.9643
cosine_accuracy@10	1.0
cosine_precision@1	0.8214
cosine_precision@3	0.3214
cosine_precision@5	0.1929
cosine_precision@10	0.1
cosine_recall@1	0.8214
cosine_recall@3	0.9643
cosine_recall@5	0.9643
cosine_recall@10	1.0
cosine_ndcg@10	0.9196
cosine_mrr@10	0.8929
cosine_map@100	0.8929

Training Details

Training Dataset

Unnamed Dataset

• Size: 122 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 122 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 7 tokens mean: 21.95 tokens max: 32 tokens	min: 25 tokens mean: 166.49 tokens max: 277 tokens

• Samples:

  sentence_0	sentence_1
  What role does the enzyme DNA-PK play in weight gain during middle age according to Jay Chung's research?	2017: Muscle Enzyme Explains Weight Gain in Middle Age. The struggle to maintain a healthy weight is a lifelong challenge for many of us. While several risk factors for weight gain, such as counting calories, are within our control, there’s a major one that isn’t: age. Jay Chung, a researcher with NIH’s National Heart, Lung, and Blood Institute, and his team discovered that the normal aging process causes levels of an enzyme called DNA-PK to rise in animals as they approach middle age. While the enzyme is known for its role in DNA repair, their studies showed it also slows down metabolism, making it more difficult to burn fat.
  How does the normal aging process affect metabolism and fat burning as described in the 2017 study?	2017: Muscle Enzyme Explains Weight Gain in Middle Age. The struggle to maintain a healthy weight is a lifelong challenge for many of us. While several risk factors for weight gain, such as counting calories, are within our control, there’s a major one that isn’t: age. Jay Chung, a researcher with NIH’s National Heart, Lung, and Blood Institute, and his team discovered that the normal aging process causes levels of an enzyme called DNA-PK to rise in animals as they approach middle age. While the enzyme is known for its role in DNA repair, their studies showed it also slows down metabolism, making it more difficult to burn fat.
  What protein did researchers led by Hyo Youl Moon and Henriette van Praag identify as being secreted by skeletal muscle cells during exercise?	2016: Exercise Releases Brain-Healthy Protein. The research literature is pretty clear: exercise is good for the brain. In this very popular post, researchers led by Hyo Youl Moon and Henriette van Praag of NIH’s National Institute on Aging identified a protein secreted by skeletal muscle cells to help explore the muscle-brain connection. In a study in Cell Metabolism, Moon and his team showed that this protein called cathepsin B makes its way into the brain and after a good workout influences the development of new neural connections. This post is also memorable to me for the photo collage that accompanied the original post. Why? If you look closely at the bottom right, you’ll see me exercising—part of my regular morning routine!

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 10
• per_device_eval_batch_size: 10
• num_train_epochs: 10
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 10
• per_device_eval_batch_size: 10
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 10
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• tp_size: 0
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	cosine_ndcg@10
1.0	13	0.8533
2.0	26	0.8682
3.0	39	0.8780
3.8462	50	0.8959
4.0	52	0.8959
5.0	65	0.8932
6.0	78	0.9196
7.0	91	0.9196
7.6923	100	0.9196
8.0	104	0.9196
9.0	117	0.9196
10.0	130	0.9196

Framework Versions

• Python: 3.11.12
• Sentence Transformers: 4.1.0
• Transformers: 4.51.3
• PyTorch: 2.6.0+cu124
• Accelerate: 1.6.0
• Datasets: 2.14.4
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}