OloriBern/musique-bge-m3-50-hybrid

CrossEncoder

This is a Cross Encoder model trained using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.

Model Details

Model Description

• Model Type: Cross Encoder
• Maximum Sequence Length: 1024 tokens
• Number of Output Labels: 1 label

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("cross_encoder_model_id")
# Get scores for pairs of texts
pairs = [
    ["What is the language for the person who translated Plato's work to Latin?", 'Southern Europe. The most widely spoken family of languages in southern Europe are the Romance languages, the heirs of Latin, which have spread from the Italian peninsula, and are emblematic of Southwestern Europe. (See the Latin Arch.) By far the most common romance languages in Southern Europe are: Italian, which is spoken by over 50 million people in Italy, San Marino, and the Vatican; and Spanish, which is spoken by over 40 million people in Spain and Gibraltar. Other common romance languages include: Romanian, which is spoken in Romania and Moldova; Portuguese, which is spoken in Portugal; Catalan, which is spoken in eastern Spain; and Galician, which is spoken in northwestern Spain.'],
    ['Which major Russian city borders the body of water in which Saaremaa is located?', "Biysky District. Biysky District () is an administrative and municipal district (raion), one of the fifty-nine in Altai Krai, Russia. It is located in the east of the krai and borders with Zonalny, Tselinny, Soltonsky, Krasnogorsky, Sovetsky, and Smolensky Districts, as well as with the territory of the City of Biysk. The area of the district is . Its administrative center is the city of Biysk (which is not administratively a part of the district). District's population:"],
    ['When did the manufacturer of a pedometer accessory for the iPod become a publicly traded company?', "Nike, Inc.. In 1976, the company hired John Brown and Partners, based in Seattle, as its first advertising agency. The following year, the agency created the first ``brand ad ''for Nike, called`` There is no finish line'', in which no Nike product was shown. By 1980, Nike had attained a 50% market share in the U.S. athletic shoe market, and the company went public in December of that year."],
    ["Where did Peter and Paul Fortress' designer die?", 'Andrey Dostoevsky. In late 1841 Andrey Dostoyevsky moved to Saint Petersburg. The following year he entered the Saint-Petersburg State University of Architecture and Civil Engineering, graduating in June 1848. Subsequently, he worked as an engineer in Saint Petersburg. He had none of the literary talent of his brothers Fyodor and Mikhail. In 1849 Andrey was arrested as a member of Petrashevsky Circle and placed in Peter and Paul Fortress, because he was mistaken for Mikhail. 13 days later Andrey was released, but this incident ruined his career. Because of the relations to Dostoyevsky family, he was sent out of Saint Petersburg and appointed as head architect in Elisavetgrad. In July 1850 Andrey Dostoyevsky married Domnika Fedorchenko. They had 2 sons and 2 daughters.'],
    ['When was the institute that owned The Collegian founded?', 'The Collegian (Houston Baptist University). The Collegian is the bi-weekly official student publication of Houston Baptist University in Houston, Texas. It was founded in 1963 as a newsletter, and adopted the newspaper format in 1990.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    "What is the language for the person who translated Plato's work to Latin?",
    [
        'Southern Europe. The most widely spoken family of languages in southern Europe are the Romance languages, the heirs of Latin, which have spread from the Italian peninsula, and are emblematic of Southwestern Europe. (See the Latin Arch.) By far the most common romance languages in Southern Europe are: Italian, which is spoken by over 50 million people in Italy, San Marino, and the Vatican; and Spanish, which is spoken by over 40 million people in Spain and Gibraltar. Other common romance languages include: Romanian, which is spoken in Romania and Moldova; Portuguese, which is spoken in Portugal; Catalan, which is spoken in eastern Spain; and Galician, which is spoken in northwestern Spain.',
        "Biysky District. Biysky District () is an administrative and municipal district (raion), one of the fifty-nine in Altai Krai, Russia. It is located in the east of the krai and borders with Zonalny, Tselinny, Soltonsky, Krasnogorsky, Sovetsky, and Smolensky Districts, as well as with the territory of the City of Biysk. The area of the district is . Its administrative center is the city of Biysk (which is not administratively a part of the district). District's population:",
        "Nike, Inc.. In 1976, the company hired John Brown and Partners, based in Seattle, as its first advertising agency. The following year, the agency created the first ``brand ad ''for Nike, called`` There is no finish line'', in which no Nike product was shown. By 1980, Nike had attained a 50% market share in the U.S. athletic shoe market, and the company went public in December of that year.",
        'Andrey Dostoevsky. In late 1841 Andrey Dostoyevsky moved to Saint Petersburg. The following year he entered the Saint-Petersburg State University of Architecture and Civil Engineering, graduating in June 1848. Subsequently, he worked as an engineer in Saint Petersburg. He had none of the literary talent of his brothers Fyodor and Mikhail. In 1849 Andrey was arrested as a member of Petrashevsky Circle and placed in Peter and Paul Fortress, because he was mistaken for Mikhail. 13 days later Andrey was released, but this incident ruined his career. Because of the relations to Dostoyevsky family, he was sent out of Saint Petersburg and appointed as head architect in Elisavetgrad. In July 1850 Andrey Dostoyevsky married Domnika Fedorchenko. They had 2 sons and 2 daughters.',
        'The Collegian (Houston Baptist University). The Collegian is the bi-weekly official student publication of Houston Baptist University in Houston, Texas. It was founded in 1963 as a newsletter, and adopted the newspaper format in 1990.',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

Evaluation

Metrics

Cross Encoder Binary Classification

• Dataset: validation
• Evaluated with CEBinaryClassificationEvaluator

Metric	Value
accuracy	0.9678
accuracy_threshold	1.0
f1	0.9677
f1_threshold	1.0
precision	0.9708
recall	0.9646
average_precision	0.9758

Training Details

Training Dataset

Unnamed Dataset

• Size: 234 training samples
• Columns: sentence_0, sentence_1, and label
• Approximate statistics based on the first 234 samples:

  	sentence_0	sentence_1	label
  type	string	string	float
  details	min: 38 characters mean: 76.04 characters max: 156 characters	min: 139 characters mean: 560.66 characters max: 2023 characters	min: 0.0 mean: 0.5 max: 1.0

• Samples:

  sentence_0	sentence_1	label
  What is the language for the person who translated Plato's work to Latin?	Southern Europe. The most widely spoken family of languages in southern Europe are the Romance languages, the heirs of Latin, which have spread from the Italian peninsula, and are emblematic of Southwestern Europe. (See the Latin Arch.) By far the most common romance languages in Southern Europe are: Italian, which is spoken by over 50 million people in Italy, San Marino, and the Vatican; and Spanish, which is spoken by over 40 million people in Spain and Gibraltar. Other common romance languages include: Romanian, which is spoken in Romania and Moldova; Portuguese, which is spoken in Portugal; Catalan, which is spoken in eastern Spain; and Galician, which is spoken in northwestern Spain.	1.0
  Which major Russian city borders the body of water in which Saaremaa is located?	Biysky District. Biysky District () is an administrative and municipal district (raion), one of the fifty-nine in Altai Krai, Russia. It is located in the east of the krai and borders with Zonalny, Tselinny, Soltonsky, Krasnogorsky, Sovetsky, and Smolensky Districts, as well as with the territory of the City of Biysk. The area of the district is . Its administrative center is the city of Biysk (which is not administratively a part of the district). District's population:	0.0
  When did the manufacturer of a pedometer accessory for the iPod become a publicly traded company?	Nike, Inc.. In 1976, the company hired John Brown and Partners, based in Seattle, as its first advertising agency. The following year, the agency created the first brand ad ''for Nike, called There is no finish line'', in which no Nike product was shown. By 1980, Nike had attained a 50% market share in the U.S. athletic shoe market, and the company went public in December of that year.	1.0

• Loss: BinaryCrossEntropyLoss with these parameters:

  {
      "activation_fn": "torch.nn.modules.linear.Identity",
      "pos_weight": null
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 4
• per_device_eval_batch_size: 4

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 4
• per_device_eval_batch_size: 4
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• eval_use_gather_object: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	validation_average_precision
1.0	59	0.9755
2.0	118	0.9758
3.0	177	0.9758

Framework Versions

• Python: 3.11.6
• Sentence Transformers: 5.2.0
• Transformers: 4.44.2
• PyTorch: 2.9.1+cu128
• Accelerate: 1.12.0
• Datasets: 4.0.0
• Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}