OloriBern/musique-bge-m3-2000-mixer-3ep

CrossEncoder

This is a Cross Encoder model trained using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.

Model Details

Model Description

• Model Type: Cross Encoder
• Maximum Sequence Length: 512 tokens
• Number of Output Labels: 1 label

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("cross_encoder_model_id")
# Get scores for pairs of texts
pairs = [
    ["In which year did the publisher of Roald Dahl's Guide to Railway Safety cease to exist?", "Glacier Park International Airport is in Flathead County, Montana, six miles northeast of Kalispell. The airport is owned and operated by the Flathead Municipal Airport Authority, a public agency created by the county in 1974.</s>Roald Dahl's Guide to Railway Safety was published in 1991 by the British Railways Board. The British Railways Board had asked Roald Dahl to write the text of the booklet, and Quentin Blake to illustrate it, to help young people enjoy using the railways safely.</s>The British Railways Board (BRB) was a nationalised industry in the United Kingdom that operated from 1963 to 2001. Until 1997 it was responsible for most railway services in Great Britain, trading under the brand name British Railways and, from 1965, British Rail. It did not operate railways in Northern Ireland, where railways were the responsibility of the Government of Northern Ireland."],
    ['Who is the brother of the developer of Buffy the Animated Series?', "Waylon Malloy Payne (born April 5, 1972) is an American country singer, songwriter, musician and actor. He is the son of the country singer Sammi Smith.</s>WVFM, known simply as FM 106.5 and formerly WQLR, is a Classic Hits-leaning Adult Contemporary outlet serving the Kalamazoo, Michigan radio market. The station's frequency is 106.5\xa0MHz on the FM dial with an ERP of 33\xa0kW. They are owned by Midwest Communications. WVFM 106.5 is located on a crowded Frequency across south-west lower Michigan. The station covers all of Kalamazoo/Battle Creek area, can be heard well in the Grand Rapids area, and reaches as far north and east as Lansing and Jackson. During summer months, the station can be received to Flint and Ann Arbor on occasion.</s>Fred Iltis (Brno, Czechoslovakia, April 20, 1923 – San Jose, California, December 11, 2008) was an American entomologist. His research focused on the biosystematics and life cycle of mosquitoes."],
    ['Who did the performer of So Under Pressure play in Home and Away?', 'The Acoma Massacre was fought in January 1599 between Spanish conquistadors and Acoma Native Americans in what is now New Mexico. After twelve soldiers were killed at Acoma Pueblo in 1598, the Spanish retaliated by launching a punitive expedition, which led to the deaths of around 800 men, women and children during a three - day battle. Several hundred survivors were also enslaved or otherwise severely punished.</s>The NBA Championship ring is an annual award given by the National Basketball Association to the team that wins the NBA Finals. Rings are presented to the team\'s players, coaches, and members of the executive front office. Red Auerbach has the most rings overall with 16. Phil Jackson has the most as coach and Bill Russell has the most as a player (11 each)</s>Tibetan sources say Deshin Shekpa also persuaded the Yongle Emperor not to impose his military might on Tibet as the Mongols had previously done. Thinley writes that before the Karmapa returned to Tibet, the Yongle Emperor began planning to send a military force into Tibet to forcibly give the Karmapa authority over all the Tibetan Buddhist schools but Deshin Shekpa dissuaded him. However, Hok-Lam Chan states that "there is little evidence that this was ever the emperor\'s intention" and that evidence indicates that Deshin Skekpa was invited strictly for religious purposes.'],
    ["Big Bill Morganfield's father is associated with which subgenre of the blues?", 'William "Big Bill" Morganfield (born June 19, 1956) is an American blues singer and guitarist, who is the son of Muddy Waters.</s>WNZK is a radio station in Dearborn Heights, Michigan, United States. It began broadcasting October 12, 1985. It broadcasts in the AM radio band at 690 kHz during the daytime and at 680\xa0kHz at night. This is to protect the nighttime pattern of Montreal, Quebec\'s CKGM, a clear-channel station on 690. WNZK is the only North American AM station to broadcast on two frequencies, at least according to the FCC online database.</s>McKinley Morganfield (April 4, 1913 -- April 30, 1983), known professionally as Muddy Waters, was an American blues musician who is often cited as the ``father of modern Chicago blues \'\'.'],
    ['Who was played by the Nothing That You Are performer in Princess Diaries?', 'The Madonna of Chancellor Rolin is an oil painting by the Early Netherlandish master Jan van Eyck, dating from around 1435. It is kept in the Musée du Louvre, Paris, and was commissioned by Nicolas Rolin, aged 60, chancellor of the Duchy of Burgundy, whose votive portrait takes up the left side of the picture, for his parish church, "Notre-Dame-du-Chastel" in Autun, where it remained until the church burnt down in 1793. After a period in Autun Cathedral, it was moved to the Louvre in 1805.</s>Jon Monday (born 1947 in San Jose, California) is an American producer and distributor of CDs and DVDs across an eclectic range of material such as Swami Prabhavananda, Aldous Huxley, Christopher Isherwood, Huston Smith, Chalmers Johnson, and Charles Bukowski. Monday directed and co-produced with Jennifer Douglas the feature-length documentary "Save KLSD: Media Consolidation and Local Radio". He is also President of Benchmark Recordings, which owns and distributes the early catalog of The Fabulous Thunderbirds CDs and a live recording of Mike Bloomfield.</s>Strictly Inc. is the self-titled project album, released by Genesis keyboardist Tony Banks, and Wang Chung lead vocalist Jack Hues, in 1995 on Virgin Records. Tony Banks wanted the album release—as the title suggested—with no reference to the band members; but the record company went against his wishes. This was Banks\' fifth studio album (his second issued under a band name and seventh album overall).'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    "In which year did the publisher of Roald Dahl's Guide to Railway Safety cease to exist?",
    [
        "Glacier Park International Airport is in Flathead County, Montana, six miles northeast of Kalispell. The airport is owned and operated by the Flathead Municipal Airport Authority, a public agency created by the county in 1974.</s>Roald Dahl's Guide to Railway Safety was published in 1991 by the British Railways Board. The British Railways Board had asked Roald Dahl to write the text of the booklet, and Quentin Blake to illustrate it, to help young people enjoy using the railways safely.</s>The British Railways Board (BRB) was a nationalised industry in the United Kingdom that operated from 1963 to 2001. Until 1997 it was responsible for most railway services in Great Britain, trading under the brand name British Railways and, from 1965, British Rail. It did not operate railways in Northern Ireland, where railways were the responsibility of the Government of Northern Ireland.",
        "Waylon Malloy Payne (born April 5, 1972) is an American country singer, songwriter, musician and actor. He is the son of the country singer Sammi Smith.</s>WVFM, known simply as FM 106.5 and formerly WQLR, is a Classic Hits-leaning Adult Contemporary outlet serving the Kalamazoo, Michigan radio market. The station's frequency is 106.5\xa0MHz on the FM dial with an ERP of 33\xa0kW. They are owned by Midwest Communications. WVFM 106.5 is located on a crowded Frequency across south-west lower Michigan. The station covers all of Kalamazoo/Battle Creek area, can be heard well in the Grand Rapids area, and reaches as far north and east as Lansing and Jackson. During summer months, the station can be received to Flint and Ann Arbor on occasion.</s>Fred Iltis (Brno, Czechoslovakia, April 20, 1923 – San Jose, California, December 11, 2008) was an American entomologist. His research focused on the biosystematics and life cycle of mosquitoes.",
        'The Acoma Massacre was fought in January 1599 between Spanish conquistadors and Acoma Native Americans in what is now New Mexico. After twelve soldiers were killed at Acoma Pueblo in 1598, the Spanish retaliated by launching a punitive expedition, which led to the deaths of around 800 men, women and children during a three - day battle. Several hundred survivors were also enslaved or otherwise severely punished.</s>The NBA Championship ring is an annual award given by the National Basketball Association to the team that wins the NBA Finals. Rings are presented to the team\'s players, coaches, and members of the executive front office. Red Auerbach has the most rings overall with 16. Phil Jackson has the most as coach and Bill Russell has the most as a player (11 each)</s>Tibetan sources say Deshin Shekpa also persuaded the Yongle Emperor not to impose his military might on Tibet as the Mongols had previously done. Thinley writes that before the Karmapa returned to Tibet, the Yongle Emperor began planning to send a military force into Tibet to forcibly give the Karmapa authority over all the Tibetan Buddhist schools but Deshin Shekpa dissuaded him. However, Hok-Lam Chan states that "there is little evidence that this was ever the emperor\'s intention" and that evidence indicates that Deshin Skekpa was invited strictly for religious purposes.',
        'William "Big Bill" Morganfield (born June 19, 1956) is an American blues singer and guitarist, who is the son of Muddy Waters.</s>WNZK is a radio station in Dearborn Heights, Michigan, United States. It began broadcasting October 12, 1985. It broadcasts in the AM radio band at 690 kHz during the daytime and at 680\xa0kHz at night. This is to protect the nighttime pattern of Montreal, Quebec\'s CKGM, a clear-channel station on 690. WNZK is the only North American AM station to broadcast on two frequencies, at least according to the FCC online database.</s>McKinley Morganfield (April 4, 1913 -- April 30, 1983), known professionally as Muddy Waters, was an American blues musician who is often cited as the ``father of modern Chicago blues \'\'.',
        'The Madonna of Chancellor Rolin is an oil painting by the Early Netherlandish master Jan van Eyck, dating from around 1435. It is kept in the Musée du Louvre, Paris, and was commissioned by Nicolas Rolin, aged 60, chancellor of the Duchy of Burgundy, whose votive portrait takes up the left side of the picture, for his parish church, "Notre-Dame-du-Chastel" in Autun, where it remained until the church burnt down in 1793. After a period in Autun Cathedral, it was moved to the Louvre in 1805.</s>Jon Monday (born 1947 in San Jose, California) is an American producer and distributor of CDs and DVDs across an eclectic range of material such as Swami Prabhavananda, Aldous Huxley, Christopher Isherwood, Huston Smith, Chalmers Johnson, and Charles Bukowski. Monday directed and co-produced with Jennifer Douglas the feature-length documentary "Save KLSD: Media Consolidation and Local Radio". He is also President of Benchmark Recordings, which owns and distributes the early catalog of The Fabulous Thunderbirds CDs and a live recording of Mike Bloomfield.</s>Strictly Inc. is the self-titled project album, released by Genesis keyboardist Tony Banks, and Wang Chung lead vocalist Jack Hues, in 1995 on Virgin Records. Tony Banks wanted the album release—as the title suggested—with no reference to the band members; but the record company went against his wishes. This was Banks\' fifth studio album (his second issued under a band name and seventh album overall).',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

Evaluation

Metrics

Cross Encoder Correlation

• Dataset: explorer-validation
• Evaluated with CECorrelationEvaluator

Metric	Value
pearson	0.9048
spearman	0.9036

Training Details

Training Dataset

Unnamed Dataset

• Size: 6,310 training samples
• Columns: sentence_0, sentence_1, and label
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1	label
  type	string	string	float
  details	min: 31 characters mean: 79.26 characters max: 168 characters	min: 606 characters mean: 1644.98 characters max: 4551 characters	min: 0.0 mean: 0.51 max: 1.0

• Samples:

  sentence_0	sentence_1	label
  In which year did the publisher of Roald Dahl's Guide to Railway Safety cease to exist?	Glacier Park International Airport is in Flathead County, Montana, six miles northeast of Kalispell. The airport is owned and operated by the Flathead Municipal Airport Authority, a public agency created by the county in 1974.Roald Dahl's Guide to Railway Safety was published in 1991 by the British Railways Board. The British Railways Board had asked Roald Dahl to write the text of the booklet, and Quentin Blake to illustrate it, to help young people enjoy using the railways safely.The British Railways Board (BRB) was a nationalised industry in the United Kingdom that operated from 1963 to 2001. Until 1997 it was responsible for most railway services in Great Britain, trading under the brand name British Railways and, from 1965, British Rail. It did not operate railways in Northern Ireland, where railways were the responsibility of the Government of Northern Ireland.	1.0
  Who is the brother of the developer of Buffy the Animated Series?	Waylon Malloy Payne (born April 5, 1972) is an American country singer, songwriter, musician and actor. He is the son of the country singer Sammi Smith.WVFM, known simply as FM 106.5 and formerly WQLR, is a Classic Hits-leaning Adult Contemporary outlet serving the Kalamazoo, Michigan radio market. The station's frequency is 106.5 MHz on the FM dial with an ERP of 33 kW. They are owned by Midwest Communications. WVFM 106.5 is located on a crowded Frequency across south-west lower Michigan. The station covers all of Kalamazoo/Battle Creek area, can be heard well in the Grand Rapids area, and reaches as far north and east as Lansing and Jackson. During summer months, the station can be received to Flint and Ann Arbor on occasion.Fred Iltis (Brno, Czechoslovakia, April 20, 1923 – San Jose, California, December 11, 2008) was an American entomologist. His research focused on the biosystematics and life cycle of mosquitoes.	0.0
  Who did the performer of So Under Pressure play in Home and Away?	The Acoma Massacre was fought in January 1599 between Spanish conquistadors and Acoma Native Americans in what is now New Mexico. After twelve soldiers were killed at Acoma Pueblo in 1598, the Spanish retaliated by launching a punitive expedition, which led to the deaths of around 800 men, women and children during a three - day battle. Several hundred survivors were also enslaved or otherwise severely punished.The NBA Championship ring is an annual award given by the National Basketball Association to the team that wins the NBA Finals. Rings are presented to the team's players, coaches, and members of the executive front office. Red Auerbach has the most rings overall with 16. Phil Jackson has the most as coach and Bill Russell has the most as a player (11 each)Tibetan sources say Deshin Shekpa also persuaded the Yongle Emperor not to impose his military might on Tibet as the Mongols had previously done. Thinley writes that before the Karmapa returned to Tibet, the Yongle Empe...	0.0

• Loss: BinaryCrossEntropyLoss with these parameters:

  {
      "activation_fn": "torch.nn.modules.linear.Identity",
      "pos_weight": null
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 4
• per_device_eval_batch_size: 4
• num_train_epochs: 1

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 4
• per_device_eval_batch_size: 4
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	explorer-validation_spearman
0.3169	500	0.3868	0.9090
0.6337	1000	0.4186	0.9041
0.9506	1500	0.3636	0.8895
1.0	1578	-	0.8906
-1	-1	-	0.8906
0.3169	500	0.3691	0.9014
0.6337	1000	0.3661	0.8900
0.9506	1500	0.3499	0.8936
1.0	1578	-	0.8942
-1	-1	-	0.8942
0.3169	500	0.3342	0.9036

Framework Versions

• Python: 3.12.11
• Sentence Transformers: 5.2.0
• Transformers: 4.57.6
• PyTorch: 2.9.1+cu128
• Accelerate: 1.12.0
• Datasets: 4.5.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}