Hugging Face's logo

Miya67
/
aiq-scoring-e5-small-wiki-absolute

Text Ranking
sentence-transformers
Safetensors
xlm-roberta
cross-encoder
reranker
Generated from Trainer
dataset_size:104687
loss:BinaryCrossEntropyLoss
text-embeddings-inference
Model card Files
xet
 Community
aiq-scoring-e5-small-wiki-absolute / README.md
Miya67's picture
Miya67
Add new CrossEncoder model
8882144 verified
preview code
|
raw
history blame contribute delete
12.3 kB
metadata 
tags:
  - sentence-transformers
  - cross-encoder
  - reranker
  - generated_from_trainer
  - dataset_size:104687
  - loss:BinaryCrossEntropyLoss
base_model: hotchpotch/japanese-reranker-cross-encoder-small-v1
pipeline_tag: text-ranking
library_name: sentence-transformers

CrossEncoder based on hotchpotch/japanese-reranker-cross-encoder-small-v1

This is a Cross Encoder model finetuned from hotchpotch/japanese-reranker-cross-encoder-small-v1 using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.

Model Details

Model Description

Model Sources

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("Miya67/aiq-scoring-e5-small-wiki-absolute")
# Get scores for pairs of texts
pairs = [
    ['問題: 俳優の哀川翔、女優の小西真奈美、歌手の長渕剛の出身都道府県はどこでしょう? / 想定解: 鹿児島県', '熊毛地域'],
    ['問題: 和名を「トウショウブ」や「オランダショウブ」という、剣のように尖った葉が特徴的なアヤメ科の植物は何でしょう? / 想定解: グラジオラス', 'グラジオラス属'],
    ['問題: 月見そばで、卵の黄身が表しているものは月ですが、白身が表しているものは何でしょう? / 想定解: 雲', '分子雲'],
    ['問題: 1950年に第1回日本シリーズの第1戦が行われた、現在はヤクルトスワローズが本拠地とする野球場はどこでしょう? / 想定解: 明治神宮野球場', '明治神宮野球大会'],
    ['問題: オーストラリアの6つの州の中で、最も面積が大きいのは西オーストラリア州ですが、最も面積が小さいのは何州でしょう? / 想定解: タスマニア州', 'たすまにあしゅうそうとく'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    '問題: 俳優の哀川翔、女優の小西真奈美、歌手の長渕剛の出身都道府県はどこでしょう? / 想定解: 鹿児島県',
    [
        '熊毛地域',
        'グラジオラス属',
        '分子雲',
        '明治神宮野球大会',
        'たすまにあしゅうそうとく',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

Training Details

Training Dataset

Unnamed Dataset

  • Size: 104,687 training samples
  • Columns: sentence_0, sentence_1, and label
  • Approximate statistics based on the first 1000 samples:
    sentence_0 sentence_1 label
    type string string float
    details
    • min: 29 characters
    • mean: 64.2 characters
    • max: 108 characters
    • min: 1 characters
    • mean: 7.33 characters
    • max: 30 characters
    • min: 0.0
    • mean: 0.59
    • max: 1.0
  • Samples:
    sentence_0 sentence_1 label
    問題: 俳優の哀川翔、女優の小西真奈美、歌手の長渕剛の出身都道府県はどこでしょう? / 想定解: 鹿児島県 熊毛地域 1.0
    問題: 和名を「トウショウブ」や「オランダショウブ」という、剣のように尖った葉が特徴的なアヤメ科の植物は何でしょう? / 想定解: グラジオラス グラジオラス属 1.0
    問題: 月見そばで、卵の黄身が表しているものは月ですが、白身が表しているものは何でしょう? / 想定解: 雲 分子雲 0.0
  • Loss: BinaryCrossEntropyLoss with these parameters:
    {
    "activation_fn": "torch.nn.modules.linear.Identity",
    "pos_weight": null
}

Training Hyperparameters

Non-Default Hyperparameters

  • per_device_train_batch_size: 32
  • num_train_epochs: 4
  • per_device_eval_batch_size: 32

All Hyperparameters

Click to expand
  • per_device_train_batch_size: 32
  • num_train_epochs: 4
  • max_steps: -1
  • learning_rate: 5e-05
  • lr_scheduler_type: linear
  • lr_scheduler_kwargs: None
  • warmup_steps: 0
  • optim: adamw_torch_fused
  • optim_args: None
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • optim_target_modules: None
  • gradient_accumulation_steps: 1
  • average_tokens_across_devices: True
  • max_grad_norm: 1
  • label_smoothing_factor: 0.0
  • bf16: False
  • fp16: False
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: None
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • use_liger_kernel: False
  • liger_kernel_config: None
  • use_cache: False
  • neftune_noise_alpha: None
  • torch_empty_cache_steps: None
  • auto_find_batch_size: False
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • include_num_input_tokens_seen: no
  • log_level: passive
  • log_level_replica: warning
  • disable_tqdm: False
  • project: huggingface
  • trackio_space_id: trackio
  • eval_strategy: no
  • per_device_eval_batch_size: 32
  • prediction_loss_only: True
  • eval_on_start: False
  • eval_do_concat_batches: True
  • eval_use_gather_object: False
  • eval_accumulation_steps: None
  • include_for_metrics: []
  • batch_eval_metrics: False
  • save_only_model: False
  • save_on_each_node: False
  • enable_jit_checkpoint: False
  • push_to_hub: False
  • hub_private_repo: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_always_push: False
  • hub_revision: None
  • load_best_model_at_end: False
  • ignore_data_skip: False
  • restore_callback_states_from_checkpoint: False
  • full_determinism: False
  • seed: 42
  • data_seed: None
  • use_cpu: False
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • parallelism_config: None
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • dataloader_prefetch_factor: None
  • remove_unused_columns: True
  • label_names: None
  • train_sampling_strategy: random
  • length_column_name: length
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • ddp_backend: None
  • ddp_timeout: 1800
  • fsdp: []
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • deepspeed: None
  • debug: []
  • skip_memory_metrics: True
  • do_predict: False
  • resume_from_checkpoint: None
  • warmup_ratio: None
  • local_rank: -1
  • prompts: None
  • batch_sampler: batch_sampler
  • multi_dataset_batch_sampler: proportional
  • router_mapping: {}
  • learning_rate_mapping: {}

Training Logs

Epoch Step Training Loss
0.1528 500 0.6662
0.3056 1000 0.6092
0.4584 1500 0.5728
0.6112 2000 0.5385
0.7641 2500 0.5165
0.9169 3000 0.4984
1.0697 3500 0.4844
1.2225 4000 0.4681
1.3753 4500 0.4557
1.5281 5000 0.4539
1.6809 5500 0.4460
1.8337 6000 0.4402
1.9866 6500 0.4317
2.1394 7000 0.4071
2.2922 7500 0.3955
2.4450 8000 0.3957
2.5978 8500 0.3874
2.7506 9000 0.3996
2.9034 9500 0.3933
3.0562 10000 0.3788
3.2090 10500 0.3635
3.3619 11000 0.3613
3.5147 11500 0.3633
3.6675 12000 0.3584
3.8203 12500 0.3592
3.9731 13000 0.3555

Framework Versions

  • Python: 3.12.12
  • Sentence Transformers: 5.2.3
  • Transformers: 5.3.0
  • PyTorch: 2.10.0+cu128
  • Accelerate: 1.12.0
  • Datasets: 4.0.0
  • Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}