--- tags: - sentence-transformers - cross-encoder - reranker - generated_from_trainer - dataset_size:1047 - loss:MultipleNegativesRankingLoss base_model: Qwen/Qwen3-Embedding-0.6B pipeline_tag: text-ranking library_name: sentence-transformers metrics: - accuracy - accuracy_threshold - f1 - f1_threshold - precision - recall - average_precision model-index: - name: CrossEncoder based on Qwen/Qwen3-Embedding-0.6B results: - task: type: cross-encoder-classification name: Cross Encoder Classification dataset: name: Unknown type: unknown metrics: - type: accuracy value: 0.9389312977099237 name: Accuracy - type: accuracy_threshold value: 0.726271390914917 name: Accuracy Threshold - type: f1 value: 0.9391634980988592 name: F1 - type: f1_threshold value: 0.726271390914917 name: F1 Threshold - type: precision value: 0.9356060606060606 name: Precision - type: recall value: 0.9427480916030534 name: Recall - type: average_precision value: 0.9508539647615596 name: Average Precision - type: accuracy value: 0.9435975609756098 name: Accuracy - type: accuracy_threshold value: 0.8168901205062866 name: Accuracy Threshold - type: f1 value: 0.944693572496263 name: F1 - type: f1_threshold value: 0.7354934215545654 name: F1 Threshold - type: precision value: 0.9266862170087976 name: Precision - type: recall value: 0.9634146341463414 name: Recall - type: average_precision value: 0.9544295903264528 name: Average Precision --- # CrossEncoder based on Qwen/Qwen3-Embedding-0.6B This is a [Cross Encoder](https://www.sbert.net/docs/cross_encoder/usage/usage.html) model finetuned from [Qwen/Qwen3-Embedding-0.6B](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B) using the [sentence-transformers](https://www.SBERT.net) library. It computes scores for pairs of texts, which can be used for text reranking and semantic search. ## Model Details ### Model Description - **Model Type:** Cross Encoder - **Base model:** [Qwen/Qwen3-Embedding-0.6B](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B) - **Maximum Sequence Length:** 32768 tokens - **Number of Output Labels:** 1 label ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Documentation:** [Cross Encoder Documentation](https://www.sbert.net/docs/cross_encoder/usage/usage.html) - **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers) - **Hugging Face:** [Cross Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=cross-encoder) ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import CrossEncoder # Download from the πŸ€— Hub model = CrossEncoder("vkimbris/qwen3_06b_items_reranker") # Get scores for pairs of texts pairs = [ ['Васаби ΠΏΠΎΡ€ΠΎΡˆΠΎΠΊ Π³ΠΎΡ€Ρ‡ΠΈΡ‡Π½Ρ‹ΠΉ ΠŸΡ€Π΅ΠΌΠΈΡƒΠΌ Fumiko Resfood 1ΠΊΠ³, 10ΡˆΡ‚/ΠΊΠΎΡ€, ΠšΠΈΡ…Π°ΠΉ', 'Васаби Fumiko Premium Π³Ρ€Π΅ΠΉΠ΄ А, 85% Ρ…Ρ€Π΅Π½Π°'], ['Боус ВСрияки Genso 1,5n/1,7ΠΊΠ³, Π±ΡˆΡ‚/ΠΊΠΎΡ€, Россия', 'Боус ВСрияки Genso'], ['Уксус рисовый Padam Prem Resfood 20Π», Россия', 'Уксус рисовый Padam Premium'], ['Π˜ΠΌΠ±ΠΈΡ€ΡŒ ΠΌΠ°Ρ€ΠΈΠ½ΠΎΠ²Π°Π½Π½Ρ‹ΠΉ Ρ€ΠΎΠ·ΠΎΠ²Ρ‹ΠΉ Tabuko Restood 1,5 ΠΊΠ³, вСс сухого Π²Π΅Ρ‰-Π²Π° 1ΠΊΠ³, 10ΡˆΡ‚/ΠΊΠΎΡ€, ΠšΠΈΡ‚Π°ΠΉ', 'Π˜ΠΌΠ±ΠΈΡ€ΡŒ ΠΌΠ°Ρ€ΠΈΠ½ΠΎΠ²Π°Π½Π½Ρ‹ΠΉ Tabuko Ρ€ΠΎΠ·ΠΎΠ²Ρ‹ΠΉ'], ["ΠŸΠ°ΡΡ‚Π° Π’ΠΎΠΌ Π―ΠΌ 'Genso' ΠΏΠ°ΠΊΠ΅Ρ‚ (0,400 ΠΊΠ³) ΡƒΠΏΠ°ΠΊ. 24 ΡˆΡ‚. Π’Π°ΠΉΠ»Π°Π½Π΄", 'ΠŸΠ°ΡΡ‚Π° Π’ΠΎΠΌ Π―ΠΌ Genso'], ] scores = model.predict(pairs) print(scores.shape) # (5,) # Or rank different texts based on similarity to a single text ranks = model.rank( 'Васаби ΠΏΠΎΡ€ΠΎΡˆΠΎΠΊ Π³ΠΎΡ€Ρ‡ΠΈΡ‡Π½Ρ‹ΠΉ ΠŸΡ€Π΅ΠΌΠΈΡƒΠΌ Fumiko Resfood 1ΠΊΠ³, 10ΡˆΡ‚/ΠΊΠΎΡ€, ΠšΠΈΡ…Π°ΠΉ', [ 'Васаби Fumiko Premium Π³Ρ€Π΅ΠΉΠ΄ А, 85% Ρ…Ρ€Π΅Π½Π°', 'Боус ВСрияки Genso', 'Уксус рисовый Padam Premium', 'Π˜ΠΌΠ±ΠΈΡ€ΡŒ ΠΌΠ°Ρ€ΠΈΠ½ΠΎΠ²Π°Π½Π½Ρ‹ΠΉ Tabuko Ρ€ΠΎΠ·ΠΎΠ²Ρ‹ΠΉ', 'ΠŸΠ°ΡΡ‚Π° Π’ΠΎΠΌ Π―ΠΌ Genso', ] ) # [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...] ``` ## Evaluation ### Metrics #### Cross Encoder Classification * Evaluated with [CrossEncoderClassificationEvaluator](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderClassificationEvaluator) | Metric | Value | |:----------------------|:-----------| | accuracy | 0.9389 | | accuracy_threshold | 0.7263 | | f1 | 0.9392 | | f1_threshold | 0.7263 | | precision | 0.9356 | | recall | 0.9427 | | **average_precision** | **0.9509** | #### Cross Encoder Classification * Evaluated with [CrossEncoderClassificationEvaluator](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderClassificationEvaluator) | Metric | Value | |:----------------------|:-----------| | accuracy | 0.9436 | | accuracy_threshold | 0.8169 | | f1 | 0.9447 | | f1_threshold | 0.7355 | | precision | 0.9267 | | recall | 0.9634 | | **average_precision** | **0.9544** | ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 1,047 training samples * Columns: premise and hypothesis * Approximate statistics based on the first 1000 samples: | | premise | hypothesis | |:--------|:------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------| | type | string | string | | details | | | * Samples: | premise | hypothesis | |:---------------------------------------------------------------------------------------|:----------------------------------------------------------------------------| | БмСсь мучная тСмпурная 'KANESHIRO' 1ΠΊΠ³ | ΠœΡƒΠΊΠ° тСмпурная Kaneshiro | | БмСсь тСмпурная Kaneshiro Resfood 1xr. 10ΡˆΡ‚/ΠΊΠΎΡ€ | ΠœΡƒΠΊΠ° тСмпурная Kaneshiro | | Π˜ΠΌΠ±ΠΈΡ€ΡŒ ΠΌΠ°Ρ€ΠΈΠ½ΠΎΠ²Π°Π½Π½Ρ‹ΠΉ Ρ€ΠΎΠ·ΠΎΠ²Ρ‹ΠΉ 'Hansey' 1,4 ΠΊΠ³*10 (Π².с. ΠšΠžΠ ΠžΠ‘ΠžΠš ПО 10 ΠŸΠΠ§Π•Πš) | Π˜ΠΌΠ±ΠΈΡ€ΡŒ ΠΌΠ°Ρ€ΠΈΠ½ΠΎΠ²Π°Π½Π½Ρ‹ΠΉ Ρ€ΠΎΠ·ΠΎΠ²Ρ‹ΠΉ Hansey, вСс сухого вСщСства 1000 Π³ | * Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/cross_encoder/losses.html#multiplenegativesrankingloss) with these parameters: ```json { "scale": 10.0, "num_negatives": 4, "activation_fn": "torch.nn.modules.activation.Sigmoid" } ``` ### Evaluation Dataset #### Unnamed Dataset * Size: 262 evaluation samples * Columns: premise and hypothesis * Approximate statistics based on the first 262 samples: | | premise | hypothesis | |:--------|:------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------| | type | string | string | | details | | | * Samples: | premise | hypothesis | |:----------------------------------------------------------------------------------|:-------------------------------------------------------| | Васаби ΠΏΠΎΡ€ΠΎΡˆΠΎΠΊ Π³ΠΎΡ€Ρ‡ΠΈΡ‡Π½Ρ‹ΠΉ ΠŸΡ€Π΅ΠΌΠΈΡƒΠΌ Fumiko Resfood 1ΠΊΠ³, 10ΡˆΡ‚/ΠΊΠΎΡ€, ΠšΠΈΡ…Π°ΠΉ | Васаби Fumiko Premium Π³Ρ€Π΅ΠΉΠ΄ А, 85% Ρ…Ρ€Π΅Π½Π° | | Боус ВСрияки Genso 1,5n/1,7ΠΊΠ³, Π±ΡˆΡ‚/ΠΊΠΎΡ€, Россия | Боус ВСрияки Genso | | Уксус рисовый Padam Prem Resfood 20Π», Россия | Уксус рисовый Padam Premium | * Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/cross_encoder/losses.html#multiplenegativesrankingloss) with these parameters: ```json { "scale": 10.0, "num_negatives": 4, "activation_fn": "torch.nn.modules.activation.Sigmoid" } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: steps - `per_device_train_batch_size`: 16 - `per_device_eval_batch_size`: 16 - `learning_rate`: 2e-05 - `num_train_epochs`: 15 - `warmup_ratio`: 0.1 - `fp16`: True #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: steps - `prediction_loss_only`: True - `per_device_train_batch_size`: 16 - `per_device_eval_batch_size`: 16 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 1 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 2e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 15 - `max_steps`: -1 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.1 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `bf16`: False - `fp16`: True - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: False - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `parallelism_config`: None - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch_fused - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `project`: huggingface - `trackio_space_id`: trackio - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: None - `hub_always_push`: False - `hub_revision`: None - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `include_for_metrics`: [] - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: no - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `use_liger_kernel`: False - `liger_kernel_config`: None - `eval_use_gather_object`: False - `average_tokens_across_devices`: True - `prompts`: None - `batch_sampler`: batch_sampler - `multi_dataset_batch_sampler`: proportional - `router_mapping`: {} - `learning_rate_mapping`: {}
### Training Logs | Epoch | Step | Training Loss | Validation Loss | average_precision | |:-------:|:----:|:-------------:|:---------------:|:-----------------:| | 1.5152 | 100 | 0.4864 | 0.1104 | 0.8944 | | 3.0303 | 200 | 0.1238 | 0.0983 | 0.9240 | | 4.5455 | 300 | 0.1106 | 0.0934 | 0.9466 | | 6.0606 | 400 | 0.1068 | 0.0939 | 0.9378 | | 7.5758 | 500 | 0.1135 | 0.1023 | 0.9232 | | 9.0909 | 600 | 0.1061 | 0.1187 | 0.9186 | | 10.6061 | 700 | 0.1074 | 0.0808 | 0.9445 | | 12.1212 | 800 | 0.1039 | 0.1153 | 0.9403 | | 13.6364 | 900 | 0.1082 | 0.0900 | 0.9509 | | -1 | -1 | - | - | 0.9544 | ### Framework Versions - Python: 3.12.3 - Sentence Transformers: 5.2.0 - Transformers: 4.57.3 - PyTorch: 2.9.1+cu128 - Accelerate: 1.12.0 - Datasets: 4.4.2 - Tokenizers: 0.22.1 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ```