SentenceTransformer

This model was finetuned with Unsloth.

based on unsloth/Qwen3-Embedding-0.6B

This is a sentence-transformers model finetuned from unsloth/Qwen3-Embedding-0.6B on the xativa-embedding-corpus dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: unsloth/Qwen3-Embedding-0.6B
Maximum Sequence Length: 8192 tokens
Output Dimensionality: 1024 dimensions
Similarity Function: Cosine Similarity
Training Dataset:
- xativa-embedding-corpus

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'PeftModelForFeatureExtraction'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Código Java para imprimir triángulo de asteriscos con recursión',
    '### Triángulo recursivo en JAVA\n\nEjecución figura recursiva usando la pila de llamadas:\n\n\n|String frase = "Hola mundo!!";<br>int cantidad = 0;<br>for(int i = 0; i < frase.length(); i++) {<br>if(frase.charAt(i) == \'a\' || frase.charAt(i) == \'A<br>cantidad++;<br>}<br>System.out.println("La letra A ha aparecido "+canti<br>String frase = "Programación en JAVA";<br>int cantidad = 0;<br>for(int i = 0; i < frase.length(); i++) {<br>if(frase.charAt(i) == \'a\' || frase.charAt|Col2|\')|\n|---|---|---|\n|**static void filaTriangulo(int n)**<br>{<br>    if(n>0) {<br>        System.out.print("* ");<br>        filaTriangulo(n - 1);<br>    }<br>    else System.out.println();<br>}<br>**static void triangulo(int n)**<br>{<br>    if(n>0) {<br>        triangulo(n - 1);<br>        filaTriangulo(n);<br>    }<br>}<br>**public static void main(String[] args)**<br>{<br>  triangulo(3);<br>}<br> *|*||\n|**static void filaTriangulo(int n)**<br>{<br>    if(n>0) {<br>        System.out.print("* ");<br>        filaTriangulo(n - 1);<br>    }<br>    else System.out.println();<br>}<br>**static void triangulo(int n)**<br>{<br>    if(n>0) {<br>        triangulo(n - 1);<br>        filaTriangulo(n);<br>    }<br>}<br>**public static void main(String[] args)**<br>{<br>  triangulo(3);<br>}<br> *|*||',
    '',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.8568, 0.1317],
#         [0.8568, 1.0000, 0.1611],
#         [0.1317, 0.1611, 1.0000]])

Evaluation

Metrics

Information Retrieval

Dataset: xativa-embedding-v2
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.7087
cosine_accuracy@3	0.8719
cosine_accuracy@5	0.9174
cosine_accuracy@10	0.9573
cosine_precision@1	0.7087
cosine_precision@3	0.2906
cosine_precision@5	0.1835
cosine_precision@10	0.0957
cosine_recall@1	0.7087
cosine_recall@3	0.8719
cosine_recall@5	0.9174
cosine_recall@10	0.9573
cosine_ndcg@10	0.8381
cosine_mrr@10	0.7994
cosine_map@100	0.8015

Training Details

Training Dataset

xativa-embedding-corpus

Dataset: xativa-embedding-corpus at ce31e12
Size: 27,570 training samples
Columns: anchor, positive, ciclo, asignatura, and source_file

Approximate statistics based on the first 1000 samples:

	anchor	positive	ciclo	asignatura	source_file
type	string	string	string	string	string
details	min: 5 tokens mean: 18.64 tokens max: 54 tokens	min: 41 tokens mean: 364.5 tokens max: 1221 tokens	min: 1 tokens mean: 1.0 tokens max: 1 tokens	min: 1 tokens mean: 1.0 tokens max: 1 tokens	min: 1 tokens mean: 1.0 tokens max: 1 tokens

Samples:

anchor	positive	ciclo	asignatura	source_file
`¿Qué es la clase Medium en php-code-coverage?`	php<br><?php declare(strict_types=1);<br>/<br> This file is part of phpunit/php-code-coverage.<br> <br> (c) Sebastian Bergmann <sebastian@phpunit.de><br> <br> For the full copyright and license information, please view the LICENSE<br> * file that was distributed with this source code.<br> /<br>namespace SebastianBergmann\CodeCoverage\Test\TestSize;<br><br>/<br> @psalm-immutable<br> /<br>final class Medium extends Known<br>{<br> /<br> @psalm-assert-if-true Medium $this<br> */<br> public function isMedium(): bool<br> {<br> return true;<br> }<br><br> public function isGreaterThan(TestSize $other): bool<br> {<br> return $other->isSmall();<br> }<br><br> public function asString(): string<br> {<br> return 'medium';<br> }<br>}<br><br>
`¿Cómo implementar clases para gestionar facturas y líneas en Java con constructores y métodos get/set?`	Examen Final DAW - PROGRAMACIÓN Nombre: Fecha: Se quiere realizar una aplicación para llevar la gestión de las Facturas de diferentes Tiendas. Se desea gestionar las Facturas y las Líneas que se realizan en cada una de ellas. Cada Tienda se identifica por un nombre y el número de facturas que tiene. Cada Factura se caracteriza por su id, fecha y hora y totalfact. Cada factura puede tener 1 o muchas líneas. Cada línea tiene un id, un producto, un precio unitario y una cantidad. Se pide: Implementar las clases que permitan almacenar los datos de una Tienda, sus Facturas y las Líneas que se representan, con sus correspondientes constructores. Los atributos de clase serán privados por lo tanto se especificarán los sets y gets correspondientes. (1 punto) Implementa los métodos asociados a Factura que se indican a continuación : añadirLinea: Permite añadir una línea. modificarLinea: Permite cambiar el precio unitario de una línea. buscarLinea: Comprueba si una lín...
`ManifestDocumentMapper map method PHP ejemplo`	```php /* * This file is part of PharIo\Manifest. * * (c) Arne Blankerts arne@blankerts.de, Sebastian Heuer sebastian@phpeople.de, Sebastian Bergmann sebastian@phpunit.de * * For the full copyright and license information, please view the LICENSE * file that was distributed with this source code. */ namespace PharIo\Manifest; use PharIo\Version\Exception as VersionException; use PharIo\Version\Version; use PharIo\Version\VersionConstraintParser; class ManifestDocumentMapper { public function map(ManifestDocument $document): Manifest { try { $contains = $document->getContainsElement(); $type = $this->mapType($contains); $copyright = $this->mapCopyright($document->getCopyrightElement()); $requirements = $this->mapRequirements($document->getRequiresElement()); $bundledComponents = $this->mapBundledComponents($document); ret...

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false,
    "directions": [
        "query_to_doc"
    ],
    "partition_mode": "joint",
    "hardness_mode": null,
    "hardness_strength": 0.0
}

Evaluation Dataset

xativa-embedding-corpus

Dataset: xativa-embedding-corpus at ce31e12
Size: 1,452 evaluation samples
Columns: anchor, positive, ciclo, asignatura, and source_file

Approximate statistics based on the first 1000 samples:

	anchor	positive	ciclo	asignatura	source_file
type	string	string	string	string	string
details	min: 5 tokens mean: 18.68 tokens max: 58 tokens	min: 41 tokens mean: 350.68 tokens max: 1416 tokens	min: 1 tokens mean: 1.0 tokens max: 1 tokens	min: 1 tokens mean: 1.0 tokens max: 1 tokens	min: 1 tokens mean: 1.0 tokens max: 1 tokens

Samples:

anchor	positive	ciclo	asignatura	source_file
`Ejemplo de comparación de cadenas en PHP con var_dump`	```php Exemple comparació cadenes Exemple comparació cadenes $cadena1 = "avellana"; $cadena2 = "meló"; echo "Valor de: $cadena1 < $cadena2 :"; $resultat1 = $cadena1 < $cadena2; var_dump($resultat1); $cadena1 = "cotxe"; $cadena2 = "vehicle"; echo "Valor de: $cadena1 > $cadena2 :"; $resultat2 = $cadena1 > $cadena2; var_dump($resultat2); $cadena1 = "ARBRE"; $cadena2 = "arbre"; echo "Valor de: $cadena1 > $cadena2 :"; $resultat3 = $cadena1 > $cadena2; var_dump($resultat3); <?php $cadena1 = "avellana"; $cadena2 = "meló"; echo "Valor de: $cadena1 < $cadena2 :"; $resultat1 = $cadena1 < $cadena2; var_dump($resultat1); echo " "; $cadena1 = "cotxe"; $cadena2 = "vehicle"; echo "Valor de: $cadena1 > $cadena2 :"; $result...
`PropertyFetch PHP Node ejemplo código`	php<br><?php declare(strict_types=1);<br><br>namespace PhpParser\Node\Expr;<br><br>use PhpParser\Node;<br>use PhpParser\Node\Expr;<br>use PhpParser\Node\Identifier;<br><br>class PropertyFetch extends Expr {<br> /** @var Expr Variable holding object /<br> public Expr $var;<br> /* @var Identifier\|Expr Property name /<br> public Node $name;<br><br> /<br> Constructs a function call node.<br> <br> @param Expr $var Variable holding object<br> * @param string\|Identifier\|Expr $name Property name<br> * @param array<string, mixed> $attributes Additional attributes<br> */<br> public function __construct(Expr $var, $name, array $attributes = []) {<br> $this->attributes = $attributes;<br> $this->var = $var;<br> $this->name = \is_string($name) ? new Identifier($name) : $name;<br> }<br><br> public function getSubNodeNames(): array {<br> return ['var', 'name'];<br> }<br><br> public function getType(): string {<br> return 'Expr_PropertyFetch';<br> }<br>}<br><br>
`¿Cómo funciona flex-shrink y order en CSS Flexbox con ejemplo práctico?`	css<br>* {<br> margin: 0;<br> padding: 0;<br> background-color: aquamarine;<br>}<br><br>#container {<br> display:flex;<br> align-items:center;<br> flex-flow:row-reverse nowrap;<br> justify-content:space-around;<br><br> background-color:coral;<br> height:200px;<br> margin:20px auto;<br> padding:10px;<br> width:60%;<br> }<br><br>#container div{<br> width:40%;<br> height:50px;<br><br> background-color: white;<br> border:1px solid black;<br> font-size:1.3em;<br> color:black;<br> font-family: sans-serif;<br> text-align: center;<br>}<br>#container div:nth-child(1){<br> order:2;<br> flex-shrink: 1;<br>}<br>#container div:nth-child(2){<br> order:3;<br> flex-shrink: 1;<br>}<br>#container div:nth-child(3){<br> order:1;<br> flex-shrink: 1;<br>}<br><br>

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false,
    "directions": [
        "query_to_doc"
    ],
    "partition_mode": "joint",
    "hardness_mode": null,
    "hardness_strength": 0.0
}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 256
num_train_epochs: 1
learning_rate: 2e-05
lr_scheduler_type: constant_with_warmup
warmup_steps: 0.1
bf16: True
eval_strategy: steps
per_device_eval_batch_size: 256
load_best_model_at_end: True
seed: 3407
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

per_device_train_batch_size: 256
num_train_epochs: 1
max_steps: -1
learning_rate: 2e-05
lr_scheduler_type: constant_with_warmup
lr_scheduler_kwargs: None
warmup_steps: 0.1
optim: adamw_torch_fused
optim_args: None
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
optim_target_modules: None
gradient_accumulation_steps: 1
average_tokens_across_devices: True
max_grad_norm: 1.0
label_smoothing_factor: 0.0
bf16: True
fp16: False
bf16_full_eval: False
fp16_full_eval: False
tf32: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
use_liger_kernel: False
liger_kernel_config: None
use_cache: False
neftune_noise_alpha: None
torch_empty_cache_steps: None
auto_find_batch_size: False
log_on_each_node: True
logging_nan_inf_filter: True
include_num_input_tokens_seen: no
log_level: passive
log_level_replica: warning
disable_tqdm: False
project: huggingface
trackio_space_id: trackio
eval_strategy: steps
per_device_eval_batch_size: 256
prediction_loss_only: True
eval_on_start: False
eval_do_concat_batches: True
eval_use_gather_object: False
eval_accumulation_steps: None
include_for_metrics: []
batch_eval_metrics: False
save_only_model: False
save_on_each_node: False
enable_jit_checkpoint: False
push_to_hub: False
hub_private_repo: None
hub_model_id: None
hub_strategy: every_save
hub_always_push: False
hub_revision: None
load_best_model_at_end: True
ignore_data_skip: False
restore_callback_states_from_checkpoint: False
full_determinism: False
seed: 3407
data_seed: None
use_cpu: False
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_pin_memory: True
dataloader_persistent_workers: False
dataloader_prefetch_factor: None
remove_unused_columns: True
label_names: None
train_sampling_strategy: random
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
ddp_backend: None
ddp_timeout: 1800
fsdp: []
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
deepspeed: None
debug: []
skip_memory_metrics: True
do_predict: False
resume_from_checkpoint: None
warmup_ratio: None
local_rank: -1
prompts: None
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	xativa-embedding-v2_cosine_ndcg@10
-1	-1	-	0.8096
0.0926	10	0.0363	-
0.1852	20	0.0044	-
0.2778	30	0.0011	-
0.3704	40	0.0068	-
0.4630	50	0.0005	-
0.5556	60	0.0050	-
0.6481	70	0.0032	-
0.7407	80	0.0000	-
0.8333	90	0.0001	-
0.9259	100	0.0000	-
-1	-1	-	0.8381

Framework Versions

Python: 3.12.3
Sentence Transformers: 5.3.0
Transformers: 5.3.0
PyTorch: 2.10.0+cu128
Accelerate: 1.13.0
Datasets: 4.3.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}