anass1209/resume-job-matcher-all-MiniLM-L6-v2

SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("anass1209/resume-job-matcher-all-MiniLM-L6-v2")
# Run inference
sentences = [
    'Developed and maintained core backend services using Python and Django, focusing on scalability and efficiency. Implemented RESTful APIs for data retrieval and manipulation.  Worked extensively with PostgreSQL for data storage and retrieval.  Responsible for optimizing database queries and improving API response times.  Experience with model fine-tuning for semantic search and document retrieval using pre-trained embedding models like Sentence Transformers or similar libraries, specifically for improving the relevance of search results and document matching within the web application.  Experience using vector databases (e.g., ChromaDB, Weaviate) preferred.',
    '## Senior Backend Engineer\n\n*   **ABC Corp** | 2020 - Present\n*   Led development of a new REST API for user authentication and profile management using Python and Django.\n*   Managed a PostgreSQL database, optimizing queries and schema design for improved performance, resulting in a 20% reduction in average API response time.\n*   Improved system scalability through efficient code design and load balancing techniques.\n*   Experience using pre-trained embedding models (BERT) for natural language processing tasks to improve search accuracy, with focus on keyphrase extraction and content similarity comparison for the recommendations engine. Proficient in Flask.',
    "PhD in Computer Science, University of California, Berkeley (2018-2023). Dissertation: 'Adversarial Robustness in NLP for Cybersecurity Applications.' Focused on fine-tuning BERT for malware detection and social engineering attacks. Proficient in Python, TensorFlow, and AWS. Published in top-tier NLP and security conferences. Experienced with large datasets and model evaluation metrics.\n\nMaster of Science in Cybersecurity, Johns Hopkins University (2016-2018). Relevant coursework included Machine Learning, Data Mining, and Network Security. Developed a system for anomaly detection using a recurrent neural network (RNN). Familiar with Python and cloud computing platforms. Good understanding of NLP concepts, but limited experience fine-tuning transformer models. Strong understanding of Information Security Principles.\n\nBachelor of Science in Computer Engineering, Carnegie Mellon University (2012-2016). Relevant coursework: Artificial Intelligence, Database Management, and Software Engineering. Project experience: Developed a web application using Python. No direct experience with fine-tuning NLP models, but a strong foundation in programming and data structures.  Familiar with cloud infrastructure concepts. Possess CISSP certification.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Semantic Similarity

• Datasets: dev_evaluation and test_evaluation
• Evaluated with EmbeddingSimilarityEvaluator

Metric	dev_evaluation	test_evaluation
pearson_cosine	0.5379	0.5379
spearman_cosine	0.6213	0.6213

Training Details

Training Dataset

Unnamed Dataset

• Size: 958 training samples
• Columns: sentence_0, sentence_1, and label
• Approximate statistics based on the first 958 samples:

  	sentence_0	sentence_1	label
  type	string	string	float
  details	min: 41 tokens mean: 110.12 tokens max: 234 tokens	min: 25 tokens mean: 134.18 tokens max: 256 tokens	min: 0.5 mean: 0.78 max: 0.96

• Samples:

  sentence_0	sentence_1	label
  Required skills include experience with embedding models, fine-tuning techniques, Python programming, and knowledge of NLP concepts. Proficiency in libraries like TensorFlow or PyTorch is essential. Familiarity with resume parsing and matching algorithms is a plus. Must be able to analyze performance metrics and iterate on model improvements.	Skills: Python, TensorFlow, NLP, Embedding Models, Fine-tuning, Resume Matching, Model Evaluation. Experienced in building and deploying machine learning models for text analysis and information retrieval. Proficient in analyzing performance using precision, recall, and F1-score to improve model accuracy. Skills: Python, PyTorch, Natural Language Processing, Text Classification, Machine Learning. Developed several machine learning models using Python and PyTorch for various text related tasks. Good understanding of model evaluation metrics. Technical Skills: Python, Scikit-learn, Data analysis, Data Visualization, Natural Language Processing basics. Projects include text classification and sentiment analysis. Knowledge of model evaluation techniques. Proficient in Python. Familiar with basic machine learning concepts and libraries. Experience with data cleaning and preprocessing. Strong analytical and problem-solving skills. Skills: Python, Pandas, Scikit-learn, Data Preprocessing...	0.8882194757461548
  Experience with embedding models and fine-tuning techniques. Ability to analyze resume data and identify relevant keywords for improved matching. Proficiency in Python and experience with relevant libraries like Transformers, Sentence Transformers, and scikit-learn. Knowledge of A/B testing and evaluation metrics (precision, recall, F1-score). Understanding of product management principles and the product development lifecycle is a plus.	Skills: * Python (proficient in Pandas, NumPy) * Machine Learning (basic understanding) * Data Analysis * A/B Testing (conducted tests for website optimization) * Excellent communication and presentation skills	0.5
  Senior DevOps Engineer to lead the implementation and optimization of our resume matching system. Responsibilities include: Fine-tuning and evaluating embedding models (e.g., Sentence Transformers, BERT) for improved semantic similarity scoring. Developing and maintaining the infrastructure for model training, evaluation, and deployment. Collaborating with data scientists and software engineers to integrate the matching system into our platform. Monitoring model performance and identifying areas for improvement, including data augmentation strategies. Strong experience with Python, cloud platforms (AWS, GCP, or Azure), containerization (Docker, Kubernetes), and CI/CD pipelines. Must have proficiency in evaluating model performance metrics (precision, recall, F1-score, AUC) and experience with model versioning and A/B testing.	## Experience Senior DevOps Engineer	Acme Corp

• Loss: CosineSimilarityLoss with these parameters:

  {
      "loss_fct": "torch.nn.modules.loss.MSELoss"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• num_train_epochs: 50
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 50
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• tp_size: 0
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	Training Loss	dev_evaluation_spearman_cosine	test_evaluation_spearman_cosine
1.0	60	-	0.4867	-
2.0	120	-	0.5612	-
3.0	180	-	0.5929	-
4.0	240	-	0.6229	-
5.0	300	-	0.6377	-
6.0	360	-	0.6434	-
7.0	420	-	0.6104	-
8.0	480	-	0.6064	-
8.3333	500	0.0122	-	-
9.0	540	-	0.6005	-
10.0	600	-	0.6064	-
11.0	660	-	0.5973	-
12.0	720	-	0.6097	-
13.0	780	-	0.5907	-
14.0	840	-	0.5870	-
15.0	900	-	0.5989	-
16.0	960	-	0.6018	-
16.6667	1000	0.0019	-	-
17.0	1020	-	0.6208	-
18.0	1080	-	0.6133	-
19.0	1140	-	0.6200	-
20.0	1200	-	0.5960	-
21.0	1260	-	0.5999	-
22.0	1320	-	0.5995	-
23.0	1380	-	0.6177	-
24.0	1440	-	0.6201	-
25.0	1500	0.0009	0.6110	-
26.0	1560	-	0.6184	-
27.0	1620	-	0.6133	-
28.0	1680	-	0.6287	-
29.0	1740	-	0.6200	-
30.0	1800	-	0.6272	-
31.0	1860	-	0.6222	-
32.0	1920	-	0.6199	-
33.0	1980	-	0.6141	-
33.3333	2000	0.0006	-	-
34.0	2040	-	0.6228	-
35.0	2100	-	0.6275	-
36.0	2160	-	0.6167	-
37.0	2220	-	0.6140	-
38.0	2280	-	0.6217	-
39.0	2340	-	0.6280	-
40.0	2400	-	0.6254	-
41.0	2460	-	0.6186	-
41.6667	2500	0.0005	-	-
42.0	2520	-	0.6185	-
43.0	2580	-	0.6242	-
44.0	2640	-	0.6183	-
45.0	2700	-	0.6213	-
46.0	2760	-	0.6220	-
47.0	2820	-	0.6213	-
48.0	2880	-	0.6213	-
49.0	2940	-	0.6214	-
50.0	3000	0.0004	0.6213	-
-1	-1	-	-	0.6213

Framework Versions

• Python: 3.11.11
• Sentence Transformers: 4.1.0
• Transformers: 4.51.1
• PyTorch: 2.5.1+cu124
• Accelerate: 1.3.0
• Datasets: 3.5.0
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}