README.md

---
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers
- agent-routing
- conversation-matching
language: en
license: apache-2.0
datasets:
- custom
metrics:
- cosine_similarity
base_model: sentence-transformers/all-MiniLM-L12-v2
---

# Gatekeeper Agent Responding Model

This is a fine-tuned [sentence-transformers](https://www.SBERT.net) model based on **all-MiniLM-L12-v2** that has been specifically trained for **agent routing and conversation matching**. The model determines whether agents should respond to conversations based on semantic similarity.

## Model Details

### Base Model
- **Base Model**: [sentence-transformers/all-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L12-v2)
- **Model Architecture**: MiniLM-L12 (Microsoft)
- **Embedding Dimensions**: 384
- **Max Sequence Length**: 256 tokens

### Training Data
The model was fine-tuned on two custom datasets using triplet training:
- **semantic_triplet_training_data_round1.pkl**: 469 samples
- **inverse_semantic_triplet_training_data.pkl**: 475 samples

Each sample contains:
- `anchor`: Conversation text or agent description
- `positive`: Similar/relevant text to the anchor
- `negative`: Dissimilar/irrelevant text to the anchor

### Training Configuration
- **Loss Function**: MultipleNegativesRankingLoss
- **Batch Size**: 16
- **Learning Rate**: 2e-5
- **Epochs**: 1
- **Warmup Ratio**: 0.1
- **Training Framework**: sentence-transformers v2.7.0+

### Performance
Evaluation results on held-out test sets:
- **Semantic Triplets Accuracy**: 97.87%
- **Inverse Semantic Triplets Accuracy**: 100.00%

## Usage

### Direct Usage (Sentence Transformers)
```python
from sentence_transformers import SentenceTransformer

# Load the model
model = SentenceTransformer('msugimura/gatekeeper_agent_responding')

# Example: Agent routing for conversation
conversation = "I've been feeling anxious and need help with stress management"
agent_descriptions = [
    "Licensed therapist specializing in anxiety and stress management",
    "Fitness trainer who creates workout routines for stress relief",
    "Financial advisor who helps with investment planning"
]

# Get embeddings
conversation_embedding = model.encode(conversation)
agent_embeddings = model.encode(agent_descriptions)

# Calculate similarities
from sentence_transformers.util import cos_sim
similarities = cos_sim(conversation_embedding, agent_embeddings)

print("Similarity scores:", similarities)
# Expected: Highest similarity with the therapist
```

### API Usage (Portcullis Service)
```python
import requests

# Example API call to Portcullis service
response = requests.post("http://localhost:8000/should_agents_respond", json={
    "conversation": "I've been feeling anxious and need help",
    "agent_descriptions": [
        "Licensed therapist specializing in anxiety treatment",
        "Fitness trainer for workout routines",
        "Financial advisor for investments"
    ],
    "threshold": 0.4
})

result = response.json()
print("Qualified agents:", result["qualified_agents"])
```

## Intended Use Cases

1. **Agent Routing**: Automatically route conversations to appropriate specialist agents
2. **Conversation Matching**: Match user queries with relevant service providers
3. **Semantic Search**: Find similar conversations or agent descriptions
4. **Content Recommendation**: Recommend agents based on conversation context

## Limitations

- **Domain Specific**: Optimized for agent-conversation matching scenarios
- **English Only**: Trained primarily on English text
- **Context Length**: Limited to 256 tokens per input
- **Training Data**: Performance depends on similarity to training domain

## Technical Details

### Model Architecture
```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
  (2): Normalize()
)
```

### Training Process
1. **Data Preprocessing**: Cleaned triplet datasets, removed extraneous columns
2. **Multi-Dataset Training**: Combined training on both semantic and inverse semantic data
3. **Loss Function**: MultipleNegativesRankingLoss with in-batch negatives
4. **Evaluation**: TripletEvaluator on held-out validation sets

## Citation

If you use this model, please cite:

```bibtex
@misc{gatekeeper_agent_responding_2024,
  title={Gatekeeper Agent Responding Model},
  author={Michael Sugimura},
  year={2024},
  publisher={Hugging Face},
  url={https://huggingface.co/msugimura/gatekeeper_agent_responding}
}
```

## Contact

For questions or issues, please contact [your-email] or open an issue in the model repository.