Model Card for Model ID
🧠 Model Card
Drug Review Sentiment Classification (3-Class)
Fine-tuned BERT on Healthcare User Reviews
📌 Model Overview
This model is a fine-tuned version of BERT-base-uncased for three-class sentiment classification on user-generated drug reviews.
The task consists of classifying textual reviews into:
- Negative
- Neutral
- Positive
The model was trained on a cleaned and preprocessed version of the Drugs.com review dataset.
📂 Dataset
Source: Drugs.com Drug Review Dataset
- User reviews
- Numerical rating (1–10 scale)
Label Mapping Strategy
Three-class sentiment labels were constructed as:
- Negative → Low ratings
- Neutral → Mid-range ratings
- Positive → High ratings
Text preprocessing pipeline:
- Lowercasing
- Punctuation removal
- Stopword filtering
- Lemmatization
- Tokenization using BERT tokenizer
🏗 Model Architecture
Base Model: BERT-base-uncased
Architecture Details:
- 12 Transformer encoder layers
- Hidden size: 768
- 12 attention heads
- Classification head added for 3 output labels
Loss Function:
- CrossEntropyLoss
Optimizer:
- AdamW
Training performed using:
- PyTorch
- HuggingFace Transformers
📊 Evaluation Metrics
Evaluation was conducted on a held-out validation set.
Metrics reported:
- Accuracy
- Precision (macro)
- Recall (macro)
- F1-score (macro)
- Confusion Matrix
Example
| Model | Feature | Accuracy | F1 |
| Logistic Regression | TF-IDF | 0.82 | 0.81 |
| Naive Bayes | TF-IDF | 0.78 | 0.77 |
| Random Forest | TF-IDF | 0.81 | 0.79 |
| LSTM | Word2Vec | 0.83 | 0.83 |
🔍 Key Findings
- Transformer-based contextual embeddings outperform classical TF-IDF based models in multi-class sentiment detection.
- The neutral class benefits significantly from contextual modeling.
- Domain-specific vocabulary (medical terminology) is better captured by BERT representations.
🧪 Example Inference
from transformers import pipeline
classifier = pipeline(
"text-classification",
model="erdemyavuz55/drug-review-sentiment-analysis-bert"
)
classifier("This medication worked great with minimal side effects.")
Example Output:
[{'label': 'POSITIVE', 'score': 0.97}]
📈 Comparison with Classical Models
In the broader study, we compared:
- TF-IDF + Logistic Regression
- TF-IDF + SVM
- Word2Vec + LSTM
- Random Forest
- Gradient Boosting
The fine-tuned BERT model achieved superior macro-F1 in the three-class setting.
⚠️ Limitations
- Trained only on English reviews
- Domain-specific slang may reduce accuracy
- Ratings-based labeling may introduce subjectivity bias
- No domain-adaptive pretraining performed
🚀 Future Work
- Domain-adaptive pretraining on medical corpora
- Model interpretability (SHAP, attention visualization)
- Deployment as REST API
- Cross-dataset generalization experiments
🤗 Citation
If you use this model, please cite:
Fine-tuned BERT for Drug Review Sentiment Classification, 2026.
- Downloads last month
- 7