maxwlnd/socialgroup_stance_classification_nli

Social Group Stance Classification (NLI)

A DeBERTa-v3-base NLI model fine-tuned to classify the author's stance toward a social group in a political text as positive, negative, or neutral.

This model is part of the group-appeal-detector package, which also provides group mention detection and mention clustering.

Model Details

• Base model: MoritzLaurer/deberta-v3-base-zeroshot-v2.0
• Task: Stance Classification (NLI-based)
• Labels: positive, negative, neutral
• Training data: 5,000 manually annotated sentences from UK House of Commons parliamentary debates (2010–2019), with the negative class oversampled by adding synthetic paraphrases for half of all negative social group appeals

How It Works

For each detected group mention, three hypotheses are formulated — one per stance class. The model chooses the class with the largest entailment probability as the predicted stance.

Performance

Cross-validated performance (95% confidence intervals based on the estimated standard error across folds in brackets):

Metric	Negative	Neutral	Positive	Macro-Avg.
F1	0.76 [0.72, 0.80]	0.80 [0.78, 0.81]	0.89 [0.89, 0.89]	0.81 [0.80, 0.83]
Precision	0.85 [0.77, 0.94]	0.81 [0.79, 0.84]	0.87 [0.86, 0.88]	0.85 [0.82, 0.87]
Recall	0.70 [0.62, 0.77]	0.78 [0.76, 0.80]	0.91 [0.89, 0.92]	0.79 [0.77, 0.82]

Usage

Via group-appeal-detector package (recommended)

pip install group-appeal-detector

from group_appeal_detector import GroupAppealDetector

detector = GroupAppealDetector(device="cpu")

sentence = "We must protect the rights of farmers."
target_group = "farmers"
result = detector.classify_stance(sentence, target_group)
print(result["predicted_stance"], result["stance_probs"])

For batch processing:

pairs = [
    ("We must protect the rights of farmers.", "farmers"),
    ("We do a lot for elderly people.", "elderly people"),
]
results_df = detector.classify_stance_batch(pairs, batch_size=8, as_df=True)

Direct usage with Transformers

from transformers import pipeline

pipe = pipeline(
    "zero-shot-classification",
    model="maxwlnd/socialgroup_stance_classification_nli",
)

sentence = "We must protect the rights of farmers."
target_group = "farmers"

hypotheses = [
    f"The text is positive towards {target_group}.",
    f"The text is negative towards {target_group}.",
    f"The text is neutral, or contains no stance, towards {target_group}.",
]

result = pipe(sentence, candidate_labels=hypotheses)
print(result["labels"][0], result["scores"][0])

Related Models

This model is one of three models in the group appeal detection pipeline:

Model	Task
maxwlnd/roberta_group_mention_detector	Detect social group mentions
maxwlnd/socialgroup_stance_classification_nli	Classify stance toward a group as positive, negative, or neutral (this model)
maxwlnd/cl_mention_embedding	Embed mentions for clustering into qualitative categories

Conceptual Background

The definition of a social group appeal used for annotating the training data is inspired by Lena Maria Huber and Alona O. Dolinsky and Will Horne, Alona O. Dolinsky and Lena Maria Huber.

A group appeal is an intentional act by a political actor that associates them with a social group in a supportive or critical manner. This model classifies whether the author's expressed stance toward the mentioned group is positive, negative, or neutral.

License

MIT