maxwlnd/roberta_group_mention_detector

RoBERTa Group Mention Detector

A RoBERTa-base token classification model fine-tuned to detect social group mentions in political text.

This model is part of the group-appeal-detector package, which also provides stance classification and mention clustering.

Model Details

• Base model: roberta-base
• Task: Sequence Labeling (BIO tagging)
• Labels: B-socialgroup, I-socialgroup, O
• Training data: 5,000 manually annotated sentences from UK House of Commons parliamentary debates (2010–2019), augmented with 25% synthetic paraphrases generated by a GPT-5-nano model

Performance

Cross-validated performance evaluated with seqeval (95% confidence intervals based on the estimated standard error across folds in brackets):

Seqeval-Metric	Score
F1	0.82 [0.82, 0.83]
Precision	0.80 [0.79, 0.81]
Recall	0.84 [0.83, 0.85]

Usage

Via group-appeal-detector package (recommended)

pip install group-appeal-detector

from group_appeal_detector import GroupAppealDetector

detector = GroupAppealDetector(device="cpu")

sentence = "Our party supports the interests of young people and working families."
mentions = detector.detect_mentions(sentence)

for m in mentions:
    print(m["span"], m["start"], m["end"])

For batch processing:

sentences = [
    "Farmers must earn more money.",
    "The government must do more to protect the women living in this country.",
]
results_df = detector.detect_mentions_batch(sentences, batch_size=8, as_df=True)

Direct usage with Transformers

from transformers import pipeline

pipe = pipeline(
    "token-classification",
    model="maxwlnd/roberta_group_mention_detector",
    aggregation_strategy="simple",
)

sentence = "Our party supports the interests of young people and working families."
results = pipe(sentence)

for r in results:
    print(r["word"], r["entity_group"], round(r["score"], 3))

Related Models

This model is one of three models in the group appeal detection pipeline:

Model	Task
maxwlnd/roberta_group_mention_detector	Detect social group mentions (this model)
maxwlnd/socialgroup_stance_classification_nli	Classify stance toward a group as positive, negative, or neutral
maxwlnd/cl_mention_embedding	Embed mentions for clustering into qualitative categories

Conceptual Background

The definitions of social group and social group appeal are inspired by Lena Maria Huber and Alona O. Dolinsky and Will Horne, Alona O. Dolinsky and Lena Maria Huber.

A social group is a segment of society whose members share common sociodemographic traits that are ascriptive and/or acquired. A reference to a social group in text is a group mention. A group appeal is an intentional act by a political actor that associates them with a social group in a supportive or critical manner.

License

MIT