README.md

---
license: apache-2.0
base_model: microsoft/deberta-v3-base
library_name: transformers
pipeline_tag: text-classification
tags:
- deberta-v3
- human value detection
- schwartz values
- moral values
- political text
- retrieval augmented classification
- rag
- multi-label classification
model-index:
- name: value-context-rag-deberta-v3-base-doc-rag
  results:
  - task:
      type: text-classification
      name: Multi-label text classification
    dataset:
      name: ValuesML / Touché24-ValueEval
      type: restricted
    metrics:
    - type: f1
      name: Macro-F1 (seed 1701)
      value: 0.3224
    - type: f1
      name: Micro-F1 (seed 1701)
      value: 0.3617
language:
- en
metrics:
- f1
---

# Value Context RAG - DeBERTa-v3-base Document RAG

This model is the representative checkpoint for the best-performing supervised
encoder condition from:

> *More Context, Larger Models, or Moral Knowledge? A Systematic Study of Schwartz Value Detection in Political Texts*  
> Víctor Yeste, Paolo Rosso (2026), arXiv:2605.22641, submitted to ARR May 2026 / EMNLP 2026.

It is a **multi-label classifier** over the **19 refined Schwartz values**. The
model was trained with **document context** and **early-fusion retrieval-augmented
classification**: the input contains a target sentence marked inside its document
context, followed by two retrieved moral-knowledge snippets.

This checkpoint corresponds to:

- Base model: `microsoft/deberta-v3-base`
- Context condition: document context
- Retrieval: enabled, early fusion
- Retrieval top-k: 2 KB chunks
- Seed: 1701
- Recommended global threshold: 0.18

The paper reports this condition as mean±standard deviation over three seeds
(7, 42, 1701). This uploaded checkpoint is the seed-1701 run of the best
aggregate condition.

---

## Intended Use

This model is intended for research on:

- Schwartz value detection
- Moral and value-aware NLP
- Political text analysis
- Retrieval-augmented text classification
- Multi-label classification under label imbalance

The model should be used as a noisy research classifier, not as an authoritative
source of a person's values or beliefs.

Do **not** use this model for:

- Individual-level profiling
- Automated moderation
- Surveillance
- Ranking political actors or viewpoints
- High-stakes decision-making

---

## Labels

The model predicts the 19 refined Schwartz values:

1. `Self-direction: thought`
2. `Self-direction: action`
3. `Stimulation`
4. `Hedonism`
5. `Achievement`
6. `Power: dominance`
7. `Power: resources`
8. `Face`
9. `Security: personal`
10. `Security: societal`
11. `Tradition`
12. `Conformity: rules`
13. `Conformity: interpersonal`
14. `Humility`
15. `Benevolence: caring`
16. `Benevolence: dependability`
17. `Universalism: concern`
18. `Universalism: nature`
19. `Universalism: tolerance`

The original benchmark distinguishes attained and constrained values. For this
model, both variants are collapsed into a single binary value-presence label.

---

## Input Format

The model was trained with early-fusion RAG inputs of the form:

```text
TEXT:
... previous document sentence. <TGT>target sentence</TGT> following document sentence ...

KNOWLEDGE:
retrieved moral knowledge snippet 1

retrieved moral knowledge snippet 2
```

For exact reproduction of the paper condition, use the accompanying GitHub
repository to build document contexts, retrieve the top-2 KB chunks, and apply
the same token budgets. The released HF model contains the classifier and
tokenizer, but not the restricted benchmark texts.

The model can still be used with manually provided document context and
knowledge snippets, but performance may differ from the paper if the input
format or retrieval setup changes.

---

## How to Use

This is a standard Transformers sequence-classification model. It does not
require `trust_remote_code=True`.

```python
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

model_id = "VictorYeste/value-context-rag-deberta-v3-base-doc-rag"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)
model.eval()

id2label = {int(k): v for k, v in model.config.id2label.items()}

def build_doc_rag_input(document_sentences, target_index, kb_snippets):
    """Build a simplified version of the paper's document+RAG input format."""
    marked_sentences = []
    for i, sentence in enumerate(document_sentences):
        if i == target_index:
            marked_sentences.append(f"<TGT>{sentence}</TGT>")
        else:
            marked_sentences.append(sentence)

    document = " ".join(marked_sentences)
    knowledge = "\n\n".join(kb_snippets[:2])
    return f"TEXT:\n{document}\n\nKNOWLEDGE:\n{knowledge}"

def predict_values(text, threshold=0.18):
    enc = tokenizer(
        text,
        return_tensors="pt",
        truncation=True,
        max_length=1024,
    )

    with torch.no_grad():
        logits = model(**enc).logits.squeeze(0)
        probs = torch.sigmoid(logits).cpu()

    scores = {id2label[i]: float(probs[i]) for i in range(len(id2label))}
    labels = [label for label, score in scores.items() if score >= threshold]

    return {
        "labels": labels,
        "scores": scores,
    }

document = [
    "The city council debated a new climate adaptation plan.",
    "The proposal aims to protect rivers, parks, and future generations.",
    "Opponents argued that the cost would be too high for local businesses.",
]

kb_snippets = [
    "Universalism: nature concerns preservation of the natural environment and protection of nature.",
    "Universalism: concern concerns equality, justice, and protection for all people.",
]

model_input = build_doc_rag_input(
    document_sentences=document,
    target_index=1,
    kb_snippets=kb_snippets,
)

print(predict_values(model_input, threshold=0.18))
```

If no labels pass the threshold, treat the prediction as `NONE`.

---

## Training Data

The model was trained on the English ValuesML / Touché24-ValueEval human value
detection benchmark.

- Domain: political and socially oriented texts
- Prediction unit: target sentence
- Context: document-level context reconstructed with `text_id` and `sent_id`
- Labels: 19 refined Schwartz values
- Label formulation: attained/constrained annotations collapsed into value
  presence

Important: the original dataset is distributed under a restricted Data Usage
Agreement. This model repository does **not** redistribute raw dataset text. To
reproduce training or evaluation, users must obtain the official dataset
separately and comply with its terms.

---

## Training Setup

- Base model: `microsoft/deberta-v3-base`
- Architecture: `DebertaV2ForSequenceClassification`
- Task: 19-label multi-label classification
- Objective: binary cross-entropy with logits
- Input condition: document context + early-fusion RAG
- RAG top-k: 2 moral-knowledge chunks
- KB budget: 200 tokens
- Maximum sequence length: 1024
- Batch size: 8
- Gradient accumulation steps: 2
- Learning rate: 1e-5
- Weight decay: 0.15
- Maximum epochs: 20
- Early stopping: validation macro-F1
- Training precision: fp32
- Gradient checkpointing: enabled
- Seed: 1701
- Decision threshold: 0.18

The moral KB contains short, paraphrased chunks based on Schwartz value
definitions, annotation guidance, and theory-level contrasts. The repository
associated with the paper documents the KB construction and retrieval setup.

---

## Performance

On the held-out test split used in the paper, this specific seed-1701 checkpoint
achieves:

| Model condition | Macro-F1 | Micro-F1 |
| --- | ---: | ---: |
| DeBERTa-v3-base doc + early RAG, seed 1701 | 0.3224 | 0.3617 |

The paper reports the same condition as an aggregate over three seeds:

| Model condition | Macro-F1 | Micro-F1 |
| --- | ---: | ---: |
| DeBERTa-v3-base doc + early RAG, seeds 7/42/1701 | 0.314 ± 0.008 | 0.369 ± 0.010 |

This was the strongest aggregate condition among the tested supervised encoders
and zero-shot LLMs in the paper.

---

## Limitations

- The model is trained and evaluated on one benchmark and one broad genre:
  political and socially oriented text.
- It may not generalize to social media, dialogue, literary text, legal text,
  other languages, or non-political domains.
- Value labels are sparse and imbalanced; rare labels such as `Humility` and
  `Conformity: interpersonal` remain difficult.
- The model depends on the input format. Using sentence-only inputs, missing KB
  snippets, or different retrieval chunks can change predictions.
- The retrieved KB can shape predictions through its wording.
- No systematic fairness audit has been conducted.
- Predictions should be treated as uncertain analytical signals, not ground
  truth about people or groups.

---

## License

The model artifacts in this repository are released under the Apache License
2.0.

This license does not grant rights over:

- The underlying ValuesML / Touché24-ValueEval dataset
- Third-party copyrighted text in the benchmark
- The upstream `microsoft/deberta-v3-base` model beyond its own license terms

Users must obtain and use the original dataset under its own Data Usage
Agreement.

---

## Citation

If you use this model, please cite the associated paper:

```bibtex
@misc{yeste2026contextlargermodelsmoral,
  title={More Context, Larger Models, or Moral Knowledge? A Systematic Study of Schwartz Value Detection in Political Texts}, 
  author={Víctor Yeste and Paolo Rosso},
  year={2026},
  eprint={2605.22641},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2605.22641}, 
}
```

Please also cite the dataset/shared-task resources where appropriate:

```bibtex
@misc{ValueEval24Zenodo,
  author    = {{The ValuesML Team}},
  title     = {Touch{\'e}24{-}ValueEval},
  year      = {2024},
  month     = {8},
  version   = {2024-08-09},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.13283288},
  url       = {https://doi.org/10.5281/zenodo.13283288}
}
```