syahrezapratama/bart-crypto-stance

An NLI-Based Approach to Asset-Specific Stance Detection in Cryptocurrency Tweets

This model classifies the stance of tweets toward Bitcoin (BTC) and Ethereum (ETH) as Bullish, Bearish, or Neutral using a Natural Language Inference (NLI) approach.

It was fine-tuned from facebook/bart-large-mnli as part of a master's thesis on NLI-based cryptocurrency stance detection.

How it works

Instead of standard 3-class classification, this model frames stance detection as an entailment task. For each tweet, three hypotheses are constructed (one per stance), and the model scores which hypothesis is most entailed by the tweet:

Stance	Hypothesis
Bullish	"The overall tone of this tweet suggests a bullish view regarding {target}."
Bearish	"The overall tone of this tweet suggests a bearish view regarding {target}."
Neutral	"The overall tone of this tweet suggests a neutral view regarding {target}."

The predicted stance is the one with the highest entailment score.

Usage

from transformers import pipeline

classifier = pipeline("zero-shot-classification", model="syahrezapratama/bart-crypto-stance")

tweet = "Bitcoin is going to the moon! $100k is just the beginning 🚀"

result = classifier(
    tweet,
    candidate_labels=["bullish", "bearish", "neutral"],
    hypothesis_template="The overall tone of this tweet suggests a {} view regarding BTC.",
)

print(result["labels"][0])  # "bullish"

Manual inference (more control)

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "syahrezapratama/bart-crypto-stance"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
model.eval()

tweet = "I'm not sure where ETH is headed, could go either way"
stances = ["bullish", "bearish", "neutral"]
template = "The overall tone of this tweet suggests a {} view regarding ETH."

scores = []
for stance in stances:
    hypothesis = template.format(stance)
    inputs = tokenizer(tweet, hypothesis, return_tensors="pt", truncation=True, max_length=128)
    with torch.no_grad():
        logits = model(**inputs).logits
    # Entailment is index 2 for BART-MNLI
    entailment_score = torch.softmax(logits, dim=-1)[0, 2].item()
    scores.append(entailment_score)

predicted = stances[scores.index(max(scores))]
print(f"Predicted stance: {predicted}")

Performance

Evaluated on a held-out test set of 450 tweets (70/15/15 train/val/test split, seed=42).

Overall metrics

Metric	Value
Accuracy	80.44%
Macro F1	0.7622
Weighted F1	0.7991

Per-class metrics

Class	Precision	Recall	F1	Support
Bearish	0.7333	0.7021	0.7174	47
Neutral	0.8046	0.9081	0.8532	272
Bullish	0.8367	0.6260	0.7162	131

Comparison with baselines

Model	Paradigm	Accuracy	Macro F1
BART-MNLI	Zero-Shot (Baseline)	44.22%	0.4359
BART-MNLI	Zero-Shot (OPRO)	59.56%	0.5212
BART-NLI	Fine-Tuned	80.44%	0.7622
GPT-4o	Zero-Shot	76.67%	0.7275

Training details

Dataset

• Source: 3,000 cryptocurrency tweets about BTC and ETH
• Labels: Bullish (873), Neutral (1,814), Bearish (313)
• Split: 2,099 train / 451 val / 450 test (seed=42)

NLI training approach

Each tweet is expanded into 3 NLI premise-hypothesis pairs:

• Correct stance → entailment
• Incorrect stances → contradiction

This results in 6,297 training pairs from 2,099 tweets.

Hyperparameters

Parameter	Value
Base model	facebook/bart-large-mnli
Learning rate	2e-5
Batch size	2 (physical) × 8 (accumulation) = 16 (effective)
Max epochs	5 (early stopping patience = 3)
Max sequence length	128
Warmup	10% linear warmup + linear decay
Weight decay	0.01
Class weights	Bearish=3.19, Neutral=0.55, Bullish=1.15
Gradient checkpointing	Enabled
Optimizer	AdamW
Best epoch	Early stopped based on validation macro F1

Hypothesis template (OPRO-optimized)

The hypothesis template was optimized using OPRO (Optimization by PROmpting) with GPT-4o-mini:

"The overall tone of this tweet suggests a {stance} view regarding {target}."

Limitations

• Domain-specific: Trained only on cryptocurrency tweets (BTC and ETH). May not generalize to other financial assets or domains.
• Class imbalance: Bearish tweets are underrepresented (10.4% of data), leading to lower recall on bearish stance despite class weighting.
• Language: English only.
• Temporal: Trained on tweets from a specific time period. Cryptocurrency language and sentiment patterns evolve rapidly.

Citation

If you use this model, please cite:

@mastersthesis{pratama2026cryptostancenli,
  title={An NLI-Based Approach to Asset-Specific Stance Detection in Cryptocurrency Tweets},
  author={Pratama, Syahreza},
  year={2026},
}

License

MIT