README.md

---
license: gemma
base_model: google/gemma-2-2b
tags:
- fine-tuned
- trading
- financial
- summarization
- lora
pipeline_tag: text-generation
---

# Gemma-2-2B Fine-tuned for Trading Journal Summarization

## Model Description
This model is a fine-tuned version of google/gemma-2-2b, specifically trained to generate structured bullet-point summaries of trading journal entries and financial documents.

## Training Details
- **Base Model**: google/gemma-2-2b
- **Fine-tuning Method**: LoRA (Low-Rank Adaptation)
- **LoRA Rank**: 16
- **LoRA Alpha**: 32
- **Target Modules**: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- **Training Epochs**: 3
- **Learning Rate**: 2e-05
- **Batch Size**: 2 (effective: 16)
- **Training Precision**: fp16

## Performance Metrics
- **Training Loss**: 0.4682
- **Validation Loss**: 0.4266
- **Perplexity**: 1.53

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "./gemma-2b-trader-fp16",
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("./gemma-2b-trader-fp16")

prompt = '''### Instruction:
Summarize the following trading journal entry using structured bullet points and precise trading terminology.

### Input:
[Your trading entry here]

### Summary:
'''

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Training Data
Fine-tuned on the EDGAR-CORPUS-Financial-Summarization dataset for financial text domain adaptation.

## Limitations
- Optimized for trading/financial context
- Best results with English text
- May require domain-specific prompting for optimal output

## Citation
If you use this model, please cite the original Gemma model and this fine-tuned version.