llama-3.2-1B-bio-kg-n200-m3-seed42

Model Description

This model is a fine-tuned version of meta-llama/Llama-3.2-1B on biographical knowledge graph data for knowledge completion tasks.

Training Data

• Dataset: Biographical KG (n=200, m=3, seed=42)
• Domain: Biographical knowledge graphs
• Task: Causal language modeling for knowledge completion
• Data Type: Synthetic data generated from knowledge graph triples

Training Details

Training Parameters

• Epochs: 20
• Batch Size: 32
• Learning Rate: 5e-05
• Lr Scheduler: cosine_with_min_lr
• Nodes: 200
• Edges Per Node: 3
• Random Seed: 42

Base Model

• Model: meta-llama/Llama-3.2-1B
• Architecture: Transformer-based causal language model

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained("r-takahashi/llama-3.2-1B-bio-kg-n200-m3-seed42")
tokenizer = AutoTokenizer.from_pretrained("r-takahashi/llama-3.2-1B-bio-kg-n200-m3-seed42")

# Example: Knowledge completion
input_text = "Albert Einstein was born in"
inputs = tokenizer(input_text, return_tensors="pt")

# Generate completion
outputs = model.generate(
    **inputs,
    max_new_tokens=50,
    do_sample=False,
    pad_token_id=tokenizer.pad_token_id
)

result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)