Qwen2.5-7B-3GPP-NR

A fine-tuned version of Qwen2.5-7B-Instruct specialized for answering technical questions about 3GPP 5G New Radio (NR) specifications.

Model Description

This model was fine-tuned on ~27,000 Q&A pairs derived from 3GPP 38-series technical specifications. It excels at answering detailed technical questions about 5G NR protocols, procedures, and configurations.

Model Details

Attribute	Value
Base Model	Qwen2.5-7B-Instruct
Fine-tuning Method	LoRA (merged)
LoRA Rank	32
LoRA Alpha	64
Training Data	26,926 Q&A pairs
Training Epochs	2
License	CC-BY-NC-4.0

Intended Use

Answering technical questions about 3GPP 5G NR specifications
Assisting telecom engineers with specification lookups
Research and educational purposes
Building domain-specific chatbots for telecommunications

How to Use

With Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "raoulbia/Qwen2.5-7B-3GPP-NR"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")

messages = [
    {"role": "system", "content": "You are a 3GPP technical specifications expert."},
    {"role": "user", "content": "What are the requirements for antenna configuration during FR1 testing?"}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.2, top_p=0.8)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

With Ollama (GGUF)

A GGUF version is available for use with Ollama. Create a Modelfile:

FROM path/to/model.gguf

TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ .Response }}<|im_end|>"""

SYSTEM "You are a 3GPP technical specifications expert."
PARAMETER temperature 0.2
PARAMETER stop "<|im_end|>"

Training Details

Training Data

Fine-tuned on raoulbia/3gpp-5g-nr-qa, a dataset of Q&A pairs generated from 3GPP 38-series specifications covering:

TS 38.211-214: Physical layer procedures
TS 38.101, 38.104: RF requirements
TS 38.133, 38.521, 38.533: Conformance testing
TS 38.331: RRC protocol
TS 38.300, 38.401: Architecture

Training Configuration

Framework: NVIDIA NeMo 25.09
Method: LoRA (Low-Rank Adaptation)
Epochs: 2
Batch Size: 1 (with gradient accumulation)
Learning Rate: 5e-6
Sequence Length: 2048
Hardware: NVIDIA DGX Spark

Evaluation

Evaluated on 50 samples from a held-out test set:

Metric	Value
Valid Responses	100%
Avg Response Length	~800 characters
Response Length vs Base	-28% (more concise)

Strengths

Concise, focused answers
Correct 3GPP terminology (RRC, E-UTRA, ENDC, SRS, NG-RAN)
Release-specific references (Rel-15, Rel-16, Rel-17)
Clean stopping behavior

Limitations

Scope: Only trained on 38-series (5G NR), not full 3GPP stack
General Knowledge: May struggle with definitional questions like "What is 3GPP?"
Verification: LLM-generated training data, not manually verified
Domain Bias: Strong bias toward technical spec patterns

Example Outputs

Question: What are the requirements for antenna configuration during FR1 testing for UEs with two or four Rx antennas?

Answer: During FR1 testing, if a User Equipment (UE) is equipped with either two or four receive (Rx) antennas, the same downlink signal must be applied to each antenna. It is essential that all receive antennas of the UE are connected unless the specific test case indicates otherwise. This configuration ensures that the testing accurately reflects the performance of the UE under typical operational conditions.

Citation

If you use this model, please cite the source dataset:

@misc{nikbakht2024tspecllm,
  title={TSpec-LLM: An Open-source Dataset for LLM Understanding of 3GPP Specifications},
  author={Rasoul Nikbakht and Mohamed Benzaghta and Giovanni Geraci},
  year={2024},
  eprint={2406.01768},
  archivePrefix={arXiv},
  primaryClass={cs.NI}
}