README_api.md

---
base_model: Qwen/Qwen2-7B-Instruct
library_name: peft
license: other
tags:
- llama-factory
- lora
- generated_from_trainer
model-index:
- name: train_2024-06-17-19-49-05
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Install some dependency 
```bash
pip install openai huggingface_hub
```

# Download lora 
```python
from huggingface_hub import snapshot_download
snapshot_download(
        repo_id="svjack/Qwen2-7B_Function_Call_tiny_lora",
        repo_type="model",
        local_dir="Qwen2-7B_Function_Call_tiny_lora",
        local_dir_use_symlinks = False
    )
```

# Start OpenAI style api server 
```bash
python src/api.py \
--model_name_or_path Qwen/Qwen2-7B-Instruct \
--template qwen \
--adapter_name_or_path Qwen2-7B_Function_Call_tiny_lora \
--quantization_bit 4
```

# Inference 
```python
import json
import os
from typing import Sequence

from openai import OpenAI
from transformers.utils.versions import require_version

require_version("openai>=1.5.0", "To fix: pip install openai>=1.5.0")

def calculate_gpa(grades: Sequence[str], hours: Sequence[int]) -> float:
    grade_to_score = {"A": 4, "B": 3, "C": 2}
    total_score, total_hour = 0, 0
    for grade, hour in zip(grades, hours):
        total_score += grade_to_score[grade] * hour
        total_hour += hour
    return round(total_score / total_hour, 2)

client = OpenAI(
        api_key="0",
        base_url="http://localhost:{}/v1".format(os.environ.get("API_PORT", 8000)),
    )

tools = [
        {
            "type": "function",
            "function": {
                "name": "calculate_gpa",
                "description": "Calculate the Grade Point Average (GPA) based on grades and credit hours",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "grades": {"type": "array", "items": {"type": "string"}, "description": "The grades"},
                        "hours": {"type": "array", "items": {"type": "integer"}, "description": "The credit hours"},
                    },
                    "required": ["grades", "hours"],
                },
            },
        }
    ]
tool_map = {"calculate_gpa": calculate_gpa}

messages = []
messages.append({"role": "user", "content": "My grades are A, A, B, and C. The credit hours are 3, 4, 3, and 2."})

result = client.chat.completions.create(messages=messages,
                                        model="Qwen/Qwen2-7B-Instruct", tools=tools)

result.choices[0].message.tool_calls

messages.append(result.choices[0].message)
tool_call = result.choices[0].message.tool_calls[0].function
print(tool_call)

name, arguments = tool_call.name, json.loads(tool_call.arguments)
tool_result = tool_map[name](**arguments)

messages.append({"role": "tool", "content": json.dumps({"gpa": tool_result}, ensure_ascii=False)})

result = client.chat.completions.create(messages=messages, model="test", tools=tools)
print(result.choices[0].message.content)
```

# Output 
```
Function(arguments='{"grades": ["A", "A", "B", "C"], "hours": [3, 4, 3, 2]}', name='calculate_gpa')
Based on the grades and credit hours you provided, your calculated GPA is 3.42.
```

# Inference 
```python
messages = []
messages.append({"role": "user", "content": "我的成绩分别是A,A,B,C学分分别是3, 4, 3,和2"})

result = client.chat.completions.create(messages=messages,
                                        model="Qwen/Qwen2-7B-Instruct", tools=tools)

result.choices[0].message.tool_calls

messages.append(result.choices[0].message)
tool_call = result.choices[0].message.tool_calls[0].function
print(tool_call)

name, arguments = tool_call.name, json.loads(tool_call.arguments)
tool_result = tool_map[name](**arguments)

messages.append({"role": "tool", "content": json.dumps({"gpa": tool_result}, ensure_ascii=False)})

result = client.chat.completions.create(messages=messages, model="test", tools=tools)
print(result.choices[0].message.content)
```

# Output 
```
Function(arguments='{"grades": ["A", "A", "B", "C"], "hours": [3, 4, 3, 2]}', name='calculate_gpa')
您提供的成绩和学分的加权平均分（GPA）是3.42。
```


# train_2024-06-17-19-49-05

This model is a fine-tuned version of [Qwen/Qwen2-7B-Instruct](https://huggingface.co/Qwen/Qwen2-7B-Instruct) on the glaive_toolcall_zh and the glaive_toolcall_en datasets.

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 1
- eval_batch_size: 8
- seed: 42
- distributed_type: multi-GPU
- num_devices: 2
- gradient_accumulation_steps: 8
- total_train_batch_size: 16
- total_eval_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- num_epochs: 3.0
- mixed_precision_training: Native AMP

### Training results



### Framework versions

- PEFT 0.11.1
- Transformers 4.41.2
- Pytorch 2.3.1+cu121
- Datasets 2.20.0
- Tokenizers 0.19.1