| ---
|
| license: cc-by-sa-4.0
|
| ---
|
|
|
| # Compare-Answer Model
|
|
|
| Welcome to the repository for the Compare-Answer Model, an innovative tool designed to enhance the accuracy and efficiency of mathematical answer comparison tasks. This model leverages advanced techniques to provide precise comparisons across a wide range of mathematical problems.
|
|
|
| ## Features
|
|
|
| - **High Accuracy**: Utilizes state-of-the-art technology to ensure high reliability in answer comparison.
|
| - **Broad Compatibility**: Supports a variety of mathematical problem types and formats.
|
| - **Easy Integration**: Designed for easy integration with existing systems and workflows.
|
|
|
| ## Installation
|
|
|
| To get started with the Compare-Answer Model, clone this repository and load model with Transformers.
|
|
|
| # Quick Start
|
| To use the model, import it and call the main comparison function with the required parameters:
|
| ```python
|
| from transformers import AutoModelForCausalLM, AutoTokenizer
|
| device = "cuda" # the device to load the model onto
|
| model = AutoModelForCausalLM.from_pretrained(
|
| model_path, torch_dtype="auto", device_map="auto"
|
| )
|
| tokenizer = AutoTokenizer.from_pretrained(model_path)
|
|
|
| def build_user_query(question, pred_answer, answer, base_prompt):
|
| input_text = base_prompt.replace("{{question}}", question)
|
| input_text = input_text.replace("{{pred_step}}", pred_answer)
|
| input_text = input_text.replace("{{answer}}", answer)
|
| input_text = input_text.replace("{{analysis}}", "") # default set analysis to blank, if exist, you can pass in the corresponding parameter.
|
| return input_text
|
|
|
| chat_prompt = """<|im_start|>system
|
| You are a helpful assistant.<|im_end|>
|
| <|im_start|>human
|
| {}<|im_end|>
|
| <|im_start|>gpt
|
| """
|
|
|
| basic_prompt = """## 任务描述\n\n你是一个数学老师，学生提交了题目的解题步骤，你需要参考`题干`，`解析`和`答案`，判断`学生解题步骤`的结果是否正确。忽略`学生解题步骤`中的错误，只关注最后的答案。答案可能出现在`解析`中，也可能出现在`答案`中。\n\n## 输入内容\n\n题干:\n\n```\n{{question}}\n```\n\n解析:\n\n```\n{{analysis}}\n\n```\n\n答案:\n\n```\n{{answer}}\n```\n\n学生解题步骤:\n\n```\n{{pred_step}}\n```\n\n输出:"""
|
| base_prompt = chat_prompt.format(basic_prompt)
|
|
|
| def build_user_query(question, pred_answer, answer, base_prompt):
|
| input_text = base_prompt.replace("{{question}}", question)
|
| input_text = input_text.replace("{{pred_step}}", pred_answer)
|
| input_text = input_text.replace("{{answer}}", answer)
|
| input_text = input_text.replace("{{analysis}}", "") # default set analysis to blank, if exist, you can pass in the corresponding parameter.
|
| return input_text
|
|
|
| prompt = build_user_query("1+1=", "3", "2", base_prompt)
|
|
|
| model_inputs = tokenizer([prompt], return_tensors="pt").to(device)
|
|
|
| generated_ids = model.generate(model_inputs.input_ids, temperature=0, max_new_tokens=16, eos_token_id=100005)
|
| generated_ids = [
|
| output_ids[len(input_ids) :]
|
| for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
|
| ]
|
|
|
| response = tokenizer.batch_decode(generated_ids, skip_special_tokens=False)[0]
|
| ```
|
|
|
| ## Documentation
|
| For more detailed information about the model's API and functionalities, please contact us.
|
|
|
| # Contributing
|
| Contributions to the Compare-Answer Model are welcome! If you have suggestions or improvements, please fork the repository and submit a pull request.
|
|
|
| # License
|
| This project is licensed under the MIT License - see the LICENSE.md file for details.
|
|
|
| # Acknowledgements
|
| Thanks to all contributors who have helped in developing this model.
|
| Special thanks to MathEval for providing the datasets and challenges that inspired this project.
|
|
|
| # Contact
|
| For any inquiries, please reach out via email at liutianqiao1@tal.com or open an issue in this repository.
|
|
|
| Thank you for using or contributing to the Compare-Answer Model! |
