wehub-resource-sync
53 lines
1.5 KiB
Markdown
53 lines
1.5 KiB
Markdown
### Requirements
|
|
You can install the required packages with the following command:
|
|
```bash
|
|
cd latex2sympy
|
|
pip install -e .
|
|
cd ..
|
|
pip install -r requirements.txt
|
|
pip install vllm==0.5.1 --no-build-isolation
|
|
pip install transformers==4.42.3
|
|
```
|
|
|
|
### Evaluation
|
|
You can evaluate Qwen2.5/Qwen2-Math-Instruct series model with the following command:
|
|
```bash
|
|
# Qwen2.5-Math-Instruct Series
|
|
PROMPT_TYPE="qwen25-math-cot"
|
|
# Qwen2.5-Math-1.5B-Instruct
|
|
export CUDA_VISIBLE_DEVICES="0"
|
|
MODEL_NAME_OR_PATH="Qwen/Qwen2.5-Math-1.5B-Instruct"
|
|
bash sh/eval.sh $PROMPT_TYPE $MODEL_NAME_OR_PATH
|
|
|
|
# Qwen2.5-Math-7B-Instruct
|
|
export CUDA_VISIBLE_DEVICES="0"
|
|
MODEL_NAME_OR_PATH="Qwen/Qwen2.5-Math-7B-Instruct"
|
|
bash sh/eval.sh $PROMPT_TYPE $MODEL_NAME_OR_PATH
|
|
|
|
# Qwen2.5-Math-72B-Instruct
|
|
export CUDA_VISIBLE_DEVICES="0,1,2,3"
|
|
MODEL_NAME_OR_PATH="Qwen/Qwen2.5-Math-72B-Instruct"
|
|
bash sh/eval.sh $PROMPT_TYPE $MODEL_NAME_OR_PATH
|
|
|
|
|
|
# Qwen2-Math-Instruct Series
|
|
PROMPT_TYPE="qwen-boxed"
|
|
# Qwen2-Math-1.5B-Instruct
|
|
export CUDA_VISIBLE_DEVICES="0"
|
|
MODEL_NAME_OR_PATH="Qwen/Qwen2-Math-1.5B-Instruct"
|
|
bash sh/eval.sh $PROMPT_TYPE $MODEL_NAME_OR_PATH
|
|
|
|
# Qwen2-Math-7B-Instruct
|
|
export CUDA_VISIBLE_DEVICES="0"
|
|
MODEL_NAME_OR_PATH="Qwen/Qwen2-Math-7B-Instruct"
|
|
bash sh/eval.sh $PROMPT_TYPE $MODEL_NAME_OR_PATH
|
|
|
|
# Qwen2-Math-72B-Instruct
|
|
export CUDA_VISIBLE_DEVICES="0,1,2,3"
|
|
MODEL_NAME_OR_PATH="Qwen/Qwen2-Math-72B-Instruct"
|
|
bash sh/eval.sh $PROMPT_TYPE $MODEL_NAME_OR_PATH
|
|
```
|
|
|
|
## Acknowledgement
|
|
The codebase is adapted from [math-evaluation-harness](https://github.com/ZubinGou/math-evaluation-harness).
|