GSM8K (Grade School Math 8K) is a dataset of 8.5K high quality, linguistically diverse grade school math word problems. The dataset was created to support the task of question answering on basic mathematical problems that require multi-step reasoning. The following steps are performed in this flow:

Read the .txt file of few-shot examples.
Use LLMLingua prompt compression tool to compress the GSM8K few-shot examples.
Test the LLM by using the compressed few-shot examples as context to determine if the answers are correct.

See the llmlingua-promptflow tool package reference documentation for further information.

Tools used in this flow:

python tool.
LLMLingua Prompt Compression Tool from the llmlingua-promptflow package.
prompt tool.
LLM tool.

Connections used in this flow:

Custom connection.
AzureOpenAI connection.

Prerequisites

Prompt flow SDK:

Install promptflow sdk and other dependencies:

pip install -r requirements.txt

Note: when using the Prompt flow SDK, it may be useful to also install the Prompt flow for VS Code extension (if using VS Code).

Azure AI/ML Studio:

Start an compute session. Required packages will automatically be installed from the requirements.txt file.

Setup connections

Custom connection

Create a connection to a MaaS resource for calculating log probability in Azure model catalog. You can use Llama, gpt-2, or other language models.

Take the Llama model as an example, you can learn how to deploy and consume Meta Llama models with model as a service by the guidance for Azure AI Studio or the guidance for Azure Machine Learning Studio .

The required keys to set are:

api_url
- This value can be found at the previously created inferencing endpoint.
api_key
- Ensure to set this as a secret value.
- This value can be found at the previously created inferencing endpoint.

Create a Custom connection with api_url and api_key.

AzureOpenAI connection

To use the LLM tool, you must have an Azure OpenAI Service Resource. Create one if necessary. From your Azure OpenAI Service Resource, obtain its api_key and endpoint.

Create a connection to your Azure OpenAI Service Resource.

Run flow

Prompt flow SDK:

# Test with default input values in flow.dag.yaml:
pf flow test --flow .

Azure AI/ML Studio:

Run flow.

Contact

Please reach out to LLMLingua Team (llmlingua@microsoft.com) with any issues.