wehub-resource-sync
3fbbd7970c
Code Quality / Python Lint & Format (push) Has been cancelled
Code Quality / Python Tests (push) Has been cancelled
Code Quality / JavaScript/TypeScript Lint (advisory) (push) Has been cancelled
Security Scan / CodeQL Analysis (python) (push) Has been cancelled
Security Scan / Dependency Review (push) Has been cancelled
Security Scan / CodeQL Analysis (javascript-typescript) (push) Has been cancelled
388 lines
19 KiB
Plaintext
388 lines
19 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Fine Tuning Open AI Models\n",
|
|
"\n",
|
|
"This notebook is based on the current guidance provided in the [Fine Tuning](https://platform.openai.com/docs/guides/fine-tuning?WT.mc_id=academic-105485-koreyst) documentation from Open AI.\n",
|
|
"\n",
|
|
"> **Note:** This notebook's example output was generated against `gpt-3.5-turbo`, which is now retired for both inference and fine-tuning on Azure OpenAI in Microsoft Foundry (and deprecated by OpenAI directly). The walkthrough and concepts below are still accurate, but if you're starting a new fine-tuning job today, target a currently supported model instead - for example `gpt-4o-mini` or `gpt-4.1-mini`. See the [Fine-tuning models list](https://learn.microsoft.com/en-us/azure/ai-foundry/foundry-models/concepts/models-sold-directly-by-azure?WT.mc_id=academic-105485-koreyst#fine-tuning-models) for the current set of fine-tunable models.\n",
|
|
"\n",
|
|
"Fine-tuning improves the performance of foundation models for your application by retraining it with additional data and context relevant to that specific use case or scenario. Note that prompt engineering techniques like _few shot learning_ and _retrieval augmented generation_ allow you to enhance the default prompt with relevant data to improve quality. However, these approaches are limited by max token window size of the targeted foundation model. \n",
|
|
"\n",
|
|
"With fine-tuning, we are effectively retraining the model itself with the required data (allowing us to use many more examples than can fit in the max token window) - and deploying a _custom_ version of the model that no longer needs to have examples provided at inference time. This not only improves the effectivenenss of our prompt design (we have more flexibility in using the token window for other things) but potentially also improves our costs (by reducing the number of tokens we need to send to the model at inference time).\n",
|
|
"\n",
|
|
"Fine tuning has 4 steps:\n",
|
|
"1. Prepare the training data and upload it.\n",
|
|
"1. Run the training job to get a fine-tuned model.\n",
|
|
"1. Evaluate the fine-tuned model and iterate for quality.\n",
|
|
"1. Deploy the fine-tuned model for inference when satisfied.\n",
|
|
"\n",
|
|
"Note that not all foundation models support fine-tuning - [check OpenAI documentation](https://platform.openai.com/docs/guides/fine-tuning/what-models-can-be-fine-tuned?WT.mc_id=academic-105485-koreyst) for the latest information. You can also fine-tune a previously fine-tuned model. In this tutorial, we'll use `gpt-35-turbo` as our target foundation model for fine-tuning (see the note above for a current, supported alternative). \n",
|
|
"\n",
|
|
"---\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Step 1.1: Prepare Your Dataset\n",
|
|
"\n",
|
|
"Let's build a chatbot that helps you understand the periodic table of elements by answering questions about an element with a limerick. In _this_ simple tutorial, we'll just create a dataset to train the model with a few sample examples of responses that show the expected format of the data. In a real-world use case, you would need to create a dataset with many more examples. You may also be able to use an open dataset (for your application domain) if one exists, and reformat it for use in fine-tuning.\n",
|
|
"\n",
|
|
"Since we are focusing on `gpt-35-turbo` and looking for a single-turn response (chat completion) we can create examples using [this suggested format](https://platform.openai.com/docs/guides/fine-tuning/preparing-your-dataset?WT.mc_id=academic-105485-koreyst) reflecting the OpenAI chat completion requirements. If you expect multi-turn conversational content, you would use the [multi-turn example format](https://platform.openai.com/docs/guides/fine-tuning/multi-turn-chat-examples?WT.mc_id=academic-105485-koreyst) which includes a `weight` parameter to signal which messages should be used (or not) in the fine-tuning process.\n",
|
|
"\n",
|
|
"We will use the simpler single-turn format for our tutorial here. The data is in the [jsonl format](https://jsonlines.org/?WT.mc_id=academic-105485-koreyst) with 1 record per line, each represented as a JSON-formatted object. The snippet below shows 2 records as a sample - see [training-data.jsonl](training-data.jsonl) for full sample set (10 examples) we'll use for our fine-training tutorial. **Note:** Each record _must_ be defined in a single line (not split across lines as is typical in a formatted JSON file)\n",
|
|
"\n",
|
|
"```json\n",
|
|
"{ \"messages\": [{\"role\": \"system\", \"content\": \"Elle is a factual chatbot that answers questions about elements in the periodic table with a limerick\"}, {\"role\": \"user\", \"content\": \"Tell me about Gallium\"}, {\"role\": \"assistant\", \"content\": \"Gallium, oh gallium, so light - Melts in your hand, oh what a sight - At 86 degrees - Its liquid with ease - And in semiconductors, it's out of sight\"}]}\n",
|
|
"{ \"messages\": [{\"role\": \"system\", \"content\": \"Elle is a factual chatbot that answers questions about elements in the periodic table with a limerick\"}, {\"role\": \"user\", \"content\": \"Tell me about Hydrogen\"}, {\"role\": \"assistant\", \"content\": \"Hydrogen, the first in the line - The lightest of all, so divine - It's in water, you see - And in stars, it's the key - The universe's most common sign\"}]}\n",
|
|
"```\n",
|
|
"\n",
|
|
"In a real-world use case you will need a much larger examples set for good results - the tradeoff will be between quality of responses and the time/costs for fine-tuning. We are using a small set so we can complete fine-tuning quickly to illustrate the process. See [this OpenAI Cookbook example](https://github.com/openai/openai-cookbook/blob/main/examples/How_to_finetune_chat_models.ipynb?WT.mc_id=academic-105485-koreyst) for a more complex fine-tuning tutorial."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"---\n",
|
|
"\n",
|
|
"### Step 1.2 Upload Your Dataset\n",
|
|
"\n",
|
|
"Upload the data using the Files API [as described here](https://platform.openai.com/docs/guides/fine-tuning/upload-a-training-file). Note that in order to run this code, you must have done the following steps first:\n",
|
|
" - Installed the `openai` Python package (make sure you use a version >=0.28.0 for latest features)\n",
|
|
" - Set the `OPENAI_API_KEY` environment variable to your OpenAI API key\n",
|
|
"To learn more, see the [Setup guide](./../../../00-course-setup/02-setup-local.md?WT.mc_id=academic-105485-koreyst) provided for the course.\n",
|
|
"\n",
|
|
"Now, run the code to create a file for upload from your local JSONL file."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 24,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"FileObject(id='file-JdAJcagdOTG6ACNlFWzuzmyV', bytes=4021, created_at=1715566183, filename='training-data.jsonl', object='file', purpose='fine-tune', status='processed', status_details=None)\n",
|
|
"Training File ID: file-JdAJcagdOTG6ACNlFWzuzmyV\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from openai import OpenAI\n",
|
|
"client = OpenAI()\n",
|
|
"\n",
|
|
"ft_file = client.files.create(\n",
|
|
" file=open(\"./training-data.jsonl\", \"rb\"),\n",
|
|
" purpose=\"fine-tune\"\n",
|
|
")\n",
|
|
"\n",
|
|
"print(ft_file)\n",
|
|
"print(\"Training File ID: \" + ft_file.id)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"---\n",
|
|
"\n",
|
|
"### Step 2.1: Create the Fine-tuning job with the SDK"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 25,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"FineTuningJob(id='ftjob-Usfb9RjasncaZ5Cjbuh1XSCh', created_at=1715566184, error=Error(code=None, message=None, param=None), fine_tuned_model=None, finished_at=None, hyperparameters=Hyperparameters(n_epochs='auto', batch_size='auto', learning_rate_multiplier='auto'), model='gpt-3.5-turbo-0125', object='fine_tuning.job', organization_id='org-EZ6ag0n0S6Zm8eV9BSWKmE6l', result_files=[], seed=830529052, status='validating_files', trained_tokens=None, training_file='file-JdAJcagdOTG6ACNlFWzuzmyV', validation_file=None, estimated_finish=None, integrations=[], user_provided_suffix=None)\n",
|
|
"Fine-tuning Job ID: ftjob-Usfb9RjasncaZ5Cjbuh1XSCh\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from openai import OpenAI\n",
|
|
"client = OpenAI()\n",
|
|
"\n",
|
|
"ft_filejob = client.fine_tuning.jobs.create(\n",
|
|
" training_file=ft_file.id, \n",
|
|
" model=\"gpt-3.5-turbo\"\n",
|
|
")\n",
|
|
"\n",
|
|
"print(ft_filejob)\n",
|
|
"print(\"Fine-tuning Job ID: \" + ft_filejob.id)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"---\n",
|
|
"\n",
|
|
"### Step 2.2: Check the Status of the job\n",
|
|
"\n",
|
|
"Here are a few things you can do with the `client.fine_tuning.jobs` API:\n",
|
|
"- `client.fine_tuning.jobs.list(limit=<n>)` - List the last n fine-tuning jobs\n",
|
|
"- `client.fine_tuning.jobs.retrieve(<job_id>)` - Get details of a specific fine-tuning job\n",
|
|
"- `client.fine_tuning.jobs.cancel(<job_id>)` - Cancel a fine-tuning job\n",
|
|
"- `client.fine_tuning.jobs.list_events(fine_tuning_job_id=<job_id>, limit=<b>)` - List up to n events from the job\n",
|
|
"- `client.fine_tuning.jobs.create(model=\"gpt-35-turbo\", training_file=\"your-training-file.jsonl\", ...)`\n",
|
|
"\n",
|
|
"The first step of the process is _validating the training file_ to make sure data is in the right format."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 26,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"SyncCursorPage[FineTuningJobEvent](data=[FineTuningJobEvent(id='ftevent-GkWiDgZmOsuv4q5cSTEGscY6', created_at=1715566184, level='info', message='Validating training file: file-JdAJcagdOTG6ACNlFWzuzmyV', object='fine_tuning.job.event', data={}, type='message'), FineTuningJobEvent(id='ftevent-3899xdVTO3LN7Q7LkKLMJUnb', created_at=1715566184, level='info', message='Created fine-tuning job: ftjob-Usfb9RjasncaZ5Cjbuh1XSCh', object='fine_tuning.job.event', data={}, type='message')], object='list', has_more=False)"
|
|
]
|
|
},
|
|
"execution_count": 26,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"from openai import OpenAI\n",
|
|
"client = OpenAI()\n",
|
|
"\n",
|
|
"# List 10 fine-tuning jobs\n",
|
|
"client.fine_tuning.jobs.list(limit=10)\n",
|
|
"\n",
|
|
"# Retrieve the state of a fine-tune\n",
|
|
"client.fine_tuning.jobs.retrieve(ft_filejob.id)\n",
|
|
"\n",
|
|
"# List up to 10 events from a fine-tuning job\n",
|
|
"client.fine_tuning.jobs.list_events(fine_tuning_job_id=ft_filejob.id, limit=10)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 30,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Job ID: ftjob-Usfb9RjasncaZ5Cjbuh1XSCh\n",
|
|
"Status: running\n",
|
|
"Trained Tokens: None\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# Once the training data is validated\n",
|
|
"# Track the job status to see if it is running and when it is complete\n",
|
|
"from openai import OpenAI\n",
|
|
"client = OpenAI()\n",
|
|
"\n",
|
|
"response = client.fine_tuning.jobs.retrieve(ft_filejob.id)\n",
|
|
"\n",
|
|
"print(\"Job ID:\", response.id)\n",
|
|
"print(\"Status:\", response.status)\n",
|
|
"print(\"Trained Tokens:\", response.trained_tokens)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"---\n",
|
|
"\n",
|
|
"### Step 2.3: Track events to monitor progress\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 44,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Step 85/100: training loss=0.14\n",
|
|
"Step 86/100: training loss=0.00\n",
|
|
"Step 87/100: training loss=0.00\n",
|
|
"Step 88/100: training loss=0.07\n",
|
|
"Step 89/100: training loss=0.00\n",
|
|
"Step 90/100: training loss=0.00\n",
|
|
"Step 91/100: training loss=0.00\n",
|
|
"Step 92/100: training loss=0.00\n",
|
|
"Step 93/100: training loss=0.00\n",
|
|
"Step 94/100: training loss=0.00\n",
|
|
"Step 95/100: training loss=0.08\n",
|
|
"Step 96/100: training loss=0.05\n",
|
|
"Step 97/100: training loss=0.00\n",
|
|
"Step 98/100: training loss=0.00\n",
|
|
"Step 99/100: training loss=0.00\n",
|
|
"Step 100/100: training loss=0.00\n",
|
|
"Checkpoint created at step 80 with Snapshot ID: ft:gpt-3.5-turbo-0125:bitnbot::9OFWyyF2:ckpt-step-80\n",
|
|
"Checkpoint created at step 90 with Snapshot ID: ft:gpt-3.5-turbo-0125:bitnbot::9OFWyzhK:ckpt-step-90\n",
|
|
"New fine-tuned model created: ft:gpt-3.5-turbo-0125:bitnbot::9OFWzNjz\n",
|
|
"The job has successfully completed\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# You can also track progress in a more granular way by checking for events\n",
|
|
"# Refresh this code till you get the `The job has successfully completed` message\n",
|
|
"response = client.fine_tuning.jobs.list_events(ft_filejob.id)\n",
|
|
"\n",
|
|
"events = response.data\n",
|
|
"events.reverse()\n",
|
|
"\n",
|
|
"for event in events:\n",
|
|
" print(event.message)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Step 2.4: View status in OpenAI Dashboard"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"You can also view the status by visiting the OpenAI website and exploring the _Fine-tuning_ section of the platform. This will show you the status of the current job, and also let you track the history of prior job execution runs. In this screenshot, you can see that the prior execution failed, and the second run succeeded. For context, this happened when the first run used a JSON file with incorrectly formatted records - once fixed, the second run completed successfully and made the model available for use.\n",
|
|
"\n",
|
|
""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"You can also view the status messages and metrics by scrolling down further in the visual dashboard as shown:\n",
|
|
"\n",
|
|
"| Messages | Metrics |\n",
|
|
"|:---|:---|\n",
|
|
"|  | |\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"---\n",
|
|
"\n",
|
|
"### Step 3.1: Retrieve ID & Test Fine-Tuned Model in Code\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 46,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Fine-tuned Model ID: ft:gpt-3.5-turbo-0125:bitnbot::9OFWzNjz\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# Retrieve the identity of the fine-tuned model once ready\n",
|
|
"response = client.fine_tuning.jobs.retrieve(ft_filejob.id)\n",
|
|
"fine_tuned_model_id = response.fine_tuned_model\n",
|
|
"print(\"Fine-tuned Model ID:\", fine_tuned_model_id)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"ChatCompletionMessage(content=\"Strontium, a metal so bright - It's in fireworks, a dazzling sight - It's in bones, you see - And in tea, it's the key - It's the fortieth, so pure, that's the right\", role='assistant', function_call=None, tool_calls=None)\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# You can then use that model to generate completions from the SDK as shown\n",
|
|
"# Or you can load that model into the OpenAI Playground (in the UI) to validate it from there.\n",
|
|
"from openai import OpenAI\n",
|
|
"client = OpenAI()\n",
|
|
"\n",
|
|
"completion = client.responses.create(\n",
|
|
" model=fine_tuned_model_id,\n",
|
|
" input=[\n",
|
|
" {\"role\": \"system\", \"content\": \"You are Elle, a factual chatbot that answers questions about elements in the periodic table with a limerick\"},\n",
|
|
" {\"role\": \"user\", \"content\": \"Tell me about Strontium\"},\n",
|
|
" ],\n",
|
|
" store=False,\n",
|
|
")\n",
|
|
"print(completion.output_text)\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"---\n",
|
|
"\n",
|
|
"### Step 3.2: Load & Test Fine-Tuned Model in Playground\n",
|
|
"\n",
|
|
"You can now test the fine-tuned model in two ways. First, you can visit the Playground and use the Models drop-down to select your newly fine-tuned model from the options listed. The other option is to use the \"Playground\" option shown in the Fine-tuning panel (see screenshot above) which launches the following _comparitive_ view which shows the foundation and fine-tuned model versions side-by-side for quick evaluation.\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"Simply fill in the system context used in your training data and provide your test question. You will notice that both sides are updated with the identical context and question. Run the comparison and you will see the difference in outputs between them. _Note how the fine-tuned model renders the response in the format you provided in your examples while the foundation model simply follows the system prompt_.\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"You will notice that that the comparison also provides the token counts for each model, and the time taken for the inference. **This specific example is a simplistic one meant to show the process but not actually reflecting a real world dataset or scenario**. You may notice that both samples show the same number of tokens (system context and user prompt are identical) with the fine-tuned model taking more time for inference (custom model).\n",
|
|
"\n",
|
|
"In real-world scenarios, you will not be using a toy example like this, but fine-tuning against real data (e.g., product catalog for customer service) where the quality of response will be much more apparent. In _that_ context, getting an equivalent response quality with the foundation model will require more custom prompt engineering which will increase token usage and potentially the related processing time for inference. _To try this out, check out the fine-tuning examples in the OpenAI Cookbook to start._\n",
|
|
"\n",
|
|
"---"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.10.13"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|