{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": { "id": "EElURkc9VGpZ" }, "outputs": [], "source": [ "# Copyright 2025 Google LLC\n", "#\n", "# Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", "# You may obtain a copy of the License at\n", "#\n", "# https://www.apache.org/licenses/LICENSE-2.0\n", "#\n", "# Unless required by applicable law or agreed to in writing, software\n", "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# See the License for the specific language governing permissions and\n", "# limitations under the License." ] }, { "cell_type": "markdown", "metadata": { "id": "l7hzJLswRlgJ" }, "source": [ "# Migrating Foundation Models: A Practical Guide with Gen AI Evaluation Serivce\n" ] }, { "cell_type": "markdown", "metadata": { "id": "U_zRPTySRjHX" }, "source": [ " \n", " \n", " \n", " \n", " \n", "
\n", " \n", " \"Google
Open in Colab\n", " \n", " 		\n", " \n", " \"Google
Open in Colab Enterprise\n", " \n", " 		\n", " \n", " \"Vertex
Open in Vertex AI Workbench\n", " \n", " 		\n", " \n", " \"GitHub
View on GitHub\n", " \n", "
\n", "\n", "
\n", "\n", "Share to:\n", "\n", "\n", " \"LinkedIn\n", "\n", "\n", "\n", " \"Bluesky\n", "\n", "\n", "\n", " \"X\n", "\n", "\n", "\n", " \"Reddit\n", "\n", "\n", "\n", " \"Facebook\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "YK2R3uVZR1RD" }, "source": [ "| Author(s) |\n", "| --- |\n", "| [Jason Dai](https://github.com/jsondai) |" ] }, { "cell_type": "markdown", "metadata": { "id": "1ijw7-AyR344" }, "source": [ "## Overview\n", "\n", "This notebook demonstrates how to use the Vertex AI SDK for Gen AI Evaluation Service to compare two first-party models, for example, when considering a migration (e.g., `Gemini 2.0 Flash` to `Gemini 2.5 Flash`). We will use various predefined adaptive rubric-based metrics. Additionally, we'll touch upon how evaluation results can guide prompt optimization.\n", "\n", "---\n", "\n", "Key features highlighted in this notebook include:\n", "\n", "\n", "* **A Complete Evaluation Workflow**: The SDK provides a seamless experience from generating model responses with `run_inference()` to detailed assessment with `evaluate()`.\n", "\n", "* **Flexible, Multi-Candidate Evaluation**: Easily analyze and compare the performance of multiple AI models, agents, or configurations in a single run. The SDK provides a unified report with comprehensive results and win-rate calculations for all contenders.\n", "\n", "* **Rich In-Notebook Visualization**: The `.show()` method, available on both `EvaluationDataset` and `EvaluationResult` objects, renders an interactive HTML report for analysis directly within your Colab and Jupyter notebooks.\n", "\n", "* **Broad Model and Data Support**: Natively evaluate models from Google, OpenAI, and other providers supported by LiteLLM, and handle various data formats automatically.\n", "\n", "* **Asynchronous Batch-style Evaluation**: For large datasets, you can now use `batch_evaluate()` to run evaluations as a long-running operation, which is ideal for large-scale jobs.\n", "\n", "\n", "* **Integrated Prompt Optimization**: Iteratively improve your prompts using the built-in `prompt_optimizer` module and immediately re-evaluate to quantify the impact of your changes.\n" ] }, { "cell_type": "markdown", "metadata": { "id": "5E1mN9wWT-Ol" }, "source": [ "### Costs\n", "\n", "This tutorial uses billable components of Google Cloud:\n", "\n", "- Vertex AI\n", "\n", "Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing) and use the [Pricing Calculator](https://cloud.google.com/products/calculator/) to generate a cost estimate based on your projected usage.\n" ] }, { "cell_type": "markdown", "metadata": { "id": "9yt-EPacUD4j" }, "source": [ "## Getting Started\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "-zTjlxYMRAog" }, "outputs": [], "source": [ "# @title ### Install Vertex AI SDK for Gen AI Evaluation Service\n", "\n", "%pip install --upgrade \"google-cloud-aiplatform[evaluation]>=1.111.0\" --force-reinstall --quiet --no-warn-conflicts" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "cellView": "form", "id": "AWP0pW9hUOBR" }, "outputs": [], "source": [ "# @title ### Authenticate your notebook environment (Colab only)\n", "# @markdown If you're running this notebook on Google Colab, run the cell below to authenticate your environment.\n", "\n", "import sys\n", "\n", "if \"google.colab\" in sys.modules:\n", " from google.colab import auth\n", "\n", " auth.authenticate_user()" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "cellView": "form", "id": "LHpeZ2pWUQag" }, "outputs": [], "source": [ "# @title ### Set Google Cloud project information\n", "# @markdown To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n", "# @markdown Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment).\n", "\n", "# @markdown ---\n", "\n", "import os\n", "PROJECT_ID = \"\" # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n", "if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n", " PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n", "LOCATION= \"us-central1\" # @param {type: \"string\", placeholder: \"us-central1\", isTemplate: true}\n", "LOCATION = os.environ.get(\"GOOGLE_CLOUD_REGION\", LOCATION)\n", "\n", "\n", "from vertexai import Client, types\n", "client = Client(project=PROJECT_ID, location=LOCATION)" ] }, { "cell_type": "code", "execution_count": 45, "metadata": { "cellView": "form", "id": "lDZ6j_p3Vct9" }, "outputs": [], "source": [ "# @title ### Prepare Dataset and Generate Rubrics\n", "# @markdown Rubrics would be saved in a group named `general_quality_rubrics`.\n", "\n", "import pandas as pd\n", "\n", "prompts_df = pd.DataFrame({\n", " \"prompt\": [\n", " \"Explain the difference between 'emergent behavior' in AI systems and 'unintended consequences' in software engineering, with examples.\",\n", " \"Write a short story (under 150 words) that includes a paradox, irony, and an emotional twist—without explicitly naming any of them.\",\n", " \"Summarize the philosophical implications of Gödel’s incompleteness theorems for modern AI research in 3 clear bullet points.\",\n", " \"Given a fictional planet where time flows backward for 50% of its inhabitants, design a basic legal system that works for all citizens.\",\n", " \"Critically compare Nietzsche’s idea of 'eternal recurrence' with the concept of simulation theory in modern tech culture.\"\n", " ]\n", "})\n", "\n", "data_with_rubrics = client.evals.generate_rubrics(\n", " src=prompts_df,\n", " rubric_group_name=\"general_quality_rubrics\",\n", " predefined_spec_name=types.RubricMetric.GENERAL_QUALITY,\n", ")" ] }, { "cell_type": "code", "execution_count": 46, "metadata": { "cellView": "form", "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, "id": "XYH5UMDzdvFf", "outputId": "07789ea5-2be6-44bd-d968-160de37a97b1" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Generating responses for gemini-2.0-flash...\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Gemini Inference: 100%|██████████| 5/5 [00:12<00:00, 2.48s/it]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Generating responses for gemini-2.5-flash...\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Gemini Inference: 100%|██████████| 5/5 [00:27<00:00, 5.46s/it]\n" ] }, { "data": { "text/html": [ "\n", "\n", "\n", "\n", " \n", " Evaluation Dataset\n", " \n", " \n", " \n", "\n", "\n", "
\n", "

Evaluation Dataset

\n", "
\n", "
\n", " \n", "\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# @title ### Run Inference for Both Models\n", "\n", "# @markdown ---\n", "# @markdown **Models to Compare**\n", "MODEL_1_ID = \"gemini-2.0-flash\" # @param {type: \"string\"}\n", "MODEL_2_ID = \"gemini-2.5-flash\" # @param {type: \"string\"}\n", "# @markdown ---\n", "\n", "print(f\"Generating responses for {MODEL_1_ID}...\")\n", "candidate_1 = client.evals.run_inference(\n", " model=MODEL_1_ID,\n", " src=data_with_rubrics,\n", " config={\n", " \"generate_content_config\": {\"temperature\": 1.6}\n", " }\n", ")\n", "\n", "print(f\"Generating responses for {MODEL_2_ID}...\")\n", "candidate_2 = client.evals.run_inference(\n", " model=MODEL_2_ID,\n", " src=data_with_rubrics,\n", ")\n", "candidate_2.show()" ] }, { "cell_type": "code", "execution_count": 47, "metadata": { "cellView": "form", "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, "id": "55AVmz7yVpPF", "outputId": "148756fd-d5b2-40fb-daa7-c04576c4110a" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Computing Metrics for Evaluation Dataset: 100%|██████████| 10/10 [00:12<00:00, 1.25s/it]\n", "/usr/local/lib/python3.12/dist-packages/pydantic/main.py:463: UserWarning: Pydantic serializer warnings:\n", " PydanticSerializationUnexpectedValue(Expected `WinRateStats` - serialized value may not be as expected [input_value={'win_rates': [0.0, 0.0], 'tie_rate': 1.0}, input_type=dict])\n", " return self.__pydantic_serializer__.to_python(\n" ] }, { "data": { "text/html": [ "\n", "\n", "\n", "\n", " \n", " Eval Comparison Report\n", " \n", " \n", " \n", "\n", "\n", "
\n", "

Eval Comparison Report

\n", "
\n", "
\n", "
\n", " \n", "\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# @title ### Evaluate and Compare\n", "# @markdown Use a list of datasets to compare the candidates.\n", "\n", "comparison_result = client.evals.evaluate(\n", " dataset=[candidate_1, candidate_2],\n", " metrics=[\n", " types.RubricMetric.GENERAL_QUALITY(\n", " rubric_group_name=\"general_quality_rubrics\",\n", " ),\n", " ]\n", ")\n", "comparison_result.show()" ] }, { "cell_type": "markdown", "metadata": { "id": "zafdt1_uCIZ-" }, "source": [ "### Evaluate Third-Party Models (e.g., OpenAI)\n", "\n", "You can use the Gen AI evaluation service to evaluate and compare models from providers such as OpenAI by passing the model name string to the `run_inference` method. The Gen AI evaluation service uses the `litellm` library to call the model API.\n", "\n", "Make sure to set the required API key as an environment variable (such as `OPENAI_API_KEY`):" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, "id": "rXLwn-9oDrzt", "outputId": "e8a6f321-6662-4b4c-8c39-77e046e0d8e9" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "LiteLLM Inference (gpt-4o): 100%|██████████| 7/7 [00:02<00:00, 2.52it/s]\n" ] }, { "data": { "text/html": [ "\n", "\n", "\n", "\n", " \n", " Evaluation Dataset\n", " \n", " \n", " \n", "\n", "\n", "
\n", "

Evaluation Dataset

\n", "
\n", "
\n", " \n", "\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import json\n", "import pandas as pd\n", "\n", "# Make sure your OPENAI_API_KEY environment variable is set.\n", "os.environ['OPENAI_API_KEY'] = \"\" # @param {type:\"string\", placeholder: \"[your-openai-api-key]\"}\n", "# WARNING: Setting API keys directly in code is insecure. Use environment variables or secure storage.\n", "\n", "# Alternative, use your OPENAI_API_KEY from Colab Secrets manager\n", "from google.colab import userdata\n", "os.environ['OPENAI_API_KEY'] = userdata.get('OPENAI_API_KEY')\n", "\n", "\n", "openai_responses = client.evals.run_inference(\n", " model=\"gpt-4o\",\n", " src=\"gs://vertex-evaluation-llm-dataset-us-central1/genai_eval_sdk/test_prompts.jsonl\",\n", ")\n", "openai_responses.show()" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, "id": "EGEYPLGcDt0K", "outputId": "24bbb346-039b-4aa0-c7c7-4b556efbe3fc" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Computing Metrics for Evaluation Dataset: 100%|██████████| 28/28 [00:01<00:00, 26.27it/s]\n" ] }, { "data": { "text/html": [ "\n", "\n", "\n", "\n", " \n", " Evaluation Report\n", " \n", " \n", " \n", "\n", "\n", "
\n", "

Evaluation Report

\n", "
\n", "
\n", "
\n", " \n", "\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "eval_result = client.evals.evaluate(\n", " dataset=openai_responses,\n", " metrics=[\n", " types.RubricMetric.COHERENCE,\n", " types.RubricMetric.FLUENCY,\n", " types.Metric(name='rouge_1'),\n", " types.Metric(name='bleu'),\n", " ]\n", ")\n", "eval_result.show()" ] }, { "cell_type": "markdown", "metadata": { "id": "ta1KAP5ABY_h" }, "source": [ "### Asynchronous and large-scale evaluation\n", "\n", "For large datasets, the Gen AI evaluation service provides an asynchronous, long-running batch evaluation method. This is ideal for scenarios where you don't need immediate results and want to offload the computation.\n", "The `batch_evaluate()` method returns an operation object that you can poll to track its progress. The parameters are compatible with the `evaluate()` method." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "ytgBpbvlBiwD" }, "outputs": [], "source": [ "GCS_DEST_BUCKET = \"\" # @param {type:\"string\", placeholder: \"[your-gcs-bucket]\"}\n", "\n", "inference_result_saved = client.evals.run_inference(\n", " model=\"gemini-2.5-flash\",\n", " src=\"gs://vertex-evaluation-llm-dataset-us-central1/genai_eval_sdk/test_prompts.jsonl\",\n", " config={'dest': GCS_DEST_BUCKET}\n", ")\n", "print(f\"Eval Dataset uploaded to: {inference_result_saved.gcs_source}\")\n", "\n", "batch_eval_job = client.evals.batch_evaluate(\n", " dataset = inference_result_saved,\n", " metrics = [\n", " types.RubricMetric.FLUENCY,\n", " types.Metric(name='bleu'),\n", " ],\n", " dest=GCS_DEST_BUCKET\n", ")\n", "batch_eval_job" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "cellView": "form", "id": "1F0iAO3GBsmv" }, "outputs": [], "source": [ "# @title #### View results\n", "def gcs_path_to_console_url(gcs_path: str) -> str:\n", " if not gcs_path.startswith(\"gs://\"):\n", " raise ValueError(\"Invalid GCS path. Must start with 'gs://'\")\n", "\n", " # Remove the 'gs://' prefix\n", " bucket_and_path = gcs_path[5:]\n", "\n", " # Construct the console URL\n", " console_url = f\"https://console.cloud.google.com/storage/browser/{bucket_and_path}\"\n", " return console_url\n", "\n", "url = gcs_path_to_console_url(GCS_DEST_BUCKET)\n", "print(f\"Results will be written to your GCS destination path: {GCS_DEST_BUCKET}\\n\", url)" ] }, { "cell_type": "markdown", "metadata": { "id": "strmA_s5jWgm" }, "source": [ "### Prompt Optimization\n", "\n", "\n", "The Vertex AI SDK includes a `prompt_optimizer` module designed to enhance your prompts. The typical workflow involves utilizing the optimizer to generate improved prompts and then re-evaluating your models using these optimized prompts." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "4r2anB3tn6f-", "outputId": "48caf757-9210-42d0-f67d-4f59eacb4cc6" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The below texts are extracted from a page of bank statement with position tags at the end of each line. The position tags are in the format XX|YY. The origin 00|00 is at the top left of the page. For example, 10|20 is directly above 10|24, and 10|20 is directly to the left of 20|20.\n", "From the bank statement, extract the named entities in json format.\n", "{\"account_number\": [], \"account_type\": [], \"bank_address\": \"\", \"bank_name\": \"\", \"client_address\": \"\", \"client_name\": [], \"currency\": [], \"ending_balance\": [], \"page_number\": [], \"starting_balance\": [], \"statement_date\": \"\", \"statement_end_date\": \"\", \"statement_start_date\": \"\"}\n", "- The extraction must respect the JSON schema.\n", "- The values must only include text strings found in the document and their respective line tags.\n", "- The line tags appear at the end of each line. They are the line's X and Y location on the document page, in the format XX|YY. The origin 00|00 is at the top left of the page. For example, 10|20 is directly above 10|24, and 10|20 is directly to the left of 20|20.\n", "- Examples of valid string value format: \"$ 1234.50 40|12\", \"John Do 55|03\", \"Jane Johan 53|89\n", "Doe 54|91\", null.\n", "- Examples of invalid string value format: \"$ 1234.50\", \"John Do\".\n", "- Examples of valid list value format: [\"1 Imaginary St 34|61\n", "MA02140 35|62\", \"7 Heaven Rd 44|03\n", "NY10011 44|06\"], [].\n", "- Do not normalize any entity value.\n", "- Do not generate \"0\" or \"0.00\" for missing numerical values.\n", "\n" ] } ], "source": [ "prompt_template = \"The below texts are extracted from a page of bank statement with position tags at the end of each line. The position tags are in the format XX|YY. The origin 00|00 is at the top left of the page. For example, 10|20 is directly above 10|24, and 10|20 is directly to the left of 20|20.\\nFrom the bank statement, extract the named entities in json format.\\n{\\\"account_number\\\": [], \\\"account_type\\\": [], \\\"bank_address\\\": \\\"\\\", \\\"bank_name\\\": \\\"\\\", \\\"client_address\\\": \\\"\\\", \\\"client_name\\\": [], \\\"currency\\\": [], \\\"ending_balance\\\": [], \\\"page_number\\\": [], \\\"starting_balance\\\": [], \\\"statement_date\\\": \\\"\\\", \\\"statement_end_date\\\": \\\"\\\", \\\"statement_start_date\\\": \\\"\\\"}\\n- The extraction must respect the JSON schema.\\n- The values must only include text strings found in the document and their respective line tags.\\n- The line tags appear at the end of each line. They are the line's X and Y location on the document page, in the format XX|YY. The origin 00|00 is at the top left of the page. For example, 10|20 is directly above 10|24, and 10|20 is directly to the left of 20|20.\\n- Examples of valid string value format: \\\"$ 1234.50 40|12\\\", \\\"John Do 55|03\\\", \\\"Jane Johan 53|89\\nDoe 54|91\\\", null.\\n- Examples of invalid string value format: \\\"$ 1234.50\\\", \\\"John Do\\\".\\n- Examples of valid list value format: [\\\"1 Imaginary St 34|61\\nMA02140 35|62\\\", \\\"7 Heaven Rd 44|03\\nNY10011 44|06\\\"], [].\\n- Do not normalize any entity value.\\n- Do not generate \\\"0\\\" or \\\"0.00\\\" for missing numerical values.\\n\"\n", "print(prompt_template)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "cellView": "form", "id": "eRDunJHkpI4Q" }, "outputs": [], "source": [ "PROJECT_NUMBER = \"\" # @param {type: \"string\"}\n", "SERVICE_ACCOUNT = f\"{PROJECT_NUMBER}-compute@developer.gserviceaccount.com\"\n", "INPUT_GCS_SOURCE = \"gs://vertex-ai-generative-ai-eval-sdk-resources/test_data/prompt_optimization_input_data.jsonl\" # @param {type: \"string\"}\n", "GCS_DEST_BUCKET = \"\" # @param {type:\"string\", placeholder: \"[your-gcs-bucket]\"}\n", "\n", "\n", "import pandas as pd\n", "df = pd.read_json(INPUT_GCS_SOURCE, lines=True)\n", "df.head()\n", "\n", "vapo_config = {\n", " \"project\": PROJECT_ID,\n", " \"eval_metric\": \"bleu\",\n", " \"target_model\": \"gemini-1.5-flash-002\",\n", " \"target_model_qps\": 5,\n", " \"target_model_location\": LOCATION,\n", " \"optimizer_model\": \"gemini-1.5-pro-002\",\n", " \"optimizer_model_qps\": 3,\n", " \"optimizer_model_location\": LOCATION,\n", " \"optimization_mode\": \"instruction\",\n", " \"instruction_optimization_method\": \"reflect\",\n", " \"input_data_path\": INPUT_GCS_SOURCE,\n", " \"data_limit\": 3,\n", " \"output_path\": GCS_DEST_BUCKET,\n", " \"prompt_template\": prompt_template,\n", " \"demo_and_query_template\": \"Texts and Position Tags: {{document}}\\nEntities: {{target}}\",\n", " \"num_steps\": 2,\n", " \"num_template_eval_per_step\": 2,\n", " \"num_demo_set_candidates\": 10,\n", " \"demo_set_size\": 3,\n", " \"test_split_ratio\": 0.5,\n", " \"eval_qps\": 5\n", "}\n", "\n", "# Write the vapo config to output gcs path.\n", "import gcsfs\n", "import json\n", "gcs_file_system = gcsfs.GCSFileSystem(project=PROJECT_ID)\n", "vapo_config_json_path = f\"{GCS_DEST_BUCKET}/config.json\"\n", "with gcs_file_system.open(vapo_config_json_path, 'w') as f:\n", " json.dump(vapo_config, f)\n", "\n", "import logging\n", "logging.basicConfig(encoding='utf-8', level=logging.INFO, force=True)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "Tx84bcF2jhqN", "outputId": "89ff9bfc-0965-4c1d-da86-2a5092f74d81" }, "outputs": [], "source": [ "prompt_optimizer_result = client.prompt_optimizer.optimize(\n", " method=\"vapo\",\n", " config={\n", " \"config_path\": vapo_config_json_path,\n", " \"service_account\": SERVICE_ACCOUNT,\n", " \"wait_for_completion\": True\n", " }\n", ")" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "cellView": "form", "colab": { "base_uri": "https://localhost:8080/", "height": 241 }, "id": "uF2sevnWuJtQ", "outputId": "3c0292e4-b013-4449-bec1-960e981ead21" }, "outputs": [], "source": [ "# @title #### Results in output path\n", "\n", "eval_results_file_path = f\"gs://{GCS_DEST_BUCKET}/instruction/eval_results.json\"\n", "print(f\"Reading results from: {eval_results_file_path}\\n\")\n", "results_df = pd.read_json(eval_results_file_path)\n", "print(\"Successfully loaded data into a DataFrame:\")\n", "display(results_df.head())" ] } ], "metadata": { "colab": { "provenance": [], "toc_visible": true }, "kernelspec": { "display_name": "Python 3", "name": "python3" }, "language_info": { "name": "python" } }, "nbformat": 4, "nbformat_minor": 0 }