chore: import upstream snapshot with attribution
Continuous Integration / Pre-commit Linter (push) Has been cancelled
Continuous Integration / Mypy Check (Python 3.10) (push) Has been cancelled
Continuous Integration / Mypy Check (Python 3.11) (push) Has been cancelled
Continuous Integration / Mypy Check (Python 3.12) (push) Has been cancelled
Continuous Integration / Mypy Check (Python 3.13) (push) Has been cancelled
Continuous Integration / Unit Tests (Python 3.10) (push) Has been cancelled
Continuous Integration / Unit Tests (Python 3.11) (push) Has been cancelled
Continuous Integration / Unit Tests (Python 3.12) (push) Has been cancelled
Continuous Integration / Unit Tests (Python 3.13) (push) Has been cancelled
Continuous Integration / Unit Tests (Python 3.14) (push) Has been cancelled
Continuous Integration / A2A v0.3 Tests (Python 3.10) (push) Has been cancelled
Continuous Integration / A2A v0.3 Tests (Python 3.11) (push) Has been cancelled
Continuous Integration / A2A v0.3 Tests (Python 3.12) (push) Has been cancelled
Copybara PR Handler / close-imported-pr (push) Has been cancelled
Continuous Integration / A2A v0.3 Tests (Python 3.13) (push) Has been cancelled
Continuous Integration / A2A v0.3 Tests (Python 3.14) (push) Has been cancelled
Continuous Integration / Pre-commit Linter (push) Has been cancelled
Continuous Integration / Mypy Check (Python 3.10) (push) Has been cancelled
Continuous Integration / Mypy Check (Python 3.11) (push) Has been cancelled
Continuous Integration / Mypy Check (Python 3.12) (push) Has been cancelled
Continuous Integration / Mypy Check (Python 3.13) (push) Has been cancelled
Continuous Integration / Unit Tests (Python 3.10) (push) Has been cancelled
Continuous Integration / Unit Tests (Python 3.11) (push) Has been cancelled
Continuous Integration / Unit Tests (Python 3.12) (push) Has been cancelled
Continuous Integration / Unit Tests (Python 3.13) (push) Has been cancelled
Continuous Integration / Unit Tests (Python 3.14) (push) Has been cancelled
Continuous Integration / A2A v0.3 Tests (Python 3.10) (push) Has been cancelled
Continuous Integration / A2A v0.3 Tests (Python 3.11) (push) Has been cancelled
Continuous Integration / A2A v0.3 Tests (Python 3.12) (push) Has been cancelled
Copybara PR Handler / close-imported-pr (push) Has been cancelled
Continuous Integration / A2A v0.3 Tests (Python 3.13) (push) Has been cancelled
Continuous Integration / A2A v0.3 Tests (Python 3.14) (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,67 @@
|
||||
# Instructions
|
||||
|
||||
## Run Evaluation
|
||||
|
||||
1. Set environment variables in your terminal:
|
||||
|
||||
```shell
|
||||
export GOOGLE_GENAI_USE_ENTERPRISE=FALSE
|
||||
export GOOGLE_API_KEY=<your_api_key>
|
||||
export GOOGLE_CLOUD_PROJECT=<your_bigquery_project>
|
||||
```
|
||||
1. Change to the current directory:
|
||||
|
||||
```shell
|
||||
cd tests/integration/fixture/bigquery_agent/
|
||||
```
|
||||
1. Customize the evaluation dataset to the environment `GOOGLE_CLOUD_PROJECT`
|
||||
by replacing the placeholder to the real project set in your environment:
|
||||
|
||||
```shell
|
||||
sed -e "s:\${GOOGLE_CLOUD_PROJECT}:${GOOGLE_CLOUD_PROJECT}:g" simple.test.json -i
|
||||
```
|
||||
1. Run the following command as per https://google.github.io/adk-docs/evaluate/#3-adk-eval-run-evaluations-via-the-cli:
|
||||
|
||||
```shell
|
||||
adk eval . simple.test.json --config_file_path=test_config.json
|
||||
```
|
||||
|
||||
If it fails, re-run with `--print_detailed_results` flag to see more details
|
||||
on turn-by-turn evaluation.
|
||||
|
||||
## Generate Evaluation dataset
|
||||
|
||||
1. Set environment variables in your terminal:
|
||||
|
||||
```shell
|
||||
export GOOGLE_GENAI_USE_ENTERPRISE=FALSE
|
||||
export GOOGLE_API_KEY=<your_api_key>
|
||||
export GOOGLE_CLOUD_PROJECT=<your_bigquery_project>
|
||||
```
|
||||
1. Set up google [application default credentials](https://cloud.google.com/docs/authentication/provide-credentials-adc)
|
||||
on your machine.
|
||||
|
||||
```shell
|
||||
gcloud auth application-default login
|
||||
```
|
||||
1. Change to the directory containing agent folder:
|
||||
|
||||
```shell
|
||||
cd tests/integration/fixture/
|
||||
```
|
||||
1. Run the following command to start the ADK web app:
|
||||
|
||||
```shell
|
||||
adk web
|
||||
```
|
||||
1. Open the ADK web UI in your browser http://127.0.0.1:8000/dev-ui/?app=bigquery_agent.
|
||||
1. Create an evaluation dataset by following [these steps](https://google.github.io/adk-docs/evaluate/#1-adk-web-run-evaluations-via-the-web-ui).
|
||||
This would generate file `bigquery_agent/simple.evalset.json`.
|
||||
1. Note that this evaluation data would be tied to the agent interaction in the
|
||||
`GOOGLE_CLOUD_PROJECT` set in your environment. To normalize it by replacing
|
||||
the real project set in your environment to a placeholder, let's run the
|
||||
following command:
|
||||
|
||||
```shell
|
||||
sed -e "s:${GOOGLE_CLOUD_PROJECT}:\${GOOGLE_CLOUD_PROJECT}:g" bigquery_agent/simple.evalset.json > bigquery_agent/simple.test.json
|
||||
```
|
||||
Reference in New Issue
Block a user