chore: import upstream snapshot with attribution
Continuous Integration / Pre-commit Linter (push) Has been cancelled
Continuous Integration / Mypy Check (Python 3.10) (push) Has been cancelled
Continuous Integration / Mypy Check (Python 3.11) (push) Has been cancelled
Continuous Integration / Mypy Check (Python 3.12) (push) Has been cancelled
Continuous Integration / Mypy Check (Python 3.13) (push) Has been cancelled
Continuous Integration / Unit Tests (Python 3.10) (push) Has been cancelled
Continuous Integration / Unit Tests (Python 3.11) (push) Has been cancelled
Continuous Integration / Unit Tests (Python 3.12) (push) Has been cancelled
Continuous Integration / Unit Tests (Python 3.13) (push) Has been cancelled
Continuous Integration / Unit Tests (Python 3.14) (push) Has been cancelled
Continuous Integration / A2A v0.3 Tests (Python 3.10) (push) Has been cancelled
Continuous Integration / A2A v0.3 Tests (Python 3.11) (push) Has been cancelled
Continuous Integration / A2A v0.3 Tests (Python 3.12) (push) Has been cancelled
Copybara PR Handler / close-imported-pr (push) Has been cancelled
Continuous Integration / A2A v0.3 Tests (Python 3.13) (push) Has been cancelled
Continuous Integration / A2A v0.3 Tests (Python 3.14) (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 13:25:13 +08:00
commit ec2b666284
2231 changed files with 491535 additions and 0 deletions
@@ -0,0 +1,78 @@
# Log Probabilities Demo Agent
## Overview
This sample demonstrates how to access and display log probabilities from language model responses using the `avg_logprobs` and `logprobs_result` fields in `LlmResponse`. It shows how to configure an ADK agent to request log probabilities and how to use an `after_model_callback` to analyze and append confidence metrics to the response.
## Sample Inputs
- `What is the capital of France?`
*A factual, straightforward question. The agent will answer confidently (e.g., "Paris"), resulting in a high average log probability and confidence score near 100%.*
- `What are the philosophical implications of artificial consciousness?`
*A complex, open-ended question. The agent will provide a nuanced answer with varied vocabulary, resulting in a lower average log probability and medium/low confidence score.*
## Graph
```mermaid
graph TD
User[User Input] --> RootAgent[root_agent: logprobs_demo_agent]
RootAgent --> Callback[after_model_callback: append_logprobs_to_response]
Callback -- Appended Logprobs Analysis --> Response[User Response]
```
## How To
### 1. Enabling Log Probabilities
To enable log probability collection, configure `generate_content_config` on the `Agent` using `types.GenerateContentConfig`:
```python
from google.genai import types
root_agent = Agent(
name="logprobs_demo_agent",
generate_content_config=types.GenerateContentConfig(
response_logprobs=True, # Enable log probability collection
logprobs=5, # Collect top 5 alternatives for analysis
temperature=0.7,
),
after_model_callback=append_logprobs_to_response,
)
```
### 2. Extracting Log Probabilities in a Callback
The `after_model_callback` receives the `LlmResponse` object, which contains the `avg_logprobs` and `logprobs_result` fields. You can use this data for confidence analysis, quality filtering, or appending information to the response content:
```python
async def append_logprobs_to_response(
callback_context: CallbackContext, llm_response: LlmResponse
) -> LlmResponse:
if llm_response.avg_logprobs is not None:
print(f"📊 Average log probability: {llm_response.avg_logprobs:.4f}")
# Analyze confidence
confidence_level = (
"High" if llm_response.avg_logprobs >= -0.5
else "Medium" if llm_response.avg_logprobs >= -1.0
else "Low"
)
# Access detailed candidates
if llm_response.logprobs_result and llm_response.logprobs_result.top_candidates:
num_candidates = len(llm_response.logprobs_result.top_candidates)
return llm_response
```
### 3. Understanding Log Probabilities
- **Range**: -∞ to 0 (0 = 100% confident, -1 ≈ 37% confident, -2 ≈ 14% confident)
- **Confidence Levels**:
- **High**: `>= -0.5` (typically factual, straightforward responses)
- **Medium**: `-1.0` to `-0.5` (reasonably confident responses)
- **Low**: `< -1.0` (uncertain or complex responses)
- **Use Cases**: Quality control, uncertainty detection, and response filtering.
@@ -0,0 +1,15 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from . import agent
+116
View File
@@ -0,0 +1,116 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Sample agent demonstrating log probability usage.
This agent shows how to access log probabilities from language model responses.
The after_model_callback appends confidence information to demonstrate how
logprobs can be extracted and used.
"""
from google.adk.agents.callback_context import CallbackContext
from google.adk.agents.llm_agent import Agent
from google.adk.models.llm_response import LlmResponse
from google.genai import types
async def append_logprobs_to_response(
callback_context: CallbackContext, llm_response: LlmResponse
) -> LlmResponse:
"""After-model callback that appends log probability information to response.
This callback demonstrates how to access avg_logprobs and logprobs_result
from the LlmResponse and append the information to the response content.
Args:
callback_context: The current callback context
llm_response: The LlmResponse containing logprobs data
Returns:
Modified LlmResponse with logprobs information appended
"""
# Build log probability analysis
if llm_response.avg_logprobs is None:
print("⚠️ No log probability data available")
logprobs_info = """
---
### 📊 Log Probability Analysis
⚠️ *No log probability data available*"""
else:
print(f"📊 Average log probability: {llm_response.avg_logprobs:.4f}")
# Build confidence analysis
confidence_level = (
"High"
if llm_response.avg_logprobs >= -0.5
else "Medium"
if llm_response.avg_logprobs >= -1.0
else "Low"
)
logprobs_info = f"""
---
### 📊 Log Probability Analysis
* **Average Log Probability**: {llm_response.avg_logprobs:.4f}
* **Confidence Level**: {confidence_level}
* **Confidence Score**: {100 * (2 ** llm_response.avg_logprobs):.1f}%"""
# Optionally include detailed logprobs_result information
if (
llm_response.logprobs_result
and llm_response.logprobs_result.top_candidates
):
logprobs_info += (
"\n* **Top Alternatives Analyzed**:"
f" {len(llm_response.logprobs_result.top_candidates)}"
)
# Append logprobs analysis to the response
if llm_response.content and llm_response.content.parts:
if not any(
"### 📊 Log Probability Analysis" in p.text
for p in llm_response.content.parts
if p.text
):
llm_response.content.parts.append(types.Part(text=logprobs_info))
return llm_response
# Create a simple agent that demonstrates logprobs usage
root_agent = Agent(
name="logprobs_demo_agent",
description=(
"A simple agent that demonstrates log probability extraction and"
" display."
),
instruction="""
You are a helpful AI assistant. Answer user questions normally and naturally.
After you respond, you'll see log probability analysis appended to your response.
You don't need to include the log probability analysis in your response yourself.
""",
generate_content_config=types.GenerateContentConfig(
response_logprobs=True, # Enable log probability collection
logprobs=5, # Collect top 5 alternatives for analysis
temperature=0.7, # Moderate temperature for varied responses
),
after_model_callback=append_logprobs_to_response,
)
@@ -0,0 +1,40 @@
{
"events": [
{
"author": "user",
"content": {
"parts": [
{
"text": "Hello"
}
],
"role": "user"
},
"id": "e-1",
"invocationId": "i-1",
"nodeInfo": {
"path": ""
}
},
{
"author": "logprobs_demo_agent",
"content": {
"parts": [
{
"text": "Hello there! How can I help you today?"
},
{
"text": "\n\n---\n\n### \ud83d\udcca Log Probability Analysis\n\n* **Average Log Probability**: -0.1564\n* **Confidence Level**: High\n* **Confidence Score**: 89.7%\n* **Top Alternatives Analyzed**: 10"
}
],
"role": "model"
},
"finishReason": "STOP",
"id": "e-2",
"invocationId": "i-1",
"nodeInfo": {
"path": "logprobs_demo_agent@1"
}
}
]
}
@@ -0,0 +1,40 @@
{
"events": [
{
"author": "user",
"content": {
"parts": [
{
"text": "Tell me a joke"
}
],
"role": "user"
},
"id": "e-1",
"invocationId": "i-1",
"nodeInfo": {
"path": ""
}
},
{
"author": "logprobs_demo_agent",
"content": {
"parts": [
{
"text": "Why don't scientists trust atoms?\n\nBecause they make up everything!"
},
{
"text": "\n\n---\n\n### \ud83d\udcca Log Probability Analysis\n\n* **Average Log Probability**: -0.4411\n* **Confidence Level**: High\n* **Confidence Score**: 73.7%\n* **Top Alternatives Analyzed**: 15"
}
],
"role": "model"
},
"finishReason": "STOP",
"id": "e-2",
"invocationId": "i-1",
"nodeInfo": {
"path": "logprobs_demo_agent@1"
}
}
]
}
@@ -0,0 +1,40 @@
{
"events": [
{
"author": "user",
"content": {
"parts": [
{
"text": "What is AI?"
}
],
"role": "user"
},
"id": "e-1",
"invocationId": "i-1",
"nodeInfo": {
"path": ""
}
},
{
"author": "logprobs_demo_agent",
"content": {
"parts": [
{
"text": "Artificial Intelligence (AI) refers to the simulation of human intelligence processes by machines, especially computer systems. These processes include learning (the acquisition of information and rules for using the information), reasoning (using rules to reach approximate or definite conclusions), and self-correction.\n\nThe ultimate goal of AI is to enable machines to perform tasks that typically require human intelligence, such as understanding natural language, recognizing patterns, solving problems, making decisions, and even learning from experience."
},
{
"text": "\n\n---\n\n### \ud83d\udcca Log Probability Analysis\n\n* **Average Log Probability**: -0.1382\n* **Confidence Level**: High\n* **Confidence Score**: 90.9%\n* **Top Alternatives Analyzed**: 92"
}
],
"role": "model"
},
"finishReason": "STOP",
"id": "e-2",
"invocationId": "i-1",
"nodeInfo": {
"path": "logprobs_demo_agent@1"
}
}
]
}