chore: import upstream snapshot with attribution
Test Browser Use CLI Install / uv pip install (ubuntu-latest) (push) Failing after 1s
Test Browser Use CLI Install / uvx browser-use from local wheel (push) Failing after 1s
Test Browser Use CLI Install / uvx browser-use[cli] from PyPI (push) Failing after 1s
package / pip-install-on-macos-latest-py-3.11 (push) Has been skipped
package / pip-install-on-macos-latest-py-3.13 (push) Has been skipped
package / pip-install-on-ubuntu-latest-py-3.11 (push) Has been skipped
package / pip-install-on-windows-latest-py-3.13 (push) Has been skipped
cloud_evals / trigger_cloud_eval_image_build (push) Failing after 1s
docker / build_publish_image (push) Failing after 1s
Test Browser Use CLI Install / browser-use skill sync (push) Failing after 1s
lint / code-style (push) Failing after 0s
lint / type-checker (push) Failing after 1s
package / pip-build (push) Failing after 1s
lint / syntax-errors (push) Failing after 3s
package / pip-install-on-ubuntu-latest-py-3.13 (push) Has been skipped
package / pip-install-on-windows-latest-py-3.11 (push) Has been skipped
test / ${{ matrix.test_filename }} (push) Has been skipped
test / evaluate-tasks (push) Has been skipped
test / setup-chromium (push) Failing after 2s
test / find_tests (push) Failing after 2s
Test Browser Use CLI Install / uv pip install (windows-latest) (push) Has been cancelled
Test Browser Use CLI Install / uv pip install (macos-latest) (push) Has been cancelled
Test Browser Use CLI Install / uv pip install (ubuntu-latest) (push) Failing after 1s
Test Browser Use CLI Install / uvx browser-use from local wheel (push) Failing after 1s
Test Browser Use CLI Install / uvx browser-use[cli] from PyPI (push) Failing after 1s
package / pip-install-on-macos-latest-py-3.11 (push) Has been skipped
package / pip-install-on-macos-latest-py-3.13 (push) Has been skipped
package / pip-install-on-ubuntu-latest-py-3.11 (push) Has been skipped
package / pip-install-on-windows-latest-py-3.13 (push) Has been skipped
cloud_evals / trigger_cloud_eval_image_build (push) Failing after 1s
docker / build_publish_image (push) Failing after 1s
Test Browser Use CLI Install / browser-use skill sync (push) Failing after 1s
lint / code-style (push) Failing after 0s
lint / type-checker (push) Failing after 1s
package / pip-build (push) Failing after 1s
lint / syntax-errors (push) Failing after 3s
package / pip-install-on-ubuntu-latest-py-3.13 (push) Has been skipped
package / pip-install-on-windows-latest-py-3.11 (push) Has been skipped
test / ${{ matrix.test_filename }} (push) Has been skipped
test / evaluate-tasks (push) Has been skipped
test / setup-chromium (push) Failing after 2s
test / find_tests (push) Failing after 2s
Test Browser Use CLI Install / uv pip install (windows-latest) (push) Has been cancelled
Test Browser Use CLI Install / uv pip install (macos-latest) (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,33 @@
|
||||
# Contributing Agent Tasks
|
||||
|
||||
Contribute your own agent tasks and we test if the agent solves them for CI testing!
|
||||
|
||||
## How to Add a Task
|
||||
|
||||
1. Create a new `.yaml` file in this directory (`tests/agent_tasks/`).
|
||||
2. Use the following format:
|
||||
|
||||
```yaml
|
||||
name: My Task Name
|
||||
task: Describe the task for the agent to perform
|
||||
judge_context:
|
||||
- List criteria for success, one per line
|
||||
max_steps: 10
|
||||
```
|
||||
|
||||
## Guidelines
|
||||
- Be specific in your task and criteria.
|
||||
- The `judge_context` should list what counts as a successful result.
|
||||
- The agent's output will be judged by an LLM using these criteria.
|
||||
|
||||
## Running the Tests
|
||||
|
||||
To run all agent tasks:
|
||||
|
||||
```bash
|
||||
pytest tests/ci/test_agent_real_tasks.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
Happy contributing!
|
||||
@@ -0,0 +1,7 @@
|
||||
name: Amazon Laptop Search
|
||||
task: Go to amazon.com, search for 'laptop', and return the first result
|
||||
judge_context:
|
||||
- The agent must navigate to amazon.com
|
||||
- The agent must search for 'laptop'
|
||||
- The agent must return name of the first laptop
|
||||
max_steps: 10
|
||||
@@ -0,0 +1,5 @@
|
||||
name: Find pip install command for browser-use
|
||||
task: Find the pip installation command for the browser-use repo
|
||||
judge_context:
|
||||
- The output must include the command ('pip install browser-use')
|
||||
max_steps: 10
|
||||
@@ -0,0 +1,21 @@
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Same-Origin Iframe</title>
|
||||
</head>
|
||||
<body style="padding: 10px; background: #fff;">
|
||||
<h3>Same-Origin Iframe Content</h3>
|
||||
<button id="iframe-btn">Iframe Button</button>
|
||||
<input type="text" id="iframe-input" placeholder="Iframe input" />
|
||||
|
||||
<script>
|
||||
// When button is clicked, increment counter in parent page using addEventListener
|
||||
document.getElementById('iframe-btn').addEventListener('click', function() {
|
||||
if (window.parent && window.parent !== window) {
|
||||
// Call parent's incrementCounter function
|
||||
window.parent.incrementCounter('Same-Origin Iframe');
|
||||
}
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,177 @@
|
||||
"""
|
||||
Test that headers are properly passed to CDPClient for authenticated remote browser connections.
|
||||
|
||||
This tests the fix for: When using browser-use with remote browser services that require
|
||||
authentication headers, these headers need to be included in the WebSocket handshake.
|
||||
"""
|
||||
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
from browser_use.browser.session import BrowserSession
|
||||
|
||||
|
||||
def test_browser_profile_headers_attribute():
|
||||
"""Test that BrowserProfile correctly stores headers attribute."""
|
||||
test_headers = {'Authorization': 'Bearer token123', 'X-API-Key': 'key456'}
|
||||
|
||||
profile = BrowserProfile(headers=test_headers)
|
||||
|
||||
# Verify headers are stored correctly
|
||||
assert profile.headers == test_headers
|
||||
|
||||
# Test with profile without headers
|
||||
profile_no_headers = BrowserProfile()
|
||||
assert profile_no_headers.headers is None
|
||||
|
||||
|
||||
def test_browser_profile_headers_inherited():
|
||||
"""Test that BrowserSession can access headers from its profile."""
|
||||
test_headers = {'Authorization': 'Bearer test-token'}
|
||||
|
||||
session = BrowserSession(cdp_url='wss://example.com/cdp', headers=test_headers)
|
||||
|
||||
assert session.browser_profile.headers == test_headers
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cdp_client_headers_passed_on_connect():
|
||||
"""Test that headers from BrowserProfile are passed to CDPClient on connect()."""
|
||||
test_headers = {
|
||||
'Authorization': 'AWS4-HMAC-SHA256 Credential=test...',
|
||||
'X-Amz-Date': '20250914T163733Z',
|
||||
'X-Amz-Security-Token': 'test-token',
|
||||
'Host': 'remote-browser.example.com',
|
||||
}
|
||||
|
||||
session = BrowserSession(cdp_url='wss://remote-browser.example.com/cdp', headers=test_headers)
|
||||
|
||||
with patch('browser_use.browser.session.TimeoutWrappedCDPClient') as mock_cdp_client_class:
|
||||
# Setup mock CDPClient instance
|
||||
mock_cdp_client = AsyncMock()
|
||||
mock_cdp_client_class.return_value = mock_cdp_client
|
||||
mock_cdp_client.start = AsyncMock()
|
||||
mock_cdp_client.stop = AsyncMock()
|
||||
|
||||
# Mock CDP methods
|
||||
mock_cdp_client.send = MagicMock()
|
||||
mock_cdp_client.send.Target = MagicMock()
|
||||
mock_cdp_client.send.Target.setAutoAttach = AsyncMock()
|
||||
mock_cdp_client.send.Target.getTargets = AsyncMock(return_value={'targetInfos': []})
|
||||
mock_cdp_client.send.Target.createTarget = AsyncMock(return_value={'targetId': 'test-target-id'})
|
||||
|
||||
# Mock SessionManager (imported inside connect() from browser_use.browser.session_manager)
|
||||
with patch('browser_use.browser.session_manager.SessionManager') as mock_session_manager_class:
|
||||
mock_session_manager = MagicMock()
|
||||
mock_session_manager_class.return_value = mock_session_manager
|
||||
mock_session_manager.start_monitoring = AsyncMock()
|
||||
mock_session_manager.get_all_page_targets = MagicMock(return_value=[])
|
||||
|
||||
try:
|
||||
await session.connect()
|
||||
except Exception:
|
||||
# May fail due to incomplete mocking, but we can still verify the key assertion
|
||||
pass
|
||||
|
||||
# Verify CDPClient was instantiated with the headers
|
||||
mock_cdp_client_class.assert_called_once()
|
||||
call_kwargs = mock_cdp_client_class.call_args
|
||||
|
||||
# Check positional args and keyword args
|
||||
assert call_kwargs[0][0] == 'wss://remote-browser.example.com/cdp', 'CDP URL should be first arg'
|
||||
actual_headers = call_kwargs[1].get('additional_headers')
|
||||
# All user-provided headers must be present
|
||||
for key, value in test_headers.items():
|
||||
assert actual_headers[key] == value, f'Header {key} should be passed as additional_headers'
|
||||
# User-Agent should be injected for remote connections
|
||||
assert 'User-Agent' in actual_headers, 'User-Agent should be injected for remote connections'
|
||||
assert actual_headers['User-Agent'].startswith('browser-use/'), 'User-Agent should start with browser-use/'
|
||||
assert call_kwargs[1].get('max_ws_frame_size') == 200 * 1024 * 1024, 'max_ws_frame_size should be set'
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cdp_client_no_headers_when_none():
|
||||
"""Test that CDPClient is created with None headers when profile has no headers."""
|
||||
session = BrowserSession(cdp_url='wss://example.com/cdp')
|
||||
|
||||
assert session.browser_profile.headers is None
|
||||
|
||||
with patch('browser_use.browser.session.TimeoutWrappedCDPClient') as mock_cdp_client_class:
|
||||
mock_cdp_client = AsyncMock()
|
||||
mock_cdp_client_class.return_value = mock_cdp_client
|
||||
mock_cdp_client.start = AsyncMock()
|
||||
mock_cdp_client.stop = AsyncMock()
|
||||
mock_cdp_client.send = MagicMock()
|
||||
mock_cdp_client.send.Target = MagicMock()
|
||||
mock_cdp_client.send.Target.setAutoAttach = AsyncMock()
|
||||
mock_cdp_client.send.Target.getTargets = AsyncMock(return_value={'targetInfos': []})
|
||||
mock_cdp_client.send.Target.createTarget = AsyncMock(return_value={'targetId': 'test-target-id'})
|
||||
|
||||
with patch('browser_use.browser.session_manager.SessionManager') as mock_session_manager_class:
|
||||
mock_session_manager = MagicMock()
|
||||
mock_session_manager_class.return_value = mock_session_manager
|
||||
mock_session_manager.start_monitoring = AsyncMock()
|
||||
mock_session_manager.get_all_page_targets = MagicMock(return_value=[])
|
||||
|
||||
try:
|
||||
await session.connect()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Verify CDPClient was called with User-Agent even when no user headers are set (remote connection)
|
||||
call_kwargs = mock_cdp_client_class.call_args
|
||||
actual_headers = call_kwargs[1].get('additional_headers')
|
||||
assert actual_headers is not None, 'Remote connections should always have headers with User-Agent'
|
||||
assert actual_headers['User-Agent'].startswith('browser-use/'), 'User-Agent should be injected for remote connections'
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_headers_used_for_json_version_endpoint():
|
||||
"""Test that headers are also used when fetching WebSocket URL from /json/version."""
|
||||
test_headers = {'Authorization': 'Bearer test-token'}
|
||||
|
||||
# Use HTTP URL (not ws://) to trigger /json/version fetch
|
||||
session = BrowserSession(cdp_url='http://remote-browser.example.com:9222', headers=test_headers)
|
||||
|
||||
with patch('browser_use.browser.session.httpx.AsyncClient') as mock_client_class:
|
||||
mock_client = AsyncMock()
|
||||
mock_client_class.return_value.__aenter__ = AsyncMock(return_value=mock_client)
|
||||
mock_client_class.return_value.__aexit__ = AsyncMock()
|
||||
|
||||
# Mock the /json/version response
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = {'webSocketDebuggerUrl': 'ws://remote-browser.example.com:9222/devtools/browser/abc'}
|
||||
mock_client.get = AsyncMock(return_value=mock_response)
|
||||
|
||||
with patch('browser_use.browser.session.TimeoutWrappedCDPClient') as mock_cdp_client_class:
|
||||
mock_cdp_client = AsyncMock()
|
||||
mock_cdp_client_class.return_value = mock_cdp_client
|
||||
mock_cdp_client.start = AsyncMock()
|
||||
mock_cdp_client.send = MagicMock()
|
||||
mock_cdp_client.send.Target = MagicMock()
|
||||
mock_cdp_client.send.Target.setAutoAttach = AsyncMock()
|
||||
|
||||
with patch('browser_use.browser.session_manager.SessionManager') as mock_sm_class:
|
||||
mock_sm = MagicMock()
|
||||
mock_sm_class.return_value = mock_sm
|
||||
mock_sm.start_monitoring = AsyncMock()
|
||||
mock_sm.get_all_page_targets = MagicMock(return_value=[])
|
||||
|
||||
try:
|
||||
await session.connect()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Verify headers were passed to the HTTP GET request
|
||||
mock_client.get.assert_called_once()
|
||||
call_kwargs = mock_client.get.call_args
|
||||
actual_headers = call_kwargs[1].get('headers')
|
||||
# All user-provided headers must be present
|
||||
for key, value in test_headers.items():
|
||||
assert actual_headers[key] == value, f'Header {key} should be passed to /json/version'
|
||||
# User-Agent should be injected
|
||||
assert actual_headers['User-Agent'].startswith('browser-use/'), (
|
||||
'User-Agent should be injected for /json/version fetch'
|
||||
)
|
||||
@@ -0,0 +1,259 @@
|
||||
"""Tests for cloud browser functionality."""
|
||||
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from browser_use.browser.cloud.cloud import (
|
||||
CloudBrowserAuthError,
|
||||
CloudBrowserClient,
|
||||
CloudBrowserError,
|
||||
)
|
||||
from browser_use.browser.cloud.views import CreateBrowserRequest
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
from browser_use.browser.session import BrowserSession
|
||||
from browser_use.sync.auth import CloudAuthConfig
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def temp_config_dir(monkeypatch):
|
||||
"""Create temporary config directory."""
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
temp_dir = Path(tmpdir) / '.config' / 'browseruse'
|
||||
temp_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Use monkeypatch to set the environment variable
|
||||
monkeypatch.setenv('BROWSER_USE_CONFIG_DIR', str(temp_dir))
|
||||
|
||||
yield temp_dir
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_auth_config(temp_config_dir):
|
||||
"""Create a mock auth config with valid token."""
|
||||
auth_config = CloudAuthConfig(api_token='test-token', user_id='test-user-id', authorized_at=None)
|
||||
auth_config.save_to_file()
|
||||
return auth_config
|
||||
|
||||
|
||||
class TestCloudBrowserClient:
|
||||
"""Test CloudBrowserClient class."""
|
||||
|
||||
async def test_create_browser_success(self, mock_auth_config, monkeypatch):
|
||||
"""Test successful cloud browser creation."""
|
||||
|
||||
# Clear environment variable so test uses mock_auth_config
|
||||
monkeypatch.delenv('BROWSER_USE_API_KEY', raising=False)
|
||||
|
||||
# Mock response data matching the API
|
||||
mock_response_data = {
|
||||
'id': 'test-browser-id',
|
||||
'status': 'active',
|
||||
'liveUrl': 'https://live.browser-use.com?wss=test',
|
||||
'cdpUrl': 'wss://test.proxy.daytona.works',
|
||||
'timeoutAt': '2025-09-17T04:35:36.049892',
|
||||
'startedAt': '2025-09-17T03:35:36.049974',
|
||||
'finishedAt': None,
|
||||
}
|
||||
|
||||
# Mock the httpx client
|
||||
with patch('httpx.AsyncClient') as mock_client_class:
|
||||
mock_response = AsyncMock()
|
||||
mock_response.status_code = 201
|
||||
mock_response.is_success = True
|
||||
mock_response.json = lambda: mock_response_data
|
||||
|
||||
mock_client = AsyncMock()
|
||||
mock_client.post.return_value = mock_response
|
||||
mock_client_class.return_value = mock_client
|
||||
|
||||
client = CloudBrowserClient()
|
||||
client.client = mock_client
|
||||
|
||||
result = await client.create_browser(CreateBrowserRequest())
|
||||
|
||||
assert result.id == 'test-browser-id'
|
||||
assert result.status == 'active'
|
||||
assert result.cdpUrl == 'wss://test.proxy.daytona.works'
|
||||
|
||||
# Verify auth headers were included
|
||||
mock_client.post.assert_called_once()
|
||||
call_args = mock_client.post.call_args
|
||||
assert 'X-Browser-Use-API-Key' in call_args.kwargs['headers']
|
||||
assert call_args.kwargs['headers']['X-Browser-Use-API-Key'] == 'test-token'
|
||||
|
||||
async def test_create_browser_auth_error(self, temp_config_dir, monkeypatch):
|
||||
"""Test cloud browser creation with auth error."""
|
||||
|
||||
# Clear environment variable and don't create auth config - should trigger auth error
|
||||
monkeypatch.delenv('BROWSER_USE_API_KEY', raising=False)
|
||||
|
||||
client = CloudBrowserClient()
|
||||
|
||||
with pytest.raises(CloudBrowserAuthError) as exc_info:
|
||||
await client.create_browser(CreateBrowserRequest())
|
||||
|
||||
assert 'BROWSER_USE_API_KEY is not set' in str(exc_info.value)
|
||||
|
||||
async def test_create_browser_http_401(self, mock_auth_config, monkeypatch):
|
||||
"""Test cloud browser creation with HTTP 401 response."""
|
||||
|
||||
# Clear environment variable so test uses mock_auth_config
|
||||
monkeypatch.delenv('BROWSER_USE_API_KEY', raising=False)
|
||||
|
||||
with patch('httpx.AsyncClient') as mock_client_class:
|
||||
mock_response = AsyncMock()
|
||||
mock_response.status_code = 401
|
||||
mock_response.is_success = False
|
||||
|
||||
mock_client = AsyncMock()
|
||||
mock_client.post.return_value = mock_response
|
||||
mock_client_class.return_value = mock_client
|
||||
|
||||
client = CloudBrowserClient()
|
||||
client.client = mock_client
|
||||
|
||||
with pytest.raises(CloudBrowserAuthError) as exc_info:
|
||||
await client.create_browser(CreateBrowserRequest())
|
||||
|
||||
assert 'BROWSER_USE_API_KEY is invalid' in str(exc_info.value)
|
||||
|
||||
async def test_create_browser_with_env_var(self, temp_config_dir, monkeypatch):
|
||||
"""Test cloud browser creation using BROWSER_USE_API_KEY environment variable."""
|
||||
|
||||
# Set environment variable
|
||||
monkeypatch.setenv('BROWSER_USE_API_KEY', 'env-test-token')
|
||||
|
||||
# Mock response data matching the API
|
||||
mock_response_data = {
|
||||
'id': 'test-browser-id',
|
||||
'status': 'active',
|
||||
'liveUrl': 'https://live.browser-use.com?wss=test',
|
||||
'cdpUrl': 'wss://test.proxy.daytona.works',
|
||||
'timeoutAt': '2025-09-17T04:35:36.049892',
|
||||
'startedAt': '2025-09-17T03:35:36.049974',
|
||||
'finishedAt': None,
|
||||
}
|
||||
|
||||
with patch('httpx.AsyncClient') as mock_client_class:
|
||||
mock_response = AsyncMock()
|
||||
mock_response.status_code = 201
|
||||
mock_response.is_success = True
|
||||
mock_response.json = lambda: mock_response_data
|
||||
|
||||
mock_client = AsyncMock()
|
||||
mock_client.post.return_value = mock_response
|
||||
mock_client_class.return_value = mock_client
|
||||
|
||||
client = CloudBrowserClient()
|
||||
client.client = mock_client
|
||||
|
||||
result = await client.create_browser(CreateBrowserRequest())
|
||||
|
||||
assert result.id == 'test-browser-id'
|
||||
assert result.status == 'active'
|
||||
assert result.cdpUrl == 'wss://test.proxy.daytona.works'
|
||||
|
||||
# Verify environment variable was used
|
||||
mock_client.post.assert_called_once()
|
||||
call_args = mock_client.post.call_args
|
||||
assert 'X-Browser-Use-API-Key' in call_args.kwargs['headers']
|
||||
assert call_args.kwargs['headers']['X-Browser-Use-API-Key'] == 'env-test-token'
|
||||
|
||||
async def test_stop_browser_success(self, mock_auth_config, monkeypatch):
|
||||
"""Test successful cloud browser session stop."""
|
||||
|
||||
# Clear environment variable so test uses mock_auth_config
|
||||
monkeypatch.delenv('BROWSER_USE_API_KEY', raising=False)
|
||||
|
||||
# Mock response data for stop
|
||||
mock_response_data = {
|
||||
'id': 'test-browser-id',
|
||||
'status': 'stopped',
|
||||
'liveUrl': 'https://live.browser-use.com?wss=test',
|
||||
'cdpUrl': 'wss://test.proxy.daytona.works',
|
||||
'timeoutAt': '2025-09-17T04:35:36.049892',
|
||||
'startedAt': '2025-09-17T03:35:36.049974',
|
||||
'finishedAt': '2025-09-17T04:35:36.049892',
|
||||
}
|
||||
|
||||
with patch('httpx.AsyncClient') as mock_client_class:
|
||||
mock_response = AsyncMock()
|
||||
mock_response.status_code = 200
|
||||
mock_response.is_success = True
|
||||
mock_response.json = lambda: mock_response_data
|
||||
|
||||
mock_client = AsyncMock()
|
||||
mock_client.patch.return_value = mock_response
|
||||
mock_client_class.return_value = mock_client
|
||||
|
||||
client = CloudBrowserClient()
|
||||
client.client = mock_client
|
||||
client.current_session_id = 'test-browser-id'
|
||||
|
||||
result = await client.stop_browser()
|
||||
|
||||
assert result.id == 'test-browser-id'
|
||||
assert result.status == 'stopped'
|
||||
assert result.finishedAt is not None
|
||||
|
||||
# Verify correct API call
|
||||
mock_client.patch.assert_called_once()
|
||||
call_args = mock_client.patch.call_args
|
||||
assert 'test-browser-id' in call_args.args[0] # URL contains session ID
|
||||
assert call_args.kwargs['json'] == {'action': 'stop'}
|
||||
assert 'X-Browser-Use-API-Key' in call_args.kwargs['headers']
|
||||
|
||||
async def test_stop_browser_session_not_found(self, mock_auth_config, monkeypatch):
|
||||
"""Test stopping a browser session that doesn't exist."""
|
||||
|
||||
# Clear environment variable so test uses mock_auth_config
|
||||
monkeypatch.delenv('BROWSER_USE_API_KEY', raising=False)
|
||||
|
||||
with patch('httpx.AsyncClient') as mock_client_class:
|
||||
mock_response = AsyncMock()
|
||||
mock_response.status_code = 404
|
||||
mock_response.is_success = False
|
||||
|
||||
mock_client = AsyncMock()
|
||||
mock_client.patch.return_value = mock_response
|
||||
mock_client_class.return_value = mock_client
|
||||
|
||||
client = CloudBrowserClient()
|
||||
client.client = mock_client
|
||||
|
||||
with pytest.raises(CloudBrowserError) as exc_info:
|
||||
await client.stop_browser('nonexistent-session')
|
||||
|
||||
assert 'not found' in str(exc_info.value)
|
||||
|
||||
|
||||
class TestBrowserSessionCloudIntegration:
|
||||
"""Test BrowserSession integration with cloud browsers."""
|
||||
|
||||
async def test_cloud_browser_profile_property(self):
|
||||
"""Test that cloud_browser property works correctly."""
|
||||
|
||||
# Just test the profile and session properties without connecting
|
||||
profile = BrowserProfile(use_cloud=True)
|
||||
session = BrowserSession(browser_profile=profile, cdp_url='ws://mock-url') # Provide CDP URL to avoid connection
|
||||
|
||||
assert session.cloud_browser is True
|
||||
assert session.browser_profile.use_cloud is True
|
||||
|
||||
async def test_browser_session_cloud_browser_logic(self, mock_auth_config, monkeypatch):
|
||||
"""Test that cloud browser profile settings work correctly."""
|
||||
|
||||
# Clear environment variable so test uses mock_auth_config
|
||||
monkeypatch.delenv('BROWSER_USE_API_KEY', raising=False)
|
||||
|
||||
# Test cloud browser profile creation
|
||||
profile = BrowserProfile(use_cloud=True)
|
||||
assert profile.use_cloud is True
|
||||
|
||||
# Test that BrowserSession respects cloud_browser setting
|
||||
# Provide CDP URL to avoid actual connection attempts
|
||||
session = BrowserSession(browser_profile=profile, cdp_url='ws://mock-url')
|
||||
assert session.cloud_browser is True
|
||||
@@ -0,0 +1,138 @@
|
||||
"""Test clicking elements inside cross-origin iframes."""
|
||||
|
||||
import asyncio
|
||||
|
||||
import pytest
|
||||
|
||||
from browser_use.browser.profile import BrowserProfile, ViewportSize
|
||||
from browser_use.browser.session import BrowserSession
|
||||
from browser_use.tools.service import Tools
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def browser_session():
|
||||
"""Create browser session with cross-origin iframe support."""
|
||||
session = BrowserSession(
|
||||
browser_profile=BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
keep_alive=True,
|
||||
window_size=ViewportSize(width=1920, height=1400),
|
||||
cross_origin_iframes=True, # Enable cross-origin iframe extraction
|
||||
)
|
||||
)
|
||||
await session.start()
|
||||
yield session
|
||||
await session.kill()
|
||||
|
||||
|
||||
class TestCrossOriginIframeClick:
|
||||
"""Test clicking elements inside cross-origin iframes."""
|
||||
|
||||
async def test_click_element_in_cross_origin_iframe(self, httpserver, browser_session: BrowserSession):
|
||||
"""Verify that elements inside iframes in different CDP targets can be clicked."""
|
||||
|
||||
# Create iframe content with clickable elements
|
||||
iframe_html = """
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Iframe Page</title></head>
|
||||
<body>
|
||||
<h1>Iframe Content</h1>
|
||||
<a href="https://test-domain.example/page" id="iframe-link">Test Link</a>
|
||||
<button id="iframe-button">Iframe Button</button>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
# Create main page with iframe pointing to our test server
|
||||
main_html = """
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Multi-Target Test</title></head>
|
||||
<body>
|
||||
<h1>Main Page</h1>
|
||||
<button id="main-button">Main Button</button>
|
||||
<iframe id="test-iframe" src="/iframe-content" style="width: 800px; height: 600px;"></iframe>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
# Serve both pages
|
||||
httpserver.expect_request('/multi-target-test').respond_with_data(main_html, content_type='text/html')
|
||||
httpserver.expect_request('/iframe-content').respond_with_data(iframe_html, content_type='text/html')
|
||||
url = httpserver.url_for('/multi-target-test')
|
||||
|
||||
# Navigate to the page
|
||||
await browser_session.navigate_to(url)
|
||||
|
||||
# Wait for iframe to load
|
||||
await asyncio.sleep(2)
|
||||
|
||||
# Get DOM state with cross-origin iframe extraction enabled
|
||||
# Use browser_session.get_browser_state_summary() instead of directly creating DomService
|
||||
# This goes through the proper event bus and watchdog system
|
||||
browser_state = await browser_session.get_browser_state_summary(
|
||||
include_screenshot=False,
|
||||
include_recent_events=False,
|
||||
)
|
||||
assert browser_state.dom_state is not None
|
||||
state = browser_state.dom_state
|
||||
|
||||
print(f'\n📊 Found {len(state.selector_map)} total elements')
|
||||
|
||||
# Find elements from different targets
|
||||
targets_found = set()
|
||||
main_page_elements = []
|
||||
iframe_elements = []
|
||||
|
||||
for idx, element in state.selector_map.items():
|
||||
target_id = element.target_id
|
||||
targets_found.add(target_id)
|
||||
|
||||
# Check if element is from iframe (identified by id attributes we set)
|
||||
# Iframe elements will have a different target_id when cross_origin_iframes=True
|
||||
if element.attributes:
|
||||
element_id = element.attributes.get('id', '')
|
||||
if element_id in ('iframe-link', 'iframe-button'):
|
||||
iframe_elements.append((idx, element))
|
||||
print(f' ✅ Found iframe element: [{idx}] {element.tag_name} id={element_id}')
|
||||
elif element_id == 'main-button':
|
||||
main_page_elements.append((idx, element))
|
||||
|
||||
# Verify we found elements from at least 2 different targets
|
||||
print(f'\n🎯 Found elements from {len(targets_found)} different CDP targets')
|
||||
|
||||
# Check if iframe elements were found
|
||||
if len(iframe_elements) == 0:
|
||||
pytest.fail('Expected to find at least one element from iframe, but found none')
|
||||
|
||||
# Verify we found at least one element from the iframe
|
||||
assert len(iframe_elements) > 0, 'Expected to find at least one element from iframe'
|
||||
|
||||
# Try clicking the iframe element
|
||||
print('\n🖱️ Testing Click on Iframe Element:')
|
||||
tools = Tools()
|
||||
|
||||
link_idx, link_element = iframe_elements[0]
|
||||
print(f' Attempting to click element [{link_idx}] from iframe...')
|
||||
|
||||
try:
|
||||
result = await tools.click(index=link_idx, browser_session=browser_session)
|
||||
|
||||
# Check for errors
|
||||
if result.error:
|
||||
pytest.fail(f'Click on iframe element [{link_idx}] failed with error: {result.error}')
|
||||
|
||||
if result.extracted_content and (
|
||||
'not available' in result.extracted_content.lower() or 'failed' in result.extracted_content.lower()
|
||||
):
|
||||
pytest.fail(f'Click on iframe element [{link_idx}] failed: {result.extracted_content}')
|
||||
|
||||
print(f' ✅ Click succeeded on iframe element [{link_idx}]!')
|
||||
print(' 🎉 Iframe element clicking works!')
|
||||
|
||||
except Exception as e:
|
||||
pytest.fail(f'Exception while clicking iframe element [{link_idx}]: {e}')
|
||||
|
||||
print('\n✅ Test passed: Iframe elements can be clicked')
|
||||
@@ -0,0 +1,585 @@
|
||||
"""
|
||||
Test DOM serializer with complex scenarios: shadow DOM, same-origin and cross-origin iframes.
|
||||
|
||||
This test verifies that the DOM serializer correctly:
|
||||
1. Extracts interactive elements from shadow DOM
|
||||
2. Processes same-origin iframes
|
||||
3. Handles cross-origin iframes (should be blocked)
|
||||
4. Generates correct selector_map with expected element counts
|
||||
|
||||
Usage:
|
||||
uv run pytest tests/ci/browser/test_dom_serializer.py -v -s
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from pytest_httpserver import HTTPServer
|
||||
|
||||
from browser_use.agent.service import Agent
|
||||
from browser_use.browser import BrowserSession
|
||||
from browser_use.browser.profile import BrowserProfile, ViewportSize
|
||||
from tests.ci.conftest import create_mock_llm
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def http_server():
|
||||
"""Create and provide a test HTTP server for DOM serializer tests."""
|
||||
from pathlib import Path
|
||||
|
||||
server = HTTPServer()
|
||||
server.start()
|
||||
|
||||
# Load HTML templates from files
|
||||
test_dir = Path(__file__).parent
|
||||
main_page_html = (test_dir / 'test_page_template.html').read_text()
|
||||
iframe_html = (test_dir / 'iframe_template.html').read_text()
|
||||
stacked_page_html = (test_dir / 'test_page_stacked_template.html').read_text()
|
||||
|
||||
# Route 1: Main page with shadow DOM and iframes
|
||||
server.expect_request('/dom-test-main').respond_with_data(main_page_html, content_type='text/html')
|
||||
|
||||
# Route 2: Same-origin iframe content
|
||||
server.expect_request('/iframe-same-origin').respond_with_data(iframe_html, content_type='text/html')
|
||||
|
||||
# Route 3: Stacked complex scenarios test page
|
||||
server.expect_request('/stacked-test').respond_with_data(stacked_page_html, content_type='text/html')
|
||||
|
||||
yield server
|
||||
server.stop()
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def base_url(http_server):
|
||||
"""Return the base URL for the test HTTP server."""
|
||||
return f'http://{http_server.host}:{http_server.port}'
|
||||
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
async def browser_session():
|
||||
"""Create a browser session for DOM serializer tests."""
|
||||
session = BrowserSession(
|
||||
browser_profile=BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
keep_alive=True,
|
||||
window_size=ViewportSize(width=1920, height=1400), # Taller window to fit all stacked elements
|
||||
cross_origin_iframes=True, # Enable cross-origin iframe extraction via CDP target switching
|
||||
)
|
||||
)
|
||||
await session.start()
|
||||
yield session
|
||||
await session.kill()
|
||||
|
||||
|
||||
class TestDOMSerializer:
|
||||
"""Test DOM serializer with complex scenarios."""
|
||||
|
||||
async def test_dom_serializer_with_shadow_dom_and_iframes(self, browser_session, base_url):
|
||||
"""Test DOM serializer extracts elements from shadow DOM, same-origin iframes, and cross-origin iframes.
|
||||
|
||||
This test verifies:
|
||||
1. Elements are in the serializer (selector_map)
|
||||
2. We can click elements using click(index)
|
||||
|
||||
Expected interactive elements:
|
||||
- Regular DOM: 3 elements (button, input, link on main page)
|
||||
- Shadow DOM: 3 elements (2 buttons, 1 input inside shadow root)
|
||||
- Same-origin iframe: 2 elements (button, input inside iframe)
|
||||
- Cross-origin iframe placeholder: about:blank (no interactive elements)
|
||||
- Iframe tags: 2 elements (the iframe elements themselves)
|
||||
Total: ~10 interactive elements
|
||||
"""
|
||||
from browser_use.tools.service import Tools
|
||||
|
||||
tools = Tools()
|
||||
|
||||
# Create mock LLM actions that will click elements from each category
|
||||
# We'll generate actions dynamically after we know the indices
|
||||
actions = [
|
||||
f"""
|
||||
{{
|
||||
"thinking": "I'll navigate to the DOM test page",
|
||||
"evaluation_previous_goal": "Starting task",
|
||||
"memory": "Navigating to test page",
|
||||
"next_goal": "Navigate to test page",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{base_url}/dom-test-main",
|
||||
"new_tab": false
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
"""
|
||||
]
|
||||
await tools.navigate(url=f'{base_url}/dom-test-main', new_tab=False, browser_session=browser_session)
|
||||
|
||||
import asyncio
|
||||
|
||||
await asyncio.sleep(1)
|
||||
|
||||
# Get the browser state to access selector_map
|
||||
browser_state_summary = await browser_session.get_browser_state_summary(
|
||||
include_screenshot=False,
|
||||
include_recent_events=False,
|
||||
)
|
||||
|
||||
assert browser_state_summary is not None, 'Browser state summary should not be None'
|
||||
assert browser_state_summary.dom_state is not None, 'DOM state should not be None'
|
||||
|
||||
selector_map = browser_state_summary.dom_state.selector_map
|
||||
print(f' Selector map: {selector_map.keys()}')
|
||||
|
||||
print('\n📊 DOM Serializer Analysis:')
|
||||
print(f' Total interactive elements found: {len(selector_map)}')
|
||||
serilized_text = browser_state_summary.dom_state.llm_representation()
|
||||
print(f' Serialized text: {serilized_text}')
|
||||
# assume all selector map keys are as text in the serialized text
|
||||
# for idx, element in selector_map.items():
|
||||
# assert str(idx) in serilized_text, f'Element {idx} should be in serialized text'
|
||||
# print(f' ✓ Element {idx} found in serialized text')
|
||||
|
||||
# assume at least 10 interactive elements are in the selector map
|
||||
assert len(selector_map) >= 10, f'Should find at least 10 interactive elements, found {len(selector_map)}'
|
||||
|
||||
# assert all interactive elements marked with [123] from serialized text are in selector map
|
||||
# find all [index] from serialized text with regex
|
||||
import re
|
||||
|
||||
indices = re.findall(r'\[(\d+)\]', serilized_text)
|
||||
for idx in indices:
|
||||
assert int(idx) in selector_map.keys(), f'Element {idx} should be in selector map'
|
||||
print(f' ✓ Element {idx} found in selector map')
|
||||
|
||||
regular_elements = []
|
||||
shadow_elements = []
|
||||
iframe_content_elements = []
|
||||
iframe_tags = []
|
||||
|
||||
# Categorize elements by their IDs (more stable than hardcoded indices)
|
||||
# Check element attributes to identify their location
|
||||
for idx, element in selector_map.items():
|
||||
# Check if this is an iframe tag (not content inside iframe)
|
||||
if element.tag_name == 'iframe':
|
||||
iframe_tags.append((idx, element))
|
||||
# Check if element has an ID attribute
|
||||
elif hasattr(element, 'attributes') and 'id' in element.attributes:
|
||||
elem_id = element.attributes['id'].lower()
|
||||
# Shadow DOM elements have IDs starting with "shadow-"
|
||||
if elem_id.startswith('shadow-'):
|
||||
shadow_elements.append((idx, element))
|
||||
# Iframe content elements have IDs starting with "iframe-"
|
||||
elif elem_id.startswith('iframe-'):
|
||||
iframe_content_elements.append((idx, element))
|
||||
# Everything else is regular DOM
|
||||
else:
|
||||
regular_elements.append((idx, element))
|
||||
# Elements without IDs are regular DOM
|
||||
else:
|
||||
regular_elements.append((idx, element))
|
||||
|
||||
# Verify element counts based on our test page structure:
|
||||
# - Regular DOM: 3-4 elements (button, input, link on main page + possible cross-origin content)
|
||||
# - Shadow DOM: 3 elements (2 buttons, 1 input inside shadow root)
|
||||
# - Iframe content: 2 elements (button, input from same-origin iframe)
|
||||
# - Iframe tags: 2 elements (the iframe elements themselves)
|
||||
# Total: ~10-11 interactive elements depending on cross-origin iframe extraction
|
||||
|
||||
print('\n✅ DOM Serializer Test Summary:')
|
||||
print(f' • Regular DOM: {len(regular_elements)} elements {"✓" if len(regular_elements) >= 3 else "✗"}')
|
||||
print(f' • Shadow DOM: {len(shadow_elements)} elements {"✓" if len(shadow_elements) >= 3 else "✗"}')
|
||||
print(
|
||||
f' • Same-origin iframe content: {len(iframe_content_elements)} elements {"✓" if len(iframe_content_elements) >= 2 else "✗"}'
|
||||
)
|
||||
print(f' • Iframe tags: {len(iframe_tags)} elements {"✓" if len(iframe_tags) >= 2 else "✗"}')
|
||||
print(f' • Total elements: {len(selector_map)}')
|
||||
|
||||
# Verify we found elements from all sources
|
||||
assert len(selector_map) >= 8, f'Should find at least 8 interactive elements, found {len(selector_map)}'
|
||||
assert len(regular_elements) >= 1, f'Should find at least 1 regular DOM element, found {len(regular_elements)}'
|
||||
assert len(shadow_elements) >= 1, f'Should find at least 1 shadow DOM element, found {len(shadow_elements)}'
|
||||
assert len(iframe_content_elements) >= 1, (
|
||||
f'Should find at least 1 iframe content element, found {len(iframe_content_elements)}'
|
||||
)
|
||||
|
||||
# Now test clicking elements from each category using tools.click(index)
|
||||
print('\n🖱️ Testing Click Functionality:')
|
||||
|
||||
# Helper to call tools.click(index) and verify it worked
|
||||
async def click(index: int, element_description: str, browser_session: BrowserSession):
|
||||
result = await tools.click(index=index, browser_session=browser_session)
|
||||
# Check both error field and extracted_content for failure messages
|
||||
if result.error:
|
||||
raise AssertionError(f'Click on {element_description} [{index}] failed: {result.error}')
|
||||
if result.extracted_content and (
|
||||
'not available' in result.extracted_content.lower() or 'failed' in result.extracted_content.lower()
|
||||
):
|
||||
raise AssertionError(f'Click on {element_description} [{index}] failed: {result.extracted_content}')
|
||||
print(f' ✓ {element_description} [{index}] clicked successfully')
|
||||
return result
|
||||
|
||||
# Test clicking a regular DOM element (button)
|
||||
if regular_elements:
|
||||
regular_button_idx = next((idx for idx, el in regular_elements if 'regular-btn' in el.attributes.get('id', '')), None)
|
||||
if regular_button_idx:
|
||||
await click(regular_button_idx, 'Regular DOM button', browser_session)
|
||||
|
||||
# Test clicking a shadow DOM element (button)
|
||||
if shadow_elements:
|
||||
shadow_button_idx = next((idx for idx, el in shadow_elements if 'btn' in el.attributes.get('id', '')), None)
|
||||
if shadow_button_idx:
|
||||
await click(shadow_button_idx, 'Shadow DOM button', browser_session)
|
||||
|
||||
# Test clicking a same-origin iframe element (button)
|
||||
if iframe_content_elements:
|
||||
iframe_button_idx = next((idx for idx, el in iframe_content_elements if 'btn' in el.attributes.get('id', '')), None)
|
||||
if iframe_button_idx:
|
||||
await click(iframe_button_idx, 'Same-origin iframe button', browser_session)
|
||||
|
||||
# Validate click counter - verify all 3 clicks actually executed JavaScript
|
||||
print('\n✅ Validating click counter...')
|
||||
|
||||
# Get the CDP session for the main page (use target from a regular DOM element)
|
||||
# Note: browser_session.agent_focus_target_id may point to a different target than the page
|
||||
if regular_elements and regular_elements[0][1].target_id:
|
||||
cdp_session = await browser_session.get_or_create_cdp_session(target_id=regular_elements[0][1].target_id)
|
||||
else:
|
||||
cdp_session = await browser_session.get_or_create_cdp_session()
|
||||
|
||||
result = await cdp_session.cdp_client.send.Runtime.evaluate(
|
||||
params={
|
||||
'expression': 'window.getClickCount()',
|
||||
'returnByValue': True,
|
||||
},
|
||||
session_id=cdp_session.session_id,
|
||||
)
|
||||
|
||||
click_count = result.get('result', {}).get('value', 0)
|
||||
print(f' Click counter value: {click_count}')
|
||||
|
||||
assert click_count == 3, (
|
||||
f'Expected 3 clicks (Regular DOM + Shadow DOM + Iframe), but counter shows {click_count}. '
|
||||
f'This means some clicks did not execute JavaScript properly.'
|
||||
)
|
||||
|
||||
print('\n🎉 DOM Serializer test completed successfully!')
|
||||
|
||||
async def test_dom_serializer_element_counts_detailed(self, browser_session, base_url):
|
||||
"""Detailed test to verify specific element types are captured correctly."""
|
||||
|
||||
actions = [
|
||||
f"""
|
||||
{{
|
||||
"thinking": "Navigating to test page",
|
||||
"evaluation_previous_goal": "Starting",
|
||||
"memory": "Navigate",
|
||||
"next_goal": "Navigate",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{base_url}/dom-test-main",
|
||||
"new_tab": false
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
""",
|
||||
"""
|
||||
{
|
||||
"thinking": "Done",
|
||||
"evaluation_previous_goal": "Navigated",
|
||||
"memory": "Complete",
|
||||
"next_goal": "Done",
|
||||
"action": [
|
||||
{
|
||||
"done": {
|
||||
"text": "Done",
|
||||
"success": true
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
]
|
||||
|
||||
mock_llm = create_mock_llm(actions=actions)
|
||||
agent = Agent(
|
||||
task=f'Navigate to {base_url}/dom-test-main',
|
||||
llm=mock_llm,
|
||||
browser_session=browser_session,
|
||||
)
|
||||
|
||||
history = await agent.run(max_steps=2)
|
||||
|
||||
# Get current browser state to access selector_map
|
||||
browser_state_summary = await browser_session.get_browser_state_summary(
|
||||
include_screenshot=False,
|
||||
include_recent_events=False,
|
||||
)
|
||||
selector_map = browser_state_summary.dom_state.selector_map
|
||||
|
||||
# Count different element types
|
||||
buttons = 0
|
||||
inputs = 0
|
||||
links = 0
|
||||
|
||||
for idx, element in selector_map.items():
|
||||
element_str = str(element).lower()
|
||||
if 'button' in element_str or '<button' in element_str:
|
||||
buttons += 1
|
||||
elif 'input' in element_str or '<input' in element_str:
|
||||
inputs += 1
|
||||
elif 'link' in element_str or '<a' in element_str or 'href' in element_str:
|
||||
links += 1
|
||||
|
||||
print('\n📊 Element Type Counts:')
|
||||
print(f' Buttons: {buttons}')
|
||||
print(f' Inputs: {inputs}')
|
||||
print(f' Links: {links}')
|
||||
print(f' Total: {len(selector_map)}')
|
||||
|
||||
# We should have at least some of each type from the regular DOM
|
||||
assert buttons >= 1, f'Should find at least 1 button, found {buttons}'
|
||||
assert inputs >= 1, f'Should find at least 1 input, found {inputs}'
|
||||
|
||||
print('\n✅ Element type verification passed!')
|
||||
|
||||
async def test_stacked_complex_scenarios(self, browser_session, base_url):
|
||||
"""Test clicking through stacked complex scenarios and verify cross-origin iframe extraction.
|
||||
|
||||
This test verifies:
|
||||
1. Open shadow DOM element interaction
|
||||
2. Closed shadow DOM element interaction (nested inside open shadow)
|
||||
3. Same-origin iframe element interaction (inside closed shadow)
|
||||
4. Cross-origin iframe placeholder with about:blank (no external dependencies)
|
||||
5. Truly nested structure: Open Shadow → Closed Shadow → Iframe
|
||||
"""
|
||||
from browser_use.tools.service import Tools
|
||||
|
||||
tools = Tools()
|
||||
|
||||
# Navigate to stacked test page
|
||||
await tools.navigate(url=f'{base_url}/stacked-test', new_tab=False, browser_session=browser_session)
|
||||
|
||||
import asyncio
|
||||
|
||||
await asyncio.sleep(1)
|
||||
|
||||
# Get browser state
|
||||
browser_state_summary = await browser_session.get_browser_state_summary(
|
||||
include_screenshot=False,
|
||||
include_recent_events=False,
|
||||
)
|
||||
|
||||
selector_map = browser_state_summary.dom_state.selector_map
|
||||
print(f'\n📊 Stacked Test - Found {len(selector_map)} elements')
|
||||
|
||||
# Debug: Show all elements
|
||||
print('\n🔍 All elements found:')
|
||||
for idx, element in selector_map.items():
|
||||
elem_id = element.attributes.get('id', 'NO_ID') if hasattr(element, 'attributes') else 'NO_ATTR'
|
||||
print(f' [{idx}] {element.tag_name} id={elem_id} target={element.target_id[-4:] if element.target_id else "None"}')
|
||||
|
||||
# Categorize elements
|
||||
open_shadow_elements = []
|
||||
closed_shadow_elements = []
|
||||
iframe_elements = []
|
||||
final_button = None
|
||||
|
||||
for idx, element in selector_map.items():
|
||||
if hasattr(element, 'attributes') and 'id' in element.attributes:
|
||||
elem_id = element.attributes['id'].lower()
|
||||
|
||||
if 'open-shadow' in elem_id:
|
||||
open_shadow_elements.append((idx, element))
|
||||
elif 'closed-shadow' in elem_id:
|
||||
closed_shadow_elements.append((idx, element))
|
||||
elif 'iframe' in elem_id and element.tag_name != 'iframe':
|
||||
iframe_elements.append((idx, element))
|
||||
elif 'final-button' in elem_id:
|
||||
final_button = (idx, element)
|
||||
|
||||
print('\n📋 Element Distribution:')
|
||||
print(f' Open Shadow: {len(open_shadow_elements)} elements')
|
||||
print(f' Closed Shadow: {len(closed_shadow_elements)} elements')
|
||||
print(f' Iframe content: {len(iframe_elements)} elements')
|
||||
print(f' Final button: {"Found" if final_button else "Not found"}')
|
||||
|
||||
# Test clicking through each stacked layer
|
||||
print('\n🖱️ Testing Click Functionality Through Stacked Layers:')
|
||||
|
||||
async def click(index: int, element_description: str, browser_session: BrowserSession):
|
||||
result = await tools.click(index=index, browser_session=browser_session)
|
||||
if result.error:
|
||||
raise AssertionError(f'Click on {element_description} [{index}] failed: {result.error}')
|
||||
if result.extracted_content and (
|
||||
'not available' in result.extracted_content.lower() or 'failed' in result.extracted_content.lower()
|
||||
):
|
||||
raise AssertionError(f'Click on {element_description} [{index}] failed: {result.extracted_content}')
|
||||
print(f' ✓ {element_description} [{index}] clicked successfully')
|
||||
return result
|
||||
|
||||
clicks_performed = 0
|
||||
|
||||
# 1. Click open shadow button
|
||||
if open_shadow_elements:
|
||||
open_shadow_btn = next((idx for idx, el in open_shadow_elements if 'btn' in el.attributes.get('id', '')), None)
|
||||
if open_shadow_btn:
|
||||
await click(open_shadow_btn, 'Open Shadow DOM button', browser_session)
|
||||
clicks_performed += 1
|
||||
|
||||
# 2. Click closed shadow button
|
||||
if closed_shadow_elements:
|
||||
closed_shadow_btn = next((idx for idx, el in closed_shadow_elements if 'btn' in el.attributes.get('id', '')), None)
|
||||
if closed_shadow_btn:
|
||||
await click(closed_shadow_btn, 'Closed Shadow DOM button', browser_session)
|
||||
clicks_performed += 1
|
||||
|
||||
# 3. Click iframe button
|
||||
if iframe_elements:
|
||||
iframe_btn = next((idx for idx, el in iframe_elements if 'btn' in el.attributes.get('id', '')), None)
|
||||
if iframe_btn:
|
||||
await click(iframe_btn, 'Same-origin iframe button', browser_session)
|
||||
clicks_performed += 1
|
||||
|
||||
# 4. Try clicking cross-origin iframe tag (can click the tag, but not elements inside)
|
||||
cross_origin_iframe_tag = None
|
||||
for idx, element in selector_map.items():
|
||||
if (
|
||||
element.tag_name == 'iframe'
|
||||
and hasattr(element, 'attributes')
|
||||
and 'cross-origin' in element.attributes.get('id', '').lower()
|
||||
):
|
||||
cross_origin_iframe_tag = (idx, element)
|
||||
break
|
||||
|
||||
# Verify cross-origin iframe extraction is working
|
||||
# Check the full DOM tree (not just selector_map which only has interactive elements)
|
||||
def count_targets_in_tree(node, targets=None):
|
||||
if targets is None:
|
||||
targets = set()
|
||||
# SimplifiedNode has original_node which is an EnhancedDOMTreeNode
|
||||
if hasattr(node, 'original_node') and node.original_node and node.original_node.target_id:
|
||||
targets.add(node.original_node.target_id)
|
||||
# Recursively check children
|
||||
if hasattr(node, 'children') and node.children:
|
||||
for child in node.children:
|
||||
count_targets_in_tree(child, targets)
|
||||
return targets
|
||||
|
||||
all_targets = count_targets_in_tree(browser_state_summary.dom_state._root)
|
||||
|
||||
print('\n📊 Cross-Origin Iframe Extraction:')
|
||||
print(f' Found elements from {len(all_targets)} different CDP targets in full DOM tree')
|
||||
|
||||
if len(all_targets) >= 2:
|
||||
print(' ✅ Multi-target iframe extraction IS WORKING!')
|
||||
print(' ✓ Successfully extracted DOM from multiple CDP targets')
|
||||
print(' ✓ CDP target switching feature is enabled and functional')
|
||||
else:
|
||||
print(' ⚠️ Only found elements from 1 target (cross-origin extraction may not be working)')
|
||||
|
||||
if cross_origin_iframe_tag:
|
||||
print(f'\n 📌 Found cross-origin iframe tag [{cross_origin_iframe_tag[0]}]')
|
||||
# Note: We don't increment clicks_performed since this doesn't trigger our counter
|
||||
# await click(cross_origin_iframe_tag[0], 'Cross-origin iframe tag (scroll)', browser_session)
|
||||
|
||||
# 5. Click final button (after all stacked elements)
|
||||
if final_button:
|
||||
await click(final_button[0], 'Final button (after stack)', browser_session)
|
||||
clicks_performed += 1
|
||||
|
||||
# Validate click counter
|
||||
print('\n✅ Validating click counter...')
|
||||
|
||||
# Get CDP session from a non-iframe element (open shadow or final button)
|
||||
if open_shadow_elements:
|
||||
cdp_session = await browser_session.get_or_create_cdp_session(target_id=open_shadow_elements[0][1].target_id)
|
||||
elif final_button:
|
||||
cdp_session = await browser_session.get_or_create_cdp_session(target_id=final_button[1].target_id)
|
||||
else:
|
||||
cdp_session = await browser_session.get_or_create_cdp_session()
|
||||
|
||||
result = await cdp_session.cdp_client.send.Runtime.evaluate(
|
||||
params={
|
||||
'expression': 'window.getClickCount()',
|
||||
'returnByValue': True,
|
||||
},
|
||||
session_id=cdp_session.session_id,
|
||||
)
|
||||
|
||||
click_count = result.get('result', {}).get('value', 0)
|
||||
print(f' Click counter value: {click_count}')
|
||||
print(f' Expected clicks: {clicks_performed}')
|
||||
|
||||
assert click_count == clicks_performed, (
|
||||
f'Expected {clicks_performed} clicks, but counter shows {click_count}. '
|
||||
f'Some clicks did not execute JavaScript properly.'
|
||||
)
|
||||
|
||||
print('\n🎉 Stacked scenario test completed successfully!')
|
||||
print(' ✓ Open shadow DOM clicks work')
|
||||
print(' ✓ Closed shadow DOM clicks work')
|
||||
print(' ✓ Same-origin iframe clicks work (can access elements inside)')
|
||||
print(' ✓ Cross-origin iframe extraction works (CDP target switching enabled)')
|
||||
print(' ✓ Truly nested structure works: Open Shadow → Closed Shadow → Iframe')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
"""Run test in debug mode with manual fixture setup."""
|
||||
import asyncio
|
||||
import logging
|
||||
|
||||
# Set up debug logging
|
||||
logging.basicConfig(
|
||||
level=logging.DEBUG,
|
||||
format='%(levelname)-8s [%(name)s] %(message)s',
|
||||
)
|
||||
|
||||
async def main():
|
||||
# Set up HTTP server fixture
|
||||
from pathlib import Path
|
||||
|
||||
from pytest_httpserver import HTTPServer
|
||||
|
||||
server = HTTPServer()
|
||||
server.start()
|
||||
|
||||
# Load HTML templates from files (same as http_server fixture)
|
||||
test_dir = Path(__file__).parent
|
||||
main_page_html = (test_dir / 'test_page_stacked_template.html').read_text()
|
||||
# Set up routes using templates
|
||||
server.expect_request('/stacked-test').respond_with_data(main_page_html, content_type='text/html')
|
||||
|
||||
base_url = f'http://{server.host}:{server.port}'
|
||||
print(f'\n🌐 HTTP Server running at {base_url}')
|
||||
|
||||
# Set up browser session
|
||||
from browser_use.browser import BrowserSession
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
|
||||
session = BrowserSession(
|
||||
browser_profile=BrowserProfile(
|
||||
headless=False, # Set to False to see browser in action
|
||||
user_data_dir=None,
|
||||
keep_alive=True,
|
||||
)
|
||||
)
|
||||
|
||||
try:
|
||||
await session.start()
|
||||
print('🚀 Browser session started\n')
|
||||
|
||||
# Run the test
|
||||
test = TestDOMSerializer()
|
||||
await test.test_stacked_complex_scenarios(session, base_url)
|
||||
|
||||
print('\n✅ Test completed successfully!')
|
||||
|
||||
finally:
|
||||
# Cleanup
|
||||
await session.kill()
|
||||
server.stop()
|
||||
print('\n🧹 Cleanup complete')
|
||||
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,396 @@
|
||||
"""
|
||||
Test navigation edge cases: broken pages, slow loading, non-existing pages.
|
||||
|
||||
Tests verify that:
|
||||
1. Agent can handle navigation to broken/malformed HTML pages
|
||||
2. Agent can handle slow-loading pages without hanging
|
||||
3. Agent can handle non-existing pages (404, connection refused, etc.)
|
||||
4. Agent can recover and continue making LLM calls after encountering these issues
|
||||
|
||||
All tests use:
|
||||
- max_steps=3 to limit agent actions
|
||||
- 120s timeout to fail if test takes too long
|
||||
- Mock LLM to verify agent can still make decisions after navigation errors
|
||||
|
||||
Usage:
|
||||
uv run pytest tests/ci/browser/test_navigation.py -v -s
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
|
||||
import pytest
|
||||
from pytest_httpserver import HTTPServer
|
||||
from werkzeug import Response
|
||||
|
||||
from browser_use.agent.service import Agent
|
||||
from browser_use.browser import BrowserSession
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
from tests.ci.conftest import create_mock_llm
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def http_server():
|
||||
"""Create and provide a test HTTP server for navigation tests."""
|
||||
server = HTTPServer()
|
||||
server.start()
|
||||
|
||||
# Route 1: Broken/malformed HTML page
|
||||
server.expect_request('/broken').respond_with_data(
|
||||
'<html><head><title>Broken Page</title></head><body><h1>Incomplete HTML',
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
# Route 2: Valid page for testing navigation after error recovery
|
||||
server.expect_request('/valid').respond_with_data(
|
||||
'<html><head><title>Valid Page</title></head><body><h1>Valid Page</h1><p>This page loaded successfully</p></body></html>',
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
# Route 3: Slow loading page - delays 10 seconds before responding
|
||||
def slow_handler(request):
|
||||
time.sleep(10)
|
||||
return Response(
|
||||
'<html><head><title>Slow Page</title></head><body><h1>Slow Loading Page</h1><p>This page took 10 seconds to load</p></body></html>',
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
server.expect_request('/slow').respond_with_handler(slow_handler)
|
||||
|
||||
# Route 4: 404 page
|
||||
server.expect_request('/notfound').respond_with_data(
|
||||
'<html><head><title>404 Not Found</title></head><body><h1>404 - Page Not Found</h1></body></html>',
|
||||
status=404,
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
yield server
|
||||
server.stop()
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def base_url(http_server):
|
||||
"""Return the base URL for the test HTTP server."""
|
||||
return f'http://{http_server.host}:{http_server.port}'
|
||||
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
async def browser_session():
|
||||
"""Create a browser session for navigation tests."""
|
||||
session = BrowserSession(
|
||||
browser_profile=BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
keep_alive=True,
|
||||
)
|
||||
)
|
||||
await session.start()
|
||||
yield session
|
||||
await session.kill()
|
||||
|
||||
|
||||
class TestNavigationEdgeCases:
|
||||
"""Test navigation error handling and recovery."""
|
||||
|
||||
async def test_broken_page_navigation(self, browser_session, base_url):
|
||||
"""Test that agent can handle broken/malformed HTML and still make LLM calls."""
|
||||
|
||||
# Create actions for the agent:
|
||||
# 1. Navigate to broken page
|
||||
# 2. Check if page exists
|
||||
# 3. Done
|
||||
actions = [
|
||||
f"""
|
||||
{{
|
||||
"thinking": "I need to navigate to the broken page",
|
||||
"evaluation_previous_goal": "Starting task",
|
||||
"memory": "Navigating to broken page",
|
||||
"next_goal": "Navigate to broken page",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{base_url}/broken"
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
""",
|
||||
"""
|
||||
{
|
||||
"thinking": "I should check if the page loaded",
|
||||
"evaluation_previous_goal": "Navigated to page",
|
||||
"memory": "Checking page state",
|
||||
"next_goal": "Verify page exists",
|
||||
"action": [
|
||||
{
|
||||
"done": {
|
||||
"text": "Page exists despite broken HTML",
|
||||
"success": true
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
]
|
||||
|
||||
mock_llm = create_mock_llm(actions=actions)
|
||||
|
||||
agent = Agent(
|
||||
task=f'Navigate to {base_url}/broken and check if page exists',
|
||||
llm=mock_llm,
|
||||
browser_session=browser_session,
|
||||
)
|
||||
|
||||
# Run with timeout - should complete within 2 minutes
|
||||
try:
|
||||
history = await asyncio.wait_for(agent.run(max_steps=3), timeout=120)
|
||||
assert len(history) > 0, 'Agent should have completed at least one step'
|
||||
# If agent completes successfully, it means LLM was called and functioning
|
||||
final_result = history.final_result()
|
||||
assert final_result is not None, 'Agent should return a final result'
|
||||
except TimeoutError:
|
||||
pytest.fail('Test timed out after 2 minutes - agent hung on broken page')
|
||||
|
||||
async def test_slow_loading_page(self, browser_session, base_url):
|
||||
"""Test that agent can handle slow-loading pages without hanging."""
|
||||
|
||||
actions = [
|
||||
f"""
|
||||
{{
|
||||
"thinking": "I need to navigate to the slow page",
|
||||
"evaluation_previous_goal": "Starting task",
|
||||
"memory": "Navigating to slow page",
|
||||
"next_goal": "Navigate to slow page",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{base_url}/slow"
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
""",
|
||||
"""
|
||||
{
|
||||
"thinking": "The page loaded, even though it was slow",
|
||||
"evaluation_previous_goal": "Successfully navigated",
|
||||
"memory": "Page loaded after delay",
|
||||
"next_goal": "Complete task",
|
||||
"action": [
|
||||
{
|
||||
"done": {
|
||||
"text": "Slow page loaded successfully",
|
||||
"success": true
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
]
|
||||
|
||||
mock_llm = create_mock_llm(actions=actions)
|
||||
|
||||
agent = Agent(
|
||||
task=f'Navigate to {base_url}/slow and wait for it to load',
|
||||
llm=mock_llm,
|
||||
browser_session=browser_session,
|
||||
)
|
||||
|
||||
# Run with timeout - should complete within 2 minutes
|
||||
start_time = time.time()
|
||||
try:
|
||||
history = await asyncio.wait_for(agent.run(max_steps=3), timeout=120)
|
||||
elapsed = time.time() - start_time
|
||||
|
||||
assert len(history) > 0, 'Agent should have completed at least one step'
|
||||
assert elapsed >= 10, f'Agent should have waited for slow page (10s delay), but only took {elapsed:.1f}s'
|
||||
final_result = history.final_result()
|
||||
assert final_result is not None, 'Agent should return a final result'
|
||||
except TimeoutError:
|
||||
pytest.fail('Test timed out after 2 minutes - agent hung on slow page')
|
||||
|
||||
async def test_nonexisting_page_404(self, browser_session, base_url):
|
||||
"""Test that agent can handle 404 pages and still make LLM calls."""
|
||||
|
||||
actions = [
|
||||
f"""
|
||||
{{
|
||||
"thinking": "I need to navigate to the non-existing page",
|
||||
"evaluation_previous_goal": "Starting task",
|
||||
"memory": "Navigating to 404 page",
|
||||
"next_goal": "Navigate to non-existing page",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{base_url}/notfound"
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
""",
|
||||
"""
|
||||
{
|
||||
"thinking": "I got a 404 error but the browser still works",
|
||||
"evaluation_previous_goal": "Navigated to 404 page",
|
||||
"memory": "Page not found",
|
||||
"next_goal": "Report that page does not exist",
|
||||
"action": [
|
||||
{
|
||||
"done": {
|
||||
"text": "Page does not exist (404 error)",
|
||||
"success": false
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
]
|
||||
|
||||
mock_llm = create_mock_llm(actions=actions)
|
||||
|
||||
agent = Agent(
|
||||
task=f'Navigate to {base_url}/notfound and check if page exists',
|
||||
llm=mock_llm,
|
||||
browser_session=browser_session,
|
||||
)
|
||||
|
||||
# Run with timeout - should complete within 2 minutes
|
||||
try:
|
||||
history = await asyncio.wait_for(agent.run(max_steps=3), timeout=120)
|
||||
assert len(history) > 0, 'Agent should have completed at least one step'
|
||||
final_result = history.final_result()
|
||||
assert final_result is not None, 'Agent should return a final result'
|
||||
except TimeoutError:
|
||||
pytest.fail('Test timed out after 2 minutes - agent hung on 404 page')
|
||||
|
||||
async def test_nonexisting_domain(self, browser_session):
|
||||
"""Test that agent can handle completely non-existing domains (connection refused)."""
|
||||
|
||||
# Use a localhost port that's not listening
|
||||
nonexisting_url = 'http://localhost:59999/page'
|
||||
|
||||
actions = [
|
||||
f"""
|
||||
{{
|
||||
"thinking": "I need to navigate to a non-existing domain",
|
||||
"evaluation_previous_goal": "Starting task",
|
||||
"memory": "Attempting to navigate",
|
||||
"next_goal": "Navigate to non-existing domain",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{nonexisting_url}"
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
""",
|
||||
"""
|
||||
{
|
||||
"thinking": "The connection failed but I can still proceed",
|
||||
"evaluation_previous_goal": "Connection failed",
|
||||
"memory": "Domain does not exist",
|
||||
"next_goal": "Report failure",
|
||||
"action": [
|
||||
{
|
||||
"done": {
|
||||
"text": "Domain does not exist (connection refused)",
|
||||
"success": false
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
]
|
||||
|
||||
mock_llm = create_mock_llm(actions=actions)
|
||||
|
||||
agent = Agent(
|
||||
task=f'Navigate to {nonexisting_url} and check if it exists',
|
||||
llm=mock_llm,
|
||||
browser_session=browser_session,
|
||||
)
|
||||
|
||||
# Run with timeout - should complete within 2 minutes
|
||||
try:
|
||||
history = await asyncio.wait_for(agent.run(max_steps=3), timeout=120)
|
||||
assert len(history) > 0, 'Agent should have completed at least one step'
|
||||
final_result = history.final_result()
|
||||
assert final_result is not None, 'Agent should return a final result'
|
||||
except TimeoutError:
|
||||
pytest.fail('Test timed out after 2 minutes - agent hung on non-existing domain')
|
||||
|
||||
async def test_recovery_after_navigation_error(self, browser_session, base_url):
|
||||
"""Test that agent can recover and navigate to valid page after encountering error."""
|
||||
|
||||
actions = [
|
||||
f"""
|
||||
{{
|
||||
"thinking": "First, I'll try the broken page",
|
||||
"evaluation_previous_goal": "Starting task",
|
||||
"memory": "Navigating to broken page",
|
||||
"next_goal": "Navigate to broken page first",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{base_url}/broken"
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
""",
|
||||
f"""
|
||||
{{
|
||||
"thinking": "That page was broken, let me try a valid page now",
|
||||
"evaluation_previous_goal": "Broken page loaded",
|
||||
"memory": "Now navigating to valid page",
|
||||
"next_goal": "Navigate to valid page",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{base_url}/valid"
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
""",
|
||||
"""
|
||||
{
|
||||
"thinking": "The valid page loaded successfully after the broken one",
|
||||
"evaluation_previous_goal": "Valid page loaded",
|
||||
"memory": "Successfully recovered from error",
|
||||
"next_goal": "Complete task",
|
||||
"action": [
|
||||
{
|
||||
"done": {
|
||||
"text": "Successfully navigated to valid page after broken page",
|
||||
"success": true
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
]
|
||||
|
||||
mock_llm = create_mock_llm(actions=actions)
|
||||
|
||||
agent = Agent(
|
||||
task=f'First navigate to {base_url}/broken, then navigate to {base_url}/valid',
|
||||
llm=mock_llm,
|
||||
browser_session=browser_session,
|
||||
)
|
||||
|
||||
# Run with timeout - should complete within 2 minutes
|
||||
try:
|
||||
history = await asyncio.wait_for(agent.run(max_steps=3), timeout=120)
|
||||
assert len(history) >= 2, 'Agent should have completed at least 2 steps (broken -> valid)'
|
||||
|
||||
# Verify final page is the valid one
|
||||
final_url = await browser_session.get_current_page_url()
|
||||
assert final_url.endswith('/valid'), f'Final URL should be /valid, got {final_url}'
|
||||
|
||||
# Verify agent completed successfully
|
||||
final_result = history.final_result()
|
||||
assert final_result is not None, 'Agent should return a final result'
|
||||
except TimeoutError:
|
||||
pytest.fail('Test timed out after 2 minutes - agent could not recover from broken page')
|
||||
@@ -0,0 +1,139 @@
|
||||
"""Regression tests: navigation readiness detection must actually detect page load.
|
||||
|
||||
cdp-use's event registry is single-slot per CDP method: registering a per-session
|
||||
Page.lifecycleEvent closure replaces the previous one, so only the most recently
|
||||
attached target recorded lifecycle events. Any earlier tab's event deque went silent,
|
||||
and every navigation on it burned the full readiness timeout (3s same-domain /
|
||||
8s cross-domain) before proceeding on a page in unknown load state.
|
||||
|
||||
Lifecycle events are now stored per-target in SessionManager, fed by one global
|
||||
handler registered once on the root CDP client.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
|
||||
import pytest
|
||||
|
||||
from browser_use.browser.events import NavigateToUrlEvent
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
from browser_use.browser.session import BrowserSession
|
||||
|
||||
SIMPLE_HTML = '<html><head><title>fast page</title></head><body>hello</body></html>'
|
||||
|
||||
# Generous bound for a local page load: well above real load time (~100ms),
|
||||
# well below the 3s that a readiness-timeout fallback burns.
|
||||
FAST_NAVIGATION_BOUND_S = 2.5
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def browser_session():
|
||||
session = BrowserSession(browser_profile=BrowserProfile(headless=True, user_data_dir=None, keep_alive=True))
|
||||
await session.start()
|
||||
yield session
|
||||
await session.kill()
|
||||
|
||||
|
||||
async def test_navigation_detects_readiness_without_burning_timeout(httpserver, browser_session: BrowserSession):
|
||||
"""A local page load must complete as soon as the load event fires, not after the fallback timeout."""
|
||||
httpserver.expect_request('/fast').respond_with_data(SIMPLE_HTML, content_type='text/html')
|
||||
|
||||
start = time.monotonic()
|
||||
await browser_session.navigate_to(httpserver.url_for('/fast'))
|
||||
elapsed = time.monotonic() - start
|
||||
|
||||
assert elapsed < FAST_NAVIGATION_BOUND_S, (
|
||||
f'navigation took {elapsed:.2f}s — readiness detection failed and the fallback timeout was burned'
|
||||
)
|
||||
|
||||
|
||||
async def test_first_tab_navigation_still_works_after_second_tab_opens(httpserver, browser_session: BrowserSession):
|
||||
"""Opening a new tab must not disable lifecycle monitoring on existing tabs.
|
||||
|
||||
Pre-fix: the second tab's per-session handler registration replaced the first
|
||||
tab's on the shared root client, freezing the first tab's event deque — every
|
||||
subsequent navigation on tab A hit the full readiness timeout.
|
||||
"""
|
||||
httpserver.expect_request('/a').respond_with_data(SIMPLE_HTML, content_type='text/html')
|
||||
httpserver.expect_request('/b').respond_with_data(SIMPLE_HTML, content_type='text/html')
|
||||
httpserver.expect_request('/a2').respond_with_data(SIMPLE_HTML, content_type='text/html')
|
||||
|
||||
await browser_session.navigate_to(httpserver.url_for('/a'))
|
||||
tab_a = browser_session.agent_focus_target_id
|
||||
assert tab_a is not None
|
||||
|
||||
# Open a second tab — its target attach re-runs page monitoring setup
|
||||
event = browser_session.event_bus.dispatch(NavigateToUrlEvent(url=httpserver.url_for('/b'), new_tab=True))
|
||||
await event
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
# Navigate the FIRST tab again (same domain -> 3s fallback timeout pre-fix)
|
||||
start = time.monotonic()
|
||||
await browser_session._navigate_and_wait(httpserver.url_for('/a2'), tab_a, wait_until='load')
|
||||
elapsed = time.monotonic() - start
|
||||
|
||||
assert elapsed < FAST_NAVIGATION_BOUND_S, (
|
||||
f'tab A navigation took {elapsed:.2f}s after tab B opened — its lifecycle monitoring was clobbered'
|
||||
)
|
||||
|
||||
|
||||
async def test_readiness_timeout_is_reported_not_swallowed(httpserver, browser_session: BrowserSession):
|
||||
"""When readiness genuinely times out, _navigate_and_wait must say so instead of
|
||||
returning indistinguishably from success (NavigationCompleteEvent.loading_status
|
||||
exists for exactly this)."""
|
||||
|
||||
def slow_image(request):
|
||||
time.sleep(5)
|
||||
from werkzeug.wrappers import Response
|
||||
|
||||
return Response(b'', content_type='image/png')
|
||||
|
||||
# DOMContentLoaded fires immediately; 'load' is held hostage by the stalled image.
|
||||
httpserver.expect_request('/hanging').respond_with_data(
|
||||
'<html><body><img src="/slow-img">never finishes loading</body></html>', content_type='text/html'
|
||||
)
|
||||
httpserver.expect_request('/slow-img').respond_with_handler(slow_image)
|
||||
|
||||
target_id = browser_session.agent_focus_target_id
|
||||
assert target_id is not None
|
||||
|
||||
status = await browser_session._navigate_and_wait(httpserver.url_for('/hanging'), target_id, timeout=1.0, wait_until='load')
|
||||
|
||||
assert status is not None and 'timeout' in status, f'readiness timeout was swallowed, got status={status!r}'
|
||||
|
||||
|
||||
async def test_successful_navigation_returns_no_timeout_status(httpserver, browser_session: BrowserSession):
|
||||
httpserver.expect_request('/ok').respond_with_data(SIMPLE_HTML, content_type='text/html')
|
||||
|
||||
target_id = browser_session.agent_focus_target_id
|
||||
assert target_id is not None
|
||||
|
||||
status = await browser_session._navigate_and_wait(httpserver.url_for('/ok'), target_id, wait_until='load')
|
||||
assert status is None
|
||||
|
||||
|
||||
async def test_same_document_navigation_completes_immediately(httpserver, browser_session: BrowserSession):
|
||||
"""Fragment/History-API navigations must not burn the readiness timeout.
|
||||
|
||||
Page.navigate returns no loaderId for same-document navigations and Chrome
|
||||
emits no new load/DOMContentLoaded lifecycle events for them — the navigation
|
||||
is already committed when Page.navigate returns.
|
||||
"""
|
||||
httpserver.expect_request('/page').respond_with_data(
|
||||
'<html><body><a id="anchor" name="section">s</a></body></html>', content_type='text/html'
|
||||
)
|
||||
|
||||
await browser_session.navigate_to(httpserver.url_for('/page'))
|
||||
target_id = browser_session.agent_focus_target_id
|
||||
assert target_id is not None
|
||||
|
||||
# Let the first load's trailing lifecycle events (networkIdle fires ~1s after
|
||||
# load) drain, so a stale event can't accidentally satisfy the fragment wait.
|
||||
await asyncio.sleep(1.5)
|
||||
|
||||
start = time.monotonic()
|
||||
status = await browser_session._navigate_and_wait(httpserver.url_for('/page') + '#section', target_id, wait_until='load')
|
||||
elapsed = time.monotonic() - start
|
||||
|
||||
assert elapsed < FAST_NAVIGATION_BOUND_S, f'same-document navigation took {elapsed:.2f}s — burned the readiness timeout'
|
||||
assert status is None, f'same-document navigation reported a bogus timeout: {status!r}'
|
||||
@@ -0,0 +1,188 @@
|
||||
"""
|
||||
Test navigation on heavy/slow-loading pages (e.g. e-commerce PDPs).
|
||||
|
||||
Reproduces the issue where navigating to heavy pages like stevemadden.com PDPs
|
||||
fails due to NavigateToUrlEvent timing out.
|
||||
|
||||
Usage:
|
||||
uv run pytest tests/ci/browser/test_navigation_slow_pages.py -v -s
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
|
||||
import pytest
|
||||
from pytest_httpserver import HTTPServer
|
||||
from werkzeug import Response
|
||||
|
||||
from browser_use.agent.service import Agent
|
||||
from browser_use.browser import BrowserSession
|
||||
from browser_use.browser.events import NavigateToUrlEvent
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
from tests.ci.conftest import create_mock_llm
|
||||
|
||||
HEAVY_PDP_HTML = """
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Frosting Black Velvet - Steve Madden</title></head>
|
||||
<body>
|
||||
<h1>FROSTING</h1>
|
||||
<p class="price">$129.95</p>
|
||||
<button id="add-to-cart">ADD TO BAG</button>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def heavy_page_server():
|
||||
server = HTTPServer()
|
||||
server.start()
|
||||
|
||||
def slow_initial_response(request):
|
||||
time.sleep(6)
|
||||
return Response(HEAVY_PDP_HTML, content_type='text/html')
|
||||
|
||||
server.expect_request('/slow-server-pdp').respond_with_handler(slow_initial_response)
|
||||
|
||||
def redirect_step1(request):
|
||||
return Response('', status=302, headers={'Location': f'http://{server.host}:{server.port}/redirect-step2'})
|
||||
|
||||
def redirect_step2(request):
|
||||
return Response('', status=302, headers={'Location': f'http://{server.host}:{server.port}/redirect-final'})
|
||||
|
||||
def redirect_final(request):
|
||||
time.sleep(3)
|
||||
return Response(HEAVY_PDP_HTML, content_type='text/html')
|
||||
|
||||
server.expect_request('/redirect-step1').respond_with_handler(redirect_step1)
|
||||
server.expect_request('/redirect-step2').respond_with_handler(redirect_step2)
|
||||
server.expect_request('/redirect-final').respond_with_handler(redirect_final)
|
||||
|
||||
server.expect_request('/fast-dom-slow-load').respond_with_data(HEAVY_PDP_HTML, content_type='text/html')
|
||||
server.expect_request('/quick-page').respond_with_data(
|
||||
'<html><body><h1>Quick Page</h1></body></html>', content_type='text/html'
|
||||
)
|
||||
|
||||
yield server
|
||||
server.stop()
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def heavy_base_url(heavy_page_server):
|
||||
return f'http://{heavy_page_server.host}:{heavy_page_server.port}'
|
||||
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
async def browser_session():
|
||||
session = BrowserSession(browser_profile=BrowserProfile(headless=True, user_data_dir=None, keep_alive=True))
|
||||
await session.start()
|
||||
yield session
|
||||
await session.kill()
|
||||
|
||||
|
||||
def _nav_actions(url: str, msg: str = 'Done') -> list[str]:
|
||||
"""Helper to build a navigate-then-done action sequence."""
|
||||
return [
|
||||
f"""
|
||||
{{
|
||||
"thinking": "Navigate to the page",
|
||||
"evaluation_previous_goal": "Starting task",
|
||||
"memory": "Navigating",
|
||||
"next_goal": "Navigate",
|
||||
"action": [{{"navigate": {{"url": "{url}"}}}}]
|
||||
}}
|
||||
""",
|
||||
f"""
|
||||
{{
|
||||
"thinking": "Page loaded",
|
||||
"evaluation_previous_goal": "Navigation completed",
|
||||
"memory": "Page loaded",
|
||||
"next_goal": "Done",
|
||||
"action": [{{"done": {{"text": "{msg}", "success": true}}}}]
|
||||
}}
|
||||
""",
|
||||
]
|
||||
|
||||
|
||||
class TestHeavyPageNavigation:
|
||||
async def test_slow_server_response_completes(self, browser_session, heavy_base_url):
|
||||
"""Navigation succeeds even when server takes 6s to respond."""
|
||||
url = f'{heavy_base_url}/slow-server-pdp'
|
||||
agent = Agent(
|
||||
task=f'Navigate to {url}',
|
||||
llm=create_mock_llm(actions=_nav_actions(url)),
|
||||
browser_session=browser_session,
|
||||
)
|
||||
start = time.time()
|
||||
history = await asyncio.wait_for(agent.run(max_steps=3), timeout=60)
|
||||
assert len(history) > 0
|
||||
assert history.final_result() is not None
|
||||
assert time.time() - start >= 5, 'Should have waited for slow server'
|
||||
|
||||
async def test_redirect_chain_completes(self, browser_session, heavy_base_url):
|
||||
"""Navigation handles multi-step redirects + slow final response."""
|
||||
url = f'{heavy_base_url}/redirect-step1'
|
||||
agent = Agent(
|
||||
task=f'Navigate to {url}',
|
||||
llm=create_mock_llm(actions=_nav_actions(url)),
|
||||
browser_session=browser_session,
|
||||
)
|
||||
history = await asyncio.wait_for(agent.run(max_steps=3), timeout=60)
|
||||
assert len(history) > 0
|
||||
assert history.final_result() is not None
|
||||
|
||||
async def test_navigate_event_accepts_domcontentloaded(self, browser_session, heavy_base_url):
|
||||
"""NavigateToUrlEvent with fast page should complete quickly via DOMContentLoaded/load."""
|
||||
url = f'{heavy_base_url}/fast-dom-slow-load'
|
||||
event = browser_session.event_bus.dispatch(NavigateToUrlEvent(url=url))
|
||||
await asyncio.wait_for(event, timeout=15)
|
||||
await event.event_result(raise_if_any=True, raise_if_none=False)
|
||||
|
||||
async def test_recovery_after_slow_navigation(self, browser_session, heavy_base_url):
|
||||
"""Agent recovers and navigates to a fast page after a slow one."""
|
||||
slow_url = f'{heavy_base_url}/slow-server-pdp'
|
||||
quick_url = f'{heavy_base_url}/quick-page'
|
||||
actions = [
|
||||
f"""
|
||||
{{
|
||||
"thinking": "Navigate to slow page",
|
||||
"evaluation_previous_goal": "Starting",
|
||||
"memory": "Going to slow page",
|
||||
"next_goal": "Navigate",
|
||||
"action": [{{"navigate": {{"url": "{slow_url}"}}}}]
|
||||
}}
|
||||
""",
|
||||
f"""
|
||||
{{
|
||||
"thinking": "Now navigate to quick page",
|
||||
"evaluation_previous_goal": "Slow page loaded",
|
||||
"memory": "Trying quick page",
|
||||
"next_goal": "Navigate",
|
||||
"action": [{{"navigate": {{"url": "{quick_url}"}}}}]
|
||||
}}
|
||||
""",
|
||||
"""
|
||||
{
|
||||
"thinking": "Both done",
|
||||
"evaluation_previous_goal": "Quick page loaded",
|
||||
"memory": "Recovery successful",
|
||||
"next_goal": "Done",
|
||||
"action": [{"done": {"text": "Recovery succeeded", "success": true}}]
|
||||
}
|
||||
""",
|
||||
]
|
||||
agent = Agent(
|
||||
task='Navigate to slow then quick page',
|
||||
llm=create_mock_llm(actions=actions),
|
||||
browser_session=browser_session,
|
||||
)
|
||||
history = await asyncio.wait_for(agent.run(max_steps=4), timeout=90)
|
||||
assert len(history) >= 2
|
||||
assert history.final_result() is not None
|
||||
|
||||
async def test_event_timeout_sufficient_for_heavy_pages(self, browser_session):
|
||||
"""event_timeout should be >= 30s to handle slow servers + redirect chains."""
|
||||
event = NavigateToUrlEvent(url='http://example.com')
|
||||
assert event.event_timeout is not None
|
||||
assert event.event_timeout >= 30.0, f'event_timeout={event.event_timeout}s is too low for heavy pages (need >= 30s)'
|
||||
@@ -0,0 +1,219 @@
|
||||
"""Test all recording and save functionality for Agent and BrowserSession."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from browser_use import Agent, AgentHistoryList
|
||||
from browser_use.browser import BrowserProfile, BrowserSession
|
||||
from tests.ci.conftest import create_mock_llm
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def test_dir(tmp_path):
|
||||
"""Create a test directory that gets cleaned up after each test."""
|
||||
test_path = tmp_path / 'test_recordings'
|
||||
test_path.mkdir(exist_ok=True)
|
||||
yield test_path
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def httpserver_url(httpserver):
|
||||
"""Simple test page."""
|
||||
# Use expect_ordered_request with multiple handlers to handle repeated requests
|
||||
for _ in range(10): # Allow up to 10 requests to the same URL
|
||||
httpserver.expect_ordered_request('/').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Test Page</title>
|
||||
</head>
|
||||
<body>
|
||||
<h1>Test Recording Page</h1>
|
||||
<input type="text" id="search" placeholder="Search here" />
|
||||
<button type="button" id="submit">Submit</button>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
return httpserver.url_for('/')
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def llm():
|
||||
"""Create mocked LLM instance for tests."""
|
||||
return create_mock_llm()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def interactive_llm(httpserver_url):
|
||||
"""Create mocked LLM that navigates to page and interacts with elements."""
|
||||
actions = [
|
||||
# First action: Navigate to the page
|
||||
f"""
|
||||
{{
|
||||
"thinking": "null",
|
||||
"evaluation_previous_goal": "Starting the task",
|
||||
"memory": "Need to navigate to the test page",
|
||||
"next_goal": "Navigate to the URL",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{httpserver_url}",
|
||||
"new_tab": false
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
""",
|
||||
# Second action: Click in the search box
|
||||
"""
|
||||
{
|
||||
"thinking": "null",
|
||||
"evaluation_previous_goal": "Successfully navigated to the page",
|
||||
"memory": "Page loaded, can see search box and submit button",
|
||||
"next_goal": "Click on the search box to focus it",
|
||||
"action": [
|
||||
{
|
||||
"click": {
|
||||
"index": 0
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
# Third action: Type text in the search box
|
||||
"""
|
||||
{
|
||||
"thinking": "null",
|
||||
"evaluation_previous_goal": "Clicked on search box",
|
||||
"memory": "Search box is focused and ready for input",
|
||||
"next_goal": "Type 'test' in the search box",
|
||||
"action": [
|
||||
{
|
||||
"input_text": {
|
||||
"index": 0,
|
||||
"text": "test"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
# Fourth action: Click submit button
|
||||
"""
|
||||
{
|
||||
"thinking": "null",
|
||||
"evaluation_previous_goal": "Typed 'test' in search box",
|
||||
"memory": "Text 'test' has been entered successfully",
|
||||
"next_goal": "Click the submit button to complete the task",
|
||||
"action": [
|
||||
{
|
||||
"click": {
|
||||
"index": 1
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
# Fifth action: Done - task completed
|
||||
"""
|
||||
{
|
||||
"thinking": "null",
|
||||
"evaluation_previous_goal": "Clicked the submit button",
|
||||
"memory": "Successfully navigated to the page, typed 'test' in the search box, and clicked submit",
|
||||
"next_goal": "Task completed",
|
||||
"action": [
|
||||
{
|
||||
"done": {
|
||||
"text": "Task completed - typed 'test' in search box and clicked submit",
|
||||
"success": true
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
]
|
||||
return create_mock_llm(actions)
|
||||
|
||||
|
||||
class TestAgentRecordings:
|
||||
"""Test Agent save_conversation_path and generate_gif parameters."""
|
||||
|
||||
@pytest.mark.parametrize('path_type', ['with_slash', 'without_slash', 'deep_directory'])
|
||||
async def test_save_conversation_path(self, test_dir, httpserver_url, llm, path_type):
|
||||
"""Test saving conversation with different path types."""
|
||||
if path_type == 'with_slash':
|
||||
conversation_path = test_dir / 'logs' / 'conversation'
|
||||
elif path_type == 'without_slash':
|
||||
conversation_path = test_dir / 'logs'
|
||||
else: # deep_directory
|
||||
conversation_path = test_dir / 'logs' / 'deep' / 'directory' / 'conversation'
|
||||
|
||||
browser_session = BrowserSession(browser_profile=BrowserProfile(headless=True, disable_security=True, user_data_dir=None))
|
||||
await browser_session.start()
|
||||
try:
|
||||
agent = Agent(
|
||||
task=f'go to {httpserver_url} and type "test" in the search box',
|
||||
llm=llm,
|
||||
browser_session=browser_session,
|
||||
save_conversation_path=str(conversation_path),
|
||||
)
|
||||
history: AgentHistoryList = await agent.run(max_steps=2)
|
||||
|
||||
result = history.final_result()
|
||||
assert result is not None
|
||||
|
||||
# Check that the conversation directory and files were created
|
||||
assert conversation_path.exists(), f'{path_type}: conversation directory was not created'
|
||||
# Files are now always created as conversation_<agent_id>_<step>.txt inside the directory
|
||||
conversation_files = list(conversation_path.glob('conversation_*.txt'))
|
||||
assert len(conversation_files) > 0, f'{path_type}: conversation file was not created in {conversation_path}'
|
||||
finally:
|
||||
await browser_session.kill()
|
||||
|
||||
@pytest.mark.skip(reason='TODO: fix')
|
||||
@pytest.mark.parametrize('generate_gif', [False, True, 'custom_path'])
|
||||
async def test_generate_gif(self, test_dir, httpserver_url, llm, generate_gif):
|
||||
"""Test GIF generation with different settings."""
|
||||
# Clean up any existing GIFs first
|
||||
for gif in Path.cwd().glob('agent_*.gif'):
|
||||
gif.unlink()
|
||||
|
||||
gif_param = generate_gif
|
||||
expected_gif_path = None
|
||||
|
||||
if generate_gif == 'custom_path':
|
||||
expected_gif_path = test_dir / 'custom_agent.gif'
|
||||
gif_param = str(expected_gif_path)
|
||||
|
||||
browser_session = BrowserSession(browser_profile=BrowserProfile(headless=True, disable_security=True, user_data_dir=None))
|
||||
await browser_session.start()
|
||||
try:
|
||||
agent = Agent(
|
||||
task=f'go to {httpserver_url}',
|
||||
llm=llm,
|
||||
browser_session=browser_session,
|
||||
generate_gif=gif_param,
|
||||
)
|
||||
history: AgentHistoryList = await agent.run(max_steps=2)
|
||||
|
||||
result = history.final_result()
|
||||
assert result is not None
|
||||
|
||||
# Check GIF creation
|
||||
if generate_gif is False:
|
||||
gif_files = list(Path.cwd().glob('*.gif'))
|
||||
assert len(gif_files) == 0, 'GIF file was created when generate_gif=False'
|
||||
elif generate_gif is True:
|
||||
# With mock LLM that doesn't navigate, all screenshots will be about:blank placeholders
|
||||
# So no GIF will be created (this is expected behavior)
|
||||
gif_files = list(Path.cwd().glob('agent_history.gif'))
|
||||
assert len(gif_files) == 0, 'GIF should not be created when all screenshots are placeholders'
|
||||
else: # custom_path
|
||||
assert expected_gif_path is not None, 'expected_gif_path should be set for custom_path'
|
||||
# With mock LLM that doesn't navigate, no GIF will be created
|
||||
assert not expected_gif_path.exists(), 'GIF should not be created when all screenshots are placeholders'
|
||||
finally:
|
||||
await browser_session.kill()
|
||||
@@ -0,0 +1,129 @@
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Stacked DOM Elements Test</title>
|
||||
<style>
|
||||
body { font-family: Arial; padding: 20px; min-height: 3000px; }
|
||||
.section {
|
||||
margin: 20px 0;
|
||||
padding: 15px;
|
||||
border: 2px solid #333;
|
||||
background: #f9f9f9;
|
||||
}
|
||||
#click-counter {
|
||||
position: fixed;
|
||||
top: 20px;
|
||||
right: 20px;
|
||||
background: #4CAF50;
|
||||
color: white;
|
||||
padding: 30px 50px;
|
||||
border-radius: 15px;
|
||||
font-size: 48px;
|
||||
font-weight: bold;
|
||||
box-shadow: 0 4px 20px rgba(0,0,0,0.3);
|
||||
transition: all 0.2s ease;
|
||||
z-index: 9999;
|
||||
}
|
||||
#counter-value {
|
||||
font-size: 64px;
|
||||
display: inline-block;
|
||||
min-width: 60px;
|
||||
text-align: center;
|
||||
}
|
||||
@keyframes flash {
|
||||
0% { transform: scale(1); }
|
||||
50% { transform: scale(1.3); background: #FFC107; }
|
||||
100% { transform: scale(1); }
|
||||
}
|
||||
.flash {
|
||||
animation: flash 0.3s ease;
|
||||
}
|
||||
.final-button {
|
||||
margin-top: 50px;
|
||||
padding: 20px 40px;
|
||||
font-size: 24px;
|
||||
background: #2196F3;
|
||||
color: white;
|
||||
border: none;
|
||||
border-radius: 8px;
|
||||
cursor: pointer;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div id="click-counter">Clicks: <span id="counter-value">0</span></div>
|
||||
<h1>Nested DOM Test</h1>
|
||||
|
||||
<!-- Root: Open Shadow DOM (contains everything else) -->
|
||||
<div class="section">
|
||||
<div id="open-shadow-host"></div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
// Global click counter
|
||||
let clickCount = 0;
|
||||
|
||||
function incrementCounter(source) {
|
||||
clickCount++;
|
||||
const counter = document.getElementById('click-counter');
|
||||
const counterValue = document.getElementById('counter-value');
|
||||
|
||||
counterValue.textContent = clickCount;
|
||||
console.log(`Click #${clickCount} from: ${source}`);
|
||||
|
||||
// Add flash animation
|
||||
counter.classList.remove('flash');
|
||||
void counter.offsetWidth; // Trigger reflow
|
||||
counter.classList.add('flash');
|
||||
}
|
||||
|
||||
// Expose counter for testing
|
||||
window.getClickCount = function() {
|
||||
return clickCount;
|
||||
};
|
||||
|
||||
// Build nested structure: Open Shadow → Closed Shadow → Iframe → Final Button
|
||||
|
||||
// 1. Create Open Shadow DOM (contains everything else)
|
||||
const openShadowHost = document.getElementById('open-shadow-host');
|
||||
const openShadowRoot = openShadowHost.attachShadow({mode: 'open'});
|
||||
openShadowRoot.innerHTML = `
|
||||
<style>
|
||||
.shadow-content { padding: 15px; background: #e3f2fd; border: 2px solid #2196F3; margin: 10px 0; }
|
||||
button { padding: 10px 20px; font-size: 16px; margin: 10px 0; display: block; }
|
||||
.nested-info { font-weight: bold; color: #1976D2; }
|
||||
</style>
|
||||
<div class="shadow-content">
|
||||
<button id="open-shadow-btn">Open Shadow Button</button>
|
||||
<div id="closed-shadow-host"></div>
|
||||
</div>
|
||||
`;
|
||||
|
||||
openShadowRoot.getElementById('open-shadow-btn').addEventListener('click', function() {
|
||||
incrementCounter('Open Shadow DOM');
|
||||
});
|
||||
|
||||
// 2. Create Closed Shadow DOM INSIDE Open Shadow (nested!)
|
||||
const closedShadowHost = openShadowRoot.getElementById('closed-shadow-host');
|
||||
const closedShadowRoot = closedShadowHost.attachShadow({mode: 'closed'});
|
||||
closedShadowRoot.innerHTML = `
|
||||
<style>
|
||||
.shadow-content { padding: 15px; background: #fff3e0; border: 2px solid #FF9800; margin: 10px 0; }
|
||||
button { padding: 10px 20px; font-size: 16px; margin: 10px 0; display: block; }
|
||||
iframe { width: 100%; height: 250px; border: 2px solid #4CAF50; margin: 10px 0; }
|
||||
.nested-info { font-weight: bold; color: #F57C00; }
|
||||
.iframe-label { font-size: 14px; color: #666; margin-top: 10px; }
|
||||
</style>
|
||||
<div class="shadow-content">
|
||||
<button id="closed-shadow-btn">Closed Shadow Button</button>
|
||||
<iframe id="cross-origin-iframe" src="about:blank"></iframe>
|
||||
<iframe id="nested-iframe" src="/iframe-same-origin"></iframe>
|
||||
</div>
|
||||
`;
|
||||
|
||||
closedShadowRoot.getElementById('closed-shadow-btn').addEventListener('click', function() {
|
||||
incrementCounter('Closed Shadow DOM');
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,118 @@
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>DOM Serializer Test - Main Page</title>
|
||||
<style>
|
||||
body { font-family: Arial; padding: 20px; }
|
||||
.section { margin: 20px 0; padding: 15px; border: 1px solid #ccc; }
|
||||
#click-counter {
|
||||
position: fixed;
|
||||
top: 20px;
|
||||
right: 20px;
|
||||
background: #4CAF50;
|
||||
color: white;
|
||||
padding: 30px 50px;
|
||||
border-radius: 15px;
|
||||
font-size: 48px;
|
||||
font-weight: bold;
|
||||
box-shadow: 0 4px 20px rgba(0,0,0,0.3);
|
||||
transition: all 0.2s ease;
|
||||
z-index: 9999;
|
||||
}
|
||||
#counter-value {
|
||||
font-size: 64px;
|
||||
display: inline-block;
|
||||
min-width: 60px;
|
||||
text-align: center;
|
||||
}
|
||||
@keyframes flash {
|
||||
0% { transform: scale(1); }
|
||||
50% { transform: scale(1.3); background: #FFC107; }
|
||||
100% { transform: scale(1); }
|
||||
}
|
||||
.flash {
|
||||
animation: flash 0.3s ease;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div id="click-counter">Clicks: <span id="counter-value">0</span></div>
|
||||
<h1>DOM Serializer Test Page</h1>
|
||||
|
||||
<!-- Regular DOM elements (3 interactive elements) -->
|
||||
<div class="section">
|
||||
<h2>Regular DOM Elements</h2>
|
||||
<button id="regular-btn-1">Regular Button 1</button>
|
||||
<input type="text" id="regular-input" placeholder="Regular input" />
|
||||
<a href="#test" id="regular-link">Regular Link</a>
|
||||
</div>
|
||||
|
||||
<!-- Shadow DOM elements (3 interactive elements inside shadow) -->
|
||||
<div class="section">
|
||||
<h2>Shadow DOM Elements</h2>
|
||||
<div id="shadow-host"></div>
|
||||
</div>
|
||||
|
||||
<!-- Same-origin iframe (2 interactive elements inside) -->
|
||||
<div class="section">
|
||||
<h2>Same-Origin Iframe</h2>
|
||||
<iframe id="same-origin-iframe" src="/iframe-same-origin" style="width:100%; height:200px; border:1px solid #999;"></iframe>
|
||||
</div>
|
||||
|
||||
<!-- Cross-origin iframe placeholder (external domain removed for test isolation) -->
|
||||
<div class="section">
|
||||
<h2>Cross-Origin Iframe (Placeholder)</h2>
|
||||
<iframe id="cross-origin-iframe" src="about:blank" style="width:100%; height:200px; border:1px solid #999;"></iframe>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
// Global click counter
|
||||
let clickCount = 0;
|
||||
|
||||
function incrementCounter(source) {
|
||||
clickCount++;
|
||||
const counter = document.getElementById('click-counter');
|
||||
const counterValue = document.getElementById('counter-value');
|
||||
|
||||
counterValue.textContent = clickCount;
|
||||
console.log(`Click #${clickCount} from: ${source}`);
|
||||
|
||||
// Add flash animation
|
||||
counter.classList.remove('flash');
|
||||
void counter.offsetWidth; // Trigger reflow
|
||||
counter.classList.add('flash');
|
||||
}
|
||||
|
||||
// Expose counter for testing
|
||||
window.getClickCount = function() {
|
||||
return clickCount;
|
||||
};
|
||||
|
||||
// Add click handler to regular button using addEventListener
|
||||
document.getElementById('regular-btn-1').addEventListener('click', function() {
|
||||
incrementCounter('Regular DOM');
|
||||
});
|
||||
|
||||
// Create shadow DOM with interactive elements
|
||||
const shadowHost = document.getElementById('shadow-host');
|
||||
const shadowRoot = shadowHost.attachShadow({mode: 'open'});
|
||||
|
||||
shadowRoot.innerHTML = `
|
||||
<style>
|
||||
.shadow-content { padding: 10px; background: #f0f0f0; }
|
||||
</style>
|
||||
<div class="shadow-content">
|
||||
<p>Content inside Shadow DOM:</p>
|
||||
<button id="shadow-btn-1">Shadow Button 1</button>
|
||||
<input type="text" id="shadow-input" placeholder="Shadow input" />
|
||||
<button id="shadow-btn-2">Shadow Button 2</button>
|
||||
</div>
|
||||
`;
|
||||
|
||||
// Add click handler to shadow DOM button using addEventListener
|
||||
shadowRoot.getElementById('shadow-btn-1').addEventListener('click', function() {
|
||||
incrementCounter('Shadow DOM');
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,63 @@
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from browser_use.browser import profile as profile_module
|
||||
from browser_use.browser.profile import BrowserChannel, BrowserProfile
|
||||
|
||||
|
||||
def _create_chrome_user_data_dir(tmp_path: Path) -> Path:
|
||||
user_data_dir = tmp_path / 'Chrome User Data'
|
||||
default_profile = user_data_dir / 'Default'
|
||||
default_profile.mkdir(parents=True)
|
||||
(default_profile / 'Preferences').write_text('{"profile": "default"}')
|
||||
(user_data_dir / 'Local State').write_text('{"browser": "chrome"}')
|
||||
return user_data_dir
|
||||
|
||||
|
||||
def test_chrome_profile_copy_skips_transient_lock_files(tmp_path: Path) -> None:
|
||||
user_data_dir = _create_chrome_user_data_dir(tmp_path)
|
||||
default_profile = user_data_dir / 'Default'
|
||||
(default_profile / 'SingletonLock').write_text('locked')
|
||||
(default_profile / 'Cookies-journal').write_text('journal')
|
||||
|
||||
browser_profile = BrowserProfile(
|
||||
user_data_dir=user_data_dir,
|
||||
channel=BrowserChannel.CHROME,
|
||||
headless=True,
|
||||
)
|
||||
|
||||
assert browser_profile.user_data_dir is not None
|
||||
temp_user_data_dir = Path(browser_profile.user_data_dir)
|
||||
try:
|
||||
assert (temp_user_data_dir / 'Default' / 'Preferences').read_text() == '{"profile": "default"}'
|
||||
assert (temp_user_data_dir / 'Local State').read_text() == '{"browser": "chrome"}'
|
||||
assert not (temp_user_data_dir / 'Default' / 'SingletonLock').exists()
|
||||
assert not (temp_user_data_dir / 'Default' / 'Cookies-journal').exists()
|
||||
finally:
|
||||
shutil.rmtree(temp_user_data_dir, ignore_errors=True)
|
||||
|
||||
|
||||
def test_chrome_profile_copy_lock_error_is_actionable(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
user_data_dir = _create_chrome_user_data_dir(tmp_path)
|
||||
temp_user_data_dir = tmp_path / 'browser-use-user-data-dir-test'
|
||||
|
||||
def fake_mkdtemp(prefix: str) -> str:
|
||||
temp_user_data_dir.mkdir()
|
||||
return str(temp_user_data_dir)
|
||||
|
||||
def fake_copytree(*_args: object, **_kwargs: object) -> None:
|
||||
raise PermissionError(13, 'The process cannot access the file because it is being used by another process')
|
||||
|
||||
monkeypatch.setattr(profile_module.tempfile, 'mkdtemp', fake_mkdtemp)
|
||||
monkeypatch.setattr(shutil, 'copytree', fake_copytree)
|
||||
|
||||
with pytest.raises(RuntimeError, match='Close any Chrome windows using this profile.*--cdp-url'):
|
||||
BrowserProfile(
|
||||
user_data_dir=user_data_dir,
|
||||
channel=BrowserChannel.CHROME,
|
||||
headless=True,
|
||||
)
|
||||
|
||||
assert not temp_user_data_dir.exists()
|
||||
@@ -0,0 +1,113 @@
|
||||
import asyncio
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
|
||||
from browser_use.browser import BrowserProfile, BrowserSession
|
||||
from browser_use.browser.profile import ProxySettings
|
||||
from browser_use.config import CONFIG
|
||||
|
||||
|
||||
def test_chromium_args_include_proxy_flags():
|
||||
profile = BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=str(CONFIG.BROWSER_USE_PROFILES_DIR / 'proxy-smoke'),
|
||||
proxy=ProxySettings(
|
||||
server='http://proxy.local:8080',
|
||||
bypass='localhost,127.0.0.1',
|
||||
),
|
||||
)
|
||||
args = profile.get_args()
|
||||
assert any(a == '--proxy-server=http://proxy.local:8080' for a in args), args
|
||||
assert any(a == '--proxy-bypass-list=localhost,127.0.0.1' for a in args), args
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cdp_proxy_auth_handler_registers_and_responds():
|
||||
# Create profile with proxy auth credentials
|
||||
profile = BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=str(CONFIG.BROWSER_USE_PROFILES_DIR / 'proxy-smoke'),
|
||||
proxy=ProxySettings(username='user', password='pass'),
|
||||
)
|
||||
session = BrowserSession(browser_profile=profile)
|
||||
|
||||
# Stub CDP client with minimal Fetch support
|
||||
class StubCDP:
|
||||
def __init__(self) -> None:
|
||||
self.enabled = False
|
||||
self.last_auth: dict[str, Any] | None = None
|
||||
self.last_default: dict[str, Any] | None = None
|
||||
self.auth_callback = None
|
||||
self.request_paused_callback = None
|
||||
|
||||
class _FetchSend:
|
||||
def __init__(self, outer: 'StubCDP') -> None:
|
||||
self._outer = outer
|
||||
|
||||
async def enable(self, params: dict, session_id: str | None = None) -> None:
|
||||
self._outer.enabled = True
|
||||
|
||||
async def continueWithAuth(self, params: dict, session_id: str | None = None) -> None:
|
||||
self._outer.last_auth = {'params': params, 'session_id': session_id}
|
||||
|
||||
async def continueRequest(self, params: dict, session_id: str | None = None) -> None:
|
||||
# no-op; included to mirror CDP API surface used by impl
|
||||
pass
|
||||
|
||||
class _Send:
|
||||
def __init__(self, outer: 'StubCDP') -> None:
|
||||
self.Fetch = _FetchSend(outer)
|
||||
|
||||
class _FetchRegister:
|
||||
def __init__(self, outer: 'StubCDP') -> None:
|
||||
self._outer = outer
|
||||
|
||||
def authRequired(self, callback) -> None:
|
||||
self._outer.auth_callback = callback
|
||||
|
||||
def requestPaused(self, callback) -> None:
|
||||
self._outer.request_paused_callback = callback
|
||||
|
||||
class _Register:
|
||||
def __init__(self, outer: 'StubCDP') -> None:
|
||||
self.Fetch = _FetchRegister(outer)
|
||||
|
||||
self.send = _Send(self)
|
||||
self.register = _Register(self)
|
||||
|
||||
root = StubCDP()
|
||||
|
||||
# Attach stubs to session
|
||||
session._cdp_client_root = root # type: ignore[attr-defined]
|
||||
# No need to attach a real CDPSession; _setup_proxy_auth works with root client
|
||||
|
||||
# Should register Fetch handler and enable auth handling without raising
|
||||
await session._setup_proxy_auth()
|
||||
|
||||
assert root.enabled is True
|
||||
assert callable(root.auth_callback)
|
||||
|
||||
# Simulate proxy auth required event
|
||||
ev = {'requestId': 'r1', 'authChallenge': {'source': 'Proxy'}}
|
||||
root.auth_callback(ev, session_id='s1') # type: ignore[misc]
|
||||
|
||||
# Let scheduled task run
|
||||
await asyncio.sleep(0.05)
|
||||
|
||||
assert root.last_auth is not None
|
||||
params = root.last_auth['params']
|
||||
assert params['authChallengeResponse']['response'] == 'ProvideCredentials'
|
||||
assert params['authChallengeResponse']['username'] == 'user'
|
||||
assert params['authChallengeResponse']['password'] == 'pass'
|
||||
assert root.last_auth['session_id'] == 's1'
|
||||
|
||||
# Now simulate a non-proxy auth challenge and ensure default handling
|
||||
ev2 = {'requestId': 'r2', 'authChallenge': {'source': 'Server'}}
|
||||
root.auth_callback(ev2, session_id='s2') # type: ignore[misc]
|
||||
await asyncio.sleep(0.05)
|
||||
# After non-proxy challenge, last_auth should reflect Default response
|
||||
assert root.last_auth is not None
|
||||
params2 = root.last_auth['params']
|
||||
assert params2['requestId'] == 'r2'
|
||||
assert params2['authChallengeResponse']['response'] == 'Default'
|
||||
@@ -0,0 +1,163 @@
|
||||
import pytest
|
||||
from pytest_httpserver import HTTPServer
|
||||
|
||||
from browser_use.agent.service import Agent
|
||||
from browser_use.browser.events import NavigateToUrlEvent
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
from browser_use.browser.session import BrowserSession
|
||||
from tests.ci.conftest import create_mock_llm
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def http_server():
|
||||
"""Create and provide a test HTTP server for screenshot tests."""
|
||||
server = HTTPServer()
|
||||
server.start()
|
||||
|
||||
# Route: Page with visible content for screenshot testing
|
||||
server.expect_request('/screenshot-page').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Screenshot Test Page</title>
|
||||
<style>
|
||||
body { font-family: Arial; padding: 20px; background: #f0f0f0; }
|
||||
h1 { color: #333; font-size: 32px; }
|
||||
.content { background: white; padding: 20px; border-radius: 8px; margin: 10px 0; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<h1>Screenshot Test Page</h1>
|
||||
<div class="content">
|
||||
<p>This page is used to test screenshot capture with vision enabled.</p>
|
||||
<p>The agent should capture a screenshot when navigating to this page.</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
yield server
|
||||
server.stop()
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def base_url(http_server):
|
||||
"""Return the base URL for the test HTTP server."""
|
||||
return f'http://{http_server.host}:{http_server.port}'
|
||||
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
async def browser_session():
|
||||
session = BrowserSession(browser_profile=BrowserProfile(headless=True))
|
||||
await session.start()
|
||||
yield session
|
||||
await session.kill()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_basic_screenshots(browser_session: BrowserSession, httpserver):
|
||||
"""Navigate to a local page and ensure screenshot helpers return bytes."""
|
||||
|
||||
html = """
|
||||
<html><body><h1 id='title'>Hello</h1><p>Screenshot demo.</p></body></html>
|
||||
"""
|
||||
httpserver.expect_request('/demo').respond_with_data(html, content_type='text/html')
|
||||
url = httpserver.url_for('/demo')
|
||||
|
||||
nav = browser_session.event_bus.dispatch(NavigateToUrlEvent(url=url, new_tab=False))
|
||||
await nav
|
||||
|
||||
data = await browser_session.take_screenshot(full_page=False)
|
||||
assert data, 'Viewport screenshot returned no data'
|
||||
|
||||
element = await browser_session.screenshot_element('h1')
|
||||
assert element, 'Element screenshot returned no data'
|
||||
|
||||
|
||||
async def test_agent_screenshot_with_vision_enabled(browser_session, base_url):
|
||||
"""Test that agent captures screenshots when vision is enabled.
|
||||
|
||||
This integration test verifies that:
|
||||
1. Agent with vision=True navigates to a page
|
||||
2. After prepare_context/update message manager, screenshot is captured
|
||||
3. Screenshot is included in the agent's history state
|
||||
"""
|
||||
|
||||
# Create mock LLM actions
|
||||
actions = [
|
||||
f"""
|
||||
{{
|
||||
"thinking": "I'll navigate to the screenshot test page",
|
||||
"evaluation_previous_goal": "Starting task",
|
||||
"memory": "Navigating to page",
|
||||
"next_goal": "Navigate to test page",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{base_url}/screenshot-page",
|
||||
"new_tab": false
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
""",
|
||||
"""
|
||||
{
|
||||
"thinking": "Page loaded, completing task",
|
||||
"evaluation_previous_goal": "Page loaded",
|
||||
"memory": "Task completed",
|
||||
"next_goal": "Complete task",
|
||||
"action": [
|
||||
{
|
||||
"done": {
|
||||
"text": "Successfully navigated and captured screenshot",
|
||||
"success": true
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
]
|
||||
|
||||
mock_llm = create_mock_llm(actions=actions)
|
||||
|
||||
# Create agent with vision enabled
|
||||
agent = Agent(
|
||||
task=f'Navigate to {base_url}/screenshot-page',
|
||||
llm=mock_llm,
|
||||
browser_session=browser_session,
|
||||
use_vision=True, # Enable vision/screenshots
|
||||
)
|
||||
|
||||
# Run agent
|
||||
history = await agent.run(max_steps=2)
|
||||
|
||||
# Verify agent completed successfully
|
||||
assert len(history) >= 1, 'Agent should have completed at least 1 step'
|
||||
final_result = history.final_result()
|
||||
assert final_result is not None, 'Agent should return a final result'
|
||||
|
||||
# Verify screenshots were captured in the history
|
||||
screenshot_found = False
|
||||
for i, step in enumerate(history.history):
|
||||
# Check if browser state has screenshot path
|
||||
if step.state and hasattr(step.state, 'screenshot_path') and step.state.screenshot_path:
|
||||
screenshot_found = True
|
||||
print(f'\n✅ Step {i + 1}: Screenshot captured at {step.state.screenshot_path}')
|
||||
|
||||
# Verify screenshot file exists (it should be saved to disk)
|
||||
import os
|
||||
|
||||
assert os.path.exists(step.state.screenshot_path), f'Screenshot file should exist at {step.state.screenshot_path}'
|
||||
|
||||
# Verify screenshot file has content
|
||||
screenshot_size = os.path.getsize(step.state.screenshot_path)
|
||||
assert screenshot_size > 0, f'Screenshot file should have content, got {screenshot_size} bytes'
|
||||
print(f' Screenshot size: {screenshot_size} bytes')
|
||||
|
||||
assert screenshot_found, 'At least one screenshot should be captured when vision is enabled'
|
||||
|
||||
print('\n🎉 Integration test passed: Screenshots are captured correctly with vision enabled')
|
||||
@@ -0,0 +1,436 @@
|
||||
"""
|
||||
Test script for BrowserSession.start() method to ensure proper initialization,
|
||||
concurrency handling, and error handling.
|
||||
|
||||
Tests cover:
|
||||
- Calling .start() on a session that's already started
|
||||
- Simultaneously calling .start() from two parallel coroutines
|
||||
- Calling .start() on a session that's started but has a closed browser connection
|
||||
- Calling .close() on a session that hasn't been started yet
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
|
||||
import pytest
|
||||
|
||||
from browser_use.browser.profile import (
|
||||
BROWSERUSE_DEFAULT_CHANNEL,
|
||||
BrowserChannel,
|
||||
BrowserProfile,
|
||||
)
|
||||
from browser_use.browser.session import BrowserSession
|
||||
from browser_use.config import CONFIG
|
||||
|
||||
# Set up test logging
|
||||
logger = logging.getLogger('browser_session_start_tests')
|
||||
# logger.setLevel(logging.DEBUG)
|
||||
|
||||
|
||||
# run with pytest -k test_user_data_dir_not_allowed_to_corrupt_default_profile
|
||||
|
||||
|
||||
class TestBrowserSessionStart:
|
||||
"""Tests for BrowserSession.start() method initialization and concurrency."""
|
||||
|
||||
@pytest.fixture(scope='module')
|
||||
async def browser_profile(self):
|
||||
"""Create and provide a BrowserProfile with headless mode."""
|
||||
profile = BrowserProfile(headless=True, user_data_dir=None, keep_alive=False)
|
||||
yield profile
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
async def browser_session(self, browser_profile):
|
||||
"""Create a BrowserSession instance without starting it."""
|
||||
session = BrowserSession(browser_profile=browser_profile)
|
||||
yield session
|
||||
await session.kill()
|
||||
|
||||
async def test_start_already_started_session(self, browser_session):
|
||||
"""Test calling .start() on a session that's already started."""
|
||||
# logger.info('Testing start on already started session')
|
||||
|
||||
# Start the session for the first time
|
||||
await browser_session.start()
|
||||
assert browser_session._cdp_client_root is not None
|
||||
|
||||
# Start the session again - should return immediately without re-initialization
|
||||
await browser_session.start()
|
||||
assert browser_session._cdp_client_root is not None
|
||||
|
||||
# @pytest.mark.skip(reason="Race condition - DOMWatchdog tries to inject scripts into tab that's being closed")
|
||||
# async def test_page_lifecycle_management(self, browser_session: BrowserSession):
|
||||
# """Test session handles page lifecycle correctly."""
|
||||
# # logger.info('Testing page lifecycle management')
|
||||
|
||||
# # Start the session and get initial state
|
||||
# await browser_session.start()
|
||||
# initial_tabs = await browser_session.get_tabs()
|
||||
# initial_count = len(initial_tabs)
|
||||
|
||||
# # Get current tab info
|
||||
# current_url = await browser_session.get_current_page_url()
|
||||
# assert current_url is not None
|
||||
|
||||
# # Get current tab ID
|
||||
# current_tab_id = browser_session.agent_focus.target_id if browser_session.agent_focus else None
|
||||
# assert current_tab_id is not None
|
||||
|
||||
# # Close the current tab using the event system
|
||||
# from browser_use.browser.events import CloseTabEvent
|
||||
|
||||
# close_event = browser_session.event_bus.dispatch(CloseTabEvent(target_id=current_tab_id))
|
||||
# await close_event
|
||||
|
||||
# # Operations should still work - may create new page or use existing
|
||||
# tabs_after_close = await browser_session.get_tabs()
|
||||
# assert isinstance(tabs_after_close, list)
|
||||
|
||||
# # Create a new tab explicitly
|
||||
# event = browser_session.event_bus.dispatch(NavigateToUrlEvent(url='about:blank', new_tab=True))
|
||||
# await event
|
||||
# await event.event_result(raise_if_any=True, raise_if_none=False)
|
||||
|
||||
# # Should have at least one tab now
|
||||
# final_tabs = await browser_session.get_tabs()
|
||||
# assert len(final_tabs) >= 1
|
||||
|
||||
async def test_user_data_dir_not_allowed_to_corrupt_default_profile(self):
|
||||
"""Test user_data_dir handling for different browser channels and version mismatches."""
|
||||
# Test 1: Chromium with default user_data_dir and default channel should work fine
|
||||
session = BrowserSession(
|
||||
browser_profile=BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=CONFIG.BROWSER_USE_DEFAULT_USER_DATA_DIR,
|
||||
channel=BROWSERUSE_DEFAULT_CHANNEL, # chromium
|
||||
keep_alive=False,
|
||||
),
|
||||
)
|
||||
|
||||
try:
|
||||
await session.start()
|
||||
assert session._cdp_client_root is not None
|
||||
# Verify the user_data_dir wasn't changed
|
||||
assert session.browser_profile.user_data_dir == CONFIG.BROWSER_USE_DEFAULT_USER_DATA_DIR
|
||||
finally:
|
||||
await session.kill()
|
||||
|
||||
# Test 2: Chrome with default user_data_dir should change dir AND copy to temp
|
||||
profile2 = BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=CONFIG.BROWSER_USE_DEFAULT_USER_DATA_DIR,
|
||||
channel=BrowserChannel.CHROME,
|
||||
keep_alive=False,
|
||||
)
|
||||
|
||||
# The validator should have changed the user_data_dir to avoid corruption
|
||||
# And then _copy_profile copies it to a temp directory (Chrome only)
|
||||
assert profile2.user_data_dir != CONFIG.BROWSER_USE_DEFAULT_USER_DATA_DIR
|
||||
assert 'browser-use-user-data-dir-' in str(profile2.user_data_dir)
|
||||
|
||||
# Test 3: Edge with default user_data_dir should also change
|
||||
profile3 = BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=CONFIG.BROWSER_USE_DEFAULT_USER_DATA_DIR,
|
||||
channel=BrowserChannel.MSEDGE,
|
||||
keep_alive=False,
|
||||
)
|
||||
|
||||
assert profile3.user_data_dir != CONFIG.BROWSER_USE_DEFAULT_USER_DATA_DIR
|
||||
assert profile3.user_data_dir == CONFIG.BROWSER_USE_DEFAULT_USER_DATA_DIR.parent / 'default-msedge'
|
||||
assert 'browser-use-user-data-dir-' not in str(profile3.user_data_dir)
|
||||
|
||||
|
||||
class TestBrowserSessionReusePatterns:
|
||||
"""Tests for all browser re-use patterns documented in docs/customize/real-browser.mdx"""
|
||||
|
||||
async def test_sequential_agents_same_profile_different_browser(self, mock_llm):
|
||||
"""Test Sequential Agents, Same Profile, Different Browser pattern"""
|
||||
from browser_use import Agent
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
|
||||
# Create a reusable profile
|
||||
reused_profile = BrowserProfile(
|
||||
user_data_dir=None, # Use temp dir for testing
|
||||
headless=True,
|
||||
)
|
||||
|
||||
# First agent
|
||||
agent1 = Agent(
|
||||
task='The first task...',
|
||||
llm=mock_llm,
|
||||
browser_profile=reused_profile,
|
||||
)
|
||||
await agent1.run()
|
||||
|
||||
# Verify first agent's session is closed
|
||||
assert agent1.browser_session is not None
|
||||
assert not agent1.browser_session._cdp_client_root is not None
|
||||
|
||||
# Second agent with same profile
|
||||
agent2 = Agent(
|
||||
task='The second task...',
|
||||
llm=mock_llm,
|
||||
browser_profile=reused_profile,
|
||||
# Disable memory for tests
|
||||
)
|
||||
await agent2.run()
|
||||
|
||||
# Verify second agent created a new session
|
||||
assert agent2.browser_session is not None
|
||||
assert agent1.browser_session is not agent2.browser_session
|
||||
assert not agent2.browser_session._cdp_client_root is not None
|
||||
|
||||
async def test_sequential_agents_same_profile_same_browser(self, mock_llm):
|
||||
"""Test Sequential Agents, Same Profile, Same Browser pattern"""
|
||||
from browser_use import Agent, BrowserSession
|
||||
|
||||
# Create a reusable session with keep_alive
|
||||
reused_session = BrowserSession(
|
||||
browser_profile=BrowserProfile(
|
||||
user_data_dir=None, # Use temp dir for testing
|
||||
headless=True,
|
||||
keep_alive=True, # Don't close browser after agent.run()
|
||||
),
|
||||
)
|
||||
|
||||
try:
|
||||
# Start the session manually (agents will reuse this initialized session)
|
||||
await reused_session.start()
|
||||
|
||||
# First agent
|
||||
agent1 = Agent(
|
||||
task='The first task...',
|
||||
llm=mock_llm,
|
||||
browser_session=reused_session,
|
||||
# Disable memory for tests
|
||||
)
|
||||
await agent1.run()
|
||||
|
||||
# Verify session is still alive
|
||||
assert reused_session._cdp_client_root is not None
|
||||
|
||||
# Second agent reusing the same session
|
||||
agent2 = Agent(
|
||||
task='The second task...',
|
||||
llm=mock_llm,
|
||||
browser_session=reused_session,
|
||||
# Disable memory for tests
|
||||
)
|
||||
await agent2.run()
|
||||
|
||||
# Verify same browser was used (using __eq__ to check browser_pid, cdp_url)
|
||||
assert agent1.browser_session == agent2.browser_session
|
||||
assert agent1.browser_session == reused_session
|
||||
assert reused_session._cdp_client_root is not None
|
||||
|
||||
finally:
|
||||
await reused_session.kill()
|
||||
|
||||
|
||||
class TestBrowserSessionEventSystem:
|
||||
"""Tests for the new event system integration in BrowserSession."""
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
async def browser_session(self):
|
||||
"""Create a BrowserSession instance for event system testing."""
|
||||
profile = BrowserProfile(headless=True, user_data_dir=None, keep_alive=False)
|
||||
session = BrowserSession(browser_profile=profile)
|
||||
yield session
|
||||
await session.kill()
|
||||
|
||||
async def test_event_bus_initialization(self, browser_session):
|
||||
"""Test that event bus is properly initialized with unique name."""
|
||||
# Event bus should be created during __init__
|
||||
assert browser_session.event_bus is not None
|
||||
assert browser_session.event_bus.name.startswith('EventBus_')
|
||||
# Event bus name format may vary, just check it exists
|
||||
|
||||
async def test_event_handlers_registration(self, browser_session: BrowserSession):
|
||||
"""Test that event handlers are properly registered."""
|
||||
# Attach all watchdogs to register their handlers
|
||||
await browser_session.attach_all_watchdogs()
|
||||
|
||||
# Check that handlers are registered in the event bus
|
||||
from browser_use.browser.events import (
|
||||
BrowserStartEvent,
|
||||
BrowserStateRequestEvent,
|
||||
BrowserStopEvent,
|
||||
ClickElementEvent,
|
||||
CloseTabEvent,
|
||||
ScreenshotEvent,
|
||||
ScrollEvent,
|
||||
TypeTextEvent,
|
||||
)
|
||||
|
||||
# These event types should have handlers registered
|
||||
event_types_with_handlers = [
|
||||
BrowserStartEvent,
|
||||
BrowserStopEvent,
|
||||
ClickElementEvent,
|
||||
TypeTextEvent,
|
||||
ScrollEvent,
|
||||
CloseTabEvent,
|
||||
BrowserStateRequestEvent,
|
||||
ScreenshotEvent,
|
||||
]
|
||||
|
||||
for event_type in event_types_with_handlers:
|
||||
handlers = browser_session.event_bus.handlers.get(event_type.__name__, [])
|
||||
assert len(handlers) > 0, f'No handlers registered for {event_type.__name__}'
|
||||
|
||||
async def test_direct_event_dispatching(self, browser_session):
|
||||
"""Test direct event dispatching without using the public API."""
|
||||
from browser_use.browser.events import BrowserConnectedEvent, BrowserStartEvent
|
||||
|
||||
# Dispatch BrowserStartEvent directly
|
||||
start_event = browser_session.event_bus.dispatch(BrowserStartEvent())
|
||||
|
||||
# Wait for event to complete
|
||||
await start_event
|
||||
|
||||
# Check if BrowserConnectedEvent was dispatched
|
||||
assert browser_session._cdp_client_root is not None
|
||||
|
||||
# Check event history
|
||||
event_history = list(browser_session.event_bus.event_history.values())
|
||||
assert len(event_history) >= 2 # BrowserStartEvent + BrowserConnectedEvent + others
|
||||
|
||||
# Find the BrowserConnectedEvent in history
|
||||
started_events = [e for e in event_history if isinstance(e, BrowserConnectedEvent)]
|
||||
assert len(started_events) >= 1
|
||||
assert started_events[0].cdp_url is not None
|
||||
|
||||
async def test_event_system_error_handling(self, browser_session):
|
||||
"""Test error handling in event system."""
|
||||
from browser_use.browser.events import BrowserStartEvent
|
||||
|
||||
# Create session with invalid CDP URL to trigger error
|
||||
error_session = BrowserSession(
|
||||
browser_profile=BrowserProfile(headless=True),
|
||||
cdp_url='http://localhost:99999', # Invalid port
|
||||
)
|
||||
|
||||
try:
|
||||
# Dispatch start event directly - should trigger error handling
|
||||
start_event = error_session.event_bus.dispatch(BrowserStartEvent())
|
||||
|
||||
# The event bus catches and logs the error, but the event awaits successfully
|
||||
await start_event
|
||||
|
||||
# The session should not be initialized due to the error
|
||||
assert error_session._cdp_client_root is None, 'Session should not be initialized after connection error'
|
||||
|
||||
# Verify the error was logged in the event history (good enough for error handling test)
|
||||
assert len(error_session.event_bus.event_history) > 0, 'Event should be tracked even with errors'
|
||||
|
||||
finally:
|
||||
await error_session.kill()
|
||||
|
||||
async def test_concurrent_event_dispatching(self, browser_session: BrowserSession):
|
||||
"""Test that concurrent events are handled properly."""
|
||||
from browser_use.browser.events import ScreenshotEvent
|
||||
|
||||
# Start browser first
|
||||
await browser_session.start()
|
||||
|
||||
# Dispatch multiple events concurrently
|
||||
screenshot_event1 = browser_session.event_bus.dispatch(ScreenshotEvent())
|
||||
screenshot_event2 = browser_session.event_bus.dispatch(ScreenshotEvent())
|
||||
|
||||
# Both should complete successfully
|
||||
results = await asyncio.gather(screenshot_event1, screenshot_event2, return_exceptions=True)
|
||||
|
||||
# Check that no exceptions were raised
|
||||
for result in results:
|
||||
assert not isinstance(result, Exception), f'Event failed with: {result}'
|
||||
|
||||
# async def test_many_parallel_browser_sessions(self):
|
||||
# """Test spawning 12 parallel browser_sessions with different settings and ensure they all work"""
|
||||
# from browser_use import BrowserSession
|
||||
|
||||
# browser_sessions = []
|
||||
|
||||
# for i in range(3):
|
||||
# browser_sessions.append(
|
||||
# BrowserSession(
|
||||
# browser_profile=BrowserProfile(
|
||||
# user_data_dir=None,
|
||||
# headless=True,
|
||||
# keep_alive=True,
|
||||
# ),
|
||||
# )
|
||||
# )
|
||||
# for i in range(3):
|
||||
# browser_sessions.append(
|
||||
# BrowserSession(
|
||||
# browser_profile=BrowserProfile(
|
||||
# user_data_dir=Path(tempfile.mkdtemp(prefix=f'browseruse-tmp-{i}')),
|
||||
# headless=True,
|
||||
# keep_alive=True,
|
||||
# ),
|
||||
# )
|
||||
# )
|
||||
# for i in range(3):
|
||||
# browser_sessions.append(
|
||||
# BrowserSession(
|
||||
# browser_profile=BrowserProfile(
|
||||
# user_data_dir=None,
|
||||
# headless=True,
|
||||
# keep_alive=False,
|
||||
# ),
|
||||
# )
|
||||
# )
|
||||
# for i in range(3):
|
||||
# browser_sessions.append(
|
||||
# BrowserSession(
|
||||
# browser_profile=BrowserProfile(
|
||||
# user_data_dir=Path(tempfile.mkdtemp(prefix=f'browseruse-tmp-{i}')),
|
||||
# headless=True,
|
||||
# keep_alive=False,
|
||||
# ),
|
||||
# )
|
||||
# )
|
||||
|
||||
# print('Starting many parallel browser sessions...')
|
||||
# await asyncio.gather(*[browser_session.start() for browser_session in browser_sessions])
|
||||
|
||||
# print('Ensuring all parallel browser sessions are connected and usable...')
|
||||
# new_tab_tasks = []
|
||||
# for browser_session in browser_sessions:
|
||||
# assert browser_session._cdp_client_root is not None
|
||||
# assert browser_session._cdp_client_root is not None
|
||||
# new_tab_tasks.append(browser_session.create_new_tab('chrome://version'))
|
||||
# await asyncio.gather(*new_tab_tasks)
|
||||
|
||||
# print('killing every 3rd browser_session to test parallel shutdown')
|
||||
# kill_tasks = []
|
||||
# for i in range(0, len(browser_sessions), 3):
|
||||
# kill_tasks.append(browser_sessions[i].kill())
|
||||
# browser_sessions[i] = None
|
||||
# results = await asyncio.gather(*kill_tasks, return_exceptions=True)
|
||||
# # Check that no exceptions were raised during cleanup
|
||||
# for i, result in enumerate(results):
|
||||
# if isinstance(result, Exception):
|
||||
# print(f'Warning: Browser session kill raised exception: {type(result).__name__}: {result}')
|
||||
|
||||
# print('ensuring the remaining browser_sessions are still connected and usable')
|
||||
# new_tab_tasks = []
|
||||
# screenshot_tasks = []
|
||||
# for browser_session in filter(bool, browser_sessions):
|
||||
# assert browser_session._cdp_client_root is not None
|
||||
# assert browser_session._cdp_client_root is not None
|
||||
# new_tab_tasks.append(browser_session.create_new_tab('chrome://version'))
|
||||
# screenshot_tasks.append(browser_session.take_screenshot())
|
||||
# await asyncio.gather(*new_tab_tasks)
|
||||
# await asyncio.gather(*screenshot_tasks)
|
||||
|
||||
# kill_tasks = []
|
||||
# print('killing the remaining browser_sessions')
|
||||
# for browser_session in filter(bool, browser_sessions):
|
||||
# kill_tasks.append(browser_session.kill())
|
||||
# results = await asyncio.gather(*kill_tasks, return_exceptions=True)
|
||||
# # Check that no exceptions were raised during cleanup
|
||||
# for i, result in enumerate(results):
|
||||
# if isinstance(result, Exception):
|
||||
# print(f'Warning: Browser session kill raised exception: {type(result).__name__}: {result}')
|
||||
@@ -0,0 +1,671 @@
|
||||
"""
|
||||
Test multi-tab operations: creation, switching, closing, and background tabs.
|
||||
|
||||
Tests verify that:
|
||||
1. Agent can create multiple tabs (3) and switch between them
|
||||
2. Agent can close tabs with vision=True
|
||||
3. Agent can handle buttons that open new tabs in background
|
||||
4. Agent can continue and call done() after each tab operation
|
||||
5. Browser state doesn't timeout during background tab operations
|
||||
|
||||
All tests use:
|
||||
- max_steps=5 to allow multiple tab operations
|
||||
- 120s timeout to fail if test takes too long
|
||||
- Mock LLM to verify agent can still make decisions after tab operations
|
||||
|
||||
Usage:
|
||||
uv run pytest tests/ci/browser/test_tabs.py -v -s
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
|
||||
import pytest
|
||||
from pytest_httpserver import HTTPServer
|
||||
|
||||
from browser_use.agent.service import Agent
|
||||
from browser_use.browser import BrowserSession
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
from tests.ci.conftest import create_mock_llm
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def http_server():
|
||||
"""Create and provide a test HTTP server for tab tests."""
|
||||
server = HTTPServer()
|
||||
server.start()
|
||||
|
||||
# Route 1: Home page
|
||||
server.expect_request('/home').respond_with_data(
|
||||
'<html><head><title>Home Page</title></head><body><h1>Home Page</h1><p>This is the home page</p></body></html>',
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
# Route 2: Page 1
|
||||
server.expect_request('/page1').respond_with_data(
|
||||
'<html><head><title>Page 1</title></head><body><h1>Page 1</h1><p>First test page</p></body></html>',
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
# Route 3: Page 2
|
||||
server.expect_request('/page2').respond_with_data(
|
||||
'<html><head><title>Page 2</title></head><body><h1>Page 2</h1><p>Second test page</p></body></html>',
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
# Route 4: Page 3
|
||||
server.expect_request('/page3').respond_with_data(
|
||||
'<html><head><title>Page 3</title></head><body><h1>Page 3</h1><p>Third test page</p></body></html>',
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
# Route 5: Background tab page - has a link that opens a new tab in the background
|
||||
server.expect_request('/background-tab-test').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Background Tab Test</title></head>
|
||||
<body style="padding: 20px; font-family: Arial;">
|
||||
<h1>Background Tab Test</h1>
|
||||
<p>Click the link below to open a new tab in the background:</p>
|
||||
<a href="/page3" target="_blank" id="open-tab-link">Open New Tab (link)</a>
|
||||
<br><br>
|
||||
<button id="open-tab-btn" onclick="window.open('/page3', '_blank'); document.getElementById('status').textContent='Tab opened!'">
|
||||
Open New Tab (button)
|
||||
</button>
|
||||
<p id="status" style="margin-top: 20px; color: green;"></p>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
yield server
|
||||
server.stop()
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def base_url(http_server):
|
||||
"""Return the base URL for the test HTTP server."""
|
||||
return f'http://{http_server.host}:{http_server.port}'
|
||||
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
async def browser_session():
|
||||
"""Create a browser session for tab tests."""
|
||||
session = BrowserSession(
|
||||
browser_profile=BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
keep_alive=True,
|
||||
)
|
||||
)
|
||||
await session.start()
|
||||
yield session
|
||||
await session.kill()
|
||||
|
||||
|
||||
class TestMultiTabOperations:
|
||||
"""Test multi-tab creation, switching, and closing."""
|
||||
|
||||
async def test_create_and_switch_three_tabs(self, browser_session, base_url):
|
||||
"""Test that agent can create 3 tabs, switch between them, and call done().
|
||||
|
||||
This test verifies that browser state is retrieved between each step.
|
||||
"""
|
||||
start_time = time.time()
|
||||
|
||||
actions = [
|
||||
# Action 1: Navigate to home page
|
||||
f"""
|
||||
{{
|
||||
"thinking": "I'll start by navigating to the home page",
|
||||
"evaluation_previous_goal": "Starting task",
|
||||
"memory": "Navigating to home page",
|
||||
"next_goal": "Navigate to home page",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{base_url}/home",
|
||||
"new_tab": false
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
""",
|
||||
# Action 2: Open page1 in new tab
|
||||
f"""
|
||||
{{
|
||||
"thinking": "Now I'll open page 1 in a new tab",
|
||||
"evaluation_previous_goal": "Home page loaded",
|
||||
"memory": "Opening page 1 in new tab",
|
||||
"next_goal": "Open page 1 in new tab",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{base_url}/page1",
|
||||
"new_tab": true
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
""",
|
||||
# Action 3: Open page2 in new tab
|
||||
f"""
|
||||
{{
|
||||
"thinking": "Now I'll open page 2 in a new tab",
|
||||
"evaluation_previous_goal": "Page 1 opened in new tab",
|
||||
"memory": "Opening page 2 in new tab",
|
||||
"next_goal": "Open page 2 in new tab",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{base_url}/page2",
|
||||
"new_tab": true
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
""",
|
||||
# Action 4: Switch to first tab
|
||||
"""
|
||||
{
|
||||
"thinking": "Now I'll switch back to the first tab",
|
||||
"evaluation_previous_goal": "Page 2 opened in new tab",
|
||||
"memory": "Switching to first tab",
|
||||
"next_goal": "Switch to first tab",
|
||||
"action": [
|
||||
{
|
||||
"switch": {
|
||||
"tab_id": "0000"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
# Action 5: Done
|
||||
"""
|
||||
{
|
||||
"thinking": "I've successfully created 3 tabs and switched between them",
|
||||
"evaluation_previous_goal": "Switched to first tab",
|
||||
"memory": "All tabs created and switched",
|
||||
"next_goal": "Complete task",
|
||||
"action": [
|
||||
{
|
||||
"done": {
|
||||
"text": "Successfully created 3 tabs and switched between them",
|
||||
"success": true
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
]
|
||||
|
||||
mock_llm = create_mock_llm(actions=actions)
|
||||
|
||||
agent = Agent(
|
||||
task=f'Navigate to {base_url}/home, then open {base_url}/page1 and {base_url}/page2 in new tabs, then switch back to the first tab',
|
||||
llm=mock_llm,
|
||||
browser_session=browser_session,
|
||||
)
|
||||
|
||||
# Run with timeout - should complete within 2 minutes
|
||||
try:
|
||||
history = await asyncio.wait_for(agent.run(max_steps=5), timeout=120)
|
||||
elapsed = time.time() - start_time
|
||||
|
||||
print(f'\n⏱️ Test completed in {elapsed:.2f} seconds')
|
||||
print(f'📊 Completed {len(history)} steps')
|
||||
|
||||
# Verify each step has browser state
|
||||
for i, step in enumerate(history.history):
|
||||
assert step.state is not None, f'Step {i} should have browser state'
|
||||
assert step.state.url is not None, f'Step {i} should have URL in browser state'
|
||||
print(f' Step {i + 1}: URL={step.state.url}, tabs={len(step.state.tabs) if step.state.tabs else 0}')
|
||||
|
||||
assert len(history) >= 4, 'Agent should have completed at least 4 steps'
|
||||
|
||||
# Verify we have 3 tabs open
|
||||
tabs = await browser_session.get_tabs()
|
||||
assert len(tabs) >= 3, f'Should have at least 3 tabs open, got {len(tabs)}'
|
||||
|
||||
# Verify agent completed successfully
|
||||
final_result = history.final_result()
|
||||
assert final_result is not None, 'Agent should return a final result'
|
||||
assert 'Successfully' in final_result, 'Agent should report success'
|
||||
|
||||
# Note: Test is fast (< 1s) because mock LLM returns instantly and pages are simple,
|
||||
# but browser state IS being retrieved correctly between steps as verified above
|
||||
except TimeoutError:
|
||||
pytest.fail('Test timed out after 2 minutes - agent hung during tab operations')
|
||||
|
||||
async def test_close_tab_with_vision(self, browser_session, base_url):
|
||||
"""Test that agent can close a tab with vision=True and call done()."""
|
||||
|
||||
actions = [
|
||||
# Action 1: Navigate to home page
|
||||
f"""
|
||||
{{
|
||||
"thinking": "I'll start by navigating to the home page",
|
||||
"evaluation_previous_goal": "Starting task",
|
||||
"memory": "Navigating to home page",
|
||||
"next_goal": "Navigate to home page",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{base_url}/home",
|
||||
"new_tab": false
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
""",
|
||||
# Action 2: Open page1 in new tab
|
||||
f"""
|
||||
{{
|
||||
"thinking": "Now I'll open page 1 in a new tab",
|
||||
"evaluation_previous_goal": "Home page loaded",
|
||||
"memory": "Opening page 1 in new tab",
|
||||
"next_goal": "Open page 1 in new tab",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{base_url}/page1",
|
||||
"new_tab": true
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
""",
|
||||
# Action 3: Close the current tab
|
||||
"""
|
||||
{
|
||||
"thinking": "Now I'll close the current tab (page1)",
|
||||
"evaluation_previous_goal": "Page 1 opened in new tab",
|
||||
"memory": "Closing current tab",
|
||||
"next_goal": "Close current tab",
|
||||
"action": [
|
||||
{
|
||||
"close": {
|
||||
"tab_id": "0001"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
# Action 4: Done
|
||||
"""
|
||||
{
|
||||
"thinking": "I've successfully closed the tab",
|
||||
"evaluation_previous_goal": "Tab closed",
|
||||
"memory": "Tab closed successfully",
|
||||
"next_goal": "Complete task",
|
||||
"action": [
|
||||
{
|
||||
"done": {
|
||||
"text": "Successfully closed the tab",
|
||||
"success": true
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
]
|
||||
|
||||
mock_llm = create_mock_llm(actions=actions)
|
||||
|
||||
agent = Agent(
|
||||
task=f'Navigate to {base_url}/home, then open {base_url}/page1 in a new tab, then close the page1 tab',
|
||||
llm=mock_llm,
|
||||
browser_session=browser_session,
|
||||
use_vision=True, # Enable vision for this test
|
||||
)
|
||||
|
||||
# Run with timeout - should complete within 2 minutes
|
||||
try:
|
||||
history = await asyncio.wait_for(agent.run(max_steps=5), timeout=120)
|
||||
assert len(history) >= 3, 'Agent should have completed at least 3 steps'
|
||||
|
||||
# Verify agent completed successfully
|
||||
final_result = history.final_result()
|
||||
assert final_result is not None, 'Agent should return a final result'
|
||||
assert 'Successfully' in final_result, 'Agent should report success'
|
||||
except TimeoutError:
|
||||
pytest.fail('Test timed out after 2 minutes - agent hung during tab closing with vision')
|
||||
|
||||
async def test_background_tab_open_no_timeout(self, browser_session, base_url):
|
||||
"""Test that browser state doesn't timeout when a new tab opens in the background."""
|
||||
start_time = time.time()
|
||||
|
||||
actions = [
|
||||
# Action 1: Navigate to home page
|
||||
f"""
|
||||
{{
|
||||
"thinking": "I'll navigate to the home page first",
|
||||
"evaluation_previous_goal": "Starting task",
|
||||
"memory": "Navigating to home page",
|
||||
"next_goal": "Navigate to home page",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{base_url}/home",
|
||||
"new_tab": false
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
""",
|
||||
# Action 2: Open page1 in new background tab (stay on home page)
|
||||
f"""
|
||||
{{
|
||||
"thinking": "I'll open page1 in a new background tab",
|
||||
"evaluation_previous_goal": "Home page loaded",
|
||||
"memory": "Opening background tab",
|
||||
"next_goal": "Open background tab without switching to it",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{base_url}/page1",
|
||||
"new_tab": true
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
""",
|
||||
# Action 3: Immediately check browser state after background tab opens
|
||||
"""
|
||||
{
|
||||
"thinking": "After opening background tab, browser state should still be accessible",
|
||||
"evaluation_previous_goal": "Background tab opened",
|
||||
"memory": "Verifying browser state works",
|
||||
"next_goal": "Complete task",
|
||||
"action": [
|
||||
{
|
||||
"done": {
|
||||
"text": "Successfully opened background tab, browser state remains accessible",
|
||||
"success": true
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
]
|
||||
|
||||
mock_llm = create_mock_llm(actions=actions)
|
||||
|
||||
agent = Agent(
|
||||
task=f'Navigate to {base_url}/home and open {base_url}/page1 in a new tab',
|
||||
llm=mock_llm,
|
||||
browser_session=browser_session,
|
||||
)
|
||||
|
||||
# Run with timeout - this tests if browser state times out when new tabs open
|
||||
try:
|
||||
history = await asyncio.wait_for(agent.run(max_steps=3), timeout=120)
|
||||
elapsed = time.time() - start_time
|
||||
|
||||
print(f'\n⏱️ Test completed in {elapsed:.2f} seconds')
|
||||
print(f'📊 Completed {len(history)} steps')
|
||||
|
||||
# Verify each step has browser state (the key test - no timeouts)
|
||||
for i, step in enumerate(history.history):
|
||||
assert step.state is not None, f'Step {i} should have browser state'
|
||||
assert step.state.url is not None, f'Step {i} should have URL in browser state'
|
||||
print(f' Step {i + 1}: URL={step.state.url}, tabs={len(step.state.tabs) if step.state.tabs else 0}')
|
||||
|
||||
assert len(history) >= 2, 'Agent should have completed at least 2 steps'
|
||||
|
||||
# Verify agent completed successfully
|
||||
final_result = history.final_result()
|
||||
assert final_result is not None, 'Agent should return a final result'
|
||||
assert 'Successfully' in final_result, 'Agent should report success'
|
||||
|
||||
# Verify we have at least 2 tabs
|
||||
tabs = await browser_session.get_tabs()
|
||||
print(f' Final tab count: {len(tabs)}')
|
||||
assert len(tabs) >= 2, f'Should have at least 2 tabs after opening background tab, got {len(tabs)}'
|
||||
|
||||
except TimeoutError:
|
||||
pytest.fail('Test timed out after 2 minutes - browser state timed out after opening background tab')
|
||||
|
||||
async def test_rapid_tab_operations_no_timeout(self, browser_session, base_url):
|
||||
"""Test that browser state doesn't timeout during rapid tab operations."""
|
||||
|
||||
actions = [
|
||||
# Action 1: Navigate to home page
|
||||
f"""
|
||||
{{
|
||||
"thinking": "I'll navigate to the home page",
|
||||
"evaluation_previous_goal": "Starting task",
|
||||
"memory": "Navigating to home page",
|
||||
"next_goal": "Navigate to home page",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{base_url}/home",
|
||||
"new_tab": false
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
""",
|
||||
# Action 2: Open page1 in new tab
|
||||
f"""
|
||||
{{
|
||||
"thinking": "Opening page1 in new tab",
|
||||
"evaluation_previous_goal": "Home page loaded",
|
||||
"memory": "Opening page1",
|
||||
"next_goal": "Open page1",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{base_url}/page1",
|
||||
"new_tab": true
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
""",
|
||||
# Action 3: Open page2 in new tab
|
||||
f"""
|
||||
{{
|
||||
"thinking": "Opening page2 in new tab",
|
||||
"evaluation_previous_goal": "Page1 opened",
|
||||
"memory": "Opening page2",
|
||||
"next_goal": "Open page2",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{base_url}/page2",
|
||||
"new_tab": true
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
""",
|
||||
# Action 4: Open page3 in new tab
|
||||
f"""
|
||||
{{
|
||||
"thinking": "Opening page3 in new tab",
|
||||
"evaluation_previous_goal": "Page2 opened",
|
||||
"memory": "Opening page3",
|
||||
"next_goal": "Open page3",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{base_url}/page3",
|
||||
"new_tab": true
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
""",
|
||||
# Action 5: Verify browser state is still accessible
|
||||
"""
|
||||
{
|
||||
"thinking": "All tabs opened rapidly, browser state should still be accessible",
|
||||
"evaluation_previous_goal": "Page3 opened",
|
||||
"memory": "All tabs opened",
|
||||
"next_goal": "Complete task",
|
||||
"action": [
|
||||
{
|
||||
"done": {
|
||||
"text": "Successfully opened 4 tabs rapidly without timeout",
|
||||
"success": true
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
]
|
||||
|
||||
mock_llm = create_mock_llm(actions=actions)
|
||||
|
||||
agent = Agent(
|
||||
task='Open multiple tabs rapidly and verify browser state remains accessible',
|
||||
llm=mock_llm,
|
||||
browser_session=browser_session,
|
||||
)
|
||||
|
||||
# Run with timeout - should complete within 2 minutes
|
||||
try:
|
||||
history = await asyncio.wait_for(agent.run(max_steps=5), timeout=120)
|
||||
assert len(history) >= 4, 'Agent should have completed at least 4 steps'
|
||||
|
||||
# Verify we have 4 tabs open
|
||||
tabs = await browser_session.get_tabs()
|
||||
assert len(tabs) >= 4, f'Should have at least 4 tabs open, got {len(tabs)}'
|
||||
|
||||
# Verify agent completed successfully
|
||||
final_result = history.final_result()
|
||||
assert final_result is not None, 'Agent should return a final result'
|
||||
assert 'Successfully' in final_result, 'Agent should report success'
|
||||
except TimeoutError:
|
||||
pytest.fail('Test timed out after 2 minutes - browser state timed out during rapid tab operations')
|
||||
|
||||
async def test_multiple_tab_switches_and_close(self, browser_session, base_url):
|
||||
"""Test that agent can switch between multiple tabs and close one."""
|
||||
|
||||
actions = [
|
||||
# Action 1: Navigate to home page
|
||||
f"""
|
||||
{{
|
||||
"thinking": "I'll start by navigating to the home page",
|
||||
"evaluation_previous_goal": "Starting task",
|
||||
"memory": "Navigating to home page",
|
||||
"next_goal": "Navigate to home page",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{base_url}/home",
|
||||
"new_tab": false
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
""",
|
||||
# Action 2: Open page1 in new tab
|
||||
f"""
|
||||
{{
|
||||
"thinking": "Opening page 1 in new tab",
|
||||
"evaluation_previous_goal": "Home page loaded",
|
||||
"memory": "Opening page 1",
|
||||
"next_goal": "Open page 1",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{base_url}/page1",
|
||||
"new_tab": true
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
""",
|
||||
# Action 3: Open page2 in new tab
|
||||
f"""
|
||||
{{
|
||||
"thinking": "Opening page 2 in new tab",
|
||||
"evaluation_previous_goal": "Page 1 opened",
|
||||
"memory": "Opening page 2",
|
||||
"next_goal": "Open page 2",
|
||||
"action": [
|
||||
{{
|
||||
"navigate": {{
|
||||
"url": "{base_url}/page2",
|
||||
"new_tab": true
|
||||
}}
|
||||
}}
|
||||
]
|
||||
}}
|
||||
""",
|
||||
# Action 4: Switch to tab 1
|
||||
"""
|
||||
{
|
||||
"thinking": "Switching to tab 1 (page1)",
|
||||
"evaluation_previous_goal": "Page 2 opened",
|
||||
"memory": "Switching to page 1",
|
||||
"next_goal": "Switch to page 1",
|
||||
"action": [
|
||||
{
|
||||
"switch": {
|
||||
"tab_id": "0001"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
# Action 5: Close current tab
|
||||
"""
|
||||
{
|
||||
"thinking": "Closing the current tab (page1)",
|
||||
"evaluation_previous_goal": "Switched to page 1",
|
||||
"memory": "Closing page 1",
|
||||
"next_goal": "Close page 1",
|
||||
"action": [
|
||||
{
|
||||
"close": {
|
||||
"tab_id": "0001"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
# Action 6: Done
|
||||
"""
|
||||
{
|
||||
"thinking": "Successfully completed all tab operations",
|
||||
"evaluation_previous_goal": "Tab closed",
|
||||
"memory": "All operations completed",
|
||||
"next_goal": "Complete task",
|
||||
"action": [
|
||||
{
|
||||
"done": {
|
||||
"text": "Successfully created, switched, and closed tabs",
|
||||
"success": true
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
]
|
||||
|
||||
mock_llm = create_mock_llm(actions=actions)
|
||||
|
||||
agent = Agent(
|
||||
task='Create 3 tabs, switch to the second one, then close it',
|
||||
llm=mock_llm,
|
||||
browser_session=browser_session,
|
||||
)
|
||||
|
||||
# Run with timeout - should complete within 2 minutes
|
||||
try:
|
||||
history = await asyncio.wait_for(agent.run(max_steps=6), timeout=120)
|
||||
assert len(history) >= 5, 'Agent should have completed at least 5 steps'
|
||||
|
||||
# Verify agent completed successfully
|
||||
final_result = history.final_result()
|
||||
assert final_result is not None, 'Agent should return a final result'
|
||||
assert 'Successfully' in final_result, 'Agent should report success'
|
||||
except TimeoutError:
|
||||
pytest.fail('Test timed out after 2 minutes - agent hung during multiple tab operations')
|
||||
@@ -0,0 +1,136 @@
|
||||
"""Test clicking elements inside TRUE cross-origin iframes (external domains)."""
|
||||
|
||||
import asyncio
|
||||
|
||||
import pytest
|
||||
|
||||
from browser_use.browser.profile import BrowserProfile, ViewportSize
|
||||
from browser_use.browser.session import BrowserSession
|
||||
from browser_use.tools.service import Tools
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def browser_session():
|
||||
"""Create browser session with cross-origin iframe support."""
|
||||
session = BrowserSession(
|
||||
browser_profile=BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
keep_alive=True,
|
||||
window_size=ViewportSize(width=1920, height=1400),
|
||||
cross_origin_iframes=True, # Enable cross-origin iframe extraction
|
||||
)
|
||||
)
|
||||
await session.start()
|
||||
yield session
|
||||
await session.kill()
|
||||
|
||||
|
||||
class TestTrueCrossOriginIframeClick:
|
||||
"""Test clicking elements inside true cross-origin iframes."""
|
||||
|
||||
async def test_click_element_in_true_cross_origin_iframe(self, httpserver, browser_session: BrowserSession):
|
||||
"""Verify that elements inside TRUE cross-origin iframes (example.com) can be clicked.
|
||||
|
||||
This test uses example.com which is a real external domain, testing actual cross-origin
|
||||
iframe extraction and clicking via CDP target switching.
|
||||
"""
|
||||
|
||||
# Create main page with TRUE cross-origin iframe pointing to example.com
|
||||
main_html = """
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>True Cross-Origin Test</title></head>
|
||||
<body>
|
||||
<h1>Main Page</h1>
|
||||
<button id="main-button">Main Button</button>
|
||||
<iframe id="cross-origin" src="https://example.com" style="width: 800px; height: 600px;"></iframe>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
# Serve the main page
|
||||
httpserver.expect_request('/true-cross-origin-test').respond_with_data(main_html, content_type='text/html')
|
||||
url = httpserver.url_for('/true-cross-origin-test')
|
||||
|
||||
# Navigate to the page
|
||||
await browser_session.navigate_to(url)
|
||||
|
||||
# Wait for cross-origin iframe to load (network request)
|
||||
await asyncio.sleep(5)
|
||||
|
||||
# Get DOM state with cross-origin iframe extraction enabled
|
||||
browser_state = await browser_session.get_browser_state_summary(
|
||||
include_screenshot=False,
|
||||
include_recent_events=False,
|
||||
)
|
||||
assert browser_state.dom_state is not None
|
||||
state = browser_state.dom_state
|
||||
|
||||
print(f'\n📊 Found {len(state.selector_map)} total elements')
|
||||
|
||||
# Find elements from different targets
|
||||
targets_found = set()
|
||||
main_page_elements = []
|
||||
cross_origin_elements = []
|
||||
|
||||
for idx, element in state.selector_map.items():
|
||||
target_id = element.target_id
|
||||
targets_found.add(target_id)
|
||||
|
||||
# Check if element is from cross-origin iframe (example.com)
|
||||
# Look for links - example.com has a link to iana.org/domains/reserved
|
||||
if element.attributes:
|
||||
href = element.attributes.get('href', '')
|
||||
element_id = element.attributes.get('id', '')
|
||||
|
||||
# example.com has a link to iana.org/domains/reserved
|
||||
if 'iana.org' in href:
|
||||
cross_origin_elements.append((idx, element))
|
||||
print(f' ✅ Found cross-origin element: [{idx}] {element.tag_name} href={href}')
|
||||
elif element_id == 'main-button':
|
||||
main_page_elements.append((idx, element))
|
||||
|
||||
# Verify we found elements from at least 2 different targets
|
||||
print(f'\n🎯 Found elements from {len(targets_found)} different CDP targets')
|
||||
|
||||
# Check if cross-origin iframe loaded
|
||||
if len(targets_found) < 2:
|
||||
print('⚠️ Warning: Cross-origin iframe did not create separate CDP target')
|
||||
print(' This may indicate cross_origin_iframes feature is not working as expected')
|
||||
pytest.skip('Cross-origin iframe did not create separate CDP target - skipping test')
|
||||
|
||||
if len(cross_origin_elements) == 0:
|
||||
print('⚠️ Warning: No elements found from example.com iframe')
|
||||
print(' Network may be restricted in CI environment')
|
||||
pytest.skip('No elements extracted from example.com - skipping click test')
|
||||
|
||||
# Verify we found at least one element from the cross-origin iframe
|
||||
assert len(cross_origin_elements) > 0, 'Expected to find at least one element from cross-origin iframe (example.com)'
|
||||
|
||||
# Try clicking the cross-origin element
|
||||
print('\n🖱️ Testing Click on True Cross-Origin Iframe Element:')
|
||||
tools = Tools()
|
||||
|
||||
link_idx, link_element = cross_origin_elements[0]
|
||||
print(f' Attempting to click element [{link_idx}] from example.com iframe...')
|
||||
|
||||
try:
|
||||
result = await tools.click(index=link_idx, browser_session=browser_session)
|
||||
|
||||
# Check for errors
|
||||
if result.error:
|
||||
pytest.fail(f'Click on cross-origin element [{link_idx}] failed with error: {result.error}')
|
||||
|
||||
if result.extracted_content and (
|
||||
'not available' in result.extracted_content.lower() or 'failed' in result.extracted_content.lower()
|
||||
):
|
||||
pytest.fail(f'Click on cross-origin element [{link_idx}] failed: {result.extracted_content}')
|
||||
|
||||
print(f' ✅ Click succeeded on cross-origin element [{link_idx}]!')
|
||||
print(' 🎉 True cross-origin iframe element clicking works!')
|
||||
|
||||
except Exception as e:
|
||||
pytest.fail(f'Exception while clicking cross-origin element [{link_idx}]: {e}')
|
||||
|
||||
print('\n✅ Test passed: True cross-origin iframe elements can be clicked')
|
||||
@@ -0,0 +1,238 @@
|
||||
"""
|
||||
Pytest configuration for browser-use CI tests.
|
||||
|
||||
Sets up environment variables to ensure tests never connect to production services.
|
||||
"""
|
||||
|
||||
import os
|
||||
import socketserver
|
||||
import tempfile
|
||||
from unittest.mock import AsyncMock
|
||||
|
||||
import pytest
|
||||
from dotenv import load_dotenv
|
||||
from pytest_httpserver import HTTPServer
|
||||
|
||||
# Fix for httpserver hanging on shutdown - prevent blocking on socket close
|
||||
# This prevents tests from hanging when shutting down HTTP servers
|
||||
socketserver.ThreadingMixIn.block_on_close = False
|
||||
# Also set daemon threads to prevent hanging
|
||||
socketserver.ThreadingMixIn.daemon_threads = True
|
||||
|
||||
from browser_use.agent.views import AgentOutput
|
||||
from browser_use.llm import BaseChatModel
|
||||
from browser_use.llm.views import ChatInvokeCompletion
|
||||
from browser_use.tools.service import Tools
|
||||
|
||||
# Load environment variables before any imports
|
||||
load_dotenv()
|
||||
|
||||
|
||||
# Skip LLM API key verification for tests
|
||||
os.environ['SKIP_LLM_API_KEY_VERIFICATION'] = 'true'
|
||||
|
||||
from bubus import BaseEvent
|
||||
|
||||
from browser_use import Agent
|
||||
from browser_use.browser import BrowserProfile, BrowserSession
|
||||
from browser_use.sync.service import CloudSync
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def setup_test_environment():
|
||||
"""
|
||||
Automatically set up test environment for all tests.
|
||||
"""
|
||||
|
||||
# Create a temporary directory for test config (but not for extensions)
|
||||
config_dir = tempfile.mkdtemp(prefix='browseruse_tests_')
|
||||
|
||||
original_env = {}
|
||||
test_env_vars = {
|
||||
'SKIP_LLM_API_KEY_VERIFICATION': 'true',
|
||||
'ANONYMIZED_TELEMETRY': 'false',
|
||||
'BROWSER_USE_CLOUD_SYNC': 'true',
|
||||
'BROWSER_USE_CLOUD_API_URL': 'http://placeholder-will-be-replaced-by-specific-test-fixtures',
|
||||
'BROWSER_USE_CLOUD_UI_URL': 'http://placeholder-will-be-replaced-by-specific-test-fixtures',
|
||||
# Don't set BROWSER_USE_CONFIG_DIR anymore - let it use the default ~/.config/browseruse
|
||||
# This way extensions will be cached in ~/.config/browseruse/extensions
|
||||
}
|
||||
|
||||
for key, value in test_env_vars.items():
|
||||
original_env[key] = os.environ.get(key)
|
||||
os.environ[key] = value
|
||||
|
||||
yield
|
||||
|
||||
# Restore original environment
|
||||
for key, value in original_env.items():
|
||||
if value is None:
|
||||
os.environ.pop(key, None)
|
||||
else:
|
||||
os.environ[key] = value
|
||||
|
||||
|
||||
# not a fixture, mock_llm() provides this in a fixture below, this is a helper so that it can accept args
|
||||
def create_mock_llm(actions: list[str] | None = None) -> BaseChatModel:
|
||||
"""Create a mock LLM that returns specified actions or a default done action.
|
||||
|
||||
Args:
|
||||
actions: Optional list of JSON strings representing actions to return in sequence.
|
||||
If not provided, returns a single done action.
|
||||
After all actions are exhausted, returns a done action.
|
||||
|
||||
Returns:
|
||||
Mock LLM that will return the actions in order, or just a done action if no actions provided.
|
||||
"""
|
||||
tools = Tools()
|
||||
ActionModel = tools.registry.create_action_model()
|
||||
AgentOutputWithActions = AgentOutput.type_with_custom_actions(ActionModel)
|
||||
|
||||
llm = AsyncMock(spec=BaseChatModel)
|
||||
llm.model = 'mock-llm'
|
||||
llm._verified_api_keys = True
|
||||
|
||||
# Add missing properties from BaseChatModel protocol
|
||||
llm.provider = 'mock'
|
||||
llm.name = 'mock-llm'
|
||||
llm.model_name = 'mock-llm' # Ensure this returns a string, not a mock
|
||||
|
||||
# Default done action
|
||||
default_done_action = """
|
||||
{
|
||||
"thinking": "null",
|
||||
"evaluation_previous_goal": "Successfully completed the task",
|
||||
"memory": "Task completed",
|
||||
"next_goal": "Task completed",
|
||||
"action": [
|
||||
{
|
||||
"done": {
|
||||
"text": "Task completed successfully",
|
||||
"success": true
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
"""
|
||||
|
||||
# Unified logic for both cases
|
||||
action_index = 0
|
||||
|
||||
def get_next_action() -> str:
|
||||
nonlocal action_index
|
||||
if actions is not None and action_index < len(actions):
|
||||
action = actions[action_index]
|
||||
action_index += 1
|
||||
return action
|
||||
else:
|
||||
return default_done_action
|
||||
|
||||
async def mock_ainvoke(*args, **kwargs):
|
||||
# Check if output_format is provided (2nd argument or in kwargs)
|
||||
output_format = None
|
||||
if len(args) >= 2:
|
||||
output_format = args[1]
|
||||
elif 'output_format' in kwargs:
|
||||
output_format = kwargs['output_format']
|
||||
|
||||
action_json = get_next_action()
|
||||
|
||||
if output_format is None:
|
||||
# Return string completion
|
||||
return ChatInvokeCompletion(completion=action_json, usage=None)
|
||||
else:
|
||||
# Parse with provided output_format (could be AgentOutputWithActions or another model)
|
||||
if output_format == AgentOutputWithActions:
|
||||
parsed = AgentOutputWithActions.model_validate_json(action_json)
|
||||
else:
|
||||
# For other output formats, try to parse the JSON with that model
|
||||
parsed = output_format.model_validate_json(action_json)
|
||||
return ChatInvokeCompletion(completion=parsed, usage=None)
|
||||
|
||||
llm.ainvoke.side_effect = mock_ainvoke
|
||||
|
||||
return llm
|
||||
|
||||
|
||||
@pytest.fixture(scope='module')
|
||||
async def browser_session():
|
||||
"""Create a real browser session for testing"""
|
||||
session = BrowserSession(
|
||||
browser_profile=BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=None, # Use temporary directory
|
||||
keep_alive=True,
|
||||
enable_default_extensions=True, # Enable extensions during tests
|
||||
)
|
||||
)
|
||||
await session.start()
|
||||
yield session
|
||||
await session.kill()
|
||||
# Ensure event bus is properly stopped
|
||||
await session.event_bus.stop(clear=True, timeout=5)
|
||||
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
def cloud_sync(httpserver: HTTPServer):
|
||||
"""
|
||||
Create a CloudSync instance configured for testing.
|
||||
|
||||
This fixture creates a real CloudSync instance and sets up the test environment
|
||||
to use the httpserver URLs.
|
||||
"""
|
||||
|
||||
# Set up test environment
|
||||
test_http_server_url = httpserver.url_for('')
|
||||
os.environ['BROWSER_USE_CLOUD_API_URL'] = test_http_server_url
|
||||
os.environ['BROWSER_USE_CLOUD_UI_URL'] = test_http_server_url
|
||||
os.environ['BROWSER_USE_CLOUD_SYNC'] = 'true'
|
||||
|
||||
# Create CloudSync with test server URL
|
||||
cloud_sync = CloudSync(
|
||||
base_url=test_http_server_url,
|
||||
)
|
||||
|
||||
return cloud_sync
|
||||
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
def mock_llm():
|
||||
"""Create a mock LLM that just returns the done action if queried"""
|
||||
return create_mock_llm(actions=None)
|
||||
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
def agent_with_cloud(browser_session, mock_llm, cloud_sync):
|
||||
"""Create agent (cloud_sync parameter removed)."""
|
||||
agent = Agent(
|
||||
task='Test task',
|
||||
llm=mock_llm,
|
||||
browser_session=browser_session,
|
||||
)
|
||||
return agent
|
||||
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
def event_collector():
|
||||
"""Helper to collect all events emitted during tests"""
|
||||
events = []
|
||||
event_order = []
|
||||
|
||||
class EventCollector:
|
||||
def __init__(self):
|
||||
self.events = events
|
||||
self.event_order = event_order
|
||||
|
||||
async def collect_event(self, event: BaseEvent):
|
||||
self.events.append(event)
|
||||
self.event_order.append(event.event_type)
|
||||
return 'collected'
|
||||
|
||||
def get_events_by_type(self, event_type: str) -> list[BaseEvent]:
|
||||
return [e for e in self.events if e.event_type == event_type]
|
||||
|
||||
def clear(self):
|
||||
self.events.clear()
|
||||
self.event_order.clear()
|
||||
|
||||
return EventCollector()
|
||||
@@ -0,0 +1,372 @@
|
||||
"""
|
||||
Runs all agent tasks in parallel (up to 10 at a time) using separate subprocesses.
|
||||
Each task gets its own Python process, preventing browser session interference.
|
||||
Fails with exit code 1 if 0% of tasks pass.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import glob
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
import warnings
|
||||
|
||||
import anyio
|
||||
import yaml
|
||||
from dotenv import load_dotenv
|
||||
from pydantic import BaseModel
|
||||
|
||||
load_dotenv()
|
||||
from browser_use import Agent, AgentHistoryList, BrowserProfile, BrowserSession, ChatBrowserUse
|
||||
from browser_use.llm.google.chat import ChatGoogle
|
||||
from browser_use.llm.messages import UserMessage
|
||||
|
||||
# --- CONFIG ---
|
||||
MAX_PARALLEL = 10
|
||||
TASK_DIR = (
|
||||
sys.argv[1]
|
||||
if len(sys.argv) > 1 and not sys.argv[1].startswith('--')
|
||||
else os.path.join(os.path.dirname(__file__), '../agent_tasks')
|
||||
)
|
||||
TASK_FILES = glob.glob(os.path.join(TASK_DIR, '*.yaml'))
|
||||
|
||||
|
||||
class JudgeResponse(BaseModel):
|
||||
success: bool
|
||||
explanation: str
|
||||
|
||||
|
||||
async def run_single_task(task_file):
|
||||
"""Run a single task in the current process (called by subprocess)"""
|
||||
try:
|
||||
print(f'[DEBUG] Starting task: {os.path.basename(task_file)}', file=sys.stderr)
|
||||
|
||||
# Suppress all logging in subprocess to avoid interfering with JSON output
|
||||
logging.getLogger().setLevel(logging.CRITICAL)
|
||||
for logger_name in ['browser_use', 'telemetry', 'message_manager']:
|
||||
logging.getLogger(logger_name).setLevel(logging.CRITICAL)
|
||||
warnings.filterwarnings('ignore')
|
||||
|
||||
print('[DEBUG] Loading task file...', file=sys.stderr)
|
||||
content = await anyio.Path(task_file).read_text()
|
||||
task_data = yaml.safe_load(content)
|
||||
task = task_data['task']
|
||||
judge_context = task_data.get('judge_context', ['The agent must solve the task'])
|
||||
max_steps = task_data.get('max_steps', 15)
|
||||
|
||||
print(f'[DEBUG] Task: {task[:100]}...', file=sys.stderr)
|
||||
print(f'[DEBUG] Max steps: {max_steps}', file=sys.stderr)
|
||||
api_key = os.getenv('BROWSER_USE_API_KEY')
|
||||
if not api_key:
|
||||
print('[SKIP] BROWSER_USE_API_KEY is not set - skipping task evaluation', file=sys.stderr)
|
||||
return {
|
||||
'file': os.path.basename(task_file),
|
||||
'success': True, # Mark as success so it doesn't fail CI
|
||||
'explanation': 'Skipped - API key not available (fork PR or missing secret)',
|
||||
}
|
||||
|
||||
agent_llm = ChatBrowserUse(api_key=api_key)
|
||||
|
||||
# Check if Google API key is available for judge LLM
|
||||
google_api_key = os.getenv('GOOGLE_API_KEY')
|
||||
if not google_api_key:
|
||||
print('[SKIP] GOOGLE_API_KEY is not set - skipping task evaluation', file=sys.stderr)
|
||||
return {
|
||||
'file': os.path.basename(task_file),
|
||||
'success': True, # Mark as success so it doesn't fail CI
|
||||
'explanation': 'Skipped - Google API key not available (fork PR or missing secret)',
|
||||
}
|
||||
|
||||
judge_llm = ChatGoogle(model='gemini-3.1-flash-lite')
|
||||
print('[DEBUG] LLMs initialized', file=sys.stderr)
|
||||
|
||||
# Each subprocess gets its own profile and session
|
||||
print('[DEBUG] Creating browser session...', file=sys.stderr)
|
||||
profile = BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
chromium_sandbox=False, # Disable sandbox for CI environment (GitHub Actions)
|
||||
)
|
||||
session = BrowserSession(browser_profile=profile)
|
||||
print('[DEBUG] Browser session created', file=sys.stderr)
|
||||
|
||||
# Test if browser is working
|
||||
try:
|
||||
await session.start()
|
||||
from browser_use.browser.events import NavigateToUrlEvent
|
||||
|
||||
event = session.event_bus.dispatch(NavigateToUrlEvent(url='https://httpbin.org/get', new_tab=True))
|
||||
await event
|
||||
print('[DEBUG] Browser test: navigation successful', file=sys.stderr)
|
||||
title = await session.get_current_page_title()
|
||||
print(f"[DEBUG] Browser test: got title '{title}'", file=sys.stderr)
|
||||
except Exception as browser_error:
|
||||
print(f'[DEBUG] Browser test failed: {str(browser_error)}', file=sys.stderr)
|
||||
print(
|
||||
f'[DEBUG] Browser error type: {type(browser_error).__name__}',
|
||||
file=sys.stderr,
|
||||
)
|
||||
|
||||
print('[DEBUG] Starting agent execution...', file=sys.stderr)
|
||||
agent = Agent(task=task, llm=agent_llm, browser_session=session)
|
||||
|
||||
try:
|
||||
history: AgentHistoryList = await agent.run(max_steps=max_steps)
|
||||
print('[DEBUG] Agent.run() returned successfully', file=sys.stderr)
|
||||
except Exception as agent_error:
|
||||
print(
|
||||
f'[DEBUG] Agent.run() failed with error: {str(agent_error)}',
|
||||
file=sys.stderr,
|
||||
)
|
||||
print(f'[DEBUG] Error type: {type(agent_error).__name__}', file=sys.stderr)
|
||||
# Re-raise to be caught by outer try-catch
|
||||
raise agent_error
|
||||
|
||||
agent_output = history.final_result() or ''
|
||||
print('[DEBUG] Agent execution completed', file=sys.stderr)
|
||||
|
||||
# Test if LLM is working by making a simple call
|
||||
try:
|
||||
response = await agent_llm.ainvoke([UserMessage(content="Say 'test'")])
|
||||
print(
|
||||
f'[DEBUG] LLM test call successful: {response.completion[:50]}',
|
||||
file=sys.stderr,
|
||||
)
|
||||
except Exception as llm_error:
|
||||
print(f'[DEBUG] LLM test call failed: {str(llm_error)}', file=sys.stderr)
|
||||
|
||||
# Debug: capture more details about the agent execution
|
||||
total_steps = len(history.history) if hasattr(history, 'history') else 0
|
||||
last_action = history.history[-1] if hasattr(history, 'history') and history.history else None
|
||||
debug_info = f'Steps: {total_steps}, Final result length: {len(agent_output)}'
|
||||
if last_action:
|
||||
debug_info += f', Last action: {type(last_action).__name__}'
|
||||
|
||||
# Log to stderr so it shows up in GitHub Actions (won't interfere with JSON output to stdout)
|
||||
print(f'[DEBUG] Task {os.path.basename(task_file)}: {debug_info}', file=sys.stderr)
|
||||
if agent_output:
|
||||
print(
|
||||
f'[DEBUG] Agent output preview: {agent_output[:200]}...',
|
||||
file=sys.stderr,
|
||||
)
|
||||
else:
|
||||
print('[DEBUG] Agent produced no output!', file=sys.stderr)
|
||||
|
||||
criteria = '\n- '.join(judge_context)
|
||||
judge_prompt = f"""
|
||||
You are a evaluator of a browser agent task inside a ci/cd pipeline. Here was the agent's task:
|
||||
{task}
|
||||
|
||||
Here is the agent's output:
|
||||
{agent_output if agent_output else '[No output provided]'}
|
||||
|
||||
Debug info: {debug_info}
|
||||
|
||||
Criteria for success:
|
||||
- {criteria}
|
||||
|
||||
Reply in JSON with keys: success (true/false), explanation (string).
|
||||
If the agent provided no output, explain what might have gone wrong.
|
||||
"""
|
||||
response = await judge_llm.ainvoke([UserMessage(content=judge_prompt)], output_format=JudgeResponse)
|
||||
judge_response = response.completion
|
||||
|
||||
result = {
|
||||
'file': os.path.basename(task_file),
|
||||
'success': judge_response.success,
|
||||
'explanation': judge_response.explanation,
|
||||
}
|
||||
|
||||
# Clean up session before returning
|
||||
await session.kill()
|
||||
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
# Ensure session cleanup even on error
|
||||
try:
|
||||
await session.kill()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return {
|
||||
'file': os.path.basename(task_file),
|
||||
'success': False,
|
||||
'explanation': f'Task failed with error: {str(e)}',
|
||||
}
|
||||
|
||||
|
||||
async def run_task_subprocess(task_file, semaphore):
|
||||
"""Run a task in a separate subprocess"""
|
||||
async with semaphore:
|
||||
try:
|
||||
# Set environment to reduce noise in subprocess
|
||||
env = os.environ.copy()
|
||||
env['PYTHONPATH'] = os.pathsep.join(sys.path)
|
||||
|
||||
proc = await asyncio.create_subprocess_exec(
|
||||
sys.executable,
|
||||
__file__,
|
||||
'--task',
|
||||
task_file,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE,
|
||||
env=env,
|
||||
)
|
||||
stdout, stderr = await proc.communicate()
|
||||
|
||||
if proc.returncode == 0:
|
||||
try:
|
||||
# Parse JSON result from subprocess
|
||||
stdout_text = stdout.decode().strip()
|
||||
stderr_text = stderr.decode().strip()
|
||||
|
||||
# Display subprocess debug logs
|
||||
if stderr_text:
|
||||
print(f'[SUBPROCESS {os.path.basename(task_file)}] Debug output:')
|
||||
for line in stderr_text.split('\n'):
|
||||
if line.strip():
|
||||
print(f' {line}')
|
||||
|
||||
# Find the JSON line (should be the last line that starts with {)
|
||||
lines = stdout_text.split('\n')
|
||||
json_line = None
|
||||
for line in reversed(lines):
|
||||
line = line.strip()
|
||||
if line.startswith('{') and line.endswith('}'):
|
||||
json_line = line
|
||||
break
|
||||
|
||||
if json_line:
|
||||
result = json.loads(json_line)
|
||||
print(f'[PARENT] Task {os.path.basename(task_file)} completed: {result["success"]}')
|
||||
else:
|
||||
raise ValueError(f'No JSON found in output: {stdout_text}')
|
||||
|
||||
except (json.JSONDecodeError, ValueError) as e:
|
||||
result = {
|
||||
'file': os.path.basename(task_file),
|
||||
'success': False,
|
||||
'explanation': f'Failed to parse subprocess result: {str(e)[:100]}',
|
||||
}
|
||||
print(f'[PARENT] Task {os.path.basename(task_file)} failed to parse: {str(e)}')
|
||||
print(f'[PARENT] Full stdout was: {stdout.decode()[:500]}')
|
||||
else:
|
||||
stderr_text = stderr.decode().strip()
|
||||
result = {
|
||||
'file': os.path.basename(task_file),
|
||||
'success': False,
|
||||
'explanation': f'Subprocess failed (code {proc.returncode}): {stderr_text[:200]}',
|
||||
}
|
||||
print(f'[PARENT] Task {os.path.basename(task_file)} subprocess failed with code {proc.returncode}')
|
||||
if stderr_text:
|
||||
print(f'[PARENT] stderr: {stderr_text[:1000]}')
|
||||
stdout_text = stdout.decode().strip()
|
||||
if stdout_text:
|
||||
print(f'[PARENT] stdout: {stdout_text[:1000]}')
|
||||
except Exception as e:
|
||||
result = {
|
||||
'file': os.path.basename(task_file),
|
||||
'success': False,
|
||||
'explanation': f'Failed to start subprocess: {str(e)}',
|
||||
}
|
||||
print(f'[PARENT] Failed to start subprocess for {os.path.basename(task_file)}: {str(e)}')
|
||||
|
||||
return result
|
||||
|
||||
|
||||
async def main():
|
||||
"""Run all tasks in parallel using subprocesses"""
|
||||
semaphore = asyncio.Semaphore(MAX_PARALLEL)
|
||||
|
||||
print(f'Found task files: {TASK_FILES}')
|
||||
|
||||
if not TASK_FILES:
|
||||
print('No task files found!')
|
||||
return 0, 0
|
||||
|
||||
# Run all tasks in parallel subprocesses
|
||||
tasks = [run_task_subprocess(task_file, semaphore) for task_file in TASK_FILES]
|
||||
results = await asyncio.gather(*tasks)
|
||||
|
||||
passed = sum(1 for r in results if r['success'])
|
||||
total = len(results)
|
||||
|
||||
print('\n' + '=' * 60)
|
||||
print(f'{"RESULTS":^60}\n')
|
||||
|
||||
# Prepare table data
|
||||
headers = ['Task', 'Success', 'Reason']
|
||||
rows = []
|
||||
for r in results:
|
||||
status = '✅' if r['success'] else '❌'
|
||||
rows.append([r['file'], status, r['explanation']])
|
||||
|
||||
# Calculate column widths
|
||||
col_widths = [max(len(str(row[i])) for row in ([headers] + rows)) for i in range(3)]
|
||||
|
||||
# Print header
|
||||
header_row = ' | '.join(headers[i].ljust(col_widths[i]) for i in range(3))
|
||||
print(header_row)
|
||||
print('-+-'.join('-' * w for w in col_widths))
|
||||
|
||||
# Print rows
|
||||
for row in rows:
|
||||
print(' | '.join(str(row[i]).ljust(col_widths[i]) for i in range(3)))
|
||||
|
||||
print('\n' + '=' * 60)
|
||||
print(f'\n{"SCORE":^60}')
|
||||
print(f'\n{"=" * 60}\n')
|
||||
print(f'\n{"*" * 10} {passed}/{total} PASSED {"*" * 10}\n')
|
||||
print('=' * 60 + '\n')
|
||||
|
||||
# Output results for GitHub Actions
|
||||
print(f'PASSED={passed}')
|
||||
print(f'TOTAL={total}')
|
||||
|
||||
# Output detailed results as JSON for GitHub Actions
|
||||
detailed_results = []
|
||||
for r in results:
|
||||
detailed_results.append(
|
||||
{
|
||||
'task': r['file'].replace('.yaml', ''),
|
||||
'success': r['success'],
|
||||
'reason': r['explanation'],
|
||||
}
|
||||
)
|
||||
|
||||
print('DETAILED_RESULTS=' + json.dumps(detailed_results))
|
||||
|
||||
return passed, total
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument('--task', type=str, help='Path to a single task YAML file (for subprocess mode)')
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.task:
|
||||
# Subprocess mode: run a single task and output ONLY JSON
|
||||
try:
|
||||
result = asyncio.run(run_single_task(args.task))
|
||||
# Output ONLY the JSON result, nothing else
|
||||
print(json.dumps(result))
|
||||
except Exception as e:
|
||||
# Even on critical failure, output valid JSON
|
||||
error_result = {
|
||||
'file': os.path.basename(args.task),
|
||||
'success': False,
|
||||
'explanation': f'Critical subprocess error: {str(e)}',
|
||||
}
|
||||
print(json.dumps(error_result))
|
||||
else:
|
||||
# Parent process mode: run all tasks in parallel subprocesses
|
||||
passed, total = asyncio.run(main())
|
||||
# Results already printed by main() function
|
||||
|
||||
# Fail if 0% pass rate (all tasks failed)
|
||||
if total > 0 and passed == 0:
|
||||
print('\n❌ CRITICAL: 0% pass rate - all tasks failed!')
|
||||
sys.exit(1)
|
||||
@@ -0,0 +1,120 @@
|
||||
"""Tests for lazy loading configuration system."""
|
||||
|
||||
import os
|
||||
|
||||
from browser_use.config import CONFIG
|
||||
|
||||
|
||||
class TestLazyConfig:
|
||||
"""Test lazy loading of environment variables through CONFIG object."""
|
||||
|
||||
def test_config_reads_env_vars_lazily(self):
|
||||
"""Test that CONFIG reads environment variables each time they're accessed."""
|
||||
# Set an env var
|
||||
original_value = os.environ.get('BROWSER_USE_LOGGING_LEVEL', '')
|
||||
try:
|
||||
os.environ['BROWSER_USE_LOGGING_LEVEL'] = 'debug'
|
||||
assert CONFIG.BROWSER_USE_LOGGING_LEVEL == 'debug'
|
||||
|
||||
# Change the env var
|
||||
os.environ['BROWSER_USE_LOGGING_LEVEL'] = 'info'
|
||||
assert CONFIG.BROWSER_USE_LOGGING_LEVEL == 'info'
|
||||
|
||||
# Delete the env var to test default
|
||||
del os.environ['BROWSER_USE_LOGGING_LEVEL']
|
||||
assert CONFIG.BROWSER_USE_LOGGING_LEVEL == 'info' # default value
|
||||
finally:
|
||||
# Restore original value
|
||||
if original_value:
|
||||
os.environ['BROWSER_USE_LOGGING_LEVEL'] = original_value
|
||||
else:
|
||||
os.environ.pop('BROWSER_USE_LOGGING_LEVEL', None)
|
||||
|
||||
def test_boolean_env_vars(self):
|
||||
"""Test boolean environment variables are parsed correctly."""
|
||||
original_value = os.environ.get('ANONYMIZED_TELEMETRY', '')
|
||||
try:
|
||||
# Test true values
|
||||
for true_val in ['true', 'True', 'TRUE', 'yes', 'Yes', '1']:
|
||||
os.environ['ANONYMIZED_TELEMETRY'] = true_val
|
||||
assert CONFIG.ANONYMIZED_TELEMETRY is True, f'Failed for value: {true_val}'
|
||||
|
||||
# Test false values
|
||||
for false_val in ['false', 'False', 'FALSE', 'no', 'No', '0']:
|
||||
os.environ['ANONYMIZED_TELEMETRY'] = false_val
|
||||
assert CONFIG.ANONYMIZED_TELEMETRY is False, f'Failed for value: {false_val}'
|
||||
finally:
|
||||
if original_value:
|
||||
os.environ['ANONYMIZED_TELEMETRY'] = original_value
|
||||
else:
|
||||
os.environ.pop('ANONYMIZED_TELEMETRY', None)
|
||||
|
||||
def test_api_keys_lazy_loading(self):
|
||||
"""Test API keys are loaded lazily."""
|
||||
original_value = os.environ.get('OPENAI_API_KEY', '')
|
||||
try:
|
||||
# Test empty default
|
||||
os.environ.pop('OPENAI_API_KEY', None)
|
||||
assert CONFIG.OPENAI_API_KEY == ''
|
||||
|
||||
# Set a value
|
||||
os.environ['OPENAI_API_KEY'] = 'test-key-123'
|
||||
assert CONFIG.OPENAI_API_KEY == 'test-key-123'
|
||||
|
||||
# Change the value
|
||||
os.environ['OPENAI_API_KEY'] = 'new-key-456'
|
||||
assert CONFIG.OPENAI_API_KEY == 'new-key-456'
|
||||
finally:
|
||||
if original_value:
|
||||
os.environ['OPENAI_API_KEY'] = original_value
|
||||
else:
|
||||
os.environ.pop('OPENAI_API_KEY', None)
|
||||
|
||||
def test_path_configuration(self):
|
||||
"""Test path configuration variables."""
|
||||
original_value = os.environ.get('XDG_CACHE_HOME', '')
|
||||
try:
|
||||
# Test custom path
|
||||
test_path = '/tmp/test-cache'
|
||||
os.environ['XDG_CACHE_HOME'] = test_path
|
||||
# Use Path().resolve() to handle symlinks (e.g., /tmp -> /private/tmp on macOS)
|
||||
from pathlib import Path
|
||||
|
||||
assert CONFIG.XDG_CACHE_HOME == Path(test_path).resolve()
|
||||
|
||||
# Test default path expansion
|
||||
os.environ.pop('XDG_CACHE_HOME', None)
|
||||
assert '/.cache' in str(CONFIG.XDG_CACHE_HOME)
|
||||
finally:
|
||||
if original_value:
|
||||
os.environ['XDG_CACHE_HOME'] = original_value
|
||||
else:
|
||||
os.environ.pop('XDG_CACHE_HOME', None)
|
||||
|
||||
def test_cloud_sync_inherits_telemetry(self):
|
||||
"""Test BROWSER_USE_CLOUD_SYNC inherits from ANONYMIZED_TELEMETRY when not set."""
|
||||
telemetry_original = os.environ.get('ANONYMIZED_TELEMETRY', '')
|
||||
sync_original = os.environ.get('BROWSER_USE_CLOUD_SYNC', '')
|
||||
try:
|
||||
# When BROWSER_USE_CLOUD_SYNC is not set, it should inherit from ANONYMIZED_TELEMETRY
|
||||
os.environ['ANONYMIZED_TELEMETRY'] = 'true'
|
||||
os.environ.pop('BROWSER_USE_CLOUD_SYNC', None)
|
||||
assert CONFIG.BROWSER_USE_CLOUD_SYNC is True
|
||||
|
||||
os.environ['ANONYMIZED_TELEMETRY'] = 'false'
|
||||
os.environ.pop('BROWSER_USE_CLOUD_SYNC', None)
|
||||
assert CONFIG.BROWSER_USE_CLOUD_SYNC is False
|
||||
|
||||
# When explicitly set, it should use its own value
|
||||
os.environ['ANONYMIZED_TELEMETRY'] = 'false'
|
||||
os.environ['BROWSER_USE_CLOUD_SYNC'] = 'true'
|
||||
assert CONFIG.BROWSER_USE_CLOUD_SYNC is True
|
||||
finally:
|
||||
if telemetry_original:
|
||||
os.environ['ANONYMIZED_TELEMETRY'] = telemetry_original
|
||||
else:
|
||||
os.environ.pop('ANONYMIZED_TELEMETRY', None)
|
||||
if sync_original:
|
||||
os.environ['BROWSER_USE_CLOUD_SYNC'] = sync_original
|
||||
else:
|
||||
os.environ.pop('BROWSER_USE_CLOUD_SYNC', None)
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,394 @@
|
||||
import asyncio
|
||||
import base64
|
||||
import socketserver
|
||||
|
||||
import pytest
|
||||
from pytest_httpserver import HTTPServer
|
||||
|
||||
from browser_use.browser import BrowserProfile, BrowserSession
|
||||
|
||||
# Fix for httpserver hanging on shutdown - prevent blocking on socket close
|
||||
socketserver.ThreadingMixIn.block_on_close = False
|
||||
socketserver.ThreadingMixIn.daemon_threads = True
|
||||
|
||||
|
||||
class TestBrowserContext:
|
||||
"""Tests for browser context functionality using real browser instances."""
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def http_server(self):
|
||||
"""Create and provide a test HTTP server that serves static content."""
|
||||
server = HTTPServer()
|
||||
server.start()
|
||||
|
||||
# Add routes for test pages
|
||||
server.expect_request('/').respond_with_data(
|
||||
'<html><head><title>Test Home Page</title></head><body><h1>Test Home Page</h1><p>Welcome to the test site</p></body></html>',
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
server.expect_request('/scroll_test').respond_with_data(
|
||||
"""
|
||||
<html>
|
||||
<head>
|
||||
<title>Scroll Test</title>
|
||||
<style>
|
||||
body { height: 3000px; }
|
||||
.marker { position: absolute; }
|
||||
#top { top: 0; }
|
||||
#middle { top: 1000px; }
|
||||
#bottom { top: 2000px; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div id="top" class="marker">Top of the page</div>
|
||||
<div id="middle" class="marker">Middle of the page</div>
|
||||
<div id="bottom" class="marker">Bottom of the page</div>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
yield server
|
||||
server.stop()
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def base_url(self, http_server):
|
||||
"""Return the base URL for the test HTTP server."""
|
||||
return f'http://{http_server.host}:{http_server.port}'
|
||||
|
||||
@pytest.fixture(scope='module')
|
||||
async def browser_session(self):
|
||||
"""Create and provide a BrowserSession instance with security disabled."""
|
||||
browser_session = BrowserSession(
|
||||
browser_profile=BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
keep_alive=True,
|
||||
)
|
||||
)
|
||||
await browser_session.start()
|
||||
yield browser_session
|
||||
await browser_session.kill()
|
||||
# Ensure event bus is properly stopped
|
||||
await browser_session.event_bus.stop(clear=True, timeout=5)
|
||||
|
||||
@pytest.mark.skip(reason='TODO: fix')
|
||||
def test_is_url_allowed(self):
|
||||
"""
|
||||
Test the _is_url_allowed method to verify that it correctly checks URLs against
|
||||
the allowed domains configuration.
|
||||
"""
|
||||
# Scenario 1: allowed_domains is None, any URL should be allowed.
|
||||
from bubus import EventBus
|
||||
|
||||
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
|
||||
|
||||
config1 = BrowserProfile(allowed_domains=None, headless=True, user_data_dir=None)
|
||||
context1 = BrowserSession(browser_profile=config1)
|
||||
event_bus1 = EventBus()
|
||||
watchdog1 = SecurityWatchdog(browser_session=context1, event_bus=event_bus1)
|
||||
assert watchdog1._is_url_allowed('http://anydomain.com') is True
|
||||
assert watchdog1._is_url_allowed('https://anotherdomain.org/path') is True
|
||||
|
||||
# Scenario 2: allowed_domains is provided.
|
||||
# Note: match_url_with_domain_pattern defaults to https:// scheme when none is specified
|
||||
allowed = ['https://example.com', 'http://example.com', 'http://*.mysite.org', 'https://*.mysite.org']
|
||||
config2 = BrowserProfile(allowed_domains=allowed, headless=True, user_data_dir=None)
|
||||
context2 = BrowserSession(browser_profile=config2)
|
||||
event_bus2 = EventBus()
|
||||
watchdog2 = SecurityWatchdog(browser_session=context2, event_bus=event_bus2)
|
||||
|
||||
# URL exactly matching
|
||||
assert watchdog2._is_url_allowed('http://example.com') is True
|
||||
# URL with subdomain (should not be allowed)
|
||||
assert watchdog2._is_url_allowed('http://sub.example.com/path') is False
|
||||
# URL with subdomain for wildcard pattern (should be allowed)
|
||||
assert watchdog2._is_url_allowed('http://sub.mysite.org') is True
|
||||
# URL that matches second allowed domain
|
||||
assert watchdog2._is_url_allowed('https://mysite.org/page') is True
|
||||
# URL with port number, still allowed (port is stripped)
|
||||
assert watchdog2._is_url_allowed('http://example.com:8080') is True
|
||||
assert watchdog2._is_url_allowed('https://example.com:443') is True
|
||||
|
||||
# Scenario 3: Malformed URL or empty domain
|
||||
# urlparse will return an empty netloc for some malformed URLs.
|
||||
assert watchdog2._is_url_allowed('notaurl') is False
|
||||
|
||||
# Method was removed from BrowserSession
|
||||
|
||||
def test_enhanced_css_selector_for_element(self):
|
||||
"""
|
||||
Test removed: _enhanced_css_selector_for_element method no longer exists.
|
||||
"""
|
||||
pass # Method was removed from BrowserSession
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.skip(reason='TODO: fix')
|
||||
async def test_navigate_and_get_current_page(self, browser_session, base_url):
|
||||
"""Test that navigate method changes the URL and get_current_page returns the proper page."""
|
||||
# Navigate to the test page
|
||||
from browser_use.browser.events import NavigateToUrlEvent
|
||||
|
||||
event = browser_session.event_bus.dispatch(NavigateToUrlEvent(url=f'{base_url}/'))
|
||||
await event
|
||||
|
||||
# Get the current page
|
||||
url = await browser_session.get_current_page_url()
|
||||
|
||||
# Verify the page URL matches what we navigated to
|
||||
assert f'{base_url}/' in url
|
||||
|
||||
# Verify the page title
|
||||
title = await browser_session.get_current_page_title()
|
||||
assert title == 'Test Home Page'
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.skip(reason='TODO: fix')
|
||||
async def test_refresh_page(self, browser_session, base_url):
|
||||
"""Test that refresh_page correctly reloads the current page."""
|
||||
# Navigate to the test page
|
||||
from browser_use.browser.events import NavigateToUrlEvent
|
||||
|
||||
event = browser_session.event_bus.dispatch(NavigateToUrlEvent(url=f'{base_url}/'))
|
||||
await event
|
||||
|
||||
# Get the current page info before refresh
|
||||
url_before = await browser_session.get_current_page_url()
|
||||
title_before = await browser_session.get_current_page_title()
|
||||
|
||||
# Refresh the page
|
||||
await browser_session.refresh()
|
||||
|
||||
# Get the current page info after refresh
|
||||
url_after = await browser_session.get_current_page_url()
|
||||
title_after = await browser_session.get_current_page_title()
|
||||
|
||||
# Verify it's still on the same URL
|
||||
assert url_after == url_before
|
||||
|
||||
# Verify the page title is still correct
|
||||
assert title_after == 'Test Home Page'
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.skip(reason='TODO: fix')
|
||||
async def test_execute_javascript(self, browser_session, base_url):
|
||||
"""Test that execute_javascript correctly executes JavaScript in the current page."""
|
||||
# Navigate to a test page
|
||||
from browser_use.browser.events import NavigateToUrlEvent
|
||||
|
||||
event = browser_session.event_bus.dispatch(NavigateToUrlEvent(url=f'{base_url}/'))
|
||||
await event
|
||||
|
||||
# Execute a simple JavaScript snippet that returns a value
|
||||
result = await browser_session.execute_javascript('document.title')
|
||||
|
||||
# Verify the result
|
||||
assert result == 'Test Home Page'
|
||||
|
||||
# Execute JavaScript that modifies the page
|
||||
await browser_session.execute_javascript("document.body.style.backgroundColor = 'red'")
|
||||
|
||||
# Verify the change by reading back the value
|
||||
bg_color = await browser_session.execute_javascript('document.body.style.backgroundColor')
|
||||
assert bg_color == 'red'
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.skip(reason='TODO: fix')
|
||||
@pytest.mark.skip(reason='get_scroll_info API changed - depends on page object that no longer exists')
|
||||
async def test_get_scroll_info(self, browser_session, base_url):
|
||||
"""Test that get_scroll_info returns the correct scroll position information."""
|
||||
# Navigate to the scroll test page
|
||||
from browser_use.browser.events import NavigateToUrlEvent
|
||||
|
||||
event = browser_session.event_bus.dispatch(NavigateToUrlEvent(url=f'{base_url}/scroll_test'))
|
||||
await event
|
||||
page = await browser_session.get_current_page()
|
||||
|
||||
# Get initial scroll info
|
||||
pixels_above_initial, pixels_below_initial = await browser_session.get_scroll_info(page)
|
||||
|
||||
# Verify initial scroll position
|
||||
assert pixels_above_initial == 0, 'Initial scroll position should be at the top'
|
||||
assert pixels_below_initial > 0, 'There should be content below the viewport'
|
||||
|
||||
# Scroll down the page
|
||||
await browser_session.execute_javascript('window.scrollBy(0, 500)')
|
||||
await asyncio.sleep(0.2) # Brief delay for scroll to complete
|
||||
|
||||
# Get new scroll info
|
||||
pixels_above_after_scroll, pixels_below_after_scroll = await browser_session.get_scroll_info(page)
|
||||
|
||||
# Verify new scroll position
|
||||
assert pixels_above_after_scroll > 0, 'Page should be scrolled down'
|
||||
assert pixels_above_after_scroll >= 400, 'Page should be scrolled down at least 400px'
|
||||
assert pixels_below_after_scroll < pixels_below_initial, 'Less content should be below viewport after scrolling'
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.skip(reason='TODO: fix')
|
||||
async def test_take_screenshot(self, browser_session, base_url):
|
||||
"""Test that take_screenshot returns a valid base64 encoded image."""
|
||||
# Navigate to the test page
|
||||
from browser_use.browser.events import NavigateToUrlEvent
|
||||
|
||||
event = browser_session.event_bus.dispatch(NavigateToUrlEvent(url=f'{base_url}/'))
|
||||
await event
|
||||
|
||||
# Take a screenshot
|
||||
screenshot_base64 = await browser_session.take_screenshot()
|
||||
|
||||
# Verify the screenshot is a valid base64 string
|
||||
assert isinstance(screenshot_base64, str)
|
||||
assert len(screenshot_base64) > 0
|
||||
|
||||
# Verify it can be decoded as base64
|
||||
try:
|
||||
image_data = base64.b64decode(screenshot_base64)
|
||||
# Verify the data starts with a valid image signature (PNG file header)
|
||||
assert image_data[:8] == b'\x89PNG\r\n\x1a\n', 'Screenshot is not a valid PNG image'
|
||||
except Exception as e:
|
||||
pytest.fail(f'Failed to decode screenshot as base64: {e}')
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.skip(reason='TODO: fix')
|
||||
async def test_switch_tab_operations(self, browser_session, base_url):
|
||||
"""Test tab creation, switching, and closing operations."""
|
||||
# Navigate to home page in first tab
|
||||
from browser_use.browser.events import NavigateToUrlEvent
|
||||
|
||||
event = browser_session.event_bus.dispatch(NavigateToUrlEvent(url=f'{base_url}/'))
|
||||
await event
|
||||
|
||||
# Create a new tab
|
||||
await browser_session.create_new_tab(f'{base_url}/scroll_test')
|
||||
|
||||
# Verify we have two tabs now
|
||||
tabs_info = await browser_session.get_tabs()
|
||||
assert len(tabs_info) == 2, 'Should have two tabs open'
|
||||
|
||||
# Verify current tab is the scroll test page
|
||||
current_url = await browser_session.get_current_page_url()
|
||||
assert f'{base_url}/scroll_test' in current_url
|
||||
|
||||
# Switch back to the first tab
|
||||
await browser_session.switch_to_tab(0)
|
||||
|
||||
# Verify we're back on the home page
|
||||
current_url = await browser_session.get_current_page_url()
|
||||
assert f'{base_url}/' in current_url
|
||||
|
||||
# Close the second tab
|
||||
await browser_session.close_tab(1)
|
||||
|
||||
# Verify we have the expected number of tabs
|
||||
# The first tab remains plus any about:blank tabs created by AboutBlankWatchdog
|
||||
tabs_info = await browser_session.get_tabs_info()
|
||||
# Filter out about:blank tabs created by the watchdog
|
||||
non_blank_tabs = [tab for tab in tabs_info if 'about:blank' not in tab.url]
|
||||
assert len(non_blank_tabs) == 1, (
|
||||
f'Should have one non-blank tab open after closing the second, but got {len(non_blank_tabs)}: {non_blank_tabs}'
|
||||
)
|
||||
assert base_url in non_blank_tabs[0].url, 'The remaining tab should be the home page'
|
||||
|
||||
# TODO: highlighting doesn't exist anymore
|
||||
# @pytest.mark.asyncio
|
||||
# async def test_remove_highlights(self, browser_session, base_url):
|
||||
# """Test that remove_highlights successfully removes highlight elements."""
|
||||
# # Navigate to a test page
|
||||
# from browser_use.browser.events import NavigateToUrlEvent; event = browser_session.event_bus.dispatch(NavigateToUrlEvent(url=f'{base_url}/')
|
||||
|
||||
# # Add a highlight via JavaScript
|
||||
# await browser_session.execute_javascript("""
|
||||
# const container = document.createElement('div');
|
||||
# container.id = 'playwright-highlight-container';
|
||||
# document.body.appendChild(container);
|
||||
|
||||
# const highlight = document.createElement('div');
|
||||
# highlight.id = 'playwright-highlight-1';
|
||||
# container.appendChild(highlight);
|
||||
|
||||
# const element = document.querySelector('h1');
|
||||
# element.setAttribute('browser-user-highlight-id', 'playwright-highlight-1');
|
||||
# """)
|
||||
|
||||
# # Verify the highlight container exists
|
||||
# container_exists = await browser_session.execute_javascript(
|
||||
# "document.getElementById('playwright-highlight-container') !== null"
|
||||
# )
|
||||
# assert container_exists, 'Highlight container should exist before removal'
|
||||
|
||||
# # Call remove_highlights
|
||||
# await browser_session.remove_highlights()
|
||||
|
||||
# # Verify the highlight container was removed
|
||||
# container_exists_after = await browser_session.execute_javascript(
|
||||
# "document.getElementById('playwright-highlight-container') !== null"
|
||||
# )
|
||||
# assert not container_exists_after, 'Highlight container should be removed'
|
||||
|
||||
# # Verify the highlight attribute was removed from the element
|
||||
# attribute_exists = await browser_session.execute_javascript(
|
||||
# "document.querySelector('h1').hasAttribute('browser-user-highlight-id')"
|
||||
# )
|
||||
# assert not attribute_exists, 'browser-user-highlight-id attribute should be removed'
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.skip(reason='TODO: fix')
|
||||
async def test_custom_action_with_no_arguments(self, browser_session, base_url):
|
||||
"""Test that custom actions with no arguments are handled correctly"""
|
||||
from browser_use.agent.views import ActionResult
|
||||
from browser_use.tools.registry.service import Registry
|
||||
|
||||
# Create a registry
|
||||
registry = Registry()
|
||||
|
||||
# Register a custom action with no arguments
|
||||
@registry.action('Some custom action with no args')
|
||||
def simple_action():
|
||||
return ActionResult(extracted_content='return some result')
|
||||
|
||||
# Navigate to a test page
|
||||
from browser_use.browser.events import NavigateToUrlEvent
|
||||
|
||||
event = browser_session.event_bus.dispatch(NavigateToUrlEvent(url=f'{base_url}/'))
|
||||
await event
|
||||
|
||||
# Execute the action
|
||||
result = await registry.execute_action('simple_action', {})
|
||||
|
||||
# Verify the result
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content == 'return some result'
|
||||
|
||||
# Test that the action model is created correctly
|
||||
action_model = registry.create_action_model()
|
||||
|
||||
# The action should be in the model fields
|
||||
assert 'simple_action' in action_model.model_fields
|
||||
|
||||
# Create an instance with the simple_action
|
||||
action_instance = action_model(simple_action={}) # type: ignore[call-arg]
|
||||
|
||||
# Test that model_dump works correctly
|
||||
dumped = action_instance.model_dump(exclude_unset=True)
|
||||
assert 'simple_action' in dumped
|
||||
assert dumped['simple_action'] == {}
|
||||
|
||||
# Test async version as well
|
||||
@registry.action('Async custom action with no args')
|
||||
async def async_simple_action():
|
||||
return ActionResult(extracted_content='async result')
|
||||
|
||||
result = await registry.execute_action('async_simple_action', {})
|
||||
assert result.extracted_content == 'async result'
|
||||
|
||||
# Test with special parameters but no regular arguments
|
||||
@registry.action('Action with only special params')
|
||||
async def special_params_only(browser_session):
|
||||
current_url = await browser_session.get_current_page_url()
|
||||
return ActionResult(extracted_content=f'Page URL: {current_url}')
|
||||
|
||||
result = await registry.execute_action('special_params_only', {}, browser_session=browser_session)
|
||||
assert 'Page URL:' in result.extracted_content
|
||||
assert base_url in result.extracted_content
|
||||
@@ -0,0 +1,544 @@
|
||||
"""
|
||||
Comprehensive tests for the action registry system - Core functionality.
|
||||
|
||||
Tests cover:
|
||||
1. Existing parameter patterns (individual params, pydantic models)
|
||||
2. Special parameter injection (browser_session, page_extraction_llm, etc.)
|
||||
3. Action-to-action calling scenarios
|
||||
4. Mixed parameter patterns
|
||||
5. Registry execution edge cases
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
|
||||
import pytest
|
||||
from pydantic import Field
|
||||
from pytest_httpserver import HTTPServer
|
||||
from pytest_httpserver.httpserver import HandlerType
|
||||
|
||||
from browser_use.agent.views import ActionResult
|
||||
from browser_use.browser import BrowserSession
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
from browser_use.llm.messages import UserMessage
|
||||
from browser_use.tools.registry.service import Registry
|
||||
from browser_use.tools.registry.views import ActionModel as BaseActionModel
|
||||
from browser_use.tools.views import (
|
||||
ClickElementAction,
|
||||
InputTextAction,
|
||||
NoParamsAction,
|
||||
SearchAction,
|
||||
)
|
||||
from tests.ci.conftest import create_mock_llm
|
||||
|
||||
# Configure logging
|
||||
logging.basicConfig(level=logging.DEBUG)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class TestContext:
|
||||
"""Simple context for testing"""
|
||||
|
||||
pass
|
||||
|
||||
|
||||
# Test parameter models
|
||||
class SimpleParams(BaseActionModel):
|
||||
"""Simple parameter model"""
|
||||
|
||||
value: str = Field(description='Test value')
|
||||
|
||||
|
||||
class ComplexParams(BaseActionModel):
|
||||
"""Complex parameter model with multiple fields"""
|
||||
|
||||
text: str = Field(description='Text input')
|
||||
number: int = Field(description='Number input', default=42)
|
||||
optional_flag: bool = Field(description='Optional boolean', default=False)
|
||||
|
||||
|
||||
# Test fixtures
|
||||
@pytest.fixture(scope='session')
|
||||
def http_server():
|
||||
"""Create and provide a test HTTP server that serves static content."""
|
||||
server = HTTPServer()
|
||||
server.start()
|
||||
|
||||
# Add a simple test page that can handle multiple requests
|
||||
server.expect_request('/test', handler_type=HandlerType.PERMANENT).respond_with_data(
|
||||
'<html><head><title>Test Page</title></head><body><h1>Test Page</h1><p>Hello from test page</p></body></html>',
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
yield server
|
||||
|
||||
server.stop()
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def base_url(http_server):
|
||||
"""Return the base URL for the test HTTP server."""
|
||||
return f'http://{http_server.host}:{http_server.port}'
|
||||
|
||||
|
||||
@pytest.fixture(scope='module')
|
||||
def mock_llm():
|
||||
"""Create a mock LLM"""
|
||||
return create_mock_llm()
|
||||
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
def registry():
|
||||
"""Create a fresh registry for each test"""
|
||||
return Registry[TestContext]()
|
||||
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
async def browser_session(base_url):
|
||||
"""Create a real BrowserSession for testing"""
|
||||
browser_session = BrowserSession(
|
||||
browser_profile=BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
keep_alive=True,
|
||||
)
|
||||
)
|
||||
await browser_session.start()
|
||||
from browser_use.browser.events import NavigateToUrlEvent
|
||||
|
||||
browser_session.event_bus.dispatch(NavigateToUrlEvent(url=f'{base_url}/test'))
|
||||
await asyncio.sleep(0.5) # Wait for navigation
|
||||
yield browser_session
|
||||
await browser_session.kill()
|
||||
|
||||
|
||||
class TestActionRegistryParameterPatterns:
|
||||
"""Test different parameter patterns that should all continue to work"""
|
||||
|
||||
async def test_individual_parameters_no_browser(self, registry):
|
||||
"""Test action with individual parameters, no special injection"""
|
||||
|
||||
@registry.action('Simple action with individual params')
|
||||
async def simple_action(text: str, number: int = 10):
|
||||
return ActionResult(extracted_content=f'Text: {text}, Number: {number}')
|
||||
|
||||
# Test execution
|
||||
result = await registry.execute_action('simple_action', {'text': 'hello', 'number': 42})
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'Text: hello, Number: 42' in result.extracted_content
|
||||
|
||||
async def test_individual_parameters_with_browser(self, registry, browser_session, base_url):
|
||||
"""Test action with individual parameters plus browser_session injection"""
|
||||
|
||||
@registry.action('Action with individual params and browser')
|
||||
async def action_with_browser(text: str, browser_session: BrowserSession):
|
||||
url = await browser_session.get_current_page_url()
|
||||
return ActionResult(extracted_content=f'Text: {text}, URL: {url}')
|
||||
|
||||
# Navigate to test page first
|
||||
from browser_use.browser.events import NavigateToUrlEvent
|
||||
|
||||
event = browser_session.event_bus.dispatch(NavigateToUrlEvent(url=f'{base_url}/test', new_tab=True))
|
||||
await event
|
||||
|
||||
# Test execution
|
||||
result = await registry.execute_action('action_with_browser', {'text': 'hello'}, browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'Text: hello, URL:' in result.extracted_content
|
||||
assert base_url in result.extracted_content
|
||||
|
||||
async def test_pydantic_model_parameters(self, registry, browser_session, base_url):
|
||||
"""Test action that takes a pydantic model as first parameter"""
|
||||
|
||||
@registry.action('Action with pydantic model', param_model=ComplexParams)
|
||||
async def pydantic_action(params: ComplexParams, browser_session: BrowserSession):
|
||||
url = await browser_session.get_current_page_url()
|
||||
return ActionResult(
|
||||
extracted_content=f'Text: {params.text}, Number: {params.number}, Flag: {params.optional_flag}, URL: {url}'
|
||||
)
|
||||
|
||||
# Navigate to test page first
|
||||
from browser_use.browser.events import NavigateToUrlEvent
|
||||
|
||||
event = browser_session.event_bus.dispatch(NavigateToUrlEvent(url=f'{base_url}/test', new_tab=True))
|
||||
await event
|
||||
|
||||
# Test execution
|
||||
result = await registry.execute_action(
|
||||
'pydantic_action', {'text': 'test', 'number': 100, 'optional_flag': True}, browser_session=browser_session
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'Text: test, Number: 100, Flag: True' in result.extracted_content
|
||||
assert base_url in result.extracted_content
|
||||
|
||||
async def test_mixed_special_parameters(self, registry, browser_session, base_url, mock_llm):
|
||||
"""Test action with multiple special injected parameters"""
|
||||
|
||||
from browser_use.llm.base import BaseChatModel
|
||||
|
||||
@registry.action('Action with multiple special params')
|
||||
async def multi_special_action(
|
||||
text: str,
|
||||
browser_session: BrowserSession,
|
||||
page_extraction_llm: BaseChatModel,
|
||||
available_file_paths: list,
|
||||
):
|
||||
llm_response = await page_extraction_llm.ainvoke([UserMessage(content='test')])
|
||||
files = available_file_paths or []
|
||||
url = await browser_session.get_current_page_url()
|
||||
|
||||
return ActionResult(
|
||||
extracted_content=f'Text: {text}, URL: {url}, LLM: {llm_response.completion}, Files: {len(files)}'
|
||||
)
|
||||
|
||||
# Navigate to test page first
|
||||
from browser_use.browser.events import NavigateToUrlEvent
|
||||
|
||||
event = browser_session.event_bus.dispatch(NavigateToUrlEvent(url=f'{base_url}/test', new_tab=True))
|
||||
await event
|
||||
|
||||
# Test execution
|
||||
result = await registry.execute_action(
|
||||
'multi_special_action',
|
||||
{'text': 'hello'},
|
||||
browser_session=browser_session,
|
||||
page_extraction_llm=mock_llm,
|
||||
available_file_paths=['file1.txt', 'file2.txt'],
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'Text: hello' in result.extracted_content
|
||||
assert base_url in result.extracted_content
|
||||
# The mock LLM returns a JSON response
|
||||
assert '"Task completed successfully"' in result.extracted_content
|
||||
assert 'Files: 2' in result.extracted_content
|
||||
|
||||
async def test_no_params_action(self, registry, browser_session):
|
||||
"""Test action with NoParamsAction model"""
|
||||
|
||||
@registry.action('No params action', param_model=NoParamsAction)
|
||||
async def no_params_action(params: NoParamsAction, browser_session: BrowserSession):
|
||||
url = await browser_session.get_current_page_url()
|
||||
return ActionResult(extracted_content=f'No params action executed on {url}')
|
||||
|
||||
# Test execution with any parameters (should be ignored)
|
||||
result = await registry.execute_action(
|
||||
'no_params_action', {'random': 'data', 'should': 'be', 'ignored': True}, browser_session=browser_session
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'No params action executed on' in result.extracted_content
|
||||
assert '/test' in result.extracted_content
|
||||
|
||||
|
||||
class TestActionToActionCalling:
|
||||
"""Test scenarios where actions call other actions"""
|
||||
|
||||
async def test_action_calling_action_with_kwargs(self, registry, browser_session):
|
||||
"""Test action calling another action using kwargs (current problematic pattern)"""
|
||||
|
||||
# Helper function that actions can call
|
||||
async def helper_function(browser_session: BrowserSession, data: str):
|
||||
url = await browser_session.get_current_page_url()
|
||||
return f'Helper processed: {data} on {url}'
|
||||
|
||||
@registry.action('First action')
|
||||
async def first_action(text: str, browser_session: BrowserSession):
|
||||
# This should work without parameter conflicts
|
||||
result = await helper_function(browser_session=browser_session, data=text)
|
||||
return ActionResult(extracted_content=f'First: {result}')
|
||||
|
||||
@registry.action('Calling action')
|
||||
async def calling_action(message: str, browser_session: BrowserSession):
|
||||
# Call the first action through the registry (simulates action-to-action calling)
|
||||
intermediate_result = await registry.execute_action(
|
||||
'first_action', {'text': message}, browser_session=browser_session
|
||||
)
|
||||
return ActionResult(extracted_content=f'Called result: {intermediate_result.extracted_content}')
|
||||
|
||||
# Test the calling chain
|
||||
result = await registry.execute_action('calling_action', {'message': 'test'}, browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'Called result: First: Helper processed: test on' in result.extracted_content
|
||||
assert '/test' in result.extracted_content
|
||||
|
||||
async def test_google_sheets_style_calling_pattern(self, registry, browser_session):
|
||||
"""Test the specific pattern from Google Sheets actions that causes the error"""
|
||||
|
||||
# Simulate the _select_cell_or_range helper function
|
||||
async def _select_cell_or_range(browser_session: BrowserSession, cell_or_range: str):
|
||||
url = await browser_session.get_current_page_url()
|
||||
return ActionResult(extracted_content=f'Selected cell {cell_or_range} on {url}')
|
||||
|
||||
@registry.action('Select cell or range')
|
||||
async def select_cell_or_range(cell_or_range: str, browser_session: BrowserSession):
|
||||
# This pattern now works with kwargs
|
||||
return await _select_cell_or_range(browser_session=browser_session, cell_or_range=cell_or_range)
|
||||
|
||||
@registry.action('Select cell or range (fixed)')
|
||||
async def select_cell_or_range_fixed(cell_or_range: str, browser_session: BrowserSession):
|
||||
# This pattern also works
|
||||
return await _select_cell_or_range(browser_session, cell_or_range)
|
||||
|
||||
@registry.action('Update range contents')
|
||||
async def update_range_contents(range_name: str, new_contents: str, browser_session: BrowserSession):
|
||||
# This action calls select_cell_or_range, simulating the real Google Sheets pattern
|
||||
# Get the action's param model to call it properly
|
||||
action = registry.registry.actions['select_cell_or_range_fixed']
|
||||
params = action.param_model(cell_or_range=range_name)
|
||||
await select_cell_or_range_fixed(cell_or_range=range_name, browser_session=browser_session)
|
||||
return ActionResult(extracted_content=f'Updated range {range_name} with {new_contents}')
|
||||
|
||||
# Test the fixed version (should work)
|
||||
result_fixed = await registry.execute_action(
|
||||
'select_cell_or_range_fixed', {'cell_or_range': 'A1:F100'}, browser_session=browser_session
|
||||
)
|
||||
assert result_fixed.extracted_content is not None
|
||||
assert 'Selected cell A1:F100 on' in result_fixed.extracted_content
|
||||
assert '/test' in result_fixed.extracted_content
|
||||
|
||||
# Test the chained calling pattern
|
||||
result_chain = await registry.execute_action(
|
||||
'update_range_contents', {'range_name': 'B2:D4', 'new_contents': 'test data'}, browser_session=browser_session
|
||||
)
|
||||
assert result_chain.extracted_content is not None
|
||||
assert 'Updated range B2:D4 with test data' in result_chain.extracted_content
|
||||
|
||||
# Test the problematic version (should work with enhanced registry)
|
||||
result_problematic = await registry.execute_action(
|
||||
'select_cell_or_range', {'cell_or_range': 'A1:F100'}, browser_session=browser_session
|
||||
)
|
||||
# With the enhanced registry, this should succeed
|
||||
assert result_problematic.extracted_content is not None
|
||||
assert 'Selected cell A1:F100 on' in result_problematic.extracted_content
|
||||
assert '/test' in result_problematic.extracted_content
|
||||
|
||||
async def test_complex_action_chain(self, registry, browser_session):
|
||||
"""Test a complex chain of actions calling other actions"""
|
||||
|
||||
@registry.action('Base action')
|
||||
async def base_action(value: str, browser_session: BrowserSession):
|
||||
url = await browser_session.get_current_page_url()
|
||||
return ActionResult(extracted_content=f'Base: {value} on {url}')
|
||||
|
||||
@registry.action('Middle action')
|
||||
async def middle_action(input_val: str, browser_session: BrowserSession):
|
||||
# Call base action
|
||||
base_result = await registry.execute_action(
|
||||
'base_action', {'value': f'processed-{input_val}'}, browser_session=browser_session
|
||||
)
|
||||
return ActionResult(extracted_content=f'Middle: {base_result.extracted_content}')
|
||||
|
||||
@registry.action('Top action')
|
||||
async def top_action(original: str, browser_session: BrowserSession):
|
||||
# Call middle action
|
||||
middle_result = await registry.execute_action(
|
||||
'middle_action', {'input_val': f'enhanced-{original}'}, browser_session=browser_session
|
||||
)
|
||||
return ActionResult(extracted_content=f'Top: {middle_result.extracted_content}')
|
||||
|
||||
# Test the full chain
|
||||
result = await registry.execute_action('top_action', {'original': 'test'}, browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'Top: Middle: Base: processed-enhanced-test on' in result.extracted_content
|
||||
assert '/test' in result.extracted_content
|
||||
|
||||
|
||||
class TestRegistryEdgeCases:
|
||||
"""Test edge cases and error conditions"""
|
||||
|
||||
async def test_decorated_action_rejects_positional_args(self, registry, browser_session):
|
||||
"""Test that decorated actions reject positional arguments"""
|
||||
|
||||
@registry.action('Action that should reject positional args')
|
||||
async def test_action(cell_or_range: str, browser_session: BrowserSession):
|
||||
url = await browser_session.get_current_page_url()
|
||||
return ActionResult(extracted_content=f'Selected cell {cell_or_range} on {url}')
|
||||
|
||||
# Test that calling with positional arguments raises TypeError
|
||||
with pytest.raises(
|
||||
TypeError, match='test_action\\(\\) does not accept positional arguments, only keyword arguments are allowed'
|
||||
):
|
||||
await test_action('A1:B2', browser_session)
|
||||
|
||||
# Test that calling with keyword arguments works
|
||||
result = await test_action(browser_session=browser_session, cell_or_range='A1:B2')
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'Selected cell A1:B2 on' in result.extracted_content
|
||||
|
||||
async def test_missing_required_browser_session(self, registry):
|
||||
"""Test that actions requiring browser_session fail appropriately when not provided"""
|
||||
|
||||
@registry.action('Requires browser')
|
||||
async def requires_browser(text: str, browser_session: BrowserSession):
|
||||
url = await browser_session.get_current_page_url()
|
||||
return ActionResult(extracted_content=f'Text: {text}, URL: {url}')
|
||||
|
||||
# Should raise RuntimeError when browser_session is required but not provided
|
||||
with pytest.raises(RuntimeError, match='requires browser_session but none provided'):
|
||||
await registry.execute_action(
|
||||
'requires_browser',
|
||||
{'text': 'test'},
|
||||
# No browser_session provided
|
||||
)
|
||||
|
||||
async def test_missing_required_llm(self, registry, browser_session):
|
||||
"""Test that actions requiring page_extraction_llm fail appropriately when not provided"""
|
||||
|
||||
from browser_use.llm.base import BaseChatModel
|
||||
|
||||
@registry.action('Requires LLM')
|
||||
async def requires_llm(text: str, browser_session: BrowserSession, page_extraction_llm: BaseChatModel):
|
||||
url = await browser_session.get_current_page_url()
|
||||
llm_response = await page_extraction_llm.ainvoke([UserMessage(content='test')])
|
||||
return ActionResult(extracted_content=f'Text: {text}, LLM: {llm_response.completion}')
|
||||
|
||||
# Should raise RuntimeError when page_extraction_llm is required but not provided
|
||||
with pytest.raises(RuntimeError, match='requires page_extraction_llm but none provided'):
|
||||
await registry.execute_action(
|
||||
'requires_llm',
|
||||
{'text': 'test'},
|
||||
browser_session=browser_session,
|
||||
# No page_extraction_llm provided
|
||||
)
|
||||
|
||||
async def test_invalid_parameters(self, registry, browser_session):
|
||||
"""Test handling of invalid parameters"""
|
||||
|
||||
@registry.action('Typed action')
|
||||
async def typed_action(number: int, browser_session: BrowserSession):
|
||||
return ActionResult(extracted_content=f'Number: {number}')
|
||||
|
||||
# Should raise RuntimeError when parameter validation fails
|
||||
with pytest.raises(RuntimeError, match='Invalid parameters'):
|
||||
await registry.execute_action(
|
||||
'typed_action',
|
||||
{'number': 'not a number'}, # Invalid type
|
||||
browser_session=browser_session,
|
||||
)
|
||||
|
||||
async def test_nonexistent_action(self, registry, browser_session):
|
||||
"""Test calling a non-existent action"""
|
||||
|
||||
with pytest.raises(ValueError, match='Action nonexistent_action not found'):
|
||||
await registry.execute_action('nonexistent_action', {'param': 'value'}, browser_session=browser_session)
|
||||
|
||||
async def test_sync_action_wrapper(self, registry, browser_session):
|
||||
"""Test that sync functions are properly wrapped to be async"""
|
||||
|
||||
@registry.action('Sync action')
|
||||
def sync_action(text: str, browser_session: BrowserSession):
|
||||
# This is a sync function that should be wrapped
|
||||
return ActionResult(extracted_content=f'Sync: {text}')
|
||||
|
||||
# Should work even though the original function is sync
|
||||
result = await registry.execute_action('sync_action', {'text': 'test'}, browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'Sync: test' in result.extracted_content
|
||||
|
||||
async def test_excluded_actions(self, browser_session):
|
||||
"""Test that excluded actions are not registered"""
|
||||
|
||||
registry_with_exclusions = Registry[TestContext](exclude_actions=['excluded_action'])
|
||||
|
||||
@registry_with_exclusions.action('Excluded action')
|
||||
async def excluded_action(text: str):
|
||||
return ActionResult(extracted_content=f'Should not execute: {text}')
|
||||
|
||||
@registry_with_exclusions.action('Included action')
|
||||
async def included_action(text: str):
|
||||
return ActionResult(extracted_content=f'Should execute: {text}')
|
||||
|
||||
# Excluded action should not be in registry
|
||||
assert 'excluded_action' not in registry_with_exclusions.registry.actions
|
||||
assert 'included_action' in registry_with_exclusions.registry.actions
|
||||
|
||||
# Should raise error when trying to execute excluded action
|
||||
with pytest.raises(ValueError, match='Action excluded_action not found'):
|
||||
await registry_with_exclusions.execute_action('excluded_action', {'text': 'test'})
|
||||
|
||||
# Included action should work
|
||||
result = await registry_with_exclusions.execute_action('included_action', {'text': 'test'})
|
||||
assert result.extracted_content is not None
|
||||
assert 'Should execute: test' in result.extracted_content
|
||||
|
||||
|
||||
class TestExistingToolsActions:
|
||||
"""Test that existing tools actions continue to work"""
|
||||
|
||||
async def test_existing_action_models(self, registry, browser_session):
|
||||
"""Test that existing action parameter models work correctly"""
|
||||
|
||||
@registry.action('Test search', param_model=SearchAction)
|
||||
async def test_search(params: SearchAction, browser_session: BrowserSession):
|
||||
return ActionResult(extracted_content=f'Searched for: {params.query}')
|
||||
|
||||
@registry.action('Test click', param_model=ClickElementAction)
|
||||
async def test_click(params: ClickElementAction, browser_session: BrowserSession):
|
||||
return ActionResult(extracted_content=f'Clicked element: {params.index}')
|
||||
|
||||
@registry.action('Test input', param_model=InputTextAction)
|
||||
async def test_input(params: InputTextAction, browser_session: BrowserSession):
|
||||
return ActionResult(extracted_content=f'Input text: {params.text} at index: {params.index}')
|
||||
|
||||
# Test SearchGoogleAction
|
||||
result1 = await registry.execute_action('test_search', {'query': 'python testing'}, browser_session=browser_session)
|
||||
assert result1.extracted_content is not None
|
||||
assert 'Searched for: python testing' in result1.extracted_content
|
||||
|
||||
# Test ClickElementAction
|
||||
result2 = await registry.execute_action('test_click', {'index': 42}, browser_session=browser_session)
|
||||
assert result2.extracted_content is not None
|
||||
assert 'Clicked element: 42' in result2.extracted_content
|
||||
|
||||
# Test InputTextAction
|
||||
result3 = await registry.execute_action('test_input', {'index': 5, 'text': 'test input'}, browser_session=browser_session)
|
||||
assert result3.extracted_content is not None
|
||||
assert 'Input text: test input at index: 5' in result3.extracted_content
|
||||
|
||||
async def test_pydantic_vs_individual_params_consistency(self, registry, browser_session):
|
||||
"""Test that pydantic and individual parameter patterns produce consistent results"""
|
||||
|
||||
# Action using individual parameters
|
||||
@registry.action('Individual params')
|
||||
async def individual_params_action(text: str, number: int, browser_session: BrowserSession):
|
||||
return ActionResult(extracted_content=f'Individual: {text}-{number}')
|
||||
|
||||
# Action using pydantic model
|
||||
class TestParams(BaseActionModel):
|
||||
text: str
|
||||
number: int
|
||||
|
||||
@registry.action('Pydantic params', param_model=TestParams)
|
||||
async def pydantic_params_action(params: TestParams, browser_session: BrowserSession):
|
||||
return ActionResult(extracted_content=f'Pydantic: {params.text}-{params.number}')
|
||||
|
||||
# Both should produce similar results
|
||||
test_data = {'text': 'hello', 'number': 42}
|
||||
|
||||
result1 = await registry.execute_action('individual_params_action', test_data, browser_session=browser_session)
|
||||
|
||||
result2 = await registry.execute_action('pydantic_params_action', test_data, browser_session=browser_session)
|
||||
|
||||
# Both should extract the same content (just different prefixes)
|
||||
assert result1.extracted_content is not None
|
||||
assert 'hello-42' in result1.extracted_content
|
||||
assert result2.extracted_content is not None
|
||||
assert 'hello-42' in result2.extracted_content
|
||||
assert 'Individual:' in result1.extracted_content
|
||||
assert 'Pydantic:' in result2.extracted_content
|
||||
@@ -0,0 +1,547 @@
|
||||
"""
|
||||
Comprehensive tests for the action registry system - Validation and patterns.
|
||||
|
||||
Tests cover:
|
||||
1. Type 1 and Type 2 patterns
|
||||
2. Validation rules
|
||||
3. Decorated function behavior
|
||||
4. Parameter model generation
|
||||
5. Parameter ordering
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
|
||||
import pytest
|
||||
from pydantic import Field
|
||||
|
||||
from browser_use.agent.views import ActionResult
|
||||
from browser_use.browser import BrowserSession
|
||||
from browser_use.tools.registry.service import Registry
|
||||
from browser_use.tools.registry.views import ActionModel as BaseActionModel
|
||||
from tests.ci.conftest import create_mock_llm
|
||||
|
||||
# Configure logging
|
||||
logging.basicConfig(level=logging.DEBUG)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class TestType1Pattern:
|
||||
"""Test Type 1 Pattern: Pydantic model first (from normalization tests)"""
|
||||
|
||||
def test_type1_with_param_model(self):
|
||||
"""Type 1: action(params: Model, special_args...) should work"""
|
||||
registry = Registry()
|
||||
|
||||
class ClickAction(BaseActionModel):
|
||||
index: int
|
||||
delay: float = 0.0
|
||||
|
||||
@registry.action('Click element', param_model=ClickAction)
|
||||
async def click_element(params: ClickAction, browser_session: BrowserSession):
|
||||
return ActionResult(extracted_content=f'Clicked {params.index}')
|
||||
|
||||
# Verify registration
|
||||
assert 'click_element' in registry.registry.actions
|
||||
action = registry.registry.actions['click_element']
|
||||
assert action.param_model == ClickAction
|
||||
|
||||
# Verify decorated function signature (should be kwargs-only)
|
||||
import inspect
|
||||
|
||||
sig = inspect.signature(click_element)
|
||||
params = list(sig.parameters.values())
|
||||
|
||||
# Should have no positional-only or positional-or-keyword params
|
||||
for param in params:
|
||||
assert param.kind in (inspect.Parameter.KEYWORD_ONLY, inspect.Parameter.VAR_KEYWORD)
|
||||
|
||||
def test_type1_with_multiple_special_params(self):
|
||||
"""Type 1 with multiple special params should work"""
|
||||
registry = Registry()
|
||||
|
||||
class ExtractAction(BaseActionModel):
|
||||
goal: str
|
||||
include_links: bool = False
|
||||
|
||||
from browser_use.llm.base import BaseChatModel
|
||||
|
||||
@registry.action('Extract content', param_model=ExtractAction)
|
||||
async def extract_content(params: ExtractAction, browser_session: BrowserSession, page_extraction_llm: BaseChatModel):
|
||||
return ActionResult(extracted_content=params.goal)
|
||||
|
||||
assert 'extract_content' in registry.registry.actions
|
||||
|
||||
|
||||
class TestType2Pattern:
|
||||
"""Test Type 2 Pattern: loose parameters (from normalization tests)"""
|
||||
|
||||
def test_type2_simple_action(self):
|
||||
"""Type 2: action(arg1, arg2, special_args...) should work"""
|
||||
registry = Registry()
|
||||
|
||||
@registry.action('Fill field')
|
||||
async def fill_field(index: int, text: str, browser_session: BrowserSession):
|
||||
return ActionResult(extracted_content=f'Filled {index} with {text}')
|
||||
|
||||
# Verify registration
|
||||
assert 'fill_field' in registry.registry.actions
|
||||
action = registry.registry.actions['fill_field']
|
||||
|
||||
# Should auto-generate param model
|
||||
assert action.param_model is not None
|
||||
assert 'index' in action.param_model.model_fields
|
||||
assert 'text' in action.param_model.model_fields
|
||||
|
||||
def test_type2_with_defaults(self):
|
||||
"""Type 2 with default values should preserve defaults"""
|
||||
registry = Registry()
|
||||
|
||||
@registry.action('Scroll page')
|
||||
async def scroll_page(direction: str = 'down', amount: int = 100, browser_session: BrowserSession = None): # type: ignore
|
||||
return ActionResult(extracted_content=f'Scrolled {direction} by {amount}')
|
||||
|
||||
action = registry.registry.actions['scroll_page']
|
||||
# Check that defaults are preserved in generated model
|
||||
schema = action.param_model.model_json_schema()
|
||||
assert schema['properties']['direction']['default'] == 'down'
|
||||
assert schema['properties']['amount']['default'] == 100
|
||||
|
||||
def test_type2_no_action_params(self):
|
||||
"""Type 2 with only special params should work"""
|
||||
registry = Registry()
|
||||
|
||||
@registry.action('Save PDF')
|
||||
async def save_pdf(browser_session: BrowserSession):
|
||||
return ActionResult(extracted_content='Saved PDF')
|
||||
|
||||
action = registry.registry.actions['save_pdf']
|
||||
# Should have empty or minimal param model
|
||||
fields = action.param_model.model_fields
|
||||
assert len(fields) == 0 or all(f in ['title'] for f in fields)
|
||||
|
||||
def test_no_special_params_action(self):
|
||||
"""Test action with no special params (like wait action in Tools)"""
|
||||
registry = Registry()
|
||||
|
||||
@registry.action('Wait for x seconds default 3')
|
||||
async def wait(seconds: int = 3):
|
||||
await asyncio.sleep(seconds)
|
||||
return ActionResult(extracted_content=f'Waited {seconds} seconds')
|
||||
|
||||
# Should register successfully
|
||||
assert 'wait' in registry.registry.actions
|
||||
action = registry.registry.actions['wait']
|
||||
|
||||
# Should have seconds in param model
|
||||
assert 'seconds' in action.param_model.model_fields
|
||||
|
||||
# Should preserve default value
|
||||
schema = action.param_model.model_json_schema()
|
||||
assert schema['properties']['seconds']['default'] == 3
|
||||
|
||||
|
||||
class TestValidationRules:
|
||||
"""Test validation rules for action registration (from normalization tests)"""
|
||||
|
||||
def test_error_on_kwargs_in_original_function(self):
|
||||
"""Should error if original function has kwargs"""
|
||||
registry = Registry()
|
||||
|
||||
with pytest.raises(ValueError, match='kwargs.*not allowed'):
|
||||
|
||||
@registry.action('Bad action')
|
||||
async def bad_action(index: int, browser_session: BrowserSession, **kwargs):
|
||||
pass
|
||||
|
||||
def test_error_on_special_param_name_with_wrong_type(self):
|
||||
"""Should error if special param name used with wrong type"""
|
||||
registry = Registry()
|
||||
|
||||
# Using 'browser_session' with wrong type should error
|
||||
with pytest.raises(ValueError, match='conflicts with special argument.*browser_session: BrowserSession'):
|
||||
|
||||
@registry.action('Bad session')
|
||||
async def bad_session(browser_session: str):
|
||||
pass
|
||||
|
||||
def test_special_params_must_match_type(self):
|
||||
"""Special params with correct types should work"""
|
||||
registry = Registry()
|
||||
|
||||
@registry.action('Good action')
|
||||
async def good_action(
|
||||
index: int,
|
||||
browser_session: BrowserSession, # Correct type
|
||||
):
|
||||
return ActionResult()
|
||||
|
||||
assert 'good_action' in registry.registry.actions
|
||||
|
||||
|
||||
class TestDecoratedFunctionBehavior:
|
||||
"""Test behavior of decorated action functions (from normalization tests)"""
|
||||
|
||||
async def test_decorated_function_only_accepts_kwargs(self):
|
||||
"""Decorated functions should only accept kwargs, no positional args"""
|
||||
registry = Registry()
|
||||
|
||||
class MockBrowserSession:
|
||||
async def get_current_page(self):
|
||||
return None
|
||||
|
||||
@registry.action('Click')
|
||||
async def click(index: int, browser_session: BrowserSession):
|
||||
return ActionResult()
|
||||
|
||||
# Should raise error when called with positional args
|
||||
with pytest.raises(TypeError, match='positional arguments'):
|
||||
await click(5, MockBrowserSession())
|
||||
|
||||
async def test_decorated_function_accepts_params_model(self):
|
||||
"""Decorated function should accept params as model"""
|
||||
registry = Registry()
|
||||
|
||||
class MockBrowserSession:
|
||||
async def get_current_page(self):
|
||||
return None
|
||||
|
||||
@registry.action('Input text')
|
||||
async def input_text(index: int, text: str, browser_session: BrowserSession):
|
||||
return ActionResult(extracted_content=f'{index}:{text}')
|
||||
|
||||
# Get the generated param model class
|
||||
action = registry.registry.actions['input_text']
|
||||
ParamsModel = action.param_model
|
||||
|
||||
# Should work with params model
|
||||
result = await input_text(params=ParamsModel(index=5, text='hello'), browser_session=MockBrowserSession())
|
||||
assert result.extracted_content == '5:hello'
|
||||
|
||||
async def test_decorated_function_ignores_extra_kwargs(self):
|
||||
"""Decorated function should ignore extra kwargs for easy unpacking"""
|
||||
registry = Registry()
|
||||
|
||||
@registry.action('Simple action')
|
||||
async def simple_action(value: int):
|
||||
return ActionResult(extracted_content=str(value))
|
||||
|
||||
# Should work even with extra kwargs
|
||||
special_context = {
|
||||
'browser_session': None,
|
||||
'page_extraction_llm': create_mock_llm(),
|
||||
'context': {'extra': 'data'},
|
||||
'unknown_param': 'ignored',
|
||||
}
|
||||
|
||||
action = registry.registry.actions['simple_action']
|
||||
ParamsModel = action.param_model
|
||||
|
||||
result = await simple_action(params=ParamsModel(value=42), **special_context)
|
||||
assert result.extracted_content == '42'
|
||||
|
||||
|
||||
class TestParamsModelGeneration:
|
||||
"""Test automatic parameter model generation (from normalization tests)"""
|
||||
|
||||
def test_generates_model_from_non_special_args(self):
|
||||
"""Should generate param model from non-special positional args"""
|
||||
registry = Registry()
|
||||
|
||||
@registry.action('Complex action')
|
||||
async def complex_action(
|
||||
query: str,
|
||||
max_results: int,
|
||||
include_images: bool = True,
|
||||
browser_session: BrowserSession = None, # type: ignore
|
||||
):
|
||||
return ActionResult()
|
||||
|
||||
action = registry.registry.actions['complex_action']
|
||||
model_fields = action.param_model.model_fields
|
||||
|
||||
# Should include only non-special params
|
||||
assert 'query' in model_fields
|
||||
assert 'max_results' in model_fields
|
||||
assert 'include_images' in model_fields
|
||||
|
||||
# Should NOT include special params
|
||||
assert 'browser_session' not in model_fields
|
||||
|
||||
def test_preserves_type_annotations(self):
|
||||
"""Generated model should preserve type annotations"""
|
||||
registry = Registry()
|
||||
|
||||
@registry.action('Typed action')
|
||||
async def typed_action(
|
||||
count: int,
|
||||
rate: float,
|
||||
enabled: bool,
|
||||
name: str | None = None,
|
||||
browser_session: BrowserSession = None, # type: ignore
|
||||
):
|
||||
return ActionResult()
|
||||
|
||||
action = registry.registry.actions['typed_action']
|
||||
schema = action.param_model.model_json_schema()
|
||||
|
||||
# Check types are preserved
|
||||
assert schema['properties']['count']['type'] == 'integer'
|
||||
assert schema['properties']['rate']['type'] == 'number'
|
||||
assert schema['properties']['enabled']['type'] == 'boolean'
|
||||
# Optional should allow null
|
||||
assert 'null' in schema['properties']['name']['anyOf'][1]['type']
|
||||
|
||||
|
||||
class TestParameterOrdering:
|
||||
"""Test mixed ordering of parameters (from normalization tests)"""
|
||||
|
||||
def test_mixed_param_ordering(self):
|
||||
"""Should handle any ordering of action params and special params"""
|
||||
registry = Registry()
|
||||
from browser_use.llm.base import BaseChatModel
|
||||
|
||||
# Special params mixed throughout
|
||||
@registry.action('Mixed params')
|
||||
async def mixed_action(
|
||||
first: str,
|
||||
browser_session: BrowserSession,
|
||||
second: int,
|
||||
third: bool = True,
|
||||
page_extraction_llm: BaseChatModel = None, # type: ignore
|
||||
):
|
||||
return ActionResult()
|
||||
|
||||
action = registry.registry.actions['mixed_action']
|
||||
model_fields = action.param_model.model_fields
|
||||
|
||||
# Only action params in model
|
||||
assert set(model_fields.keys()) == {'first', 'second', 'third'}
|
||||
assert model_fields['third'].default is True
|
||||
|
||||
def test_extract_content_pattern_registration(self):
|
||||
"""Test that the extract_content pattern with mixed params registers correctly"""
|
||||
registry = Registry()
|
||||
|
||||
# This is the problematic pattern: positional arg, then special args, then kwargs with defaults
|
||||
@registry.action('Extract content from page')
|
||||
async def extract_content(
|
||||
goal: str,
|
||||
page_extraction_llm,
|
||||
include_links: bool = False,
|
||||
):
|
||||
return ActionResult(extracted_content=f'Goal: {goal}, include_links: {include_links}')
|
||||
|
||||
# Verify registration
|
||||
assert 'extract_content' in registry.registry.actions
|
||||
action = registry.registry.actions['extract_content']
|
||||
|
||||
# Check that the param model only includes user-facing params
|
||||
model_fields = action.param_model.model_fields
|
||||
assert 'goal' in model_fields
|
||||
assert 'include_links' in model_fields
|
||||
assert model_fields['include_links'].default is False
|
||||
|
||||
# Special params should NOT be in the model
|
||||
assert 'page' not in model_fields
|
||||
assert 'page_extraction_llm' not in model_fields
|
||||
|
||||
# Verify the action was properly registered
|
||||
assert action.name == 'extract_content'
|
||||
assert action.description == 'Extract content from page'
|
||||
|
||||
|
||||
class TestParamsModelArgsAndKwargs:
|
||||
async def test_browser_session_double_kwarg(self):
|
||||
"""Run the test to diagnose browser_session parameter issue
|
||||
|
||||
This test demonstrates the problem and our fix. The issue happens because:
|
||||
|
||||
1. In tools/service.py, we have:
|
||||
```python
|
||||
@registry.action('Google Sheets: Select a specific cell or range of cells')
|
||||
async def select_cell_or_range(browser_session: BrowserSession, cell_or_range: str):
|
||||
return await _select_cell_or_range(browser_session=browser_session, cell_or_range=cell_or_range)
|
||||
```
|
||||
|
||||
2. When registry.execute_action calls this function, it adds browser_session to extra_args:
|
||||
```python
|
||||
# In registry/service.py
|
||||
if 'browser_session' in parameter_names:
|
||||
extra_args['browser_session'] = browser_session
|
||||
```
|
||||
|
||||
3. Then later, when calling action.function:
|
||||
```python
|
||||
return await action.function(**params_dict, **extra_args)
|
||||
```
|
||||
|
||||
4. This effectively means browser_session is passed twice:
|
||||
- Once through extra_args['browser_session']
|
||||
- And again through params_dict['browser_session'] (from the original function)
|
||||
|
||||
The fix is to pass browser_session positionally in select_cell_or_range:
|
||||
```python
|
||||
return await _select_cell_or_range(browser_session, cell_or_range)
|
||||
```
|
||||
|
||||
This test confirms that this approach works.
|
||||
"""
|
||||
|
||||
from browser_use.tools.registry.service import Registry
|
||||
from browser_use.tools.registry.views import ActionModel
|
||||
|
||||
# Simple context for testing
|
||||
class TestContext:
|
||||
pass
|
||||
|
||||
class MockBrowserSession:
|
||||
async def get_current_page(self):
|
||||
return None
|
||||
|
||||
browser_session = MockBrowserSession()
|
||||
|
||||
# Create registry
|
||||
registry = Registry[TestContext]()
|
||||
|
||||
# Model that doesn't include browser_session (renamed to avoid pytest collecting it)
|
||||
class CellActionParams(ActionModel):
|
||||
value: str = Field(description='Test value')
|
||||
|
||||
# Model that includes browser_session
|
||||
class ModelWithBrowser(ActionModel):
|
||||
value: str = Field(description='Test value')
|
||||
browser_session: BrowserSession = None # type: ignore
|
||||
|
||||
# Create a custom param model for select_cell_or_range
|
||||
class CellRangeParams(ActionModel):
|
||||
cell_or_range: str = Field(description='Cell or range to select')
|
||||
|
||||
# Use the provided real browser session
|
||||
|
||||
# Test with the real issue: select_cell_or_range
|
||||
# logger.info('\n\n=== Test: Simulating select_cell_or_range issue with correct model ===')
|
||||
|
||||
# Define the function without using our registry - this will be a helper function
|
||||
async def _select_cell_or_range(browser_session, cell_or_range):
|
||||
"""Helper function for select_cell_or_range"""
|
||||
return f'Selected cell {cell_or_range}'
|
||||
|
||||
# This simulates the actual issue we're seeing in the real code
|
||||
# The browser_session parameter is in both the function signature and passed as a named arg
|
||||
@registry.action('Google Sheets: Select a cell or range', param_model=CellRangeParams)
|
||||
async def select_cell_or_range(browser_session: BrowserSession, cell_or_range: str):
|
||||
# logger.info(f'select_cell_or_range called with browser_session={browser_session}, cell_or_range={cell_or_range}')
|
||||
|
||||
# PROBLEMATIC LINE: browser_session is passed by name, matching the parameter name
|
||||
# This is what causes the "got multiple values" error in the real code
|
||||
return await _select_cell_or_range(browser_session=browser_session, cell_or_range=cell_or_range)
|
||||
|
||||
# Fix attempt: Register a version that uses positional args instead
|
||||
@registry.action('Google Sheets: Select a cell or range (fixed)', param_model=CellRangeParams)
|
||||
async def select_cell_or_range_fixed(browser_session: BrowserSession, cell_or_range: str):
|
||||
# logger.info(f'select_cell_or_range_fixed called with browser_session={browser_session}, cell_or_range={cell_or_range}')
|
||||
|
||||
# FIXED LINE: browser_session is passed positionally, avoiding the parameter name conflict
|
||||
return await _select_cell_or_range(browser_session, cell_or_range)
|
||||
|
||||
# Another attempt: explicitly call using **kwargs to simulate what the registry does
|
||||
@registry.action('Google Sheets: Select with kwargs', param_model=CellRangeParams)
|
||||
async def select_with_kwargs(browser_session: BrowserSession, cell_or_range: str):
|
||||
# logger.info(f'select_with_kwargs called with browser_session={browser_session}, cell_or_range={cell_or_range}')
|
||||
|
||||
# Get params and extra_args, like in Registry.execute_action
|
||||
params = {'cell_or_range': cell_or_range, 'browser_session': browser_session}
|
||||
extra_args = {'browser_session': browser_session}
|
||||
|
||||
# Try to call _select_cell_or_range with both params and extra_args
|
||||
# This will fail with "got multiple values for keyword argument 'browser_session'"
|
||||
try:
|
||||
# logger.info('Attempting to call with both params and extra_args (should fail):')
|
||||
await _select_cell_or_range(**params, **extra_args)
|
||||
except TypeError as e:
|
||||
# logger.info(f'Expected error: {e}')
|
||||
|
||||
# Remove browser_session from params to avoid the conflict
|
||||
params_fixed = dict(params)
|
||||
del params_fixed['browser_session']
|
||||
|
||||
# logger.info(f'Fixed params: {params_fixed}')
|
||||
|
||||
# This should work
|
||||
result = await _select_cell_or_range(**params_fixed, **extra_args)
|
||||
# logger.info(f'Success after fix: {result}')
|
||||
return result
|
||||
|
||||
# Test the original problematic version
|
||||
# logger.info('\n--- Testing original problematic version ---')
|
||||
try:
|
||||
result1 = await registry.execute_action(
|
||||
'select_cell_or_range',
|
||||
{'cell_or_range': 'A1:F100'},
|
||||
browser_session=browser_session, # type: ignore
|
||||
)
|
||||
# logger.info(f'Success! Result: {result1}')
|
||||
except Exception as e:
|
||||
logger.error(f'Error: {str(e)}')
|
||||
|
||||
# Test the fixed version (using positional args)
|
||||
# logger.info('\n--- Testing fixed version (positional args) ---')
|
||||
try:
|
||||
result2 = await registry.execute_action(
|
||||
'select_cell_or_range_fixed',
|
||||
{'cell_or_range': 'A1:F100'},
|
||||
browser_session=browser_session, # type: ignore
|
||||
)
|
||||
# logger.info(f'Success! Result: {result2}')
|
||||
except Exception as e:
|
||||
logger.error(f'Error: {str(e)}')
|
||||
|
||||
# Test with kwargs version that simulates what Registry.execute_action does
|
||||
# logger.info('\n--- Testing kwargs simulation version ---')
|
||||
try:
|
||||
result3 = await registry.execute_action(
|
||||
'select_with_kwargs',
|
||||
{'cell_or_range': 'A1:F100'},
|
||||
browser_session=browser_session, # type: ignore
|
||||
)
|
||||
# logger.info(f'Success! Result: {result3}')
|
||||
except Exception as e:
|
||||
logger.error(f'Error: {str(e)}')
|
||||
|
||||
# Manual test of our theory: browser_session is passed twice
|
||||
# logger.info('\n--- Direct test of our theory ---')
|
||||
try:
|
||||
# Create the model instance
|
||||
params = CellRangeParams(cell_or_range='A1:F100')
|
||||
|
||||
# First check if the extra_args approach works
|
||||
# logger.info('Checking if extra_args approach works:')
|
||||
extra_args = {'browser_session': browser_session}
|
||||
|
||||
# If we were to modify Registry.execute_action:
|
||||
# 1. Check if the function parameter needs browser_session
|
||||
parameter_names = ['browser_session', 'cell_or_range']
|
||||
browser_keys = ['browser_session', 'browser', 'browser_context']
|
||||
|
||||
# Create params dict
|
||||
param_dict = params.model_dump()
|
||||
# logger.info(f'params dict before: {param_dict}')
|
||||
|
||||
# Apply our fix: remove browser_session from params dict
|
||||
for key in browser_keys:
|
||||
if key in param_dict and key in extra_args:
|
||||
# logger.info(f'Removing {key} from params dict')
|
||||
del param_dict[key]
|
||||
|
||||
# logger.info(f'params dict after: {param_dict}')
|
||||
# logger.info(f'extra_args: {extra_args}')
|
||||
|
||||
# This would be the fixed code:
|
||||
# return await action.function(**param_dict, **extra_args)
|
||||
|
||||
# Call directly to test
|
||||
result3 = await select_cell_or_range(**param_dict, **extra_args)
|
||||
# logger.info(f'Success with our fix! Result: {result3}')
|
||||
except Exception as e:
|
||||
logger.error(f'Error with our manual test: {str(e)}')
|
||||
@@ -0,0 +1,161 @@
|
||||
"""
|
||||
Simplified tests for URL shortening functionality in Agent service.
|
||||
|
||||
Three focused tests:
|
||||
1. Input message processing with URL shortening
|
||||
2. Output processing with custom actions and URL restoration
|
||||
3. End-to-end pipeline test
|
||||
"""
|
||||
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from browser_use.agent.service import Agent
|
||||
from browser_use.agent.views import AgentOutput
|
||||
from browser_use.llm.messages import AssistantMessage, BaseMessage, UserMessage
|
||||
|
||||
# Super long URL to reuse across tests - much longer than the 25 character limit
|
||||
# Includes both query params (?...) and fragment params (#...)
|
||||
SUPER_LONG_URL = 'https://documentation.example-company.com/api/v3/enterprise/user-management/endpoints/administration/create-new-user-account-with-permissions/advanced-settings?format=detailed-json&version=3.2.1×tamp=1699123456789&session_id=abc123def456ghi789&authentication_token=very_long_authentication_token_string_here&include_metadata=true&expand_relationships=user_groups,permissions,roles&sort_by=created_at&order=desc&page_size=100&include_deprecated_fields=false&api_key=super_long_api_key_that_exceeds_normal_limits#section=user_management&tab=advanced&view=detailed&scroll_to=permissions_table&highlight=admin_settings&filter=active_users&expand_all=true&debug_mode=enabled'
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def agent():
|
||||
"""Create an agent instance for testing URL shortening functionality."""
|
||||
from tests.ci.conftest import create_mock_llm
|
||||
|
||||
return Agent(task='Test URL shortening', llm=create_mock_llm(), url_shortening_limit=25)
|
||||
|
||||
|
||||
class TestUrlShorteningInputProcessing:
|
||||
"""Test URL shortening for input messages."""
|
||||
|
||||
def test_process_input_messages_with_url_shortening(self, agent: Agent):
|
||||
"""Test that long URLs in input messages are shortened and mappings stored."""
|
||||
original_content = f'Please visit {SUPER_LONG_URL} and extract information'
|
||||
|
||||
messages: list[BaseMessage] = [UserMessage(content=original_content)]
|
||||
|
||||
# Process messages (modifies messages in-place and returns URL mappings)
|
||||
url_mappings = agent._process_messsages_and_replace_long_urls_shorter_ones(messages)
|
||||
|
||||
# Verify URL was shortened in the message (modified in-place)
|
||||
processed_content = messages[0].content or ''
|
||||
assert processed_content != original_content
|
||||
assert 'https://documentation.example-company.com' in processed_content
|
||||
assert len(processed_content) < len(original_content)
|
||||
|
||||
# Verify URL mapping was returned
|
||||
assert len(url_mappings) == 1
|
||||
shortened_url = next(iter(url_mappings.keys()))
|
||||
assert url_mappings[shortened_url] == SUPER_LONG_URL
|
||||
|
||||
def test_process_user_and_assistant_messages_with_url_shortening(self, agent: Agent):
|
||||
"""Test URL shortening in both UserMessage and AssistantMessage."""
|
||||
user_content = f'I need to access {SUPER_LONG_URL} for the API documentation'
|
||||
assistant_content = f'I will help you navigate to {SUPER_LONG_URL} to retrieve the documentation'
|
||||
|
||||
messages: list[BaseMessage] = [UserMessage(content=user_content), AssistantMessage(content=assistant_content)]
|
||||
|
||||
# Process messages (modifies messages in-place and returns URL mappings)
|
||||
url_mappings = agent._process_messsages_and_replace_long_urls_shorter_ones(messages)
|
||||
|
||||
# Verify URL was shortened in both messages
|
||||
user_processed_content = messages[0].content or ''
|
||||
assistant_processed_content = messages[1].content or ''
|
||||
|
||||
assert user_processed_content != user_content
|
||||
assert assistant_processed_content != assistant_content
|
||||
assert 'https://documentation.example-company.com' in user_processed_content
|
||||
assert 'https://documentation.example-company.com' in assistant_processed_content
|
||||
assert len(user_processed_content) < len(user_content)
|
||||
assert len(assistant_processed_content) < len(assistant_content)
|
||||
|
||||
# Verify URL mapping was returned (should be same shortened URL for both occurrences)
|
||||
assert len(url_mappings) == 1
|
||||
shortened_url = next(iter(url_mappings.keys()))
|
||||
assert url_mappings[shortened_url] == SUPER_LONG_URL
|
||||
|
||||
|
||||
class TestUrlShorteningOutputProcessing:
|
||||
"""Test URL restoration for output processing with custom actions."""
|
||||
|
||||
def test_process_output_with_custom_actions_and_url_restoration(self, agent: Agent):
|
||||
"""Test that shortened URLs in AgentOutput with custom actions are restored."""
|
||||
# Set up URL mapping (simulating previous shortening)
|
||||
shortened_url: str = agent._replace_urls_in_text(SUPER_LONG_URL)[0]
|
||||
url_mappings = {shortened_url: SUPER_LONG_URL}
|
||||
|
||||
# Create AgentOutput with shortened URLs using JSON parsing
|
||||
output_json = {
|
||||
'thinking': f'I need to navigate to {shortened_url} for documentation',
|
||||
'evaluation_previous_goal': 'Successfully processed the request',
|
||||
'memory': f'Found useful info at {shortened_url}',
|
||||
'next_goal': 'Complete the documentation review',
|
||||
'action': [{'navigate': {'url': shortened_url, 'new_tab': False}}],
|
||||
}
|
||||
|
||||
# Create properly typed AgentOutput with custom actions
|
||||
tools = agent.tools
|
||||
ActionModel = tools.registry.create_action_model()
|
||||
AgentOutputWithActions = AgentOutput.type_with_custom_actions(ActionModel)
|
||||
agent_output = AgentOutputWithActions.model_validate_json(json.dumps(output_json))
|
||||
|
||||
# Process the output to restore URLs (modifies agent_output in-place)
|
||||
agent._recursive_process_all_strings_inside_pydantic_model(agent_output, url_mappings)
|
||||
|
||||
# Verify URLs were restored in all locations
|
||||
assert SUPER_LONG_URL in (agent_output.thinking or '')
|
||||
assert SUPER_LONG_URL in (agent_output.memory or '')
|
||||
action_data = agent_output.action[0].model_dump()
|
||||
assert action_data['navigate']['url'] == SUPER_LONG_URL
|
||||
|
||||
|
||||
class TestUrlShorteningEndToEnd:
|
||||
"""Test complete URL shortening pipeline end-to-end."""
|
||||
|
||||
def test_complete_url_shortening_pipeline(self, agent: Agent):
|
||||
"""Test the complete pipeline: input shortening -> processing -> output restoration."""
|
||||
|
||||
# Step 1: Input processing with URL shortening
|
||||
original_content = f'Navigate to {SUPER_LONG_URL} and extract the API documentation'
|
||||
|
||||
messages: list[BaseMessage] = [UserMessage(content=original_content)]
|
||||
|
||||
url_mappings = agent._process_messsages_and_replace_long_urls_shorter_ones(messages)
|
||||
|
||||
# Verify URL was shortened in input
|
||||
assert len(url_mappings) == 1
|
||||
shortened_url = next(iter(url_mappings.keys()))
|
||||
assert url_mappings[shortened_url] == SUPER_LONG_URL
|
||||
assert shortened_url in (messages[0].content or '')
|
||||
|
||||
# Step 2: Simulate agent output with shortened URL
|
||||
output_json = {
|
||||
'thinking': f'I will navigate to {shortened_url} to get the documentation',
|
||||
'evaluation_previous_goal': 'Starting documentation extraction',
|
||||
'memory': f'Target URL: {shortened_url}',
|
||||
'next_goal': 'Extract API documentation',
|
||||
'action': [{'navigate': {'url': shortened_url, 'new_tab': True}}],
|
||||
}
|
||||
|
||||
# Create AgentOutput with custom actions
|
||||
tools = agent.tools
|
||||
ActionModel = tools.registry.create_action_model()
|
||||
AgentOutputWithActions = AgentOutput.type_with_custom_actions(ActionModel)
|
||||
agent_output = AgentOutputWithActions.model_validate_json(json.dumps(output_json))
|
||||
|
||||
# Step 3: Output processing with URL restoration (modifies agent_output in-place)
|
||||
agent._recursive_process_all_strings_inside_pydantic_model(agent_output, url_mappings)
|
||||
|
||||
# Verify complete pipeline worked correctly
|
||||
assert SUPER_LONG_URL in (agent_output.thinking or '')
|
||||
assert SUPER_LONG_URL in (agent_output.memory or '')
|
||||
action_data = agent_output.action[0].model_dump()
|
||||
assert action_data['navigate']['url'] == SUPER_LONG_URL
|
||||
assert action_data['navigate']['new_tab'] is True
|
||||
|
||||
# Verify original shortened content is no longer present
|
||||
assert shortened_url not in (agent_output.thinking or '')
|
||||
assert shortened_url not in (agent_output.memory or '')
|
||||
@@ -0,0 +1,13 @@
|
||||
from browser_use.utils import _is_newer_browser_use_version
|
||||
|
||||
|
||||
def test_prerelease_is_newer_than_previous_stable():
|
||||
assert _is_newer_browser_use_version('0.12.9', '0.13.0rc3') is False
|
||||
|
||||
|
||||
def test_stable_release_is_newer_than_same_release_candidate():
|
||||
assert _is_newer_browser_use_version('0.13.0', '0.13.0rc3') is True
|
||||
|
||||
|
||||
def test_later_release_candidate_is_newer():
|
||||
assert _is_newer_browser_use_version('0.13.0rc4', '0.13.0rc3') is True
|
||||
@@ -0,0 +1,382 @@
|
||||
"""Test autocomplete/combobox field detection, value readback, and input clearing.
|
||||
|
||||
Tests cover:
|
||||
- Value mismatch detection when JS rewrites input value
|
||||
- Combobox field detection (role=combobox + aria-autocomplete)
|
||||
- Datalist field detection (input with list attribute)
|
||||
- No false positives on plain inputs
|
||||
- Sensitive data skips value verification
|
||||
- Pre-filled input clearing (clear=True default)
|
||||
- Pre-filled input appending (clear=False)
|
||||
- Concatenation auto-retry when clear fails
|
||||
- Autocomplete delay before next action
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
import pytest
|
||||
from pytest_httpserver import HTTPServer
|
||||
|
||||
from browser_use.agent.views import ActionResult
|
||||
from browser_use.browser import BrowserSession
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
from browser_use.tools.service import Tools
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def http_server():
|
||||
"""Create and provide a test HTTP server with autocomplete test pages."""
|
||||
server = HTTPServer()
|
||||
server.start()
|
||||
|
||||
# Page 1: Input with JS that rewrites value on change (simulates autocomplete replacing text)
|
||||
server.expect_request('/autocomplete-rewrite').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Autocomplete Rewrite Test</title></head>
|
||||
<body>
|
||||
<input id="search" type="text" />
|
||||
<script>
|
||||
const input = document.getElementById('search');
|
||||
input.addEventListener('change', function() {
|
||||
// Simulate autocomplete rewriting the value
|
||||
this.value = 'REWRITTEN_' + this.value;
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
# Page 2: Input with role=combobox + aria-autocomplete=list + aria-controls + listbox
|
||||
server.expect_request('/combobox-field').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Combobox Field Test</title></head>
|
||||
<body>
|
||||
<div>
|
||||
<input id="combo" type="text" role="combobox"
|
||||
aria-autocomplete="list" aria-controls="suggestions-list"
|
||||
aria-expanded="false" />
|
||||
<ul id="suggestions-list" role="listbox" style="display:none;">
|
||||
<li role="option">Option A</li>
|
||||
<li role="option">Option B</li>
|
||||
</ul>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
# Page 3: Input with list attribute pointing to a datalist
|
||||
server.expect_request('/datalist-field').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Datalist Field Test</title></head>
|
||||
<body>
|
||||
<input id="city" type="text" list="suggestions" />
|
||||
<datalist id="suggestions">
|
||||
<option value="New York">
|
||||
<option value="Los Angeles">
|
||||
<option value="Chicago">
|
||||
</datalist>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
# Page 4: Plain input with no autocomplete attributes
|
||||
server.expect_request('/normal-input').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Normal Input Test</title></head>
|
||||
<body>
|
||||
<input id="plain" type="text" placeholder="Just a normal input" />
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
# Page 5: Pre-filled input to test clear=True behavior
|
||||
server.expect_request('/prefilled-input').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Pre-filled Input Test</title></head>
|
||||
<body>
|
||||
<input id="prefilled" type="text" value="old value" />
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
# Page 6: Input where clear fails — input event listener restores old text
|
||||
# Simulates a framework-controlled input where clearing triggers re-render with old state
|
||||
server.expect_request('/sticky-input').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Sticky Input Test</title></head>
|
||||
<body>
|
||||
<input id="sticky" type="text" value="prefix_" />
|
||||
<script>
|
||||
var el = document.getElementById('sticky');
|
||||
var clearAttempts = 0;
|
||||
// Intercept value clears: restore old value on first two clears
|
||||
// (simulates framework re-rendering with stale state)
|
||||
el.addEventListener('input', function() {
|
||||
if (el.value === '' && clearAttempts < 2) {
|
||||
clearAttempts++;
|
||||
el.value = 'prefix_';
|
||||
}
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
yield server
|
||||
server.stop()
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def base_url(http_server):
|
||||
"""Return the base URL for the test HTTP server."""
|
||||
return f'http://{http_server.host}:{http_server.port}'
|
||||
|
||||
|
||||
@pytest.fixture(scope='module')
|
||||
async def browser_session():
|
||||
"""Create and provide a Browser instance for testing."""
|
||||
browser_session = BrowserSession(
|
||||
browser_profile=BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
keep_alive=True,
|
||||
chromium_sandbox=False,
|
||||
)
|
||||
)
|
||||
await browser_session.start()
|
||||
yield browser_session
|
||||
await browser_session.kill()
|
||||
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
def tools():
|
||||
"""Create and provide a Tools instance."""
|
||||
return Tools()
|
||||
|
||||
|
||||
class TestAutocompleteInteraction:
|
||||
"""Test autocomplete/combobox detection and value readback."""
|
||||
|
||||
async def test_value_mismatch_detected(self, tools: Tools, browser_session: BrowserSession, base_url: str):
|
||||
"""Type into a field whose JS rewrites the value on change. Assert the ActionResult notes the mismatch."""
|
||||
await tools.navigate(url=f'{base_url}/autocomplete-rewrite', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.3)
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
input_index = await browser_session.get_index_by_id('search')
|
||||
assert input_index is not None, 'Could not find search input'
|
||||
|
||||
result = await tools.input(index=input_index, text='hello', browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'differs from typed text' in result.extracted_content, (
|
||||
f'Expected mismatch note in extracted_content, got: {result.extracted_content}'
|
||||
)
|
||||
|
||||
async def test_combobox_field_detected(self, tools: Tools, browser_session: BrowserSession, base_url: str):
|
||||
"""Type into a combobox field. Assert the ActionResult includes autocomplete guidance."""
|
||||
await tools.navigate(url=f'{base_url}/combobox-field', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.3)
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
combo_index = await browser_session.get_index_by_id('combo')
|
||||
assert combo_index is not None, 'Could not find combobox input'
|
||||
|
||||
result = await tools.input(index=combo_index, text='test', browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'autocomplete field' in result.extracted_content, (
|
||||
f'Expected autocomplete guidance in extracted_content, got: {result.extracted_content}'
|
||||
)
|
||||
|
||||
async def test_datalist_field_detected(self, tools: Tools, browser_session: BrowserSession, base_url: str):
|
||||
"""Type into a datalist-backed field. Assert the ActionResult includes autocomplete guidance."""
|
||||
await tools.navigate(url=f'{base_url}/datalist-field', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.3)
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
city_index = await browser_session.get_index_by_id('city')
|
||||
assert city_index is not None, 'Could not find datalist input'
|
||||
|
||||
result = await tools.input(index=city_index, text='New', browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'autocomplete field' in result.extracted_content, (
|
||||
f'Expected autocomplete guidance in extracted_content, got: {result.extracted_content}'
|
||||
)
|
||||
|
||||
async def test_normal_input_no_false_positive(self, tools: Tools, browser_session: BrowserSession, base_url: str):
|
||||
"""Type into a plain input. Assert the ActionResult does NOT contain autocomplete guidance."""
|
||||
await tools.navigate(url=f'{base_url}/normal-input', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.3)
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
plain_index = await browser_session.get_index_by_id('plain')
|
||||
assert plain_index is not None, 'Could not find plain input'
|
||||
|
||||
result = await tools.input(index=plain_index, text='hello', browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'autocomplete field' not in result.extracted_content, (
|
||||
f'Got false positive autocomplete guidance on plain input: {result.extracted_content}'
|
||||
)
|
||||
|
||||
async def test_sensitive_data_skips_value_verification(self, tools: Tools, browser_session: BrowserSession, base_url: str):
|
||||
"""Type sensitive data into the rewrite field. Assert no 'differs from typed text' note appears."""
|
||||
await tools.navigate(url=f'{base_url}/autocomplete-rewrite', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.3)
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
input_index = await browser_session.get_index_by_id('search')
|
||||
assert input_index is not None, 'Could not find search input'
|
||||
|
||||
# Use tools.act() with sensitive_data to trigger the sensitive code path
|
||||
result = await tools.input(
|
||||
index=input_index,
|
||||
text='secret123',
|
||||
browser_session=browser_session,
|
||||
sensitive_data={'password': 'secret123'},
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'differs from typed text' not in result.extracted_content, (
|
||||
f'Sensitive data should not show value mismatch: {result.extracted_content}'
|
||||
)
|
||||
|
||||
async def test_prefilled_input_cleared_by_default(self, tools: Tools, browser_session: BrowserSession, base_url: str):
|
||||
"""Type into a pre-filled input with clear=True (default). Field should contain only the new text."""
|
||||
await tools.navigate(url=f'{base_url}/prefilled-input', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.3)
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
idx = await browser_session.get_index_by_id('prefilled')
|
||||
assert idx is not None, 'Could not find prefilled input'
|
||||
|
||||
result = await tools.input(index=idx, text='new value', browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is None, f'Input action failed: {result.error}'
|
||||
|
||||
# Read back the actual DOM value via CDP
|
||||
cdp_session = await browser_session.get_or_create_cdp_session()
|
||||
readback = await cdp_session.cdp_client.send.Runtime.evaluate(
|
||||
params={'expression': "document.getElementById('prefilled').value"},
|
||||
session_id=cdp_session.session_id,
|
||||
)
|
||||
actual = readback.get('result', {}).get('value', '')
|
||||
assert actual == 'new value', f'Expected "new value", got "{actual}" — clear=True did not remove old text'
|
||||
|
||||
async def test_prefilled_input_append_with_clear_false(self, tools: Tools, browser_session: BrowserSession, base_url: str):
|
||||
"""Type into a pre-filled input with clear=False. Field should contain old + new text."""
|
||||
await tools.navigate(url=f'{base_url}/prefilled-input', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.3)
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
idx = await browser_session.get_index_by_id('prefilled')
|
||||
assert idx is not None, 'Could not find prefilled input'
|
||||
|
||||
result = await tools.input(index=idx, text=' appended', clear=False, browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is None, f'Input action failed: {result.error}'
|
||||
|
||||
# Read back the actual DOM value via CDP
|
||||
cdp_session = await browser_session.get_or_create_cdp_session()
|
||||
readback = await cdp_session.cdp_client.send.Runtime.evaluate(
|
||||
params={'expression': "document.getElementById('prefilled').value"},
|
||||
session_id=cdp_session.session_id,
|
||||
)
|
||||
actual = readback.get('result', {}).get('value', '')
|
||||
assert 'old value' in actual and 'appended' in actual, f'Expected old text + appended text, got "{actual}"'
|
||||
|
||||
async def test_concatenation_retry_on_sticky_field(self, tools: Tools, browser_session: BrowserSession, base_url: str):
|
||||
"""Type into a field where clearing is resisted by JS. The retry should fix the value."""
|
||||
await tools.navigate(url=f'{base_url}/sticky-input', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.3)
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
idx = await browser_session.get_index_by_id('sticky')
|
||||
assert idx is not None, 'Could not find sticky input'
|
||||
|
||||
result = await tools.input(index=idx, text='typed_text', browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is None, f'Input action failed: {result.error}'
|
||||
|
||||
# The retry mechanism uses a native setter to bypass the event listener.
|
||||
# Read back the final DOM value.
|
||||
cdp_session = await browser_session.get_or_create_cdp_session()
|
||||
readback = await cdp_session.cdp_client.send.Runtime.evaluate(
|
||||
params={'expression': "document.getElementById('sticky').value"},
|
||||
session_id=cdp_session.session_id,
|
||||
)
|
||||
actual = readback.get('result', {}).get('value', '')
|
||||
# The retry should have set the value to just "typed_text" via the native setter.
|
||||
# Even if the event listener fires on the retry's dispatched events, the native setter
|
||||
# bypasses instance-level interception. The value may or may not be perfect depending
|
||||
# on how the JS listener interacts, but it should not be "prefix_typed_text" (raw concatenation).
|
||||
assert actual != 'prefix_typed_text', f'Got raw concatenation "{actual}" — retry should have prevented this'
|
||||
|
||||
async def test_combobox_field_adds_delay(self, tools: Tools, browser_session: BrowserSession, base_url: str):
|
||||
"""Typing into a combobox (role=combobox) field should take >= 400ms due to the mechanical delay."""
|
||||
import time
|
||||
|
||||
await tools.navigate(url=f'{base_url}/combobox-field', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.3)
|
||||
await browser_session.get_browser_state_summary()
|
||||
combo_idx = await browser_session.get_index_by_id('combo')
|
||||
assert combo_idx is not None
|
||||
|
||||
t0 = time.monotonic()
|
||||
await tools.input(index=combo_idx, text='hi', browser_session=browser_session)
|
||||
duration = time.monotonic() - t0
|
||||
|
||||
# The 400ms sleep is a hard floor — total duration must exceed it
|
||||
assert duration >= 0.4, f'Combobox delay not present: input took only {duration:.3f}s (expected >= 0.4s)'
|
||||
|
||||
async def test_datalist_field_no_delay(self, tools: Tools, browser_session: BrowserSession, base_url: str):
|
||||
"""Native datalist fields should NOT get the 400ms delay — browser handles them instantly."""
|
||||
import time
|
||||
|
||||
await tools.navigate(url=f'{base_url}/datalist-field', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.3)
|
||||
await browser_session.get_browser_state_summary()
|
||||
city_idx = await browser_session.get_index_by_id('city')
|
||||
assert city_idx is not None
|
||||
|
||||
t0 = time.monotonic()
|
||||
await tools.input(index=city_idx, text='Chi', browser_session=browser_session)
|
||||
duration = time.monotonic() - t0
|
||||
|
||||
# Datalist fields should complete without the 400ms tax.
|
||||
# Normal typing for 3 chars takes well under 400ms.
|
||||
assert duration < 0.4, f'Datalist field got unexpected delay: {duration:.3f}s (should be < 0.4s)'
|
||||
@@ -0,0 +1,275 @@
|
||||
import pytest
|
||||
from pytest_httpserver import HTTPServer
|
||||
|
||||
from browser_use.agent.views import ActionResult
|
||||
from browser_use.browser import BrowserSession
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
from browser_use.tools.service import Tools
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def http_server():
|
||||
"""Create and provide a test HTTP server that serves static content."""
|
||||
server = HTTPServer()
|
||||
server.start()
|
||||
|
||||
# Add route for ARIA menu test page
|
||||
server.expect_request('/aria-menu').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>ARIA Menu Test</title>
|
||||
<style>
|
||||
.menu {
|
||||
list-style: none;
|
||||
padding: 0;
|
||||
margin: 0;
|
||||
border: 1px solid #ccc;
|
||||
background: white;
|
||||
width: 200px;
|
||||
}
|
||||
.menu-item {
|
||||
padding: 10px 20px;
|
||||
border-bottom: 1px solid #eee;
|
||||
}
|
||||
.menu-item:hover {
|
||||
background: #f0f0f0;
|
||||
}
|
||||
.menu-item-anchor {
|
||||
text-decoration: none;
|
||||
color: #333;
|
||||
display: block;
|
||||
}
|
||||
#result {
|
||||
margin-top: 20px;
|
||||
padding: 10px;
|
||||
border: 1px solid #ddd;
|
||||
min-height: 20px;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<h1>ARIA Menu Test</h1>
|
||||
<p>This menu uses ARIA roles instead of native select elements</p>
|
||||
|
||||
<!-- Exactly like the HTML provided in the issue -->
|
||||
<ul class="menu menu-format-standard menu-regular" role="menu" id="pyNavigation1752753375773" style="display: block;">
|
||||
<li class="menu-item menu-item-enabled" role="presentation">
|
||||
<a href="#" onclick="pd(event);" class="menu-item-anchor" tabindex="0" role="menuitem">
|
||||
<span class="menu-item-title-wrap"><span class="menu-item-title">Filter</span></span>
|
||||
</a>
|
||||
</li>
|
||||
<li class="menu-item menu-item-enabled" role="presentation" id="menu-item-$PpyNavigation1752753375773$ppyElements$l2">
|
||||
<a href="#" onclick="pd(event);" class="menu-item-anchor menu-item-expand" tabindex="0" role="menuitem" aria-haspopup="true">
|
||||
<span class="menu-item-title-wrap"><span class="menu-item-title">Sort</span></span>
|
||||
</a>
|
||||
<div class="menu-panel-wrapper">
|
||||
<ul class="menu menu-format-standard menu-regular" role="menu" id="$PpyNavigation1752753375773$ppyElements$l2">
|
||||
<li class="menu-item menu-item-enabled" role="presentation">
|
||||
<a href="#" onclick="pd(event);" class="menu-item-anchor" tabindex="0" role="menuitem">
|
||||
<span class="menu-item-title-wrap"><span class="menu-item-title">Lowest to highest</span></span>
|
||||
</a>
|
||||
</li>
|
||||
<li class="menu-item menu-item-enabled" role="presentation">
|
||||
<a href="#" onclick="pd(event);" class="menu-item-anchor" tabindex="0" role="menuitem">
|
||||
<span class="menu-item-title-wrap"><span class="menu-item-title">Highest to lowest</span></span>
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
</li>
|
||||
<li class="menu-item menu-item-enabled" role="presentation">
|
||||
<a href="#" onclick="pd(event);" class="menu-item-anchor" tabindex="0" role="menuitem">
|
||||
<span class="menu-item-title-wrap"><span class="menu-item-title">Appearance</span></span>
|
||||
</a>
|
||||
</li>
|
||||
<li class="menu-item menu-item-enabled" role="presentation">
|
||||
<a href="#" onclick="pd(event);" class="menu-item-anchor" tabindex="0" role="menuitem">
|
||||
<span class="menu-item-title-wrap"><span class="menu-item-title">Summarize</span></span>
|
||||
</a>
|
||||
</li>
|
||||
<li class="menu-item menu-item-enabled" role="presentation">
|
||||
<a href="#" onclick="pd(event);" class="menu-item-anchor" tabindex="0" role="menuitem">
|
||||
<span class="menu-item-title-wrap"><span class="menu-item-title">Delete</span></span>
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<div id="result">Click an option to see the result</div>
|
||||
|
||||
<script>
|
||||
// Mock the pd function that prevents default
|
||||
function pd(event) {
|
||||
event.preventDefault();
|
||||
const text = event.target.closest('[role="menuitem"]').textContent.trim();
|
||||
document.getElementById('result').textContent = 'Clicked: ' + text;
|
||||
}
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
yield server
|
||||
server.stop()
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def base_url(http_server):
|
||||
"""Return the base URL for the test HTTP server."""
|
||||
return f'http://{http_server.host}:{http_server.port}'
|
||||
|
||||
|
||||
@pytest.fixture(scope='module')
|
||||
async def browser_session():
|
||||
"""Create and provide a Browser instance with security disabled."""
|
||||
browser_session = BrowserSession(
|
||||
browser_profile=BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
keep_alive=True,
|
||||
chromium_sandbox=False, # Disable sandbox for CI environment
|
||||
)
|
||||
)
|
||||
await browser_session.start()
|
||||
yield browser_session
|
||||
await browser_session.kill()
|
||||
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
def tools():
|
||||
"""Create and provide a Tools instance."""
|
||||
return Tools()
|
||||
|
||||
|
||||
class TestARIAMenuDropdown:
|
||||
"""Test ARIA menu support for get_dropdown_options and select_dropdown_option."""
|
||||
|
||||
@pytest.mark.skip(reason='TODO: fix')
|
||||
async def test_get_dropdown_options_with_aria_menu(self, tools, browser_session: BrowserSession, base_url):
|
||||
"""Test that get_dropdown_options can retrieve options from ARIA menus."""
|
||||
# Navigate to the ARIA menu test page
|
||||
await tools.navigate(url=f'{base_url}/aria-menu', new_tab=False, browser_session=browser_session)
|
||||
|
||||
# Wait for the page to load
|
||||
from browser_use.browser.events import NavigationCompleteEvent
|
||||
|
||||
await browser_session.event_bus.expect(NavigationCompleteEvent, timeout=10.0)
|
||||
|
||||
# Initialize the DOM state to populate the selector map
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
# Find the ARIA menu element by ID
|
||||
menu_index = await browser_session.get_index_by_id('pyNavigation1752753375773')
|
||||
|
||||
assert menu_index is not None, 'Could not find ARIA menu element'
|
||||
|
||||
# Execute the action with the menu index
|
||||
result = await tools.dropdown_options(index=menu_index, browser_session=browser_session)
|
||||
|
||||
# Verify the result structure
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
|
||||
# Expected ARIA menu options
|
||||
expected_options = ['Filter', 'Sort', 'Appearance', 'Summarize', 'Delete']
|
||||
|
||||
# Verify all options are returned
|
||||
for option in expected_options:
|
||||
assert option in result.extracted_content, f"Option '{option}' not found in result content"
|
||||
|
||||
# Verify the instruction for using the text in select_dropdown is included
|
||||
assert 'Use the exact text string in select_dropdown' in result.extracted_content
|
||||
|
||||
@pytest.mark.skip(reason='TODO: fix')
|
||||
async def test_select_dropdown_option_with_aria_menu(self, tools, browser_session: BrowserSession, base_url):
|
||||
"""Test that select_dropdown_option can select an option from ARIA menus."""
|
||||
# Navigate to the ARIA menu test page
|
||||
await tools.navigate(url=f'{base_url}/aria-menu', new_tab=False, browser_session=browser_session)
|
||||
|
||||
# Wait for the page to load
|
||||
from browser_use.browser.events import NavigationCompleteEvent
|
||||
|
||||
await browser_session.event_bus.expect(NavigationCompleteEvent, timeout=10.0)
|
||||
|
||||
# Initialize the DOM state to populate the selector map
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
# Find the ARIA menu element by ID
|
||||
menu_index = await browser_session.get_index_by_id('pyNavigation1752753375773')
|
||||
|
||||
assert menu_index is not None, 'Could not find ARIA menu element'
|
||||
|
||||
# Execute the action with the menu index to select "Filter"
|
||||
result = await tools.select_dropdown(index=menu_index, text='Filter', browser_session=browser_session)
|
||||
|
||||
# Verify the result structure
|
||||
assert isinstance(result, ActionResult)
|
||||
|
||||
# Core logic validation: Verify selection was successful
|
||||
assert result.extracted_content is not None
|
||||
assert 'selected option' in result.extracted_content.lower() or 'clicked' in result.extracted_content.lower()
|
||||
assert 'Filter' in result.extracted_content
|
||||
|
||||
# Verify the click actually had an effect on the page using CDP
|
||||
cdp_session = await browser_session.get_or_create_cdp_session()
|
||||
result = await cdp_session.cdp_client.send.Runtime.evaluate(
|
||||
params={'expression': "document.getElementById('result').textContent", 'returnByValue': True},
|
||||
session_id=cdp_session.session_id,
|
||||
)
|
||||
result_text = result.get('result', {}).get('value', '')
|
||||
assert 'Filter' in result_text, f"Expected 'Filter' in result text, got '{result_text}'"
|
||||
|
||||
@pytest.mark.skip(reason='TODO: fix')
|
||||
async def test_get_dropdown_options_with_nested_aria_menu(self, tools, browser_session: BrowserSession, base_url):
|
||||
"""Test that get_dropdown_options can handle nested ARIA menus (like Sort submenu)."""
|
||||
# Navigate to the ARIA menu test page
|
||||
await tools.navigate(url=f'{base_url}/aria-menu', new_tab=False, browser_session=browser_session)
|
||||
|
||||
# Wait for the page to load
|
||||
from browser_use.browser.events import NavigationCompleteEvent
|
||||
|
||||
await browser_session.event_bus.expect(NavigationCompleteEvent, timeout=10.0)
|
||||
|
||||
# Initialize the DOM state to populate the selector map
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
# Get the selector map
|
||||
selector_map = await browser_session.get_selector_map()
|
||||
|
||||
# Find the nested ARIA menu element in the selector map
|
||||
nested_menu_index = None
|
||||
for idx, element in selector_map.items():
|
||||
# Look for the nested UL with id containing "$PpyNavigation"
|
||||
if (
|
||||
element.tag_name.lower() == 'ul'
|
||||
and '$PpyNavigation' in str(element.attributes.get('id', ''))
|
||||
and element.attributes.get('role') == 'menu'
|
||||
):
|
||||
nested_menu_index = idx
|
||||
break
|
||||
|
||||
# The nested menu might not be in the selector map initially if it's hidden
|
||||
# In that case, we should test the main menu
|
||||
if nested_menu_index is None:
|
||||
# Find the main menu instead
|
||||
for idx, element in selector_map.items():
|
||||
if element.tag_name.lower() == 'ul' and element.attributes.get('id') == 'pyNavigation1752753375773':
|
||||
nested_menu_index = idx
|
||||
break
|
||||
|
||||
assert nested_menu_index is not None, (
|
||||
f'Could not find any ARIA menu element in selector map. Available elements: {[f"{idx}: {element.tag_name}" for idx, element in selector_map.items()]}'
|
||||
)
|
||||
|
||||
# Execute the action with the menu index
|
||||
result = await tools.dropdown_options(index=nested_menu_index, browser_session=browser_session)
|
||||
|
||||
# Verify the result structure
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
|
||||
# The action should return some menu options
|
||||
assert 'Use the exact text string in select_dropdown' in result.extracted_content
|
||||
@@ -0,0 +1,517 @@
|
||||
"""Test GetDropdownOptionsEvent and SelectDropdownOptionEvent functionality.
|
||||
|
||||
This file consolidates all tests related to dropdown functionality including:
|
||||
- Native <select> dropdowns
|
||||
- ARIA role="menu" dropdowns
|
||||
- Custom dropdown implementations
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from pytest_httpserver import HTTPServer
|
||||
|
||||
from browser_use.agent.views import ActionResult
|
||||
from browser_use.browser import BrowserSession
|
||||
from browser_use.browser.events import GetDropdownOptionsEvent, NavigationCompleteEvent, SelectDropdownOptionEvent
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
from browser_use.tools.service import Tools
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def http_server():
|
||||
"""Create and provide a test HTTP server that serves static content."""
|
||||
server = HTTPServer()
|
||||
server.start()
|
||||
|
||||
# Add route for native dropdown test page
|
||||
server.expect_request('/native-dropdown').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Native Dropdown Test</title>
|
||||
</head>
|
||||
<body>
|
||||
<h1>Native Dropdown Test</h1>
|
||||
<select id="test-dropdown" name="test-dropdown">
|
||||
<option value="">Please select</option>
|
||||
<option value="option1">First Option</option>
|
||||
<option value="option2">Second Option</option>
|
||||
<option value="option3">Third Option</option>
|
||||
</select>
|
||||
<div id="result">No selection made</div>
|
||||
<script>
|
||||
document.getElementById('test-dropdown').addEventListener('change', function(e) {
|
||||
document.getElementById('result').textContent = 'Selected: ' + e.target.options[e.target.selectedIndex].text;
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
# Add route for ARIA menu test page
|
||||
server.expect_request('/aria-menu').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>ARIA Menu Test</title>
|
||||
<style>
|
||||
.menu {
|
||||
list-style: none;
|
||||
padding: 0;
|
||||
margin: 0;
|
||||
border: 1px solid #ccc;
|
||||
background: white;
|
||||
width: 200px;
|
||||
}
|
||||
.menu-item {
|
||||
padding: 10px 20px;
|
||||
border-bottom: 1px solid #eee;
|
||||
}
|
||||
.menu-item:hover {
|
||||
background: #f0f0f0;
|
||||
}
|
||||
.menu-item-anchor {
|
||||
text-decoration: none;
|
||||
color: #333;
|
||||
display: block;
|
||||
}
|
||||
#result {
|
||||
margin-top: 20px;
|
||||
padding: 10px;
|
||||
border: 1px solid #ddd;
|
||||
min-height: 20px;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<h1>ARIA Menu Test</h1>
|
||||
<p>This menu uses ARIA roles instead of native select elements</p>
|
||||
|
||||
<ul class="menu menu-format-standard menu-regular" role="menu" id="pyNavigation1752753375773" style="display: block;">
|
||||
<li class="menu-item menu-item-enabled" role="presentation">
|
||||
<a href="#" onclick="pd(event);" class="menu-item-anchor" tabindex="0" role="menuitem">
|
||||
<span class="menu-item-title-wrap"><span class="menu-item-title">Filter</span></span>
|
||||
</a>
|
||||
</li>
|
||||
<li class="menu-item menu-item-enabled" role="presentation" id="menu-item-$PpyNavigation1752753375773$ppyElements$l2">
|
||||
<a href="#" onclick="pd(event);" class="menu-item-anchor menu-item-expand" tabindex="0" role="menuitem" aria-haspopup="true">
|
||||
<span class="menu-item-title-wrap"><span class="menu-item-title">Sort</span></span>
|
||||
</a>
|
||||
<div class="menu-panel-wrapper">
|
||||
<ul class="menu menu-format-standard menu-regular" role="menu" id="$PpyNavigation1752753375773$ppyElements$l2">
|
||||
<li class="menu-item menu-item-enabled" role="presentation">
|
||||
<a href="#" onclick="pd(event);" class="menu-item-anchor" tabindex="0" role="menuitem">
|
||||
<span class="menu-item-title-wrap"><span class="menu-item-title">Lowest to highest</span></span>
|
||||
</a>
|
||||
</li>
|
||||
<li class="menu-item menu-item-enabled" role="presentation">
|
||||
<a href="#" onclick="pd(event);" class="menu-item-anchor" tabindex="0" role="menuitem">
|
||||
<span class="menu-item-title-wrap"><span class="menu-item-title">Highest to lowest</span></span>
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
</li>
|
||||
<li class="menu-item menu-item-enabled" role="presentation">
|
||||
<a href="#" onclick="pd(event);" class="menu-item-anchor" tabindex="0" role="menuitem">
|
||||
<span class="menu-item-title-wrap"><span class="menu-item-title">Appearance</span></span>
|
||||
</a>
|
||||
</li>
|
||||
<li class="menu-item menu-item-enabled" role="presentation">
|
||||
<a href="#" onclick="pd(event);" class="menu-item-anchor" tabindex="0" role="menuitem">
|
||||
<span class="menu-item-title-wrap"><span class="menu-item-title">Summarize</span></span>
|
||||
</a>
|
||||
</li>
|
||||
<li class="menu-item menu-item-enabled" role="presentation">
|
||||
<a href="#" onclick="pd(event);" class="menu-item-anchor" tabindex="0" role="menuitem">
|
||||
<span class="menu-item-title-wrap"><span class="menu-item-title">Delete</span></span>
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<div id="result">Click an option to see the result</div>
|
||||
|
||||
<script>
|
||||
// Mock the pd function that prevents default
|
||||
function pd(event) {
|
||||
event.preventDefault();
|
||||
const text = event.target.closest('[role="menuitem"]').textContent.trim();
|
||||
document.getElementById('result').textContent = 'Clicked: ' + text;
|
||||
}
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
# Add route for custom dropdown test page
|
||||
server.expect_request('/custom-dropdown').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Custom Dropdown Test</title>
|
||||
<style>
|
||||
.dropdown {
|
||||
position: relative;
|
||||
display: inline-block;
|
||||
width: 200px;
|
||||
}
|
||||
.dropdown-button {
|
||||
padding: 10px;
|
||||
border: 1px solid #ccc;
|
||||
background: white;
|
||||
cursor: pointer;
|
||||
width: 100%;
|
||||
}
|
||||
.dropdown-menu {
|
||||
position: absolute;
|
||||
top: 100%;
|
||||
left: 0;
|
||||
right: 0;
|
||||
border: 1px solid #ccc;
|
||||
background: white;
|
||||
display: block;
|
||||
z-index: 1000;
|
||||
}
|
||||
.dropdown-menu.hidden {
|
||||
display: none;
|
||||
}
|
||||
.dropdown .item {
|
||||
padding: 10px;
|
||||
cursor: pointer;
|
||||
}
|
||||
.dropdown .item:hover {
|
||||
background: #f0f0f0;
|
||||
}
|
||||
.dropdown .item.selected {
|
||||
background: #e0e0e0;
|
||||
}
|
||||
#result {
|
||||
margin-top: 20px;
|
||||
padding: 10px;
|
||||
border: 1px solid #ddd;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<h1>Custom Dropdown Test</h1>
|
||||
<p>This is a custom dropdown implementation (like Semantic UI)</p>
|
||||
|
||||
<div class="dropdown ui" id="custom-dropdown">
|
||||
<div class="dropdown-button" onclick="toggleDropdown()">
|
||||
<span id="selected-text">Choose an option</span>
|
||||
</div>
|
||||
<div class="dropdown-menu" id="dropdown-menu">
|
||||
<div class="item" data-value="red" onclick="selectOption('Red', 'red')">Red</div>
|
||||
<div class="item" data-value="green" onclick="selectOption('Green', 'green')">Green</div>
|
||||
<div class="item" data-value="blue" onclick="selectOption('Blue', 'blue')">Blue</div>
|
||||
<div class="item" data-value="yellow" onclick="selectOption('Yellow', 'yellow')">Yellow</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="result">No selection made</div>
|
||||
|
||||
<script>
|
||||
function toggleDropdown() {
|
||||
const menu = document.getElementById('dropdown-menu');
|
||||
menu.classList.toggle('hidden');
|
||||
}
|
||||
|
||||
function selectOption(text, value) {
|
||||
document.getElementById('selected-text').textContent = text;
|
||||
document.getElementById('result').textContent = 'Selected: ' + text + ' (value: ' + value + ')';
|
||||
// Mark as selected
|
||||
document.querySelectorAll('.item').forEach(item => item.classList.remove('selected'));
|
||||
event.target.classList.add('selected');
|
||||
// Close dropdown
|
||||
document.getElementById('dropdown-menu').classList.add('hidden');
|
||||
}
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
yield server
|
||||
server.stop()
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def base_url(http_server):
|
||||
"""Return the base URL for the test HTTP server."""
|
||||
return f'http://{http_server.host}:{http_server.port}'
|
||||
|
||||
|
||||
@pytest.fixture(scope='module')
|
||||
async def browser_session():
|
||||
"""Create and provide a Browser instance with security disabled."""
|
||||
browser_session = BrowserSession(
|
||||
browser_profile=BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
keep_alive=True,
|
||||
chromium_sandbox=False, # Disable sandbox for CI environment
|
||||
)
|
||||
)
|
||||
await browser_session.start()
|
||||
yield browser_session
|
||||
await browser_session.kill()
|
||||
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
def tools():
|
||||
"""Create and provide a Tools instance."""
|
||||
return Tools()
|
||||
|
||||
|
||||
class TestGetDropdownOptionsEvent:
|
||||
"""Test GetDropdownOptionsEvent functionality for various dropdown types."""
|
||||
|
||||
@pytest.mark.skip(reason='Dropdown text assertion issue - test expects specific text format')
|
||||
async def test_native_select_dropdown(self, tools, browser_session: BrowserSession, base_url):
|
||||
"""Test get_dropdown_options with native HTML select element."""
|
||||
# Navigate to the native dropdown test page
|
||||
await tools.navigate(url=f'{base_url}/native-dropdown', new_tab=False, browser_session=browser_session)
|
||||
|
||||
# Initialize the DOM state to populate the selector map
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
# Find the select element by ID
|
||||
dropdown_index = await browser_session.get_index_by_id('test-dropdown')
|
||||
|
||||
assert dropdown_index is not None, 'Could not find select element'
|
||||
|
||||
# Test via tools action
|
||||
result = await tools.dropdown_options(index=dropdown_index, browser_session=browser_session)
|
||||
|
||||
# Verify the result
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
|
||||
# Verify all expected options are present
|
||||
expected_options = ['Please select', 'First Option', 'Second Option', 'Third Option']
|
||||
for option in expected_options:
|
||||
assert option in result.extracted_content, f"Option '{option}' not found in result content"
|
||||
|
||||
# Verify instruction is included
|
||||
assert 'Use the exact text string' in result.extracted_content and 'select_dropdown' in result.extracted_content
|
||||
|
||||
# Also test direct event dispatch
|
||||
node = await browser_session.get_element_by_index(dropdown_index)
|
||||
assert node is not None
|
||||
event = browser_session.event_bus.dispatch(GetDropdownOptionsEvent(node=node))
|
||||
dropdown_data = await event.event_result(timeout=3.0)
|
||||
|
||||
assert dropdown_data is not None
|
||||
assert 'options' in dropdown_data
|
||||
assert 'type' in dropdown_data
|
||||
assert dropdown_data['type'] == 'select'
|
||||
|
||||
@pytest.mark.skip(reason='ARIA menu detection issue - element not found in selector map')
|
||||
async def test_aria_menu_dropdown(self, tools, browser_session: BrowserSession, base_url):
|
||||
"""Test get_dropdown_options with ARIA role='menu' element."""
|
||||
# Navigate to the ARIA menu test page
|
||||
await tools.navigate(url=f'{base_url}/aria-menu', new_tab=False, browser_session=browser_session)
|
||||
|
||||
# Initialize the DOM state
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
# Find the ARIA menu by ID
|
||||
menu_index = await browser_session.get_index_by_id('pyNavigation1752753375773')
|
||||
|
||||
assert menu_index is not None, 'Could not find ARIA menu element'
|
||||
|
||||
# Test via tools action
|
||||
result = await tools.dropdown_options(index=menu_index, browser_session=browser_session)
|
||||
|
||||
# Verify the result
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
|
||||
# Verify expected ARIA menu options are present
|
||||
expected_options = ['Filter', 'Sort', 'Appearance', 'Summarize', 'Delete']
|
||||
for option in expected_options:
|
||||
assert option in result.extracted_content, f"Option '{option}' not found in result content"
|
||||
|
||||
# Also test direct event dispatch
|
||||
node = await browser_session.get_element_by_index(menu_index)
|
||||
assert node is not None
|
||||
event = browser_session.event_bus.dispatch(GetDropdownOptionsEvent(node=node))
|
||||
dropdown_data = await event.event_result(timeout=3.0)
|
||||
|
||||
assert dropdown_data is not None
|
||||
assert 'options' in dropdown_data
|
||||
assert 'type' in dropdown_data
|
||||
assert dropdown_data['type'] == 'aria'
|
||||
|
||||
@pytest.mark.skip(reason='Custom dropdown detection issue - element not found in selector map')
|
||||
async def test_custom_dropdown(self, tools, browser_session: BrowserSession, base_url):
|
||||
"""Test get_dropdown_options with custom dropdown implementation."""
|
||||
# Navigate to the custom dropdown test page
|
||||
await tools.navigate(url=f'{base_url}/custom-dropdown', new_tab=False, browser_session=browser_session)
|
||||
|
||||
# Initialize the DOM state
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
# Find the custom dropdown by ID
|
||||
dropdown_index = await browser_session.get_index_by_id('custom-dropdown')
|
||||
|
||||
assert dropdown_index is not None, 'Could not find custom dropdown element'
|
||||
|
||||
# Test via tools action
|
||||
result = await tools.dropdown_options(index=dropdown_index, browser_session=browser_session)
|
||||
|
||||
# Verify the result
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
|
||||
# Verify expected custom dropdown options are present
|
||||
expected_options = ['Red', 'Green', 'Blue', 'Yellow']
|
||||
for option in expected_options:
|
||||
assert option in result.extracted_content, f"Option '{option}' not found in result content"
|
||||
|
||||
# Also test direct event dispatch
|
||||
node = await browser_session.get_element_by_index(dropdown_index)
|
||||
assert node is not None
|
||||
event = browser_session.event_bus.dispatch(GetDropdownOptionsEvent(node=node))
|
||||
dropdown_data = await event.event_result(timeout=3.0)
|
||||
|
||||
assert dropdown_data is not None
|
||||
assert 'options' in dropdown_data
|
||||
assert 'type' in dropdown_data
|
||||
assert dropdown_data['type'] == 'custom'
|
||||
|
||||
|
||||
class TestSelectDropdownOptionEvent:
|
||||
"""Test SelectDropdownOptionEvent functionality for various dropdown types."""
|
||||
|
||||
@pytest.mark.skip(reason='Timeout issue - test takes too long to complete')
|
||||
async def test_select_native_dropdown_option(self, tools, browser_session: BrowserSession, base_url):
|
||||
"""Test select_dropdown_option with native HTML select element."""
|
||||
# Navigate to the native dropdown test page
|
||||
await tools.navigate(url=f'{base_url}/native-dropdown', new_tab=False, browser_session=browser_session)
|
||||
await browser_session.event_bus.expect(NavigationCompleteEvent, timeout=10.0)
|
||||
|
||||
# Initialize the DOM state
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
# Find the select element by ID
|
||||
dropdown_index = await browser_session.get_index_by_id('test-dropdown')
|
||||
|
||||
assert dropdown_index is not None
|
||||
|
||||
# Test via tools action
|
||||
result = await tools.select_dropdown(index=dropdown_index, text='Second Option', browser_session=browser_session)
|
||||
|
||||
# Verify the result
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'Second Option' in result.extracted_content
|
||||
|
||||
# Verify the selection actually worked using CDP
|
||||
cdp_session = await browser_session.get_or_create_cdp_session()
|
||||
result = await cdp_session.cdp_client.send.Runtime.evaluate(
|
||||
params={'expression': "document.getElementById('test-dropdown').selectedIndex", 'returnByValue': True},
|
||||
session_id=cdp_session.session_id,
|
||||
)
|
||||
selected_index = result.get('result', {}).get('value', -1)
|
||||
assert selected_index == 2, f'Expected selected index 2, got {selected_index}'
|
||||
|
||||
@pytest.mark.skip(reason='Timeout issue - test takes too long to complete')
|
||||
async def test_select_aria_menu_option(self, tools, browser_session: BrowserSession, base_url):
|
||||
"""Test select_dropdown_option with ARIA menu."""
|
||||
# Navigate to the ARIA menu test page
|
||||
await tools.navigate(url=f'{base_url}/aria-menu', new_tab=False, browser_session=browser_session)
|
||||
await browser_session.event_bus.expect(NavigationCompleteEvent, timeout=10.0)
|
||||
|
||||
# Initialize the DOM state
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
# Find the ARIA menu by ID
|
||||
menu_index = await browser_session.get_index_by_id('pyNavigation1752753375773')
|
||||
|
||||
assert menu_index is not None
|
||||
|
||||
# Test via tools action
|
||||
result = await tools.select_dropdown(index=menu_index, text='Filter', browser_session=browser_session)
|
||||
|
||||
# Verify the result
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'Filter' in result.extracted_content
|
||||
|
||||
# Verify the click had an effect using CDP
|
||||
cdp_session = await browser_session.get_or_create_cdp_session()
|
||||
result = await cdp_session.cdp_client.send.Runtime.evaluate(
|
||||
params={'expression': "document.getElementById('result').textContent", 'returnByValue': True},
|
||||
session_id=cdp_session.session_id,
|
||||
)
|
||||
result_text = result.get('result', {}).get('value', '')
|
||||
assert 'Filter' in result_text, f"Expected 'Filter' in result text, got '{result_text}'"
|
||||
|
||||
@pytest.mark.skip(reason='Timeout issue - test takes too long to complete')
|
||||
async def test_select_custom_dropdown_option(self, tools, browser_session: BrowserSession, base_url):
|
||||
"""Test select_dropdown_option with custom dropdown."""
|
||||
# Navigate to the custom dropdown test page
|
||||
await tools.navigate(url=f'{base_url}/custom-dropdown', new_tab=False, browser_session=browser_session)
|
||||
await browser_session.event_bus.expect(NavigationCompleteEvent, timeout=10.0)
|
||||
|
||||
# Initialize the DOM state
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
# Find the custom dropdown by ID
|
||||
dropdown_index = await browser_session.get_index_by_id('custom-dropdown')
|
||||
|
||||
assert dropdown_index is not None
|
||||
|
||||
# Test via tools action
|
||||
result = await tools.select_dropdown(index=dropdown_index, text='Blue', browser_session=browser_session)
|
||||
|
||||
# Verify the result
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'Blue' in result.extracted_content
|
||||
|
||||
# Verify the selection worked using CDP
|
||||
cdp_session = await browser_session.get_or_create_cdp_session()
|
||||
result = await cdp_session.cdp_client.send.Runtime.evaluate(
|
||||
params={'expression': "document.getElementById('result').textContent", 'returnByValue': True},
|
||||
session_id=cdp_session.session_id,
|
||||
)
|
||||
result_text = result.get('result', {}).get('value', '')
|
||||
assert 'Blue' in result_text, f"Expected 'Blue' in result text, got '{result_text}'"
|
||||
|
||||
@pytest.mark.skip(reason='Timeout issue - test takes too long to complete')
|
||||
async def test_select_invalid_option_error(self, tools, browser_session: BrowserSession, base_url):
|
||||
"""Test select_dropdown_option with non-existent option text."""
|
||||
# Navigate to the native dropdown test page
|
||||
await tools.navigate(url=f'{base_url}/native-dropdown', new_tab=False, browser_session=browser_session)
|
||||
await browser_session.event_bus.expect(NavigationCompleteEvent, timeout=10.0)
|
||||
|
||||
# Initialize the DOM state
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
# Find the select element by ID
|
||||
dropdown_index = await browser_session.get_index_by_id('test-dropdown')
|
||||
|
||||
assert dropdown_index is not None
|
||||
|
||||
# Try to select non-existent option via direct event
|
||||
node = await browser_session.get_element_by_index(dropdown_index)
|
||||
assert node is not None
|
||||
event = browser_session.event_bus.dispatch(SelectDropdownOptionEvent(node=node, text='Non-existent Option'))
|
||||
|
||||
try:
|
||||
selection_data = await event.event_result(timeout=3.0)
|
||||
# Should have an error in the result
|
||||
assert selection_data is not None
|
||||
assert 'error' in selection_data or 'not found' in str(selection_data).lower()
|
||||
except Exception as e:
|
||||
# Or raise an exception
|
||||
assert 'not found' in str(e).lower() or 'no option' in str(e).lower()
|
||||
@@ -0,0 +1,274 @@
|
||||
"""Test radio button click interactions with various label association patterns.
|
||||
|
||||
Verifies that the occlusion check correctly identifies label-input associations
|
||||
so that CDP click dispatch works for radio buttons whose labels visually overlap them.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from pytest_httpserver import HTTPServer
|
||||
|
||||
from browser_use.browser import BrowserSession
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
from browser_use.tools.service import Tools
|
||||
|
||||
# -- HTML fixtures --
|
||||
|
||||
RADIO_SIBLING_HTML = """
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Radio Sibling Label Test</title>
|
||||
<style>
|
||||
.radio-group { display: flex; flex-direction: column; gap: 8px; }
|
||||
.radio-group input[type="radio"] { width: 16px; height: 16px; }
|
||||
.radio-group label { cursor: pointer; padding: 4px 8px; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="radio-group">
|
||||
<div>
|
||||
<input type="radio" name="color" value="red" id="radio-red">
|
||||
<label for="radio-red">Red</label>
|
||||
</div>
|
||||
<div>
|
||||
<input type="radio" name="color" value="blue" id="radio-blue">
|
||||
<label for="radio-blue">Blue</label>
|
||||
</div>
|
||||
<div>
|
||||
<input type="radio" name="color" value="green" id="radio-green">
|
||||
<label for="radio-green">Green</label>
|
||||
</div>
|
||||
</div>
|
||||
<div id="result"></div>
|
||||
<script>
|
||||
document.querySelectorAll('input[name="color"]').forEach(r => {
|
||||
r.addEventListener('change', () => {
|
||||
document.getElementById('result').textContent = 'selected:' + r.value;
|
||||
});
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
RADIO_WRAPPED_HTML = """
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Radio Wrapped Label Test</title>
|
||||
<style>
|
||||
.radio-group label { display: block; cursor: pointer; padding: 8px; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="radio-group">
|
||||
<label><input type="radio" name="fruit" value="apple" id="radio-apple"> Apple</label>
|
||||
<label><input type="radio" name="fruit" value="banana" id="radio-banana"> Banana</label>
|
||||
<label><input type="radio" name="fruit" value="cherry" id="radio-cherry"> Cherry</label>
|
||||
</div>
|
||||
<div id="result"></div>
|
||||
<script>
|
||||
document.querySelectorAll('input[name="fruit"]').forEach(r => {
|
||||
r.addEventListener('change', () => {
|
||||
document.getElementById('result').textContent = 'selected:' + r.value;
|
||||
});
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
RADIO_CUSTOM_HTML = """
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Radio Custom Styled Test</title>
|
||||
<style>
|
||||
.radio-group { display: flex; flex-direction: column; gap: 8px; }
|
||||
.radio-group input[type="radio"] {
|
||||
appearance: none;
|
||||
-webkit-appearance: none;
|
||||
width: 20px;
|
||||
height: 20px;
|
||||
border: 2px solid #999;
|
||||
border-radius: 50%;
|
||||
position: relative;
|
||||
}
|
||||
.radio-group input[type="radio"]:checked {
|
||||
border-color: #007bff;
|
||||
background: #007bff;
|
||||
}
|
||||
.radio-group label {
|
||||
cursor: pointer;
|
||||
padding: 4px 8px;
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="radio-group">
|
||||
<div>
|
||||
<input type="radio" name="size" value="small" id="radio-small">
|
||||
<label for="radio-small">Small</label>
|
||||
</div>
|
||||
<div>
|
||||
<input type="radio" name="size" value="medium" id="radio-medium">
|
||||
<label for="radio-medium">Medium</label>
|
||||
</div>
|
||||
<div>
|
||||
<input type="radio" name="size" value="large" id="radio-large">
|
||||
<label for="radio-large">Large</label>
|
||||
</div>
|
||||
</div>
|
||||
<div id="result"></div>
|
||||
<script>
|
||||
document.querySelectorAll('input[name="size"]').forEach(r => {
|
||||
r.addEventListener('change', () => {
|
||||
document.getElementById('result').textContent = 'selected:' + r.value;
|
||||
});
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def http_server():
|
||||
server = HTTPServer()
|
||||
server.start()
|
||||
|
||||
server.expect_request('/radio-sibling').respond_with_data(RADIO_SIBLING_HTML, content_type='text/html')
|
||||
server.expect_request('/radio-wrapped').respond_with_data(RADIO_WRAPPED_HTML, content_type='text/html')
|
||||
server.expect_request('/radio-custom').respond_with_data(RADIO_CUSTOM_HTML, content_type='text/html')
|
||||
|
||||
yield server
|
||||
server.stop()
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def base_url(http_server):
|
||||
return f'http://{http_server.host}:{http_server.port}'
|
||||
|
||||
|
||||
@pytest.fixture(scope='module')
|
||||
async def browser_session():
|
||||
browser_session = BrowserSession(
|
||||
browser_profile=BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
keep_alive=True,
|
||||
chromium_sandbox=False,
|
||||
)
|
||||
)
|
||||
await browser_session.start()
|
||||
yield browser_session
|
||||
await browser_session.kill()
|
||||
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
def tools():
|
||||
return Tools()
|
||||
|
||||
|
||||
async def _get_checked_and_result(browser_session: BrowserSession, input_id: str) -> tuple[bool, str]:
|
||||
"""Helper: returns (is_checked, result_div_text) via CDP."""
|
||||
cdp_session = await browser_session.get_or_create_cdp_session()
|
||||
sid = cdp_session.session_id
|
||||
|
||||
checked_result = await cdp_session.cdp_client.send.Runtime.evaluate(
|
||||
params={
|
||||
'expression': f"document.getElementById('{input_id}').checked",
|
||||
'returnByValue': True,
|
||||
},
|
||||
session_id=sid,
|
||||
)
|
||||
is_checked = checked_result.get('result', {}).get('value', False)
|
||||
|
||||
text_result = await cdp_session.cdp_client.send.Runtime.evaluate(
|
||||
params={
|
||||
'expression': "document.getElementById('result').textContent",
|
||||
'returnByValue': True,
|
||||
},
|
||||
session_id=sid,
|
||||
)
|
||||
result_text = text_result.get('result', {}).get('value', '')
|
||||
|
||||
return is_checked, result_text
|
||||
|
||||
|
||||
class TestRadioButtons:
|
||||
"""Test radio button clicks across label association patterns."""
|
||||
|
||||
async def test_sibling_label_radio_click(self, tools: Tools, browser_session: BrowserSession, base_url: str):
|
||||
"""Click a radio whose sibling <label for=...> may occlude it."""
|
||||
await tools.navigate(url=f'{base_url}/radio-sibling', new_tab=False, browser_session=browser_session)
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
idx = await browser_session.get_index_by_id('radio-blue')
|
||||
assert idx is not None, 'Could not find radio-blue in selector map'
|
||||
|
||||
result = await tools.click(index=idx, browser_session=browser_session)
|
||||
assert result.error is None, f'Click failed: {result.error}'
|
||||
|
||||
is_checked, result_text = await _get_checked_and_result(browser_session, 'radio-blue')
|
||||
assert is_checked, 'radio-blue should be checked after click'
|
||||
assert 'selected:blue' in result_text, f'Change event not fired, result: {result_text}'
|
||||
|
||||
async def test_wrapped_label_radio_click(self, tools: Tools, browser_session: BrowserSession, base_url: str):
|
||||
"""Click a radio wrapped inside its <label>."""
|
||||
await tools.navigate(url=f'{base_url}/radio-wrapped', new_tab=False, browser_session=browser_session)
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
idx = await browser_session.get_index_by_id('radio-banana')
|
||||
assert idx is not None, 'Could not find radio-banana in selector map'
|
||||
|
||||
result = await tools.click(index=idx, browser_session=browser_session)
|
||||
assert result.error is None, f'Click failed: {result.error}'
|
||||
|
||||
is_checked, result_text = await _get_checked_and_result(browser_session, 'radio-banana')
|
||||
assert is_checked, 'radio-banana should be checked after click'
|
||||
assert 'selected:banana' in result_text, f'Change event not fired, result: {result_text}'
|
||||
|
||||
async def test_custom_styled_radio_click(self, tools: Tools, browser_session: BrowserSession, base_url: str):
|
||||
"""Click a CSS-customized radio (appearance:none) with sibling label."""
|
||||
await tools.navigate(url=f'{base_url}/radio-custom', new_tab=False, browser_session=browser_session)
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
idx = await browser_session.get_index_by_id('radio-large')
|
||||
assert idx is not None, 'Could not find radio-large in selector map'
|
||||
|
||||
result = await tools.click(index=idx, browser_session=browser_session)
|
||||
assert result.error is None, f'Click failed: {result.error}'
|
||||
|
||||
is_checked, result_text = await _get_checked_and_result(browser_session, 'radio-large')
|
||||
assert is_checked, 'radio-large should be checked after click'
|
||||
assert 'selected:large' in result_text, f'Change event not fired, result: {result_text}'
|
||||
|
||||
async def test_radio_group_switching(self, tools: Tools, browser_session: BrowserSession, base_url: str):
|
||||
"""Click one radio then another in the same group; first should uncheck."""
|
||||
await tools.navigate(url=f'{base_url}/radio-sibling', new_tab=False, browser_session=browser_session)
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
# Click red first
|
||||
red_idx = await browser_session.get_index_by_id('radio-red')
|
||||
assert red_idx is not None
|
||||
result = await tools.click(index=red_idx, browser_session=browser_session)
|
||||
assert result.error is None, f'Click red failed: {result.error}'
|
||||
|
||||
is_red_checked, _ = await _get_checked_and_result(browser_session, 'radio-red')
|
||||
assert is_red_checked, 'radio-red should be checked'
|
||||
|
||||
# Re-fetch state so indices are current, then click green
|
||||
await browser_session.get_browser_state_summary()
|
||||
green_idx = await browser_session.get_index_by_id('radio-green')
|
||||
assert green_idx is not None
|
||||
result = await tools.click(index=green_idx, browser_session=browser_session)
|
||||
assert result.error is None, f'Click green failed: {result.error}'
|
||||
|
||||
is_green_checked, result_text = await _get_checked_and_result(browser_session, 'radio-green')
|
||||
assert is_green_checked, 'radio-green should be checked'
|
||||
assert 'selected:green' in result_text
|
||||
|
||||
# Verify red is now unchecked
|
||||
is_red_checked, _ = await _get_checked_and_result(browser_session, 'radio-red')
|
||||
assert not is_red_checked, 'radio-red should be unchecked after selecting green'
|
||||
@@ -0,0 +1,109 @@
|
||||
"""Shared test helper for LLM model tests."""
|
||||
|
||||
import os
|
||||
|
||||
import pytest
|
||||
|
||||
from browser_use.agent.service import Agent
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
from browser_use.browser.session import BrowserSession
|
||||
|
||||
|
||||
async def run_model_button_click_test(
|
||||
model_class,
|
||||
model_name: str,
|
||||
api_key_env: str | None,
|
||||
extra_kwargs: dict,
|
||||
httpserver,
|
||||
):
|
||||
"""Test that an LLM model can click a button.
|
||||
|
||||
This test verifies:
|
||||
1. Model can be initialized with API key
|
||||
2. Agent can navigate and click a button
|
||||
3. Button click is verified by checking page state change
|
||||
4. Completes within max 2 steps
|
||||
"""
|
||||
# Handle API key validation - skip test if not available
|
||||
if api_key_env is not None:
|
||||
api_key = os.getenv(api_key_env)
|
||||
if not api_key:
|
||||
pytest.skip(f'{api_key_env} not set - skipping test')
|
||||
else:
|
||||
api_key = None
|
||||
|
||||
# Handle Azure-specific endpoint validation
|
||||
from browser_use.llm.azure.chat import ChatAzureOpenAI
|
||||
|
||||
if model_class is ChatAzureOpenAI:
|
||||
azure_endpoint = os.getenv('AZURE_OPENAI_ENDPOINT')
|
||||
if not azure_endpoint:
|
||||
pytest.skip('AZURE_OPENAI_ENDPOINT not set - skipping test')
|
||||
# Add the azure_endpoint to extra_kwargs
|
||||
extra_kwargs = {**extra_kwargs, 'azure_endpoint': azure_endpoint}
|
||||
|
||||
# Create HTML page with a button that changes page content when clicked
|
||||
html = """
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Button Test</title></head>
|
||||
<body>
|
||||
<h1>Button Click Test</h1>
|
||||
<button id="test-button" onclick="document.getElementById('result').innerText='SUCCESS'">
|
||||
Click Me
|
||||
</button>
|
||||
<div id="result">NOT_CLICKED</div>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
httpserver.expect_request('/').respond_with_data(html, content_type='text/html')
|
||||
|
||||
# Create LLM instance with extra kwargs if provided
|
||||
llm_kwargs = {'model': model_name}
|
||||
if api_key is not None:
|
||||
llm_kwargs['api_key'] = api_key
|
||||
llm_kwargs.update(extra_kwargs)
|
||||
llm = model_class(**llm_kwargs) # type: ignore[arg-type]
|
||||
|
||||
# Create browser session
|
||||
browser = BrowserSession(
|
||||
browser_profile=BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=None, # Use temporary directory
|
||||
)
|
||||
)
|
||||
|
||||
try:
|
||||
# Start browser
|
||||
await browser.start()
|
||||
|
||||
# Create agent with button click task (URL in task triggers auto-navigation)
|
||||
test_url = httpserver.url_for('/')
|
||||
agent = Agent(
|
||||
task=f'{test_url} - Click the button',
|
||||
llm=llm,
|
||||
browser_session=browser,
|
||||
max_steps=2, # Max 2 steps as per requirements
|
||||
)
|
||||
|
||||
# Run the agent
|
||||
result = await agent.run()
|
||||
|
||||
# Verify task completed
|
||||
assert result is not None
|
||||
assert len(result.history) > 0
|
||||
|
||||
# Verify button was clicked by checking page state across any step
|
||||
button_clicked = False
|
||||
for step in result.history:
|
||||
# Check state_message which contains browser state with page text
|
||||
if step.state_message and 'SUCCESS' in step.state_message:
|
||||
button_clicked = True
|
||||
break
|
||||
|
||||
# Check if SUCCESS appears in any step (indicating button was clicked)
|
||||
assert button_clicked, 'Button was not clicked - SUCCESS not found in any page state'
|
||||
|
||||
finally:
|
||||
# Clean up browser session
|
||||
await browser.kill()
|
||||
@@ -0,0 +1,266 @@
|
||||
"""Tests for Azure OpenAI Responses API support."""
|
||||
|
||||
import os
|
||||
|
||||
import pytest
|
||||
|
||||
from browser_use.llm.azure.chat import RESPONSES_API_ONLY_MODELS, ChatAzureOpenAI
|
||||
from browser_use.llm.messages import (
|
||||
AssistantMessage,
|
||||
ContentPartImageParam,
|
||||
ContentPartTextParam,
|
||||
Function,
|
||||
ImageURL,
|
||||
SystemMessage,
|
||||
ToolCall,
|
||||
UserMessage,
|
||||
)
|
||||
from browser_use.llm.openai.responses_serializer import ResponsesAPIMessageSerializer
|
||||
|
||||
|
||||
class TestResponsesAPIMessageSerializer:
|
||||
"""Tests for the ResponsesAPIMessageSerializer class."""
|
||||
|
||||
def test_serialize_user_message_string_content(self):
|
||||
"""Test serializing a user message with string content."""
|
||||
message = UserMessage(content='Hello, world!')
|
||||
result = ResponsesAPIMessageSerializer.serialize(message)
|
||||
|
||||
assert result['role'] == 'user'
|
||||
assert result['content'] == 'Hello, world!'
|
||||
|
||||
def test_serialize_user_message_text_parts(self):
|
||||
"""Test serializing a user message with text content parts."""
|
||||
message = UserMessage(
|
||||
content=[
|
||||
ContentPartTextParam(type='text', text='First part'),
|
||||
ContentPartTextParam(type='text', text='Second part'),
|
||||
]
|
||||
)
|
||||
result = ResponsesAPIMessageSerializer.serialize(message)
|
||||
|
||||
assert result['role'] == 'user'
|
||||
assert isinstance(result['content'], list)
|
||||
assert len(result['content']) == 2
|
||||
assert result['content'][0]['type'] == 'input_text'
|
||||
assert result['content'][0]['text'] == 'First part'
|
||||
assert result['content'][1]['type'] == 'input_text'
|
||||
assert result['content'][1]['text'] == 'Second part'
|
||||
|
||||
def test_serialize_user_message_with_image(self):
|
||||
"""Test serializing a user message with image content."""
|
||||
message = UserMessage(
|
||||
content=[
|
||||
ContentPartTextParam(type='text', text='What is in this image?'),
|
||||
ContentPartImageParam(
|
||||
type='image_url',
|
||||
image_url=ImageURL(url='https://example.com/image.png', detail='auto'),
|
||||
),
|
||||
]
|
||||
)
|
||||
result = ResponsesAPIMessageSerializer.serialize(message)
|
||||
|
||||
assert result['role'] == 'user'
|
||||
assert isinstance(result['content'], list)
|
||||
assert len(result['content']) == 2
|
||||
assert result['content'][0]['type'] == 'input_text'
|
||||
assert result['content'][1]['type'] == 'input_image'
|
||||
assert result['content'][1].get('image_url') == 'https://example.com/image.png'
|
||||
assert result['content'][1].get('detail') == 'auto'
|
||||
|
||||
def test_serialize_system_message_string_content(self):
|
||||
"""Test serializing a system message with string content."""
|
||||
message = SystemMessage(content='You are a helpful assistant.')
|
||||
result = ResponsesAPIMessageSerializer.serialize(message)
|
||||
|
||||
assert result['role'] == 'system'
|
||||
assert result['content'] == 'You are a helpful assistant.'
|
||||
|
||||
def test_serialize_system_message_text_parts(self):
|
||||
"""Test serializing a system message with text content parts."""
|
||||
message = SystemMessage(content=[ContentPartTextParam(type='text', text='System instruction')])
|
||||
result = ResponsesAPIMessageSerializer.serialize(message)
|
||||
|
||||
assert result['role'] == 'system'
|
||||
assert isinstance(result['content'], list)
|
||||
assert len(result['content']) == 1
|
||||
assert result['content'][0]['type'] == 'input_text'
|
||||
|
||||
def test_serialize_assistant_message_string_content(self):
|
||||
"""Test serializing an assistant message with string content."""
|
||||
message = AssistantMessage(content='Here is my response.')
|
||||
result = ResponsesAPIMessageSerializer.serialize(message)
|
||||
|
||||
assert result['role'] == 'assistant'
|
||||
assert result['content'] == 'Here is my response.'
|
||||
|
||||
def test_serialize_assistant_message_none_content_with_tool_calls(self):
|
||||
"""Test serializing an assistant message with None content and tool calls."""
|
||||
message = AssistantMessage(
|
||||
content=None,
|
||||
tool_calls=[
|
||||
ToolCall(
|
||||
id='call_123',
|
||||
type='function',
|
||||
function=Function(name='search', arguments='{"query": "test"}'),
|
||||
)
|
||||
],
|
||||
)
|
||||
result = ResponsesAPIMessageSerializer.serialize(message)
|
||||
|
||||
assert result['role'] == 'assistant'
|
||||
assert '[Tool call: search({"query": "test"})]' in result['content']
|
||||
|
||||
def test_serialize_assistant_message_none_content_no_tool_calls(self):
|
||||
"""Test serializing an assistant message with None content and no tool calls."""
|
||||
message = AssistantMessage(content=None)
|
||||
result = ResponsesAPIMessageSerializer.serialize(message)
|
||||
|
||||
assert result['role'] == 'assistant'
|
||||
assert result['content'] == ''
|
||||
|
||||
def test_serialize_messages_list(self):
|
||||
"""Test serializing a list of messages."""
|
||||
messages = [
|
||||
SystemMessage(content='You are helpful.'),
|
||||
UserMessage(content='Hello!'),
|
||||
AssistantMessage(content='Hi there!'),
|
||||
]
|
||||
results = ResponsesAPIMessageSerializer.serialize_messages(messages)
|
||||
|
||||
assert len(results) == 3
|
||||
assert results[0]['role'] == 'system'
|
||||
assert results[1]['role'] == 'user'
|
||||
assert results[2]['role'] == 'assistant'
|
||||
|
||||
|
||||
class TestChatAzureOpenAIShouldUseResponsesAPI:
|
||||
"""Tests for the _should_use_responses_api method."""
|
||||
|
||||
def test_use_responses_api_true(self):
|
||||
"""Test that use_responses_api=True forces Responses API."""
|
||||
llm = ChatAzureOpenAI(
|
||||
model='gpt-4o',
|
||||
api_key='test',
|
||||
azure_endpoint='https://test.openai.azure.com',
|
||||
use_responses_api=True,
|
||||
)
|
||||
assert llm._should_use_responses_api() is True
|
||||
|
||||
def test_use_responses_api_false(self):
|
||||
"""Test that use_responses_api=False forces Chat Completions API."""
|
||||
llm = ChatAzureOpenAI(
|
||||
model='gpt-5.1-codex-mini', # Even with a Responses-only model
|
||||
api_key='test',
|
||||
azure_endpoint='https://test.openai.azure.com',
|
||||
use_responses_api=False,
|
||||
)
|
||||
assert llm._should_use_responses_api() is False
|
||||
|
||||
def test_use_responses_api_auto_with_responses_only_model(self):
|
||||
"""Test that auto mode detects Responses-only models."""
|
||||
for model_name in RESPONSES_API_ONLY_MODELS:
|
||||
llm = ChatAzureOpenAI(
|
||||
model=model_name,
|
||||
api_key='test',
|
||||
azure_endpoint='https://test.openai.azure.com',
|
||||
use_responses_api='auto',
|
||||
)
|
||||
assert llm._should_use_responses_api() is True, f'Expected Responses API for {model_name}'
|
||||
|
||||
def test_use_responses_api_auto_with_regular_model(self):
|
||||
"""Test that auto mode uses Chat Completions for regular models."""
|
||||
regular_models = ['gpt-4o', 'gpt-4.1-mini', 'gpt-3.5-turbo', 'gpt-4']
|
||||
for model_name in regular_models:
|
||||
llm = ChatAzureOpenAI(
|
||||
model=model_name,
|
||||
api_key='test',
|
||||
azure_endpoint='https://test.openai.azure.com',
|
||||
use_responses_api='auto',
|
||||
)
|
||||
assert llm._should_use_responses_api() is False, f'Expected Chat Completions for {model_name}'
|
||||
|
||||
def test_use_responses_api_auto_is_default(self):
|
||||
"""Test that 'auto' is the default value for use_responses_api."""
|
||||
llm = ChatAzureOpenAI(
|
||||
model='gpt-4o',
|
||||
api_key='test',
|
||||
azure_endpoint='https://test.openai.azure.com',
|
||||
)
|
||||
assert llm.use_responses_api == 'auto'
|
||||
|
||||
def test_responses_api_only_models_list(self):
|
||||
"""Test that the RESPONSES_API_ONLY_MODELS list contains expected models."""
|
||||
expected_models = [
|
||||
'gpt-5.1-codex',
|
||||
'gpt-5.1-codex-mini',
|
||||
'gpt-5.1-codex-max',
|
||||
'gpt-5-codex',
|
||||
'codex-mini-latest',
|
||||
'computer-use-preview',
|
||||
]
|
||||
for model in expected_models:
|
||||
assert model in RESPONSES_API_ONLY_MODELS, f'{model} should be in RESPONSES_API_ONLY_MODELS'
|
||||
|
||||
|
||||
class TestChatAzureOpenAIIntegration:
|
||||
"""Integration tests for Azure OpenAI with Responses API.
|
||||
|
||||
These tests require valid Azure OpenAI credentials and are skipped if not available.
|
||||
"""
|
||||
|
||||
@pytest.fixture
|
||||
def azure_credentials(self):
|
||||
"""Get Azure OpenAI credentials from environment."""
|
||||
api_key = os.getenv('AZURE_OPENAI_KEY') or os.getenv('AZURE_OPENAI_API_KEY')
|
||||
endpoint = os.getenv('AZURE_OPENAI_ENDPOINT')
|
||||
if not api_key or not endpoint:
|
||||
pytest.skip('Azure OpenAI credentials not available')
|
||||
return {'api_key': api_key, 'azure_endpoint': endpoint}
|
||||
|
||||
async def test_chat_completions_api_basic_call(self, azure_credentials):
|
||||
"""Test basic call using Chat Completions API."""
|
||||
llm = ChatAzureOpenAI(
|
||||
model='gpt-4.1-mini',
|
||||
api_key=azure_credentials['api_key'],
|
||||
azure_endpoint=azure_credentials['azure_endpoint'],
|
||||
use_responses_api=False, # Force Chat Completions API
|
||||
)
|
||||
|
||||
messages = [
|
||||
SystemMessage(content='You are a helpful assistant.'),
|
||||
UserMessage(content='Say "hello" and nothing else.'),
|
||||
]
|
||||
|
||||
result = await llm.ainvoke(messages)
|
||||
assert result.completion is not None
|
||||
assert 'hello' in result.completion.lower()
|
||||
|
||||
async def test_responses_api_basic_call(self, azure_credentials):
|
||||
"""Test basic call using Responses API.
|
||||
|
||||
This test only runs if the Azure deployment supports the Responses API
|
||||
(api_version >= 2025-03-01-preview).
|
||||
"""
|
||||
llm = ChatAzureOpenAI(
|
||||
model='gpt-4.1-mini',
|
||||
api_key=azure_credentials['api_key'],
|
||||
azure_endpoint=azure_credentials['azure_endpoint'],
|
||||
api_version='2025-03-01-preview', # Required for Responses API
|
||||
use_responses_api=True, # Force Responses API
|
||||
)
|
||||
|
||||
messages = [
|
||||
SystemMessage(content='You are a helpful assistant.'),
|
||||
UserMessage(content='Say "hello" and nothing else.'),
|
||||
]
|
||||
|
||||
try:
|
||||
result = await llm.ainvoke(messages)
|
||||
assert result.completion is not None
|
||||
assert 'hello' in result.completion.lower()
|
||||
except Exception as e:
|
||||
# Skip if Responses API is not supported
|
||||
if 'Responses API' in str(e) or '404' in str(e):
|
||||
pytest.skip('Responses API not supported by this Azure deployment')
|
||||
raise
|
||||
@@ -0,0 +1,15 @@
|
||||
"""Test Anthropic model button click."""
|
||||
|
||||
from browser_use.llm.anthropic.chat import ChatAnthropic
|
||||
from tests.ci.models.model_test_helper import run_model_button_click_test
|
||||
|
||||
|
||||
async def test_anthropic_claude_sonnet_4_6(httpserver):
|
||||
"""Test Anthropic claude-sonnet-4-6 can click a button."""
|
||||
await run_model_button_click_test(
|
||||
model_class=ChatAnthropic,
|
||||
model_name='claude-sonnet-4-6',
|
||||
api_key_env='ANTHROPIC_API_KEY',
|
||||
extra_kwargs={},
|
||||
httpserver=httpserver,
|
||||
)
|
||||
@@ -0,0 +1,15 @@
|
||||
"""Test Azure OpenAI model button click."""
|
||||
|
||||
from browser_use.llm.azure.chat import ChatAzureOpenAI
|
||||
from tests.ci.models.model_test_helper import run_model_button_click_test
|
||||
|
||||
|
||||
async def test_azure_gpt_4_1_mini(httpserver):
|
||||
"""Test Azure OpenAI gpt-4.1-mini can click a button."""
|
||||
await run_model_button_click_test(
|
||||
model_class=ChatAzureOpenAI,
|
||||
model_name='gpt-4.1-mini',
|
||||
api_key_env='AZURE_OPENAI_KEY',
|
||||
extra_kwargs={}, # Azure endpoint will be added by helper
|
||||
httpserver=httpserver,
|
||||
)
|
||||
@@ -0,0 +1,118 @@
|
||||
"""Tests for the ChatBrowserUse cloud client."""
|
||||
|
||||
import pytest
|
||||
|
||||
from browser_use.llm.browser_use.chat import ChatBrowserUse
|
||||
from browser_use.llm.messages import UserMessage
|
||||
from tests.ci.models.model_test_helper import run_model_button_click_test
|
||||
|
||||
# A syntactically-valid key so the constructor doesn't bail before we reach the
|
||||
# code under test. These unit tests never hit the network.
|
||||
TEST_API_KEY = 'test-key-not-real'
|
||||
|
||||
|
||||
async def test_browseruse_bu_latest(httpserver):
|
||||
"""Test Browser Use bu-latest can click a button."""
|
||||
await run_model_button_click_test(
|
||||
model_class=ChatBrowserUse,
|
||||
model_name='bu-latest',
|
||||
api_key_env='BROWSER_USE_API_KEY',
|
||||
extra_kwargs={},
|
||||
httpserver=httpserver,
|
||||
)
|
||||
|
||||
|
||||
# --- Model validation -------------------------------------------------------
|
||||
|
||||
|
||||
def test_default_model_is_bu_2_0():
|
||||
chat = ChatBrowserUse(api_key=TEST_API_KEY)
|
||||
assert chat.model == 'bu-2-0'
|
||||
assert chat.provider == 'browser-use'
|
||||
|
||||
|
||||
@pytest.mark.parametrize('alias', ['bu-1-0', 'bu-2-0'])
|
||||
def test_bu_aliases_are_accepted(alias):
|
||||
chat = ChatBrowserUse(model=alias, api_key=TEST_API_KEY)
|
||||
assert chat.model == alias
|
||||
assert chat.name == alias
|
||||
assert chat.provider == 'browser-use'
|
||||
|
||||
|
||||
def test_bu_latest_normalizes_to_bu_2_0():
|
||||
chat = ChatBrowserUse(model='bu-latest', api_key=TEST_API_KEY)
|
||||
assert chat.model == 'bu-2-0'
|
||||
assert chat.name == 'bu-2-0'
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
'model',
|
||||
[
|
||||
'anthropic/claude-sonnet-4-6',
|
||||
'openai/gpt-5.5',
|
||||
'google/gemini-3-pro',
|
||||
'browser-use/bu-30b-a3b-preview',
|
||||
],
|
||||
)
|
||||
def test_provider_prefixed_models_are_accepted(model):
|
||||
"""Provider-prefixed ids are accepted and forwarded verbatim (the gateway resolves them)."""
|
||||
chat = ChatBrowserUse(model=model, api_key=TEST_API_KEY)
|
||||
assert chat.model == model
|
||||
assert chat.name == model
|
||||
# Always routes through the browser-use gateway, whatever the target model.
|
||||
assert chat.provider == 'browser-use'
|
||||
|
||||
|
||||
@pytest.mark.parametrize('model', ['gpt-5', 'claude-sonnet-4-6', 'bu-9-9', 'random-model'])
|
||||
def test_bare_model_ids_are_rejected(model):
|
||||
"""Bare ids (no bu-* alias, no provider/ prefix) are rejected."""
|
||||
with pytest.raises(ValueError, match='Invalid model'):
|
||||
ChatBrowserUse(model=model, api_key=TEST_API_KEY)
|
||||
|
||||
|
||||
async def test_provider_prefixed_model_forwarded_in_payload(httpserver):
|
||||
"""The provider-prefixed id must be sent verbatim in the request body."""
|
||||
httpserver.expect_request('/v1/chat/completions', method='POST').respond_with_json({'completion': 'hello from gateway'})
|
||||
|
||||
chat = ChatBrowserUse(
|
||||
model='anthropic/claude-sonnet-4-6',
|
||||
api_key=TEST_API_KEY,
|
||||
base_url=httpserver.url_for('/').rstrip('/'),
|
||||
)
|
||||
result = await chat.ainvoke([UserMessage(content='hi')])
|
||||
|
||||
assert result.completion == 'hello from gateway'
|
||||
|
||||
# Check the posted body.
|
||||
request, _ = httpserver.log[-1]
|
||||
body = request.get_json()
|
||||
assert body['model'] == 'anthropic/claude-sonnet-4-6'
|
||||
assert body['request_type'] == 'browser_agent'
|
||||
|
||||
|
||||
# --- Agent screenshot auto-config -------------------------------------------
|
||||
|
||||
|
||||
# Both the classic and beta agents auto-config the Claude screenshot size, so both
|
||||
# must strip the provider prefix for gateway ids.
|
||||
@pytest.mark.parametrize('agent_path', ['classic', 'beta'])
|
||||
@pytest.mark.parametrize(
|
||||
'model,expected_size',
|
||||
[
|
||||
# Claude Sonnet via the gateway keeps the auto-config; the prefix must not break detection.
|
||||
('anthropic/claude-sonnet-4-6', (1400, 850)),
|
||||
# Non-Claude models keep the default.
|
||||
('bu-2-0', None),
|
||||
('openai/gpt-5.5', None),
|
||||
],
|
||||
)
|
||||
def test_claude_sonnet_screenshot_autoconfig_through_gateway(agent_path, model, expected_size):
|
||||
if agent_path == 'classic':
|
||||
from browser_use.agent.service import Agent
|
||||
else:
|
||||
from browser_use.beta import Agent
|
||||
|
||||
llm = ChatBrowserUse(model=model, api_key=TEST_API_KEY)
|
||||
agent = Agent(task='test', llm=llm)
|
||||
assert agent.browser_session is not None
|
||||
assert agent.browser_session.llm_screenshot_size == expected_size
|
||||
@@ -0,0 +1,71 @@
|
||||
"""Test Google model button click."""
|
||||
|
||||
from browser_use.llm.google.chat import ChatGoogle
|
||||
from tests.ci.models.model_test_helper import run_model_button_click_test
|
||||
|
||||
|
||||
async def test_google_gemini_3_flash_preview(httpserver):
|
||||
"""Test Google gemini-3-flash-preview can click a button."""
|
||||
await run_model_button_click_test(
|
||||
model_class=ChatGoogle,
|
||||
model_name='gemini-3-flash-preview',
|
||||
api_key_env='GOOGLE_API_KEY',
|
||||
extra_kwargs={},
|
||||
httpserver=httpserver,
|
||||
)
|
||||
|
||||
|
||||
def test_x_goog_api_client_header_is_set():
|
||||
"""Test that the x-goog-api-client header is correctly set in the HTTP options."""
|
||||
chat = ChatGoogle(model='gemini-flash-latest', api_key='fake')
|
||||
|
||||
# Generate the params used for genai.Client
|
||||
params = chat._get_client_params()
|
||||
|
||||
# Extract the header
|
||||
http_options = params.get('http_options', {})
|
||||
headers = http_options.get('headers', {})
|
||||
|
||||
assert 'x-goog-api-client' in headers, 'x-goog-api-client header missing'
|
||||
assert 'browser-use/' in headers['x-goog-api-client'], 'browser-use not found in x-goog-api-client header'
|
||||
|
||||
|
||||
def test_x_goog_api_client_header_with_none_http_options():
|
||||
"""Test setting header when http_options is None."""
|
||||
chat = ChatGoogle(model='gemini-flash-latest', api_key='fake', http_options=None)
|
||||
params = chat._get_client_params()
|
||||
http_opts = params.get('http_options', {})
|
||||
assert http_opts.get('headers', {}).get('x-goog-api-client', '').startswith('browser-use/')
|
||||
|
||||
|
||||
def test_x_goog_api_client_header_with_pydantic_http_options():
|
||||
"""Test setting header when http_options is a types.HttpOptions Pydantic model."""
|
||||
from google.genai import types
|
||||
|
||||
pydantic_opts = types.HttpOptions(timeout=30, headers={'custom-header': 'value'})
|
||||
chat = ChatGoogle(model='gemini-flash-latest', api_key='fake', http_options=pydantic_opts)
|
||||
params = chat._get_client_params()
|
||||
http_opts = params.get('http_options', {})
|
||||
|
||||
# Verify it extracts and preserves timeout and custom-header
|
||||
assert http_opts.get('timeout') == 30
|
||||
assert http_opts.get('headers', {}).get('custom-header') == 'value'
|
||||
assert http_opts.get('headers', {}).get('x-goog-api-client', '').startswith('browser-use/')
|
||||
|
||||
|
||||
def test_x_goog_api_client_header_with_dict_http_options():
|
||||
"""Test setting header when http_options is a dictionary (types.HttpOptionsDict)."""
|
||||
from google.genai import types
|
||||
|
||||
dict_opts: types.HttpOptionsDict = {
|
||||
'timeout': 45,
|
||||
'headers': {'another-header': 'another-value'},
|
||||
}
|
||||
chat = ChatGoogle(model='gemini-flash-latest', api_key='fake', http_options=dict_opts)
|
||||
params = chat._get_client_params()
|
||||
http_opts = params.get('http_options', {})
|
||||
|
||||
# Verify it preserves dictionary values and appends the tracking header
|
||||
assert http_opts.get('timeout') == 45
|
||||
assert http_opts.get('headers', {}).get('another-header') == 'another-value'
|
||||
assert http_opts.get('headers', {}).get('x-goog-api-client', '').startswith('browser-use/')
|
||||
@@ -0,0 +1,12 @@
|
||||
from browser_use.llm.anthropic.chat import ChatAnthropic
|
||||
from browser_use.llm.models import get_llm_by_name
|
||||
|
||||
|
||||
def test_get_llm_by_name_resolves_anthropic_from_env(monkeypatch):
|
||||
monkeypatch.setenv('ANTHROPIC_API_KEY', 'anthropic-test-key')
|
||||
|
||||
llm = get_llm_by_name('anthropic_claude_sonnet_4_0')
|
||||
|
||||
assert isinstance(llm, ChatAnthropic)
|
||||
assert llm.model == 'claude-sonnet-4-0'
|
||||
assert llm.api_key == 'anthropic-test-key'
|
||||
@@ -0,0 +1,15 @@
|
||||
"""Test OpenAI model button click."""
|
||||
|
||||
from browser_use.llm.openai.chat import ChatOpenAI
|
||||
from tests.ci.models.model_test_helper import run_model_button_click_test
|
||||
|
||||
|
||||
async def test_openai_gpt_4_1_mini(httpserver):
|
||||
"""Test OpenAI gpt-4.1-mini can click a button."""
|
||||
await run_model_button_click_test(
|
||||
model_class=ChatOpenAI,
|
||||
model_name='gpt-4.1-mini',
|
||||
api_key_env='OPENAI_API_KEY',
|
||||
extra_kwargs={},
|
||||
httpserver=httpserver,
|
||||
)
|
||||
@@ -0,0 +1,76 @@
|
||||
"""
|
||||
Tests for the SchemaOptimizer to ensure it correctly processes and
|
||||
optimizes the schemas for agent actions without losing information.
|
||||
"""
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
from browser_use.agent.views import AgentOutput
|
||||
from browser_use.llm.schema import SchemaOptimizer
|
||||
from browser_use.tools.service import Tools
|
||||
|
||||
|
||||
class ProductInfo(BaseModel):
|
||||
"""A sample structured output model with multiple fields."""
|
||||
|
||||
price: str
|
||||
title: str
|
||||
rating: float | None = None
|
||||
|
||||
|
||||
def test_optimizer_preserves_all_fields_in_structured_done_action():
|
||||
"""
|
||||
Ensures the SchemaOptimizer does not drop fields from a custom structured
|
||||
output model when creating the schema for the 'done' action.
|
||||
|
||||
This test specifically checks for a bug where fields were being lost
|
||||
during the optimization process.
|
||||
"""
|
||||
# 1. Setup a tools with a custom output model, simulating an Agent
|
||||
# being created with an `output_model_schema`.
|
||||
tools = Tools(output_model=ProductInfo)
|
||||
|
||||
# 2. Get the dynamically created AgentOutput model, which includes all registered actions.
|
||||
ActionModel = tools.registry.create_action_model()
|
||||
agent_output_model = AgentOutput.type_with_custom_actions(ActionModel)
|
||||
|
||||
# 3. Run the schema optimizer on the agent's output model.
|
||||
optimized_schema = SchemaOptimizer.create_optimized_json_schema(agent_output_model)
|
||||
|
||||
# 4. Find the 'done' action schema within the optimized output.
|
||||
# The path is properties -> action -> items -> anyOf -> [schema with 'done'].
|
||||
done_action_schema = None
|
||||
actions_schemas = optimized_schema.get('properties', {}).get('action', {}).get('items', {}).get('anyOf', [])
|
||||
for action_schema in actions_schemas:
|
||||
if 'done' in action_schema.get('properties', {}):
|
||||
done_action_schema = action_schema
|
||||
break
|
||||
|
||||
# 5. Assert that the 'done' action schema was successfully found.
|
||||
assert done_action_schema is not None, "Could not find 'done' action in the optimized schema."
|
||||
|
||||
# 6. Navigate to the schema for our custom data model within the 'done' action.
|
||||
# The path is properties -> done -> properties -> data -> properties.
|
||||
done_params_schema = done_action_schema.get('properties', {}).get('done', {})
|
||||
structured_data_schema = done_params_schema.get('properties', {}).get('data', {})
|
||||
final_properties = structured_data_schema.get('properties', {})
|
||||
|
||||
# 7. Assert that the set of fields in the optimized schema matches the original model's fields.
|
||||
original_fields = set(ProductInfo.model_fields.keys())
|
||||
optimized_fields = set(final_properties.keys())
|
||||
|
||||
assert original_fields == optimized_fields, (
|
||||
f"Field mismatch between original and optimized structured 'done' action schema.\n"
|
||||
f'Missing from optimized: {original_fields - optimized_fields}\n'
|
||||
f'Unexpected in optimized: {optimized_fields - original_fields}'
|
||||
)
|
||||
|
||||
|
||||
def test_gemini_schema_retains_required_fields():
|
||||
"""Gemini schema should keep explicit required arrays for mandatory fields."""
|
||||
schema = SchemaOptimizer.create_gemini_optimized_schema(ProductInfo)
|
||||
|
||||
assert 'required' in schema, 'Gemini schema removed required fields.'
|
||||
|
||||
required_fields = set(schema['required'])
|
||||
assert {'price', 'title'}.issubset(required_fields), 'Mandatory fields must stay required for Gemini.'
|
||||
@@ -0,0 +1,569 @@
|
||||
from browser_use.browser import BrowserProfile, BrowserSession
|
||||
|
||||
|
||||
class TestUrlAllowlistSecurity:
|
||||
"""Tests for URL allowlist security bypass prevention and URL allowlist glob pattern matching."""
|
||||
|
||||
def test_authentication_bypass_prevention(self):
|
||||
"""Test that the URL allowlist cannot be bypassed using authentication credentials."""
|
||||
from bubus import EventBus
|
||||
|
||||
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
|
||||
|
||||
# Create a context config with a sample allowed domain
|
||||
browser_profile = BrowserProfile(allowed_domains=['example.com'], headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Security vulnerability test cases
|
||||
# These should all be detected as malicious despite containing "example.com"
|
||||
assert watchdog._is_url_allowed('https://example.com:password@malicious.com') is False
|
||||
assert watchdog._is_url_allowed('https://example.com@malicious.com') is False
|
||||
assert watchdog._is_url_allowed('https://example.com%20@malicious.com') is False
|
||||
assert watchdog._is_url_allowed('https://example.com%3A@malicious.com') is False
|
||||
|
||||
# Make sure legitimate auth credentials still work
|
||||
assert watchdog._is_url_allowed('https://user:password@example.com') is True
|
||||
|
||||
def test_glob_pattern_matching(self):
|
||||
"""Test that glob patterns in allowed_domains work correctly."""
|
||||
from bubus import EventBus
|
||||
|
||||
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
|
||||
|
||||
# Test *.example.com pattern (should match subdomains and main domain)
|
||||
browser_profile = BrowserProfile(allowed_domains=['*.example.com'], headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Should match subdomains
|
||||
assert watchdog._is_url_allowed('https://sub.example.com') is True
|
||||
assert watchdog._is_url_allowed('https://deep.sub.example.com') is True
|
||||
|
||||
# Should also match main domain
|
||||
assert watchdog._is_url_allowed('https://example.com') is True
|
||||
|
||||
# Should not match other domains
|
||||
assert watchdog._is_url_allowed('https://notexample.com') is False
|
||||
assert watchdog._is_url_allowed('https://example.org') is False
|
||||
|
||||
# Test more complex glob patterns
|
||||
browser_profile = BrowserProfile(
|
||||
allowed_domains=[
|
||||
'*.google.com',
|
||||
'https://wiki.org',
|
||||
'https://good.com',
|
||||
'https://*.test.com',
|
||||
'chrome://version',
|
||||
'brave://*',
|
||||
],
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Should match domains ending with google.com
|
||||
assert watchdog._is_url_allowed('https://google.com') is True
|
||||
assert watchdog._is_url_allowed('https://www.google.com') is True
|
||||
assert (
|
||||
watchdog._is_url_allowed('https://evilgood.com') is False
|
||||
) # make sure we dont allow *good.com patterns, only *.good.com
|
||||
|
||||
# Should match domains starting with wiki
|
||||
assert watchdog._is_url_allowed('http://wiki.org') is False
|
||||
assert watchdog._is_url_allowed('https://wiki.org') is True
|
||||
|
||||
# Should not match internal domains because scheme was not provided
|
||||
assert watchdog._is_url_allowed('chrome://google.com') is False
|
||||
assert watchdog._is_url_allowed('chrome://abc.google.com') is False
|
||||
|
||||
# Test browser internal URLs
|
||||
assert watchdog._is_url_allowed('chrome://settings') is False
|
||||
assert watchdog._is_url_allowed('chrome://version') is True
|
||||
assert watchdog._is_url_allowed('chrome-extension://version/') is False
|
||||
assert watchdog._is_url_allowed('brave://anything/') is True
|
||||
assert watchdog._is_url_allowed('about:blank') is True
|
||||
assert watchdog._is_url_allowed('chrome://new-tab-page/') is True
|
||||
assert watchdog._is_url_allowed('chrome://new-tab-page') is True
|
||||
|
||||
# Test security for glob patterns (authentication credentials bypass attempts)
|
||||
# These should all be detected as malicious despite containing allowed domain patterns
|
||||
assert watchdog._is_url_allowed('https://allowed.example.com:password@notallowed.com') is False
|
||||
assert watchdog._is_url_allowed('https://subdomain.example.com@evil.com') is False
|
||||
assert watchdog._is_url_allowed('https://sub.example.com%20@malicious.org') is False
|
||||
assert watchdog._is_url_allowed('https://anygoogle.com@evil.org') is False
|
||||
|
||||
# Test pattern matching
|
||||
assert watchdog._is_url_allowed('https://www.test.com') is True
|
||||
assert watchdog._is_url_allowed('https://www.testx.com') is False
|
||||
|
||||
def test_glob_pattern_edge_cases(self):
|
||||
"""Test edge cases for glob pattern matching to ensure proper behavior."""
|
||||
from bubus import EventBus
|
||||
|
||||
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
|
||||
|
||||
# Test with domains containing glob pattern in the middle
|
||||
browser_profile = BrowserProfile(allowed_domains=['*.google.com', 'https://wiki.org'], headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Verify that 'wiki*' pattern doesn't match domains that merely contain 'wiki' in the middle
|
||||
assert watchdog._is_url_allowed('https://notawiki.com') is False
|
||||
assert watchdog._is_url_allowed('https://havewikipages.org') is False
|
||||
assert watchdog._is_url_allowed('https://my-wiki-site.com') is False
|
||||
|
||||
# Verify that '*google.com' doesn't match domains that have 'google' in the middle
|
||||
assert watchdog._is_url_allowed('https://mygoogle.company.com') is False
|
||||
|
||||
# Create context with potentially risky glob pattern that demonstrates security concerns
|
||||
browser_profile = BrowserProfile(allowed_domains=['*.google.com', '*.google.co.uk'], headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Should match legitimate Google domains
|
||||
assert watchdog._is_url_allowed('https://www.google.com') is True
|
||||
assert watchdog._is_url_allowed('https://mail.google.co.uk') is True
|
||||
|
||||
# Shouldn't match potentially malicious domains with a similar structure
|
||||
# This demonstrates why the previous pattern was risky and why it's now rejected
|
||||
assert watchdog._is_url_allowed('https://www.google.evil.com') is False
|
||||
|
||||
def test_automatic_www_subdomain_addition(self):
|
||||
"""Test that root domains automatically allow www subdomain."""
|
||||
from bubus import EventBus
|
||||
|
||||
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
|
||||
|
||||
# Test with simple root domains
|
||||
browser_profile = BrowserProfile(allowed_domains=['example.com', 'test.org'], headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Root domain should allow itself
|
||||
assert watchdog._is_url_allowed('https://example.com') is True
|
||||
assert watchdog._is_url_allowed('https://test.org') is True
|
||||
|
||||
# Root domain should automatically allow www subdomain
|
||||
assert watchdog._is_url_allowed('https://www.example.com') is True
|
||||
assert watchdog._is_url_allowed('https://www.test.org') is True
|
||||
|
||||
# Should not allow other subdomains
|
||||
assert watchdog._is_url_allowed('https://mail.example.com') is False
|
||||
assert watchdog._is_url_allowed('https://sub.test.org') is False
|
||||
|
||||
# Should not allow unrelated domains
|
||||
assert watchdog._is_url_allowed('https://notexample.com') is False
|
||||
assert watchdog._is_url_allowed('https://www.notexample.com') is False
|
||||
|
||||
def test_www_subdomain_not_added_for_country_tlds(self):
|
||||
"""Test www subdomain is NOT automatically added for country-specific TLDs (2+ dots)."""
|
||||
from bubus import EventBus
|
||||
|
||||
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
|
||||
|
||||
# Test with country-specific TLDs - these should NOT get automatic www
|
||||
browser_profile = BrowserProfile(
|
||||
allowed_domains=['example.co.uk', 'test.com.au', 'site.co.jp'], headless=True, user_data_dir=None
|
||||
)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Root domains should work exactly as specified
|
||||
assert watchdog._is_url_allowed('https://example.co.uk') is True
|
||||
assert watchdog._is_url_allowed('https://test.com.au') is True
|
||||
assert watchdog._is_url_allowed('https://site.co.jp') is True
|
||||
|
||||
# www subdomains should NOT work automatically (user must specify explicitly)
|
||||
assert watchdog._is_url_allowed('https://www.example.co.uk') is False
|
||||
assert watchdog._is_url_allowed('https://www.test.com.au') is False
|
||||
assert watchdog._is_url_allowed('https://www.site.co.jp') is False
|
||||
|
||||
# Other subdomains should not work
|
||||
assert watchdog._is_url_allowed('https://mail.example.co.uk') is False
|
||||
assert watchdog._is_url_allowed('https://api.test.com.au') is False
|
||||
|
||||
def test_www_subdomain_not_added_for_existing_subdomains(self):
|
||||
"""Test that www is not automatically added for domains that already have subdomains."""
|
||||
from bubus import EventBus
|
||||
|
||||
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
|
||||
|
||||
# Test with existing subdomains - should NOT get automatic www
|
||||
browser_profile = BrowserProfile(allowed_domains=['mail.example.com', 'api.test.org'], headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Exact subdomain should work
|
||||
assert watchdog._is_url_allowed('https://mail.example.com') is True
|
||||
assert watchdog._is_url_allowed('https://api.test.org') is True
|
||||
|
||||
# www should NOT be automatically added to subdomains
|
||||
assert watchdog._is_url_allowed('https://www.mail.example.com') is False
|
||||
assert watchdog._is_url_allowed('https://www.api.test.org') is False
|
||||
|
||||
# Root domains should not work either
|
||||
assert watchdog._is_url_allowed('https://example.com') is False
|
||||
assert watchdog._is_url_allowed('https://test.org') is False
|
||||
|
||||
def test_www_subdomain_not_added_for_wildcard_patterns(self):
|
||||
"""Test that www is not automatically added for wildcard patterns."""
|
||||
from bubus import EventBus
|
||||
|
||||
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
|
||||
|
||||
# Test with wildcard patterns - should NOT get automatic www logic
|
||||
browser_profile = BrowserProfile(allowed_domains=['*.example.com'], headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Wildcard should match everything including root and www
|
||||
assert watchdog._is_url_allowed('https://example.com') is True
|
||||
assert watchdog._is_url_allowed('https://www.example.com') is True
|
||||
assert watchdog._is_url_allowed('https://mail.example.com') is True
|
||||
|
||||
def test_www_subdomain_not_added_for_url_patterns(self):
|
||||
"""Test that www is not automatically added for full URL patterns."""
|
||||
from bubus import EventBus
|
||||
|
||||
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
|
||||
|
||||
# Test with full URL patterns - should NOT get automatic www logic
|
||||
browser_profile = BrowserProfile(
|
||||
allowed_domains=['https://example.com', 'http://test.org'], headless=True, user_data_dir=None
|
||||
)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Exact URL should work
|
||||
assert watchdog._is_url_allowed('https://example.com/path') is True
|
||||
assert watchdog._is_url_allowed('http://test.org/page') is True
|
||||
|
||||
# www should NOT be automatically added for full URL patterns
|
||||
assert watchdog._is_url_allowed('https://www.example.com') is False
|
||||
assert watchdog._is_url_allowed('http://www.test.org') is False
|
||||
|
||||
def test_is_root_domain_helper(self):
|
||||
"""Test the _is_root_domain helper method logic."""
|
||||
from bubus import EventBus
|
||||
|
||||
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
|
||||
|
||||
browser_profile = BrowserProfile(allowed_domains=['example.com'], headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Simple root domains (1 dot) - should return True
|
||||
assert watchdog._is_root_domain('example.com') is True
|
||||
assert watchdog._is_root_domain('test.org') is True
|
||||
assert watchdog._is_root_domain('site.net') is True
|
||||
|
||||
# Subdomains (more than 1 dot) - should return False
|
||||
assert watchdog._is_root_domain('www.example.com') is False
|
||||
assert watchdog._is_root_domain('mail.example.com') is False
|
||||
assert watchdog._is_root_domain('example.co.uk') is False
|
||||
assert watchdog._is_root_domain('test.com.au') is False
|
||||
|
||||
# Wildcards - should return False
|
||||
assert watchdog._is_root_domain('*.example.com') is False
|
||||
assert watchdog._is_root_domain('*example.com') is False
|
||||
|
||||
# Full URLs - should return False
|
||||
assert watchdog._is_root_domain('https://example.com') is False
|
||||
assert watchdog._is_root_domain('http://test.org') is False
|
||||
|
||||
# Invalid domains - should return False
|
||||
assert watchdog._is_root_domain('example') is False
|
||||
assert watchdog._is_root_domain('') is False
|
||||
|
||||
|
||||
class TestUrlProhibitlistSecurity:
|
||||
"""Tests for URL prohibitlist (blocked domains) behavior and matching semantics."""
|
||||
|
||||
def test_simple_prohibited_domains(self):
|
||||
"""Domain-only patterns block exact host and www, but not other subdomains."""
|
||||
from bubus import EventBus
|
||||
|
||||
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
|
||||
|
||||
browser_profile = BrowserProfile(prohibited_domains=['example.com', 'test.org'], headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Block exact and www
|
||||
assert watchdog._is_url_allowed('https://example.com') is False
|
||||
assert watchdog._is_url_allowed('https://www.example.com') is False
|
||||
assert watchdog._is_url_allowed('https://test.org') is False
|
||||
assert watchdog._is_url_allowed('https://www.test.org') is False
|
||||
|
||||
# Allow other subdomains when only root is prohibited
|
||||
assert watchdog._is_url_allowed('https://mail.example.com') is True
|
||||
assert watchdog._is_url_allowed('https://api.test.org') is True
|
||||
|
||||
# Allow unrelated domains
|
||||
assert watchdog._is_url_allowed('https://notexample.com') is True
|
||||
|
||||
def test_glob_pattern_prohibited(self):
|
||||
"""Wildcard patterns block subdomains and main domain for http/https only."""
|
||||
from bubus import EventBus
|
||||
|
||||
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
|
||||
|
||||
browser_profile = BrowserProfile(prohibited_domains=['*.example.com'], headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Block subdomains and main domain
|
||||
assert watchdog._is_url_allowed('https://example.com') is False
|
||||
assert watchdog._is_url_allowed('https://www.example.com') is False
|
||||
assert watchdog._is_url_allowed('https://mail.example.com') is False
|
||||
|
||||
# Allow other domains
|
||||
assert watchdog._is_url_allowed('https://notexample.com') is True
|
||||
|
||||
# Wildcard with domain-only should not apply to non-http(s)
|
||||
assert watchdog._is_url_allowed('chrome://abc.example.com') is True
|
||||
|
||||
def test_full_url_prohibited_patterns(self):
|
||||
"""Full URL patterns block only matching scheme/host/prefix."""
|
||||
from bubus import EventBus
|
||||
|
||||
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
|
||||
|
||||
browser_profile = BrowserProfile(prohibited_domains=['https://wiki.org', 'brave://*'], headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Scheme-specific blocking
|
||||
assert watchdog._is_url_allowed('http://wiki.org') is True
|
||||
assert watchdog._is_url_allowed('https://wiki.org') is False
|
||||
assert watchdog._is_url_allowed('https://wiki.org/path') is False
|
||||
|
||||
# Internal URL prefix blocking
|
||||
assert watchdog._is_url_allowed('brave://anything/') is False
|
||||
assert watchdog._is_url_allowed('chrome://settings') is True
|
||||
|
||||
def test_internal_urls_allowed_even_when_prohibited(self):
|
||||
"""Internal new-tab/blank URLs are always allowed regardless of prohibited list."""
|
||||
from bubus import EventBus
|
||||
|
||||
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
|
||||
|
||||
browser_profile = BrowserProfile(prohibited_domains=['*'], headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
assert watchdog._is_url_allowed('about:blank') is True
|
||||
assert watchdog._is_url_allowed('chrome://new-tab-page/') is True
|
||||
assert watchdog._is_url_allowed('chrome://new-tab-page') is True
|
||||
assert watchdog._is_url_allowed('chrome://newtab/') is True
|
||||
|
||||
def test_prohibited_ignored_when_allowlist_present(self):
|
||||
"""When allowlist is set, prohibited list is ignored by design."""
|
||||
from bubus import EventBus
|
||||
|
||||
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
|
||||
|
||||
browser_profile = BrowserProfile(
|
||||
allowed_domains=['*.example.com'],
|
||||
prohibited_domains=['https://example.com'],
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Allowed by allowlist even though exact URL is in prohibited list
|
||||
assert watchdog._is_url_allowed('https://example.com') is True
|
||||
assert watchdog._is_url_allowed('https://www.example.com') is True
|
||||
|
||||
# Not in allowlist => blocked (prohibited list is not consulted in this mode)
|
||||
assert watchdog._is_url_allowed('https://api.example.com') is True # wildcard allowlist includes this
|
||||
# A domain outside the allowlist should be blocked
|
||||
assert watchdog._is_url_allowed('https://notexample.com') is False
|
||||
|
||||
def test_auth_credentials_do_not_cause_false_block(self):
|
||||
"""Credentials injection with prohibited domain in username should not block unrelated hosts."""
|
||||
from bubus import EventBus
|
||||
|
||||
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
|
||||
|
||||
browser_profile = BrowserProfile(prohibited_domains=['example.com'], headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Host is malicious.com, should not be blocked just because username contains example.com
|
||||
assert watchdog._is_url_allowed('https://example.com:password@malicious.com') is True
|
||||
assert watchdog._is_url_allowed('https://example.com@malicious.com') is True
|
||||
assert watchdog._is_url_allowed('https://example.com%20@malicious.com') is True
|
||||
assert watchdog._is_url_allowed('https://example.com%3A@malicious.com') is True
|
||||
|
||||
# Legitimate credentials to a prohibited host should be blocked
|
||||
assert watchdog._is_url_allowed('https://user:password@example.com') is False
|
||||
|
||||
def test_case_insensitive_prohibited_domains(self):
|
||||
"""Prohibited domain matching should be case-insensitive."""
|
||||
from bubus import EventBus
|
||||
|
||||
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
|
||||
|
||||
browser_profile = BrowserProfile(prohibited_domains=['Example.COM'], headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
assert watchdog._is_url_allowed('https://example.com') is False
|
||||
assert watchdog._is_url_allowed('https://WWW.EXAMPLE.COM') is False
|
||||
assert watchdog._is_url_allowed('https://mail.example.com') is True
|
||||
|
||||
|
||||
class TestDomainListOptimization:
|
||||
"""Tests for domain list optimization (set conversion for large lists)."""
|
||||
|
||||
def test_small_list_keeps_pattern_support(self):
|
||||
"""Test that lists < 100 items keep pattern matching support."""
|
||||
from bubus import EventBus
|
||||
|
||||
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
|
||||
|
||||
browser_profile = BrowserProfile(
|
||||
prohibited_domains=['*.google.com', 'x.com', 'facebook.com'], headless=True, user_data_dir=None
|
||||
)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Should still be a list
|
||||
assert isinstance(browser_session.browser_profile.prohibited_domains, list)
|
||||
|
||||
# Pattern matching should work
|
||||
assert watchdog._is_url_allowed('https://www.google.com') is False
|
||||
assert watchdog._is_url_allowed('https://mail.google.com') is False
|
||||
assert watchdog._is_url_allowed('https://google.com') is False
|
||||
|
||||
# Exact matches should work
|
||||
assert watchdog._is_url_allowed('https://x.com') is False
|
||||
assert watchdog._is_url_allowed('https://facebook.com') is False
|
||||
|
||||
# Other domains should be allowed
|
||||
assert watchdog._is_url_allowed('https://example.com') is True
|
||||
|
||||
def test_large_list_converts_to_set(self):
|
||||
"""Test that lists >= 100 items are converted to sets."""
|
||||
from bubus import EventBus
|
||||
|
||||
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
|
||||
|
||||
# Create a list of 100 domains
|
||||
large_list = [f'blocked{i}.com' for i in range(100)]
|
||||
|
||||
browser_profile = BrowserProfile(prohibited_domains=large_list, headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Should be converted to set
|
||||
assert isinstance(browser_session.browser_profile.prohibited_domains, set)
|
||||
assert len(browser_session.browser_profile.prohibited_domains) == 100
|
||||
|
||||
# Exact matches should work
|
||||
assert watchdog._is_url_allowed('https://blocked0.com') is False
|
||||
assert watchdog._is_url_allowed('https://blocked50.com') is False
|
||||
assert watchdog._is_url_allowed('https://blocked99.com') is False
|
||||
|
||||
# Other domains should be allowed
|
||||
assert watchdog._is_url_allowed('https://example.com') is True
|
||||
assert watchdog._is_url_allowed('https://blocked100.com') is True # Not in list
|
||||
|
||||
def test_www_variant_matching_with_sets(self):
|
||||
"""Test that www variants are checked in set-based lookups."""
|
||||
from bubus import EventBus
|
||||
|
||||
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
|
||||
|
||||
# Create a list with 100 domains (some with www, some without)
|
||||
large_list = [f'site{i}.com' for i in range(50)] + [f'www.domain{i}.org' for i in range(50)]
|
||||
|
||||
browser_profile = BrowserProfile(prohibited_domains=large_list, headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Should be converted to set
|
||||
assert isinstance(browser_session.browser_profile.prohibited_domains, set)
|
||||
|
||||
# Test www variant matching for domains without www prefix
|
||||
assert watchdog._is_url_allowed('https://site0.com') is False
|
||||
assert watchdog._is_url_allowed('https://www.site0.com') is False # Should also be blocked
|
||||
|
||||
# Test www variant matching for domains with www prefix
|
||||
assert watchdog._is_url_allowed('https://www.domain0.org') is False
|
||||
assert watchdog._is_url_allowed('https://domain0.org') is False # Should also be blocked
|
||||
|
||||
# Test that unrelated domains are allowed
|
||||
assert watchdog._is_url_allowed('https://example.com') is True
|
||||
assert watchdog._is_url_allowed('https://www.example.com') is True
|
||||
|
||||
def test_allowed_domains_with_sets(self):
|
||||
"""Test that allowed_domains also works with set optimization."""
|
||||
from bubus import EventBus
|
||||
|
||||
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
|
||||
|
||||
# Create a large allowlist
|
||||
large_list = [f'allowed{i}.com' for i in range(100)]
|
||||
|
||||
browser_profile = BrowserProfile(allowed_domains=large_list, headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Should be converted to set
|
||||
assert isinstance(browser_session.browser_profile.allowed_domains, set)
|
||||
|
||||
# Allowed domains should work
|
||||
assert watchdog._is_url_allowed('https://allowed0.com') is True
|
||||
assert watchdog._is_url_allowed('https://www.allowed0.com') is True
|
||||
assert watchdog._is_url_allowed('https://allowed99.com') is True
|
||||
|
||||
# Other domains should be blocked
|
||||
assert watchdog._is_url_allowed('https://example.com') is False
|
||||
assert watchdog._is_url_allowed('https://notallowed.com') is False
|
||||
|
||||
def test_manual_set_input(self):
|
||||
"""Test that users can directly provide a set."""
|
||||
from bubus import EventBus
|
||||
|
||||
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
|
||||
|
||||
blocked_set = {f'blocked{i}.com' for i in range(50)}
|
||||
|
||||
browser_profile = BrowserProfile(prohibited_domains=blocked_set, headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Should remain a set
|
||||
assert isinstance(browser_session.browser_profile.prohibited_domains, set)
|
||||
|
||||
# Should work correctly
|
||||
assert watchdog._is_url_allowed('https://blocked0.com') is False
|
||||
assert watchdog._is_url_allowed('https://example.com') is True
|
||||
@@ -0,0 +1,116 @@
|
||||
"""Tests for download filename sanitization (GHSA-rv9j-wqjp-2fv4,
|
||||
GHSA-66xh-g88g-2h8j, GHSA-hpr4-fqgr-xhj9).
|
||||
|
||||
`DownloadsWatchdog` historically joined attacker-controlled filenames from CDP
|
||||
(`Page.downloadWillBegin.suggestedFilename`) and `Content-Disposition` headers
|
||||
directly into the configured `downloads_path`. Strings like `../../escape.bin`
|
||||
or `/etc/shadow.bak` would `os.path.join` outside the downloads directory,
|
||||
writing the fetched bytes (also attacker-controlled — the response body is the
|
||||
exploit content) to an arbitrary location with the agent's process privileges.
|
||||
|
||||
`download_file_from_url` triggers passively for any
|
||||
`Content-Disposition: attachment` response, so this is reachable from any
|
||||
visited site — `allowed_domains` does not mitigate it.
|
||||
|
||||
The fix funnels every attacker-controlled filename through
|
||||
`DownloadsWatchdog._sanitize_download_filename`, which keeps only the basename
|
||||
and rejects pure-traversal names. Each on-disk sink additionally verifies
|
||||
containment via `os.path.realpath`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from browser_use.browser.watchdogs.downloads_watchdog import DownloadsWatchdog
|
||||
|
||||
|
||||
class TestSanitizeDownloadFilename:
|
||||
def test_strips_relative_traversal(self) -> None:
|
||||
assert DownloadsWatchdog._sanitize_download_filename('../../etc/passwd') == 'passwd'
|
||||
|
||||
def test_strips_absolute_unix_path(self) -> None:
|
||||
assert DownloadsWatchdog._sanitize_download_filename('/etc/shadow') == 'shadow'
|
||||
|
||||
def test_strips_windows_backslash_paths(self) -> None:
|
||||
assert DownloadsWatchdog._sanitize_download_filename('..\\..\\Windows\\System32\\config.txt') == 'config.txt'
|
||||
|
||||
def test_strips_mixed_separators(self) -> None:
|
||||
assert DownloadsWatchdog._sanitize_download_filename('a/b\\c/../d.pdf') == 'd.pdf'
|
||||
|
||||
def test_pure_traversal_falls_back_to_download(self) -> None:
|
||||
for malicious in ('..', '.', '/', '\\', '../', '..\\', '/.', '\\.', '/..'):
|
||||
assert DownloadsWatchdog._sanitize_download_filename(malicious) == 'download', (
|
||||
f'{malicious!r} should fall back to default'
|
||||
)
|
||||
|
||||
def test_null_byte_stripped(self) -> None:
|
||||
# Null bytes can be used to confuse path handling on some platforms.
|
||||
assert DownloadsWatchdog._sanitize_download_filename('file.txt\x00.exe') == 'file.txt.exe'
|
||||
|
||||
def test_empty_or_none_falls_back(self) -> None:
|
||||
assert DownloadsWatchdog._sanitize_download_filename('') == 'download'
|
||||
assert DownloadsWatchdog._sanitize_download_filename(None) == 'download'
|
||||
|
||||
def test_normal_filenames_preserved(self) -> None:
|
||||
# Make sure we don't over-sanitize legitimate names.
|
||||
assert DownloadsWatchdog._sanitize_download_filename('report.pdf') == 'report.pdf'
|
||||
assert DownloadsWatchdog._sanitize_download_filename('file with spaces.pdf') == 'file with spaces.pdf'
|
||||
assert DownloadsWatchdog._sanitize_download_filename('file-with_underscores.csv') == 'file-with_underscores.csv'
|
||||
# Dotfiles are allowed as long as they're not just dots.
|
||||
assert DownloadsWatchdog._sanitize_download_filename('.bashrc') == '.bashrc'
|
||||
|
||||
def test_unicode_preserved(self) -> None:
|
||||
# Filenames with non-ASCII characters should survive (common for i18n filenames).
|
||||
assert DownloadsWatchdog._sanitize_download_filename('résumé.pdf') == 'résumé.pdf'
|
||||
assert DownloadsWatchdog._sanitize_download_filename('文档.pdf') == '文档.pdf'
|
||||
|
||||
|
||||
class TestIsPathContained:
|
||||
def test_file_inside_dir_returns_true(self, tmp_path: Path) -> None:
|
||||
f = tmp_path / 'a.txt'
|
||||
f.write_text('x')
|
||||
assert DownloadsWatchdog._is_path_contained(f, tmp_path) is True
|
||||
|
||||
def test_nested_file_inside_dir_returns_true(self, tmp_path: Path) -> None:
|
||||
nested = tmp_path / 'sub' / 'a.txt'
|
||||
nested.parent.mkdir()
|
||||
nested.write_text('x')
|
||||
assert DownloadsWatchdog._is_path_contained(nested, tmp_path) is True
|
||||
|
||||
def test_escaping_path_returns_false(self, tmp_path: Path) -> None:
|
||||
escape = tmp_path / '..' / 'a.txt'
|
||||
assert DownloadsWatchdog._is_path_contained(escape, tmp_path) is False
|
||||
|
||||
def test_dir_itself_returns_true(self, tmp_path: Path) -> None:
|
||||
assert DownloadsWatchdog._is_path_contained(tmp_path, tmp_path) is True
|
||||
|
||||
def test_sibling_dir_returns_false(self, tmp_path: Path) -> None:
|
||||
sibling = tmp_path.parent / (tmp_path.name + '_sibling')
|
||||
sibling.mkdir(exist_ok=True)
|
||||
try:
|
||||
f = sibling / 'a.txt'
|
||||
f.write_text('x')
|
||||
assert DownloadsWatchdog._is_path_contained(f, tmp_path) is False
|
||||
finally:
|
||||
f.unlink(missing_ok=True)
|
||||
sibling.rmdir()
|
||||
|
||||
|
||||
class TestUniqueFilenameOperatesOnSanitizedBasename:
|
||||
"""`_get_unique_filename` must only ever receive a sanitized basename; if a
|
||||
traversal string slips through, the (1)/(2) collision-avoidance logic
|
||||
silently writes outside the intended directory."""
|
||||
|
||||
async def test_unique_filename_on_basename_stays_inside_dir(self, tmp_path: Path) -> None:
|
||||
# Pre-sanitized name — what the caller should always pass.
|
||||
result = await DownloadsWatchdog._get_unique_filename(str(tmp_path), 'report.pdf')
|
||||
assert result == 'report.pdf'
|
||||
# The resolved path lives inside tmp_path.
|
||||
assert DownloadsWatchdog._is_path_contained(tmp_path / result, tmp_path)
|
||||
|
||||
async def test_unique_filename_collision_handling_stays_inside_dir(self, tmp_path: Path) -> None:
|
||||
(tmp_path / 'report.pdf').write_text('x')
|
||||
result = await DownloadsWatchdog._get_unique_filename(str(tmp_path), 'report.pdf')
|
||||
assert result == 'report (1).pdf'
|
||||
assert DownloadsWatchdog._is_path_contained(tmp_path / result, tmp_path)
|
||||
@@ -0,0 +1,675 @@
|
||||
"""
|
||||
Comprehensive tests for IP address blocking in SecurityWatchdog.
|
||||
|
||||
Tests cover IPv4, IPv6, localhost, private networks, edge cases, and interactions
|
||||
with allowed_domains and prohibited_domains configurations.
|
||||
"""
|
||||
|
||||
from bubus import EventBus
|
||||
|
||||
from browser_use.browser import BrowserProfile, BrowserSession
|
||||
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
|
||||
|
||||
|
||||
class TestIPv4Blocking:
|
||||
"""Test blocking of IPv4 addresses."""
|
||||
|
||||
def test_block_public_ipv4_addresses(self):
|
||||
"""Test that public IPv4 addresses are blocked when block_ip_addresses=True."""
|
||||
browser_profile = BrowserProfile(block_ip_addresses=True, headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Public IPv4 addresses should be blocked
|
||||
assert watchdog._is_url_allowed('http://180.1.1.1/supersafe.txt') is False
|
||||
assert watchdog._is_url_allowed('https://8.8.8.8/') is False
|
||||
assert watchdog._is_url_allowed('http://1.1.1.1:8080/api') is False
|
||||
assert watchdog._is_url_allowed('https://142.250.185.46/search') is False
|
||||
assert watchdog._is_url_allowed('http://93.184.216.34/') is False
|
||||
|
||||
def test_block_private_ipv4_networks(self):
|
||||
"""Test that private network IPv4 addresses are blocked."""
|
||||
browser_profile = BrowserProfile(block_ip_addresses=True, headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Private network ranges (RFC 1918)
|
||||
assert watchdog._is_url_allowed('http://192.168.1.1/') is False
|
||||
assert watchdog._is_url_allowed('http://192.168.0.100/admin') is False
|
||||
assert watchdog._is_url_allowed('http://10.0.0.1/') is False
|
||||
assert watchdog._is_url_allowed('http://10.255.255.255/') is False
|
||||
assert watchdog._is_url_allowed('http://172.16.0.1/') is False
|
||||
assert watchdog._is_url_allowed('http://172.31.255.254/') is False
|
||||
|
||||
def test_block_localhost_ipv4(self):
|
||||
"""Test that localhost IPv4 addresses are blocked."""
|
||||
browser_profile = BrowserProfile(block_ip_addresses=True, headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Localhost/loopback addresses
|
||||
assert watchdog._is_url_allowed('http://127.0.0.1/') is False
|
||||
assert watchdog._is_url_allowed('http://127.0.0.1:8080/') is False
|
||||
assert watchdog._is_url_allowed('https://127.0.0.1:3000/api/test') is False
|
||||
assert watchdog._is_url_allowed('http://127.1.2.3/') is False # Any 127.x.x.x
|
||||
|
||||
def test_block_ipv4_with_ports_and_paths(self):
|
||||
"""Test that IPv4 addresses with ports and paths are blocked."""
|
||||
browser_profile = BrowserProfile(block_ip_addresses=True, headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# With various ports
|
||||
assert watchdog._is_url_allowed('http://8.8.8.8:80/') is False
|
||||
assert watchdog._is_url_allowed('https://8.8.8.8:443/') is False
|
||||
assert watchdog._is_url_allowed('http://192.168.1.1:8080/') is False
|
||||
assert watchdog._is_url_allowed('http://10.0.0.1:3000/api') is False
|
||||
|
||||
# With paths and query strings
|
||||
assert watchdog._is_url_allowed('http://1.2.3.4/path/to/resource') is False
|
||||
assert watchdog._is_url_allowed('http://5.6.7.8/api?key=value') is False
|
||||
assert watchdog._is_url_allowed('https://9.10.11.12/path/to/file.html#anchor') is False
|
||||
|
||||
def test_allow_ipv4_when_blocking_disabled(self):
|
||||
"""Test that IPv4 addresses are allowed when block_ip_addresses=False (default)."""
|
||||
browser_profile = BrowserProfile(block_ip_addresses=False, headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# All IP addresses should be allowed when blocking is disabled
|
||||
assert watchdog._is_url_allowed('http://180.1.1.1/supersafe.txt') is True
|
||||
assert watchdog._is_url_allowed('http://192.168.1.1/') is True
|
||||
assert watchdog._is_url_allowed('http://127.0.0.1:8080/') is True
|
||||
assert watchdog._is_url_allowed('http://8.8.8.8/') is True
|
||||
|
||||
|
||||
class TestIPv6Blocking:
|
||||
"""Test blocking of IPv6 addresses."""
|
||||
|
||||
def test_block_ipv6_addresses(self):
|
||||
"""Test that IPv6 addresses are blocked when block_ip_addresses=True."""
|
||||
browser_profile = BrowserProfile(block_ip_addresses=True, headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Public IPv6 addresses (with brackets as per URL standard)
|
||||
assert watchdog._is_url_allowed('http://[2001:db8::1]/') is False
|
||||
assert watchdog._is_url_allowed('https://[2001:4860:4860::8888]/') is False
|
||||
assert watchdog._is_url_allowed('http://[2606:4700:4700::1111]/path') is False
|
||||
assert watchdog._is_url_allowed('https://[2001:db8:85a3::8a2e:370:7334]/api') is False
|
||||
|
||||
def test_block_ipv6_localhost(self):
|
||||
"""Test that IPv6 localhost addresses are blocked."""
|
||||
browser_profile = BrowserProfile(block_ip_addresses=True, headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# IPv6 loopback
|
||||
assert watchdog._is_url_allowed('http://[::1]/') is False
|
||||
assert watchdog._is_url_allowed('http://[::1]:8080/') is False
|
||||
assert watchdog._is_url_allowed('https://[::1]:3000/api') is False
|
||||
assert watchdog._is_url_allowed('http://[0:0:0:0:0:0:0:1]/') is False # Expanded form
|
||||
|
||||
def test_block_ipv6_with_ports_and_paths(self):
|
||||
"""Test that IPv6 addresses with ports and paths are blocked."""
|
||||
browser_profile = BrowserProfile(block_ip_addresses=True, headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# IPv6 with ports
|
||||
assert watchdog._is_url_allowed('http://[2001:db8::1]:80/') is False
|
||||
assert watchdog._is_url_allowed('https://[2001:db8::1]:443/') is False
|
||||
assert watchdog._is_url_allowed('http://[::1]:8080/api') is False
|
||||
|
||||
# IPv6 with paths
|
||||
assert watchdog._is_url_allowed('http://[2001:db8::1]/path/to/resource') is False
|
||||
assert watchdog._is_url_allowed('https://[2001:db8::1]/api?key=value') is False
|
||||
|
||||
def test_allow_ipv6_when_blocking_disabled(self):
|
||||
"""Test that IPv6 addresses are allowed when block_ip_addresses=False."""
|
||||
browser_profile = BrowserProfile(block_ip_addresses=False, headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# All IPv6 addresses should be allowed
|
||||
assert watchdog._is_url_allowed('http://[2001:db8::1]/') is True
|
||||
assert watchdog._is_url_allowed('http://[::1]:8080/') is True
|
||||
assert watchdog._is_url_allowed('https://[2001:4860:4860::8888]/') is True
|
||||
|
||||
|
||||
class TestDomainNamesStillAllowed:
|
||||
"""Test that regular domain names are not affected by IP blocking."""
|
||||
|
||||
def test_domain_names_allowed_with_ip_blocking(self):
|
||||
"""Test that domain names continue to work when IP blocking is enabled."""
|
||||
browser_profile = BrowserProfile(block_ip_addresses=True, headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Regular domain names should still be allowed
|
||||
assert watchdog._is_url_allowed('https://example.com') is True
|
||||
assert watchdog._is_url_allowed('https://www.google.com') is True
|
||||
assert watchdog._is_url_allowed('http://subdomain.example.org/path') is True
|
||||
assert watchdog._is_url_allowed('https://api.github.com/repos') is True
|
||||
assert watchdog._is_url_allowed('http://localhost/') is True # "localhost" is a domain name, not IP
|
||||
assert watchdog._is_url_allowed('http://localhost:8080/api') is True
|
||||
|
||||
def test_domains_with_numbers_allowed(self):
|
||||
"""Test that domain names containing numbers are still allowed."""
|
||||
browser_profile = BrowserProfile(block_ip_addresses=True, headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Domains with numbers (but not valid IP addresses)
|
||||
assert watchdog._is_url_allowed('https://example123.com') is True
|
||||
assert watchdog._is_url_allowed('https://123example.com') is True
|
||||
assert watchdog._is_url_allowed('https://server1.example.com') is True
|
||||
assert watchdog._is_url_allowed('http://web2.site.org') is True
|
||||
|
||||
|
||||
class TestIPBlockingWithAllowedDomains:
|
||||
"""Test interaction between IP blocking and allowed_domains."""
|
||||
|
||||
def test_ip_blocked_even_in_allowed_domains(self):
|
||||
"""Test that IPs are blocked even if they're in allowed_domains list."""
|
||||
# Note: It doesn't make sense to add IPs to allowed_domains, but if someone does,
|
||||
# IP blocking should take precedence
|
||||
browser_profile = BrowserProfile(
|
||||
block_ip_addresses=True,
|
||||
allowed_domains=['example.com', '192.168.1.1'], # IP in allowlist
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# IP should be blocked despite being in allowed_domains
|
||||
assert watchdog._is_url_allowed('http://192.168.1.1/') is False
|
||||
|
||||
# Regular domain should work as expected
|
||||
assert watchdog._is_url_allowed('https://example.com') is True
|
||||
|
||||
# Other domains not in allowed_domains should be blocked
|
||||
assert watchdog._is_url_allowed('https://other.com') is False
|
||||
|
||||
def test_allowed_domains_with_ip_blocking_enabled(self):
|
||||
"""Test that allowed_domains works normally with IP blocking enabled."""
|
||||
browser_profile = BrowserProfile(
|
||||
block_ip_addresses=True, allowed_domains=['example.com', '*.google.com'], headless=True, user_data_dir=None
|
||||
)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Allowed domains should work
|
||||
assert watchdog._is_url_allowed('https://example.com') is True
|
||||
assert watchdog._is_url_allowed('https://www.google.com') is True
|
||||
|
||||
# Not allowed domains should be blocked
|
||||
assert watchdog._is_url_allowed('https://other.com') is False
|
||||
|
||||
# IPs should be blocked regardless
|
||||
assert watchdog._is_url_allowed('http://8.8.8.8/') is False
|
||||
assert watchdog._is_url_allowed('http://192.168.1.1/') is False
|
||||
|
||||
|
||||
class TestIPBlockingWithProhibitedDomains:
|
||||
"""Test interaction between IP blocking and prohibited_domains."""
|
||||
|
||||
def test_ip_blocked_regardless_of_prohibited_domains(self):
|
||||
"""Test that IPs are blocked when IP blocking is on, independent of prohibited_domains."""
|
||||
browser_profile = BrowserProfile(
|
||||
block_ip_addresses=True, prohibited_domains=['example.com'], headless=True, user_data_dir=None
|
||||
)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# IPs should be blocked due to IP blocking
|
||||
assert watchdog._is_url_allowed('http://192.168.1.1/') is False
|
||||
assert watchdog._is_url_allowed('http://8.8.8.8/') is False
|
||||
|
||||
# Prohibited domain should be blocked
|
||||
assert watchdog._is_url_allowed('https://example.com') is False
|
||||
|
||||
# Other domains should be allowed
|
||||
assert watchdog._is_url_allowed('https://other.com') is True
|
||||
|
||||
def test_prohibited_domains_without_ip_blocking(self):
|
||||
"""Test that prohibited_domains works normally when IP blocking is disabled."""
|
||||
browser_profile = BrowserProfile(
|
||||
block_ip_addresses=False, prohibited_domains=['example.com', '8.8.8.8'], headless=True, user_data_dir=None
|
||||
)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Prohibited domain should be blocked
|
||||
assert watchdog._is_url_allowed('https://example.com') is False
|
||||
|
||||
# IP in prohibited list should be blocked (by prohibited_domains, not IP blocking)
|
||||
assert watchdog._is_url_allowed('http://8.8.8.8/') is False
|
||||
|
||||
# Other IPs should be allowed (IP blocking is off)
|
||||
assert watchdog._is_url_allowed('http://192.168.1.1/') is True
|
||||
|
||||
# Other domains should be allowed
|
||||
assert watchdog._is_url_allowed('https://other.com') is True
|
||||
|
||||
|
||||
class TestEdgeCases:
|
||||
"""Test edge cases and invalid inputs."""
|
||||
|
||||
def test_invalid_urls_handled_gracefully(self):
|
||||
"""Test that invalid URLs don't cause crashes."""
|
||||
browser_profile = BrowserProfile(block_ip_addresses=True, headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Invalid URLs should return False
|
||||
assert watchdog._is_url_allowed('not-a-url') is False
|
||||
assert watchdog._is_url_allowed('') is False
|
||||
assert watchdog._is_url_allowed('http://') is False
|
||||
assert watchdog._is_url_allowed('://example.com') is False
|
||||
|
||||
def test_internal_browser_urls_allowed(self):
|
||||
"""Test that internal browser URLs are still allowed with IP blocking."""
|
||||
browser_profile = BrowserProfile(block_ip_addresses=True, headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Internal URLs should always be allowed
|
||||
assert watchdog._is_url_allowed('about:blank') is True
|
||||
assert watchdog._is_url_allowed('chrome://new-tab-page/') is True
|
||||
assert watchdog._is_url_allowed('chrome://new-tab-page') is True
|
||||
assert watchdog._is_url_allowed('chrome://newtab/') is True
|
||||
|
||||
def test_ipv4_lookalike_domains_allowed(self):
|
||||
"""Test that strings that look like IPs but cannot be resolved as IPs by
|
||||
the kernel/browser are still treated as domain names and allowed.
|
||||
|
||||
Note: short-form IPv4 strings such as `1.2.3` (which Chromium resolves
|
||||
as `1.2.0.3`) ARE recognized as IPs — see TestNonStandardIPv4Representations.
|
||||
"""
|
||||
browser_profile = BrowserProfile(block_ip_addresses=True, headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# These look like IPs but cannot resolve as IPv4 in any form the browser accepts:
|
||||
# - 999.999.999.999: out of range for every octet
|
||||
# - 1.2.3.4.5: too many octets
|
||||
assert watchdog._is_url_allowed('http://999.999.999.999/') is True
|
||||
assert watchdog._is_url_allowed('http://1.2.3.4.5/') is True
|
||||
|
||||
def test_different_schemes_with_ips(self):
|
||||
"""Test that IP blocking works across different URL schemes."""
|
||||
browser_profile = BrowserProfile(block_ip_addresses=True, headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# HTTP and HTTPS
|
||||
assert watchdog._is_url_allowed('http://192.168.1.1/') is False
|
||||
assert watchdog._is_url_allowed('https://192.168.1.1/') is False
|
||||
|
||||
# FTP (if browser supports it)
|
||||
assert watchdog._is_url_allowed('ftp://192.168.1.1/') is False
|
||||
|
||||
# WebSocket (parsed as regular URL)
|
||||
assert watchdog._is_url_allowed('ws://192.168.1.1:8080/') is False
|
||||
assert watchdog._is_url_allowed('wss://192.168.1.1:8080/') is False
|
||||
|
||||
|
||||
class TestIsIPAddressHelper:
|
||||
"""Test the _is_ip_address helper method directly."""
|
||||
|
||||
def test_valid_ipv4_detection(self):
|
||||
"""Test that valid IPv4 addresses are correctly detected."""
|
||||
browser_profile = BrowserProfile(block_ip_addresses=True, headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Valid IPv4 addresses
|
||||
assert watchdog._is_ip_address('127.0.0.1') is True
|
||||
assert watchdog._is_ip_address('192.168.1.1') is True
|
||||
assert watchdog._is_ip_address('8.8.8.8') is True
|
||||
assert watchdog._is_ip_address('255.255.255.255') is True
|
||||
assert watchdog._is_ip_address('0.0.0.0') is True
|
||||
|
||||
def test_valid_ipv6_detection(self):
|
||||
"""Test that valid IPv6 addresses are correctly detected."""
|
||||
browser_profile = BrowserProfile(block_ip_addresses=True, headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Valid IPv6 addresses (without brackets - those are URL-specific)
|
||||
assert watchdog._is_ip_address('::1') is True
|
||||
assert watchdog._is_ip_address('2001:db8::1') is True
|
||||
assert watchdog._is_ip_address('2001:4860:4860::8888') is True
|
||||
assert watchdog._is_ip_address('fe80::1') is True
|
||||
assert watchdog._is_ip_address('2001:db8:85a3::8a2e:370:7334') is True
|
||||
|
||||
def test_invalid_ip_detection(self):
|
||||
"""Test that non-IP strings are correctly identified as not IPs."""
|
||||
browser_profile = BrowserProfile(block_ip_addresses=True, headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Domain names (not IPs)
|
||||
assert watchdog._is_ip_address('example.com') is False
|
||||
assert watchdog._is_ip_address('www.google.com') is False
|
||||
assert watchdog._is_ip_address('localhost') is False
|
||||
|
||||
# Invalid IPs (rejected by both ipaddress and inet_aton)
|
||||
assert watchdog._is_ip_address('999.999.999.999') is False
|
||||
assert watchdog._is_ip_address('1.2.3.4.5') is False
|
||||
assert watchdog._is_ip_address('not-an-ip') is False
|
||||
assert watchdog._is_ip_address('') is False
|
||||
|
||||
# Short-form IPv4 strings (1.2.3 == 1.2.0.3) ARE valid IPs that
|
||||
# browsers/kernel resolve — covered by TestNonStandardIPv4Representations.
|
||||
|
||||
# IPs with ports or paths (not valid for the helper - it only checks hostnames)
|
||||
assert watchdog._is_ip_address('192.168.1.1:8080') is False
|
||||
assert watchdog._is_ip_address('192.168.1.1/path') is False
|
||||
|
||||
|
||||
class TestDefaultBehavior:
|
||||
"""Test that default behavior (no IP blocking) is maintained."""
|
||||
|
||||
def test_default_block_ip_addresses_is_false(self):
|
||||
"""Test that block_ip_addresses defaults to False."""
|
||||
browser_profile = BrowserProfile(headless=True, user_data_dir=None)
|
||||
|
||||
# Default should be False
|
||||
assert browser_profile.block_ip_addresses is False
|
||||
|
||||
def test_no_blocking_by_default(self):
|
||||
"""Test that IPs are not blocked by default."""
|
||||
browser_profile = BrowserProfile(headless=True, user_data_dir=None)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# All IPs should be allowed by default
|
||||
assert watchdog._is_url_allowed('http://180.1.1.1/supersafe.txt') is True
|
||||
assert watchdog._is_url_allowed('http://192.168.1.1/') is True
|
||||
assert watchdog._is_url_allowed('http://127.0.0.1:8080/') is True
|
||||
assert watchdog._is_url_allowed('http://[::1]/') is True
|
||||
assert watchdog._is_url_allowed('https://8.8.8.8/') is True
|
||||
|
||||
|
||||
class TestComplexScenarios:
|
||||
"""Test complex real-world scenarios."""
|
||||
|
||||
def test_mixed_configuration_comprehensive(self):
|
||||
"""Test a complex configuration with multiple security settings."""
|
||||
browser_profile = BrowserProfile(
|
||||
block_ip_addresses=True,
|
||||
allowed_domains=['example.com', '*.google.com'],
|
||||
prohibited_domains=['bad.example.com'], # Should be ignored when allowlist is set
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Allowed domains should work
|
||||
assert watchdog._is_url_allowed('https://example.com') is True
|
||||
assert watchdog._is_url_allowed('https://www.google.com') is True
|
||||
assert watchdog._is_url_allowed('https://mail.google.com') is True
|
||||
|
||||
# IPs should be blocked
|
||||
assert watchdog._is_url_allowed('http://8.8.8.8/') is False
|
||||
assert watchdog._is_url_allowed('http://192.168.1.1/') is False
|
||||
|
||||
# Domains not in allowlist should be blocked
|
||||
assert watchdog._is_url_allowed('https://other.com') is False
|
||||
|
||||
def test_localhost_development_scenario(self):
|
||||
"""Test typical local development scenario."""
|
||||
# Developer wants to block external IPs but allow domain names
|
||||
browser_profile = BrowserProfile(
|
||||
block_ip_addresses=True,
|
||||
headless=True,
|
||||
user_data_dir=None, # No domain restrictions
|
||||
)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Domain names should work (including localhost as a name)
|
||||
assert watchdog._is_url_allowed('http://localhost:3000/') is True
|
||||
assert watchdog._is_url_allowed('http://localhost:8080/api') is True
|
||||
|
||||
# But localhost IP should be blocked
|
||||
assert watchdog._is_url_allowed('http://127.0.0.1:3000/') is False
|
||||
|
||||
# External domains should work
|
||||
assert watchdog._is_url_allowed('https://api.example.com') is True
|
||||
|
||||
# External IPs should be blocked
|
||||
assert watchdog._is_url_allowed('http://8.8.8.8/') is False
|
||||
|
||||
def test_security_hardening_scenario(self):
|
||||
"""Test maximum security scenario with IP blocking and domain restrictions."""
|
||||
browser_profile = BrowserProfile(
|
||||
block_ip_addresses=True,
|
||||
allowed_domains=['example.com', 'api.example.com'],
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
)
|
||||
browser_session = BrowserSession(browser_profile=browser_profile)
|
||||
event_bus = EventBus()
|
||||
watchdog = SecurityWatchdog(browser_session=browser_session, event_bus=event_bus)
|
||||
|
||||
# Only specified domains allowed
|
||||
assert watchdog._is_url_allowed('https://example.com') is True
|
||||
assert watchdog._is_url_allowed('https://api.example.com') is True
|
||||
|
||||
# IPs blocked
|
||||
assert watchdog._is_url_allowed('http://192.168.1.1/') is False
|
||||
|
||||
# Other domains blocked
|
||||
assert watchdog._is_url_allowed('https://other.com') is False
|
||||
|
||||
# Even localhost blocked
|
||||
assert watchdog._is_url_allowed('http://127.0.0.1/') is False
|
||||
|
||||
|
||||
class TestNonStandardIPv4Representations:
|
||||
"""Regression tests for GHSA-xrfv-gg9f-wwjp / GHSA-g27c-8gp4-28cv.
|
||||
|
||||
`ipaddress.ip_address()` only accepts the canonical dotted-quad form, but
|
||||
Chromium also resolves decimal, hex, octal, and short-form IPv4 strings.
|
||||
Without canonicalization, `block_ip_addresses=True` could be bypassed via:
|
||||
|
||||
http://2130706433/ → 127.0.0.1 (decimal)
|
||||
http://0x7f000001/ → 127.0.0.1 (hex)
|
||||
http://0177.0.0.1/ → 127.0.0.1 (octal)
|
||||
http://127.1/ → 127.0.0.1 (short-form)
|
||||
http://127.0.1/ → 127.0.0.1 (short-form)
|
||||
"""
|
||||
|
||||
def _watchdog(self) -> SecurityWatchdog:
|
||||
profile = BrowserProfile(block_ip_addresses=True, headless=True, user_data_dir=None)
|
||||
session = BrowserSession(browser_profile=profile)
|
||||
return SecurityWatchdog(browser_session=session, event_bus=EventBus())
|
||||
|
||||
def test_decimal_ipv4_blocked(self):
|
||||
"""Decimal integer IPv4 strings (browser-accepted) must be blocked."""
|
||||
watchdog = self._watchdog()
|
||||
assert watchdog._is_url_allowed('http://2130706433/') is False # 127.0.0.1
|
||||
assert watchdog._is_url_allowed('http://3232235521/') is False # 192.168.0.1
|
||||
|
||||
def test_hex_ipv4_blocked(self):
|
||||
"""Hex IPv4 strings (browser-accepted) must be blocked."""
|
||||
watchdog = self._watchdog()
|
||||
assert watchdog._is_url_allowed('http://0x7f000001/') is False # 127.0.0.1
|
||||
assert watchdog._is_url_allowed('http://0x7F.0x0.0x0.0x1/') is False
|
||||
|
||||
def test_octal_ipv4_blocked(self):
|
||||
"""Octal IPv4 strings (browser-accepted) must be blocked."""
|
||||
watchdog = self._watchdog()
|
||||
assert watchdog._is_url_allowed('http://0177.0.0.1/') is False # 127.0.0.1
|
||||
|
||||
def test_short_form_ipv4_blocked(self):
|
||||
"""Short-form IPv4 strings (browser-accepted) must be blocked."""
|
||||
watchdog = self._watchdog()
|
||||
assert watchdog._is_url_allowed('http://127.1/') is False
|
||||
assert watchdog._is_url_allowed('http://127.0.1/') is False
|
||||
assert watchdog._is_url_allowed('http://10.1/') is False
|
||||
|
||||
def test_lookalike_domains_still_allowed(self):
|
||||
"""Hostnames that look IP-ish but are not (e.g. embedded labels) must
|
||||
remain unaffected — only browser-resolvable IP forms should be blocked."""
|
||||
watchdog = self._watchdog()
|
||||
# Real domains, not IPs — should not be incorrectly blocked.
|
||||
assert watchdog._is_url_allowed('http://127.0.0.1.evil.com/') is True
|
||||
assert watchdog._is_url_allowed('http://2130706433.evil.com/') is True
|
||||
assert watchdog._is_url_allowed('http://example.com/') is True
|
||||
|
||||
def test_non_standard_forms_allowed_when_blocking_disabled(self):
|
||||
"""Without block_ip_addresses, non-standard IPv4 strings are not blocked."""
|
||||
profile = BrowserProfile(block_ip_addresses=False, headless=True, user_data_dir=None)
|
||||
session = BrowserSession(browser_profile=profile)
|
||||
watchdog = SecurityWatchdog(browser_session=session, event_bus=EventBus())
|
||||
assert watchdog._is_url_allowed('http://2130706433/') is True
|
||||
assert watchdog._is_url_allowed('http://0x7f000001/') is True
|
||||
|
||||
def test_non_standard_forms_blocked_inside_allowed_domains(self):
|
||||
"""block_ip_addresses must override allowed_domains for non-standard forms
|
||||
just as it does for the canonical form."""
|
||||
profile = BrowserProfile(
|
||||
block_ip_addresses=True,
|
||||
allowed_domains=['*'],
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
)
|
||||
session = BrowserSession(browser_profile=profile)
|
||||
watchdog = SecurityWatchdog(browser_session=session, event_bus=EventBus())
|
||||
# Even with '*' allowlist, the IP block must still fire.
|
||||
assert watchdog._is_url_allowed('http://2130706433/') is False
|
||||
assert watchdog._is_url_allowed('http://0x7f000001/') is False
|
||||
# Sanity: legitimate domains still allowed.
|
||||
assert watchdog._is_url_allowed('https://example.com/') is True
|
||||
|
||||
def test_malformed_unicode_hostnames_do_not_crash_classifier(self):
|
||||
"""Hostnames with lone surrogates (e.g. `\\udcff` from URL-decoded
|
||||
malformed UTF-8) must not crash `_is_ip_address` — `socket.inet_aton`
|
||||
raises `UnicodeEncodeError` (not `OSError`) for surrogates, and the
|
||||
classifier must treat that as "not an IP" rather than propagating.
|
||||
|
||||
Pre-existing defensive behavior in the original code (caught
|
||||
`Exception`) — must be preserved when extending with `inet_aton`.
|
||||
"""
|
||||
watchdog = self._watchdog()
|
||||
# Lone surrogates — common in URLs containing percent-encoded malformed UTF-8.
|
||||
assert watchdog._is_ip_address('\udcff') is False
|
||||
assert watchdog._is_ip_address('\ud800') is False
|
||||
assert watchdog._is_ip_address('caf\udce9.local') is False
|
||||
assert watchdog._is_ip_address('\udcff.example.com') is False
|
||||
|
||||
def test_percent_encoded_ipv4_blocked(self):
|
||||
"""Percent-encoded hostnames that decode to IPs must be blocked.
|
||||
|
||||
Chromium percent-decodes the host before resolving — so
|
||||
`http://%30x7f000001/` decodes to `0x7f000001` → `127.0.0.1`, and
|
||||
`http://%31%32%37.0.0.1/` decodes to `127.0.0.1`. Without
|
||||
percent-decoding inside the classifier, the IP block is bypassed
|
||||
whenever the URL contains any `%`-encoded host bytes.
|
||||
"""
|
||||
watchdog = self._watchdog()
|
||||
# Mixed encoding: %30 = '0', rest literal → '0x7f000001'
|
||||
assert watchdog._is_url_allowed('http://%30x7f000001/') is False
|
||||
# Fully encoded canonical form: %31%32%37 = '127'
|
||||
assert watchdog._is_url_allowed('http://%31%32%37.0.0.1/') is False
|
||||
# Fully encoded decimal form: → '2130706433'
|
||||
assert watchdog._is_url_allowed('http://%32%31%33%30%37%30%36%34%33%33/') is False
|
||||
# Direct classifier checks for the same decoded forms.
|
||||
assert watchdog._is_ip_address('%30x7f000001') is True
|
||||
assert watchdog._is_ip_address('%31%32%37.0.0.1') is True
|
||||
assert watchdog._is_ip_address('%32%31%33%30%37%30%36%34%33%33') is True
|
||||
|
||||
def test_malformed_percent_encoding_does_not_crash(self):
|
||||
"""Hostnames with malformed `%` escapes must not crash the classifier."""
|
||||
watchdog = self._watchdog()
|
||||
# `unquote` leaves bad `%`-sequences as-is; we must still treat the
|
||||
# result as a non-IP rather than blowing up.
|
||||
assert watchdog._is_ip_address('%') is False
|
||||
assert watchdog._is_ip_address('%zz') is False
|
||||
assert watchdog._is_ip_address('%2') is False
|
||||
|
||||
def test_unicode_normalized_ipv4_blocked(self):
|
||||
"""Hostnames using fullwidth, circled, or other Unicode digit variants
|
||||
that NFKC/IDNA-normalize to ASCII IPv4 literals must be blocked.
|
||||
|
||||
WHATWG URL canonicalization maps `127.0.0.1` (fullwidth digits),
|
||||
`0x7f000001` (fullwidth zero + ASCII hex), and `①②⑦.⓪.⓪.①` (circled
|
||||
digits) all to `127.0.0.1`. Without NFKC normalization in the
|
||||
classifier, the new non-standard IPv4 blocking can be bypassed with
|
||||
any equivalent Unicode digit form.
|
||||
"""
|
||||
watchdog = self._watchdog()
|
||||
# Fullwidth digits (U+FF10..U+FF19) — NFKC → ASCII digits.
|
||||
assert watchdog._is_url_allowed('http://127.0.0.1/') is False
|
||||
# Fullwidth zero + ASCII hex 7f000001.
|
||||
assert watchdog._is_url_allowed('http://0x7f000001/') is False
|
||||
# Circled digits (U+2460+, U+24EA for zero).
|
||||
assert watchdog._is_url_allowed('http://①②⑦.⓪.⓪.①/') is False
|
||||
# Direct classifier checks.
|
||||
assert watchdog._is_ip_address('127.0.0.1') is True
|
||||
assert watchdog._is_ip_address('0x7f000001') is True
|
||||
assert watchdog._is_ip_address('①②⑦.⓪.⓪.①') is True
|
||||
|
||||
def test_idn_domains_not_misclassified_as_ip(self):
|
||||
"""Defense against false positives from the new normalization step:
|
||||
legitimate IDN domains (Unicode letters / punycode) MUST NOT be
|
||||
classified as IPs after NFKC."""
|
||||
watchdog = self._watchdog()
|
||||
assert watchdog._is_ip_address('café.example') is False
|
||||
assert watchdog._is_ip_address('xn--caf-dma.example') is False
|
||||
assert watchdog._is_ip_address('日本.example') is False
|
||||
assert watchdog._is_ip_address('xn--wgv71a.example') is False
|
||||
|
||||
def test_idna_dot_separators_blocked(self):
|
||||
"""Per RFC 3490 / UTS46, four code points act as label separators in
|
||||
IDNA processing — `.` (U+002E), `。` (U+3002 IDEOGRAPHIC FULL STOP),
|
||||
`.` (U+FF0E FULLWIDTH FULL STOP), `。` (U+FF61 HALFWIDTH IDEOGRAPHIC
|
||||
FULL STOP). WHATWG URL parsing maps all four to `.` before resolution,
|
||||
so `http://127。0。0。1/` etc. reach 127.0.0.1.
|
||||
|
||||
NFKC alone is insufficient — it maps U+FF0E → U+002E and U+FF61 →
|
||||
U+3002, but leaves U+3002 (the most common one) untouched. Classifier
|
||||
must additionally fold U+3002 and U+FF61 to U+002E before IP parsing.
|
||||
"""
|
||||
watchdog = self._watchdog()
|
||||
# All four dot variants must result in the IP being blocked.
|
||||
assert watchdog._is_url_allowed('http://127。0。0。1/') is False # U+3002
|
||||
assert watchdog._is_url_allowed('http://127。0。0。1/') is False # U+FF61
|
||||
assert watchdog._is_url_allowed('http://127.0.0.1/') is False # U+FF0E
|
||||
# Combined with circled-digit normalization.
|
||||
assert watchdog._is_url_allowed('http://①②⑦。⓪。⓪。①/') is False
|
||||
# Direct classifier checks.
|
||||
assert watchdog._is_ip_address('127。0。0。1') is True
|
||||
assert watchdog._is_ip_address('127。0。0。1') is True
|
||||
assert watchdog._is_ip_address('127.0.0.1') is True
|
||||
@@ -0,0 +1,100 @@
|
||||
"""Tests for MCP server allowed_domains dispatch (GHSA-vfcm-843v-w6v3).
|
||||
|
||||
The MCP `retry_with_browser_use_agent` tool used to default `allowed_domains` to
|
||||
`[]` when the client omitted the argument. That value was then forwarded to
|
||||
`BrowserProfile(allowed_domains=[])`, which `SecurityWatchdog` interprets as
|
||||
"no allowlist configured — allow every URL", silently disabling any
|
||||
admin-configured restrictions on the underlying profile.
|
||||
|
||||
The fix is to default to `None` so the admin-configured profile defaults are
|
||||
preserved when the client omits the argument.
|
||||
"""
|
||||
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
|
||||
from browser_use.mcp.server import BrowserUseServer
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def server() -> BrowserUseServer:
|
||||
return BrowserUseServer()
|
||||
|
||||
|
||||
async def test_dispatch_passes_none_when_allowed_domains_omitted(server: BrowserUseServer) -> None:
|
||||
"""Client omits `allowed_domains` → dispatcher must pass `None` to the agent runner."""
|
||||
captured: dict[str, Any] = {}
|
||||
|
||||
async def recording_stub(**kwargs: Any) -> str:
|
||||
captured.update(kwargs)
|
||||
return 'ok'
|
||||
|
||||
server._retry_with_browser_use_agent = recording_stub # type: ignore[method-assign]
|
||||
await server._execute_tool('retry_with_browser_use_agent', {'task': 'noop'})
|
||||
|
||||
assert 'allowed_domains' in captured, 'dispatcher must forward the allowed_domains kwarg'
|
||||
assert captured['allowed_domains'] is None, (
|
||||
f'Expected None to preserve admin-configured profile defaults; '
|
||||
f'got {captured["allowed_domains"]!r} which would disable SecurityWatchdog '
|
||||
f'when passed as BrowserProfile(allowed_domains=...).'
|
||||
)
|
||||
|
||||
|
||||
async def test_dispatch_forwards_explicit_list(server: BrowserUseServer) -> None:
|
||||
"""Explicit non-empty list from client must reach the agent runner unchanged."""
|
||||
captured: dict[str, Any] = {}
|
||||
|
||||
async def recording_stub(**kwargs: Any) -> str:
|
||||
captured.update(kwargs)
|
||||
return 'ok'
|
||||
|
||||
server._retry_with_browser_use_agent = recording_stub # type: ignore[method-assign]
|
||||
await server._execute_tool(
|
||||
'retry_with_browser_use_agent',
|
||||
{'task': 'noop', 'allowed_domains': ['example.test']},
|
||||
)
|
||||
assert captured['allowed_domains'] == ['example.test']
|
||||
|
||||
|
||||
async def test_explicit_empty_list_does_not_override_profile_defaults(server: BrowserUseServer) -> None:
|
||||
"""Defense in depth: even when a client explicitly sends `allowed_domains=[]`,
|
||||
`_retry_with_browser_use_agent` must NOT wipe profile defaults — empty list is
|
||||
semantically equivalent to "no override" for the security boundary."""
|
||||
# Patch the boundary just below the override decision: the BrowserProfile constructor.
|
||||
# We capture the resolved allowed_domains and short-circuit before LLM/agent setup.
|
||||
captured_profile_kwargs: dict[str, Any] = {}
|
||||
|
||||
from browser_use.mcp import server as server_module
|
||||
|
||||
original_browser_profile = server_module.BrowserProfile
|
||||
|
||||
class CapturingBrowserProfile:
|
||||
def __init__(self, **kwargs: Any) -> None:
|
||||
captured_profile_kwargs.update(kwargs)
|
||||
raise _StopAgentSetup('captured')
|
||||
|
||||
server_module.BrowserProfile = CapturingBrowserProfile # type: ignore[misc]
|
||||
|
||||
# Stub out the LLM construction path so we don't need real credentials —
|
||||
# the override decision happens before BrowserProfile() is called regardless,
|
||||
# but we need _retry_with_browser_use_agent to reach that point.
|
||||
try:
|
||||
with pytest.raises(_StopAgentSetup):
|
||||
await server._retry_with_browser_use_agent(
|
||||
task='noop',
|
||||
allowed_domains=[],
|
||||
)
|
||||
finally:
|
||||
server_module.BrowserProfile = original_browser_profile # type: ignore[misc]
|
||||
|
||||
# The profile dict should NOT have allowed_domains overridden to [] —
|
||||
# either the key is absent (profile defaults apply) or it carries whatever
|
||||
# the admin-configured default was. Either way: not [].
|
||||
assert captured_profile_kwargs.get('allowed_domains') != [], (
|
||||
'Empty list from client wiped profile defaults — SecurityWatchdog would be disabled.'
|
||||
)
|
||||
|
||||
|
||||
class _StopAgentSetup(Exception):
|
||||
"""Marker exception used to short-circuit agent construction during testing."""
|
||||
@@ -0,0 +1,83 @@
|
||||
"""Test that disable_security flag properly merges --disable-features flags without breaking extensions."""
|
||||
|
||||
import tempfile
|
||||
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
|
||||
|
||||
class TestBrowserProfileDisableSecurity:
|
||||
"""Test disable_security flag behavior."""
|
||||
|
||||
def test_disable_security_preserves_extension_features(self):
|
||||
"""Test that disable_security=True doesn't break extension features by properly merging --disable-features flags."""
|
||||
|
||||
# Test with disable_security=False (baseline)
|
||||
profile_normal = BrowserProfile(disable_security=False, user_data_dir=tempfile.mkdtemp(prefix='test-normal-'))
|
||||
profile_normal.detect_display_configuration()
|
||||
args_normal = profile_normal.get_args()
|
||||
|
||||
# Test with disable_security=True
|
||||
profile_security_disabled = BrowserProfile(disable_security=True, user_data_dir=tempfile.mkdtemp(prefix='test-security-'))
|
||||
profile_security_disabled.detect_display_configuration()
|
||||
args_security_disabled = profile_security_disabled.get_args()
|
||||
|
||||
# Extract disable-features args
|
||||
def extract_disable_features(args):
|
||||
for arg in args:
|
||||
if arg.startswith('--disable-features='):
|
||||
return set(arg.split('=', 1)[1].split(','))
|
||||
return set()
|
||||
|
||||
features_normal = extract_disable_features(args_normal)
|
||||
features_security_disabled = extract_disable_features(args_security_disabled)
|
||||
|
||||
# Check that extension-related features are preserved
|
||||
extension_features = {
|
||||
'ExtensionManifestV2Disabled',
|
||||
'ExtensionDisableUnsupportedDeveloper',
|
||||
'ExtensionManifestV2Unsupported',
|
||||
}
|
||||
|
||||
security_features = {'IsolateOrigins', 'site-per-process'}
|
||||
|
||||
# Verify that security disabled has both extension and security features
|
||||
missing_extension_features = extension_features - features_security_disabled
|
||||
missing_security_features = security_features - features_security_disabled
|
||||
|
||||
assert not missing_extension_features, (
|
||||
f'Missing extension features when disable_security=True: {missing_extension_features}'
|
||||
)
|
||||
assert not missing_security_features, f'Missing security features when disable_security=True: {missing_security_features}'
|
||||
|
||||
# Verify that security disabled profile has more features than normal (due to added security features)
|
||||
assert len(features_security_disabled) > len(features_normal), (
|
||||
'Security disabled profile should have more features than normal profile'
|
||||
)
|
||||
|
||||
# Verify all normal features are preserved in security disabled profile
|
||||
missing_normal_features = features_normal - features_security_disabled
|
||||
assert not missing_normal_features, f'Normal features missing from security disabled profile: {missing_normal_features}'
|
||||
|
||||
def test_disable_features_flag_deduplication(self):
|
||||
"""Test that duplicate --disable-features values are properly deduplicated."""
|
||||
|
||||
profile = BrowserProfile(
|
||||
disable_security=True,
|
||||
user_data_dir=tempfile.mkdtemp(prefix='test-dedup-'),
|
||||
# Add duplicate features to test deduplication
|
||||
args=['--disable-features=TestFeature1,TestFeature2', '--disable-features=TestFeature2,TestFeature3'],
|
||||
)
|
||||
profile.detect_display_configuration()
|
||||
args = profile.get_args()
|
||||
|
||||
# Extract disable-features args
|
||||
disable_features_args = [arg for arg in args if arg.startswith('--disable-features=')]
|
||||
|
||||
# Should only have one consolidated --disable-features flag
|
||||
assert len(disable_features_args) == 1, f'Expected 1 disable-features flag, got {len(disable_features_args)}'
|
||||
|
||||
features = set(disable_features_args[0].split('=', 1)[1].split(','))
|
||||
|
||||
# Should have all test features without duplicates
|
||||
expected_test_features = {'TestFeature1', 'TestFeature2', 'TestFeature3'}
|
||||
assert expected_test_features.issubset(features), f'Missing test features: {expected_test_features - features}'
|
||||
@@ -0,0 +1,590 @@
|
||||
import pytest
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from browser_use.agent.message_manager.service import MessageManager
|
||||
from browser_use.agent.views import ActionResult, AgentOutput, AgentStepInfo, MessageManagerState
|
||||
from browser_use.browser.views import BrowserStateSummary
|
||||
from browser_use.dom.views import SerializedDOMState
|
||||
from browser_use.filesystem.file_system import FileSystem
|
||||
from browser_use.llm import SystemMessage, UserMessage
|
||||
from browser_use.llm.messages import ContentPartTextParam
|
||||
from browser_use.tools.registry.service import Registry
|
||||
from browser_use.utils import is_new_tab_page, match_url_with_domain_pattern
|
||||
|
||||
|
||||
class SensitiveParams(BaseModel):
|
||||
"""Test parameter model for sensitive data testing."""
|
||||
|
||||
text: str = Field(description='Text with sensitive data placeholders')
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def registry():
|
||||
return Registry()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def message_manager():
|
||||
import os
|
||||
import tempfile
|
||||
import uuid
|
||||
|
||||
base_tmp = tempfile.gettempdir() # e.g., /tmp on Unix
|
||||
file_system_path = os.path.join(base_tmp, str(uuid.uuid4()))
|
||||
return MessageManager(
|
||||
task='Test task',
|
||||
system_message=SystemMessage(content='System message'),
|
||||
state=MessageManagerState(),
|
||||
file_system=FileSystem(file_system_path),
|
||||
)
|
||||
|
||||
|
||||
def test_replace_sensitive_data_with_missing_keys(registry, caplog):
|
||||
"""Test that _replace_sensitive_data handles missing keys gracefully"""
|
||||
# Create a simple Pydantic model with sensitive data placeholders
|
||||
params = SensitiveParams(text='Please enter <secret>username</secret> and <secret>password</secret>')
|
||||
|
||||
# Case 1: All keys present - both placeholders should be replaced
|
||||
sensitive_data = {'username': 'user123', 'password': 'pass456'}
|
||||
result = registry._replace_sensitive_data(params, sensitive_data)
|
||||
assert result.text == 'Please enter user123 and pass456'
|
||||
assert '<secret>' not in result.text # No secret tags should remain
|
||||
|
||||
# Case 2: One key missing - only available key should be replaced
|
||||
sensitive_data = {'username': 'user123'} # password is missing
|
||||
result = registry._replace_sensitive_data(params, sensitive_data)
|
||||
assert result.text == 'Please enter user123 and <secret>password</secret>'
|
||||
assert 'user123' in result.text
|
||||
assert '<secret>password</secret>' in result.text # Missing key's tag remains
|
||||
|
||||
# Case 3: Multiple keys missing - all tags should be preserved
|
||||
sensitive_data = {} # both keys missing
|
||||
result = registry._replace_sensitive_data(params, sensitive_data)
|
||||
assert result.text == 'Please enter <secret>username</secret> and <secret>password</secret>'
|
||||
assert '<secret>username</secret>' in result.text
|
||||
assert '<secret>password</secret>' in result.text
|
||||
|
||||
# Case 4: One key empty - empty values are treated as missing
|
||||
sensitive_data = {'username': 'user123', 'password': ''}
|
||||
result = registry._replace_sensitive_data(params, sensitive_data)
|
||||
assert result.text == 'Please enter user123 and <secret>password</secret>'
|
||||
assert 'user123' in result.text
|
||||
assert '<secret>password</secret>' in result.text # Empty value's tag remains
|
||||
|
||||
|
||||
def test_simple_domain_specific_sensitive_data(registry, caplog):
|
||||
"""Test the basic functionality of domain-specific sensitive data replacement"""
|
||||
# Create a simple Pydantic model with sensitive data placeholders
|
||||
params = SensitiveParams(text='Please enter <secret>username</secret> and <secret>password</secret>')
|
||||
|
||||
# Simple test with directly instantiable values
|
||||
sensitive_data = {
|
||||
'example.com': {'username': 'example_user'},
|
||||
'other_data': 'non_secret_value', # Old format mixed with new
|
||||
}
|
||||
|
||||
# Without a URL, domain-specific secrets should NOT be exposed
|
||||
result = registry._replace_sensitive_data(params, sensitive_data)
|
||||
assert result.text == 'Please enter <secret>username</secret> and <secret>password</secret>'
|
||||
assert '<secret>username</secret>' in result.text # Should NOT be replaced without URL
|
||||
assert '<secret>password</secret>' in result.text # Password is missing in sensitive_data
|
||||
assert 'example_user' not in result.text # Domain-specific value should not appear
|
||||
|
||||
# Test with a matching URL - domain-specific secrets should be exposed
|
||||
result = registry._replace_sensitive_data(params, sensitive_data, 'https://example.com/login')
|
||||
assert result.text == 'Please enter example_user and <secret>password</secret>'
|
||||
assert 'example_user' in result.text # Should be replaced with matching URL
|
||||
assert '<secret>password</secret>' in result.text # Password is still missing
|
||||
assert '<secret>username</secret>' not in result.text # Username tag should be replaced
|
||||
|
||||
|
||||
def test_match_url_with_domain_pattern():
|
||||
"""Test that the domain pattern matching utility works correctly"""
|
||||
|
||||
# Test exact domain matches
|
||||
assert match_url_with_domain_pattern('https://example.com', 'example.com') is True
|
||||
assert match_url_with_domain_pattern('http://example.com', 'example.com') is False # Default scheme is now https
|
||||
assert match_url_with_domain_pattern('https://google.com', 'example.com') is False
|
||||
|
||||
# Test subdomain pattern matches
|
||||
assert match_url_with_domain_pattern('https://sub.example.com', '*.example.com') is True
|
||||
assert match_url_with_domain_pattern('https://example.com', '*.example.com') is True # Base domain should match too
|
||||
assert match_url_with_domain_pattern('https://sub.sub.example.com', '*.example.com') is True
|
||||
assert match_url_with_domain_pattern('https://example.org', '*.example.com') is False
|
||||
|
||||
# Test protocol pattern matches
|
||||
assert match_url_with_domain_pattern('https://example.com', 'http*://example.com') is True
|
||||
assert match_url_with_domain_pattern('http://example.com', 'http*://example.com') is True
|
||||
assert match_url_with_domain_pattern('ftp://example.com', 'http*://example.com') is False
|
||||
|
||||
# Test explicit http protocol
|
||||
assert match_url_with_domain_pattern('http://example.com', 'http://example.com') is True
|
||||
assert match_url_with_domain_pattern('https://example.com', 'http://example.com') is False
|
||||
|
||||
# Test Chrome extension pattern
|
||||
assert match_url_with_domain_pattern('chrome-extension://abcdefghijkl', 'chrome-extension://*') is True
|
||||
assert match_url_with_domain_pattern('chrome-extension://mnopqrstuvwx', 'chrome-extension://abcdefghijkl') is False
|
||||
|
||||
# Test new tab page handling
|
||||
assert match_url_with_domain_pattern('about:blank', 'example.com') is False
|
||||
assert match_url_with_domain_pattern('about:blank', '*://*') is False
|
||||
assert match_url_with_domain_pattern('chrome://new-tab-page/', 'example.com') is False
|
||||
assert match_url_with_domain_pattern('chrome://new-tab-page/', '*://*') is False
|
||||
assert match_url_with_domain_pattern('chrome://new-tab-page', 'example.com') is False
|
||||
assert match_url_with_domain_pattern('chrome://new-tab-page', '*://*') is False
|
||||
|
||||
|
||||
def test_unsafe_domain_patterns():
|
||||
"""Test that unsafe domain patterns are rejected"""
|
||||
|
||||
# These are unsafe patterns that could match too many domains
|
||||
assert match_url_with_domain_pattern('https://evil.com', '*google.com') is False
|
||||
assert match_url_with_domain_pattern('https://google.com.evil.com', '*.*.com') is False
|
||||
assert match_url_with_domain_pattern('https://google.com', '**google.com') is False
|
||||
assert match_url_with_domain_pattern('https://google.com', 'g*e.com') is False
|
||||
assert match_url_with_domain_pattern('https://google.com', '*com*') is False
|
||||
|
||||
# Test with patterns that have multiple asterisks in different positions
|
||||
assert match_url_with_domain_pattern('https://subdomain.example.com', '*domain*example*') is False
|
||||
assert match_url_with_domain_pattern('https://sub.domain.example.com', '*.*.example.com') is False
|
||||
|
||||
# Test patterns with wildcards in TLD part
|
||||
assert match_url_with_domain_pattern('https://example.com', 'example.*') is False
|
||||
assert match_url_with_domain_pattern('https://example.org', 'example.*') is False
|
||||
|
||||
|
||||
def test_malformed_urls_and_patterns():
|
||||
"""Test handling of malformed URLs or patterns"""
|
||||
|
||||
# Malformed URLs
|
||||
assert match_url_with_domain_pattern('not-a-url', 'example.com') is False
|
||||
assert match_url_with_domain_pattern('http://', 'example.com') is False
|
||||
assert match_url_with_domain_pattern('https://', 'example.com') is False
|
||||
assert match_url_with_domain_pattern('ftp:/example.com', 'example.com') is False # Missing slash
|
||||
|
||||
# Empty URLs or patterns
|
||||
assert match_url_with_domain_pattern('', 'example.com') is False
|
||||
assert match_url_with_domain_pattern('https://example.com', '') is False
|
||||
|
||||
# URLs with no hostname
|
||||
assert match_url_with_domain_pattern('file:///path/to/file.txt', 'example.com') is False
|
||||
|
||||
# Invalid pattern formats
|
||||
assert match_url_with_domain_pattern('https://example.com', '..example.com') is False
|
||||
assert match_url_with_domain_pattern('https://example.com', '.*.example.com') is False
|
||||
assert match_url_with_domain_pattern('https://example.com', '**') is False
|
||||
|
||||
# Nested URL attacks in path, query or fragments
|
||||
assert match_url_with_domain_pattern('https://example.com/redirect?url=https://evil.com', 'example.com') is True
|
||||
assert match_url_with_domain_pattern('https://example.com/path/https://evil.com', 'example.com') is True
|
||||
assert match_url_with_domain_pattern('https://example.com#https://evil.com', 'example.com') is True
|
||||
# These should match example.com, not evil.com since urlparse extracts the hostname correctly
|
||||
|
||||
# Complex URL obfuscation attempts
|
||||
assert match_url_with_domain_pattern('https://example.com/path?next=//evil.com/attack', 'example.com') is True
|
||||
assert match_url_with_domain_pattern('https://example.com@evil.com', 'example.com') is False
|
||||
assert match_url_with_domain_pattern('https://evil.com?example.com', 'example.com') is False
|
||||
assert match_url_with_domain_pattern('https://user:example.com@evil.com', 'example.com') is False
|
||||
# urlparse correctly identifies evil.com as the hostname in these cases
|
||||
|
||||
|
||||
def test_url_components():
|
||||
"""Test handling of URL components like credentials, ports, fragments, etc."""
|
||||
|
||||
# URLs with credentials (username:password@)
|
||||
assert match_url_with_domain_pattern('https://user:pass@example.com', 'example.com') is True
|
||||
assert match_url_with_domain_pattern('https://user:pass@example.com', '*.example.com') is True
|
||||
|
||||
# URLs with ports
|
||||
assert match_url_with_domain_pattern('https://example.com:8080', 'example.com') is True
|
||||
assert match_url_with_domain_pattern('https://example.com:8080', 'example.com:8080') is True # Port is stripped from pattern
|
||||
|
||||
# URLs with paths
|
||||
assert match_url_with_domain_pattern('https://example.com/path/to/page', 'example.com') is True
|
||||
assert (
|
||||
match_url_with_domain_pattern('https://example.com/path/to/page', 'example.com/path') is False
|
||||
) # Paths in patterns are not supported
|
||||
|
||||
# URLs with query parameters
|
||||
assert match_url_with_domain_pattern('https://example.com?param=value', 'example.com') is True
|
||||
|
||||
# URLs with fragments
|
||||
assert match_url_with_domain_pattern('https://example.com#section', 'example.com') is True
|
||||
|
||||
# URLs with all components
|
||||
assert match_url_with_domain_pattern('https://user:pass@example.com:8080/path?query=val#fragment', 'example.com') is True
|
||||
|
||||
|
||||
def test_filter_sensitive_data(message_manager):
|
||||
"""Test that _filter_sensitive_data handles all sensitive data scenarios correctly"""
|
||||
# Set up a message with sensitive information
|
||||
message = UserMessage(content='My username is admin and password is secret123')
|
||||
|
||||
# Case 1: No sensitive data provided
|
||||
message_manager.sensitive_data = None
|
||||
result = message_manager._filter_sensitive_data(message)
|
||||
assert result.content == 'My username is admin and password is secret123'
|
||||
|
||||
# Case 2: All sensitive data is properly replaced
|
||||
message_manager.sensitive_data = {'username': 'admin', 'password': 'secret123'}
|
||||
result = message_manager._filter_sensitive_data(message)
|
||||
assert '<secret>username</secret>' in result.content
|
||||
assert '<secret>password</secret>' in result.content
|
||||
|
||||
# Case 3: Make sure it works with nested content
|
||||
nested_message = UserMessage(content=[ContentPartTextParam(text='My username is admin and password is secret123')])
|
||||
result = message_manager._filter_sensitive_data(nested_message)
|
||||
assert '<secret>username</secret>' in result.content[0].text
|
||||
assert '<secret>password</secret>' in result.content[0].text
|
||||
|
||||
# Case 4: Test with empty values
|
||||
message_manager.sensitive_data = {'username': 'admin', 'password': ''}
|
||||
result = message_manager._filter_sensitive_data(message)
|
||||
assert '<secret>username</secret>' in result.content
|
||||
# Only username should be replaced since password is empty
|
||||
|
||||
# Case 5: Test with domain-specific sensitive data format
|
||||
message_manager.sensitive_data = {
|
||||
'example.com': {'username': 'admin', 'password': 'secret123'},
|
||||
'google.com': {'email': 'user@example.com', 'password': 'google_pass'},
|
||||
}
|
||||
# Update the message to include the values we're going to test
|
||||
message = UserMessage(content='My username is admin, email is user@example.com and password is secret123 or google_pass')
|
||||
result = message_manager._filter_sensitive_data(message)
|
||||
# All sensitive values should be replaced regardless of domain
|
||||
assert '<secret>username</secret>' in result.content
|
||||
assert '<secret>password</secret>' in result.content
|
||||
assert '<secret>email</secret>' in result.content
|
||||
|
||||
|
||||
def test_is_new_tab_page():
|
||||
"""Test is_new_tab_page function"""
|
||||
# Test about:blank
|
||||
assert is_new_tab_page('about:blank') is True
|
||||
|
||||
# Test chrome://new-tab-page variations
|
||||
assert is_new_tab_page('chrome://new-tab-page/') is True
|
||||
assert is_new_tab_page('chrome://new-tab-page') is True
|
||||
|
||||
# Test regular URLs
|
||||
assert is_new_tab_page('https://example.com') is False
|
||||
assert is_new_tab_page('http://google.com') is False
|
||||
assert is_new_tab_page('') is False
|
||||
assert is_new_tab_page('chrome://settings') is False
|
||||
|
||||
|
||||
def test_sensitive_data_filtered_from_action_results():
|
||||
"""
|
||||
Test that sensitive data in action results is filtered before being sent to the LLM.
|
||||
|
||||
This tests the full flow:
|
||||
1. Agent outputs actions with <secret>password</secret> placeholder
|
||||
2. Placeholder gets replaced with real value 'secret_pass123' during action execution
|
||||
3. Action result contains: "Typed 'secret_pass123' into password field"
|
||||
4. When state messages are created, the real value should be replaced back to placeholder
|
||||
5. The LLM should never see the real password value
|
||||
"""
|
||||
import os
|
||||
import tempfile
|
||||
import uuid
|
||||
|
||||
base_tmp = tempfile.gettempdir()
|
||||
file_system_path = os.path.join(base_tmp, str(uuid.uuid4()))
|
||||
|
||||
sensitive_data: dict[str, str | dict[str, str]] = {'username': 'admin_user', 'password': 'secret_pass123'}
|
||||
|
||||
message_manager = MessageManager(
|
||||
task='Login to the website',
|
||||
system_message=SystemMessage(content='You are a browser automation agent'),
|
||||
state=MessageManagerState(),
|
||||
file_system=FileSystem(file_system_path),
|
||||
sensitive_data=sensitive_data,
|
||||
)
|
||||
|
||||
# Create browser state
|
||||
dom_state = SerializedDOMState(_root=None, selector_map={})
|
||||
browser_state = BrowserStateSummary(
|
||||
dom_state=dom_state,
|
||||
url='https://example.com/login',
|
||||
title='Login Page',
|
||||
tabs=[],
|
||||
)
|
||||
|
||||
# Simulate action result containing sensitive data after placeholder replacement
|
||||
# This represents what happens after typing a password into a form field
|
||||
action_results = [
|
||||
ActionResult(
|
||||
long_term_memory="Successfully typed 'secret_pass123' into the password field",
|
||||
error=None,
|
||||
)
|
||||
]
|
||||
|
||||
# Create model output for step 1
|
||||
model_output = AgentOutput(
|
||||
evaluation_previous_goal='Navigated to login page',
|
||||
memory='On login page, need to enter credentials',
|
||||
next_goal='Submit login form',
|
||||
action=[],
|
||||
)
|
||||
|
||||
step_info = AgentStepInfo(step_number=1, max_steps=10)
|
||||
|
||||
# Create state messages - this should filter sensitive data
|
||||
message_manager.create_state_messages(
|
||||
browser_state_summary=browser_state,
|
||||
model_output=model_output,
|
||||
result=action_results,
|
||||
step_info=step_info,
|
||||
use_vision=False,
|
||||
)
|
||||
|
||||
# Get messages that would be sent to LLM
|
||||
messages = message_manager.get_messages()
|
||||
|
||||
# Extract all text content from messages
|
||||
all_text = []
|
||||
for msg in messages:
|
||||
if isinstance(msg.content, str):
|
||||
all_text.append(msg.content)
|
||||
elif isinstance(msg.content, list):
|
||||
for part in msg.content:
|
||||
if isinstance(part, ContentPartTextParam):
|
||||
all_text.append(part.text)
|
||||
|
||||
combined_text = '\n'.join(all_text)
|
||||
|
||||
# Verify the bug is fixed: plaintext password should NOT appear in messages
|
||||
assert 'secret_pass123' not in combined_text, (
|
||||
'Sensitive data leaked! Real password value found in LLM messages. '
|
||||
'The _filter_sensitive_data method should replace it with <secret>password</secret>'
|
||||
)
|
||||
|
||||
# Verify the filtered placeholder IS present (proves filtering happened)
|
||||
assert '<secret>password</secret>' in combined_text, (
|
||||
'Filtering did not work correctly. Expected <secret>password</secret> placeholder in messages.'
|
||||
)
|
||||
|
||||
|
||||
def test_sensitive_data_filtered_with_domain_specific_format():
|
||||
"""Test that domain-specific sensitive data format is also filtered from action results."""
|
||||
import os
|
||||
import tempfile
|
||||
import uuid
|
||||
|
||||
base_tmp = tempfile.gettempdir()
|
||||
file_system_path = os.path.join(base_tmp, str(uuid.uuid4()))
|
||||
|
||||
# Use domain-specific format
|
||||
sensitive_data: dict[str, str | dict[str, str]] = {
|
||||
'example.com': {'api_key': 'sk-secret-api-key-12345'},
|
||||
}
|
||||
|
||||
message_manager = MessageManager(
|
||||
task='Use the API',
|
||||
system_message=SystemMessage(content='You are a browser automation agent'),
|
||||
state=MessageManagerState(),
|
||||
file_system=FileSystem(file_system_path),
|
||||
sensitive_data=sensitive_data,
|
||||
)
|
||||
|
||||
dom_state = SerializedDOMState(_root=None, selector_map={})
|
||||
browser_state = BrowserStateSummary(
|
||||
dom_state=dom_state,
|
||||
url='https://example.com/api',
|
||||
title='API Page',
|
||||
tabs=[],
|
||||
)
|
||||
|
||||
# Action result with API key that should be filtered
|
||||
action_results = [
|
||||
ActionResult(
|
||||
long_term_memory="Set API key to 'sk-secret-api-key-12345' in the input field",
|
||||
error=None,
|
||||
)
|
||||
]
|
||||
|
||||
model_output = AgentOutput(
|
||||
evaluation_previous_goal='Opened API settings',
|
||||
memory='Need to configure API key',
|
||||
next_goal='Save settings',
|
||||
action=[],
|
||||
)
|
||||
|
||||
step_info = AgentStepInfo(step_number=1, max_steps=10)
|
||||
|
||||
message_manager.create_state_messages(
|
||||
browser_state_summary=browser_state,
|
||||
model_output=model_output,
|
||||
result=action_results,
|
||||
step_info=step_info,
|
||||
use_vision=False,
|
||||
)
|
||||
|
||||
messages = message_manager.get_messages()
|
||||
|
||||
all_text = []
|
||||
for msg in messages:
|
||||
if isinstance(msg.content, str):
|
||||
all_text.append(msg.content)
|
||||
elif isinstance(msg.content, list):
|
||||
for part in msg.content:
|
||||
if isinstance(part, ContentPartTextParam):
|
||||
all_text.append(part.text)
|
||||
|
||||
combined_text = '\n'.join(all_text)
|
||||
|
||||
# API key should be filtered out
|
||||
assert 'sk-secret-api-key-12345' not in combined_text, 'API key leaked into LLM messages!'
|
||||
assert '<secret>api_key</secret>' in combined_text, 'API key placeholder not found in messages'
|
||||
|
||||
|
||||
# ─── Tests for password field value redaction in DOM snapshots ────────────────
|
||||
|
||||
|
||||
def _make_dom_node(
|
||||
tag_name: str,
|
||||
attributes: dict[str, str],
|
||||
ax_value: str | None = None,
|
||||
):
|
||||
"""Create a minimal EnhancedDOMTreeNode for serializer testing."""
|
||||
from browser_use.dom.views import (
|
||||
EnhancedAXNode,
|
||||
EnhancedAXProperty,
|
||||
EnhancedDOMTreeNode,
|
||||
NodeType,
|
||||
)
|
||||
|
||||
ax_node = None
|
||||
if ax_value is not None:
|
||||
ax_node = EnhancedAXNode(
|
||||
ax_node_id='ax-1',
|
||||
ignored=False,
|
||||
role='textbox',
|
||||
name=None,
|
||||
description=None,
|
||||
properties=[
|
||||
EnhancedAXProperty(name='valuetext', value=ax_value),
|
||||
],
|
||||
child_ids=None,
|
||||
)
|
||||
|
||||
return EnhancedDOMTreeNode(
|
||||
node_id=1,
|
||||
backend_node_id=1,
|
||||
node_type=NodeType.ELEMENT_NODE,
|
||||
node_name=tag_name.upper(),
|
||||
node_value='',
|
||||
attributes=attributes,
|
||||
is_scrollable=None,
|
||||
is_visible=True,
|
||||
absolute_position=None,
|
||||
target_id='target-1',
|
||||
frame_id=None,
|
||||
session_id=None,
|
||||
content_document=None,
|
||||
shadow_root_type=None,
|
||||
shadow_roots=None,
|
||||
parent_node=None,
|
||||
children_nodes=None,
|
||||
ax_node=ax_node,
|
||||
snapshot_node=None,
|
||||
)
|
||||
|
||||
|
||||
def test_password_field_value_excluded_from_dom_snapshot():
|
||||
"""
|
||||
Password field values must never appear in DOM snapshots sent to the LLM.
|
||||
|
||||
When a user types a password into <input type="password">, the accessibility tree
|
||||
stores the real value. The serializer extracts AX tree values for all form elements
|
||||
to show the LLM what's been typed. Without filtering, the password appears in the
|
||||
DOM text representation sent to the LLM on every subsequent step.
|
||||
|
||||
Attack scenario:
|
||||
1. Agent types password into <input type="password"> via sensitive_data placeholder
|
||||
2. Next step: DOM snapshot extracts the typed value from the AX tree
|
||||
3. Password appears in plaintext in the LLM context
|
||||
4. Prompt injection on a later page can exfiltrate it
|
||||
"""
|
||||
from browser_use.dom.serializer.serializer import DOMTreeSerializer
|
||||
from browser_use.dom.views import DEFAULT_INCLUDE_ATTRIBUTES
|
||||
|
||||
secret_password = 'hubble_space_telescope'
|
||||
|
||||
node = _make_dom_node(
|
||||
tag_name='input',
|
||||
attributes={'type': 'password', 'name': 'password', 'id': 'pw-field'},
|
||||
ax_value=secret_password,
|
||||
)
|
||||
|
||||
attrs_str = DOMTreeSerializer._build_attributes_string(node, list(DEFAULT_INCLUDE_ATTRIBUTES), '')
|
||||
|
||||
assert secret_password not in attrs_str, (
|
||||
f'Password "{secret_password}" leaked into DOM serialization! '
|
||||
'Password field values must be excluded from DOM snapshots sent to the LLM.'
|
||||
)
|
||||
# The type attribute should still be present so the LLM knows it's a password field
|
||||
assert 'type=password' in attrs_str, 'Password field type attribute should be preserved'
|
||||
|
||||
|
||||
def test_password_field_value_excluded_even_from_html_attributes():
|
||||
"""
|
||||
Even if the DOM attribute 'value' is set (e.g. <input type="password" value="preset">),
|
||||
the serializer must strip it for password fields.
|
||||
"""
|
||||
from browser_use.dom.serializer.serializer import DOMTreeSerializer
|
||||
from browser_use.dom.views import DEFAULT_INCLUDE_ATTRIBUTES
|
||||
|
||||
preset_password = 'hubble_space_telescope'
|
||||
|
||||
node = _make_dom_node(
|
||||
tag_name='input',
|
||||
attributes={'type': 'password', 'name': 'password', 'value': preset_password},
|
||||
ax_value=None, # no AX value, but HTML attribute has it
|
||||
)
|
||||
|
||||
attrs_str = DOMTreeSerializer._build_attributes_string(node, list(DEFAULT_INCLUDE_ATTRIBUTES), '')
|
||||
|
||||
assert preset_password not in attrs_str, (
|
||||
f'Preset password "{preset_password}" leaked via HTML value attribute! '
|
||||
'Password field values must be stripped regardless of source.'
|
||||
)
|
||||
|
||||
|
||||
def test_text_input_value_preserved():
|
||||
"""Non-password input values should still be included (backward compatibility)."""
|
||||
from browser_use.dom.serializer.serializer import DOMTreeSerializer
|
||||
from browser_use.dom.views import DEFAULT_INCLUDE_ATTRIBUTES
|
||||
|
||||
username = 'john.doe@example.com'
|
||||
|
||||
node = _make_dom_node(
|
||||
tag_name='input',
|
||||
attributes={'type': 'text', 'name': 'username'},
|
||||
ax_value=username,
|
||||
)
|
||||
|
||||
attrs_str = DOMTreeSerializer._build_attributes_string(node, list(DEFAULT_INCLUDE_ATTRIBUTES), '')
|
||||
|
||||
assert username in attrs_str, 'Non-password input values should be preserved in DOM snapshots'
|
||||
|
||||
|
||||
def test_password_field_without_type_attribute():
|
||||
"""
|
||||
An input without an explicit type attribute defaults to 'text' — its value
|
||||
should NOT be stripped. Only explicit type="password" fields are protected.
|
||||
"""
|
||||
from browser_use.dom.serializer.serializer import DOMTreeSerializer
|
||||
from browser_use.dom.views import DEFAULT_INCLUDE_ATTRIBUTES
|
||||
|
||||
value = 'some_text_value'
|
||||
|
||||
node = _make_dom_node(
|
||||
tag_name='input',
|
||||
attributes={'name': 'search'},
|
||||
ax_value=value,
|
||||
)
|
||||
|
||||
attrs_str = DOMTreeSerializer._build_attributes_string(node, list(DEFAULT_INCLUDE_ATTRIBUTES), '')
|
||||
|
||||
assert value in attrs_str, 'Input without type attribute should preserve its value'
|
||||
@@ -0,0 +1,196 @@
|
||||
"""Tests for upload_file FileSystem path containment (GHSA-j9hj-92j8-jv9h).
|
||||
|
||||
The `upload_file` action used to construct the absolute upload path by joining
|
||||
`file_system.get_dir()` with the agent-controlled `params.path`. Because
|
||||
`FileSystem.get_file()` matches by basename (it does `os.path.basename` first),
|
||||
an agent-controlled path like `../note.md` would:
|
||||
|
||||
1. Pass `get_file()` lookup if a file named `note.md` exists in the FileSystem.
|
||||
2. Be naively joined to `data_dir` producing `data_dir/../note.md`, which
|
||||
resolves *outside* the FileSystem directory.
|
||||
3. Be uploaded to the browser as the resolved escaped file.
|
||||
|
||||
The fix uses `file_obj.full_name` (the FileSystem-owned basename) for the join
|
||||
and additionally verifies via `os.path.realpath` that the result is contained
|
||||
in `data_dir`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
|
||||
from browser_use.agent.views import ActionResult
|
||||
from browser_use.filesystem.file_system import FileSystem
|
||||
from browser_use.tools.service import Tools
|
||||
|
||||
|
||||
class _StubBrowserSession:
|
||||
"""Minimal stub exposing only the attributes `upload_file` touches before it
|
||||
would hand off to CDP. We stop the action just after path resolution.
|
||||
|
||||
Per the project rule "use real objects", this stub stands in only for the
|
||||
browser/CDP boundary that has no bearing on the security property under test
|
||||
(file path containment); the FileSystem and Tools registry are real.
|
||||
"""
|
||||
|
||||
is_local = True
|
||||
downloaded_files: list[str] = []
|
||||
agent_focus_target_id: str | None = None
|
||||
session_manager: Any = None
|
||||
cdp_client: Any = None
|
||||
|
||||
# Capture the params.path passed in by the action so the test can inspect it.
|
||||
captured_resolved_path: str | None = None
|
||||
|
||||
async def get_current_page_url(self) -> str:
|
||||
return 'about:blank'
|
||||
|
||||
async def get_selector_map(self) -> dict:
|
||||
# Returning an empty map causes upload_file to bail with an
|
||||
# "Element ... does not exist" error AFTER it has resolved the path.
|
||||
return {}
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def stub_session() -> _StubBrowserSession:
|
||||
return _StubBrowserSession()
|
||||
|
||||
|
||||
async def test_traversal_in_agent_path_does_not_escape_filesystem_dir(
|
||||
tmp_path,
|
||||
stub_session: _StubBrowserSession,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""GHSA-j9hj-92j8-jv9h: agent-controlled `../note.md` must not cause
|
||||
the upload path to escape `file_system.get_dir()`."""
|
||||
fs = FileSystem(base_dir=tmp_path)
|
||||
# Legitimate FileSystem-owned file.
|
||||
await fs.write_file('note.md', 'safe content')
|
||||
data_dir_real = os.path.realpath(str(fs.get_dir()))
|
||||
|
||||
# Capture every path that the action checks for existence — the first one
|
||||
# is the resolved upload path. Pre-fix this is the escape; post-fix it must
|
||||
# be the inside path.
|
||||
exists_calls: list[str] = []
|
||||
real_exists = os.path.exists
|
||||
|
||||
def capturing_exists(path: str) -> bool:
|
||||
exists_calls.append(str(path))
|
||||
return real_exists(path)
|
||||
|
||||
monkeypatch.setattr('browser_use.tools.service.os.path.exists', capturing_exists)
|
||||
|
||||
tools = Tools()
|
||||
result = await tools.registry.execute_action(
|
||||
'upload_file',
|
||||
{'index': 0, 'path': '../note.md'},
|
||||
browser_session=stub_session, # type: ignore[arg-type]
|
||||
file_system=fs,
|
||||
available_file_paths=[],
|
||||
)
|
||||
|
||||
# Whatever the eventual ActionResult, the first os.path.exists call inside
|
||||
# upload_file is on the resolved upload path. It MUST be contained in data_dir.
|
||||
assert exists_calls, 'upload_file should call os.path.exists on the resolved path'
|
||||
resolved = os.path.realpath(exists_calls[0])
|
||||
assert resolved == data_dir_real or resolved.startswith(data_dir_real + os.sep), (
|
||||
f'Upload path escaped FileSystem directory.\n'
|
||||
f' data_dir : {data_dir_real}\n'
|
||||
f' resolved : {resolved}\n'
|
||||
f' raw : {exists_calls[0]}'
|
||||
)
|
||||
|
||||
# Sanity: the resolved path should point at the FileSystem-owned file.
|
||||
assert resolved == os.path.realpath(str(fs.get_dir() / 'note.md'))
|
||||
|
||||
# The action itself will fail with an "Element does not exist" error from
|
||||
# the stub selector map — that's fine; we're only checking the path.
|
||||
assert isinstance(result, ActionResult)
|
||||
|
||||
|
||||
async def test_traversal_with_no_basename_match_still_fails_safely(
|
||||
tmp_path,
|
||||
stub_session: _StubBrowserSession,
|
||||
) -> None:
|
||||
"""Sanity: if the basename does not match any FileSystem-owned file, the
|
||||
action must reject with an availability error — never reach the join sink."""
|
||||
fs = FileSystem(base_dir=tmp_path)
|
||||
# Note: no `note.md` registered in the FileSystem.
|
||||
|
||||
tools = Tools()
|
||||
result = await tools.registry.execute_action(
|
||||
'upload_file',
|
||||
{'index': 0, 'path': '../note.md'},
|
||||
browser_session=stub_session, # type: ignore[arg-type]
|
||||
file_system=fs,
|
||||
available_file_paths=[],
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is not None
|
||||
# Should be the "not available" rejection, not a path-resolution success.
|
||||
assert 'not available' in result.error.lower() or 'does not exist' in result.error.lower()
|
||||
|
||||
|
||||
class _StubRemoteBrowserSession(_StubBrowserSession):
|
||||
"""Stub for a remote (non-local) browser session — the upload action's
|
||||
rules differ here. Remote paths are passed through to the browser process."""
|
||||
|
||||
is_local = False
|
||||
|
||||
|
||||
async def test_remote_session_does_not_rewrite_to_local_filesystem_on_basename_collision(
|
||||
tmp_path,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""For remote sessions, an absolute remote path that happens to share a
|
||||
basename with a local FileSystem-managed file MUST be passed through
|
||||
unchanged. Rewriting to the local FileSystem path would silently upload
|
||||
the wrong file (the local managed one) instead of the remote-machine file
|
||||
the agent referenced.
|
||||
|
||||
Pre-existing issue surfaced by the GHSA-j9hj-92j8-jv9h review (codex bot).
|
||||
"""
|
||||
fs = FileSystem(base_dir=tmp_path)
|
||||
# Create a local managed file with a basename that collides with the
|
||||
# remote path the agent will reference.
|
||||
await fs.write_file('note.md', 'LOCAL content the agent did NOT ask for')
|
||||
|
||||
# What the remote-session agent would pass — a path valid on the remote
|
||||
# machine, NOT in the local FileSystem.
|
||||
remote_path = '/tmp/note.md'
|
||||
|
||||
# Capture every path that the action checks for existence. For remote
|
||||
# sessions the local-existence check is skipped, so we instead capture
|
||||
# the resolved path via the same os.path.exists hook used by the local
|
||||
# test — if the rewrite-to-local-filesystem branch fires, the next call
|
||||
# will be against the data_dir path; if not, no call should reference it.
|
||||
exists_calls: list[str] = []
|
||||
real_exists = os.path.exists
|
||||
|
||||
def capturing_exists(path: str) -> bool:
|
||||
exists_calls.append(str(path))
|
||||
return real_exists(path)
|
||||
|
||||
monkeypatch.setattr('browser_use.tools.service.os.path.exists', capturing_exists)
|
||||
|
||||
tools = Tools()
|
||||
await tools.registry.execute_action(
|
||||
'upload_file',
|
||||
{'index': 0, 'path': remote_path},
|
||||
browser_session=_StubRemoteBrowserSession(), # type: ignore[arg-type]
|
||||
file_system=fs,
|
||||
available_file_paths=[],
|
||||
)
|
||||
|
||||
# The action should NOT have rewritten to a local FileSystem path. If it
|
||||
# had, we'd see the data_dir/note.md path appear in the captured calls.
|
||||
data_dir_str = str(fs.get_dir())
|
||||
rewrites = [p for p in exists_calls if p.startswith(data_dir_str)]
|
||||
assert not rewrites, (
|
||||
f'Remote session upload silently rewrote to local FileSystem path: {rewrites}. '
|
||||
f'The agent intended to upload {remote_path!r}; the rewrite would have uploaded the local managed file instead.'
|
||||
)
|
||||
@@ -0,0 +1,191 @@
|
||||
"""Tests for AGI-497: SPA/JS-heavy page renders blank.
|
||||
|
||||
Covers:
|
||||
1. Skeleton screen detection — many elements but near-zero text triggers a warning in page_stats.
|
||||
2. Navigate reload fallback — empty-body page triggers retry cycle but ultimately succeeds
|
||||
(no error) because the DOM root exists. Error is only returned when _root is None.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import tempfile
|
||||
|
||||
import pytest
|
||||
from pytest_httpserver import HTTPServer
|
||||
|
||||
from browser_use.agent.views import ActionResult
|
||||
from browser_use.browser import BrowserProfile, BrowserSession
|
||||
from browser_use.tools.service import Tools
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Fixtures
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def http_server():
|
||||
"""Session-scoped HTTP server for blank-page tests."""
|
||||
server = HTTPServer()
|
||||
server.start()
|
||||
|
||||
# --- skeleton page: 30 empty divs, essentially no text ---
|
||||
skeleton_html = '<!DOCTYPE html><html><head><title>Loading...</title></head><body>'
|
||||
skeleton_html += ''.join(f'<div class="skeleton-item skeleton-{i}"></div>' for i in range(30))
|
||||
skeleton_html += '</body></html>'
|
||||
server.expect_request('/skeleton').respond_with_data(skeleton_html, content_type='text/html')
|
||||
|
||||
# --- rich page: real text content (should NOT be flagged as skeleton) ---
|
||||
server.expect_request('/products').respond_with_data(
|
||||
"""<!DOCTYPE html>
|
||||
<html><head><title>Products</title></head>
|
||||
<body>
|
||||
<h1>Product Catalog</h1>
|
||||
<div class="product"><h2>Widget A</h2><p>Price: $29.99 - A sturdy widget for everyday use.</p></div>
|
||||
<div class="product"><h2>Widget B</h2><p>Price: $49.99 - Premium widget with extended warranty.</p></div>
|
||||
<div class="product"><h2>Gadget C</h2><p>Price: $19.50 - Compact gadget, fits in your pocket.</p></div>
|
||||
<div class="product"><h2>Gadget D</h2><p>Price: $99.00 - Professional-grade gadget for power users.</p></div>
|
||||
</body>
|
||||
</html>""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
# --- empty body page: body is always empty ---
|
||||
# Used to test the navigate retry cycle. _root is NOT None (body element exists),
|
||||
# so navigate retries but ultimately succeeds (no error).
|
||||
server.expect_request('/always-empty').respond_with_data(
|
||||
'<html><body></body></html>',
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
yield server
|
||||
server.stop()
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def base_url(http_server):
|
||||
return f'http://{http_server.host}:{http_server.port}'
|
||||
|
||||
|
||||
@pytest.fixture(scope='module')
|
||||
async def browser_session():
|
||||
session = BrowserSession(
|
||||
browser_profile=BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
keep_alive=True,
|
||||
)
|
||||
)
|
||||
await session.start()
|
||||
yield session
|
||||
await session.kill()
|
||||
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
def tools():
|
||||
return Tools()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helper
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def _navigate(tools, browser_session, url):
|
||||
"""Navigate to url and give the page a moment to settle."""
|
||||
await tools.navigate(url=url, new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
|
||||
def _make_prompt(state):
|
||||
"""Build an AgentMessagePrompt from a BrowserStateSummary using a temp dir for FileSystem."""
|
||||
from browser_use.agent.prompts import AgentMessagePrompt
|
||||
from browser_use.filesystem.file_system import FileSystem
|
||||
|
||||
tmp_dir = tempfile.mkdtemp(prefix='browseruse_test_')
|
||||
file_system = FileSystem(base_dir=tmp_dir, create_default_files=False)
|
||||
return AgentMessagePrompt(
|
||||
browser_state_summary=state,
|
||||
file_system=file_system,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Test 1: Skeleton screen detection in _extract_page_statistics()
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestSkeletonScreenDetection:
|
||||
"""_extract_page_statistics() should flag pages with many elements but very little text."""
|
||||
|
||||
async def test_skeleton_page_low_text_chars(self, tools, browser_session, base_url):
|
||||
"""Skeleton page: total_elements > 20 but text_chars < total_elements * 5."""
|
||||
await _navigate(tools, browser_session, f'{base_url}/skeleton')
|
||||
|
||||
state = await browser_session.get_browser_state_summary(include_screenshot=False)
|
||||
prompt = _make_prompt(state)
|
||||
page_stats = prompt._extract_page_statistics()
|
||||
|
||||
assert page_stats['total_elements'] > 20, f'Expected >20 elements for skeleton page, got {page_stats["total_elements"]}'
|
||||
assert page_stats['text_chars'] < page_stats['total_elements'] * 5, (
|
||||
f'Expected text_chars ({page_stats["text_chars"]}) < total_elements*5 '
|
||||
f'({page_stats["total_elements"] * 5}) for skeleton page'
|
||||
)
|
||||
|
||||
async def test_skeleton_page_description_contains_warning(self, tools, browser_session, base_url):
|
||||
"""_get_browser_state_description() includes skeleton warning for placeholder pages."""
|
||||
await _navigate(tools, browser_session, f'{base_url}/skeleton')
|
||||
|
||||
state = await browser_session.get_browser_state_summary(include_screenshot=False)
|
||||
prompt = _make_prompt(state)
|
||||
description = prompt._get_browser_state_description()
|
||||
|
||||
assert 'skeleton' in description.lower() or 'placeholder' in description.lower(), (
|
||||
f'Expected skeleton/placeholder warning in description, got:\n{description[:500]}'
|
||||
)
|
||||
|
||||
async def test_rich_page_not_flagged_as_skeleton(self, tools, browser_session, base_url):
|
||||
"""A page with real text content should NOT be flagged as skeleton."""
|
||||
await _navigate(tools, browser_session, f'{base_url}/products')
|
||||
|
||||
state = await browser_session.get_browser_state_summary(include_screenshot=False)
|
||||
prompt = _make_prompt(state)
|
||||
page_stats = prompt._extract_page_statistics()
|
||||
description = prompt._get_browser_state_description()
|
||||
|
||||
# Rich page should have substantial text relative to element count
|
||||
assert page_stats['text_chars'] >= page_stats['total_elements'] * 5, (
|
||||
f'Rich page should have text_chars ({page_stats["text_chars"]}) >= total_elements*5 '
|
||||
f'({page_stats["total_elements"] * 5})'
|
||||
)
|
||||
# No skeleton warning in description
|
||||
assert 'skeleton' not in description.lower() and 'placeholder' not in description.lower(), (
|
||||
'Rich page should NOT produce a skeleton warning'
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Test 2: Navigate reload fallback — empty-body page triggers retry but succeeds
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestNavigateReloadFallback:
|
||||
"""Navigate retries when llm_representation() is empty, but only errors when _root is None."""
|
||||
|
||||
async def test_empty_body_page_retries_then_succeeds(self, tools, browser_session, base_url):
|
||||
"""
|
||||
Navigating to a page with an empty body triggers the health-check retry cycle
|
||||
(empty llm_representation) but ultimately succeeds (no error) because the page
|
||||
HAS a DOM root (_root is not None — the body element exists and is visible).
|
||||
|
||||
The error path only fires when _root is None (truly unloadable pages like those
|
||||
blocked by anti-bot or returning empty HTTP responses), which avoids false positives
|
||||
on image-only or other non-interactive-but-valid pages.
|
||||
"""
|
||||
empty_url = f'{base_url}/always-empty'
|
||||
|
||||
# Triggers: health check -> 3s wait -> reload -> 5s wait -> check _root -> not None -> success
|
||||
result = await tools.navigate(url=empty_url, new_tab=False, browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
# No error — the body IS a valid DOM root, just visually empty.
|
||||
# Skeleton detection (in AgentMessagePrompt._get_browser_state_description) warns the LLM.
|
||||
assert result.error is None, f'Expected no error for empty-body page, got: {result.error}'
|
||||
@@ -0,0 +1,404 @@
|
||||
"""Tests for action loop detection — behavioral cycle breaking (PR #4)."""
|
||||
|
||||
from browser_use.agent.service import Agent
|
||||
from browser_use.agent.views import (
|
||||
ActionLoopDetector,
|
||||
PageFingerprint,
|
||||
compute_action_hash,
|
||||
)
|
||||
from browser_use.llm.messages import UserMessage
|
||||
from tests.ci.conftest import create_mock_llm
|
||||
|
||||
|
||||
def _get_context_messages(agent: Agent) -> list[str]:
|
||||
"""Extract text content from the agent's context messages."""
|
||||
msgs = agent._message_manager.state.history.context_messages
|
||||
return [m.content for m in msgs if isinstance(m, UserMessage) and isinstance(m.content, str)]
|
||||
|
||||
|
||||
# ─── Action hash normalization tests ─────────────────────────────────────────
|
||||
|
||||
|
||||
def test_search_normalization_ignores_keyword_order():
|
||||
"""Two searches with the same keywords in different order should produce the same hash."""
|
||||
h1 = compute_action_hash('search', {'query': 'site:example.com answers votes'})
|
||||
h2 = compute_action_hash('search', {'query': 'votes answers site:example.com'})
|
||||
assert h1 == h2
|
||||
|
||||
|
||||
def test_search_normalization_ignores_case():
|
||||
"""Search normalization is case-insensitive."""
|
||||
h1 = compute_action_hash('search', {'query': 'Python Tutorial'})
|
||||
h2 = compute_action_hash('search', {'query': 'python tutorial'})
|
||||
assert h1 == h2
|
||||
|
||||
|
||||
def test_search_normalization_ignores_punctuation():
|
||||
"""Search normalization strips punctuation."""
|
||||
h1 = compute_action_hash('search', {'query': 'site:hinative.com "answers" votes'})
|
||||
h2 = compute_action_hash('search', {'query': 'site:hinative.com answers, votes'})
|
||||
assert h1 == h2
|
||||
|
||||
|
||||
def test_search_normalization_deduplicates_tokens():
|
||||
"""Duplicate tokens in a search query produce the same hash as single tokens."""
|
||||
h1 = compute_action_hash('search', {'query': 'python python tutorial'})
|
||||
h2 = compute_action_hash('search', {'query': 'python tutorial'})
|
||||
assert h1 == h2
|
||||
|
||||
|
||||
def test_search_different_queries_produce_different_hashes():
|
||||
"""Fundamentally different search queries should NOT match."""
|
||||
h1 = compute_action_hash('search', {'query': 'python web scraping'})
|
||||
h2 = compute_action_hash('search', {'query': 'javascript testing framework'})
|
||||
assert h1 != h2
|
||||
|
||||
|
||||
def test_click_same_index_same_hash():
|
||||
"""Clicking the same element index produces the same hash."""
|
||||
h1 = compute_action_hash('click', {'index': 5})
|
||||
h2 = compute_action_hash('click', {'index': 5})
|
||||
assert h1 == h2
|
||||
|
||||
|
||||
def test_click_different_index_different_hash():
|
||||
"""Clicking different element indices produces different hashes."""
|
||||
h1 = compute_action_hash('click', {'index': 5})
|
||||
h2 = compute_action_hash('click', {'index': 12})
|
||||
assert h1 != h2
|
||||
|
||||
|
||||
def test_input_same_element_same_text():
|
||||
"""Same element + same text = same hash."""
|
||||
h1 = compute_action_hash('input', {'index': 3, 'text': 'hello world', 'clear': True})
|
||||
h2 = compute_action_hash('input', {'index': 3, 'text': 'hello world', 'clear': False})
|
||||
assert h1 == h2 # clear flag doesn't affect hash
|
||||
|
||||
|
||||
def test_input_different_text_different_hash():
|
||||
"""Same element but different text = different hash."""
|
||||
h1 = compute_action_hash('input', {'index': 3, 'text': 'hello'})
|
||||
h2 = compute_action_hash('input', {'index': 3, 'text': 'goodbye'})
|
||||
assert h1 != h2
|
||||
|
||||
|
||||
def test_navigate_same_url_same_hash():
|
||||
"""Navigating to the exact same URL produces the same hash."""
|
||||
h1 = compute_action_hash('navigate', {'url': 'https://example.com/page1'})
|
||||
h2 = compute_action_hash('navigate', {'url': 'https://example.com/page1'})
|
||||
assert h1 == h2
|
||||
|
||||
|
||||
def test_navigate_different_paths_different_hash():
|
||||
"""Navigating to different paths on the same domain produces different hashes — this is genuine exploration."""
|
||||
h1 = compute_action_hash('navigate', {'url': 'https://example.com/page1'})
|
||||
h2 = compute_action_hash('navigate', {'url': 'https://example.com/page2'})
|
||||
assert h1 != h2
|
||||
|
||||
|
||||
def test_navigate_different_domain_different_hash():
|
||||
"""Navigate to different domains produces different hashes."""
|
||||
h1 = compute_action_hash('navigate', {'url': 'https://example.com/page1'})
|
||||
h2 = compute_action_hash('navigate', {'url': 'https://other.com/page1'})
|
||||
assert h1 != h2
|
||||
|
||||
|
||||
def test_scroll_direction_matters():
|
||||
"""Scroll up and scroll down are different actions."""
|
||||
h1 = compute_action_hash('scroll', {'down': True, 'index': None})
|
||||
h2 = compute_action_hash('scroll', {'down': False, 'index': None})
|
||||
assert h1 != h2
|
||||
|
||||
|
||||
def test_scroll_different_elements_different_hash():
|
||||
"""Scrolling different elements produces different hashes."""
|
||||
h1 = compute_action_hash('scroll', {'down': True, 'index': 5})
|
||||
h2 = compute_action_hash('scroll', {'down': True, 'index': 10})
|
||||
assert h1 != h2
|
||||
|
||||
|
||||
def test_scroll_same_element_same_hash():
|
||||
"""Scrolling the same element in the same direction produces the same hash."""
|
||||
h1 = compute_action_hash('scroll', {'down': True, 'index': 5})
|
||||
h2 = compute_action_hash('scroll', {'down': True, 'index': 5})
|
||||
assert h1 == h2
|
||||
|
||||
|
||||
def test_different_action_types_different_hashes():
|
||||
"""Different action types always produce different hashes."""
|
||||
h1 = compute_action_hash('click', {'index': 5})
|
||||
h2 = compute_action_hash('scroll', {'down': True, 'index': None})
|
||||
h3 = compute_action_hash('search', {'query': 'test'})
|
||||
assert len({h1, h2, h3}) == 3
|
||||
|
||||
|
||||
# ─── ActionLoopDetector unit tests ───────────────────────────────────────────
|
||||
|
||||
|
||||
def test_detector_no_nudge_for_diverse_actions():
|
||||
"""No nudge when actions are all different."""
|
||||
detector = ActionLoopDetector(window_size=20)
|
||||
detector.record_action('click', {'index': 1})
|
||||
detector.record_action('scroll', {'down': True, 'index': None})
|
||||
detector.record_action('click', {'index': 2})
|
||||
detector.record_action('search', {'query': 'something'})
|
||||
detector.record_action('navigate', {'url': 'https://example.com'})
|
||||
assert detector.get_nudge_message() is None
|
||||
|
||||
|
||||
def test_detector_nudge_at_5_repeats():
|
||||
"""Nudge triggers at 5 repetitions of the same action."""
|
||||
detector = ActionLoopDetector(window_size=20)
|
||||
for _ in range(5):
|
||||
detector.record_action('search', {'query': 'site:hinative.com answers votes'})
|
||||
msg = detector.get_nudge_message()
|
||||
assert msg is not None
|
||||
assert 'repeated a similar action' in msg
|
||||
assert '5 times' in msg
|
||||
|
||||
|
||||
def test_detector_no_nudge_at_4_repeats():
|
||||
"""No nudge at only 4 repetitions (below threshold)."""
|
||||
detector = ActionLoopDetector(window_size=20)
|
||||
for _ in range(4):
|
||||
detector.record_action('search', {'query': 'site:hinative.com answers votes'})
|
||||
assert detector.get_nudge_message() is None
|
||||
|
||||
|
||||
def test_detector_nudge_escalates_at_8_repeats():
|
||||
"""Stronger nudge at 8 repetitions."""
|
||||
detector = ActionLoopDetector(window_size=20)
|
||||
for _ in range(8):
|
||||
detector.record_action('search', {'query': 'site:hinative.com answers votes'})
|
||||
msg = detector.get_nudge_message()
|
||||
assert msg is not None
|
||||
assert 'still making progress' in msg
|
||||
assert '8 times' in msg
|
||||
|
||||
|
||||
def test_detector_nudge_escalates_at_12_repeats():
|
||||
"""Most urgent nudge at 12 repetitions."""
|
||||
detector = ActionLoopDetector(window_size=20)
|
||||
for _ in range(12):
|
||||
detector.record_action('search', {'query': 'site:hinative.com answers votes'})
|
||||
msg = detector.get_nudge_message()
|
||||
assert msg is not None
|
||||
assert 'making progress with each repetition' in msg
|
||||
assert '12 times' in msg
|
||||
|
||||
|
||||
def test_detector_critical_message_no_done_directive():
|
||||
"""Critical nudge should NOT tell the agent to call done — just a gentle heads up."""
|
||||
detector = ActionLoopDetector(window_size=20)
|
||||
for _ in range(12):
|
||||
detector.record_action('search', {'query': 'site:hinative.com answers votes'})
|
||||
msg = detector.get_nudge_message()
|
||||
assert msg is not None
|
||||
assert 'done action' not in msg
|
||||
assert 'different approach' in msg
|
||||
|
||||
|
||||
def test_detector_first_nudge_no_cannot_complete():
|
||||
"""First nudge should NOT say task 'cannot be completed' — just raise awareness."""
|
||||
detector = ActionLoopDetector(window_size=20)
|
||||
for _ in range(5):
|
||||
detector.record_action('search', {'query': 'site:hinative.com answers votes'})
|
||||
msg = detector.get_nudge_message()
|
||||
assert msg is not None
|
||||
assert 'cannot be completed' not in msg
|
||||
assert 'reconsidering your approach' in msg
|
||||
|
||||
|
||||
def test_detector_window_slides():
|
||||
"""Old actions fall out of the window."""
|
||||
detector = ActionLoopDetector(window_size=10)
|
||||
# Fill window with repeated actions
|
||||
for _ in range(5):
|
||||
detector.record_action('click', {'index': 7})
|
||||
assert detector.max_repetition_count == 5
|
||||
|
||||
# Push them out with diverse actions
|
||||
for i in range(10):
|
||||
detector.record_action('click', {'index': 100 + i})
|
||||
# The 5 old repeated actions should have been pushed out
|
||||
assert detector.max_repetition_count < 5
|
||||
assert detector.get_nudge_message() is None
|
||||
|
||||
|
||||
def test_detector_search_variations_detected_as_same():
|
||||
"""Minor variations of the same search (the hinative pattern) are detected as repetition."""
|
||||
detector = ActionLoopDetector(window_size=20)
|
||||
# These are the kind of variations the agent produces
|
||||
queries = [
|
||||
'site:hinative.com answers votes questions',
|
||||
'site:hinative.com questions answers votes',
|
||||
'site:hinative.com votes answers questions',
|
||||
'site:hinative.com questions votes answers',
|
||||
'site:hinative.com answers questions votes',
|
||||
]
|
||||
for q in queries:
|
||||
detector.record_action('search', {'query': q})
|
||||
assert detector.max_repetition_count == 5
|
||||
assert detector.get_nudge_message() is not None
|
||||
|
||||
|
||||
# ─── Page stagnation detection tests ─────────────────────────────────────────
|
||||
|
||||
|
||||
def test_page_stagnation_no_nudge_when_pages_change():
|
||||
"""No stagnation nudge when page content changes each step."""
|
||||
detector = ActionLoopDetector(window_size=20)
|
||||
detector.record_page_state('https://example.com', 'page content 1', 50)
|
||||
detector.record_page_state('https://example.com', 'page content 2', 55)
|
||||
detector.record_page_state('https://example.com', 'page content 3', 60)
|
||||
assert detector.consecutive_stagnant_pages == 0
|
||||
assert detector.get_nudge_message() is None
|
||||
|
||||
|
||||
def test_page_stagnation_nudge_at_5_identical_pages():
|
||||
"""Stagnation nudge fires after 5 consecutive identical page states."""
|
||||
detector = ActionLoopDetector(window_size=20)
|
||||
# First recording establishes baseline (doesn't count as stagnant)
|
||||
for _ in range(6):
|
||||
detector.record_page_state('https://example.com', 'same content', 50)
|
||||
assert detector.consecutive_stagnant_pages >= 5
|
||||
msg = detector.get_nudge_message()
|
||||
assert msg is not None
|
||||
assert 'page content has not changed' in msg
|
||||
|
||||
|
||||
def test_page_stagnation_no_nudge_at_4_identical_pages():
|
||||
"""No stagnation nudge at only 4 consecutive identical pages (below threshold)."""
|
||||
detector = ActionLoopDetector(window_size=20)
|
||||
# First recording establishes baseline, then 4 stagnant = 5 total recordings
|
||||
for _ in range(5):
|
||||
detector.record_page_state('https://example.com', 'same content', 50)
|
||||
assert detector.consecutive_stagnant_pages == 4
|
||||
assert detector.get_nudge_message() is None
|
||||
|
||||
|
||||
def test_page_stagnation_resets_on_change():
|
||||
"""Stagnation counter resets when page content changes."""
|
||||
detector = ActionLoopDetector(window_size=20)
|
||||
detector.record_page_state('https://example.com', 'same content', 50)
|
||||
detector.record_page_state('https://example.com', 'same content', 50)
|
||||
detector.record_page_state('https://example.com', 'same content', 50)
|
||||
assert detector.consecutive_stagnant_pages == 2
|
||||
# Page changes
|
||||
detector.record_page_state('https://example.com', 'different content', 55)
|
||||
assert detector.consecutive_stagnant_pages == 0
|
||||
|
||||
|
||||
def test_combined_loop_and_stagnation():
|
||||
"""Both action loop and page stagnation messages appear together."""
|
||||
detector = ActionLoopDetector(window_size=20)
|
||||
# Create action repetition (8 for STRONG LOOP WARNING)
|
||||
for _ in range(8):
|
||||
detector.record_action('click', {'index': 7})
|
||||
# Create page stagnation (need 5 consecutive stagnant)
|
||||
detector.record_page_state('https://example.com', 'same', 50)
|
||||
for _ in range(5):
|
||||
detector.record_page_state('https://example.com', 'same', 50)
|
||||
msg = detector.get_nudge_message()
|
||||
assert msg is not None
|
||||
assert 'still making progress' in msg
|
||||
assert 'page content has not changed' in msg
|
||||
|
||||
|
||||
# ─── PageFingerprint tests ───────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_page_fingerprint_same_content_equal():
|
||||
"""Same content produces equal fingerprints."""
|
||||
fp1 = PageFingerprint.from_browser_state('https://example.com', 'hello world', 50)
|
||||
fp2 = PageFingerprint.from_browser_state('https://example.com', 'hello world', 50)
|
||||
assert fp1 == fp2
|
||||
|
||||
|
||||
def test_page_fingerprint_different_content_not_equal():
|
||||
"""Different content produces different fingerprints."""
|
||||
fp1 = PageFingerprint.from_browser_state('https://example.com', 'hello world', 50)
|
||||
fp2 = PageFingerprint.from_browser_state('https://example.com', 'goodbye world', 50)
|
||||
assert fp1 != fp2
|
||||
|
||||
|
||||
def test_page_fingerprint_different_url_not_equal():
|
||||
"""Different URL produces different fingerprint even with same content."""
|
||||
fp1 = PageFingerprint.from_browser_state('https://example.com', 'hello world', 50)
|
||||
fp2 = PageFingerprint.from_browser_state('https://other.com', 'hello world', 50)
|
||||
assert fp1 != fp2
|
||||
|
||||
|
||||
def test_page_fingerprint_different_element_count_not_equal():
|
||||
"""Different element count produces different fingerprint."""
|
||||
fp1 = PageFingerprint.from_browser_state('https://example.com', 'hello world', 50)
|
||||
fp2 = PageFingerprint.from_browser_state('https://example.com', 'hello world', 51)
|
||||
assert fp1 != fp2
|
||||
|
||||
|
||||
# ─── Agent integration tests ─────────────────────────────────────────────────
|
||||
|
||||
|
||||
async def test_loop_nudge_injected_into_context():
|
||||
"""Loop detection nudge is injected as a context message in the agent."""
|
||||
llm = create_mock_llm()
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
|
||||
# Simulate 5 repeated actions (new threshold)
|
||||
for _ in range(5):
|
||||
agent.state.loop_detector.record_action('search', {'query': 'site:example.com answers'})
|
||||
|
||||
agent._inject_loop_detection_nudge()
|
||||
|
||||
messages = _get_context_messages(agent)
|
||||
assert len(messages) == 1
|
||||
assert 'repeated a similar action' in messages[0]
|
||||
|
||||
|
||||
async def test_no_loop_nudge_when_disabled():
|
||||
"""No loop nudge when loop_detection_enabled is False."""
|
||||
llm = create_mock_llm()
|
||||
agent = Agent(
|
||||
task='Test task',
|
||||
llm=llm,
|
||||
loop_detection_enabled=False,
|
||||
)
|
||||
|
||||
# Simulate 8 repeated actions
|
||||
for _ in range(8):
|
||||
agent.state.loop_detector.record_action('search', {'query': 'site:example.com answers'})
|
||||
|
||||
agent._inject_loop_detection_nudge()
|
||||
|
||||
messages = _get_context_messages(agent)
|
||||
assert len(messages) == 0
|
||||
|
||||
|
||||
async def test_no_loop_nudge_for_diverse_actions():
|
||||
"""No loop nudge when actions are diverse."""
|
||||
llm = create_mock_llm()
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
|
||||
agent.state.loop_detector.record_action('click', {'index': 1})
|
||||
agent.state.loop_detector.record_action('scroll', {'down': True, 'index': None})
|
||||
agent.state.loop_detector.record_action('click', {'index': 2})
|
||||
|
||||
agent._inject_loop_detection_nudge()
|
||||
|
||||
messages = _get_context_messages(agent)
|
||||
assert len(messages) == 0
|
||||
|
||||
|
||||
async def test_loop_detector_initialized_from_settings():
|
||||
"""Loop detector window size is set from agent settings."""
|
||||
llm = create_mock_llm()
|
||||
agent = Agent(task='Test task', llm=llm, loop_detection_window=30)
|
||||
assert agent.state.loop_detector.window_size == 30
|
||||
|
||||
|
||||
async def test_loop_detector_default_window_size():
|
||||
"""Loop detection default window size is 20."""
|
||||
llm = create_mock_llm()
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
assert agent.settings.loop_detection_enabled is True
|
||||
assert agent.state.loop_detector.window_size == 20
|
||||
@@ -0,0 +1,152 @@
|
||||
"""Tests for the RecordingWatchdog start/stop API and the `browser-use record` CLI command.
|
||||
|
||||
The watchdog drives CDP screencast (`Page.startScreencast`/`stopScreencast`) and
|
||||
`VideoRecorderService` (imageio+ffmpeg) to produce an MP4. These tests exercise
|
||||
the full stack against a real headless browser.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
|
||||
try:
|
||||
import imageio.v2 as iio # type: ignore[import-not-found]
|
||||
|
||||
IMAGEIO_AVAILABLE = True
|
||||
except ImportError:
|
||||
IMAGEIO_AVAILABLE = False
|
||||
|
||||
from browser_use.browser.events import NavigateToUrlEvent
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
from browser_use.browser.session import BrowserSession
|
||||
|
||||
pytestmark = pytest.mark.skipif(
|
||||
not IMAGEIO_AVAILABLE,
|
||||
reason='Recording requires the [video] extra: pip install "browser-use[video]"',
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def browser_session():
|
||||
session = BrowserSession(browser_profile=BrowserProfile(headless=True))
|
||||
await session.start()
|
||||
yield session
|
||||
await session.kill()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def page_url(httpserver):
|
||||
httpserver.expect_request('/recpage').respond_with_data(
|
||||
"""
|
||||
<html>
|
||||
<body style='background:#f0f;padding:40px;'>
|
||||
<h1 id='title'>Recording test</h1>
|
||||
<p>This content should appear in the captured video.</p>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
return httpserver.url_for('/recpage')
|
||||
|
||||
|
||||
async def _drive_browser_briefly(bs: BrowserSession, url: str, ticks: int = 8) -> None:
|
||||
"""Navigate + poke the page so screencast emits a few frames."""
|
||||
await bs.event_bus.dispatch(NavigateToUrlEvent(url=url, new_tab=False))
|
||||
# Screencast emits frames as the page changes; give it enough time to collect some
|
||||
for _ in range(ticks):
|
||||
await asyncio.sleep(0.15)
|
||||
|
||||
|
||||
async def test_start_stop_recording_produces_video(browser_session: BrowserSession, page_url: str, tmp_path: Path):
|
||||
"""start_recording → activity → stop_recording should write a valid MP4."""
|
||||
watchdog = browser_session._recording_watchdog
|
||||
assert watchdog is not None, 'BrowserSession should always attach a RecordingWatchdog'
|
||||
|
||||
out_path = tmp_path / 'session.mp4'
|
||||
assert not watchdog.is_recording
|
||||
|
||||
saved = await watchdog.start_recording(out_path)
|
||||
assert saved == out_path
|
||||
assert watchdog.is_recording
|
||||
|
||||
await _drive_browser_briefly(browser_session, page_url)
|
||||
|
||||
final = await watchdog.stop_recording()
|
||||
assert final == out_path
|
||||
assert not watchdog.is_recording
|
||||
assert out_path.exists(), 'recording stop should leave a file on disk'
|
||||
assert out_path.stat().st_size > 0, 'recorded video must be non-empty'
|
||||
|
||||
# Confirm the file is actually a decodable video with at least one frame.
|
||||
reader: Any = iio.get_reader(str(out_path))
|
||||
try:
|
||||
frame: Any = reader.get_next_data()
|
||||
assert frame is not None and frame.size > 0
|
||||
finally:
|
||||
reader.close()
|
||||
|
||||
|
||||
async def test_start_recording_twice_raises(browser_session: BrowserSession, tmp_path: Path):
|
||||
watchdog = browser_session._recording_watchdog
|
||||
assert watchdog is not None
|
||||
|
||||
await watchdog.start_recording(tmp_path / 'first.mp4')
|
||||
try:
|
||||
with pytest.raises(RuntimeError, match='already in progress'):
|
||||
await watchdog.start_recording(tmp_path / 'second.mp4')
|
||||
finally:
|
||||
await watchdog.stop_recording()
|
||||
|
||||
|
||||
async def test_stop_without_start_returns_none(browser_session: BrowserSession):
|
||||
watchdog = browser_session._recording_watchdog
|
||||
assert watchdog is not None
|
||||
assert await watchdog.stop_recording() is None
|
||||
|
||||
|
||||
async def test_on_browser_connected_degrades_gracefully_when_recording_fails(
|
||||
browser_session: BrowserSession, tmp_path: Path, monkeypatch
|
||||
):
|
||||
"""If start_recording() raises (e.g. missing [video] deps), profile-driven recording
|
||||
must degrade to a warning instead of breaking BrowserSession startup (see PR #4710 review)."""
|
||||
from browser_use.browser.events import BrowserConnectedEvent
|
||||
from browser_use.browser.watchdogs import recording_watchdog as rw_mod
|
||||
|
||||
watchdog = browser_session._recording_watchdog
|
||||
assert watchdog is not None
|
||||
|
||||
async def fake_start_recording(self: Any, *_args: Any, **_kwargs: Any) -> Path:
|
||||
raise RuntimeError('simulated missing video deps')
|
||||
|
||||
monkeypatch.setattr(rw_mod.RecordingWatchdog, 'start_recording', fake_start_recording)
|
||||
browser_session.browser_profile.record_video_dir = tmp_path
|
||||
|
||||
# Must not raise — watchdog should catch the RuntimeError and just log a warning.
|
||||
await watchdog.on_BrowserConnectedEvent(BrowserConnectedEvent(cdp_url=browser_session.cdp_url or ''))
|
||||
assert not watchdog.is_recording
|
||||
|
||||
|
||||
async def test_profile_record_video_dir_still_works(page_url: str, tmp_path: Path):
|
||||
"""The existing event-driven flow (profile.record_video_dir) must keep working."""
|
||||
session = BrowserSession(
|
||||
browser_profile=BrowserProfile(headless=True, record_video_dir=tmp_path),
|
||||
)
|
||||
await session.start()
|
||||
try:
|
||||
watchdog = session._recording_watchdog
|
||||
assert watchdog is not None
|
||||
# on_BrowserConnectedEvent should have auto-started recording via the watchdog
|
||||
assert watchdog.is_recording, 'profile.record_video_dir should have auto-started recording'
|
||||
await _drive_browser_briefly(session, page_url)
|
||||
finally:
|
||||
await session.kill()
|
||||
|
||||
# After kill, BrowserStopEvent should have finalized the video file into tmp_path
|
||||
videos = list(tmp_path.glob('*.mp4'))
|
||||
assert videos, f'expected at least one recorded mp4 in {tmp_path}'
|
||||
assert videos[0].stat().st_size > 0
|
||||
@@ -0,0 +1,384 @@
|
||||
import asyncio
|
||||
import tempfile
|
||||
|
||||
import anyio
|
||||
import pytest
|
||||
from pytest_httpserver import HTTPServer
|
||||
|
||||
from browser_use.agent.views import ActionResult
|
||||
from browser_use.browser import BrowserSession
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
from browser_use.filesystem.file_system import FileSystem
|
||||
from browser_use.tools.service import Tools
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def http_server():
|
||||
server = HTTPServer()
|
||||
server.start()
|
||||
|
||||
server.expect_request('/pdf-test').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>PDF Test Page</title></head>
|
||||
<body>
|
||||
<h1>Hello PDF</h1>
|
||||
<p>This page should be saved as a PDF document.</p>
|
||||
<ul>
|
||||
<li>Item 1</li>
|
||||
<li>Item 2</li>
|
||||
<li>Item 3</li>
|
||||
</ul>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
server.expect_request('/pdf-styled').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Styled PDF Page</title>
|
||||
<style>
|
||||
body { background-color: #f0f0f0; font-family: sans-serif; }
|
||||
h1 { color: navy; }
|
||||
.highlight { background-color: yellow; padding: 10px; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<h1>Styled Content</h1>
|
||||
<div class="highlight">This has a background color that should appear when print_background=True.</div>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
yield server
|
||||
server.stop()
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def base_url(http_server):
|
||||
return f'http://{http_server.host}:{http_server.port}'
|
||||
|
||||
|
||||
@pytest.fixture(scope='module')
|
||||
async def browser_session():
|
||||
browser_session = BrowserSession(
|
||||
browser_profile=BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
keep_alive=True,
|
||||
)
|
||||
)
|
||||
await browser_session.start()
|
||||
yield browser_session
|
||||
await browser_session.kill()
|
||||
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
def tools():
|
||||
return Tools()
|
||||
|
||||
|
||||
def _get_attachments(result: ActionResult) -> list[str]:
|
||||
"""Helper to extract attachments with pyright-safe narrowing."""
|
||||
assert result.attachments is not None
|
||||
return result.attachments
|
||||
|
||||
|
||||
class TestSaveAsPdf:
|
||||
"""Tests for the save_as_pdf action."""
|
||||
|
||||
async def test_save_as_pdf_registered(self, tools):
|
||||
"""save_as_pdf action is in the default action registry."""
|
||||
assert 'save_as_pdf' in tools.registry.registry.actions
|
||||
action = tools.registry.registry.actions['save_as_pdf']
|
||||
assert action.function is not None
|
||||
assert 'PDF' in action.description
|
||||
|
||||
async def test_save_as_pdf_default_filename(self, tools, browser_session, base_url):
|
||||
"""save_as_pdf with no filename uses the page title."""
|
||||
await tools.navigate(url=f'{base_url}/pdf-test', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
file_system = FileSystem(temp_dir)
|
||||
result = await tools.save_as_pdf(browser_session=browser_session, file_system=file_system)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'Saved page as PDF' in result.extracted_content
|
||||
|
||||
attachments = _get_attachments(result)
|
||||
assert len(attachments) == 1
|
||||
|
||||
pdf_path = attachments[0]
|
||||
assert pdf_path.endswith('.pdf')
|
||||
assert await anyio.Path(pdf_path).exists()
|
||||
|
||||
# Verify it's actually a PDF (starts with %PDF magic bytes)
|
||||
header = await anyio.Path(pdf_path).read_bytes()
|
||||
assert header[:5] == b'%PDF-', f'File does not start with PDF magic bytes: {header[:5]!r}'
|
||||
|
||||
async def test_save_as_pdf_custom_filename(self, tools, browser_session, base_url):
|
||||
"""save_as_pdf with a custom filename uses that name."""
|
||||
await tools.navigate(url=f'{base_url}/pdf-test', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
file_system = FileSystem(temp_dir)
|
||||
result = await tools.save_as_pdf(
|
||||
file_name='my-report',
|
||||
browser_session=browser_session,
|
||||
file_system=file_system,
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
attachments = _get_attachments(result)
|
||||
assert len(attachments) == 1
|
||||
|
||||
pdf_path = attachments[0]
|
||||
assert 'my-report.pdf' in pdf_path
|
||||
assert await anyio.Path(pdf_path).exists()
|
||||
|
||||
async def test_save_as_pdf_custom_filename_with_extension(self, tools, browser_session, base_url):
|
||||
"""save_as_pdf doesn't double the .pdf extension."""
|
||||
await tools.navigate(url=f'{base_url}/pdf-test', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
file_system = FileSystem(temp_dir)
|
||||
result = await tools.save_as_pdf(
|
||||
file_name='already.pdf',
|
||||
browser_session=browser_session,
|
||||
file_system=file_system,
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
attachments = _get_attachments(result)
|
||||
pdf_path = attachments[0]
|
||||
# Should not be "already.pdf.pdf"
|
||||
assert pdf_path.endswith('already.pdf')
|
||||
assert not pdf_path.endswith('.pdf.pdf')
|
||||
|
||||
async def test_save_as_pdf_duplicate_filename(self, tools, browser_session, base_url):
|
||||
"""save_as_pdf increments filename when a duplicate exists."""
|
||||
await tools.navigate(url=f'{base_url}/pdf-test', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
file_system = FileSystem(temp_dir)
|
||||
|
||||
# Save first PDF
|
||||
result1 = await tools.save_as_pdf(
|
||||
file_name='duplicate',
|
||||
browser_session=browser_session,
|
||||
file_system=file_system,
|
||||
)
|
||||
attachments1 = _get_attachments(result1)
|
||||
assert await anyio.Path(attachments1[0]).exists()
|
||||
assert attachments1[0].endswith('duplicate.pdf')
|
||||
|
||||
# Save second PDF with same name
|
||||
result2 = await tools.save_as_pdf(
|
||||
file_name='duplicate',
|
||||
browser_session=browser_session,
|
||||
file_system=file_system,
|
||||
)
|
||||
attachments2 = _get_attachments(result2)
|
||||
assert await anyio.Path(attachments2[0]).exists()
|
||||
assert 'duplicate (1).pdf' in attachments2[0]
|
||||
|
||||
async def test_save_as_pdf_landscape(self, tools, browser_session, base_url):
|
||||
"""save_as_pdf with landscape=True produces a valid PDF."""
|
||||
await tools.navigate(url=f'{base_url}/pdf-test', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
file_system = FileSystem(temp_dir)
|
||||
result = await tools.save_as_pdf(
|
||||
file_name='landscape-test',
|
||||
landscape=True,
|
||||
browser_session=browser_session,
|
||||
file_system=file_system,
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
attachments = _get_attachments(result)
|
||||
assert await anyio.Path(attachments[0]).exists()
|
||||
|
||||
header = await anyio.Path(attachments[0]).read_bytes()
|
||||
assert header[:5] == b'%PDF-'
|
||||
|
||||
async def test_save_as_pdf_a4_format(self, tools, browser_session, base_url):
|
||||
"""save_as_pdf with paper_format='A4' produces a valid PDF."""
|
||||
await tools.navigate(url=f'{base_url}/pdf-test', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
file_system = FileSystem(temp_dir)
|
||||
result = await tools.save_as_pdf(
|
||||
file_name='a4-test',
|
||||
paper_format='A4',
|
||||
browser_session=browser_session,
|
||||
file_system=file_system,
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
attachments = _get_attachments(result)
|
||||
assert await anyio.Path(attachments[0]).exists()
|
||||
|
||||
async def test_save_as_pdf_with_background(self, tools, browser_session, base_url):
|
||||
"""save_as_pdf with print_background=True on a styled page produces a valid PDF."""
|
||||
await tools.navigate(url=f'{base_url}/pdf-styled', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
file_system = FileSystem(temp_dir)
|
||||
result = await tools.save_as_pdf(
|
||||
file_name='styled-with-bg',
|
||||
print_background=True,
|
||||
browser_session=browser_session,
|
||||
file_system=file_system,
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
attachments = _get_attachments(result)
|
||||
pdf_path = attachments[0]
|
||||
assert await anyio.Path(pdf_path).exists()
|
||||
|
||||
# Verify file size is non-trivial (has actual rendered content)
|
||||
stat = await anyio.Path(pdf_path).stat()
|
||||
assert stat.st_size > 1000, f'PDF seems too small ({stat.st_size} bytes), may be empty'
|
||||
|
||||
async def test_save_as_pdf_header_footer_renders_url(self, tools, browser_session, http_server, base_url):
|
||||
"""display_header_footer=True (the default) prints the page URL into the footer."""
|
||||
import pypdf
|
||||
|
||||
page_url = f'{base_url}/pdf-test'
|
||||
await tools.navigate(url=page_url, new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
file_system = FileSystem(temp_dir)
|
||||
result = await tools.save_as_pdf(
|
||||
file_name='with-metadata',
|
||||
browser_session=browser_session,
|
||||
file_system=file_system,
|
||||
)
|
||||
|
||||
pdf_path = _get_attachments(result)[0]
|
||||
assert await anyio.Path(pdf_path).exists()
|
||||
|
||||
reader = pypdf.PdfReader(pdf_path)
|
||||
text_no_ws = ''.join(reader.pages[0].extract_text().split())
|
||||
|
||||
# The footer metadata (page URL) should be embedded in the rendered PDF.
|
||||
expected = f'{http_server.host}:{http_server.port}/pdf-test'
|
||||
assert expected in text_no_ws, f'URL footer metadata not found in PDF text: {text_no_ws!r}'
|
||||
|
||||
async def test_save_as_pdf_without_header_footer(self, tools, browser_session, http_server, base_url):
|
||||
"""display_header_footer=False omits the URL/date metadata from the PDF."""
|
||||
import pypdf
|
||||
|
||||
page_url = f'{base_url}/pdf-test'
|
||||
await tools.navigate(url=page_url, new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
file_system = FileSystem(temp_dir)
|
||||
result = await tools.save_as_pdf(
|
||||
file_name='no-metadata',
|
||||
display_header_footer=False,
|
||||
browser_session=browser_session,
|
||||
file_system=file_system,
|
||||
)
|
||||
|
||||
pdf_path = _get_attachments(result)[0]
|
||||
assert await anyio.Path(pdf_path).exists()
|
||||
|
||||
reader = pypdf.PdfReader(pdf_path)
|
||||
text_no_ws = ''.join(reader.pages[0].extract_text().split())
|
||||
|
||||
# Page body never contains the URL, so it must not appear without the footer.
|
||||
netloc = f'{http_server.host}:{http_server.port}'
|
||||
assert netloc not in text_no_ws, f'URL leaked into PDF without header/footer: {text_no_ws!r}'
|
||||
|
||||
async def test_save_as_pdf_custom_templates(self, tools, browser_session, base_url):
|
||||
"""Custom header/footer templates are honored over the defaults."""
|
||||
import pypdf
|
||||
|
||||
await tools.navigate(url=f'{base_url}/pdf-test', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
file_system = FileSystem(temp_dir)
|
||||
result = await tools.save_as_pdf(
|
||||
file_name='custom-templates',
|
||||
header_template='<div style="font-size:12px;">CONFIDENTIAL DRAFT</div>',
|
||||
footer_template='<div style="font-size:12px;"><span class="title"></span></div>',
|
||||
browser_session=browser_session,
|
||||
file_system=file_system,
|
||||
)
|
||||
|
||||
pdf_path = _get_attachments(result)[0]
|
||||
reader = pypdf.PdfReader(pdf_path)
|
||||
text_no_ws = ''.join(reader.pages[0].extract_text().split())
|
||||
|
||||
assert 'CONFIDENTIALDRAFT' in text_no_ws, f'Custom header not found in PDF text: {text_no_ws!r}'
|
||||
|
||||
async def test_save_as_pdf_long_url_keeps_page_number(self, tools, browser_session, base_url):
|
||||
"""A long footer URL truncates instead of pushing the page number off the printable area."""
|
||||
import pypdf
|
||||
|
||||
# Long, unbroken URL — without min-width:0 + ellipsis on the url span this
|
||||
# overflows the footer and pushes the page count past the page edge.
|
||||
long_url = f'{base_url}/pdf-test?q={"x" * 400}'
|
||||
await tools.navigate(url=long_url, new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
file_system = FileSystem(temp_dir)
|
||||
result = await tools.save_as_pdf(
|
||||
file_name='long-url',
|
||||
# Suppress the header date so the page-number check below is unambiguous.
|
||||
header_template='<span></span>',
|
||||
browser_session=browser_session,
|
||||
file_system=file_system,
|
||||
)
|
||||
|
||||
pdf_path = _get_attachments(result)[0]
|
||||
reader = pypdf.PdfReader(pdf_path)
|
||||
text_no_ws = ''.join(reader.pages[0].extract_text().split())
|
||||
|
||||
# The footer page count must still render despite the very long URL.
|
||||
assert '1/1' in text_no_ws, f'page number pushed off-page by long URL: {text_no_ws!r}'
|
||||
|
||||
async def test_save_as_pdf_param_model_schema(self):
|
||||
"""SaveAsPdfAction schema exposes the right fields with defaults."""
|
||||
from browser_use.tools.views import SaveAsPdfAction
|
||||
|
||||
schema = SaveAsPdfAction.model_json_schema()
|
||||
props = schema['properties']
|
||||
|
||||
assert 'file_name' in props
|
||||
assert 'print_background' in props
|
||||
assert 'landscape' in props
|
||||
assert 'scale' in props
|
||||
assert 'paper_format' in props
|
||||
assert 'display_header_footer' in props
|
||||
assert 'header_template' in props
|
||||
assert 'footer_template' in props
|
||||
|
||||
# Check defaults
|
||||
assert props['print_background']['default'] is True
|
||||
assert props['landscape']['default'] is False
|
||||
assert props['scale']['default'] == 1.0
|
||||
assert props['paper_format']['default'] == 'Letter'
|
||||
# Header/footer metadata is on by default, matching the browser Print dialog
|
||||
assert props['display_header_footer']['default'] is True
|
||||
@@ -0,0 +1,176 @@
|
||||
"""Per-action timeout regression test.
|
||||
|
||||
When a CDP WebSocket goes silent (common failure mode with remote / cloud browsers),
|
||||
action handlers can await event-bus dispatches that never resolve — individual CDP
|
||||
calls like Page.navigate() have their own timeouts, but the surrounding event
|
||||
plumbing does not. Without a per-action cap, `tools.act()` hangs indefinitely and
|
||||
agents never emit a step, producing empty history traces.
|
||||
|
||||
This test replaces `registry.execute_action` with a coroutine that sleeps longer
|
||||
than the per-action cap, then asserts that `tools.act()` returns within the cap
|
||||
with an ActionResult(error=...) instead of hanging.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
|
||||
from browser_use.agent.views import ActionModel, ActionResult
|
||||
from browser_use.tools.service import Tools
|
||||
|
||||
|
||||
class _StubActionModel(ActionModel):
|
||||
"""ActionModel with two arbitrary named slots for tools.act() plumbing tests.
|
||||
|
||||
Tests target tools.act() behaviour (timeout wrapping, error handling), not any
|
||||
registered action — so we declare fixed slots here and stub out execute_action.
|
||||
"""
|
||||
|
||||
hung_action: dict[str, Any] | None = None
|
||||
fast_action: dict[str, Any] | None = None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_act_enforces_per_action_timeout_on_hung_handler():
|
||||
"""tools.act() must return within action_timeout even if the handler hangs."""
|
||||
tools = Tools()
|
||||
|
||||
# Replace the action executor with one that hangs far past the timeout.
|
||||
sleep_seconds = 30.0
|
||||
call_count = {'n': 0}
|
||||
|
||||
async def _hanging_execute_action(**_kwargs):
|
||||
call_count['n'] += 1
|
||||
await asyncio.sleep(sleep_seconds)
|
||||
return ActionResult(extracted_content='should never be reached')
|
||||
|
||||
tools.registry.execute_action = _hanging_execute_action # type: ignore[assignment]
|
||||
|
||||
# Build an ActionModel with a single slot — act() iterates model_dump(exclude_unset=True).
|
||||
action = _StubActionModel(hung_action={'url': 'https://example.com'})
|
||||
|
||||
# Use a tight timeout so the test runs in under a second.
|
||||
action_timeout = 0.5
|
||||
start = time.monotonic()
|
||||
result = await tools.act(action=action, browser_session=None, action_timeout=action_timeout) # type: ignore[arg-type]
|
||||
elapsed = time.monotonic() - start
|
||||
|
||||
# Handler got invoked exactly once.
|
||||
assert call_count['n'] == 1
|
||||
|
||||
# Returned well before the sleep would have finished.
|
||||
assert elapsed < sleep_seconds / 2, f'act() did not honor timeout; took {elapsed:.2f}s'
|
||||
# And returned close to the timeout itself (with a reasonable grace margin).
|
||||
assert elapsed < action_timeout + 2.0, f'act() overshot timeout; took {elapsed:.2f}s'
|
||||
|
||||
# Returned a proper ActionResult describing the timeout.
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is not None
|
||||
assert 'timed out' in result.error.lower()
|
||||
assert 'hung_action' in result.error
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_act_passes_through_fast_handler():
|
||||
"""When the handler finishes fast, act() returns its result unchanged."""
|
||||
tools = Tools()
|
||||
|
||||
async def _fast_execute_action(**_kwargs):
|
||||
return ActionResult(extracted_content='done')
|
||||
|
||||
tools.registry.execute_action = _fast_execute_action # type: ignore[assignment]
|
||||
|
||||
action = _StubActionModel(fast_action={'x': 1})
|
||||
result = await tools.act(action=action, browser_session=None, action_timeout=5.0) # type: ignore[arg-type]
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is None
|
||||
assert result.extracted_content == 'done'
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_act_rejects_invalid_action_timeout_override():
|
||||
"""An invalid action_timeout override (nan / inf / <=0) must fall back to
|
||||
the default, not silently defeat the timeout (nan → immediate timeout,
|
||||
inf → no timeout at all)."""
|
||||
tools = Tools()
|
||||
|
||||
calls = {'n': 0}
|
||||
|
||||
async def _fast_execute_action(**_kwargs):
|
||||
calls['n'] += 1
|
||||
return ActionResult(extracted_content='done')
|
||||
|
||||
tools.registry.execute_action = _fast_execute_action # type: ignore[assignment]
|
||||
|
||||
# nan would otherwise produce an immediate TimeoutError; we expect the
|
||||
# coercion to fall back to the default, so the fast handler runs to
|
||||
# completion and returns the success result.
|
||||
action = _StubActionModel(fast_action={'x': 1})
|
||||
result = await tools.act(action=action, browser_session=None, action_timeout=float('nan')) # type: ignore[arg-type]
|
||||
assert calls['n'] == 1
|
||||
assert result.error is None
|
||||
assert result.extracted_content == 'done'
|
||||
|
||||
# inf / non-positive values also fall back cleanly.
|
||||
for bad in (float('inf'), 0.0, -5.0):
|
||||
result = await tools.act(action=action, browser_session=None, action_timeout=bad) # type: ignore[arg-type]
|
||||
assert result.error is None, f'override {bad!r} should have fallen back'
|
||||
|
||||
|
||||
def test_default_action_timeout_accommodates_extract_action():
|
||||
"""The module-level default must sit above extract's 120s LLM inner cap."""
|
||||
from browser_use.tools.service import _DEFAULT_ACTION_TIMEOUT_S
|
||||
|
||||
# extract action uses page_extraction_llm.ainvoke(..., timeout=120.0); the
|
||||
# outer per-action cap must not truncate it.
|
||||
assert _DEFAULT_ACTION_TIMEOUT_S >= 150.0, (
|
||||
f'Default action cap ({_DEFAULT_ACTION_TIMEOUT_S}s) is below the 120s '
|
||||
f'extract timeout + grace — slow but valid extractions would be killed.'
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def _restore_service_module():
|
||||
"""Reload browser_use.tools.service without any env override on teardown.
|
||||
|
||||
Tests in this file intentionally reload the module with BROWSER_USE_ACTION_TIMEOUT_S
|
||||
set to various values; without this fixture, the last reload's default leaks into
|
||||
every later test in the same worker.
|
||||
"""
|
||||
import importlib
|
||||
import os
|
||||
|
||||
import browser_use.tools.service as svc_module
|
||||
|
||||
yield svc_module
|
||||
os.environ.pop('BROWSER_USE_ACTION_TIMEOUT_S', None)
|
||||
importlib.reload(svc_module)
|
||||
|
||||
|
||||
def test_malformed_env_timeout_does_not_break_import(monkeypatch, _restore_service_module):
|
||||
"""Bad BROWSER_USE_ACTION_TIMEOUT_S values must fall back, not crash or misbehave.
|
||||
|
||||
Covers three failure modes:
|
||||
- Non-numeric / empty (ValueError from float()): would crash module import.
|
||||
- NaN: parses fine but makes asyncio.wait_for time out immediately for every action.
|
||||
- Infinity / negative / zero: parses fine but effectively disables the hang guard.
|
||||
"""
|
||||
import importlib
|
||||
|
||||
svc_module = _restore_service_module
|
||||
|
||||
bad_values = ('', 'not-a-number', 'abc', 'nan', 'NaN', 'inf', '-inf', '0', '-5')
|
||||
for bad_value in bad_values:
|
||||
monkeypatch.setenv('BROWSER_USE_ACTION_TIMEOUT_S', bad_value)
|
||||
reloaded = importlib.reload(svc_module)
|
||||
assert reloaded._DEFAULT_ACTION_TIMEOUT_S == 180.0, (
|
||||
f'Expected fallback 180.0 for bad env {bad_value!r}, got {reloaded._DEFAULT_ACTION_TIMEOUT_S}'
|
||||
)
|
||||
|
||||
# Valid finite positive values still take effect.
|
||||
monkeypatch.setenv('BROWSER_USE_ACTION_TIMEOUT_S', '45')
|
||||
reloaded = importlib.reload(svc_module)
|
||||
assert reloaded._DEFAULT_ACTION_TIMEOUT_S == 45.0
|
||||
@@ -0,0 +1,387 @@
|
||||
"""Tests for inline task planning feature.
|
||||
|
||||
Covers: plan generation, step advancement, replanning, rendering,
|
||||
disabled planning, replan nudge, flash mode schema, and edge cases.
|
||||
"""
|
||||
|
||||
import json
|
||||
|
||||
from browser_use.agent.views import (
|
||||
AgentOutput,
|
||||
PlanItem,
|
||||
)
|
||||
from browser_use.tools.service import Tools
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _make_agent_output(**overrides) -> AgentOutput:
|
||||
"""Build a minimal AgentOutput with plan fields."""
|
||||
tools = Tools()
|
||||
ActionModel = tools.registry.create_action_model()
|
||||
OutputType = AgentOutput.type_with_custom_actions(ActionModel)
|
||||
action_json = json.dumps(
|
||||
{
|
||||
'evaluation_previous_goal': 'Success',
|
||||
'memory': 'mem',
|
||||
'next_goal': 'goal',
|
||||
**{k: v for k, v in overrides.items() if k in ('current_plan_item', 'plan_update')},
|
||||
'action': [{'done': {'text': 'ok', 'success': True}}],
|
||||
}
|
||||
)
|
||||
return OutputType.model_validate_json(action_json)
|
||||
|
||||
|
||||
def _make_agent(browser_session, mock_llm, **kwargs):
|
||||
"""Create an Agent with defaults suitable for unit tests."""
|
||||
from browser_use import Agent
|
||||
|
||||
return Agent(task='Test task', llm=mock_llm, browser_session=browser_session, **kwargs)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. Plan generation from plan_update on step 1
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_plan_generation_from_plan_update(browser_session, mock_llm):
|
||||
agent = _make_agent(browser_session, mock_llm)
|
||||
output = _make_agent_output(plan_update=['Navigate to page', 'Search for item', 'Extract price'])
|
||||
|
||||
agent._update_plan_from_model_output(output)
|
||||
|
||||
assert agent.state.plan is not None
|
||||
assert len(agent.state.plan) == 3
|
||||
assert agent.state.plan[0].status == 'current'
|
||||
assert agent.state.plan[1].status == 'pending'
|
||||
assert agent.state.plan[2].status == 'pending'
|
||||
assert agent.state.current_plan_item_index == 0
|
||||
assert agent.state.plan_generation_step == agent.state.n_steps
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. Plan step advancement via current_plan_item
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_plan_step_advancement(browser_session, mock_llm):
|
||||
agent = _make_agent(browser_session, mock_llm)
|
||||
# Seed a plan
|
||||
agent.state.plan = [
|
||||
PlanItem(text='Step A', status='current'),
|
||||
PlanItem(text='Step B'),
|
||||
PlanItem(text='Step C'),
|
||||
]
|
||||
agent.state.current_plan_item_index = 0
|
||||
|
||||
output = _make_agent_output(current_plan_item=2)
|
||||
agent._update_plan_from_model_output(output)
|
||||
|
||||
assert agent.state.plan[0].status == 'done'
|
||||
assert agent.state.plan[1].status == 'done'
|
||||
assert agent.state.plan[2].status == 'current'
|
||||
assert agent.state.current_plan_item_index == 2
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. Replanning replaces old plan
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_replanning_replaces_old_plan(browser_session, mock_llm):
|
||||
agent = _make_agent(browser_session, mock_llm)
|
||||
agent.state.plan = [
|
||||
PlanItem(text='Old step 1', status='done'),
|
||||
PlanItem(text='Old step 2', status='current'),
|
||||
]
|
||||
agent.state.current_plan_item_index = 1
|
||||
agent.state.plan_generation_step = 1
|
||||
|
||||
output = _make_agent_output(plan_update=['New step A', 'New step B', 'New step C'])
|
||||
agent._update_plan_from_model_output(output)
|
||||
|
||||
assert len(agent.state.plan) == 3
|
||||
assert agent.state.plan[0].text == 'New step A'
|
||||
assert agent.state.plan[0].status == 'current'
|
||||
assert agent.state.current_plan_item_index == 0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. _render_plan_description output format
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_render_plan_description(browser_session, mock_llm):
|
||||
agent = _make_agent(browser_session, mock_llm)
|
||||
agent.state.plan = [
|
||||
PlanItem(text='Navigate to search page', status='done'),
|
||||
PlanItem(text='Search for "laptop"', status='current'),
|
||||
PlanItem(text='Extract price from results', status='pending'),
|
||||
PlanItem(text='Skipped step', status='skipped'),
|
||||
]
|
||||
|
||||
result = agent._render_plan_description()
|
||||
assert result is not None
|
||||
lines = result.split('\n')
|
||||
assert lines[0] == '[x] 0: Navigate to search page'
|
||||
assert lines[1] == '[>] 1: Search for "laptop"'
|
||||
assert lines[2] == '[ ] 2: Extract price from results'
|
||||
assert lines[3] == '[-] 3: Skipped step'
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 5. Planning disabled returns None
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_planning_disabled_returns_none(browser_session, mock_llm):
|
||||
agent = _make_agent(browser_session, mock_llm, enable_planning=False)
|
||||
agent.state.plan = [PlanItem(text='Should not render')]
|
||||
|
||||
assert agent._render_plan_description() is None
|
||||
|
||||
# Also verify update is a no-op
|
||||
output = _make_agent_output(plan_update=['New plan'])
|
||||
agent._update_plan_from_model_output(output)
|
||||
# Plan should remain unchanged (the method returns early)
|
||||
assert agent.state.plan[0].text == 'Should not render'
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 6. Replan nudge injection at threshold
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_replan_nudge_injected_at_threshold(browser_session, mock_llm):
|
||||
agent = _make_agent(browser_session, mock_llm, planning_replan_on_stall=3)
|
||||
agent.state.plan = [PlanItem(text='Step 1', status='current')]
|
||||
agent.state.consecutive_failures = 3
|
||||
|
||||
# Track context messages
|
||||
initial_count = len(agent._message_manager.state.history.context_messages)
|
||||
agent._inject_replan_nudge()
|
||||
after_count = len(agent._message_manager.state.history.context_messages)
|
||||
|
||||
assert after_count == initial_count + 1
|
||||
msg = agent._message_manager.state.history.context_messages[-1]
|
||||
assert isinstance(msg.content, str) and 'REPLAN SUGGESTED' in msg.content
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 7. No nudge below threshold
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_no_replan_nudge_below_threshold(browser_session, mock_llm):
|
||||
agent = _make_agent(browser_session, mock_llm, planning_replan_on_stall=3)
|
||||
agent.state.plan = [PlanItem(text='Step 1', status='current')]
|
||||
agent.state.consecutive_failures = 2
|
||||
|
||||
initial_count = len(agent._message_manager.state.history.context_messages)
|
||||
agent._inject_replan_nudge()
|
||||
after_count = len(agent._message_manager.state.history.context_messages)
|
||||
|
||||
assert after_count == initial_count
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 8. Flash mode schema excludes plan fields
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_flash_mode_schema_excludes_plan_fields():
|
||||
tools = Tools()
|
||||
ActionModel = tools.registry.create_action_model()
|
||||
FlashOutput = AgentOutput.type_with_custom_actions_flash_mode(ActionModel)
|
||||
|
||||
schema = FlashOutput.model_json_schema()
|
||||
assert 'current_plan_item' not in schema['properties']
|
||||
assert 'plan_update' not in schema['properties']
|
||||
assert 'thinking' not in schema['properties']
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 9. Full mode schema includes plan fields as optional
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_full_mode_schema_includes_plan_fields_optional():
|
||||
tools = Tools()
|
||||
ActionModel = tools.registry.create_action_model()
|
||||
FullOutput = AgentOutput.type_with_custom_actions(ActionModel)
|
||||
|
||||
schema = FullOutput.model_json_schema()
|
||||
assert 'current_plan_item' in schema['properties']
|
||||
assert 'plan_update' in schema['properties']
|
||||
# They should NOT be in required
|
||||
assert 'current_plan_item' not in schema.get('required', [])
|
||||
assert 'plan_update' not in schema.get('required', [])
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 10. Out-of-bounds current_plan_item handled gracefully
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_out_of_bounds_plan_step_clamped(browser_session, mock_llm):
|
||||
agent = _make_agent(browser_session, mock_llm)
|
||||
agent.state.plan = [
|
||||
PlanItem(text='Step A', status='current'),
|
||||
PlanItem(text='Step B'),
|
||||
]
|
||||
agent.state.current_plan_item_index = 0
|
||||
|
||||
# Way out of bounds high
|
||||
output = _make_agent_output(current_plan_item=999)
|
||||
agent._update_plan_from_model_output(output)
|
||||
assert agent.state.current_plan_item_index == 1 # clamped to last valid index
|
||||
assert agent.state.plan[0].status == 'done'
|
||||
assert agent.state.plan[1].status == 'current'
|
||||
|
||||
# Negative index
|
||||
agent.state.plan = [
|
||||
PlanItem(text='Step X', status='current'),
|
||||
PlanItem(text='Step Y'),
|
||||
]
|
||||
agent.state.current_plan_item_index = 1
|
||||
output2 = _make_agent_output(current_plan_item=-5)
|
||||
agent._update_plan_from_model_output(output2)
|
||||
assert agent.state.current_plan_item_index == 0 # clamped to 0
|
||||
assert agent.state.plan[0].status == 'current'
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 11. No plan means render returns None
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_no_plan_render_returns_none(browser_session, mock_llm):
|
||||
agent = _make_agent(browser_session, mock_llm)
|
||||
assert agent.state.plan is None
|
||||
assert agent._render_plan_description() is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 12. Replan nudge disabled when planning_replan_on_stall=0
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_replan_nudge_disabled_when_zero(browser_session, mock_llm):
|
||||
agent = _make_agent(browser_session, mock_llm, planning_replan_on_stall=0)
|
||||
agent.state.plan = [PlanItem(text='Step 1', status='current')]
|
||||
agent.state.consecutive_failures = 100 # high but doesn't matter
|
||||
|
||||
initial_count = len(agent._message_manager.state.history.context_messages)
|
||||
agent._inject_replan_nudge()
|
||||
after_count = len(agent._message_manager.state.history.context_messages)
|
||||
assert after_count == initial_count
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 13. No nudge when no plan exists
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_no_replan_nudge_without_plan(browser_session, mock_llm):
|
||||
agent = _make_agent(browser_session, mock_llm, planning_replan_on_stall=1)
|
||||
agent.state.consecutive_failures = 5 # above threshold
|
||||
|
||||
initial_count = len(agent._message_manager.state.history.context_messages)
|
||||
agent._inject_replan_nudge()
|
||||
after_count = len(agent._message_manager.state.history.context_messages)
|
||||
assert after_count == initial_count
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 14. Exploration nudge fires when no plan exists after N steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_exploration_nudge_fires_after_limit(browser_session, mock_llm):
|
||||
agent = _make_agent(browser_session, mock_llm, planning_exploration_limit=3)
|
||||
agent.state.plan = None
|
||||
agent.state.n_steps = 3 # at the limit
|
||||
|
||||
initial_count = len(agent._message_manager.state.history.context_messages)
|
||||
agent._inject_exploration_nudge()
|
||||
after_count = len(agent._message_manager.state.history.context_messages)
|
||||
|
||||
assert after_count == initial_count + 1
|
||||
msg = agent._message_manager.state.history.context_messages[-1]
|
||||
assert isinstance(msg.content, str) and 'PLANNING NUDGE' in msg.content
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 15. No exploration nudge when plan already exists
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_no_exploration_nudge_when_plan_exists(browser_session, mock_llm):
|
||||
agent = _make_agent(browser_session, mock_llm, planning_exploration_limit=3)
|
||||
agent.state.plan = [PlanItem(text='Step 1', status='current')]
|
||||
agent.state.n_steps = 10 # well above limit
|
||||
|
||||
initial_count = len(agent._message_manager.state.history.context_messages)
|
||||
agent._inject_exploration_nudge()
|
||||
after_count = len(agent._message_manager.state.history.context_messages)
|
||||
assert after_count == initial_count
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 16. No exploration nudge below the limit
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_no_exploration_nudge_below_limit(browser_session, mock_llm):
|
||||
agent = _make_agent(browser_session, mock_llm, planning_exploration_limit=5)
|
||||
agent.state.plan = None
|
||||
agent.state.n_steps = 4 # below the limit
|
||||
|
||||
initial_count = len(agent._message_manager.state.history.context_messages)
|
||||
agent._inject_exploration_nudge()
|
||||
after_count = len(agent._message_manager.state.history.context_messages)
|
||||
assert after_count == initial_count
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 17. Exploration nudge disabled when planning_exploration_limit=0
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_exploration_nudge_disabled_when_zero(browser_session, mock_llm):
|
||||
agent = _make_agent(browser_session, mock_llm, planning_exploration_limit=0)
|
||||
agent.state.plan = None
|
||||
agent.state.n_steps = 100 # high but doesn't matter
|
||||
|
||||
initial_count = len(agent._message_manager.state.history.context_messages)
|
||||
agent._inject_exploration_nudge()
|
||||
after_count = len(agent._message_manager.state.history.context_messages)
|
||||
assert after_count == initial_count
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 18. Exploration nudge disabled when enable_planning=False
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_exploration_nudge_disabled_when_planning_off(browser_session, mock_llm):
|
||||
agent = _make_agent(browser_session, mock_llm, enable_planning=False, planning_exploration_limit=3)
|
||||
agent.state.plan = None
|
||||
agent.state.n_steps = 10 # above limit
|
||||
|
||||
initial_count = len(agent._message_manager.state.history.context_messages)
|
||||
agent._inject_exploration_nudge()
|
||||
after_count = len(agent._message_manager.state.history.context_messages)
|
||||
assert after_count == initial_count
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 19. Flash mode forces enable_planning=False
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_flash_mode_disables_planning(browser_session, mock_llm):
|
||||
agent = _make_agent(browser_session, mock_llm, flash_mode=True)
|
||||
assert agent.settings.enable_planning is False
|
||||
@@ -0,0 +1,120 @@
|
||||
"""Tests for AI step private method used during rerun"""
|
||||
|
||||
from unittest.mock import AsyncMock
|
||||
|
||||
from browser_use.agent.service import Agent
|
||||
from browser_use.agent.views import ActionResult
|
||||
from tests.ci.conftest import create_mock_llm
|
||||
|
||||
|
||||
async def test_execute_ai_step_basic():
|
||||
"""Test that _execute_ai_step extracts content with AI"""
|
||||
|
||||
# Create mock LLM that returns text response
|
||||
async def custom_ainvoke(*args, **kwargs):
|
||||
from browser_use.llm.views import ChatInvokeCompletion
|
||||
|
||||
return ChatInvokeCompletion(completion='Extracted: Test content from page', usage=None)
|
||||
|
||||
mock_llm = AsyncMock()
|
||||
mock_llm.ainvoke.side_effect = custom_ainvoke
|
||||
mock_llm.model = 'mock-model'
|
||||
|
||||
llm = create_mock_llm(actions=None)
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
await agent.browser_session.start()
|
||||
|
||||
try:
|
||||
# Execute _execute_ai_step with mock LLM
|
||||
result = await agent._execute_ai_step(
|
||||
query='Extract the main heading',
|
||||
include_screenshot=False,
|
||||
extract_links=False,
|
||||
ai_step_llm=mock_llm,
|
||||
)
|
||||
|
||||
# Verify result
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'Extracted: Test content from page' in result.extracted_content
|
||||
assert result.long_term_memory is not None
|
||||
|
||||
finally:
|
||||
await agent.close()
|
||||
|
||||
|
||||
async def test_execute_ai_step_with_screenshot():
|
||||
"""Test that _execute_ai_step includes screenshot when requested"""
|
||||
|
||||
# Create mock LLM
|
||||
async def custom_ainvoke(*args, **kwargs):
|
||||
from browser_use.llm.views import ChatInvokeCompletion
|
||||
|
||||
# Verify that we received a message with image content
|
||||
messages = args[0] if args else []
|
||||
assert len(messages) >= 1, 'Should have at least one message'
|
||||
|
||||
# Check if any message has image content
|
||||
has_image = False
|
||||
for msg in messages:
|
||||
if hasattr(msg, 'content') and isinstance(msg.content, list):
|
||||
for part in msg.content:
|
||||
if hasattr(part, 'type') and part.type == 'image_url':
|
||||
has_image = True
|
||||
break
|
||||
|
||||
assert has_image, 'Should include screenshot in message'
|
||||
return ChatInvokeCompletion(completion='Extracted content with screenshot analysis', usage=None)
|
||||
|
||||
mock_llm = AsyncMock()
|
||||
mock_llm.ainvoke.side_effect = custom_ainvoke
|
||||
mock_llm.model = 'mock-model'
|
||||
|
||||
llm = create_mock_llm(actions=None)
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
await agent.browser_session.start()
|
||||
|
||||
try:
|
||||
# Execute _execute_ai_step with screenshot
|
||||
result = await agent._execute_ai_step(
|
||||
query='Analyze this page',
|
||||
include_screenshot=True,
|
||||
extract_links=False,
|
||||
ai_step_llm=mock_llm,
|
||||
)
|
||||
|
||||
# Verify result
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'Extracted content with screenshot analysis' in result.extracted_content
|
||||
|
||||
finally:
|
||||
await agent.close()
|
||||
|
||||
|
||||
async def test_execute_ai_step_error_handling():
|
||||
"""Test that _execute_ai_step handles errors gracefully"""
|
||||
# Create mock LLM that raises an error
|
||||
mock_llm = AsyncMock()
|
||||
mock_llm.ainvoke.side_effect = Exception('LLM service unavailable')
|
||||
mock_llm.model = 'mock-model'
|
||||
|
||||
llm = create_mock_llm(actions=None)
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
await agent.browser_session.start()
|
||||
|
||||
try:
|
||||
# Execute _execute_ai_step - should return ActionResult with error
|
||||
result = await agent._execute_ai_step(
|
||||
query='Extract data',
|
||||
include_screenshot=False,
|
||||
ai_step_llm=mock_llm,
|
||||
)
|
||||
|
||||
# Verify error is in result (not raised)
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is not None
|
||||
assert 'AI step failed' in result.error
|
||||
|
||||
finally:
|
||||
await agent.close()
|
||||
@@ -0,0 +1,558 @@
|
||||
"""Tests for ax_name (accessible name) element matching in history rerun.
|
||||
|
||||
This tests Level 4 matching which uses the accessibility tree's name property
|
||||
to match elements when hash, stable_hash, and xpath all fail.
|
||||
This is particularly useful for dynamic SPAs where DOM structure changes
|
||||
but accessible names remain stable.
|
||||
|
||||
Also tests dropdown/menu re-opening behavior when menu items can't be found
|
||||
because the dropdown closed during the wait between steps.
|
||||
"""
|
||||
|
||||
from unittest.mock import AsyncMock
|
||||
|
||||
from browser_use.agent.service import Agent
|
||||
from browser_use.agent.views import ActionResult, AgentHistory, AgentHistoryList, RerunSummaryAction, StepMetadata
|
||||
from browser_use.browser.views import BrowserStateHistory
|
||||
from browser_use.dom.views import DOMInteractedElement, DOMRect, MatchLevel, NodeType
|
||||
from tests.ci.conftest import create_mock_llm
|
||||
|
||||
|
||||
async def test_ax_name_matching_succeeds_when_hash_fails(httpserver):
|
||||
"""Test that ax_name matching finds elements when hash/xpath matching fails.
|
||||
|
||||
This simulates a dynamic SPA where the element hash and xpath change between
|
||||
sessions, but the accessible name (ax_name) remains stable.
|
||||
"""
|
||||
# Set up a test page with a menu item that has an aria-label
|
||||
# The aria-label becomes the accessible name (ax_name)
|
||||
test_html = """<!DOCTYPE html>
|
||||
<html>
|
||||
<body>
|
||||
<div role="menuitem" aria-label="New Contact" id="menu-1">New Contact</div>
|
||||
<div role="menuitem" aria-label="Search" id="menu-2">Search</div>
|
||||
</body>
|
||||
</html>"""
|
||||
httpserver.expect_request('/test').respond_with_data(test_html, content_type='text/html')
|
||||
test_url = httpserver.url_for('/test')
|
||||
|
||||
# Create a mock LLM for summary
|
||||
summary_action = RerunSummaryAction(
|
||||
summary='Rerun completed',
|
||||
success=True,
|
||||
completion_status='complete',
|
||||
)
|
||||
|
||||
async def custom_ainvoke(*args, **kwargs):
|
||||
output_format = args[1] if len(args) > 1 else kwargs.get('output_format')
|
||||
if output_format is RerunSummaryAction:
|
||||
from browser_use.llm.views import ChatInvokeCompletion
|
||||
|
||||
return ChatInvokeCompletion(completion=summary_action, usage=None)
|
||||
raise ValueError('Unexpected output_format')
|
||||
|
||||
mock_summary_llm = AsyncMock()
|
||||
mock_summary_llm.ainvoke.side_effect = custom_ainvoke
|
||||
|
||||
llm = create_mock_llm(actions=None)
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
AgentOutput = agent.AgentOutput
|
||||
|
||||
# Create an element with DIFFERENT hash/xpath but SAME ax_name as the real element
|
||||
# This simulates what happens in dynamic SPAs where the DOM changes but
|
||||
# accessible names remain stable
|
||||
historical_element = DOMInteractedElement(
|
||||
node_id=9999, # Different node_id
|
||||
backend_node_id=9999, # Different backend_node_id
|
||||
frame_id=None,
|
||||
node_type=NodeType.ELEMENT_NODE,
|
||||
node_value='',
|
||||
node_name='DIV', # Same node type
|
||||
# Note: aria-label is NOT in attributes - this tests that ax_name matching
|
||||
# is used as a fallback when attribute matching fails
|
||||
attributes={'role': 'menuitem', 'class': 'dynamic-class-12345'},
|
||||
x_path='html/body/div[1]/div[4]/div[4]/div[1]', # Different xpath
|
||||
element_hash=123456789, # Different hash (won't match)
|
||||
stable_hash=987654321, # Different stable_hash (won't match)
|
||||
bounds=DOMRect(x=0, y=0, width=100, height=50),
|
||||
ax_name='New Contact', # SAME ax_name - this should match!
|
||||
)
|
||||
|
||||
# Step 1: Navigate to test page
|
||||
navigate_step = AgentHistory(
|
||||
model_output=AgentOutput(
|
||||
evaluation_previous_goal=None,
|
||||
memory='Navigate to test page',
|
||||
next_goal=None,
|
||||
action=[{'navigate': {'url': test_url}}], # type: ignore[arg-type]
|
||||
),
|
||||
result=[ActionResult(long_term_memory='Navigated')],
|
||||
state=BrowserStateHistory(
|
||||
url=test_url,
|
||||
title='Test Page',
|
||||
tabs=[],
|
||||
interacted_element=[None],
|
||||
),
|
||||
metadata=StepMetadata(
|
||||
step_start_time=0,
|
||||
step_end_time=1,
|
||||
step_number=1,
|
||||
step_interval=0.1,
|
||||
),
|
||||
)
|
||||
|
||||
# Step 2: Click on element that has different hash/xpath but same ax_name
|
||||
click_step = AgentHistory(
|
||||
model_output=AgentOutput(
|
||||
evaluation_previous_goal=None,
|
||||
memory='Click New Contact menu',
|
||||
next_goal=None,
|
||||
action=[{'click': {'index': 100}}], # type: ignore[arg-type] # Original index doesn't matter
|
||||
),
|
||||
result=[ActionResult(long_term_memory='Clicked New Contact')],
|
||||
state=BrowserStateHistory(
|
||||
url=test_url,
|
||||
title='Test Page',
|
||||
tabs=[],
|
||||
interacted_element=[historical_element],
|
||||
),
|
||||
metadata=StepMetadata(
|
||||
step_start_time=1,
|
||||
step_end_time=2,
|
||||
step_number=2,
|
||||
step_interval=0.1,
|
||||
),
|
||||
)
|
||||
|
||||
history = AgentHistoryList(history=[navigate_step, click_step])
|
||||
|
||||
try:
|
||||
# Run rerun - should succeed because ax_name matching finds the element
|
||||
results = await agent.rerun_history(
|
||||
history,
|
||||
skip_failures=False,
|
||||
max_retries=1,
|
||||
summary_llm=mock_summary_llm,
|
||||
)
|
||||
|
||||
# Should have 3 results: navigate + click + AI summary
|
||||
assert len(results) == 3
|
||||
|
||||
# First result should be navigation success
|
||||
nav_result = results[0]
|
||||
assert nav_result.error is None
|
||||
|
||||
# Second result should be click success (matched via ax_name)
|
||||
click_result = results[1]
|
||||
assert click_result.error is None, f'Click should succeed via ax_name matching, got error: {click_result.error}'
|
||||
|
||||
# Third result should be AI summary
|
||||
summary_result = results[2]
|
||||
assert summary_result.is_done is True
|
||||
|
||||
finally:
|
||||
await agent.close()
|
||||
|
||||
|
||||
async def test_ax_name_matching_requires_same_node_type(httpserver):
|
||||
"""Test that ax_name matching also requires matching node type.
|
||||
|
||||
Even if ax_name matches, the node type (DIV, BUTTON, etc.) must also match.
|
||||
"""
|
||||
test_html = """<!DOCTYPE html>
|
||||
<html>
|
||||
<body>
|
||||
<button aria-label="Submit">Submit</button>
|
||||
<div aria-label="Submit">Submit Label</div>
|
||||
</body>
|
||||
</html>"""
|
||||
httpserver.expect_request('/test').respond_with_data(test_html, content_type='text/html')
|
||||
test_url = httpserver.url_for('/test')
|
||||
|
||||
llm = create_mock_llm(actions=None)
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
AgentOutput = agent.AgentOutput
|
||||
|
||||
# Historical element is a SPAN with ax_name "Submit"
|
||||
# Page has BUTTON and DIV with same ax_name, but no SPAN
|
||||
historical_element = DOMInteractedElement(
|
||||
node_id=1,
|
||||
backend_node_id=1,
|
||||
frame_id=None,
|
||||
node_type=NodeType.ELEMENT_NODE,
|
||||
node_value='',
|
||||
node_name='SPAN', # SPAN - won't match BUTTON or DIV
|
||||
attributes={},
|
||||
x_path='html/body/span',
|
||||
element_hash=111,
|
||||
stable_hash=111,
|
||||
bounds=DOMRect(x=0, y=0, width=100, height=50),
|
||||
ax_name='Submit', # Same ax_name, but wrong node type
|
||||
)
|
||||
|
||||
navigate_step = AgentHistory(
|
||||
model_output=AgentOutput(
|
||||
evaluation_previous_goal=None,
|
||||
memory='Navigate',
|
||||
next_goal=None,
|
||||
action=[{'navigate': {'url': test_url}}], # type: ignore[arg-type]
|
||||
),
|
||||
result=[ActionResult(long_term_memory='Navigated')],
|
||||
state=BrowserStateHistory(
|
||||
url=test_url,
|
||||
title='Test',
|
||||
tabs=[],
|
||||
interacted_element=[None],
|
||||
),
|
||||
metadata=StepMetadata(step_start_time=0, step_end_time=1, step_number=1, step_interval=0.1),
|
||||
)
|
||||
|
||||
click_step = AgentHistory(
|
||||
model_output=AgentOutput(
|
||||
evaluation_previous_goal=None,
|
||||
memory='Click Submit',
|
||||
next_goal=None,
|
||||
action=[{'click': {'index': 1}}], # type: ignore[arg-type]
|
||||
),
|
||||
result=[ActionResult(long_term_memory='Clicked')],
|
||||
state=BrowserStateHistory(
|
||||
url=test_url,
|
||||
title='Test',
|
||||
tabs=[],
|
||||
interacted_element=[historical_element],
|
||||
),
|
||||
metadata=StepMetadata(step_start_time=1, step_end_time=2, step_number=2, step_interval=0.1),
|
||||
)
|
||||
|
||||
history = AgentHistoryList(history=[navigate_step, click_step])
|
||||
|
||||
try:
|
||||
# Should fail because no SPAN with ax_name "Submit" exists
|
||||
await agent.rerun_history(
|
||||
history,
|
||||
skip_failures=False,
|
||||
max_retries=1,
|
||||
)
|
||||
assert False, 'Expected RuntimeError - no matching SPAN element'
|
||||
except RuntimeError as e:
|
||||
# Expected - no SPAN element with ax_name "Submit"
|
||||
assert 'failed after 1 attempts' in str(e)
|
||||
finally:
|
||||
await agent.close()
|
||||
|
||||
|
||||
def test_match_level_enum_includes_ax_name():
|
||||
"""Test that MatchLevel enum includes AX_NAME level."""
|
||||
assert hasattr(MatchLevel, 'AX_NAME')
|
||||
assert MatchLevel.AX_NAME.value == 4
|
||||
assert MatchLevel.ATTRIBUTE.value == 5 # AX_NAME comes before ATTRIBUTE
|
||||
|
||||
|
||||
async def test_ax_name_matching_before_attribute_matching(httpserver):
|
||||
"""Test that ax_name matching (Level 4) is tried before attribute matching (Level 5).
|
||||
|
||||
This ensures the correct matching order: EXACT -> STABLE -> XPATH -> AX_NAME -> ATTRIBUTE
|
||||
"""
|
||||
# Page has element with text content that becomes its ax_name
|
||||
# The DIV has role="menuitem" and text "Contact" which becomes its accessible name
|
||||
# but NO aria-label/id/name attributes - so attribute matching will fail but ax_name should work
|
||||
test_html = """<!DOCTYPE html>
|
||||
<html>
|
||||
<body>
|
||||
<div role="menuitem">Contact</div>
|
||||
</body>
|
||||
</html>"""
|
||||
httpserver.expect_request('/test').respond_with_data(test_html, content_type='text/html')
|
||||
test_url = httpserver.url_for('/test')
|
||||
|
||||
summary_action = RerunSummaryAction(
|
||||
summary='Rerun completed',
|
||||
success=True,
|
||||
completion_status='complete',
|
||||
)
|
||||
|
||||
async def custom_ainvoke(*args, **kwargs):
|
||||
output_format = args[1] if len(args) > 1 else kwargs.get('output_format')
|
||||
if output_format is RerunSummaryAction:
|
||||
from browser_use.llm.views import ChatInvokeCompletion
|
||||
|
||||
return ChatInvokeCompletion(completion=summary_action, usage=None)
|
||||
raise ValueError('Unexpected output_format')
|
||||
|
||||
mock_summary_llm = AsyncMock()
|
||||
mock_summary_llm.ainvoke.side_effect = custom_ainvoke
|
||||
|
||||
llm = create_mock_llm(actions=None)
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
AgentOutput = agent.AgentOutput
|
||||
|
||||
# Historical element has NO aria-label attribute (attribute matching will fail)
|
||||
# but HAS ax_name (ax_name matching should work)
|
||||
historical_element = DOMInteractedElement(
|
||||
node_id=1,
|
||||
backend_node_id=1,
|
||||
frame_id=None,
|
||||
node_type=NodeType.ELEMENT_NODE,
|
||||
node_value='',
|
||||
node_name='DIV',
|
||||
# No aria-label, id, or name - attribute matching will fail
|
||||
attributes={'role': 'menuitem'},
|
||||
x_path='html/body/div[99]', # Wrong xpath
|
||||
element_hash=12345, # Wrong hash
|
||||
stable_hash=12345, # Wrong stable hash
|
||||
bounds=DOMRect(x=0, y=0, width=100, height=50),
|
||||
ax_name='Contact', # ax_name from accessibility tree
|
||||
)
|
||||
|
||||
navigate_step = AgentHistory(
|
||||
model_output=AgentOutput(
|
||||
evaluation_previous_goal=None,
|
||||
memory='Navigate',
|
||||
next_goal=None,
|
||||
action=[{'navigate': {'url': test_url}}], # type: ignore[arg-type]
|
||||
),
|
||||
result=[ActionResult(long_term_memory='Navigated')],
|
||||
state=BrowserStateHistory(
|
||||
url=test_url,
|
||||
title='Test',
|
||||
tabs=[],
|
||||
interacted_element=[None],
|
||||
),
|
||||
metadata=StepMetadata(step_start_time=0, step_end_time=1, step_number=1, step_interval=0.1),
|
||||
)
|
||||
|
||||
click_step = AgentHistory(
|
||||
model_output=AgentOutput(
|
||||
evaluation_previous_goal=None,
|
||||
memory='Click Contact',
|
||||
next_goal=None,
|
||||
action=[{'click': {'index': 1}}], # type: ignore[arg-type]
|
||||
),
|
||||
result=[ActionResult(long_term_memory='Clicked')],
|
||||
state=BrowserStateHistory(
|
||||
url=test_url,
|
||||
title='Test',
|
||||
tabs=[],
|
||||
interacted_element=[historical_element],
|
||||
),
|
||||
metadata=StepMetadata(step_start_time=1, step_end_time=2, step_number=2, step_interval=0.1),
|
||||
)
|
||||
|
||||
history = AgentHistoryList(history=[navigate_step, click_step])
|
||||
|
||||
try:
|
||||
# Should succeed via ax_name matching (Level 4)
|
||||
# since hash, stable_hash, xpath all fail but ax_name matches
|
||||
results = await agent.rerun_history(
|
||||
history,
|
||||
skip_failures=False,
|
||||
max_retries=1,
|
||||
summary_llm=mock_summary_llm,
|
||||
)
|
||||
|
||||
# Navigation + click + summary
|
||||
assert len(results) == 3
|
||||
# Click should succeed (matched via ax_name)
|
||||
click_result = results[1]
|
||||
assert click_result.error is None, f'Expected ax_name match to succeed, got: {click_result.error}'
|
||||
|
||||
finally:
|
||||
await agent.close()
|
||||
|
||||
|
||||
# Tests for dropdown/menu re-opening behavior
|
||||
|
||||
|
||||
def test_is_menu_opener_step_detects_aria_haspopup():
|
||||
"""Test that _is_menu_opener_step detects aria-haspopup elements."""
|
||||
llm = create_mock_llm(actions=None)
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
AgentOutput = agent.AgentOutput
|
||||
|
||||
# Element with aria-haspopup="true" should be detected as menu opener
|
||||
opener_element = DOMInteractedElement(
|
||||
node_id=1,
|
||||
backend_node_id=1,
|
||||
frame_id=None,
|
||||
node_type=NodeType.ELEMENT_NODE,
|
||||
node_value='',
|
||||
node_name='DIV',
|
||||
attributes={'aria-haspopup': 'true', 'class': 'dropdown-trigger'},
|
||||
x_path='html/body/div',
|
||||
element_hash=12345,
|
||||
stable_hash=12345,
|
||||
bounds=DOMRect(x=0, y=0, width=100, height=50),
|
||||
ax_name='Contact',
|
||||
)
|
||||
|
||||
history_item = AgentHistory(
|
||||
model_output=AgentOutput(
|
||||
evaluation_previous_goal=None,
|
||||
memory='Click dropdown',
|
||||
next_goal=None,
|
||||
action=[{'click': {'index': 1}}], # type: ignore[arg-type]
|
||||
),
|
||||
result=[ActionResult(long_term_memory='Clicked')],
|
||||
state=BrowserStateHistory(
|
||||
url='http://test.com',
|
||||
title='Test',
|
||||
tabs=[],
|
||||
interacted_element=[opener_element],
|
||||
),
|
||||
metadata=StepMetadata(step_start_time=0, step_end_time=1, step_number=1, step_interval=0.1),
|
||||
)
|
||||
|
||||
assert agent._is_menu_opener_step(history_item) is True
|
||||
|
||||
|
||||
def test_is_menu_opener_step_detects_guidewire_toggle():
|
||||
"""Test that _is_menu_opener_step detects Guidewire toggleSubMenu pattern."""
|
||||
llm = create_mock_llm(actions=None)
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
AgentOutput = agent.AgentOutput
|
||||
|
||||
# Element with data-gw-click="toggleSubMenu" should be detected
|
||||
opener_element = DOMInteractedElement(
|
||||
node_id=1,
|
||||
backend_node_id=1,
|
||||
frame_id=None,
|
||||
node_type=NodeType.ELEMENT_NODE,
|
||||
node_value='',
|
||||
node_name='DIV',
|
||||
attributes={'data-gw-click': 'toggleSubMenu', 'class': 'gw-action--expand-button'},
|
||||
x_path='html/body/div',
|
||||
element_hash=12345,
|
||||
stable_hash=12345,
|
||||
bounds=DOMRect(x=0, y=0, width=100, height=50),
|
||||
ax_name=None,
|
||||
)
|
||||
|
||||
history_item = AgentHistory(
|
||||
model_output=AgentOutput(
|
||||
evaluation_previous_goal=None,
|
||||
memory='Toggle menu',
|
||||
next_goal=None,
|
||||
action=[{'click': {'index': 1}}], # type: ignore[arg-type]
|
||||
),
|
||||
result=[ActionResult(long_term_memory='Toggled')],
|
||||
state=BrowserStateHistory(
|
||||
url='http://test.com',
|
||||
title='Test',
|
||||
tabs=[],
|
||||
interacted_element=[opener_element],
|
||||
),
|
||||
metadata=StepMetadata(step_start_time=0, step_end_time=1, step_number=1, step_interval=0.1),
|
||||
)
|
||||
|
||||
assert agent._is_menu_opener_step(history_item) is True
|
||||
|
||||
|
||||
def test_is_menu_opener_step_returns_false_for_regular_element():
|
||||
"""Test that _is_menu_opener_step returns False for non-menu elements."""
|
||||
llm = create_mock_llm(actions=None)
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
AgentOutput = agent.AgentOutput
|
||||
|
||||
# Regular button without menu attributes
|
||||
regular_element = DOMInteractedElement(
|
||||
node_id=1,
|
||||
backend_node_id=1,
|
||||
frame_id=None,
|
||||
node_type=NodeType.ELEMENT_NODE,
|
||||
node_value='',
|
||||
node_name='BUTTON',
|
||||
attributes={'class': 'submit-btn', 'type': 'submit'},
|
||||
x_path='html/body/button',
|
||||
element_hash=12345,
|
||||
stable_hash=12345,
|
||||
bounds=DOMRect(x=0, y=0, width=100, height=50),
|
||||
ax_name='Submit',
|
||||
)
|
||||
|
||||
history_item = AgentHistory(
|
||||
model_output=AgentOutput(
|
||||
evaluation_previous_goal=None,
|
||||
memory='Click submit',
|
||||
next_goal=None,
|
||||
action=[{'click': {'index': 1}}], # type: ignore[arg-type]
|
||||
),
|
||||
result=[ActionResult(long_term_memory='Clicked')],
|
||||
state=BrowserStateHistory(
|
||||
url='http://test.com',
|
||||
title='Test',
|
||||
tabs=[],
|
||||
interacted_element=[regular_element],
|
||||
),
|
||||
metadata=StepMetadata(step_start_time=0, step_end_time=1, step_number=1, step_interval=0.1),
|
||||
)
|
||||
|
||||
assert agent._is_menu_opener_step(history_item) is False
|
||||
|
||||
|
||||
def test_is_menu_item_element_detects_role_menuitem():
|
||||
"""Test that _is_menu_item_element detects role=menuitem."""
|
||||
llm = create_mock_llm(actions=None)
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
|
||||
menu_item = DOMInteractedElement(
|
||||
node_id=1,
|
||||
backend_node_id=1,
|
||||
frame_id=None,
|
||||
node_type=NodeType.ELEMENT_NODE,
|
||||
node_value='',
|
||||
node_name='DIV',
|
||||
attributes={'role': 'menuitem', 'class': 'menu-option'},
|
||||
x_path='html/body/div/div',
|
||||
element_hash=12345,
|
||||
stable_hash=12345,
|
||||
bounds=DOMRect(x=0, y=0, width=100, height=50),
|
||||
ax_name='New Contact',
|
||||
)
|
||||
|
||||
assert agent._is_menu_item_element(menu_item) is True
|
||||
|
||||
|
||||
def test_is_menu_item_element_detects_guidewire_class():
|
||||
"""Test that _is_menu_item_element detects Guidewire gw-action--inner class."""
|
||||
llm = create_mock_llm(actions=None)
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
|
||||
menu_item = DOMInteractedElement(
|
||||
node_id=1,
|
||||
backend_node_id=1,
|
||||
frame_id=None,
|
||||
node_type=NodeType.ELEMENT_NODE,
|
||||
node_value='',
|
||||
node_name='DIV',
|
||||
attributes={'class': 'gw-action--inner gw-hasDivider', 'aria-haspopup': 'true'},
|
||||
x_path='html/body/div/div',
|
||||
element_hash=12345,
|
||||
stable_hash=12345,
|
||||
bounds=DOMRect(x=0, y=0, width=100, height=50),
|
||||
ax_name='New Contact',
|
||||
)
|
||||
|
||||
assert agent._is_menu_item_element(menu_item) is True
|
||||
|
||||
|
||||
def test_is_menu_item_element_returns_false_for_regular_element():
|
||||
"""Test that _is_menu_item_element returns False for non-menu elements."""
|
||||
llm = create_mock_llm(actions=None)
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
|
||||
regular_element = DOMInteractedElement(
|
||||
node_id=1,
|
||||
backend_node_id=1,
|
||||
frame_id=None,
|
||||
node_type=NodeType.ELEMENT_NODE,
|
||||
node_value='',
|
||||
node_name='BUTTON',
|
||||
attributes={'class': 'submit-btn', 'type': 'submit'},
|
||||
x_path='html/body/button',
|
||||
element_hash=12345,
|
||||
stable_hash=12345,
|
||||
bounds=DOMRect(x=0, y=0, width=100, height=50),
|
||||
ax_name='Submit',
|
||||
)
|
||||
|
||||
assert agent._is_menu_item_element(regular_element) is False
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,46 @@
|
||||
"""Regression test: BrowserError raised inside an action must reach Tools.act with its
|
||||
structured short/long-term memory intact, instead of being flattened into a generic
|
||||
'Error executing action ...' RuntimeError by execute_action's catch-all handler."""
|
||||
|
||||
from browser_use.browser.views import BrowserError
|
||||
from browser_use.tools.service import Tools
|
||||
|
||||
|
||||
async def test_browser_error_memory_survives_execute_action():
|
||||
tools = Tools()
|
||||
|
||||
@tools.registry.action(description='Test action that raises a structured BrowserError')
|
||||
async def raise_structured_error():
|
||||
raise BrowserError(
|
||||
message='element is a select, not clickable',
|
||||
short_term_memory='Available options: alpha, beta, gamma',
|
||||
long_term_memory='Tried to click a dropdown; use select_dropdown instead',
|
||||
)
|
||||
|
||||
ActionModel = tools.registry.create_action_model()
|
||||
action = ActionModel(**{'raise_structured_error': {}})
|
||||
|
||||
result = await tools.act(action, browser_session=None) # type: ignore[arg-type] -- action doesn't touch the browser
|
||||
|
||||
assert result.error == 'Tried to click a dropdown; use select_dropdown instead', (
|
||||
f'structured long_term_memory lost: {result.error!r}'
|
||||
)
|
||||
assert result.extracted_content == 'Available options: alpha, beta, gamma'
|
||||
|
||||
|
||||
async def test_plain_browser_error_still_returns_recoverable_action_result():
|
||||
"""A BrowserError without long_term_memory must not escape Tools.act as an
|
||||
exception — it must still come back as a recoverable ActionResult (as it did
|
||||
when the generic execute_action handler flattened it)."""
|
||||
tools = Tools()
|
||||
|
||||
@tools.registry.action(description='Test action that raises a plain BrowserError')
|
||||
async def raise_plain_error():
|
||||
raise BrowserError(message='element with index 5 does not exist')
|
||||
|
||||
ActionModel = tools.registry.create_action_model()
|
||||
action = ActionModel(**{'raise_plain_error': {}})
|
||||
|
||||
result = await tools.act(action, browser_session=None) # type: ignore[arg-type] -- action doesn't touch the browser
|
||||
|
||||
assert result.error is not None and 'element with index 5 does not exist' in result.error
|
||||
@@ -0,0 +1,48 @@
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
|
||||
|
||||
def _run_browser_use_cli(*args: str) -> subprocess.CompletedProcess[str]:
|
||||
env = os.environ.copy()
|
||||
env['PYTHONPATH'] = os.pathsep.join(part for part in (str(ROOT), env.get('PYTHONPATH', '')) if part)
|
||||
return subprocess.run(
|
||||
[sys.executable, '-m', 'browser_use.cli', *args],
|
||||
cwd=ROOT,
|
||||
env=env,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=20,
|
||||
)
|
||||
|
||||
|
||||
def test_browser_use_doctor_help_prints_browser_use_usage():
|
||||
result = _run_browser_use_cli('doctor', '--help')
|
||||
|
||||
assert result.returncode == 0
|
||||
assert result.stdout == 'usage: browser-use doctor [--fix-snap]\n'
|
||||
assert result.stderr == ''
|
||||
|
||||
|
||||
def test_normalize_captured_cli_output_handles_string_system_exit(capsys):
|
||||
from browser_use.cli import _normalize_captured_cli_output
|
||||
|
||||
def exits_with_string(_argv):
|
||||
raise SystemExit('browser-harness failed')
|
||||
|
||||
assert _normalize_captured_cli_output(exits_with_string, []) == 1
|
||||
captured = capsys.readouterr()
|
||||
assert captured.out == ''
|
||||
assert captured.err == 'browser-use failed\n'
|
||||
|
||||
|
||||
def test_browser_use_tui_is_deprecated_alias(monkeypatch, capsys):
|
||||
import browser_use.cli as browser_use_cli
|
||||
|
||||
monkeypatch.setattr(browser_use_cli, 'main', lambda: 0)
|
||||
|
||||
assert browser_use_cli.browser_use_tui_main() == 0
|
||||
assert capsys.readouterr().err == 'browser-use-tui is deprecated; use browser-use instead.\n'
|
||||
@@ -0,0 +1,119 @@
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
BROWSER_USE_REPO_SKILL_URL = 'https://raw.githubusercontent.com/browser-use/browser-use/main/skills/browser-use/SKILL.md'
|
||||
EXPECTED_SKILL_INSTALL_PATHS = (
|
||||
Path('.agents') / 'skills' / 'browser-use' / 'SKILL.md',
|
||||
Path('.claude') / 'skills' / 'browser-use' / 'SKILL.md',
|
||||
Path('.codex') / 'skills' / 'browser-use' / 'SKILL.md',
|
||||
Path('.copilot') / 'skills' / 'browser-use' / 'SKILL.md',
|
||||
Path('.cursor') / 'skills' / 'browser-use' / 'SKILL.md',
|
||||
Path('.gemini') / 'skills' / 'browser-use' / 'SKILL.md',
|
||||
Path('.config') / 'opencode' / 'skills' / 'browser-use' / 'SKILL.md',
|
||||
)
|
||||
|
||||
|
||||
def _fake_browser_harness_tools(tmp_path: Path, skill_text: str) -> Path:
|
||||
bin_dir = tmp_path / 'bin'
|
||||
bin_dir.mkdir()
|
||||
|
||||
uv = bin_dir / 'uv'
|
||||
uv.write_text(
|
||||
'#!/usr/bin/env python3\n'
|
||||
'import os, pathlib, sys\n'
|
||||
'pathlib.Path(os.environ["UV_TOOL_INSTALL_ARGS_FILE"]).write_text(" ".join(sys.argv[1:]), encoding="utf-8")\n',
|
||||
encoding='utf-8',
|
||||
)
|
||||
uv.chmod(0o755)
|
||||
|
||||
browser_harness = bin_dir / 'browser-harness'
|
||||
browser_harness.write_text(
|
||||
'#!/usr/bin/env python3\n'
|
||||
'import sys\n'
|
||||
f'text = {skill_text!r}\n'
|
||||
'if sys.argv[1:] == ["skill"]:\n'
|
||||
' print(text, end="")\n'
|
||||
'else:\n'
|
||||
' print("usage: browser-harness skill", file=sys.stderr)\n'
|
||||
' sys.exit(2)\n',
|
||||
encoding='utf-8',
|
||||
)
|
||||
browser_harness.chmod(0o755)
|
||||
return bin_dir
|
||||
|
||||
|
||||
def test_docs_install_browser_use_skill_from_package_alias():
|
||||
readme = (ROOT / 'README.md').read_text(encoding='utf-8')
|
||||
|
||||
assert 'run `browser-use skill install` to register the skill' in readme
|
||||
assert 'mkdir -p ~/.claude/skills/browser-use' not in readme
|
||||
assert 'uv run --with "browser-use[browser-harness]" python -c' not in readme
|
||||
assert 'from browser_use.skills import browser_use_skill_text' not in readme
|
||||
assert BROWSER_USE_REPO_SKILL_URL not in readme
|
||||
assert 'raw.githubusercontent.com/browser-use/browser-harness/main/SKILL.md' not in readme
|
||||
|
||||
|
||||
def test_browser_use_cli_installs_browser_harness_package_skill(tmp_path):
|
||||
bin_dir = _fake_browser_harness_tools(tmp_path, '---\nname: browser-harness\n---\n\n# Browser Harness\n')
|
||||
|
||||
home = tmp_path / 'home'
|
||||
for stale in (home / path for path in EXPECTED_SKILL_INSTALL_PATHS):
|
||||
stale.parent.mkdir(parents=True)
|
||||
stale.write_text('stale browser-use skill', encoding='utf-8')
|
||||
|
||||
uv_args = tmp_path / 'uv-args.txt'
|
||||
env = os.environ.copy()
|
||||
env['HOME'] = str(home)
|
||||
env['PATH'] = os.pathsep.join(part for part in (str(bin_dir), env.get('PATH', '')) if part)
|
||||
env['PYTHONPATH'] = os.pathsep.join(part for part in (str(ROOT), env.get('PYTHONPATH', '')) if part)
|
||||
env['UV_TOOL_INSTALL_ARGS_FILE'] = str(uv_args)
|
||||
|
||||
result = subprocess.run(
|
||||
[sys.executable, '-m', 'browser_use.cli', 'skill', 'install'],
|
||||
cwd=ROOT,
|
||||
env=env,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=10,
|
||||
)
|
||||
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert uv_args.read_text(encoding='utf-8') == 'tool install --python 3.12 --upgrade --force browser-use'
|
||||
expected = (
|
||||
'---\n'
|
||||
'name: browser-use\n'
|
||||
'description: "Direct browser control via CDP for web interaction: automation, scraping, testing, screenshots, and site/app work."\n'
|
||||
'---\n\n'
|
||||
'# Browser Use\n'
|
||||
)
|
||||
for installed in (home / path for path in EXPECTED_SKILL_INSTALL_PATHS):
|
||||
assert installed.read_text(encoding='utf-8') == expected
|
||||
|
||||
|
||||
def test_browser_use_cli_validates_destination_before_installing_harness(tmp_path):
|
||||
bin_dir = _fake_browser_harness_tools(tmp_path, '---\nname: browser-harness\n---\n\n# Browser Harness\n')
|
||||
blocking_file = tmp_path / 'not-a-directory'
|
||||
blocking_file.write_text('blocks skill directory creation', encoding='utf-8')
|
||||
|
||||
uv_args = tmp_path / 'uv-args.txt'
|
||||
env = os.environ.copy()
|
||||
env['HOME'] = str(tmp_path / 'home')
|
||||
env['PATH'] = os.pathsep.join(part for part in (str(bin_dir), env.get('PATH', '')) if part)
|
||||
env['PYTHONPATH'] = os.pathsep.join(part for part in (str(ROOT), env.get('PYTHONPATH', '')) if part)
|
||||
env['UV_TOOL_INSTALL_ARGS_FILE'] = str(uv_args)
|
||||
|
||||
result = subprocess.run(
|
||||
[sys.executable, '-m', 'browser_use.cli', 'skill', 'install', '--path', str(blocking_file / 'nested')],
|
||||
cwd=ROOT,
|
||||
env=env,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=10,
|
||||
)
|
||||
|
||||
assert result.returncode == 1
|
||||
assert 'is not a directory' in result.stderr
|
||||
assert not uv_args.exists()
|
||||
@@ -0,0 +1,156 @@
|
||||
"""Tests for step budget warning injection (IMP-7a)."""
|
||||
|
||||
from browser_use.agent.service import Agent
|
||||
from browser_use.agent.views import AgentStepInfo
|
||||
from browser_use.llm.messages import UserMessage
|
||||
from tests.ci.conftest import create_mock_llm
|
||||
|
||||
|
||||
def _get_context_messages(agent: Agent) -> list[str]:
|
||||
"""Extract text content from the agent's context messages."""
|
||||
msgs = agent._message_manager.state.history.context_messages
|
||||
return [m.content for m in msgs if isinstance(m, UserMessage) and isinstance(m.content, str)]
|
||||
|
||||
|
||||
async def test_budget_warning_injected_at_75_percent():
|
||||
"""Budget warning should be injected when step >= 75% of max_steps."""
|
||||
llm = create_mock_llm()
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
|
||||
step_info = AgentStepInfo(step_number=74, max_steps=100) # step 75/100 = 75%
|
||||
await agent._inject_budget_warning(step_info)
|
||||
|
||||
messages = _get_context_messages(agent)
|
||||
assert len(messages) == 1
|
||||
assert 'BUDGET WARNING' in messages[0]
|
||||
assert '75/100' in messages[0]
|
||||
assert '25 steps remaining' in messages[0]
|
||||
|
||||
|
||||
async def test_budget_warning_injected_at_90_percent():
|
||||
"""Budget warning should fire at 90% too."""
|
||||
llm = create_mock_llm()
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
|
||||
step_info = AgentStepInfo(step_number=89, max_steps=100) # step 90/100 = 90%
|
||||
await agent._inject_budget_warning(step_info)
|
||||
|
||||
messages = _get_context_messages(agent)
|
||||
assert len(messages) == 1
|
||||
assert 'BUDGET WARNING' in messages[0]
|
||||
assert '90/100' in messages[0]
|
||||
assert '10 steps remaining' in messages[0]
|
||||
|
||||
|
||||
async def test_no_budget_warning_below_75_percent():
|
||||
"""No warning should be injected when step < 75% of max_steps."""
|
||||
llm = create_mock_llm()
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
|
||||
step_info = AgentStepInfo(step_number=73, max_steps=100) # step 74/100 = 74%
|
||||
await agent._inject_budget_warning(step_info)
|
||||
|
||||
messages = _get_context_messages(agent)
|
||||
assert len(messages) == 0
|
||||
|
||||
|
||||
async def test_no_budget_warning_on_last_step():
|
||||
"""No budget warning on the last step — _force_done_after_last_step handles that."""
|
||||
llm = create_mock_llm()
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
|
||||
step_info = AgentStepInfo(step_number=99, max_steps=100) # last step
|
||||
assert step_info.is_last_step()
|
||||
await agent._inject_budget_warning(step_info)
|
||||
|
||||
messages = _get_context_messages(agent)
|
||||
assert len(messages) == 0
|
||||
|
||||
|
||||
async def test_no_budget_warning_when_step_info_is_none():
|
||||
"""No warning when step_info is None."""
|
||||
llm = create_mock_llm()
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
|
||||
await agent._inject_budget_warning(None)
|
||||
|
||||
messages = _get_context_messages(agent)
|
||||
assert len(messages) == 0
|
||||
|
||||
|
||||
async def test_budget_warning_exact_threshold():
|
||||
"""The warning should fire at exactly 75% (step 15/20)."""
|
||||
llm = create_mock_llm()
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
|
||||
# step_number=14 means step 15 (1-indexed), 15/20 = 75%
|
||||
step_info = AgentStepInfo(step_number=14, max_steps=20)
|
||||
await agent._inject_budget_warning(step_info)
|
||||
|
||||
messages = _get_context_messages(agent)
|
||||
assert len(messages) == 1
|
||||
assert '15/20' in messages[0]
|
||||
assert '5 steps remaining' in messages[0]
|
||||
|
||||
|
||||
async def test_budget_warning_just_below_threshold():
|
||||
"""No warning at 74% — just below threshold."""
|
||||
llm = create_mock_llm()
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
|
||||
# step_number=13 means step 14 (1-indexed), 14/20 = 70%
|
||||
step_info = AgentStepInfo(step_number=13, max_steps=20)
|
||||
await agent._inject_budget_warning(step_info)
|
||||
|
||||
messages = _get_context_messages(agent)
|
||||
assert len(messages) == 0
|
||||
|
||||
|
||||
async def test_budget_warning_small_max_steps():
|
||||
"""Budget warning works correctly with small max_steps values."""
|
||||
llm = create_mock_llm()
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
|
||||
# step_number=3 means step 4 (1-indexed), 4/4 = 100% but is_last_step
|
||||
step_info = AgentStepInfo(step_number=3, max_steps=4)
|
||||
assert step_info.is_last_step()
|
||||
await agent._inject_budget_warning(step_info)
|
||||
messages = _get_context_messages(agent)
|
||||
assert len(messages) == 0 # last step, no warning
|
||||
|
||||
# step_number=2 means step 3 (1-indexed), 3/4 = 75%
|
||||
step_info = AgentStepInfo(step_number=2, max_steps=4)
|
||||
await agent._inject_budget_warning(step_info)
|
||||
messages = _get_context_messages(agent)
|
||||
assert len(messages) == 1
|
||||
assert '3/4' in messages[0]
|
||||
|
||||
|
||||
async def test_budget_warning_percentage_display():
|
||||
"""The percentage in the warning should be integer (no decimals)."""
|
||||
llm = create_mock_llm()
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
|
||||
# step 76/100 = 76%
|
||||
step_info = AgentStepInfo(step_number=75, max_steps=100)
|
||||
await agent._inject_budget_warning(step_info)
|
||||
|
||||
messages = _get_context_messages(agent)
|
||||
assert '76%' in messages[0]
|
||||
# Should not have decimal
|
||||
assert '76.0%' not in messages[0]
|
||||
|
||||
|
||||
async def test_budget_warning_contains_actionable_guidance():
|
||||
"""The warning message should include actionable guidance for the agent."""
|
||||
llm = create_mock_llm()
|
||||
agent = Agent(task='Test task', llm=llm)
|
||||
|
||||
step_info = AgentStepInfo(step_number=74, max_steps=100)
|
||||
await agent._inject_budget_warning(step_info)
|
||||
|
||||
messages = _get_context_messages(agent)
|
||||
msg = messages[0]
|
||||
assert 'consolidate your results' in msg.lower()
|
||||
assert 'done' in msg.lower()
|
||||
assert 'partial results' in msg.lower()
|
||||
@@ -0,0 +1,116 @@
|
||||
"""Regression tests for TimeoutWrappedCDPClient.
|
||||
|
||||
cdp_use.CDPClient.send_raw awaits a future that only resolves when the browser
|
||||
sends a matching response. When the server goes silent (observed against cloud
|
||||
browsers whose WebSocket stays connected at TCP/keepalive layer but never
|
||||
replies), send_raw hangs forever. The wrapper turns that hang into a fast
|
||||
TimeoutError.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from browser_use.browser._cdp_timeout import (
|
||||
DEFAULT_CDP_REQUEST_TIMEOUT_S,
|
||||
TimeoutWrappedCDPClient,
|
||||
_coerce_valid_timeout,
|
||||
_parse_env_cdp_timeout,
|
||||
)
|
||||
|
||||
|
||||
def _make_wrapped_client_without_websocket(timeout_s: float) -> TimeoutWrappedCDPClient:
|
||||
"""Build a TimeoutWrappedCDPClient without opening a real WebSocket.
|
||||
|
||||
Calling `CDPClient.__init__` directly would try to construct a working
|
||||
client. We only want to exercise the timeout-wrapper `send_raw` path, so
|
||||
we construct the object via __new__ and set the single attribute the
|
||||
wrapper needs.
|
||||
"""
|
||||
client = TimeoutWrappedCDPClient.__new__(TimeoutWrappedCDPClient)
|
||||
client._cdp_request_timeout_s = timeout_s
|
||||
return client
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_raw_times_out_on_silent_server():
|
||||
"""The production TimeoutWrappedCDPClient.send_raw must cap a hung parent
|
||||
send_raw within the configured timeout.
|
||||
|
||||
We deliberately exercise the real `send_raw` (not an inline copy) so
|
||||
regressions in the wrapper itself — e.g. accidentally removing the
|
||||
asyncio.wait_for — fail this test.
|
||||
"""
|
||||
client = _make_wrapped_client_without_websocket(timeout_s=0.5)
|
||||
call_count = {'n': 0}
|
||||
|
||||
async def _hanging_super_send_raw(self, method, params=None, session_id=None):
|
||||
call_count['n'] += 1
|
||||
await asyncio.sleep(30)
|
||||
return {}
|
||||
|
||||
# Patch the parent class's send_raw so TimeoutWrappedCDPClient.send_raw's
|
||||
# `super().send_raw(...)` call lands on our hanging stub.
|
||||
with patch('browser_use.browser._cdp_timeout.CDPClient.send_raw', _hanging_super_send_raw):
|
||||
start = time.monotonic()
|
||||
with pytest.raises(TimeoutError) as exc:
|
||||
await client.send_raw('Target.getTargets')
|
||||
elapsed = time.monotonic() - start
|
||||
|
||||
assert call_count['n'] == 1
|
||||
# Returned within the cap (plus scheduling margin), not after the full 30s.
|
||||
assert elapsed < 2.0, f'wrapper did not enforce timeout; took {elapsed:.2f}s'
|
||||
assert 'Target.getTargets' in str(exc.value)
|
||||
# Error message mentions "within 0s" (0.5 rounded with %.0f) or "within 1s".
|
||||
assert 'within' in str(exc.value)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_raw_passes_through_when_fast():
|
||||
"""A parent send_raw that returns quickly should bubble the result up unchanged."""
|
||||
client = _make_wrapped_client_without_websocket(timeout_s=5.0)
|
||||
|
||||
async def _fast_super_send_raw(self, method, params=None, session_id=None):
|
||||
return {'ok': True, 'method': method}
|
||||
|
||||
with patch('browser_use.browser._cdp_timeout.CDPClient.send_raw', _fast_super_send_raw):
|
||||
result = await client.send_raw('Target.getTargets')
|
||||
|
||||
assert result == {'ok': True, 'method': 'Target.getTargets'}
|
||||
|
||||
|
||||
def test_constructor_rejects_invalid_timeout():
|
||||
"""Non-finite / non-positive constructor args must fall back to the default,
|
||||
mirroring the env-var path in _parse_env_cdp_timeout."""
|
||||
# None → default.
|
||||
assert _coerce_valid_timeout(None) == DEFAULT_CDP_REQUEST_TIMEOUT_S
|
||||
# Invalid values → default, with a warning.
|
||||
for bad in (float('nan'), float('inf'), float('-inf'), 0.0, -5.0, -0.01):
|
||||
assert _coerce_valid_timeout(bad) == DEFAULT_CDP_REQUEST_TIMEOUT_S, f'Expected fallback for {bad!r}, got something else'
|
||||
# Valid finite positives are preserved.
|
||||
assert _coerce_valid_timeout(0.1) == 0.1
|
||||
assert _coerce_valid_timeout(30.0) == 30.0
|
||||
|
||||
|
||||
def test_default_cdp_timeout_is_reasonable():
|
||||
"""Default must give headroom above typical slow CDP calls but stay below
|
||||
the 180s agent step_timeout so hangs surface before step-level kills."""
|
||||
assert 10.0 <= DEFAULT_CDP_REQUEST_TIMEOUT_S <= 120.0, (
|
||||
f'Default CDP timeout ({DEFAULT_CDP_REQUEST_TIMEOUT_S}s) is outside the sensible 10–120s range'
|
||||
)
|
||||
|
||||
|
||||
def test_parse_env_rejects_malformed_values():
|
||||
"""Mirrors the defensive parse used for BROWSER_USE_ACTION_TIMEOUT_S."""
|
||||
for bad in ('', 'nan', 'NaN', 'inf', '-inf', '0', '-5', 'abc'):
|
||||
assert _parse_env_cdp_timeout(bad) == 60.0, f'Expected fallback for {bad!r}'
|
||||
|
||||
# Finite positive values take effect.
|
||||
assert _parse_env_cdp_timeout('30') == 30.0
|
||||
assert _parse_env_cdp_timeout('15.5') == 15.5
|
||||
# None (env var not set) also falls back.
|
||||
assert _parse_env_cdp_timeout(None) == 60.0
|
||||
@@ -0,0 +1,54 @@
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from browser_use.browser import chrome
|
||||
|
||||
|
||||
def test_macos_profile_path_matches_detected_browser_variant(monkeypatch, tmp_path):
|
||||
monkeypatch.setattr(chrome.platform, 'system', lambda: 'Darwin')
|
||||
monkeypatch.setattr(Path, 'home', lambda: tmp_path)
|
||||
|
||||
assert chrome.get_chrome_profile_path(
|
||||
None,
|
||||
executable_path='/Applications/Chromium.app/Contents/MacOS/Chromium',
|
||||
) == str(tmp_path / 'Library' / 'Application Support' / 'Chromium')
|
||||
assert chrome.get_chrome_profile_path(
|
||||
None,
|
||||
executable_path='/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary',
|
||||
) == str(tmp_path / 'Library' / 'Application Support' / 'Google' / 'Chrome Canary')
|
||||
assert chrome.get_chrome_profile_path(
|
||||
None,
|
||||
executable_path='/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
|
||||
) == str(tmp_path / 'Library' / 'Application Support' / 'Google' / 'Chrome')
|
||||
|
||||
|
||||
def test_list_chrome_profiles_uses_detected_browser_variant(monkeypatch, tmp_path):
|
||||
monkeypatch.setattr(chrome.platform, 'system', lambda: 'Darwin')
|
||||
monkeypatch.setattr(Path, 'home', lambda: tmp_path)
|
||||
monkeypatch.setattr(chrome, 'find_chrome_executable', lambda: '/Applications/Chromium.app/Contents/MacOS/Chromium')
|
||||
|
||||
user_data_dir = tmp_path / 'Library' / 'Application Support' / 'Chromium'
|
||||
user_data_dir.mkdir(parents=True)
|
||||
(user_data_dir / 'Local State').write_text(
|
||||
json.dumps({'profile': {'info_cache': {'Profile 1': {'name': 'Work'}}}}),
|
||||
encoding='utf-8',
|
||||
)
|
||||
|
||||
assert chrome.list_chrome_profiles() == [{'directory': 'Profile 1', 'name': 'Work'}]
|
||||
|
||||
|
||||
def test_list_chrome_profiles_returns_empty_for_unexpected_json_shapes(monkeypatch, tmp_path):
|
||||
monkeypatch.setattr(chrome.platform, 'system', lambda: 'Darwin')
|
||||
monkeypatch.setattr(Path, 'home', lambda: tmp_path)
|
||||
monkeypatch.setattr(chrome, 'find_chrome_executable', lambda: '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome')
|
||||
|
||||
user_data_dir = tmp_path / 'Library' / 'Application Support' / 'Google' / 'Chrome'
|
||||
user_data_dir.mkdir(parents=True)
|
||||
|
||||
for payload in (
|
||||
[],
|
||||
{'profile': {'info_cache': []}},
|
||||
{'profile': {'info_cache': {'Default': []}}},
|
||||
):
|
||||
(user_data_dir / 'Local State').write_text(json.dumps(payload), encoding='utf-8')
|
||||
assert chrome.list_chrome_profiles() == []
|
||||
@@ -0,0 +1,183 @@
|
||||
"""Tests for coordinate clicking feature.
|
||||
|
||||
This feature allows certain models (Claude Sonnet 4, Claude Opus 4, Gemini 3 Pro, browser-use/* models)
|
||||
to use coordinate-based clicking, while other models only get index-based clicking.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
|
||||
from browser_use.tools.service import Tools
|
||||
from browser_use.tools.views import ClickElementAction, ClickElementActionIndexOnly
|
||||
|
||||
|
||||
class TestCoordinateClickingTools:
|
||||
"""Test the Tools class coordinate clicking functionality."""
|
||||
|
||||
def test_default_coordinate_clicking_disabled(self):
|
||||
"""By default, coordinate clicking should be disabled."""
|
||||
tools = Tools()
|
||||
|
||||
assert tools._coordinate_clicking_enabled is False
|
||||
|
||||
def test_default_uses_index_only_action(self):
|
||||
"""Default Tools should use ClickElementActionIndexOnly."""
|
||||
tools = Tools()
|
||||
|
||||
click_action = tools.registry.registry.actions.get('click')
|
||||
assert click_action is not None
|
||||
assert click_action.param_model == ClickElementActionIndexOnly
|
||||
|
||||
def test_default_click_schema_has_only_index(self):
|
||||
"""Default click action schema should only have index property."""
|
||||
tools = Tools()
|
||||
|
||||
click_action = tools.registry.registry.actions.get('click')
|
||||
assert click_action is not None
|
||||
schema = click_action.param_model.model_json_schema()
|
||||
|
||||
assert 'index' in schema['properties']
|
||||
assert 'coordinate_x' not in schema['properties']
|
||||
assert 'coordinate_y' not in schema['properties']
|
||||
|
||||
def test_enable_coordinate_clicking(self):
|
||||
"""Enabling coordinate clicking should switch to ClickElementAction."""
|
||||
tools = Tools()
|
||||
tools.set_coordinate_clicking(True)
|
||||
|
||||
assert tools._coordinate_clicking_enabled is True
|
||||
|
||||
click_action = tools.registry.registry.actions.get('click')
|
||||
assert click_action is not None
|
||||
assert click_action.param_model == ClickElementAction
|
||||
|
||||
def test_enabled_click_schema_has_coordinates(self):
|
||||
"""Enabled click action schema should have index and coordinate properties."""
|
||||
tools = Tools()
|
||||
tools.set_coordinate_clicking(True)
|
||||
|
||||
click_action = tools.registry.registry.actions.get('click')
|
||||
assert click_action is not None
|
||||
schema = click_action.param_model.model_json_schema()
|
||||
|
||||
assert 'index' in schema['properties']
|
||||
assert 'coordinate_x' in schema['properties']
|
||||
assert 'coordinate_y' in schema['properties']
|
||||
|
||||
def test_disable_coordinate_clicking(self):
|
||||
"""Disabling coordinate clicking should switch back to index-only."""
|
||||
tools = Tools()
|
||||
tools.set_coordinate_clicking(True)
|
||||
tools.set_coordinate_clicking(False)
|
||||
|
||||
assert tools._coordinate_clicking_enabled is False
|
||||
|
||||
click_action = tools.registry.registry.actions.get('click')
|
||||
assert click_action is not None
|
||||
assert click_action.param_model == ClickElementActionIndexOnly
|
||||
|
||||
def test_set_coordinate_clicking_idempotent(self):
|
||||
"""Setting the same value twice should not cause issues."""
|
||||
tools = Tools()
|
||||
|
||||
# Enable twice
|
||||
tools.set_coordinate_clicking(True)
|
||||
tools.set_coordinate_clicking(True)
|
||||
assert tools._coordinate_clicking_enabled is True
|
||||
|
||||
# Disable twice
|
||||
tools.set_coordinate_clicking(False)
|
||||
tools.set_coordinate_clicking(False)
|
||||
assert tools._coordinate_clicking_enabled is False
|
||||
|
||||
def test_schema_title_consistent(self):
|
||||
"""Schema title should be 'ClickElementAction' regardless of mode."""
|
||||
tools = Tools()
|
||||
|
||||
# Check default (disabled)
|
||||
click_action = tools.registry.registry.actions.get('click')
|
||||
assert click_action is not None
|
||||
schema = click_action.param_model.model_json_schema()
|
||||
assert schema['title'] == 'ClickElementAction'
|
||||
|
||||
# Check enabled
|
||||
tools.set_coordinate_clicking(True)
|
||||
click_action = tools.registry.registry.actions.get('click')
|
||||
assert click_action is not None
|
||||
schema = click_action.param_model.model_json_schema()
|
||||
assert schema['title'] == 'ClickElementAction'
|
||||
|
||||
|
||||
class TestCoordinateClickingModelDetection:
|
||||
"""Test the model detection logic for coordinate clicking."""
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
'model_name,expected_coords',
|
||||
[
|
||||
# Models that SHOULD have coordinate clicking (claude-sonnet-4*, claude-opus-4*, gemini-3-pro*, browser-use/*)
|
||||
('claude-sonnet-4-5', True),
|
||||
('claude-sonnet-4-5-20250101', True),
|
||||
('claude-sonnet-4-0', True),
|
||||
('claude-sonnet-4', True),
|
||||
('claude-opus-4-5', True),
|
||||
('claude-opus-4-5-latest', True),
|
||||
('claude-opus-4-0', True),
|
||||
('claude-opus-4', True),
|
||||
('gemini-3-pro-preview', True),
|
||||
('gemini-3-pro', True),
|
||||
('browser-use/fast', True),
|
||||
('browser-use/accurate', True),
|
||||
('CLAUDE-SONNET-4-5', True), # Case insensitive
|
||||
('CLAUDE-SONNET-4', True), # Case insensitive
|
||||
('GEMINI-3-PRO', True), # Case insensitive
|
||||
# Models that should NOT have coordinate clicking
|
||||
('claude-3-5-sonnet', False),
|
||||
('claude-sonnet-3-5', False),
|
||||
('gpt-4o', False),
|
||||
('gpt-4-turbo', False),
|
||||
('gemini-2.0-flash', False),
|
||||
('gemini-1.5-pro', False),
|
||||
('llama-3.1-70b', False),
|
||||
('mistral-large', False),
|
||||
],
|
||||
)
|
||||
def test_model_detection_patterns(self, model_name: str, expected_coords: bool):
|
||||
"""Test that the model detection patterns correctly identify coordinate-capable models."""
|
||||
model_lower = model_name.lower()
|
||||
supports_coords = any(
|
||||
pattern in model_lower for pattern in ['claude-sonnet-4', 'claude-opus-4', 'gemini-3-pro', 'browser-use/']
|
||||
)
|
||||
assert supports_coords == expected_coords, f'Model {model_name}: expected {expected_coords}, got {supports_coords}'
|
||||
|
||||
|
||||
class TestCoordinateClickingWithPassedTools:
|
||||
"""Test that coordinate clicking works correctly when Tools is passed to Agent."""
|
||||
|
||||
def test_tools_can_be_modified_after_creation(self):
|
||||
"""Tools created externally can have coordinate clicking enabled."""
|
||||
tools = Tools()
|
||||
assert tools._coordinate_clicking_enabled is False
|
||||
|
||||
# Simulate what Agent does for coordinate-capable models
|
||||
tools.set_coordinate_clicking(True)
|
||||
|
||||
click_action = tools.registry.registry.actions.get('click')
|
||||
assert click_action is not None
|
||||
assert click_action.param_model == ClickElementAction
|
||||
|
||||
def test_tools_state_preserved_after_modification(self):
|
||||
"""Verify that other tool state is preserved when toggling coordinate clicking."""
|
||||
tools = Tools(exclude_actions=['search'])
|
||||
|
||||
# Search should be excluded
|
||||
assert 'search' not in tools.registry.registry.actions
|
||||
|
||||
# Enable coordinate clicking
|
||||
tools.set_coordinate_clicking(True)
|
||||
|
||||
# Search should still be excluded
|
||||
assert 'search' not in tools.registry.registry.actions
|
||||
|
||||
# Click should have coordinates
|
||||
click_action = tools.registry.registry.actions.get('click')
|
||||
assert click_action is not None
|
||||
assert click_action.param_model == ClickElementAction
|
||||
@@ -0,0 +1,103 @@
|
||||
"""Tests for DomService.is_element_visible_according_to_all_parents.
|
||||
|
||||
Regression tests for the in-place mutation of shared snapshot bounds: the
|
||||
visibility check must never modify node.snapshot_node.bounds, and a frame
|
||||
node must not be transformed by its own bounds when it appears in its own
|
||||
frame chain (as built by _construct_enhanced_node).
|
||||
"""
|
||||
|
||||
from browser_use.dom.service import DomService
|
||||
from browser_use.dom.views import DOMRect, EnhancedDOMTreeNode, EnhancedSnapshotNode, NodeType
|
||||
|
||||
|
||||
def _make_snapshot(
|
||||
bounds: DOMRect | None = None,
|
||||
client_rects: DOMRect | None = None,
|
||||
scroll_rects: DOMRect | None = None,
|
||||
) -> EnhancedSnapshotNode:
|
||||
return EnhancedSnapshotNode(
|
||||
is_clickable=None,
|
||||
cursor_style=None,
|
||||
bounds=bounds,
|
||||
clientRects=client_rects,
|
||||
scrollRects=scroll_rects,
|
||||
computed_styles={},
|
||||
paint_order=None,
|
||||
stacking_contexts=None,
|
||||
)
|
||||
|
||||
|
||||
def _make_node(node_name: str, snapshot_node: EnhancedSnapshotNode | None) -> EnhancedDOMTreeNode:
|
||||
return EnhancedDOMTreeNode(
|
||||
node_id=1,
|
||||
backend_node_id=1,
|
||||
node_type=NodeType.ELEMENT_NODE,
|
||||
node_name=node_name,
|
||||
node_value='',
|
||||
attributes={},
|
||||
is_scrollable=None,
|
||||
is_visible=None,
|
||||
absolute_position=None,
|
||||
target_id='test-target',
|
||||
frame_id=None,
|
||||
session_id=None,
|
||||
content_document=None,
|
||||
shadow_root_type=None,
|
||||
shadow_roots=None,
|
||||
parent_node=None,
|
||||
children_nodes=None,
|
||||
ax_node=None,
|
||||
snapshot_node=snapshot_node,
|
||||
)
|
||||
|
||||
|
||||
def _make_html_frame(viewport_width: float = 1280, viewport_height: float = 900, scroll_y: float = 0) -> EnhancedDOMTreeNode:
|
||||
return _make_node(
|
||||
'HTML',
|
||||
_make_snapshot(
|
||||
bounds=DOMRect(x=0, y=0, width=viewport_width, height=viewport_height),
|
||||
client_rects=DOMRect(x=0, y=0, width=viewport_width, height=viewport_height),
|
||||
scroll_rects=DOMRect(x=0, y=scroll_y, width=viewport_width, height=5000),
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
class TestVisibilityDoesNotMutateBounds:
|
||||
def test_bounds_unchanged_after_visibility_check(self):
|
||||
"""The check must not modify the shared snapshot bounds object."""
|
||||
html = _make_html_frame(scroll_y=100)
|
||||
element = _make_node('DIV', _make_snapshot(bounds=DOMRect(x=10, y=500, width=100, height=50)))
|
||||
|
||||
DomService.is_element_visible_according_to_all_parents(element, [html], viewport_threshold=1000)
|
||||
|
||||
assert element.snapshot_node is not None and element.snapshot_node.bounds is not None
|
||||
assert element.snapshot_node.bounds.x == 10
|
||||
assert element.snapshot_node.bounds.y == 500
|
||||
|
||||
def test_visibility_check_is_idempotent(self):
|
||||
"""Calling the check twice must return the same result (fails if bounds drift)."""
|
||||
html = _make_html_frame(viewport_height=900, scroll_y=1000)
|
||||
# Element exactly inside the top threshold window: with in-place mutation
|
||||
# the first check returns True, then the drifted second check returns False.
|
||||
element = _make_node('DIV', _make_snapshot(bounds=DOMRect(x=10, y=0, width=100, height=50)))
|
||||
|
||||
first = DomService.is_element_visible_according_to_all_parents(element, [html], viewport_threshold=1000)
|
||||
second = DomService.is_element_visible_according_to_all_parents(element, [html], viewport_threshold=1000)
|
||||
|
||||
assert first is True
|
||||
assert second is True
|
||||
|
||||
def test_iframe_not_double_transformed_by_own_bounds(self):
|
||||
"""An iframe checked against a frame chain containing itself must not have its
|
||||
coordinates doubled (y=1200 on a 900px viewport with threshold 1000 is visible;
|
||||
doubled to 2400 it would not be)."""
|
||||
html = _make_html_frame(viewport_height=900, scroll_y=0)
|
||||
iframe = _make_node('IFRAME', _make_snapshot(bounds=DOMRect(x=0, y=1200, width=600, height=400)))
|
||||
|
||||
# _construct_enhanced_node appends the iframe to its own frame chain before
|
||||
# computing its visibility — reproduce that here.
|
||||
visible = DomService.is_element_visible_according_to_all_parents(iframe, [html, iframe], viewport_threshold=1000)
|
||||
|
||||
assert visible is True
|
||||
assert iframe.snapshot_node is not None and iframe.snapshot_node.bounds is not None
|
||||
assert iframe.snapshot_node.bounds.y == 1200
|
||||
@@ -0,0 +1,51 @@
|
||||
from browser_use.browser.watchdogs.downloads_watchdog import _should_auto_download_network_response
|
||||
|
||||
|
||||
def test_downloads_watchdog_skips_generic_text_attachment_without_file_url():
|
||||
assert not _should_auto_download_network_response(
|
||||
url='https://www.google.com/complete/search?q=test&client=gws-wiz&xssi=t',
|
||||
content_type='text/plain',
|
||||
is_pdf=False,
|
||||
is_download_attachment=True,
|
||||
suggested_filename='f.txt',
|
||||
)
|
||||
|
||||
|
||||
def test_downloads_watchdog_keeps_pdf_network_response():
|
||||
assert _should_auto_download_network_response(
|
||||
url='https://example.com/view?id=123',
|
||||
content_type='application/pdf',
|
||||
is_pdf=True,
|
||||
is_download_attachment=False,
|
||||
suggested_filename=None,
|
||||
)
|
||||
|
||||
|
||||
def test_downloads_watchdog_keeps_named_file_attachment():
|
||||
assert _should_auto_download_network_response(
|
||||
url='https://example.com/download?id=123',
|
||||
content_type='text/csv',
|
||||
is_pdf=False,
|
||||
is_download_attachment=True,
|
||||
suggested_filename='report.csv',
|
||||
)
|
||||
|
||||
|
||||
def test_downloads_watchdog_keeps_text_attachment_with_file_url():
|
||||
assert _should_auto_download_network_response(
|
||||
url='https://example.com/files/summary.txt?download=1',
|
||||
content_type='text/plain',
|
||||
is_pdf=False,
|
||||
is_download_attachment=True,
|
||||
suggested_filename='f.txt',
|
||||
)
|
||||
|
||||
|
||||
def test_downloads_watchdog_keeps_attachment_without_known_extension():
|
||||
assert _should_auto_download_network_response(
|
||||
url='https://example.com/download?id=123',
|
||||
content_type='application/vnd.example.custom',
|
||||
is_pdf=False,
|
||||
is_download_attachment=True,
|
||||
suggested_filename='statement',
|
||||
)
|
||||
@@ -0,0 +1,85 @@
|
||||
"""Regression tests for ENG-5280: V2 worker crash on warm-Lambda resume.
|
||||
|
||||
A reused keep_alive session bus is stopped and nulled by Agent.close(); on resume the worker
|
||||
can step() it before a dispatch() restarts it, which makes stock bubus assert. ResilientEventBus
|
||||
no-ops step()/wait_until_idle() in that state while still restarting on the next dispatch().
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
from bubus import BaseEvent, EventBus
|
||||
|
||||
from browser_use.browser import BrowserProfile, BrowserSession
|
||||
from browser_use.browser.session import ResilientEventBus
|
||||
|
||||
|
||||
class ResiliencePingEvent(BaseEvent):
|
||||
pass
|
||||
|
||||
|
||||
def _tear_down_like_agent_close(bus: EventBus) -> None:
|
||||
"""Mimic the keep_alive teardown in Agent.close() that triggers ENG-5280."""
|
||||
bus.event_queue = None
|
||||
bus._on_idle = None
|
||||
|
||||
|
||||
def test_browser_session_uses_resilient_event_bus():
|
||||
"""The session's default event bus must be the resilient subclass."""
|
||||
session = BrowserSession(browser_profile=BrowserProfile(keep_alive=True))
|
||||
assert isinstance(session.event_bus, ResilientEventBus)
|
||||
|
||||
|
||||
def test_resilient_event_bus_keeps_event_bus_name_prefix():
|
||||
"""The subclass must keep the ``EventBus_`` default name (not ``ResilientEventBus_``)."""
|
||||
assert ResilientEventBus().name.startswith('EventBus_')
|
||||
# Explicit names are still honored.
|
||||
assert ResilientEventBus(name='EventBus_custom').name == 'EventBus_custom'
|
||||
|
||||
|
||||
async def test_step_on_torn_down_bus_is_noop_not_assertion():
|
||||
"""Stepping a stopped+nulled bus returns None instead of raising AssertionError."""
|
||||
bus = ResilientEventBus(name='ResilientWarmResumeStep')
|
||||
bus.dispatch(ResiliencePingEvent())
|
||||
await asyncio.sleep(0.1)
|
||||
await bus.stop(clear=False, timeout=1.0)
|
||||
_tear_down_like_agent_close(bus)
|
||||
|
||||
# Warm-Lambda resume: worker steps the torn-down bus. Must not raise.
|
||||
assert await bus.step() is None
|
||||
assert await bus.wait_until_idle(timeout=0.1) is None
|
||||
|
||||
|
||||
async def test_torn_down_bus_still_restarts_on_dispatch():
|
||||
"""A later dispatch() must recreate the queue and process events normally."""
|
||||
processed: list[BaseEvent] = []
|
||||
bus = ResilientEventBus(name='ResilientWarmResumeRestart')
|
||||
bus.on('ResiliencePingEvent', lambda event: processed.append(event))
|
||||
|
||||
bus.dispatch(ResiliencePingEvent())
|
||||
await asyncio.sleep(0.1)
|
||||
await bus.stop(clear=False, timeout=1.0)
|
||||
_tear_down_like_agent_close(bus)
|
||||
|
||||
# No-op step on the torn-down bus, then dispatch again -> bus restarts.
|
||||
assert await bus.step() is None
|
||||
processed.clear()
|
||||
event = bus.dispatch(ResiliencePingEvent())
|
||||
await asyncio.wait_for(event, timeout=2.0)
|
||||
assert len(processed) == 1
|
||||
await bus.stop(timeout=0.5)
|
||||
|
||||
|
||||
async def test_stock_event_bus_reproduces_the_crash():
|
||||
"""Document the upstream bug the subclass guards against (stock bus still asserts)."""
|
||||
bus = EventBus(name='StockWarmResume')
|
||||
bus.dispatch(ResiliencePingEvent())
|
||||
await asyncio.sleep(0.1)
|
||||
await bus.stop(clear=False, timeout=1.0)
|
||||
_tear_down_like_agent_close(bus)
|
||||
|
||||
try:
|
||||
await bus.step()
|
||||
raised = False
|
||||
except AssertionError:
|
||||
raised = True
|
||||
assert raised, 'stock EventBus.step() should assert on a torn-down bus'
|
||||
@@ -0,0 +1,123 @@
|
||||
"""Tests for extension configuration environment variables."""
|
||||
|
||||
import os
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
class TestDisableExtensionsEnvVar:
|
||||
"""Test BROWSER_USE_DISABLE_EXTENSIONS environment variable."""
|
||||
|
||||
def test_default_value_is_true(self):
|
||||
"""Without env var set, enable_default_extensions should default to True."""
|
||||
# Clear the env var if it exists
|
||||
original = os.environ.pop('BROWSER_USE_DISABLE_EXTENSIONS', None)
|
||||
try:
|
||||
# Import fresh to get the default
|
||||
from browser_use.browser.profile import _get_enable_default_extensions_default
|
||||
|
||||
assert _get_enable_default_extensions_default() is True
|
||||
finally:
|
||||
if original is not None:
|
||||
os.environ['BROWSER_USE_DISABLE_EXTENSIONS'] = original
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
'env_value,expected_enabled',
|
||||
[
|
||||
# Truthy values for DISABLE = extensions disabled (False)
|
||||
('true', False),
|
||||
('True', False),
|
||||
('TRUE', False),
|
||||
('1', False),
|
||||
('yes', False),
|
||||
('on', False),
|
||||
# Falsy values for DISABLE = extensions enabled (True)
|
||||
('false', True),
|
||||
('False', True),
|
||||
('FALSE', True),
|
||||
('0', True),
|
||||
('no', True),
|
||||
('off', True),
|
||||
('', True),
|
||||
],
|
||||
)
|
||||
def test_env_var_values(self, env_value: str, expected_enabled: bool):
|
||||
"""Test various env var values are parsed correctly."""
|
||||
original = os.environ.get('BROWSER_USE_DISABLE_EXTENSIONS')
|
||||
try:
|
||||
os.environ['BROWSER_USE_DISABLE_EXTENSIONS'] = env_value
|
||||
from browser_use.browser.profile import _get_enable_default_extensions_default
|
||||
|
||||
result = _get_enable_default_extensions_default()
|
||||
assert result is expected_enabled, (
|
||||
f"Expected enable_default_extensions={expected_enabled} for DISABLE_EXTENSIONS='{env_value}', got {result}"
|
||||
)
|
||||
finally:
|
||||
if original is not None:
|
||||
os.environ['BROWSER_USE_DISABLE_EXTENSIONS'] = original
|
||||
else:
|
||||
os.environ.pop('BROWSER_USE_DISABLE_EXTENSIONS', None)
|
||||
|
||||
def test_browser_profile_uses_env_var(self):
|
||||
"""Test that BrowserProfile picks up the env var."""
|
||||
original = os.environ.get('BROWSER_USE_DISABLE_EXTENSIONS')
|
||||
try:
|
||||
# Test with env var set to true (disable extensions)
|
||||
os.environ['BROWSER_USE_DISABLE_EXTENSIONS'] = 'true'
|
||||
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
|
||||
profile = BrowserProfile(headless=True)
|
||||
assert profile.enable_default_extensions is False, (
|
||||
'BrowserProfile should disable extensions when BROWSER_USE_DISABLE_EXTENSIONS=true'
|
||||
)
|
||||
|
||||
# Test with env var set to false (enable extensions)
|
||||
os.environ['BROWSER_USE_DISABLE_EXTENSIONS'] = 'false'
|
||||
profile2 = BrowserProfile(headless=True)
|
||||
assert profile2.enable_default_extensions is True, (
|
||||
'BrowserProfile should enable extensions when BROWSER_USE_DISABLE_EXTENSIONS=false'
|
||||
)
|
||||
|
||||
finally:
|
||||
if original is not None:
|
||||
os.environ['BROWSER_USE_DISABLE_EXTENSIONS'] = original
|
||||
else:
|
||||
os.environ.pop('BROWSER_USE_DISABLE_EXTENSIONS', None)
|
||||
|
||||
def test_explicit_param_overrides_env_var(self):
|
||||
"""Test that explicit enable_default_extensions parameter overrides env var."""
|
||||
original = os.environ.get('BROWSER_USE_DISABLE_EXTENSIONS')
|
||||
try:
|
||||
os.environ['BROWSER_USE_DISABLE_EXTENSIONS'] = 'true'
|
||||
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
|
||||
# Explicitly set to True should override env var
|
||||
profile = BrowserProfile(headless=True, enable_default_extensions=True)
|
||||
assert profile.enable_default_extensions is True, 'Explicit param should override env var'
|
||||
|
||||
finally:
|
||||
if original is not None:
|
||||
os.environ['BROWSER_USE_DISABLE_EXTENSIONS'] = original
|
||||
else:
|
||||
os.environ.pop('BROWSER_USE_DISABLE_EXTENSIONS', None)
|
||||
|
||||
def test_browser_session_uses_env_var(self):
|
||||
"""Test that BrowserSession picks up the env var via BrowserProfile."""
|
||||
original = os.environ.get('BROWSER_USE_DISABLE_EXTENSIONS')
|
||||
try:
|
||||
os.environ['BROWSER_USE_DISABLE_EXTENSIONS'] = '1'
|
||||
|
||||
from browser_use.browser import BrowserSession
|
||||
|
||||
session = BrowserSession(headless=True)
|
||||
assert session.browser_profile.enable_default_extensions is False, (
|
||||
'BrowserSession should disable extensions when BROWSER_USE_DISABLE_EXTENSIONS=1'
|
||||
)
|
||||
|
||||
finally:
|
||||
if original is not None:
|
||||
os.environ['BROWSER_USE_DISABLE_EXTENSIONS'] = original
|
||||
else:
|
||||
os.environ.pop('BROWSER_USE_DISABLE_EXTENSIONS', None)
|
||||
@@ -0,0 +1,296 @@
|
||||
"""Tests for extract_images support in extract_clean_markdown.
|
||||
|
||||
Root cause of AGI-101: markdownify strips img src URLs when images appear inside table cells
|
||||
(<td>/<th>) or heading elements, because those contexts set the _inline flag. The extract_images
|
||||
parameter fixes this by adding those tags to keep_inline_images_in.
|
||||
|
||||
Block-level images (direct children of <div>, <figure>, etc.) are ALWAYS included in markdown
|
||||
regardless of extract_images. The parameter only matters for images inside <td>, <th>, <h1>-<h6>.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
import pytest
|
||||
from pytest_httpserver import HTTPServer
|
||||
|
||||
from browser_use.browser import BrowserProfile, BrowserSession
|
||||
from browser_use.dom.markdown_extractor import extract_clean_markdown
|
||||
|
||||
# --- Fixtures ---
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def http_server():
|
||||
"""Test HTTP server serving pages with product images."""
|
||||
server = HTTPServer()
|
||||
server.start()
|
||||
|
||||
# Table-layout products — images are in <td>, the actual bug scenario.
|
||||
# With extract_images=False (default), img in td becomes just alt text (no URL).
|
||||
# With extract_images=True, img in td becomes  with the real URL.
|
||||
server.expect_request('/products-table').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Products Table</title></head>
|
||||
<body>
|
||||
<h1>Product Catalog</h1>
|
||||
<table>
|
||||
<thead>
|
||||
<tr><th>Image</th><th>Name</th><th>Price</th></tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><img src="http://localhost/images/widget-a.jpg" alt="Widget A"></td>
|
||||
<td>Widget A</td>
|
||||
<td>$29.99</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><img src="http://localhost/images/widget-b.jpg" alt="Widget B"></td>
|
||||
<td>Widget B</td>
|
||||
<td>$49.99</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><img src="http://localhost/images/gadget-c.png" alt="Gadget C"></td>
|
||||
<td>Gadget C</td>
|
||||
<td>$19.50</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
# Block-level products — images in <div>/<figure> are ALWAYS included in markdown,
|
||||
# regardless of extract_images value.
|
||||
server.expect_request('/products-block').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Products Block</title></head>
|
||||
<body>
|
||||
<div class="product">
|
||||
<img src="http://localhost/images/widget-a.jpg" alt="Widget A">
|
||||
<p>Widget A - $29.99</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
server.expect_request('/text-only').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Text Only</title></head>
|
||||
<body>
|
||||
<h1>No Images Here</h1>
|
||||
<p>Just some text content with no images at all.</p>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
yield server
|
||||
server.stop()
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def base_url(http_server):
|
||||
return f'http://{http_server.host}:{http_server.port}'
|
||||
|
||||
|
||||
@pytest.fixture(scope='module')
|
||||
async def browser_session():
|
||||
session = BrowserSession(
|
||||
browser_profile=BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
keep_alive=True,
|
||||
)
|
||||
)
|
||||
await session.start()
|
||||
yield session
|
||||
await session.kill()
|
||||
|
||||
|
||||
# --- Helper ---
|
||||
|
||||
|
||||
async def _navigate(browser_session, url: str):
|
||||
"""Navigate to URL and wait for page load."""
|
||||
await browser_session.navigate_to(url)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
|
||||
# --- Tests ---
|
||||
|
||||
|
||||
class TestExtractCleanMarkdown:
|
||||
"""Tests for extract_clean_markdown with extract_images parameter."""
|
||||
|
||||
async def test_table_images_excluded_by_default(self, browser_session, base_url):
|
||||
"""Images inside <td> lose their URL with extract_images=False (default).
|
||||
|
||||
This is the AGI-101 root cause: markdownify strips img src in _inline contexts
|
||||
(td/th/headings) when keep_inline_images_in=[]. The alt text is kept but not the URL.
|
||||
"""
|
||||
await _navigate(browser_session, f'{base_url}/products-table')
|
||||
|
||||
content, _ = await extract_clean_markdown(browser_session=browser_session, extract_images=False)
|
||||
|
||||
# Alt text (product names) should still be present
|
||||
assert 'Widget A' in content
|
||||
assert 'Widget B' in content
|
||||
# But image URLs should NOT appear — img in td is stripped to alt text
|
||||
assert 'widget-a.jpg' not in content
|
||||
assert 'widget-b.jpg' not in content
|
||||
assert 'gadget-c.png' not in content
|
||||
# No markdown image syntax
|
||||
assert ' regardless
|
||||
assert '![' in content_false
|
||||
assert 'widget-a.jpg' in content_false
|
||||
assert '![' in content_true
|
||||
assert 'widget-a.jpg' in content_true
|
||||
|
||||
async def test_false_is_default(self, browser_session, base_url):
|
||||
"""Calling extract_clean_markdown without extract_images behaves same as extract_images=False."""
|
||||
await _navigate(browser_session, f'{base_url}/products-table')
|
||||
|
||||
content_default, _ = await extract_clean_markdown(browser_session=browser_session)
|
||||
content_false, _ = await extract_clean_markdown(browser_session=browser_session, extract_images=False)
|
||||
|
||||
assert content_default == content_false
|
||||
|
||||
async def test_no_images_on_text_only_page(self, browser_session, base_url):
|
||||
"""extract_images=True on a page with no images returns no image markdown."""
|
||||
await _navigate(browser_session, f'{base_url}/text-only')
|
||||
|
||||
content, _ = await extract_clean_markdown(browser_session=browser_session, extract_images=True)
|
||||
|
||||
assert '![' not in content
|
||||
assert 'No Images Here' in content or 'text content' in content
|
||||
|
||||
|
||||
class TestExtractImagesAutoDetection:
|
||||
"""Tests for auto-detection of image-related queries in the extract action."""
|
||||
|
||||
async def test_auto_detect_image_url_query(self, browser_session, base_url, tmp_path):
|
||||
"""Query containing 'image url' auto-enables extract_images: table-cell img URLs appear in LLM input."""
|
||||
from unittest.mock import AsyncMock
|
||||
|
||||
from browser_use.filesystem.file_system import FileSystem
|
||||
from browser_use.llm import BaseChatModel
|
||||
from browser_use.llm.views import ChatInvokeCompletion
|
||||
from browser_use.tools.service import Tools
|
||||
|
||||
await _navigate(browser_session, f'{base_url}/products-table')
|
||||
|
||||
captured_content: list[str] = []
|
||||
|
||||
mock_llm = AsyncMock(spec=BaseChatModel)
|
||||
mock_llm.model = 'mock-llm'
|
||||
mock_llm._verified_api_keys = True
|
||||
mock_llm.provider = 'mock'
|
||||
mock_llm.name = 'mock-llm'
|
||||
mock_llm.model_name = 'mock-llm'
|
||||
|
||||
async def capture_ainvoke(*args, **kwargs):
|
||||
if args:
|
||||
for msg in args[0]:
|
||||
content = getattr(msg, 'content', '')
|
||||
if isinstance(content, str):
|
||||
captured_content.append(content)
|
||||
elif isinstance(content, list):
|
||||
for part in content:
|
||||
if isinstance(part, dict) and part.get('type') == 'text':
|
||||
captured_content.append(part.get('text', ''))
|
||||
return ChatInvokeCompletion(completion='Widget A image: http://localhost/images/widget-a.jpg', usage=None)
|
||||
|
||||
mock_llm.ainvoke.side_effect = capture_ainvoke
|
||||
|
||||
tools = Tools()
|
||||
await tools.extract(
|
||||
query='get image url for each product',
|
||||
browser_session=browser_session,
|
||||
page_extraction_llm=mock_llm,
|
||||
file_system=FileSystem(base_dir=str(tmp_path)),
|
||||
)
|
||||
|
||||
# The LLM should have received content that includes image markdown (td images with URLs)
|
||||
all_content = ' '.join(captured_content)
|
||||
assert '![' in all_content or 'widget-a.jpg' in all_content or 'widget-b.jpg' in all_content, (
|
||||
f'Expected image URLs in LLM input but got: {all_content[:500]}'
|
||||
)
|
||||
|
||||
async def test_no_auto_detect_without_image_keyword(self, browser_session, base_url, tmp_path):
|
||||
"""Query without image keywords does NOT auto-enable extract_images: table-cell img URLs absent."""
|
||||
from unittest.mock import AsyncMock
|
||||
|
||||
from browser_use.filesystem.file_system import FileSystem
|
||||
from browser_use.llm import BaseChatModel
|
||||
from browser_use.llm.views import ChatInvokeCompletion
|
||||
from browser_use.tools.service import Tools
|
||||
|
||||
await _navigate(browser_session, f'{base_url}/products-table')
|
||||
|
||||
captured_content: list[str] = []
|
||||
|
||||
mock_llm = AsyncMock(spec=BaseChatModel)
|
||||
mock_llm.model = 'mock-llm'
|
||||
mock_llm._verified_api_keys = True
|
||||
mock_llm.provider = 'mock'
|
||||
mock_llm.name = 'mock-llm'
|
||||
mock_llm.model_name = 'mock-llm'
|
||||
|
||||
async def capture_ainvoke(*args, **kwargs):
|
||||
if args:
|
||||
for msg in args[0]:
|
||||
content = getattr(msg, 'content', '')
|
||||
if isinstance(content, str):
|
||||
captured_content.append(content)
|
||||
elif isinstance(content, list):
|
||||
for part in content:
|
||||
if isinstance(part, dict) and part.get('type') == 'text':
|
||||
captured_content.append(part.get('text', ''))
|
||||
return ChatInvokeCompletion(completion='Widget A - $29.99, Widget B - $49.99', usage=None)
|
||||
|
||||
mock_llm.ainvoke.side_effect = capture_ainvoke
|
||||
|
||||
tools = Tools()
|
||||
await tools.extract(
|
||||
query='get product names and prices',
|
||||
browser_session=browser_session,
|
||||
page_extraction_llm=mock_llm,
|
||||
file_system=FileSystem(base_dir=str(tmp_path)),
|
||||
)
|
||||
|
||||
# Table-cell image URLs should NOT appear (extract_images=False, no auto-detect)
|
||||
all_content = ' '.join(captured_content)
|
||||
assert 'widget-a.jpg' not in all_content and 'widget-b.jpg' not in all_content, (
|
||||
f'Did not expect image URLs in LLM input but got: {all_content[:500]}'
|
||||
)
|
||||
@@ -0,0 +1,439 @@
|
||||
"""
|
||||
Tests for the fallback_llm feature in Agent.
|
||||
|
||||
Tests verify that when the primary LLM fails with rate limit (429) or server errors (503, 502, 500, 504),
|
||||
the agent automatically switches to the fallback LLM and continues execution.
|
||||
"""
|
||||
|
||||
from unittest.mock import AsyncMock
|
||||
|
||||
import pytest
|
||||
|
||||
from browser_use.agent.views import AgentOutput
|
||||
from browser_use.llm import BaseChatModel
|
||||
from browser_use.llm.exceptions import ModelProviderError, ModelRateLimitError
|
||||
from browser_use.llm.views import ChatInvokeCompletion
|
||||
from browser_use.tools.service import Tools
|
||||
|
||||
|
||||
def create_mock_llm(
|
||||
model_name: str = 'mock-llm',
|
||||
should_fail: bool = False,
|
||||
fail_with: type[Exception] | None = None,
|
||||
fail_status_code: int = 429,
|
||||
fail_message: str = 'Rate limit exceeded',
|
||||
) -> BaseChatModel:
|
||||
"""Create a mock LLM for testing.
|
||||
|
||||
Args:
|
||||
model_name: Name of the mock model
|
||||
should_fail: If True, the LLM will raise an exception
|
||||
fail_with: Exception type to raise (ModelRateLimitError or ModelProviderError)
|
||||
fail_status_code: HTTP status code for the error
|
||||
fail_message: Error message
|
||||
"""
|
||||
tools = Tools()
|
||||
ActionModel = tools.registry.create_action_model()
|
||||
AgentOutputWithActions = AgentOutput.type_with_custom_actions(ActionModel)
|
||||
|
||||
llm = AsyncMock(spec=BaseChatModel)
|
||||
llm.model = model_name
|
||||
llm._verified_api_keys = True
|
||||
llm.provider = 'mock'
|
||||
llm.name = model_name
|
||||
llm.model_name = model_name
|
||||
|
||||
default_done_action = """
|
||||
{
|
||||
"thinking": "null",
|
||||
"evaluation_previous_goal": "Successfully completed the task",
|
||||
"memory": "Task completed",
|
||||
"next_goal": "Task completed",
|
||||
"action": [
|
||||
{
|
||||
"done": {
|
||||
"text": "Task completed successfully",
|
||||
"success": true
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
"""
|
||||
|
||||
async def mock_ainvoke(*args, **kwargs):
|
||||
if should_fail:
|
||||
if fail_with == ModelRateLimitError:
|
||||
raise ModelRateLimitError(message=fail_message, status_code=fail_status_code, model=model_name)
|
||||
elif fail_with == ModelProviderError:
|
||||
raise ModelProviderError(message=fail_message, status_code=fail_status_code, model=model_name)
|
||||
else:
|
||||
raise Exception(fail_message)
|
||||
|
||||
output_format = kwargs.get('output_format')
|
||||
if output_format is None:
|
||||
return ChatInvokeCompletion(completion=default_done_action, usage=None)
|
||||
else:
|
||||
parsed = output_format.model_validate_json(default_done_action)
|
||||
return ChatInvokeCompletion(completion=parsed, usage=None)
|
||||
|
||||
llm.ainvoke.side_effect = mock_ainvoke
|
||||
|
||||
return llm
|
||||
|
||||
|
||||
class TestFallbackLLMParameter:
|
||||
"""Test fallback_llm parameter initialization."""
|
||||
|
||||
def test_fallback_llm_none_by_default(self):
|
||||
"""Verify fallback_llm defaults to None."""
|
||||
from browser_use import Agent
|
||||
|
||||
primary = create_mock_llm('primary-model')
|
||||
agent = Agent(task='Test task', llm=primary)
|
||||
|
||||
assert agent._fallback_llm is None
|
||||
assert agent._using_fallback_llm is False
|
||||
assert agent._original_llm is primary
|
||||
|
||||
def test_fallback_llm_single_model(self):
|
||||
"""Test passing a fallback LLM."""
|
||||
from browser_use import Agent
|
||||
|
||||
primary = create_mock_llm('primary-model')
|
||||
fallback = create_mock_llm('fallback-model')
|
||||
|
||||
agent = Agent(task='Test task', llm=primary, fallback_llm=fallback)
|
||||
|
||||
assert agent._fallback_llm is fallback
|
||||
assert agent._using_fallback_llm is False
|
||||
|
||||
def test_public_properties(self):
|
||||
"""Test the public properties for fallback status."""
|
||||
from browser_use import Agent
|
||||
|
||||
primary = create_mock_llm('primary-model')
|
||||
fallback = create_mock_llm('fallback-model')
|
||||
|
||||
agent = Agent(task='Test task', llm=primary, fallback_llm=fallback)
|
||||
|
||||
# Before fallback
|
||||
assert agent.is_using_fallback_llm is False
|
||||
assert agent.current_llm_model == 'primary-model'
|
||||
|
||||
# Trigger fallback
|
||||
error = ModelRateLimitError(message='Rate limit', status_code=429, model='primary')
|
||||
agent._try_switch_to_fallback_llm(error)
|
||||
|
||||
# After fallback
|
||||
assert agent.is_using_fallback_llm is True
|
||||
assert agent.current_llm_model == 'fallback-model'
|
||||
|
||||
|
||||
class TestFallbackLLMSwitching:
|
||||
"""Test the fallback switching logic in _try_switch_to_fallback_llm."""
|
||||
|
||||
def test_switch_on_rate_limit_error(self):
|
||||
"""Test that agent switches to fallback on ModelRateLimitError."""
|
||||
from browser_use import Agent
|
||||
|
||||
primary = create_mock_llm('primary-model')
|
||||
fallback = create_mock_llm('fallback-model')
|
||||
|
||||
agent = Agent(task='Test task', llm=primary, fallback_llm=fallback)
|
||||
|
||||
error = ModelRateLimitError(message='Rate limit exceeded', status_code=429, model='primary-model')
|
||||
result = agent._try_switch_to_fallback_llm(error)
|
||||
|
||||
assert result is True
|
||||
assert agent.llm is fallback
|
||||
assert agent._using_fallback_llm is True
|
||||
|
||||
def test_switch_on_503_error(self):
|
||||
"""Test that agent switches to fallback on 503 Service Unavailable."""
|
||||
from browser_use import Agent
|
||||
|
||||
primary = create_mock_llm('primary-model')
|
||||
fallback = create_mock_llm('fallback-model')
|
||||
|
||||
agent = Agent(task='Test task', llm=primary, fallback_llm=fallback)
|
||||
|
||||
error = ModelProviderError(message='Service unavailable', status_code=503, model='primary-model')
|
||||
result = agent._try_switch_to_fallback_llm(error)
|
||||
|
||||
assert result is True
|
||||
assert agent.llm is fallback
|
||||
assert agent._using_fallback_llm is True
|
||||
|
||||
def test_switch_on_500_error(self):
|
||||
"""Test that agent switches to fallback on 500 Internal Server Error."""
|
||||
from browser_use import Agent
|
||||
|
||||
primary = create_mock_llm('primary-model')
|
||||
fallback = create_mock_llm('fallback-model')
|
||||
|
||||
agent = Agent(task='Test task', llm=primary, fallback_llm=fallback)
|
||||
|
||||
error = ModelProviderError(message='Internal server error', status_code=500, model='primary-model')
|
||||
result = agent._try_switch_to_fallback_llm(error)
|
||||
|
||||
assert result is True
|
||||
assert agent.llm is fallback
|
||||
|
||||
def test_switch_on_502_error(self):
|
||||
"""Test that agent switches to fallback on 502 Bad Gateway."""
|
||||
from browser_use import Agent
|
||||
|
||||
primary = create_mock_llm('primary-model')
|
||||
fallback = create_mock_llm('fallback-model')
|
||||
|
||||
agent = Agent(task='Test task', llm=primary, fallback_llm=fallback)
|
||||
|
||||
error = ModelProviderError(message='Bad gateway', status_code=502, model='primary-model')
|
||||
result = agent._try_switch_to_fallback_llm(error)
|
||||
|
||||
assert result is True
|
||||
assert agent.llm is fallback
|
||||
|
||||
def test_no_switch_on_400_error(self):
|
||||
"""Test that agent does NOT switch on 400 Bad Request (not retryable)."""
|
||||
from browser_use import Agent
|
||||
|
||||
primary = create_mock_llm('primary-model')
|
||||
fallback = create_mock_llm('fallback-model')
|
||||
|
||||
agent = Agent(task='Test task', llm=primary, fallback_llm=fallback)
|
||||
|
||||
error = ModelProviderError(message='Bad request', status_code=400, model='primary-model')
|
||||
result = agent._try_switch_to_fallback_llm(error)
|
||||
|
||||
assert result is False
|
||||
assert agent.llm is primary # Still using primary
|
||||
assert agent._using_fallback_llm is False
|
||||
|
||||
def test_switch_on_401_error(self):
|
||||
"""Test that agent switches to fallback on 401 Unauthorized (API key error)."""
|
||||
from browser_use import Agent
|
||||
|
||||
primary = create_mock_llm('primary-model')
|
||||
fallback = create_mock_llm('fallback-model')
|
||||
|
||||
agent = Agent(task='Test task', llm=primary, fallback_llm=fallback)
|
||||
|
||||
error = ModelProviderError(message='Invalid API key', status_code=401, model='primary-model')
|
||||
result = agent._try_switch_to_fallback_llm(error)
|
||||
|
||||
assert result is True
|
||||
assert agent.llm is fallback
|
||||
assert agent._using_fallback_llm is True
|
||||
|
||||
def test_switch_on_402_error(self):
|
||||
"""Test that agent switches to fallback on 402 Payment Required (insufficient credits)."""
|
||||
from browser_use import Agent
|
||||
|
||||
primary = create_mock_llm('primary-model')
|
||||
fallback = create_mock_llm('fallback-model')
|
||||
|
||||
agent = Agent(task='Test task', llm=primary, fallback_llm=fallback)
|
||||
|
||||
error = ModelProviderError(message='Insufficient credits', status_code=402, model='primary-model')
|
||||
result = agent._try_switch_to_fallback_llm(error)
|
||||
|
||||
assert result is True
|
||||
assert agent.llm is fallback
|
||||
assert agent._using_fallback_llm is True
|
||||
|
||||
def test_no_switch_when_no_fallback_configured(self):
|
||||
"""Test that agent returns False when no fallback is configured."""
|
||||
from browser_use import Agent
|
||||
|
||||
primary = create_mock_llm('primary-model')
|
||||
agent = Agent(task='Test task', llm=primary)
|
||||
|
||||
error = ModelRateLimitError(message='Rate limit exceeded', status_code=429, model='primary-model')
|
||||
result = agent._try_switch_to_fallback_llm(error)
|
||||
|
||||
assert result is False
|
||||
assert agent.llm is primary
|
||||
|
||||
def test_no_switch_when_already_using_fallback(self):
|
||||
"""Test that agent doesn't switch again when already using fallback."""
|
||||
from browser_use import Agent
|
||||
|
||||
primary = create_mock_llm('primary-model')
|
||||
fallback = create_mock_llm('fallback-model')
|
||||
|
||||
agent = Agent(task='Test task', llm=primary, fallback_llm=fallback)
|
||||
|
||||
# First switch succeeds
|
||||
error = ModelRateLimitError(message='Rate limit', status_code=429, model='primary')
|
||||
result = agent._try_switch_to_fallback_llm(error)
|
||||
assert result is True
|
||||
assert agent.llm is fallback
|
||||
|
||||
# Second switch fails - already using fallback
|
||||
result = agent._try_switch_to_fallback_llm(error)
|
||||
assert result is False
|
||||
assert agent.llm is fallback # Still on fallback
|
||||
|
||||
|
||||
class TestFallbackLLMIntegration:
|
||||
"""Integration tests for fallback LLM behavior in get_model_output."""
|
||||
|
||||
def _create_failing_mock_llm(
|
||||
self,
|
||||
model_name: str,
|
||||
fail_with: type[Exception],
|
||||
fail_status_code: int = 429,
|
||||
fail_message: str = 'Rate limit exceeded',
|
||||
) -> BaseChatModel:
|
||||
"""Create a mock LLM that always fails with the specified error."""
|
||||
llm = AsyncMock(spec=BaseChatModel)
|
||||
llm.model = model_name
|
||||
llm._verified_api_keys = True
|
||||
llm.provider = 'mock'
|
||||
llm.name = model_name
|
||||
llm.model_name = model_name
|
||||
|
||||
async def mock_ainvoke(*args, **kwargs):
|
||||
if fail_with == ModelRateLimitError:
|
||||
raise ModelRateLimitError(message=fail_message, status_code=fail_status_code, model=model_name)
|
||||
elif fail_with == ModelProviderError:
|
||||
raise ModelProviderError(message=fail_message, status_code=fail_status_code, model=model_name)
|
||||
else:
|
||||
raise Exception(fail_message)
|
||||
|
||||
llm.ainvoke.side_effect = mock_ainvoke
|
||||
return llm
|
||||
|
||||
def _create_succeeding_mock_llm(self, model_name: str, agent) -> BaseChatModel:
|
||||
"""Create a mock LLM that succeeds and returns a valid AgentOutput."""
|
||||
llm = AsyncMock(spec=BaseChatModel)
|
||||
llm.model = model_name
|
||||
llm._verified_api_keys = True
|
||||
llm.provider = 'mock'
|
||||
llm.name = model_name
|
||||
llm.model_name = model_name
|
||||
|
||||
default_done_action = """
|
||||
{
|
||||
"thinking": "null",
|
||||
"evaluation_previous_goal": "Successfully completed the task",
|
||||
"memory": "Task completed",
|
||||
"next_goal": "Task completed",
|
||||
"action": [
|
||||
{
|
||||
"done": {
|
||||
"text": "Task completed successfully",
|
||||
"success": true
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
"""
|
||||
|
||||
# Capture the agent reference for use in the closure
|
||||
captured_agent = agent
|
||||
|
||||
async def mock_ainvoke(*args, **kwargs):
|
||||
# Get the output format from kwargs and use it to parse
|
||||
output_format = kwargs.get('output_format')
|
||||
if output_format is not None:
|
||||
parsed = output_format.model_validate_json(default_done_action)
|
||||
return ChatInvokeCompletion(completion=parsed, usage=None)
|
||||
# Fallback: use the agent's AgentOutput type
|
||||
parsed = captured_agent.AgentOutput.model_validate_json(default_done_action)
|
||||
return ChatInvokeCompletion(completion=parsed, usage=None)
|
||||
|
||||
llm.ainvoke.side_effect = mock_ainvoke
|
||||
return llm
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_model_output_switches_to_fallback_on_rate_limit(self, browser_session):
|
||||
"""Test that get_model_output automatically switches to fallback on rate limit."""
|
||||
from browser_use import Agent
|
||||
|
||||
# Create agent first with a working mock LLM
|
||||
placeholder = create_mock_llm('placeholder')
|
||||
agent = Agent(task='Test task', llm=placeholder, browser_session=browser_session)
|
||||
|
||||
# Create a failing primary and succeeding fallback
|
||||
primary = self._create_failing_mock_llm(
|
||||
'primary-model',
|
||||
fail_with=ModelRateLimitError,
|
||||
fail_status_code=429,
|
||||
fail_message='Rate limit exceeded',
|
||||
)
|
||||
fallback = self._create_succeeding_mock_llm('fallback-model', agent)
|
||||
|
||||
# Replace the LLM and set up fallback
|
||||
agent.llm = primary
|
||||
agent._original_llm = primary
|
||||
agent._fallback_llm = fallback
|
||||
|
||||
from browser_use.llm.messages import BaseMessage, UserMessage
|
||||
|
||||
messages: list[BaseMessage] = [UserMessage(content='Test message')]
|
||||
|
||||
# This should switch to fallback and succeed
|
||||
result = await agent.get_model_output(messages)
|
||||
|
||||
assert result is not None
|
||||
assert agent.llm is fallback
|
||||
assert agent._using_fallback_llm is True
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_model_output_raises_when_no_fallback(self, browser_session):
|
||||
"""Test that get_model_output raises error when no fallback is configured."""
|
||||
from browser_use import Agent
|
||||
|
||||
# Create agent first with a working mock LLM
|
||||
placeholder = create_mock_llm('placeholder')
|
||||
agent = Agent(task='Test task', llm=placeholder, browser_session=browser_session)
|
||||
|
||||
# Replace with failing LLM
|
||||
primary = self._create_failing_mock_llm(
|
||||
'primary-model',
|
||||
fail_with=ModelRateLimitError,
|
||||
fail_status_code=429,
|
||||
fail_message='Rate limit exceeded',
|
||||
)
|
||||
agent.llm = primary
|
||||
agent._original_llm = primary
|
||||
agent._fallback_llm = None # No fallback
|
||||
|
||||
from browser_use.llm.messages import BaseMessage, UserMessage
|
||||
|
||||
messages: list[BaseMessage] = [UserMessage(content='Test message')]
|
||||
|
||||
# This should raise since no fallback is configured
|
||||
with pytest.raises(ModelRateLimitError):
|
||||
await agent.get_model_output(messages)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_model_output_raises_when_fallback_also_fails(self, browser_session):
|
||||
"""Test that error is raised when fallback also fails."""
|
||||
from browser_use import Agent
|
||||
|
||||
# Create agent first with a working mock LLM
|
||||
placeholder = create_mock_llm('placeholder')
|
||||
agent = Agent(task='Test task', llm=placeholder, browser_session=browser_session)
|
||||
|
||||
# Both models fail
|
||||
primary = self._create_failing_mock_llm('primary', fail_with=ModelRateLimitError, fail_status_code=429)
|
||||
fallback = self._create_failing_mock_llm('fallback', fail_with=ModelProviderError, fail_status_code=503)
|
||||
|
||||
agent.llm = primary
|
||||
agent._original_llm = primary
|
||||
agent._fallback_llm = fallback
|
||||
|
||||
from browser_use.llm.messages import BaseMessage, UserMessage
|
||||
|
||||
messages: list[BaseMessage] = [UserMessage(content='Test message')]
|
||||
|
||||
# Should fail after fallback also fails
|
||||
with pytest.raises((ModelRateLimitError, ModelProviderError)):
|
||||
await agent.get_model_output(messages)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
pytest.main([__file__, '-v'])
|
||||
@@ -0,0 +1,192 @@
|
||||
"""Tests for DOCX file support in the FileSystem."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from browser_use.filesystem.file_system import (
|
||||
DocxFile,
|
||||
FileSystem,
|
||||
)
|
||||
|
||||
|
||||
class TestDocxFile:
|
||||
"""Test DOCX file operations."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_create_docx_file(self, tmp_path: Path):
|
||||
"""Test creating a DOCX file."""
|
||||
fs = FileSystem(tmp_path)
|
||||
content = """# Heading 1
|
||||
## Heading 2
|
||||
### Heading 3
|
||||
Regular paragraph text.
|
||||
|
||||
Another paragraph."""
|
||||
|
||||
result = await fs.write_file('test.docx', content)
|
||||
assert 'successfully' in result.lower()
|
||||
assert 'test.docx' in fs.list_files()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_read_docx_file_internal(self, tmp_path: Path):
|
||||
"""Test reading internal DOCX file."""
|
||||
fs = FileSystem(tmp_path)
|
||||
content = """# Title
|
||||
Some content here."""
|
||||
|
||||
await fs.write_file('test.docx', content)
|
||||
result = await fs.read_file('test.docx')
|
||||
|
||||
assert 'test.docx' in result
|
||||
assert 'Title' in result or 'content' in result
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_read_docx_file_external(self, tmp_path: Path):
|
||||
"""Test reading external DOCX file."""
|
||||
from docx import Document
|
||||
|
||||
# Create an external DOCX file
|
||||
external_file = tmp_path / 'external.docx'
|
||||
doc = Document()
|
||||
doc.add_heading('Test Heading', level=1)
|
||||
doc.add_paragraph('Test paragraph content.')
|
||||
doc.save(str(external_file))
|
||||
|
||||
fs = FileSystem(tmp_path / 'workspace')
|
||||
structured_result = await fs.read_file_structured(str(external_file), external_file=True)
|
||||
|
||||
assert 'message' in structured_result
|
||||
assert 'Test Heading' in structured_result['message']
|
||||
assert 'Test paragraph content' in structured_result['message']
|
||||
|
||||
def test_docx_file_extension(self):
|
||||
"""Test DOCX file extension property."""
|
||||
docx_file = DocxFile(name='test')
|
||||
assert docx_file.extension == 'docx'
|
||||
assert docx_file.full_name == 'test.docx'
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_docx_with_unicode_characters(self, tmp_path: Path):
|
||||
"""Test DOCX with unicode and emoji content."""
|
||||
fs = FileSystem(tmp_path)
|
||||
content = """# Unicode Test 🚀
|
||||
Chinese: 你好世界
|
||||
Arabic: مرحبا بالعالم
|
||||
Emoji: 😀 👍 🎉"""
|
||||
|
||||
result = await fs.write_file('unicode.docx', content)
|
||||
assert 'successfully' in result.lower()
|
||||
|
||||
read_result = await fs.read_file('unicode.docx')
|
||||
assert 'Unicode Test' in read_result
|
||||
# Note: Emoji may not be preserved in all systems
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_empty_docx_file(self, tmp_path: Path):
|
||||
"""Test creating an empty DOCX file."""
|
||||
fs = FileSystem(tmp_path)
|
||||
result = await fs.write_file('empty.docx', '')
|
||||
assert 'successfully' in result.lower()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_large_docx_file(self, tmp_path: Path):
|
||||
"""Test creating a large DOCX file."""
|
||||
fs = FileSystem(tmp_path)
|
||||
# Create content with 1000 lines
|
||||
lines = [f'Line {i}: This is a test line with some content.' for i in range(1000)]
|
||||
content = '\n'.join(lines)
|
||||
|
||||
result = await fs.write_file('large.docx', content)
|
||||
assert 'successfully' in result.lower()
|
||||
|
||||
# Verify it can be read back
|
||||
read_result = await fs.read_file('large.docx')
|
||||
assert 'Line 0:' in read_result
|
||||
assert 'Line 999:' in read_result
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_corrupted_docx_file(self, tmp_path: Path):
|
||||
"""Test reading a corrupted DOCX file."""
|
||||
# Create a corrupted DOCX file
|
||||
external_file = tmp_path / 'corrupted.docx'
|
||||
external_file.write_bytes(b'This is not a valid DOCX file')
|
||||
|
||||
fs = FileSystem(tmp_path / 'workspace')
|
||||
structured_result = await fs.read_file_structured(str(external_file), external_file=True)
|
||||
|
||||
assert 'message' in structured_result
|
||||
assert 'error' in structured_result['message'].lower() or 'could not' in structured_result['message'].lower()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_docx_with_multiple_paragraphs(self, tmp_path: Path):
|
||||
"""Test DOCX with various paragraph styles."""
|
||||
fs = FileSystem(tmp_path)
|
||||
content = """# Main Title
|
||||
## Subtitle
|
||||
This is a regular paragraph.
|
||||
|
||||
This is another paragraph with some text.
|
||||
|
||||
### Section 3
|
||||
Final paragraph here."""
|
||||
|
||||
await fs.write_file('multi.docx', content)
|
||||
result = await fs.read_file('multi.docx')
|
||||
|
||||
# Should contain all the text (headings converted to paragraphs)
|
||||
assert 'Main Title' in result
|
||||
assert 'Subtitle' in result
|
||||
assert 'regular paragraph' in result
|
||||
assert 'Final paragraph' in result
|
||||
|
||||
|
||||
class TestFileSystemDocxIntegration:
|
||||
"""Integration tests for DOCX file type."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_multiple_file_types_with_docx(self, tmp_path: Path):
|
||||
"""Test working with DOCX alongside other file types."""
|
||||
fs = FileSystem(tmp_path)
|
||||
|
||||
# Create different file types
|
||||
await fs.write_file('doc.docx', '# Document\nContent here')
|
||||
await fs.write_file('data.json', '{"key": "value"}')
|
||||
await fs.write_file('notes.txt', 'Some notes')
|
||||
|
||||
# Verify all files exist
|
||||
files = fs.list_files()
|
||||
assert 'doc.docx' in files
|
||||
assert 'data.json' in files
|
||||
assert 'notes.txt' in files
|
||||
assert 'todo.md' in files # Default file
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_file_system_state_with_docx(self, tmp_path: Path):
|
||||
"""Test FileSystem state serialization with DOCX files."""
|
||||
fs = FileSystem(tmp_path)
|
||||
|
||||
# Create files
|
||||
await fs.write_file('test.docx', '# Title\nContent')
|
||||
await fs.write_file('data.txt', 'Some text')
|
||||
|
||||
# Get state
|
||||
state = fs.get_state()
|
||||
assert 'test.docx' in state.files
|
||||
assert 'data.txt' in state.files
|
||||
|
||||
# Restore from state
|
||||
fs2 = FileSystem.from_state(state)
|
||||
assert 'test.docx' in fs2.list_files()
|
||||
assert 'data.txt' in fs2.list_files()
|
||||
|
||||
def test_allowed_extensions_include_docx(self, tmp_path: Path):
|
||||
"""Test that DOCX is in allowed extensions."""
|
||||
fs = FileSystem(tmp_path)
|
||||
allowed = fs.get_allowed_extensions()
|
||||
|
||||
assert 'docx' in allowed
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
pytest.main([__file__, '-v'])
|
||||
@@ -0,0 +1,249 @@
|
||||
"""Tests for image file support in the FileSystem."""
|
||||
|
||||
import base64
|
||||
import io
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from PIL import Image
|
||||
|
||||
from browser_use.filesystem.file_system import FileSystem
|
||||
|
||||
|
||||
class TestImageFiles:
|
||||
"""Test image file operations - only external reading supported."""
|
||||
|
||||
def create_test_image(self, width: int = 100, height: int = 100, format: str = 'PNG') -> bytes:
|
||||
"""Create a test image and return bytes."""
|
||||
img = Image.new('RGB', (width, height), color='red')
|
||||
buffer = io.BytesIO()
|
||||
img.save(buffer, format=format)
|
||||
buffer.seek(0)
|
||||
return buffer.read()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_read_external_png_image(self, tmp_path: Path):
|
||||
"""Test reading external PNG image file."""
|
||||
# Create an external image file
|
||||
external_file = tmp_path / 'test.png'
|
||||
img_bytes = self.create_test_image(width=300, height=200, format='PNG')
|
||||
external_file.write_bytes(img_bytes)
|
||||
|
||||
fs = FileSystem(tmp_path / 'workspace')
|
||||
structured_result = await fs.read_file_structured(str(external_file), external_file=True)
|
||||
|
||||
assert 'message' in structured_result
|
||||
assert 'Read image file' in structured_result['message']
|
||||
assert 'images' in structured_result
|
||||
assert structured_result['images'] is not None
|
||||
assert len(structured_result['images']) == 1
|
||||
|
||||
img_data = structured_result['images'][0]
|
||||
assert img_data['name'] == 'test.png'
|
||||
assert 'data' in img_data
|
||||
# Verify base64 is valid
|
||||
decoded = base64.b64decode(img_data['data'])
|
||||
assert decoded == img_bytes
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_read_external_jpg_image(self, tmp_path: Path):
|
||||
"""Test reading external JPG image file."""
|
||||
# Create an external image file
|
||||
external_file = tmp_path / 'photo.jpg'
|
||||
img_bytes = self.create_test_image(width=150, height=100, format='JPEG')
|
||||
external_file.write_bytes(img_bytes)
|
||||
|
||||
fs = FileSystem(tmp_path / 'workspace')
|
||||
structured_result = await fs.read_file_structured(str(external_file), external_file=True)
|
||||
|
||||
assert 'message' in structured_result
|
||||
assert 'images' in structured_result
|
||||
assert structured_result['images'] is not None
|
||||
|
||||
img_data = structured_result['images'][0]
|
||||
assert img_data['name'] == 'photo.jpg'
|
||||
decoded = base64.b64decode(img_data['data'])
|
||||
assert len(decoded) > 0
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_read_jpeg_extension(self, tmp_path: Path):
|
||||
"""Test reading .jpeg extension (not just .jpg)."""
|
||||
external_file = tmp_path / 'test.jpeg'
|
||||
img_bytes = self.create_test_image(format='JPEG')
|
||||
external_file.write_bytes(img_bytes)
|
||||
|
||||
fs = FileSystem(tmp_path / 'workspace')
|
||||
structured_result = await fs.read_file_structured(str(external_file), external_file=True)
|
||||
|
||||
assert structured_result['images'] is not None
|
||||
assert structured_result['images'][0]['name'] == 'test.jpeg'
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_read_nonexistent_image(self, tmp_path: Path):
|
||||
"""Test reading a nonexistent image file."""
|
||||
fs = FileSystem(tmp_path / 'workspace')
|
||||
structured_result = await fs.read_file_structured('/path/to/nonexistent.png', external_file=True)
|
||||
|
||||
assert 'message' in structured_result
|
||||
assert 'not found' in structured_result['message'].lower()
|
||||
assert structured_result['images'] is None
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_corrupted_image_file(self, tmp_path: Path):
|
||||
"""Test reading a corrupted image file."""
|
||||
external_file = tmp_path / 'corrupted.png'
|
||||
# Write invalid PNG data
|
||||
external_file.write_bytes(b'Not a valid PNG file')
|
||||
|
||||
fs = FileSystem(tmp_path / 'workspace')
|
||||
structured_result = await fs.read_file_structured(str(external_file), external_file=True)
|
||||
|
||||
# Should still return base64 data (we don't validate image format)
|
||||
assert 'message' in structured_result
|
||||
assert 'Read image file' in structured_result['message']
|
||||
# Base64 encoding will succeed even for invalid image data
|
||||
assert structured_result['images'] is not None
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_large_image_file(self, tmp_path: Path):
|
||||
"""Test reading a large image file."""
|
||||
# Create a large image (2000x2000)
|
||||
external_file = tmp_path / 'large.png'
|
||||
img = Image.new('RGB', (2000, 2000), color='blue')
|
||||
img.save(str(external_file), format='PNG')
|
||||
|
||||
fs = FileSystem(tmp_path / 'workspace')
|
||||
structured_result = await fs.read_file_structured(str(external_file), external_file=True)
|
||||
|
||||
assert 'images' in structured_result
|
||||
assert structured_result['images'] is not None
|
||||
# Verify base64 data is present and substantial
|
||||
assert len(structured_result['images'][0]['data']) > 10000
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_multiple_images_in_sequence(self, tmp_path: Path):
|
||||
"""Test reading multiple images in sequence."""
|
||||
fs = FileSystem(tmp_path / 'workspace')
|
||||
|
||||
# Create three different images
|
||||
for i, color in enumerate(['red', 'green', 'blue']):
|
||||
img_file = tmp_path / f'image_{i}.png'
|
||||
img = Image.new('RGB', (100, 100), color=color)
|
||||
img.save(str(img_file), format='PNG')
|
||||
|
||||
# Read them all
|
||||
results = []
|
||||
for i in range(3):
|
||||
img_file = tmp_path / f'image_{i}.png'
|
||||
result = await fs.read_file_structured(str(img_file), external_file=True)
|
||||
results.append(result)
|
||||
|
||||
# Verify all were read successfully
|
||||
for i, result in enumerate(results):
|
||||
assert result['images'] is not None
|
||||
assert result['images'][0]['name'] == f'image_{i}.png'
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_different_image_formats(self, tmp_path: Path):
|
||||
"""Test reading different image format variations."""
|
||||
fs = FileSystem(tmp_path / 'workspace')
|
||||
|
||||
# Test .jpg
|
||||
jpg_file = tmp_path / 'test.jpg'
|
||||
img = Image.new('RGB', (50, 50), color='yellow')
|
||||
img.save(str(jpg_file), format='JPEG')
|
||||
result_jpg = await fs.read_file_structured(str(jpg_file), external_file=True)
|
||||
assert result_jpg['images'] is not None
|
||||
|
||||
# Test .jpeg
|
||||
jpeg_file = tmp_path / 'test.jpeg'
|
||||
img.save(str(jpeg_file), format='JPEG')
|
||||
result_jpeg = await fs.read_file_structured(str(jpeg_file), external_file=True)
|
||||
assert result_jpeg['images'] is not None
|
||||
|
||||
# Test .png
|
||||
png_file = tmp_path / 'test.png'
|
||||
img.save(str(png_file), format='PNG')
|
||||
result_png = await fs.read_file_structured(str(png_file), external_file=True)
|
||||
assert result_png['images'] is not None
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_image_with_transparency(self, tmp_path: Path):
|
||||
"""Test reading PNG with transparency (RGBA)."""
|
||||
external_file = tmp_path / 'transparent.png'
|
||||
# Create RGBA image with transparency
|
||||
img = Image.new('RGBA', (100, 100), color=(255, 0, 0, 128))
|
||||
img.save(str(external_file), format='PNG')
|
||||
|
||||
fs = FileSystem(tmp_path / 'workspace')
|
||||
structured_result = await fs.read_file_structured(str(external_file), external_file=True)
|
||||
|
||||
assert structured_result['images'] is not None
|
||||
assert len(structured_result['images'][0]['data']) > 0
|
||||
|
||||
|
||||
class TestActionResultImages:
|
||||
"""Test ActionResult with images field."""
|
||||
|
||||
def test_action_result_with_images(self):
|
||||
"""Test creating ActionResult with images."""
|
||||
from browser_use.agent.views import ActionResult
|
||||
|
||||
images = [{'name': 'test.png', 'data': 'base64_encoded_data_here'}]
|
||||
|
||||
result = ActionResult(
|
||||
extracted_content='Read image file test.png',
|
||||
long_term_memory='Read image file test.png',
|
||||
images=images,
|
||||
include_extracted_content_only_once=True,
|
||||
)
|
||||
|
||||
assert result.images is not None
|
||||
assert len(result.images) == 1
|
||||
assert result.images[0]['name'] == 'test.png'
|
||||
assert result.images[0]['data'] == 'base64_encoded_data_here'
|
||||
|
||||
def test_action_result_without_images(self):
|
||||
"""Test ActionResult without images (default behavior)."""
|
||||
from browser_use.agent.views import ActionResult
|
||||
|
||||
result = ActionResult(extracted_content='Some text', long_term_memory='Memory')
|
||||
|
||||
assert result.images is None
|
||||
|
||||
def test_action_result_with_multiple_images(self):
|
||||
"""Test ActionResult with multiple images."""
|
||||
from browser_use.agent.views import ActionResult
|
||||
|
||||
images = [
|
||||
{'name': 'image1.png', 'data': 'base64_data_1'},
|
||||
{'name': 'image2.jpg', 'data': 'base64_data_2'},
|
||||
]
|
||||
|
||||
result = ActionResult(
|
||||
extracted_content='Read multiple images',
|
||||
long_term_memory='Read image files',
|
||||
images=images,
|
||||
include_extracted_content_only_once=True,
|
||||
)
|
||||
|
||||
assert result.images is not None
|
||||
assert len(result.images) == 2
|
||||
assert result.images[0]['name'] == 'image1.png'
|
||||
assert result.images[1]['name'] == 'image2.jpg'
|
||||
|
||||
def test_action_result_with_empty_images_list(self):
|
||||
"""Test ActionResult with empty images list."""
|
||||
from browser_use.agent.views import ActionResult
|
||||
|
||||
result = ActionResult(
|
||||
extracted_content='No images',
|
||||
images=[],
|
||||
)
|
||||
|
||||
# Empty list is still valid
|
||||
assert result.images == []
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
pytest.main([__file__, '-v'])
|
||||
@@ -0,0 +1,386 @@
|
||||
"""Integration tests for DOCX and image file support in LLM messages."""
|
||||
|
||||
import base64
|
||||
import io
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from PIL import Image
|
||||
|
||||
from browser_use.agent.message_manager.service import MessageManager
|
||||
from browser_use.agent.prompts import AgentMessagePrompt
|
||||
from browser_use.agent.views import ActionResult, AgentStepInfo
|
||||
from browser_use.browser.views import BrowserStateSummary, TabInfo
|
||||
from browser_use.dom.views import SerializedDOMState
|
||||
from browser_use.filesystem.file_system import FileSystem
|
||||
from browser_use.llm.messages import ContentPartImageParam, ContentPartTextParam, SystemMessage
|
||||
|
||||
|
||||
class TestImageInLLMMessages:
|
||||
"""Test that images flow correctly through to LLM messages."""
|
||||
|
||||
def create_test_image(self, width: int = 100, height: int = 100) -> bytes:
|
||||
"""Create a test image and return bytes."""
|
||||
img = Image.new('RGB', (width, height), color='red')
|
||||
buffer = io.BytesIO()
|
||||
img.save(buffer, format='PNG')
|
||||
buffer.seek(0)
|
||||
return buffer.read()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_image_stored_in_message_manager(self, tmp_path: Path):
|
||||
"""Test that images are stored in MessageManager state."""
|
||||
fs = FileSystem(tmp_path)
|
||||
system_message = SystemMessage(content='Test system message')
|
||||
mm = MessageManager(task='test', system_message=system_message, file_system=fs)
|
||||
|
||||
# Create ActionResult with images
|
||||
images = [{'name': 'test.png', 'data': 'base64_test_data'}]
|
||||
action_results = [
|
||||
ActionResult(
|
||||
extracted_content='Read image file test.png',
|
||||
long_term_memory='Read image file test.png',
|
||||
images=images,
|
||||
include_extracted_content_only_once=True,
|
||||
)
|
||||
]
|
||||
|
||||
# Update message manager with results
|
||||
step_info = AgentStepInfo(step_number=1, max_steps=10)
|
||||
mm._update_agent_history_description(model_output=None, result=action_results, step_info=step_info)
|
||||
|
||||
# Verify images are stored
|
||||
assert mm.state.read_state_images is not None
|
||||
assert len(mm.state.read_state_images) == 1
|
||||
assert mm.state.read_state_images[0]['name'] == 'test.png'
|
||||
assert mm.state.read_state_images[0]['data'] == 'base64_test_data'
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_images_cleared_after_step(self, tmp_path: Path):
|
||||
"""Test that images are cleared after each step."""
|
||||
fs = FileSystem(tmp_path)
|
||||
system_message = SystemMessage(content='Test system message')
|
||||
mm = MessageManager(task='test', system_message=system_message, file_system=fs)
|
||||
|
||||
# First step with images
|
||||
images = [{'name': 'test.png', 'data': 'base64_data'}]
|
||||
action_results = [ActionResult(images=images, include_extracted_content_only_once=True)]
|
||||
step_info = AgentStepInfo(step_number=1, max_steps=10)
|
||||
mm._update_agent_history_description(model_output=None, result=action_results, step_info=step_info)
|
||||
|
||||
assert len(mm.state.read_state_images) == 1
|
||||
|
||||
# Second step without images - should clear
|
||||
action_results_2 = [ActionResult(extracted_content='No images')]
|
||||
step_info_2 = AgentStepInfo(step_number=2, max_steps=10)
|
||||
mm._update_agent_history_description(model_output=None, result=action_results_2, step_info=step_info_2)
|
||||
|
||||
assert len(mm.state.read_state_images) == 0
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_multiple_images_accumulated(self, tmp_path: Path):
|
||||
"""Test that multiple images in one step are accumulated."""
|
||||
fs = FileSystem(tmp_path)
|
||||
system_message = SystemMessage(content='Test system message')
|
||||
mm = MessageManager(task='test', system_message=system_message, file_system=fs)
|
||||
|
||||
# Multiple action results with images
|
||||
action_results = [
|
||||
ActionResult(images=[{'name': 'img1.png', 'data': 'data1'}], include_extracted_content_only_once=True),
|
||||
ActionResult(images=[{'name': 'img2.jpg', 'data': 'data2'}], include_extracted_content_only_once=True),
|
||||
]
|
||||
step_info = AgentStepInfo(step_number=1, max_steps=10)
|
||||
mm._update_agent_history_description(model_output=None, result=action_results, step_info=step_info)
|
||||
|
||||
assert len(mm.state.read_state_images) == 2
|
||||
assert mm.state.read_state_images[0]['name'] == 'img1.png'
|
||||
assert mm.state.read_state_images[1]['name'] == 'img2.jpg'
|
||||
|
||||
def test_agent_message_prompt_includes_images(self, tmp_path: Path):
|
||||
"""Test that AgentMessagePrompt includes images in message content."""
|
||||
fs = FileSystem(tmp_path)
|
||||
|
||||
# Create browser state
|
||||
browser_state = BrowserStateSummary(
|
||||
url='https://example.com',
|
||||
title='Test',
|
||||
tabs=[TabInfo(target_id='test-0', url='https://example.com', title='Test')],
|
||||
screenshot=None,
|
||||
dom_state=SerializedDOMState(_root=None, selector_map={}),
|
||||
)
|
||||
|
||||
# Create images
|
||||
read_state_images = [{'name': 'test.png', 'data': 'base64_image_data_here'}]
|
||||
|
||||
# Create message prompt
|
||||
prompt = AgentMessagePrompt(
|
||||
browser_state_summary=browser_state,
|
||||
file_system=fs,
|
||||
read_state_images=read_state_images,
|
||||
)
|
||||
|
||||
# Get user message with vision enabled
|
||||
user_message = prompt.get_user_message(use_vision=True)
|
||||
|
||||
# Verify message has content parts (not just string)
|
||||
assert isinstance(user_message.content, list)
|
||||
|
||||
# Find image content parts
|
||||
image_parts = [part for part in user_message.content if isinstance(part, ContentPartImageParam)]
|
||||
text_parts = [part for part in user_message.content if isinstance(part, ContentPartTextParam)]
|
||||
|
||||
# Should have at least one image
|
||||
assert len(image_parts) >= 1
|
||||
|
||||
# Should have text label
|
||||
image_labels = [part.text for part in text_parts if 'test.png' in part.text]
|
||||
assert len(image_labels) >= 1
|
||||
|
||||
# Verify image data URL format
|
||||
img_part = image_parts[0]
|
||||
assert 'data:image/' in img_part.image_url.url
|
||||
assert 'base64,base64_image_data_here' in img_part.image_url.url
|
||||
|
||||
def test_agent_message_prompt_png_vs_jpg_media_type(self, tmp_path: Path):
|
||||
"""Test that AgentMessagePrompt correctly detects PNG vs JPG media types."""
|
||||
fs = FileSystem(tmp_path)
|
||||
|
||||
browser_state = BrowserStateSummary(
|
||||
url='https://example.com',
|
||||
title='Test',
|
||||
tabs=[TabInfo(target_id='test-0', url='https://example.com', title='Test')],
|
||||
screenshot=None,
|
||||
dom_state=SerializedDOMState(_root=None, selector_map={}),
|
||||
)
|
||||
|
||||
# Test PNG
|
||||
read_state_images_png = [{'name': 'test.png', 'data': 'data'}]
|
||||
prompt_png = AgentMessagePrompt(
|
||||
browser_state_summary=browser_state,
|
||||
file_system=fs,
|
||||
read_state_images=read_state_images_png,
|
||||
)
|
||||
message_png = prompt_png.get_user_message(use_vision=True)
|
||||
image_parts_png = [part for part in message_png.content if isinstance(part, ContentPartImageParam)]
|
||||
assert 'data:image/png;base64' in image_parts_png[0].image_url.url
|
||||
|
||||
# Test JPG
|
||||
read_state_images_jpg = [{'name': 'photo.jpg', 'data': 'data'}]
|
||||
prompt_jpg = AgentMessagePrompt(
|
||||
browser_state_summary=browser_state,
|
||||
file_system=fs,
|
||||
read_state_images=read_state_images_jpg,
|
||||
)
|
||||
message_jpg = prompt_jpg.get_user_message(use_vision=True)
|
||||
image_parts_jpg = [part for part in message_jpg.content if isinstance(part, ContentPartImageParam)]
|
||||
assert 'data:image/jpeg;base64' in image_parts_jpg[0].image_url.url
|
||||
|
||||
def test_agent_message_prompt_no_images(self, tmp_path: Path):
|
||||
"""Test that message works correctly when no images are present."""
|
||||
fs = FileSystem(tmp_path)
|
||||
|
||||
browser_state = BrowserStateSummary(
|
||||
url='https://example.com',
|
||||
title='Test',
|
||||
tabs=[TabInfo(target_id='test-0', url='https://example.com', title='Test')],
|
||||
screenshot=None,
|
||||
dom_state=SerializedDOMState(_root=None, selector_map={}),
|
||||
)
|
||||
|
||||
# No images
|
||||
prompt = AgentMessagePrompt(
|
||||
browser_state_summary=browser_state,
|
||||
file_system=fs,
|
||||
read_state_images=[],
|
||||
)
|
||||
|
||||
# Get user message without vision
|
||||
user_message = prompt.get_user_message(use_vision=False)
|
||||
|
||||
# Should be plain text, not content parts
|
||||
assert isinstance(user_message.content, str)
|
||||
|
||||
def test_agent_message_prompt_empty_base64_skipped(self, tmp_path: Path):
|
||||
"""Test that images with empty base64 data are skipped."""
|
||||
fs = FileSystem(tmp_path)
|
||||
|
||||
browser_state = BrowserStateSummary(
|
||||
url='https://example.com',
|
||||
title='Test',
|
||||
tabs=[TabInfo(target_id='test-0', url='https://example.com', title='Test')],
|
||||
screenshot=None,
|
||||
dom_state=SerializedDOMState(_root=None, selector_map={}),
|
||||
)
|
||||
|
||||
# Image with empty data field
|
||||
read_state_images = [
|
||||
{'name': 'empty.png', 'data': ''}, # Empty - should be skipped
|
||||
{'name': 'valid.png', 'data': 'valid_data'}, # Valid
|
||||
]
|
||||
|
||||
prompt = AgentMessagePrompt(
|
||||
browser_state_summary=browser_state,
|
||||
file_system=fs,
|
||||
read_state_images=read_state_images,
|
||||
)
|
||||
|
||||
user_message = prompt.get_user_message(use_vision=True)
|
||||
image_parts = [part for part in user_message.content if isinstance(part, ContentPartImageParam)]
|
||||
|
||||
# Should only have 1 image (the valid one)
|
||||
assert len(image_parts) == 1
|
||||
assert 'valid_data' in image_parts[0].image_url.url
|
||||
|
||||
|
||||
class TestDocxInLLMMessages:
|
||||
"""Test that DOCX content flows correctly through to LLM messages."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_docx_in_extracted_content(self, tmp_path: Path):
|
||||
"""Test that DOCX text appears in extracted_content."""
|
||||
fs = FileSystem(tmp_path)
|
||||
|
||||
# Create DOCX file
|
||||
content = """# Title
|
||||
Some important content here."""
|
||||
await fs.write_file('test.docx', content)
|
||||
|
||||
# Read it
|
||||
result = await fs.read_file('test.docx')
|
||||
|
||||
# Verify content is in the result
|
||||
assert 'Title' in result
|
||||
assert 'important content' in result
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_docx_in_message_manager(self, tmp_path: Path):
|
||||
"""Test that DOCX content appears in message manager state."""
|
||||
fs = FileSystem(tmp_path)
|
||||
system_message = SystemMessage(content='Test system message')
|
||||
mm = MessageManager(task='test', system_message=system_message, file_system=fs)
|
||||
|
||||
# Simulate read_file action result
|
||||
docx_content = """Read from file test.docx.
|
||||
<content>
|
||||
Title
|
||||
Some content here.
|
||||
</content>"""
|
||||
|
||||
action_results = [
|
||||
ActionResult(
|
||||
extracted_content=docx_content,
|
||||
long_term_memory='Read file test.docx',
|
||||
include_extracted_content_only_once=True,
|
||||
)
|
||||
]
|
||||
|
||||
step_info = AgentStepInfo(step_number=1, max_steps=10)
|
||||
mm._update_agent_history_description(model_output=None, result=action_results, step_info=step_info)
|
||||
|
||||
# Verify it's in read_state_description
|
||||
assert 'Title' in mm.state.read_state_description
|
||||
assert 'Some content' in mm.state.read_state_description
|
||||
|
||||
|
||||
class TestEndToEndIntegration:
|
||||
"""End-to-end tests for file reading and LLM message creation."""
|
||||
|
||||
def create_test_image(self) -> bytes:
|
||||
"""Create a test image."""
|
||||
img = Image.new('RGB', (50, 50), color='blue')
|
||||
buffer = io.BytesIO()
|
||||
img.save(buffer, format='PNG')
|
||||
buffer.seek(0)
|
||||
return buffer.read()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_image_end_to_end(self, tmp_path: Path):
|
||||
"""Test complete flow: external image → FileSystem → ActionResult → MessageManager → Prompt."""
|
||||
# Step 1: Create external image
|
||||
external_file = tmp_path / 'photo.png'
|
||||
img_bytes = self.create_test_image()
|
||||
external_file.write_bytes(img_bytes)
|
||||
|
||||
# Step 2: Read via FileSystem
|
||||
fs = FileSystem(tmp_path / 'workspace')
|
||||
structured_result = await fs.read_file_structured(str(external_file), external_file=True)
|
||||
|
||||
assert structured_result['images'] is not None
|
||||
|
||||
# Step 3: Create ActionResult (simulating tools/service.py)
|
||||
action_result = ActionResult(
|
||||
extracted_content=structured_result['message'],
|
||||
long_term_memory='Read image file photo.png',
|
||||
images=structured_result['images'],
|
||||
include_extracted_content_only_once=True,
|
||||
)
|
||||
|
||||
# Step 4: Process in MessageManager
|
||||
system_message = SystemMessage(content='Test system message')
|
||||
mm = MessageManager(task='test', system_message=system_message, file_system=fs)
|
||||
step_info = AgentStepInfo(step_number=1, max_steps=10)
|
||||
mm._update_agent_history_description(model_output=None, result=[action_result], step_info=step_info)
|
||||
|
||||
# Verify images stored
|
||||
assert len(mm.state.read_state_images) == 1
|
||||
assert mm.state.read_state_images[0]['name'] == 'photo.png'
|
||||
|
||||
# Step 5: Create message with AgentMessagePrompt
|
||||
browser_state = BrowserStateSummary(
|
||||
url='https://example.com',
|
||||
title='Test',
|
||||
tabs=[TabInfo(target_id='test-0', url='https://example.com', title='Test')],
|
||||
screenshot=None,
|
||||
dom_state=SerializedDOMState(_root=None, selector_map={}),
|
||||
)
|
||||
|
||||
prompt = AgentMessagePrompt(
|
||||
browser_state_summary=browser_state,
|
||||
file_system=fs,
|
||||
read_state_images=mm.state.read_state_images,
|
||||
)
|
||||
|
||||
user_message = prompt.get_user_message(use_vision=True)
|
||||
|
||||
# Verify image is in message
|
||||
assert isinstance(user_message.content, list)
|
||||
image_parts = [part for part in user_message.content if isinstance(part, ContentPartImageParam)]
|
||||
assert len(image_parts) >= 1
|
||||
|
||||
# Verify image data is correct
|
||||
base64_str = base64.b64encode(img_bytes).decode('utf-8')
|
||||
assert base64_str in image_parts[0].image_url.url
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_docx_end_to_end(self, tmp_path: Path):
|
||||
"""Test complete flow: DOCX file → FileSystem → ActionResult → MessageManager."""
|
||||
# Step 1: Create DOCX
|
||||
fs = FileSystem(tmp_path)
|
||||
docx_content = """# Important Document
|
||||
This is critical information."""
|
||||
|
||||
await fs.write_file('important.docx', docx_content)
|
||||
|
||||
# Step 2: Read it
|
||||
read_result = await fs.read_file('important.docx')
|
||||
|
||||
# Step 3: Create ActionResult (simulating tools/service.py)
|
||||
action_result = ActionResult(
|
||||
extracted_content=read_result,
|
||||
long_term_memory=read_result[:100] if len(read_result) > 100 else read_result,
|
||||
include_extracted_content_only_once=True,
|
||||
)
|
||||
|
||||
# Step 4: Process in MessageManager
|
||||
system_message = SystemMessage(content='Test system message')
|
||||
mm = MessageManager(task='test', system_message=system_message, file_system=fs)
|
||||
step_info = AgentStepInfo(step_number=1, max_steps=10)
|
||||
mm._update_agent_history_description(model_output=None, result=[action_result], step_info=step_info)
|
||||
|
||||
# Verify content is in read_state
|
||||
assert 'Important Document' in mm.state.read_state_description
|
||||
assert 'critical information' in mm.state.read_state_description
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
pytest.main([__file__, '-v'])
|
||||
@@ -0,0 +1,119 @@
|
||||
from browser_use.agent.views import StepMetadata
|
||||
|
||||
|
||||
def test_step_metadata_has_step_interval_field():
|
||||
"""Test that StepMetadata includes step_interval field"""
|
||||
metadata = StepMetadata(step_number=1, step_start_time=10.0, step_end_time=12.5, step_interval=2.5)
|
||||
|
||||
assert hasattr(metadata, 'step_interval')
|
||||
assert metadata.step_interval == 2.5
|
||||
|
||||
|
||||
def test_step_metadata_step_interval_optional():
|
||||
"""Test that step_interval is optional (None for first step)"""
|
||||
# Explicitly None
|
||||
metadata_none = StepMetadata(step_number=0, step_start_time=0.0, step_end_time=1.0, step_interval=None)
|
||||
assert metadata_none.step_interval is None
|
||||
|
||||
# Omitted (defaults to None)
|
||||
metadata_default = StepMetadata(step_number=0, step_start_time=0.0, step_end_time=1.0)
|
||||
assert metadata_default.step_interval is None
|
||||
|
||||
|
||||
def test_step_interval_calculation():
|
||||
"""Test step_interval calculation logic (uses previous step's duration)"""
|
||||
# Previous step (Step 1): runs from 100.0 to 102.5 (duration: 2.5s)
|
||||
previous_start = 100.0
|
||||
previous_end = 102.5
|
||||
previous_duration = previous_end - previous_start
|
||||
|
||||
# Current step (Step 2): should have step_interval = previous step's duration
|
||||
# This tells the rerun system "wait 2.5s before executing Step 2"
|
||||
expected_step_interval = previous_duration
|
||||
calculated_step_interval = max(0, previous_end - previous_start)
|
||||
|
||||
assert abs(calculated_step_interval - expected_step_interval) < 0.001 # Float comparison
|
||||
assert calculated_step_interval == 2.5
|
||||
|
||||
|
||||
def test_step_metadata_serialization_with_step_interval():
|
||||
"""Test that step_interval is included in metadata serialization"""
|
||||
# With step_interval
|
||||
metadata_with_wait = StepMetadata(step_number=1, step_start_time=10.0, step_end_time=12.5, step_interval=2.5)
|
||||
|
||||
data = metadata_with_wait.model_dump()
|
||||
assert 'step_interval' in data
|
||||
assert data['step_interval'] == 2.5
|
||||
|
||||
# Without step_interval (None)
|
||||
metadata_without_wait = StepMetadata(step_number=0, step_start_time=0.0, step_end_time=1.0, step_interval=None)
|
||||
|
||||
data = metadata_without_wait.model_dump()
|
||||
assert 'step_interval' in data
|
||||
assert data['step_interval'] is None
|
||||
|
||||
|
||||
def test_step_metadata_deserialization_with_step_interval():
|
||||
"""Test that step_interval can be loaded from dict"""
|
||||
# Load with step_interval
|
||||
data_with_wait = {'step_number': 1, 'step_start_time': 10.0, 'step_end_time': 12.5, 'step_interval': 2.5}
|
||||
|
||||
metadata = StepMetadata.model_validate(data_with_wait)
|
||||
assert metadata.step_interval == 2.5
|
||||
|
||||
# Load without step_interval (old format)
|
||||
data_without_wait = {
|
||||
'step_number': 0,
|
||||
'step_start_time': 0.0,
|
||||
'step_end_time': 1.0,
|
||||
# step_interval is missing
|
||||
}
|
||||
|
||||
metadata = StepMetadata.model_validate(data_without_wait)
|
||||
assert metadata.step_interval is None # Defaults to None
|
||||
|
||||
|
||||
def test_step_interval_backwards_compatibility():
|
||||
"""Test that old metadata without step_interval still works"""
|
||||
# Simulate old format from JSON
|
||||
old_metadata_dict = {
|
||||
'step_number': 0,
|
||||
'step_start_time': 1000.0,
|
||||
'step_end_time': 1002.5,
|
||||
# step_interval field doesn't exist (old format)
|
||||
}
|
||||
|
||||
# Should load successfully with step_interval defaulting to None
|
||||
metadata = StepMetadata.model_validate(old_metadata_dict)
|
||||
|
||||
assert metadata.step_number == 0
|
||||
assert metadata.step_start_time == 1000.0
|
||||
assert metadata.step_end_time == 1002.5
|
||||
assert metadata.step_interval is None # Default value
|
||||
|
||||
|
||||
def test_duration_seconds_property_still_works():
|
||||
"""Test that existing duration_seconds property still works"""
|
||||
metadata = StepMetadata(step_number=1, step_start_time=10.0, step_end_time=13.5, step_interval=2.0)
|
||||
|
||||
# duration_seconds should be 3.5 (13.5 - 10.0)
|
||||
assert metadata.duration_seconds == 3.5
|
||||
|
||||
# step_interval is separate from duration
|
||||
assert metadata.step_interval == 2.0
|
||||
|
||||
|
||||
def test_step_metadata_json_round_trip():
|
||||
"""Test that step_interval survives JSON serialization round-trip"""
|
||||
metadata = StepMetadata(step_number=1, step_start_time=100.0, step_end_time=102.5, step_interval=1.5)
|
||||
|
||||
# Serialize to JSON
|
||||
json_str = metadata.model_dump_json()
|
||||
|
||||
# Deserialize from JSON
|
||||
loaded = StepMetadata.model_validate_json(json_str)
|
||||
|
||||
assert loaded.step_interval == 1.5
|
||||
assert loaded.step_number == 1
|
||||
assert loaded.step_start_time == 100.0
|
||||
assert loaded.step_end_time == 102.5
|
||||
@@ -0,0 +1,141 @@
|
||||
"""Regression test: structured output cut off at the completion-token cap must raise a
|
||||
clear truncation error, not a misleading JSON parse error ('Unterminated string...')."""
|
||||
|
||||
import pytest
|
||||
from pydantic import BaseModel
|
||||
|
||||
from browser_use.llm.exceptions import ModelProviderError
|
||||
from browser_use.llm.messages import UserMessage
|
||||
from browser_use.llm.openai.chat import ChatOpenAI
|
||||
|
||||
|
||||
class AnswerFormat(BaseModel):
|
||||
answer: str
|
||||
|
||||
|
||||
async def test_openai_truncated_structured_output_raises_clear_error(httpserver):
|
||||
"""finish_reason='length' with JSON cut mid-string must surface the token cap, not a parse error."""
|
||||
httpserver.expect_request('/v1/chat/completions', method='POST').respond_with_json(
|
||||
{
|
||||
'id': 'chatcmpl-test',
|
||||
'object': 'chat.completion',
|
||||
'created': 0,
|
||||
'model': 'gpt-4o',
|
||||
'choices': [
|
||||
{
|
||||
'index': 0,
|
||||
'message': {'role': 'assistant', 'content': '{"answer": "this output was cut off mid-sent'},
|
||||
'finish_reason': 'length',
|
||||
}
|
||||
],
|
||||
'usage': {'prompt_tokens': 10, 'completion_tokens': 4096, 'total_tokens': 4106},
|
||||
}
|
||||
)
|
||||
|
||||
llm = ChatOpenAI(model='gpt-4o', api_key='test-key', base_url=httpserver.url_for('/v1'))
|
||||
|
||||
with pytest.raises(ModelProviderError) as exc_info:
|
||||
await llm.ainvoke([UserMessage(content='answer at length')], output_format=AnswerFormat)
|
||||
|
||||
assert 'truncated' in str(exc_info.value), f'expected a truncation error, got: {exc_info.value}'
|
||||
assert 'max_completion_tokens' in str(exc_info.value)
|
||||
|
||||
|
||||
async def test_openai_normal_structured_output_still_parses(httpserver):
|
||||
httpserver.expect_request('/v1/chat/completions', method='POST').respond_with_json(
|
||||
{
|
||||
'id': 'chatcmpl-test',
|
||||
'object': 'chat.completion',
|
||||
'created': 0,
|
||||
'model': 'gpt-4o',
|
||||
'choices': [
|
||||
{
|
||||
'index': 0,
|
||||
'message': {'role': 'assistant', 'content': '{"answer": "complete answer"}'},
|
||||
'finish_reason': 'stop',
|
||||
}
|
||||
],
|
||||
'usage': {'prompt_tokens': 10, 'completion_tokens': 8, 'total_tokens': 18},
|
||||
}
|
||||
)
|
||||
|
||||
llm = ChatOpenAI(model='gpt-4o', api_key='test-key', base_url=httpserver.url_for('/v1'))
|
||||
|
||||
result = await llm.ainvoke([UserMessage(content='answer briefly')], output_format=AnswerFormat)
|
||||
assert result.completion.answer == 'complete answer'
|
||||
|
||||
|
||||
async def test_truncation_error_readable_when_cap_unset(httpserver):
|
||||
"""With max_completion_tokens=None, the error must describe the limit without printing 'None'."""
|
||||
httpserver.expect_request('/v1/chat/completions', method='POST').respond_with_json(
|
||||
{
|
||||
'id': 'chatcmpl-test',
|
||||
'object': 'chat.completion',
|
||||
'created': 0,
|
||||
'model': 'gpt-4o',
|
||||
'choices': [
|
||||
{
|
||||
'index': 0,
|
||||
'message': {'role': 'assistant', 'content': '{"answer": "cut off mid'},
|
||||
'finish_reason': 'length',
|
||||
}
|
||||
],
|
||||
'usage': {'prompt_tokens': 10, 'completion_tokens': 16384, 'total_tokens': 16394},
|
||||
}
|
||||
)
|
||||
|
||||
llm = ChatOpenAI(model='gpt-4o', api_key='test-key', base_url=httpserver.url_for('/v1'), max_completion_tokens=None)
|
||||
|
||||
with pytest.raises(ModelProviderError) as exc_info:
|
||||
await llm.ainvoke([UserMessage(content='answer at length')], output_format=AnswerFormat)
|
||||
|
||||
assert 'truncated' in str(exc_info.value)
|
||||
assert 'None' not in str(exc_info.value), f'error message leaks None: {exc_info.value}'
|
||||
|
||||
|
||||
def test_truncation_error_triggers_fallback_llm_switch(tmp_path):
|
||||
"""A truncation error must allow switching to a configured fallback LLM — a
|
||||
fallback with a different output cap can succeed where the primary truncated."""
|
||||
from browser_use.agent.service import Agent
|
||||
from browser_use.llm.exceptions import ModelOutputTruncatedError
|
||||
from tests.ci.conftest import create_mock_llm
|
||||
|
||||
agent = Agent(
|
||||
task='test fallback on truncation',
|
||||
llm=create_mock_llm(),
|
||||
fallback_llm=create_mock_llm(),
|
||||
file_system_path=str(tmp_path / 'agent-files'),
|
||||
)
|
||||
|
||||
error = ModelOutputTruncatedError(message='Model output was truncated at max_completion_tokens=4096', model='gpt-4o')
|
||||
assert agent._try_switch_to_fallback_llm(error) is True
|
||||
assert agent._using_fallback_llm is True
|
||||
|
||||
|
||||
async def test_truncation_detected_when_content_is_null(httpserver):
|
||||
"""Reasoning models can spend the whole completion budget on hidden reasoning:
|
||||
finish_reason='length' with content=null. That must surface as truncation, not
|
||||
the generic 'Failed to parse structured output'."""
|
||||
httpserver.expect_request('/v1/chat/completions', method='POST').respond_with_json(
|
||||
{
|
||||
'id': 'chatcmpl-test',
|
||||
'object': 'chat.completion',
|
||||
'created': 0,
|
||||
'model': 'o3',
|
||||
'choices': [
|
||||
{
|
||||
'index': 0,
|
||||
'message': {'role': 'assistant', 'content': None},
|
||||
'finish_reason': 'length',
|
||||
}
|
||||
],
|
||||
'usage': {'prompt_tokens': 10, 'completion_tokens': 4096, 'total_tokens': 4106},
|
||||
}
|
||||
)
|
||||
|
||||
llm = ChatOpenAI(model='o3', api_key='test-key', base_url=httpserver.url_for('/v1'))
|
||||
|
||||
with pytest.raises(ModelProviderError) as exc_info:
|
||||
await llm.ainvoke([UserMessage(content='think hard')], output_format=AnswerFormat)
|
||||
|
||||
assert 'truncated' in str(exc_info.value), f'expected truncation signal, got: {exc_info.value}'
|
||||
@@ -0,0 +1,304 @@
|
||||
"""
|
||||
Test retry logic with exponential backoff for LLM clients.
|
||||
"""
|
||||
|
||||
import time
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
|
||||
class TestChatBrowserUseRetries:
|
||||
"""Test retry logic for ChatBrowserUse."""
|
||||
|
||||
@pytest.fixture
|
||||
def mock_env(self, monkeypatch):
|
||||
"""Set up environment for ChatBrowserUse."""
|
||||
monkeypatch.setenv('BROWSER_USE_API_KEY', 'test-api-key')
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_retries_on_503_with_exponential_backoff(self, mock_env):
|
||||
"""Test that 503 errors trigger retries with exponential backoff."""
|
||||
from browser_use.llm.browser_use.chat import ChatBrowserUse
|
||||
from browser_use.llm.messages import UserMessage
|
||||
|
||||
# Track timing of each attempt
|
||||
attempt_times: list[float] = []
|
||||
attempt_count = 0
|
||||
|
||||
async def mock_post(*args, **kwargs):
|
||||
nonlocal attempt_count
|
||||
attempt_times.append(time.monotonic())
|
||||
attempt_count += 1
|
||||
|
||||
if attempt_count < 3:
|
||||
# First 2 attempts fail with 503
|
||||
response = MagicMock()
|
||||
response.status_code = 503
|
||||
response.json.return_value = {'detail': 'Service temporarily unavailable'}
|
||||
raise httpx.HTTPStatusError('503', request=MagicMock(), response=response)
|
||||
else:
|
||||
# Third attempt succeeds
|
||||
response = MagicMock()
|
||||
response.json.return_value = {
|
||||
'completion': 'Success!',
|
||||
'usage': {
|
||||
'prompt_tokens': 10,
|
||||
'completion_tokens': 5,
|
||||
'total_tokens': 15,
|
||||
'prompt_cached_tokens': None,
|
||||
'prompt_cache_creation_tokens': None,
|
||||
'prompt_image_tokens': None,
|
||||
},
|
||||
}
|
||||
response.raise_for_status = MagicMock()
|
||||
return response
|
||||
|
||||
with patch('httpx.AsyncClient') as mock_client_class:
|
||||
mock_client = AsyncMock()
|
||||
mock_client.post = mock_post
|
||||
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
|
||||
mock_client.__aexit__ = AsyncMock(return_value=None)
|
||||
mock_client_class.return_value = mock_client
|
||||
|
||||
# Use short delays for testing
|
||||
client = ChatBrowserUse(retry_base_delay=0.1, retry_max_delay=1.0)
|
||||
result = await client.ainvoke([UserMessage(content='test')])
|
||||
|
||||
# Should have made 3 attempts
|
||||
assert attempt_count == 3
|
||||
assert result.completion == 'Success!'
|
||||
|
||||
# Verify exponential backoff timing
|
||||
# base_delay=0.1s, so first retry sleeps ~0.1s, second ~0.2s (2x exponential)
|
||||
delay_1 = attempt_times[1] - attempt_times[0]
|
||||
delay_2 = attempt_times[2] - attempt_times[1]
|
||||
|
||||
# Allow some tolerance for CI runner scheduling jitter
|
||||
assert 0.05 <= delay_1 <= 0.3, f'First delay {delay_1:.3f}s not in expected range'
|
||||
assert 0.1 <= delay_2 <= 0.5, f'Second delay {delay_2:.3f}s not in expected range'
|
||||
# Second delay should be roughly 2x the first (exponential)
|
||||
assert delay_2 > delay_1, 'Second delay should be longer than first (exponential backoff)'
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_no_retry_on_401(self, mock_env):
|
||||
"""Test that 401 errors do NOT trigger retries."""
|
||||
from browser_use.llm.browser_use.chat import ChatBrowserUse
|
||||
from browser_use.llm.exceptions import ModelProviderError
|
||||
from browser_use.llm.messages import UserMessage
|
||||
|
||||
attempt_count = 0
|
||||
|
||||
async def mock_post(*args, **kwargs):
|
||||
nonlocal attempt_count
|
||||
attempt_count += 1
|
||||
response = MagicMock()
|
||||
response.status_code = 401
|
||||
response.json.return_value = {'detail': 'Invalid API key'}
|
||||
raise httpx.HTTPStatusError('401', request=MagicMock(), response=response)
|
||||
|
||||
with patch('httpx.AsyncClient') as mock_client_class:
|
||||
mock_client = AsyncMock()
|
||||
mock_client.post = mock_post
|
||||
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
|
||||
mock_client.__aexit__ = AsyncMock(return_value=None)
|
||||
mock_client_class.return_value = mock_client
|
||||
|
||||
client = ChatBrowserUse(retry_base_delay=0.01)
|
||||
|
||||
with pytest.raises(ModelProviderError, match='Invalid API key'):
|
||||
await client.ainvoke([UserMessage(content='test')])
|
||||
|
||||
# Should only attempt once (no retries for 401)
|
||||
assert attempt_count == 1
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_retries_on_timeout(self, mock_env):
|
||||
"""Test that timeouts trigger retries."""
|
||||
from browser_use.llm.browser_use.chat import ChatBrowserUse
|
||||
from browser_use.llm.messages import UserMessage
|
||||
|
||||
attempt_count = 0
|
||||
|
||||
async def mock_post(*args, **kwargs):
|
||||
nonlocal attempt_count
|
||||
attempt_count += 1
|
||||
if attempt_count < 2:
|
||||
raise httpx.TimeoutException('Request timed out')
|
||||
# Second attempt succeeds (with no usage data to test None handling)
|
||||
response = MagicMock()
|
||||
response.json.return_value = {'completion': 'Success after timeout!', 'usage': None}
|
||||
response.raise_for_status = MagicMock()
|
||||
return response
|
||||
|
||||
with patch('httpx.AsyncClient') as mock_client_class:
|
||||
mock_client = AsyncMock()
|
||||
mock_client.post = mock_post
|
||||
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
|
||||
mock_client.__aexit__ = AsyncMock(return_value=None)
|
||||
mock_client_class.return_value = mock_client
|
||||
|
||||
client = ChatBrowserUse(retry_base_delay=0.01)
|
||||
result = await client.ainvoke([UserMessage(content='test')])
|
||||
|
||||
assert attempt_count == 2
|
||||
assert result.completion == 'Success after timeout!'
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_max_retries_exhausted(self, mock_env):
|
||||
"""Test that error is raised after max retries exhausted."""
|
||||
from browser_use.llm.browser_use.chat import ChatBrowserUse
|
||||
from browser_use.llm.exceptions import ModelProviderError
|
||||
from browser_use.llm.messages import UserMessage
|
||||
|
||||
attempt_count = 0
|
||||
|
||||
async def mock_post(*args, **kwargs):
|
||||
nonlocal attempt_count
|
||||
attempt_count += 1
|
||||
response = MagicMock()
|
||||
response.status_code = 503
|
||||
response.json.return_value = {'detail': 'Service unavailable'}
|
||||
raise httpx.HTTPStatusError('503', request=MagicMock(), response=response)
|
||||
|
||||
with patch('httpx.AsyncClient') as mock_client_class:
|
||||
mock_client = AsyncMock()
|
||||
mock_client.post = mock_post
|
||||
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
|
||||
mock_client.__aexit__ = AsyncMock(return_value=None)
|
||||
mock_client_class.return_value = mock_client
|
||||
|
||||
client = ChatBrowserUse(max_retries=3, retry_base_delay=0.01)
|
||||
|
||||
with pytest.raises(ModelProviderError, match='Server error'):
|
||||
await client.ainvoke([UserMessage(content='test')])
|
||||
|
||||
# Should have attempted max_retries times
|
||||
assert attempt_count == 3
|
||||
|
||||
|
||||
class TestChatGoogleRetries:
|
||||
"""Test retry logic for ChatGoogle."""
|
||||
|
||||
@pytest.fixture
|
||||
def mock_env(self, monkeypatch):
|
||||
"""Set up environment for ChatGoogle."""
|
||||
monkeypatch.setenv('GOOGLE_API_KEY', 'test-api-key')
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_retries_on_503_with_exponential_backoff(self, mock_env):
|
||||
"""Test that 503 errors trigger retries with exponential backoff."""
|
||||
from browser_use.llm.exceptions import ModelProviderError
|
||||
from browser_use.llm.google.chat import ChatGoogle
|
||||
from browser_use.llm.messages import UserMessage
|
||||
|
||||
attempt_times: list[float] = []
|
||||
attempt_count = 0
|
||||
|
||||
# Mock the genai client
|
||||
with patch('browser_use.llm.google.chat.genai') as mock_genai:
|
||||
mock_client = MagicMock()
|
||||
mock_genai.Client.return_value = mock_client
|
||||
|
||||
async def mock_generate(*args, **kwargs):
|
||||
nonlocal attempt_count
|
||||
attempt_times.append(time.monotonic())
|
||||
attempt_count += 1
|
||||
|
||||
if attempt_count < 3:
|
||||
raise ModelProviderError(message='Service unavailable', status_code=503, model='gemini-2.0-flash')
|
||||
else:
|
||||
# Success on third attempt
|
||||
mock_response = MagicMock()
|
||||
mock_response.text = 'Success!'
|
||||
mock_response.usage_metadata = MagicMock(
|
||||
prompt_token_count=10, candidates_token_count=5, total_token_count=15, cached_content_token_count=0
|
||||
)
|
||||
mock_response.candidates = [MagicMock(content=MagicMock(parts=[MagicMock(text='Success!')]))]
|
||||
return mock_response
|
||||
|
||||
# Mock the aio.models.generate_content path
|
||||
mock_client.aio.models.generate_content = mock_generate
|
||||
|
||||
client = ChatGoogle(model='gemini-2.0-flash', api_key='test', retry_base_delay=0.1, retry_max_delay=1.0)
|
||||
result = await client.ainvoke([UserMessage(content='test')])
|
||||
|
||||
assert attempt_count == 3
|
||||
assert result.completion == 'Success!'
|
||||
|
||||
# Verify exponential backoff
|
||||
delay_1 = attempt_times[1] - attempt_times[0]
|
||||
delay_2 = attempt_times[2] - attempt_times[1]
|
||||
|
||||
assert 0.05 <= delay_1 <= 0.3, f'First delay {delay_1:.3f}s not in expected range'
|
||||
assert 0.1 <= delay_2 <= 0.5, f'Second delay {delay_2:.3f}s not in expected range'
|
||||
assert delay_2 > delay_1, 'Second delay should be longer than first'
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_no_retry_on_400(self, mock_env):
|
||||
"""Test that 400 errors do NOT trigger retries."""
|
||||
from browser_use.llm.exceptions import ModelProviderError
|
||||
from browser_use.llm.google.chat import ChatGoogle
|
||||
from browser_use.llm.messages import UserMessage
|
||||
|
||||
attempt_count = 0
|
||||
|
||||
with patch('browser_use.llm.google.chat.genai') as mock_genai:
|
||||
mock_client = MagicMock()
|
||||
mock_genai.Client.return_value = mock_client
|
||||
|
||||
async def mock_generate(*args, **kwargs):
|
||||
nonlocal attempt_count
|
||||
attempt_count += 1
|
||||
raise ModelProviderError(message='Bad request', status_code=400, model='gemini-2.0-flash')
|
||||
|
||||
mock_client.aio.models.generate_content = mock_generate
|
||||
|
||||
client = ChatGoogle(model='gemini-2.0-flash', api_key='test', retry_base_delay=0.01)
|
||||
|
||||
with pytest.raises(ModelProviderError):
|
||||
await client.ainvoke([UserMessage(content='test')])
|
||||
|
||||
# Should only attempt once (400 is not retryable)
|
||||
assert attempt_count == 1
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_retries_on_429_rate_limit(self, mock_env):
|
||||
"""Test that 429 rate limit errors trigger retries."""
|
||||
from browser_use.llm.exceptions import ModelProviderError
|
||||
from browser_use.llm.google.chat import ChatGoogle
|
||||
from browser_use.llm.messages import UserMessage
|
||||
|
||||
attempt_count = 0
|
||||
|
||||
with patch('browser_use.llm.google.chat.genai') as mock_genai:
|
||||
mock_client = MagicMock()
|
||||
mock_genai.Client.return_value = mock_client
|
||||
|
||||
async def mock_generate(*args, **kwargs):
|
||||
nonlocal attempt_count
|
||||
attempt_count += 1
|
||||
|
||||
if attempt_count < 2:
|
||||
raise ModelProviderError(message='Rate limit exceeded', status_code=429, model='gemini-2.0-flash')
|
||||
else:
|
||||
mock_response = MagicMock()
|
||||
mock_response.text = 'Success after rate limit!'
|
||||
mock_response.usage_metadata = MagicMock(
|
||||
prompt_token_count=10, candidates_token_count=5, total_token_count=15, cached_content_token_count=0
|
||||
)
|
||||
mock_response.candidates = [MagicMock(content=MagicMock(parts=[MagicMock(text='Success after rate limit!')]))]
|
||||
return mock_response
|
||||
|
||||
mock_client.aio.models.generate_content = mock_generate
|
||||
|
||||
client = ChatGoogle(model='gemini-2.0-flash', api_key='test', retry_base_delay=0.01)
|
||||
result = await client.ainvoke([UserMessage(content='test')])
|
||||
|
||||
assert attempt_count == 2
|
||||
assert result.completion == 'Success after rate limit!'
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
pytest.main([__file__, '-v'])
|
||||
@@ -0,0 +1,358 @@
|
||||
"""Tests for structure-aware markdown chunking."""
|
||||
|
||||
from markdownify import markdownify as md
|
||||
from pytest_httpserver import HTTPServer
|
||||
|
||||
from browser_use.dom.markdown_extractor import chunk_markdown_by_structure
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Unit tests — synchronous, no browser needed
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestChunkMarkdownBasic:
|
||||
"""Basic chunking behaviour."""
|
||||
|
||||
def test_short_content_single_chunk(self):
|
||||
content = '# Hello\n\nSome short content.'
|
||||
chunks = chunk_markdown_by_structure(content, max_chunk_chars=100_000)
|
||||
assert len(chunks) == 1
|
||||
assert chunks[0].content == content
|
||||
assert chunks[0].chunk_index == 0
|
||||
assert chunks[0].total_chunks == 1
|
||||
assert chunks[0].has_more is False
|
||||
|
||||
def test_empty_content(self):
|
||||
chunks = chunk_markdown_by_structure('', max_chunk_chars=100)
|
||||
assert len(chunks) == 1
|
||||
assert chunks[0].content == ''
|
||||
assert chunks[0].has_more is False
|
||||
|
||||
def test_chunk_offsets_cover_full_content(self):
|
||||
"""Chunk offsets should cover the entire original content without gaps."""
|
||||
content = '# Header\n\nParagraph one.\n\n# Header 2\n\nParagraph two.'
|
||||
chunks = chunk_markdown_by_structure(content, max_chunk_chars=20)
|
||||
# Verify no gaps between consecutive chunks
|
||||
for i in range(1, len(chunks)):
|
||||
assert chunks[i].char_offset_start == chunks[i - 1].char_offset_end, (
|
||||
f'Gap between chunk {i - 1} end ({chunks[i - 1].char_offset_end}) '
|
||||
f'and chunk {i} start ({chunks[i].char_offset_start})'
|
||||
)
|
||||
# First chunk starts at 0
|
||||
assert chunks[0].char_offset_start == 0
|
||||
# Last chunk ends at content length
|
||||
assert chunks[-1].char_offset_end == len(content)
|
||||
|
||||
|
||||
class TestChunkMarkdownHeaders:
|
||||
"""Header boundary splitting."""
|
||||
|
||||
def test_splits_at_header_boundary(self):
|
||||
"""Chunks should prefer splitting at header boundaries."""
|
||||
section_a = '# Section A\n\n' + 'x' * 50
|
||||
section_b = '\n\n# Section B\n\n' + 'y' * 50
|
||||
content = section_a + section_b
|
||||
# Set limit so section_a fits but section_a + section_b doesn't
|
||||
chunks = chunk_markdown_by_structure(content, max_chunk_chars=len(section_a) + 5)
|
||||
assert len(chunks) >= 2
|
||||
# First chunk should contain Section A header
|
||||
assert '# Section A' in chunks[0].content
|
||||
# Second chunk should start with or contain Section B header
|
||||
assert '# Section B' in chunks[1].content
|
||||
|
||||
|
||||
class TestChunkMarkdownHeaderPreferred:
|
||||
"""Header-preferred splitting ensures chunks start at semantic boundaries."""
|
||||
|
||||
def test_header_preferred_split_moves_header_to_next_chunk(self):
|
||||
"""When a header sits in the middle of an overflowing chunk, split before it."""
|
||||
# Build content: para_a (big) + header_b + para_b (small)
|
||||
para_a = 'A' * 600
|
||||
header_b = '# Section B'
|
||||
para_b = 'B' * 100
|
||||
content = f'{para_a}\n\n{header_b}\n\n{para_b}'
|
||||
# Limit forces a split; header is near end of first chunk
|
||||
chunks = chunk_markdown_by_structure(content, max_chunk_chars=700)
|
||||
assert len(chunks) >= 2
|
||||
# The header should be the START of the second chunk, not the end of the first
|
||||
assert chunks[1].content.lstrip().startswith('# Section B')
|
||||
# First chunk should NOT contain the header
|
||||
assert '# Section B' not in chunks[0].content
|
||||
|
||||
def test_header_preferred_split_doesnt_create_tiny_chunks(self):
|
||||
"""Don't split at a header that would make the prefix chunk < 50% of limit."""
|
||||
header_a = '# Section A'
|
||||
para_a = 'A' * 30 # very small before header
|
||||
header_b = '# Section B'
|
||||
para_b = 'B' * 600
|
||||
content = f'{header_a}\n\n{para_a}\n\n{header_b}\n\n{para_b}'
|
||||
# With a limit of 700, header_b is near the start — splitting there would
|
||||
# leave a tiny prefix chunk. The algo should NOT split there.
|
||||
chunks = chunk_markdown_by_structure(content, max_chunk_chars=700)
|
||||
# First chunk should contain both headers (no tiny split)
|
||||
assert '# Section A' in chunks[0].content
|
||||
assert '# Section B' in chunks[0].content
|
||||
|
||||
|
||||
class TestChunkMarkdownCodeFence:
|
||||
"""Code fence blocks never split."""
|
||||
|
||||
def test_code_fence_not_split(self):
|
||||
code_block = '```python\n' + 'x = 1\n' * 100 + '```'
|
||||
content = '# Title\n\n' + code_block + '\n\n# Footer\n\nDone.'
|
||||
# Limit smaller than the code block — it should still stay in one chunk (soft limit)
|
||||
chunks = chunk_markdown_by_structure(content, max_chunk_chars=50)
|
||||
# Find the chunk containing the code block
|
||||
code_chunks = [c for c in chunks if '```python' in c.content and '```' in c.content.split('```python')[1]]
|
||||
assert len(code_chunks) >= 1, 'Code fence should appear intact in at least one chunk'
|
||||
|
||||
def test_unclosed_code_fence(self):
|
||||
"""Unclosed code fence should still be kept as one block."""
|
||||
content = '# Title\n\n```python\nx = 1\ny = 2'
|
||||
chunks = chunk_markdown_by_structure(content, max_chunk_chars=100_000)
|
||||
assert len(chunks) == 1
|
||||
assert '```python' in chunks[0].content
|
||||
assert 'y = 2' in chunks[0].content
|
||||
|
||||
|
||||
class TestChunkMarkdownTable:
|
||||
"""Table rows never split mid-row."""
|
||||
|
||||
def test_table_not_split_mid_row(self):
|
||||
header = '| Name | Value |'
|
||||
separator = '| --- | --- |'
|
||||
rows = [f'| item{i} | val{i} |' for i in range(50)]
|
||||
table = '\n'.join([header, separator] + rows)
|
||||
content = '# Data\n\n' + table
|
||||
|
||||
# Use a limit that would fall in the middle of the table
|
||||
chunks = chunk_markdown_by_structure(content, max_chunk_chars=200)
|
||||
|
||||
for chunk in chunks:
|
||||
lines = chunk.content.split('\n')
|
||||
for line in lines:
|
||||
stripped = line.strip()
|
||||
if stripped.startswith('|') and stripped.endswith('|'):
|
||||
# Each table row line should be complete (start and end with |)
|
||||
assert stripped.count('|') >= 3, f'Incomplete table row: {stripped}'
|
||||
|
||||
def test_table_header_in_overlap_for_continuation(self):
|
||||
"""When a table spans multiple chunks, the header should be in the overlap prefix."""
|
||||
header = '| Col1 | Col2 |'
|
||||
separator = '| --- | --- |'
|
||||
rows = [f'| r{i} | d{i} |' for i in range(100)]
|
||||
table = '\n'.join([header, separator] + rows)
|
||||
content = table
|
||||
|
||||
# Force split within the table
|
||||
chunks = chunk_markdown_by_structure(content, max_chunk_chars=300)
|
||||
|
||||
if len(chunks) > 1:
|
||||
# Second chunk should have table header in overlap
|
||||
assert '| Col1 | Col2 |' in chunks[1].overlap_prefix
|
||||
assert '| --- | --- |' in chunks[1].overlap_prefix
|
||||
|
||||
def test_table_header_carried_across_three_plus_chunks(self):
|
||||
"""Table header must persist in overlap for ALL continuation chunks, not just the second."""
|
||||
header = '| Col1 | Col2 |'
|
||||
separator = '| --- | --- |'
|
||||
rows = [f'| row{i} | data{i} |' for i in range(200)]
|
||||
table = '\n'.join([header, separator] + rows)
|
||||
content = table
|
||||
|
||||
# Force many small chunks
|
||||
chunks = chunk_markdown_by_structure(content, max_chunk_chars=200)
|
||||
assert len(chunks) >= 3, f'Expected >=3 chunks, got {len(chunks)}'
|
||||
|
||||
# Every chunk after the first should carry the table header in its overlap
|
||||
for i in range(1, len(chunks)):
|
||||
assert '| Col1 | Col2 |' in chunks[i].overlap_prefix, f'Chunk {i} missing table header in overlap'
|
||||
assert '| --- | --- |' in chunks[i].overlap_prefix, f'Chunk {i} missing table separator in overlap'
|
||||
|
||||
|
||||
class TestChunkMarkdownListItems:
|
||||
"""List item continuations stay together."""
|
||||
|
||||
def test_list_items_not_split(self):
|
||||
items = '\n'.join([f'- Item {i} with some description text' for i in range(50)])
|
||||
content = '# List\n\n' + items
|
||||
|
||||
chunks = chunk_markdown_by_structure(content, max_chunk_chars=200)
|
||||
for chunk in chunks:
|
||||
lines = chunk.content.split('\n')
|
||||
for line in lines:
|
||||
stripped = line.strip()
|
||||
if stripped.startswith('- '):
|
||||
# Each list item should be a complete item
|
||||
assert 'Item' in stripped
|
||||
|
||||
|
||||
class TestChunkMarkdownStartFromChar:
|
||||
"""start_from_char parameter returns correct chunk."""
|
||||
|
||||
def test_start_from_char_returns_correct_chunk(self):
|
||||
section_a = '# A\n\nContent A here.'
|
||||
section_b = '\n\n# B\n\nContent B here.'
|
||||
content = section_a + section_b
|
||||
# Chunk at header boundaries
|
||||
all_chunks = chunk_markdown_by_structure(content, max_chunk_chars=len(section_a) + 5)
|
||||
|
||||
if len(all_chunks) > 1:
|
||||
# Request from char offset within second chunk
|
||||
mid = all_chunks[1].char_offset_start + 1
|
||||
filtered = chunk_markdown_by_structure(content, max_chunk_chars=len(section_a) + 5, start_from_char=mid)
|
||||
assert len(filtered) >= 1
|
||||
assert filtered[0].chunk_index == all_chunks[1].chunk_index
|
||||
|
||||
def test_start_from_char_past_end_returns_empty(self):
|
||||
content = '# Hello\n\nWorld.'
|
||||
chunks = chunk_markdown_by_structure(content, max_chunk_chars=100_000, start_from_char=99999)
|
||||
assert chunks == []
|
||||
|
||||
def test_start_from_char_zero_returns_all(self):
|
||||
content = '# Hello\n\nWorld.'
|
||||
chunks = chunk_markdown_by_structure(content, max_chunk_chars=100_000, start_from_char=0)
|
||||
assert len(chunks) == 1
|
||||
|
||||
|
||||
class TestChunkMarkdownOverlap:
|
||||
"""Overlap lines carry context."""
|
||||
|
||||
def test_overlap_lines_carry_context(self):
|
||||
lines_content = '\n'.join([f'Line {i}' for i in range(100)])
|
||||
content = lines_content
|
||||
|
||||
chunks = chunk_markdown_by_structure(content, max_chunk_chars=200, overlap_lines=3)
|
||||
if len(chunks) > 1:
|
||||
# Second chunk should have overlap from first chunk
|
||||
assert chunks[1].overlap_prefix != ''
|
||||
# Overlap should contain lines from the end of the previous chunk
|
||||
overlap_lines = chunks[1].overlap_prefix.split('\n')
|
||||
assert len(overlap_lines) <= 3 + 2 # some flexibility for table headers etc.
|
||||
|
||||
def test_no_overlap_on_first_chunk(self):
|
||||
content = '# A\n\nSome content.\n\n# B\n\nMore content.'
|
||||
chunks = chunk_markdown_by_structure(content, max_chunk_chars=25)
|
||||
assert chunks[0].overlap_prefix == ''
|
||||
|
||||
|
||||
class TestChunkMarkdownMixed:
|
||||
"""Mixed content scenarios."""
|
||||
|
||||
def test_paragraph_splitting(self):
|
||||
"""Paragraphs separated by blank lines are separate blocks."""
|
||||
p1 = 'First paragraph with text.'
|
||||
p2 = 'Second paragraph with more text.'
|
||||
content = f'{p1}\n\n{p2}'
|
||||
chunks = chunk_markdown_by_structure(content, max_chunk_chars=30)
|
||||
# Should produce multiple chunks
|
||||
assert len(chunks) >= 2
|
||||
|
||||
def test_single_oversized_block_allowed(self):
|
||||
"""A single block bigger than max_chunk_chars is allowed (soft limit)."""
|
||||
big_para = 'x' * 200
|
||||
content = big_para
|
||||
chunks = chunk_markdown_by_structure(content, max_chunk_chars=50)
|
||||
assert len(chunks) == 1
|
||||
assert chunks[0].content == big_para
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# HTML → markdown → chunk pipeline tests
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestHTMLToMarkdownChunking:
|
||||
"""End-to-end: HTML table → markdown → chunks."""
|
||||
|
||||
def test_large_table_produces_valid_chunks(self):
|
||||
"""200-row HTML table → markdown → chunks should produce valid table rows in every chunk."""
|
||||
rows = ''.join(f'<tr><td>Row {i}</td><td>Val {i}</td></tr>' for i in range(200))
|
||||
html = f'<table><thead><tr><th>Name</th><th>Value</th></tr></thead><tbody>{rows}</tbody></table>'
|
||||
markdown = md(html, heading_style='ATX')
|
||||
|
||||
chunks = chunk_markdown_by_structure(markdown, max_chunk_chars=500)
|
||||
assert len(chunks) > 1, 'Should produce multiple chunks for 200 rows'
|
||||
|
||||
for chunk in chunks:
|
||||
lines = chunk.content.strip().split('\n')
|
||||
for line in lines:
|
||||
s = line.strip()
|
||||
if s.startswith('|') and s.endswith('|'):
|
||||
# Every table line should have consistent column count
|
||||
assert s.count('|') >= 3
|
||||
|
||||
def test_table_without_thead_normalization(self):
|
||||
"""Table with <th> in first <tr> but no <thead> should still produce proper markdown."""
|
||||
html = '<table><tr><th>A</th><th>B</th></tr><tr><td>1</td><td>2</td></tr><tr><td>3</td><td>4</td></tr></table>'
|
||||
markdown = md(html, heading_style='ATX')
|
||||
# Verify markdownify produced a proper table (with separator row)
|
||||
assert '---' in markdown or '| A |' in markdown
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Integration tests — require browser + httpserver
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestTableNormalizationIntegration:
|
||||
"""Integration tests using browser session and httpserver."""
|
||||
|
||||
async def test_table_without_thead_normalized_via_serializer(self, browser_session, httpserver: HTTPServer):
|
||||
"""Tables without <thead> should get normalized by HTMLSerializer during extraction."""
|
||||
html = """
|
||||
<html><body>
|
||||
<table>
|
||||
<tr><th>Header1</th><th>Header2</th></tr>
|
||||
<tr><td>data1</td><td>data2</td></tr>
|
||||
<tr><td>data3</td><td>data4</td></tr>
|
||||
</table>
|
||||
</body></html>
|
||||
"""
|
||||
httpserver.expect_request('/table-test').respond_with_data(html, content_type='text/html')
|
||||
url = httpserver.url_for('/table-test')
|
||||
|
||||
await browser_session.navigate_to(url)
|
||||
|
||||
from browser_use.dom.markdown_extractor import extract_clean_markdown
|
||||
|
||||
content, _ = await extract_clean_markdown(browser_session=browser_session)
|
||||
|
||||
# Should have proper markdown table with separator
|
||||
assert '|' in content
|
||||
# The header should be present
|
||||
assert 'Header1' in content
|
||||
assert 'Header2' in content
|
||||
|
||||
async def test_large_table_extraction_preserves_structure(self, browser_session, httpserver: HTTPServer):
|
||||
"""Large table extraction should produce structure-aware chunks."""
|
||||
rows = ''.join(f'<tr><td>Name{i}</td><td>Value{i}</td></tr>' for i in range(300))
|
||||
html = f"""
|
||||
<html><body>
|
||||
<table>
|
||||
<tr><th>Name</th><th>Value</th></tr>
|
||||
{rows}
|
||||
</table>
|
||||
</body></html>
|
||||
"""
|
||||
httpserver.expect_request('/big-table').respond_with_data(html, content_type='text/html')
|
||||
url = httpserver.url_for('/big-table')
|
||||
|
||||
await browser_session.navigate_to(url)
|
||||
|
||||
from browser_use.dom.markdown_extractor import extract_clean_markdown
|
||||
|
||||
content, _ = await extract_clean_markdown(browser_session=browser_session)
|
||||
|
||||
# Chunk with a small limit to force multiple chunks
|
||||
chunks = chunk_markdown_by_structure(content, max_chunk_chars=2000)
|
||||
|
||||
# Should produce multiple chunks
|
||||
assert len(chunks) > 1
|
||||
|
||||
# Each chunk should have complete table rows
|
||||
for chunk in chunks:
|
||||
for line in chunk.content.split('\n'):
|
||||
s = line.strip()
|
||||
if s.startswith('|') and s.endswith('|'):
|
||||
assert s.count('|') >= 3, f'Incomplete table row: {s}'
|
||||
@@ -0,0 +1,146 @@
|
||||
"""Tests for markdown extractor preprocessing."""
|
||||
|
||||
from browser_use.dom.markdown_extractor import _preprocess_markdown_content
|
||||
|
||||
|
||||
class TestPreprocessMarkdownContent:
|
||||
"""Tests for _preprocess_markdown_content function."""
|
||||
|
||||
def test_preserves_short_lines(self):
|
||||
"""Short lines (1-2 chars) should be preserved, not removed."""
|
||||
content = '# Items\na\nb\nc\nOK\nNo'
|
||||
filtered, _ = _preprocess_markdown_content(content)
|
||||
|
||||
assert 'a' in filtered.split('\n')
|
||||
assert 'b' in filtered.split('\n')
|
||||
assert 'c' in filtered.split('\n')
|
||||
assert 'OK' in filtered.split('\n')
|
||||
assert 'No' in filtered.split('\n')
|
||||
|
||||
def test_preserves_single_digit_numbers(self):
|
||||
"""Single digit page numbers should be preserved."""
|
||||
content = 'Page navigation:\n1\n2\n3\n10'
|
||||
filtered, _ = _preprocess_markdown_content(content)
|
||||
|
||||
lines = filtered.split('\n')
|
||||
assert '1' in lines
|
||||
assert '2' in lines
|
||||
assert '3' in lines
|
||||
assert '10' in lines
|
||||
|
||||
def test_preserves_markdown_list_items(self):
|
||||
"""Markdown list items with short content should be preserved."""
|
||||
content = 'Shopping list:\n- a\n- b\n- OK\n- No'
|
||||
filtered, _ = _preprocess_markdown_content(content)
|
||||
|
||||
assert '- a' in filtered
|
||||
assert '- b' in filtered
|
||||
assert '- OK' in filtered
|
||||
assert '- No' in filtered
|
||||
|
||||
def test_preserves_state_codes(self):
|
||||
"""Two-letter state codes should be preserved."""
|
||||
content = 'States:\nCA\nNY\nTX'
|
||||
filtered, _ = _preprocess_markdown_content(content)
|
||||
|
||||
lines = filtered.split('\n')
|
||||
assert 'CA' in lines
|
||||
assert 'NY' in lines
|
||||
assert 'TX' in lines
|
||||
|
||||
def test_removes_empty_lines(self):
|
||||
"""Empty and whitespace-only lines should be removed."""
|
||||
content = 'Header\n\n \n\nContent'
|
||||
filtered, _ = _preprocess_markdown_content(content)
|
||||
|
||||
# Should not have empty lines
|
||||
for line in filtered.split('\n'):
|
||||
assert line.strip(), f'Found empty line in output: {repr(line)}'
|
||||
|
||||
def test_removes_large_json_blobs(self):
|
||||
"""Large JSON-like lines (>100 chars) should be removed."""
|
||||
# Create a JSON blob > 100 chars
|
||||
json_blob = '{"key": "' + 'x' * 100 + '"}'
|
||||
content = f'Header\n{json_blob}\nFooter'
|
||||
filtered, _ = _preprocess_markdown_content(content)
|
||||
|
||||
assert json_blob not in filtered
|
||||
assert 'Header' in filtered
|
||||
assert 'Footer' in filtered
|
||||
|
||||
def test_preserves_small_json(self):
|
||||
"""Small JSON objects (<100 chars) should be preserved."""
|
||||
small_json = '{"key": "value"}'
|
||||
content = f'Header\n{small_json}\nFooter'
|
||||
filtered, _ = _preprocess_markdown_content(content)
|
||||
|
||||
assert small_json in filtered
|
||||
|
||||
def test_compresses_multiple_newlines(self):
|
||||
"""4+ consecutive newlines should be compressed to max_newlines."""
|
||||
content = 'Header\n\n\n\n\nFooter'
|
||||
filtered, _ = _preprocess_markdown_content(content, max_newlines=2)
|
||||
|
||||
# After filtering empty lines, we should have just Header and Footer
|
||||
lines = [line for line in filtered.split('\n') if line.strip()]
|
||||
assert lines == ['Header', 'Footer']
|
||||
|
||||
def test_returns_chars_filtered_count(self):
|
||||
"""Should return count of characters removed."""
|
||||
content = 'Header\n\n\n\n\nFooter'
|
||||
_, chars_filtered = _preprocess_markdown_content(content)
|
||||
|
||||
assert chars_filtered > 0
|
||||
|
||||
def test_strips_result(self):
|
||||
"""Result should be stripped of leading/trailing whitespace."""
|
||||
content = ' \n\nContent\n\n '
|
||||
filtered, _ = _preprocess_markdown_content(content)
|
||||
|
||||
assert not filtered.startswith(' ')
|
||||
assert not filtered.startswith('\n')
|
||||
assert not filtered.endswith(' ')
|
||||
assert not filtered.endswith('\n')
|
||||
|
||||
|
||||
class TestPreservesLinksAndEncoding:
|
||||
"""Regression tests: markdown links and percent-encoded URLs must survive filtering."""
|
||||
|
||||
def test_preserves_long_markdown_link_lines(self):
|
||||
"""A long markdown link line (>100 chars) must not be dropped by the JSON heuristic."""
|
||||
link = '[Read the full quarterly earnings report for fiscal year 2025](https://example.com/investor-relations/reports/q4-2025-earnings-full.pdf)'
|
||||
assert len(link) > 100
|
||||
content = f'Header\n{link}\nFooter'
|
||||
filtered, _ = _preprocess_markdown_content(content)
|
||||
|
||||
assert link in filtered
|
||||
|
||||
def test_preserves_long_image_link_lines(self):
|
||||
"""A long clickable-image line (starts with [](https://example.com/products/deluxe-widget)'
|
||||
assert len(line) > 100
|
||||
content = f'Intro\n{line}\nOutro'
|
||||
filtered, _ = _preprocess_markdown_content(content)
|
||||
|
||||
assert line in filtered
|
||||
|
||||
def test_removes_long_json_array_lines(self):
|
||||
"""A valid JSON array blob >100 chars should still be dropped."""
|
||||
json_array = '[' + ', '.join(f'{{"id": {i}, "name": "item-{i}"}}' for i in range(10)) + ']'
|
||||
assert len(json_array) > 100
|
||||
content = f'Header\n{json_array}\nFooter'
|
||||
filtered, _ = _preprocess_markdown_content(content)
|
||||
|
||||
assert json_array not in filtered
|
||||
assert 'Header' in filtered
|
||||
|
||||
def test_preserves_percent_encoded_urls(self):
|
||||
"""Percent-encodings in URLs must survive HTML -> markdown conversion."""
|
||||
from browser_use.dom.markdown_extractor import convert_html_to_markdown
|
||||
|
||||
html = '<p>See <a href="https://example.com/my%20file%2Fv2?q=a%26b">the doc</a> here</p>'
|
||||
content, _, _ = convert_html_to_markdown(html)
|
||||
|
||||
assert '%20' in content
|
||||
assert '%2F' in content
|
||||
assert '%26' in content
|
||||
@@ -0,0 +1,290 @@
|
||||
"""
|
||||
Tests for multi_act() page-change guards.
|
||||
|
||||
Verifies:
|
||||
1. Metadata: terminates_sequence flags are set correctly on built-in actions
|
||||
2. Static guard: actions tagged terminates_sequence abort remaining queued actions
|
||||
3. Runtime guard: URL/focus changes detected after click-on-link abort remaining actions
|
||||
4. Safe chain: multiple inputs execute without interruption
|
||||
|
||||
Usage:
|
||||
uv run pytest tests/ci/test_multi_act_guards.py -v -s
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
import pytest
|
||||
from pytest_httpserver import HTTPServer
|
||||
|
||||
from browser_use.agent.service import Agent
|
||||
from browser_use.browser import BrowserSession
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
from browser_use.tools.service import Tools
|
||||
from tests.ci.conftest import create_mock_llm
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Fixtures
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def http_server():
|
||||
"""Test HTTP server with pages for guard tests."""
|
||||
server = HTTPServer()
|
||||
server.start()
|
||||
|
||||
server.expect_request('/form').respond_with_data(
|
||||
"""<html><head><title>Form Page</title></head><body>
|
||||
<h1>Form</h1>
|
||||
<input id="field1" type="text" placeholder="Field 1" />
|
||||
<input id="field2" type="text" placeholder="Field 2" />
|
||||
<input id="field3" type="text" placeholder="Field 3" />
|
||||
<button id="submit" type="submit">Submit</button>
|
||||
</body></html>""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
server.expect_request('/page_a').respond_with_data(
|
||||
"""<html><head><title>Page A</title></head><body>
|
||||
<h1>Page A</h1>
|
||||
<a id="link_b" href="/page_b">Go to Page B</a>
|
||||
</body></html>""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
server.expect_request('/page_b').respond_with_data(
|
||||
"""<html><head><title>Page B</title></head><body>
|
||||
<h1>Page B</h1>
|
||||
<p>You arrived at Page B</p>
|
||||
</body></html>""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
server.expect_request('/static').respond_with_data(
|
||||
"""<html><head><title>Static Page</title></head><body>
|
||||
<h1>Static</h1>
|
||||
<p>Nothing changes here</p>
|
||||
<input id="safe_input" type="text" />
|
||||
</body></html>""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
yield server
|
||||
server.stop()
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def base_url(http_server):
|
||||
return f'http://{http_server.host}:{http_server.port}'
|
||||
|
||||
|
||||
@pytest.fixture(scope='module')
|
||||
async def browser_session():
|
||||
session = BrowserSession(
|
||||
browser_profile=BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
keep_alive=True,
|
||||
)
|
||||
)
|
||||
await session.start()
|
||||
yield session
|
||||
await session.kill()
|
||||
await session.event_bus.stop(clear=True, timeout=5)
|
||||
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
def tools():
|
||||
return Tools()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. Metadata tests — verify terminates_sequence flags
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestTerminatesSequenceMetadata:
|
||||
"""Verify that built-in actions have correct terminates_sequence flags."""
|
||||
|
||||
def test_navigate_terminates(self, tools):
|
||||
action = tools.registry.registry.actions.get('navigate')
|
||||
assert action is not None
|
||||
assert action.terminates_sequence is True
|
||||
|
||||
def test_search_terminates(self, tools):
|
||||
action = tools.registry.registry.actions.get('search')
|
||||
assert action is not None
|
||||
assert action.terminates_sequence is True
|
||||
|
||||
def test_go_back_terminates(self, tools):
|
||||
action = tools.registry.registry.actions.get('go_back')
|
||||
assert action is not None
|
||||
assert action.terminates_sequence is True
|
||||
|
||||
def test_switch_terminates(self, tools):
|
||||
action = tools.registry.registry.actions.get('switch')
|
||||
assert action is not None
|
||||
assert action.terminates_sequence is True
|
||||
|
||||
def test_click_does_not_terminate(self, tools):
|
||||
action = tools.registry.registry.actions.get('click')
|
||||
assert action is not None
|
||||
assert action.terminates_sequence is False
|
||||
|
||||
def test_input_does_not_terminate(self, tools):
|
||||
action = tools.registry.registry.actions.get('input')
|
||||
assert action is not None
|
||||
assert action.terminates_sequence is False
|
||||
|
||||
def test_scroll_does_not_terminate(self, tools):
|
||||
action = tools.registry.registry.actions.get('scroll')
|
||||
assert action is not None
|
||||
assert action.terminates_sequence is False
|
||||
|
||||
def test_extract_does_not_terminate(self, tools):
|
||||
action = tools.registry.registry.actions.get('extract')
|
||||
assert action is not None
|
||||
assert action.terminates_sequence is False
|
||||
|
||||
def test_evaluate_terminates(self, tools):
|
||||
"""evaluate() can mutate the DOM in unpredictable ways (e.g. dismiss cookie overlays),
|
||||
so any actions queued after it should be skipped to avoid stale element references."""
|
||||
action = tools.registry.registry.actions.get('evaluate')
|
||||
assert action is not None
|
||||
assert action.terminates_sequence is True
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. Static guard — navigate as non-last action skips remaining
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestStaticGuard:
|
||||
"""Verify that terminates_sequence actions abort the remaining queue."""
|
||||
|
||||
async def test_navigate_aborts_remaining_actions(self, browser_session, base_url, tools):
|
||||
"""When navigate is action 2/3, action 3 should never execute."""
|
||||
# Start on a known page
|
||||
await tools.navigate(url=f'{base_url}/static', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
# Build action models: [scroll_down, navigate_to_page_a, scroll_down]
|
||||
ActionModel = tools.registry.create_action_model()
|
||||
actions = [
|
||||
ActionModel.model_validate({'scroll': {'down': True, 'pages': 1}}),
|
||||
ActionModel.model_validate({'navigate': {'url': f'{base_url}/page_a'}}),
|
||||
ActionModel.model_validate({'scroll': {'down': True, 'pages': 1}}),
|
||||
]
|
||||
|
||||
mock_llm = create_mock_llm()
|
||||
agent = Agent(task='test', llm=mock_llm, browser_session=browser_session, tools=tools)
|
||||
|
||||
results = await agent.multi_act(actions)
|
||||
|
||||
# Should have executed exactly 2 actions (scroll + navigate), third skipped
|
||||
assert len(results) == 2, f'Expected 2 results but got {len(results)}: {results}'
|
||||
|
||||
# Verify we actually navigated
|
||||
url = await browser_session.get_current_page_url()
|
||||
assert '/page_a' in url
|
||||
|
||||
async def test_go_back_aborts_remaining_actions(self, browser_session, base_url, tools):
|
||||
"""go_back should abort remaining queued actions."""
|
||||
# Navigate to page_a then page_b so go_back has somewhere to go
|
||||
await tools.navigate(url=f'{base_url}/page_a', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.3)
|
||||
await tools.navigate(url=f'{base_url}/page_b', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.3)
|
||||
|
||||
ActionModel = tools.registry.create_action_model()
|
||||
actions = [
|
||||
ActionModel.model_validate({'go_back': {}}),
|
||||
ActionModel.model_validate({'scroll': {'down': True, 'pages': 1}}),
|
||||
]
|
||||
|
||||
mock_llm = create_mock_llm()
|
||||
agent = Agent(task='test', llm=mock_llm, browser_session=browser_session, tools=tools)
|
||||
|
||||
results = await agent.multi_act(actions)
|
||||
|
||||
# go_back should terminate the sequence — only 1 result
|
||||
assert len(results) == 1, f'Expected 1 result but got {len(results)}: {results}'
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. Runtime guard — click on link changes URL, remaining actions skipped
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestRuntimeGuard:
|
||||
"""Verify that URL/focus changes detected at runtime abort remaining actions."""
|
||||
|
||||
async def test_click_link_aborts_remaining(self, browser_session, base_url, tools):
|
||||
"""Click a link that navigates to another page — remaining actions skipped."""
|
||||
await tools.navigate(url=f'{base_url}/page_a', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
# Get the selector map to find the link index
|
||||
state = await browser_session.get_browser_state_summary()
|
||||
assert state.dom_state is not None
|
||||
selector_map = state.dom_state.selector_map
|
||||
|
||||
# Find the link element (a#link_b)
|
||||
link_index = None
|
||||
for idx, element in selector_map.items():
|
||||
if hasattr(element, 'tag_name') and element.tag_name == 'a':
|
||||
link_index = idx
|
||||
break
|
||||
|
||||
assert link_index is not None, 'Could not find link element in selector map'
|
||||
|
||||
ActionModel = tools.registry.create_action_model()
|
||||
actions = [
|
||||
ActionModel.model_validate({'click': {'index': link_index}}),
|
||||
ActionModel.model_validate({'scroll': {'down': True, 'pages': 1}}),
|
||||
ActionModel.model_validate({'scroll': {'down': True, 'pages': 1}}),
|
||||
]
|
||||
|
||||
mock_llm = create_mock_llm()
|
||||
agent = Agent(task='test', llm=mock_llm, browser_session=browser_session, tools=tools)
|
||||
|
||||
results = await agent.multi_act(actions)
|
||||
|
||||
# Click navigated to page_b — runtime guard should stop at 1
|
||||
assert len(results) == 1, f'Expected 1 result but got {len(results)}: {results}'
|
||||
|
||||
# Verify we're on page_b
|
||||
url = await browser_session.get_current_page_url()
|
||||
assert '/page_b' in url
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. Safe chain — multiple non-page-changing actions all execute
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestSafeChain:
|
||||
"""Verify that non-page-changing actions execute without interruption."""
|
||||
|
||||
async def test_multiple_scrolls_all_execute(self, browser_session, base_url, tools):
|
||||
"""Multiple scroll actions should all execute."""
|
||||
await tools.navigate(url=f'{base_url}/static', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
ActionModel = tools.registry.create_action_model()
|
||||
actions = [
|
||||
ActionModel.model_validate({'scroll': {'down': True, 'pages': 0.5}}),
|
||||
ActionModel.model_validate({'scroll': {'down': True, 'pages': 0.5}}),
|
||||
ActionModel.model_validate({'scroll': {'down': False, 'pages': 0.5}}),
|
||||
]
|
||||
|
||||
mock_llm = create_mock_llm()
|
||||
agent = Agent(task='test', llm=mock_llm, browser_session=browser_session, tools=tools)
|
||||
|
||||
results = await agent.multi_act(actions)
|
||||
|
||||
# All 3 scrolls should execute
|
||||
assert len(results) == 3, f'Expected 3 results but got {len(results)}: {results}'
|
||||
# None should have errors
|
||||
for r in results:
|
||||
assert r.error is None, f'Unexpected error: {r.error}'
|
||||
@@ -0,0 +1,153 @@
|
||||
import pytest
|
||||
|
||||
from browser_use.llm.openrouter.chat import ChatOpenRouter
|
||||
from browser_use.llm.views import ChatInvokeUsage
|
||||
from browser_use.tokens import openrouter_pricing
|
||||
from browser_use.tokens.openrouter_pricing import model_pricing_from_openrouter_metadata
|
||||
from browser_use.tokens.service import TokenCost
|
||||
from browser_use.tokens.views import ModelPricing
|
||||
|
||||
|
||||
def _openrouter_metadata() -> dict:
|
||||
return {
|
||||
'id': 'deepseek/deepseek-v4-flash',
|
||||
'context_length': 1_048_576,
|
||||
'top_provider': {'max_completion_tokens': 16_384},
|
||||
'pricing': {
|
||||
'prompt': '0.0000001',
|
||||
'completion': '0.0000002',
|
||||
'input_cache_read': '0.00000002',
|
||||
'input_cache_write': '0.00000003',
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def test_model_pricing_from_openrouter_metadata() -> None:
|
||||
pricing = model_pricing_from_openrouter_metadata('deepseek/deepseek-v4-flash', _openrouter_metadata())
|
||||
|
||||
assert pricing is not None
|
||||
assert pricing.model == 'deepseek/deepseek-v4-flash'
|
||||
assert pricing.input_cost_per_token == pytest.approx(0.10 / 1_000_000)
|
||||
assert pricing.output_cost_per_token == pytest.approx(0.20 / 1_000_000)
|
||||
assert pricing.cache_read_input_token_cost == pytest.approx(0.02 / 1_000_000)
|
||||
assert pricing.cache_creation_input_token_cost == pytest.approx(0.03 / 1_000_000)
|
||||
assert pricing.max_tokens == 1_048_576
|
||||
assert pricing.max_input_tokens == 1_048_576
|
||||
assert pricing.max_output_tokens == 16_384
|
||||
|
||||
|
||||
async def test_openrouter_pricing_accepts_litellm_prefixed_model_ids(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
async def fake_get_openrouter_models_metadata(refresh: bool = False) -> dict[str, dict]:
|
||||
return {'deepseek/deepseek-v4-flash': _openrouter_metadata()}
|
||||
|
||||
monkeypatch.setattr(openrouter_pricing, 'get_openrouter_models_metadata', fake_get_openrouter_models_metadata)
|
||||
|
||||
pricing = await openrouter_pricing.get_openrouter_model_pricing('openrouter/deepseek/deepseek-v4-flash')
|
||||
|
||||
assert pricing is not None
|
||||
assert pricing.model == 'openrouter/deepseek/deepseek-v4-flash'
|
||||
assert pricing.input_cost_per_token == pytest.approx(0.10 / 1_000_000)
|
||||
|
||||
|
||||
async def test_token_cost_falls_back_to_openrouter_pricing(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
async def fake_openrouter_pricing(model_name: str) -> ModelPricing:
|
||||
assert model_name == 'deepseek/deepseek-v4-flash'
|
||||
return ModelPricing(
|
||||
model=model_name,
|
||||
input_cost_per_token=0.10 / 1_000_000,
|
||||
output_cost_per_token=0.20 / 1_000_000,
|
||||
cache_read_input_token_cost=0.02 / 1_000_000,
|
||||
cache_creation_input_token_cost=None,
|
||||
max_tokens=1_048_576,
|
||||
max_input_tokens=1_048_576,
|
||||
max_output_tokens=16_384,
|
||||
)
|
||||
|
||||
monkeypatch.setattr('browser_use.tokens.service.get_openrouter_model_pricing', fake_openrouter_pricing)
|
||||
|
||||
token_cost = TokenCost(include_cost=True)
|
||||
token_cost._initialized = True
|
||||
token_cost._pricing_data = {}
|
||||
|
||||
pricing = await token_cost.get_model_pricing('deepseek/deepseek-v4-flash')
|
||||
|
||||
assert pricing is not None
|
||||
assert pricing.input_cost_per_token == pytest.approx(0.10 / 1_000_000)
|
||||
assert pricing.output_cost_per_token == pytest.approx(0.20 / 1_000_000)
|
||||
|
||||
|
||||
async def test_calculate_cost_uses_openrouter_cache_pricing(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
async def fake_openrouter_pricing(model_name: str) -> ModelPricing:
|
||||
return ModelPricing(
|
||||
model=model_name,
|
||||
input_cost_per_token=0.10 / 1_000_000,
|
||||
output_cost_per_token=0.20 / 1_000_000,
|
||||
cache_read_input_token_cost=0.02 / 1_000_000,
|
||||
cache_creation_input_token_cost=None,
|
||||
max_tokens=None,
|
||||
max_input_tokens=None,
|
||||
max_output_tokens=None,
|
||||
)
|
||||
|
||||
monkeypatch.setattr('browser_use.tokens.service.get_openrouter_model_pricing', fake_openrouter_pricing)
|
||||
|
||||
token_cost = TokenCost(include_cost=True)
|
||||
token_cost._initialized = True
|
||||
token_cost._pricing_data = {}
|
||||
|
||||
cost = await token_cost.calculate_cost(
|
||||
'deepseek/deepseek-v4-flash',
|
||||
ChatInvokeUsage(
|
||||
prompt_tokens=110,
|
||||
prompt_cached_tokens=10,
|
||||
prompt_cache_creation_tokens=None,
|
||||
prompt_image_tokens=None,
|
||||
completion_tokens=20,
|
||||
total_tokens=130,
|
||||
),
|
||||
)
|
||||
|
||||
assert cost is not None
|
||||
assert cost.new_prompt_cost == pytest.approx(100 * 0.10 / 1_000_000)
|
||||
assert cost.prompt_read_cached_cost == pytest.approx(10 * 0.02 / 1_000_000)
|
||||
assert cost.completion_cost == pytest.approx(20 * 0.20 / 1_000_000)
|
||||
|
||||
|
||||
async def test_registered_openrouter_llm_forces_openrouter_pricing(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
seen_model_names = []
|
||||
|
||||
async def fake_openrouter_pricing(model_name: str) -> ModelPricing:
|
||||
seen_model_names.append(model_name)
|
||||
return ModelPricing(
|
||||
model=model_name,
|
||||
input_cost_per_token=0.10 / 1_000_000,
|
||||
output_cost_per_token=0.20 / 1_000_000,
|
||||
cache_read_input_token_cost=None,
|
||||
cache_creation_input_token_cost=None,
|
||||
max_tokens=None,
|
||||
max_input_tokens=None,
|
||||
max_output_tokens=None,
|
||||
)
|
||||
|
||||
monkeypatch.setattr('browser_use.tokens.service.get_openrouter_model_pricing', fake_openrouter_pricing)
|
||||
|
||||
token_cost = TokenCost(include_cost=True)
|
||||
token_cost._initialized = True
|
||||
token_cost._pricing_data = {'openai/gpt-4o-mini': {'input_cost_per_token': 99, 'output_cost_per_token': 99}}
|
||||
token_cost.register_llm(ChatOpenRouter(model='openai/gpt-4o-mini', api_key='test-key'))
|
||||
|
||||
cost = await token_cost.calculate_cost(
|
||||
'openai/gpt-4o-mini',
|
||||
ChatInvokeUsage(
|
||||
prompt_tokens=10,
|
||||
prompt_cached_tokens=None,
|
||||
prompt_cache_creation_tokens=None,
|
||||
prompt_image_tokens=None,
|
||||
completion_tokens=5,
|
||||
total_tokens=15,
|
||||
),
|
||||
)
|
||||
|
||||
assert cost is not None
|
||||
assert seen_model_names == ['openrouter/openai/gpt-4o-mini']
|
||||
assert cost.total_cost == pytest.approx(10 * 0.10 / 1_000_000 + 5 * 0.20 / 1_000_000)
|
||||
@@ -0,0 +1,106 @@
|
||||
"""Lock per-step varying metadata (step counter, wall-clock date) at the tail of the user message.
|
||||
|
||||
The agent's user message looks roughly:
|
||||
<user_request>...</user_request>
|
||||
<agent_history>...</agent_history>
|
||||
<agent_state>...</agent_state>
|
||||
<browser_state>...</browser_state>
|
||||
[<read_state>...</read_state>]
|
||||
[<page_specific_actions>...</page_specific_actions>]
|
||||
[unavailable skills info]
|
||||
<step_info>...</step_info> <-- new home
|
||||
|
||||
`<step_info>` carries a step counter and `datetime.now()`, both of which change between
|
||||
calls. Keeping it at the very end means everything above it can in principle be cached;
|
||||
moving it back into `<agent_state>` would silently shrink the cacheable prefix.
|
||||
"""
|
||||
|
||||
from browser_use.agent.prompts import AgentMessagePrompt
|
||||
from browser_use.agent.views import AgentStepInfo
|
||||
from browser_use.browser.views import BrowserStateSummary, PageInfo, TabInfo
|
||||
from browser_use.dom.views import SerializedDOMState
|
||||
from browser_use.filesystem.file_system import FileSystem
|
||||
|
||||
|
||||
def _make_prompt(tmp_path, step_number: int = 3) -> AgentMessagePrompt:
|
||||
dom_state = SerializedDOMState(_root=None, selector_map={})
|
||||
bs = BrowserStateSummary(
|
||||
url='https://example.test/foo',
|
||||
title='Test',
|
||||
tabs=[TabInfo(target_id='abcd1234', url='https://example.test/foo', title='Test')],
|
||||
page_info=PageInfo(
|
||||
viewport_width=1280,
|
||||
viewport_height=720,
|
||||
page_width=1280,
|
||||
page_height=1440,
|
||||
scroll_x=0,
|
||||
scroll_y=0,
|
||||
pixels_above=0,
|
||||
pixels_below=720,
|
||||
pixels_left=0,
|
||||
pixels_right=0,
|
||||
),
|
||||
dom_state=dom_state,
|
||||
is_pdf_viewer=False,
|
||||
recent_events=None,
|
||||
closed_popup_messages=[],
|
||||
screenshot=None,
|
||||
)
|
||||
fs = FileSystem(base_dir=str(tmp_path), create_default_files=False)
|
||||
return AgentMessagePrompt(
|
||||
browser_state_summary=bs,
|
||||
file_system=fs,
|
||||
agent_history_description='<step>existing history</step>',
|
||||
task='Do the test thing',
|
||||
step_info=AgentStepInfo(step_number=step_number, max_steps=50),
|
||||
)
|
||||
|
||||
|
||||
def test_step_info_lives_at_suffix(tmp_path):
|
||||
content = _make_prompt(tmp_path).get_user_message(use_vision=False).content
|
||||
assert isinstance(content, str)
|
||||
|
||||
assert '<step_info>' in content
|
||||
assert content.index('<user_request>') < content.index('<agent_history>')
|
||||
assert content.index('</agent_history>') < content.index('<agent_state>')
|
||||
# Suffix: step_info must come after agent_state and browser_state.
|
||||
assert content.index('<agent_state>') < content.index('<step_info>')
|
||||
assert content.index('<browser_state>') < content.index('<step_info>')
|
||||
|
||||
# And it must NOT live inside <agent_state> any more.
|
||||
state_start = content.index('<agent_state>')
|
||||
state_end = content.index('</agent_state>')
|
||||
assert '<step_info>' not in content[state_start:state_end], 'per-step metadata leaked into <agent_state> prefix region'
|
||||
assert '<user_request>' not in content[state_start:state_end], 'user request leaked back into <agent_state>'
|
||||
|
||||
|
||||
def test_agent_history_end_marker_is_present_once(tmp_path):
|
||||
content = _make_prompt(tmp_path).get_user_message(use_vision=False).content
|
||||
assert isinstance(content, str)
|
||||
|
||||
assert content.count('</agent_history>') == 1, (
|
||||
'super important: LLM gateway cache splitting expects exactly one </agent_history> marker'
|
||||
)
|
||||
|
||||
|
||||
def test_prefix_up_to_step_info_is_stable_across_steps(tmp_path):
|
||||
"""Step number and date can change; the bytes before <step_info> must not."""
|
||||
a = _make_prompt(tmp_path, step_number=3).get_user_message(use_vision=False).content
|
||||
b = _make_prompt(tmp_path, step_number=4).get_user_message(use_vision=False).content
|
||||
assert isinstance(a, str) and isinstance(b, str)
|
||||
prefix_end = a.index('<step_info>')
|
||||
assert a[:prefix_end] == b[:prefix_end], 'message bytes diverged before <step_info> — step counter is leaking into the prefix'
|
||||
# Sanity: the tails do differ (step counter advanced).
|
||||
assert a[prefix_end:] != b[prefix_end:]
|
||||
|
||||
|
||||
def test_agent_state_block_unaffected_by_step_change(tmp_path):
|
||||
"""<agent_state> should not include per-step-varying metadata."""
|
||||
a = _make_prompt(tmp_path, step_number=3).get_user_message(use_vision=False).content
|
||||
b = _make_prompt(tmp_path, step_number=4).get_user_message(use_vision=False).content
|
||||
assert isinstance(a, str) and isinstance(b, str)
|
||||
|
||||
def agent_state_block(s: str) -> str:
|
||||
return s[s.index('<agent_state>') : s.index('</agent_state>')]
|
||||
|
||||
assert agent_state_block(a) == agent_state_block(b)
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,227 @@
|
||||
"""
|
||||
Tests for sandbox structured output handling.
|
||||
|
||||
Tests that output_model_schema works correctly when using @sandbox decorator,
|
||||
specifically that the _output_model_schema private attribute is preserved
|
||||
through serialization/deserialization.
|
||||
"""
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
from browser_use.agent.views import ActionResult, AgentHistory, AgentHistoryList, BrowserStateHistory
|
||||
from browser_use.sandbox.sandbox import _parse_with_type_annotation
|
||||
|
||||
|
||||
class ExtractedData(BaseModel):
|
||||
"""Example structured output model"""
|
||||
|
||||
title: str
|
||||
price: float
|
||||
in_stock: bool
|
||||
|
||||
|
||||
class NestedModel(BaseModel):
|
||||
"""Nested model for testing complex structures"""
|
||||
|
||||
items: list[ExtractedData]
|
||||
total_count: int
|
||||
|
||||
|
||||
class TestGetStructuredOutput:
|
||||
"""Tests for AgentHistoryList.get_structured_output method"""
|
||||
|
||||
def test_get_structured_output_parses_final_result(self):
|
||||
"""Test that get_structured_output correctly parses final result with provided schema"""
|
||||
# Create history with structured JSON as final result
|
||||
json_result = '{"title": "Test Product", "price": 29.99, "in_stock": true}'
|
||||
|
||||
history = AgentHistoryList(
|
||||
history=[
|
||||
AgentHistory(
|
||||
model_output=None,
|
||||
result=[ActionResult(extracted_content=json_result, is_done=True)],
|
||||
state=BrowserStateHistory(url='https://example.com', title='Test', tabs=[], interacted_element=[]),
|
||||
)
|
||||
]
|
||||
)
|
||||
|
||||
# Use get_structured_output with explicit schema
|
||||
result = history.get_structured_output(ExtractedData)
|
||||
|
||||
assert result is not None
|
||||
assert isinstance(result, ExtractedData)
|
||||
assert result.title == 'Test Product'
|
||||
assert result.price == 29.99
|
||||
assert result.in_stock is True
|
||||
|
||||
def test_get_structured_output_returns_none_when_no_final_result(self):
|
||||
"""Test that get_structured_output returns None when there's no final result"""
|
||||
history = AgentHistoryList(
|
||||
history=[
|
||||
AgentHistory(
|
||||
model_output=None,
|
||||
result=[ActionResult(extracted_content=None)],
|
||||
state=BrowserStateHistory(url='https://example.com', title='Test', tabs=[], interacted_element=[]),
|
||||
)
|
||||
]
|
||||
)
|
||||
|
||||
result = history.get_structured_output(ExtractedData)
|
||||
assert result is None
|
||||
|
||||
def test_get_structured_output_with_nested_model(self):
|
||||
"""Test get_structured_output works with nested Pydantic models"""
|
||||
json_result = """
|
||||
{
|
||||
"items": [
|
||||
{"title": "Item 1", "price": 10.0, "in_stock": true},
|
||||
{"title": "Item 2", "price": 20.0, "in_stock": false}
|
||||
],
|
||||
"total_count": 2
|
||||
}
|
||||
"""
|
||||
|
||||
history = AgentHistoryList(
|
||||
history=[
|
||||
AgentHistory(
|
||||
model_output=None,
|
||||
result=[ActionResult(extracted_content=json_result, is_done=True)],
|
||||
state=BrowserStateHistory(url='https://example.com', title='Test', tabs=[], interacted_element=[]),
|
||||
)
|
||||
]
|
||||
)
|
||||
|
||||
result = history.get_structured_output(NestedModel)
|
||||
|
||||
assert result is not None
|
||||
assert len(result.items) == 2
|
||||
assert result.items[0].title == 'Item 1'
|
||||
assert result.total_count == 2
|
||||
|
||||
|
||||
class TestSandboxStructuredOutputParsing:
|
||||
"""Tests for _parse_with_type_annotation handling of AgentHistoryList[T]"""
|
||||
|
||||
def test_parse_agent_history_list_without_generic(self):
|
||||
"""Test parsing AgentHistoryList without generic parameter"""
|
||||
data = {
|
||||
'history': [
|
||||
{
|
||||
'model_output': None,
|
||||
'result': [{'extracted_content': '{"title": "Test", "price": 9.99, "in_stock": true}', 'is_done': True}],
|
||||
'state': {'url': 'https://example.com', 'title': 'Test', 'tabs': []},
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
result = _parse_with_type_annotation(data, AgentHistoryList)
|
||||
|
||||
assert isinstance(result, AgentHistoryList)
|
||||
assert len(result.history) == 1
|
||||
# Without generic, _output_model_schema should be None
|
||||
assert result._output_model_schema is None
|
||||
|
||||
def test_parse_agent_history_list_with_generic_parameter(self):
|
||||
"""Test parsing AgentHistoryList[ExtractedData] preserves output model schema"""
|
||||
data = {
|
||||
'history': [
|
||||
{
|
||||
'model_output': None,
|
||||
'result': [{'extracted_content': '{"title": "Test", "price": 9.99, "in_stock": true}', 'is_done': True}],
|
||||
'state': {'url': 'https://example.com', 'title': 'Test', 'tabs': []},
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
# Parse with generic type annotation
|
||||
result = _parse_with_type_annotation(data, AgentHistoryList[ExtractedData])
|
||||
|
||||
assert isinstance(result, AgentHistoryList)
|
||||
assert len(result.history) == 1
|
||||
# With generic, _output_model_schema should be set
|
||||
assert result._output_model_schema is ExtractedData
|
||||
|
||||
# Now structured_output property should work
|
||||
structured = result.structured_output
|
||||
assert structured is not None
|
||||
assert isinstance(structured, ExtractedData)
|
||||
assert structured.title == 'Test'
|
||||
assert structured.price == 9.99
|
||||
assert structured.in_stock is True
|
||||
|
||||
def test_parse_agent_history_list_structured_output_after_sandbox(self):
|
||||
"""Simulate full sandbox round-trip with AgentHistoryList[T]"""
|
||||
# This simulates what happens when sandbox returns data
|
||||
json_content = '{"title": "Product", "price": 49.99, "in_stock": false}'
|
||||
|
||||
data = {
|
||||
'history': [
|
||||
{
|
||||
'model_output': None,
|
||||
'result': [{'extracted_content': json_content, 'is_done': True}],
|
||||
'state': {'url': 'https://shop.com', 'title': 'Shop', 'tabs': []},
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
# Sandbox parses with return type annotation AgentHistoryList[ExtractedData]
|
||||
result = _parse_with_type_annotation(data, AgentHistoryList[ExtractedData])
|
||||
|
||||
# User accesses structured_output property
|
||||
output = result.structured_output
|
||||
|
||||
assert output is not None
|
||||
assert output.title == 'Product'
|
||||
assert output.price == 49.99
|
||||
assert output.in_stock is False
|
||||
|
||||
|
||||
class TestStructuredOutputPropertyFallback:
|
||||
"""Tests for structured_output property behavior with and without _output_model_schema"""
|
||||
|
||||
def test_structured_output_property_works_when_schema_set(self):
|
||||
"""Test structured_output property works when _output_model_schema is set"""
|
||||
json_result = '{"title": "Test", "price": 5.0, "in_stock": true}'
|
||||
|
||||
history = AgentHistoryList(
|
||||
history=[
|
||||
AgentHistory(
|
||||
model_output=None,
|
||||
result=[ActionResult(extracted_content=json_result, is_done=True)],
|
||||
state=BrowserStateHistory(url='https://example.com', title='Test', tabs=[], interacted_element=[]),
|
||||
)
|
||||
]
|
||||
)
|
||||
# Manually set the schema (as Agent.run() does)
|
||||
history._output_model_schema = ExtractedData
|
||||
|
||||
result = history.structured_output
|
||||
|
||||
assert result is not None
|
||||
assert isinstance(result, ExtractedData)
|
||||
assert result.title == 'Test'
|
||||
|
||||
def test_structured_output_property_returns_none_without_schema(self):
|
||||
"""Test structured_output property returns None when _output_model_schema is not set"""
|
||||
json_result = '{"title": "Test", "price": 5.0, "in_stock": true}'
|
||||
|
||||
history = AgentHistoryList(
|
||||
history=[
|
||||
AgentHistory(
|
||||
model_output=None,
|
||||
result=[ActionResult(extracted_content=json_result, is_done=True)],
|
||||
state=BrowserStateHistory(url='https://example.com', title='Test', tabs=[], interacted_element=[]),
|
||||
)
|
||||
]
|
||||
)
|
||||
# Don't set _output_model_schema
|
||||
|
||||
result = history.structured_output
|
||||
|
||||
# Property returns None because schema is not set
|
||||
assert result is None
|
||||
|
||||
# But get_structured_output with explicit schema works
|
||||
explicit_result = history.get_structured_output(ExtractedData)
|
||||
assert explicit_result is not None
|
||||
assert explicit_result.title == 'Test'
|
||||
@@ -0,0 +1,173 @@
|
||||
"""Test that screenshot action is excluded when use_vision != 'auto'."""
|
||||
|
||||
import pytest
|
||||
|
||||
from browser_use.agent.service import Agent
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
from browser_use.browser.session import BrowserSession
|
||||
from browser_use.tools.service import Tools
|
||||
from tests.ci.conftest import create_mock_llm
|
||||
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
async def browser_session():
|
||||
session = BrowserSession(browser_profile=BrowserProfile(headless=True))
|
||||
await session.start()
|
||||
yield session
|
||||
await session.kill()
|
||||
|
||||
|
||||
def test_screenshot_excluded_with_use_vision_false():
|
||||
"""Test that screenshot action is excluded when use_vision=False."""
|
||||
mock_llm = create_mock_llm(actions=['{"action": [{"done": {"text": "test", "success": true}}]}'])
|
||||
|
||||
agent = Agent(
|
||||
task='test',
|
||||
llm=mock_llm,
|
||||
use_vision=False,
|
||||
)
|
||||
|
||||
# Verify screenshot is not in the registry
|
||||
assert 'screenshot' not in agent.tools.registry.registry.actions, 'Screenshot should be excluded when use_vision=False'
|
||||
|
||||
|
||||
def test_screenshot_excluded_with_use_vision_true():
|
||||
"""Test that screenshot action is excluded when use_vision=True."""
|
||||
mock_llm = create_mock_llm(actions=['{"action": [{"done": {"text": "test", "success": true}}]}'])
|
||||
|
||||
agent = Agent(
|
||||
task='test',
|
||||
llm=mock_llm,
|
||||
use_vision=True,
|
||||
)
|
||||
|
||||
# Verify screenshot is not in the registry
|
||||
assert 'screenshot' not in agent.tools.registry.registry.actions, 'Screenshot should be excluded when use_vision=True'
|
||||
|
||||
|
||||
def test_screenshot_included_with_use_vision_auto():
|
||||
"""Test that screenshot action is included when use_vision='auto'."""
|
||||
mock_llm = create_mock_llm(actions=['{"action": [{"done": {"text": "test", "success": true}}]}'])
|
||||
|
||||
agent = Agent(
|
||||
task='test',
|
||||
llm=mock_llm,
|
||||
use_vision='auto',
|
||||
)
|
||||
|
||||
# Verify screenshot IS in the registry
|
||||
assert 'screenshot' in agent.tools.registry.registry.actions, 'Screenshot should be included when use_vision="auto"'
|
||||
|
||||
|
||||
def test_screenshot_excluded_with_custom_tools_and_use_vision_false():
|
||||
"""Test that screenshot action is excluded even when user passes custom tools and use_vision=False.
|
||||
|
||||
This is the critical test case that verifies the fix:
|
||||
When users pass their own Tools instance with screenshot included,
|
||||
the Agent should still enforce the exclusion if use_vision != 'auto'.
|
||||
"""
|
||||
mock_llm = create_mock_llm(actions=['{"action": [{"done": {"text": "test", "success": true}}]}'])
|
||||
|
||||
# Create custom tools that includes screenshot action
|
||||
custom_tools = Tools()
|
||||
assert 'screenshot' in custom_tools.registry.registry.actions, 'Custom tools should have screenshot by default'
|
||||
|
||||
# Pass custom tools to agent with use_vision=False
|
||||
agent = Agent(
|
||||
task='test',
|
||||
llm=mock_llm,
|
||||
tools=custom_tools,
|
||||
use_vision=False,
|
||||
)
|
||||
|
||||
# Verify screenshot is excluded even though user passed custom tools
|
||||
assert 'screenshot' not in agent.tools.registry.registry.actions, (
|
||||
'Screenshot should be excluded when use_vision=False, even with custom tools'
|
||||
)
|
||||
|
||||
|
||||
def test_screenshot_excluded_with_custom_tools_and_use_vision_true():
|
||||
"""Test that screenshot action is excluded even when user passes custom tools and use_vision=True.
|
||||
|
||||
This is another critical test case:
|
||||
When users pass their own Tools instance with screenshot included,
|
||||
the Agent should still enforce the exclusion if use_vision != 'auto'.
|
||||
"""
|
||||
mock_llm = create_mock_llm(actions=['{"action": [{"done": {"text": "test", "success": true}}]}'])
|
||||
|
||||
# Create custom tools - by default Tools() includes screenshot
|
||||
# (unless exclude_actions is passed)
|
||||
custom_tools = Tools()
|
||||
# Note: We check if screenshot exists in the default set, but it might not
|
||||
# exist if use_vision defaults have changed. The key is that after passing
|
||||
# to Agent with use_vision=True, it should be excluded.
|
||||
has_screenshot_before = 'screenshot' in custom_tools.registry.registry.actions
|
||||
|
||||
# Pass custom tools to agent with use_vision=True
|
||||
agent = Agent(
|
||||
task='test',
|
||||
llm=mock_llm,
|
||||
tools=custom_tools,
|
||||
use_vision=True,
|
||||
)
|
||||
|
||||
# Verify screenshot is excluded even though user passed custom tools
|
||||
# The key test: screenshot should be excluded after Agent init
|
||||
assert 'screenshot' not in agent.tools.registry.registry.actions, (
|
||||
f'Screenshot should be excluded when use_vision=True, even with custom tools (had screenshot before: {has_screenshot_before})'
|
||||
)
|
||||
|
||||
|
||||
def test_screenshot_included_with_custom_tools_and_use_vision_auto():
|
||||
"""Test that screenshot action is kept when user passes custom tools and use_vision='auto'."""
|
||||
mock_llm = create_mock_llm(actions=['{"action": [{"done": {"text": "test", "success": true}}]}'])
|
||||
|
||||
# Create custom tools that includes screenshot action
|
||||
custom_tools = Tools()
|
||||
assert 'screenshot' in custom_tools.registry.registry.actions, 'Custom tools should have screenshot by default'
|
||||
|
||||
# Pass custom tools to agent with use_vision='auto'
|
||||
agent = Agent(
|
||||
task='test',
|
||||
llm=mock_llm,
|
||||
tools=custom_tools,
|
||||
use_vision='auto',
|
||||
)
|
||||
|
||||
# Verify screenshot is kept when use_vision='auto'
|
||||
assert 'screenshot' in agent.tools.registry.registry.actions, (
|
||||
'Screenshot should be included when use_vision="auto", even with custom tools'
|
||||
)
|
||||
|
||||
|
||||
def test_tools_exclude_action_method():
|
||||
"""Test the Tools.exclude_action() method directly."""
|
||||
tools = Tools()
|
||||
|
||||
# Verify screenshot is included initially
|
||||
assert 'screenshot' in tools.registry.registry.actions, 'Screenshot should be included by default'
|
||||
|
||||
# Exclude screenshot
|
||||
tools.exclude_action('screenshot')
|
||||
|
||||
# Verify screenshot is excluded
|
||||
assert 'screenshot' not in tools.registry.registry.actions, 'Screenshot should be excluded after calling exclude_action()'
|
||||
assert 'screenshot' in tools.registry.exclude_actions, 'Screenshot should be in exclude_actions list'
|
||||
|
||||
|
||||
def test_exclude_action_prevents_re_registration():
|
||||
"""Test that excluded actions cannot be re-registered."""
|
||||
tools = Tools()
|
||||
|
||||
# Exclude screenshot
|
||||
tools.exclude_action('screenshot')
|
||||
assert 'screenshot' not in tools.registry.registry.actions
|
||||
|
||||
# Try to re-register screenshot (simulating what happens in __init__)
|
||||
# The decorator should skip registration since it's in exclude_actions
|
||||
@tools.registry.action('Test screenshot action')
|
||||
async def screenshot():
|
||||
return 'test'
|
||||
|
||||
# Verify it was not re-registered
|
||||
assert 'screenshot' not in tools.registry.registry.actions, 'Excluded action should not be re-registered'
|
||||
@@ -0,0 +1,465 @@
|
||||
"""Tests for search_page and find_elements actions."""
|
||||
|
||||
import asyncio
|
||||
|
||||
import pytest
|
||||
from pytest_httpserver import HTTPServer
|
||||
|
||||
from browser_use.agent.views import ActionResult
|
||||
from browser_use.browser import BrowserProfile, BrowserSession
|
||||
from browser_use.tools.service import Tools
|
||||
|
||||
# --- Fixtures ---
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def http_server():
|
||||
"""Test HTTP server serving pages for search/find tests."""
|
||||
server = HTTPServer()
|
||||
server.start()
|
||||
|
||||
server.expect_request('/products').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Products</title></head>
|
||||
<body>
|
||||
<h1>Product Catalog</h1>
|
||||
<div id="main">
|
||||
<table class="products">
|
||||
<thead>
|
||||
<tr><th>Name</th><th>Price</th><th>Rating</th></tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr class="product-row"><td>Widget A</td><td>$29.99</td><td>4.5 stars</td></tr>
|
||||
<tr class="product-row"><td>Widget B</td><td>$49.99</td><td>4.2 stars</td></tr>
|
||||
<tr class="product-row"><td>Gadget C</td><td>$19.50</td><td>3.8 stars</td></tr>
|
||||
<tr class="product-row"><td>Gadget D</td><td>$99.00</td><td>4.9 stars</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<div class="pagination">
|
||||
<a href="/products?page=1" class="page-link active">1</a>
|
||||
<a href="/products?page=2" class="page-link">2</a>
|
||||
<a href="/products?page=3" class="page-link">3</a>
|
||||
</div>
|
||||
</div>
|
||||
<footer id="footer">
|
||||
<p>Best price guarantee on all items.</p>
|
||||
<p>Contact us at support@example.com</p>
|
||||
</footer>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
server.expect_request('/articles').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Articles</title></head>
|
||||
<body>
|
||||
<article id="post-1">
|
||||
<h2>Introduction to Python</h2>
|
||||
<p>Python is a versatile programming language used in web development, data science, and automation.</p>
|
||||
<a href="/articles/python" class="read-more">Read more</a>
|
||||
</article>
|
||||
<article id="post-2">
|
||||
<h2>JavaScript for Beginners</h2>
|
||||
<p>JavaScript powers the interactive web. Learn about DOM manipulation and event handling.</p>
|
||||
<a href="/articles/javascript" class="read-more">Read more</a>
|
||||
</article>
|
||||
<article id="post-3">
|
||||
<h2>Advanced CSS Techniques</h2>
|
||||
<p>Master CSS Grid, Flexbox, and custom properties for modern web layouts.</p>
|
||||
<a href="/articles/css" class="read-more">Read more</a>
|
||||
</article>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
server.expect_request('/empty').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Empty</title></head>
|
||||
<body>
|
||||
<div id="content"></div>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
server.expect_request('/case-test').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Case Test</title></head>
|
||||
<body>
|
||||
<p>The Quick Brown Fox jumps over the lazy dog.</p>
|
||||
<p>QUICK BROWN FOX is an uppercase variant.</p>
|
||||
<p>quick brown fox is a lowercase variant.</p>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
# /images-page route is registered dynamically in base_url fixture once port is known
|
||||
yield server
|
||||
server.stop()
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def base_url(http_server):
|
||||
url = f'http://{http_server.host}:{http_server.port}'
|
||||
# Register images page here so we can embed the absolute img src URL
|
||||
http_server.expect_request('/images-page').respond_with_data(
|
||||
f"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Images Page</title></head>
|
||||
<body>
|
||||
<img src="{url}/images/product.jpg" alt="Product">
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
return url
|
||||
|
||||
|
||||
@pytest.fixture(scope='module')
|
||||
async def browser_session():
|
||||
session = BrowserSession(
|
||||
browser_profile=BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
keep_alive=True,
|
||||
)
|
||||
)
|
||||
await session.start()
|
||||
yield session
|
||||
await session.kill()
|
||||
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
def tools():
|
||||
return Tools()
|
||||
|
||||
|
||||
# --- Helper ---
|
||||
|
||||
|
||||
async def _navigate_and_wait(tools, browser_session, url):
|
||||
"""Navigate to URL and wait for page load."""
|
||||
await tools.navigate(url=url, new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
|
||||
# --- search_page tests ---
|
||||
|
||||
|
||||
class TestSearchPage:
|
||||
"""Tests for the search_page action."""
|
||||
|
||||
async def test_literal_text_search(self, tools, browser_session, base_url):
|
||||
"""Literal text search finds matches with context."""
|
||||
await _navigate_and_wait(tools, browser_session, f'{base_url}/products')
|
||||
|
||||
result = await tools.search_page(pattern='Widget A', browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is None
|
||||
assert result.extracted_content is not None
|
||||
assert 'Widget A' in result.extracted_content
|
||||
assert '1 match' in result.extracted_content
|
||||
|
||||
async def test_regex_search_prices(self, tools, browser_session, base_url):
|
||||
"""Regex search finds all price patterns on the page."""
|
||||
await _navigate_and_wait(tools, browser_session, f'{base_url}/products')
|
||||
|
||||
result = await tools.search_page(pattern=r'\$\d+\.\d{2}', regex=True, browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is None
|
||||
assert result.extracted_content is not None
|
||||
# Should find $29.99, $49.99, $19.50, $99.00
|
||||
assert '4 matches' in result.extracted_content
|
||||
assert '$29.99' in result.extracted_content
|
||||
assert '$49.99' in result.extracted_content
|
||||
|
||||
async def test_css_scope_limits_search(self, tools, browser_session, base_url):
|
||||
"""css_scope limits search to elements within the selector."""
|
||||
await _navigate_and_wait(tools, browser_session, f'{base_url}/products')
|
||||
|
||||
# Search only in footer
|
||||
result = await tools.search_page(pattern='price', css_scope='#footer', browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is None
|
||||
assert result.extracted_content is not None
|
||||
# "Best price guarantee" is in footer
|
||||
assert '1 match' in result.extracted_content
|
||||
assert 'guarantee' in result.extracted_content
|
||||
|
||||
async def test_case_insensitive_default(self, tools, browser_session, base_url):
|
||||
"""Search is case-insensitive by default."""
|
||||
await _navigate_and_wait(tools, browser_session, f'{base_url}/case-test')
|
||||
|
||||
result = await tools.search_page(pattern='quick brown fox', browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is None
|
||||
assert result.extracted_content is not None
|
||||
# Should match all three variants (Quick, QUICK, quick)
|
||||
assert '3 matches' in result.extracted_content
|
||||
|
||||
async def test_case_sensitive(self, tools, browser_session, base_url):
|
||||
"""case_sensitive=True restricts to exact case."""
|
||||
await _navigate_and_wait(tools, browser_session, f'{base_url}/case-test')
|
||||
|
||||
result = await tools.search_page(pattern='QUICK BROWN FOX', case_sensitive=True, browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is None
|
||||
assert result.extracted_content is not None
|
||||
assert '1 match' in result.extracted_content
|
||||
|
||||
async def test_max_results(self, tools, browser_session, base_url):
|
||||
"""max_results limits the number of returned matches."""
|
||||
await _navigate_and_wait(tools, browser_session, f'{base_url}/products')
|
||||
|
||||
result = await tools.search_page(pattern=r'\$\d+\.\d{2}', regex=True, max_results=2, browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is None
|
||||
assert result.extracted_content is not None
|
||||
# Total should still show 4, but only 2 are displayed
|
||||
assert '4 matches' in result.extracted_content
|
||||
assert 'Increase max_results' in result.extracted_content
|
||||
|
||||
async def test_no_matches(self, tools, browser_session, base_url):
|
||||
"""No matches returns a clean message, not an error."""
|
||||
await _navigate_and_wait(tools, browser_session, f'{base_url}/products')
|
||||
|
||||
result = await tools.search_page(pattern='xyznonexistent', browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is None
|
||||
assert result.extracted_content is not None
|
||||
assert 'No matches found' in result.extracted_content
|
||||
|
||||
async def test_element_path_in_results(self, tools, browser_session, base_url):
|
||||
"""Matches include the element path for context."""
|
||||
await _navigate_and_wait(tools, browser_session, f'{base_url}/products')
|
||||
|
||||
result = await tools.search_page(pattern='guarantee', browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is None
|
||||
assert result.extracted_content is not None
|
||||
# Should show element path containing footer
|
||||
assert '(in' in result.extracted_content
|
||||
|
||||
async def test_invalid_css_scope(self, tools, browser_session, base_url):
|
||||
"""Invalid css_scope returns a clear error."""
|
||||
await _navigate_and_wait(tools, browser_session, f'{base_url}/products')
|
||||
|
||||
result = await tools.search_page(pattern='test', css_scope='#nonexistent-scope', browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is not None
|
||||
assert 'scope' in result.error.lower() or 'not found' in result.error.lower()
|
||||
|
||||
async def test_memory_set(self, tools, browser_session, base_url):
|
||||
"""long_term_memory is set with match count summary."""
|
||||
await _navigate_and_wait(tools, browser_session, f'{base_url}/products')
|
||||
|
||||
result = await tools.search_page(pattern='Widget', browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.long_term_memory is not None
|
||||
assert 'Widget' in result.long_term_memory
|
||||
assert 'match' in result.long_term_memory
|
||||
|
||||
|
||||
# --- find_elements tests ---
|
||||
|
||||
|
||||
class TestFindElements:
|
||||
"""Tests for the find_elements action."""
|
||||
|
||||
async def test_basic_selector(self, tools, browser_session, base_url):
|
||||
"""Basic CSS selector returns correct elements."""
|
||||
await _navigate_and_wait(tools, browser_session, f'{base_url}/products')
|
||||
|
||||
result = await tools.find_elements(selector='tr.product-row', browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is None
|
||||
assert result.extracted_content is not None
|
||||
assert '4 elements' in result.extracted_content
|
||||
assert 'Widget A' in result.extracted_content
|
||||
assert 'Gadget D' in result.extracted_content
|
||||
|
||||
async def test_attribute_extraction(self, tools, browser_session, base_url):
|
||||
"""attributes parameter extracts specific attributes from elements."""
|
||||
await _navigate_and_wait(tools, browser_session, f'{base_url}/products')
|
||||
|
||||
result = await tools.find_elements(
|
||||
selector='a.page-link',
|
||||
attributes=['href', 'class'],
|
||||
browser_session=browser_session,
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is None
|
||||
assert result.extracted_content is not None
|
||||
assert '3 elements' in result.extracted_content
|
||||
assert 'href=' in result.extracted_content
|
||||
assert '/products?page=' in result.extracted_content
|
||||
|
||||
async def test_max_results_limiting(self, tools, browser_session, base_url):
|
||||
"""max_results limits displayed elements while showing total count."""
|
||||
await _navigate_and_wait(tools, browser_session, f'{base_url}/products')
|
||||
|
||||
result = await tools.find_elements(selector='tr.product-row', max_results=2, browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is None
|
||||
assert result.extracted_content is not None
|
||||
assert '4 elements' in result.extracted_content
|
||||
assert 'Showing 2 of 4' in result.extracted_content
|
||||
|
||||
async def test_no_matching_elements(self, tools, browser_session, base_url):
|
||||
"""No matches returns a clean message, not an error."""
|
||||
await _navigate_and_wait(tools, browser_session, f'{base_url}/products')
|
||||
|
||||
result = await tools.find_elements(selector='div.nonexistent', browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is None
|
||||
assert result.extracted_content is not None
|
||||
assert 'No elements found' in result.extracted_content
|
||||
|
||||
async def test_invalid_selector(self, tools, browser_session, base_url):
|
||||
"""Invalid CSS selector returns a clear error, not a crash."""
|
||||
await _navigate_and_wait(tools, browser_session, f'{base_url}/products')
|
||||
|
||||
result = await tools.find_elements(selector='[[[invalid', browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is not None
|
||||
assert 'selector' in result.error.lower() or 'invalid' in result.error.lower()
|
||||
|
||||
async def test_include_text_false(self, tools, browser_session, base_url):
|
||||
"""include_text=False omits text content from results."""
|
||||
await _navigate_and_wait(tools, browser_session, f'{base_url}/articles')
|
||||
|
||||
result = await tools.find_elements(selector='article', include_text=False, browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is None
|
||||
assert result.extracted_content is not None
|
||||
assert '3 elements' in result.extracted_content
|
||||
# Text content should not appear (no article body text)
|
||||
# But the tag and children count should still be present
|
||||
assert '<article>' in result.extracted_content
|
||||
|
||||
async def test_nested_selectors(self, tools, browser_session, base_url):
|
||||
"""Nested CSS selectors (child combinator) work correctly."""
|
||||
await _navigate_and_wait(tools, browser_session, f'{base_url}/articles')
|
||||
|
||||
result = await tools.find_elements(
|
||||
selector='article a.read-more',
|
||||
attributes=['href'],
|
||||
browser_session=browser_session,
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is None
|
||||
assert result.extracted_content is not None
|
||||
assert '3 elements' in result.extracted_content
|
||||
assert '/articles/python' in result.extracted_content
|
||||
assert '/articles/javascript' in result.extracted_content
|
||||
assert '/articles/css' in result.extracted_content
|
||||
|
||||
async def test_children_count(self, tools, browser_session, base_url):
|
||||
"""Elements show children count."""
|
||||
await _navigate_and_wait(tools, browser_session, f'{base_url}/products')
|
||||
|
||||
result = await tools.find_elements(selector='table.products thead tr', browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is None
|
||||
assert result.extracted_content is not None
|
||||
assert '1 element' in result.extracted_content
|
||||
# The header row has 3 <th> children
|
||||
assert '3 children' in result.extracted_content
|
||||
|
||||
async def test_memory_set(self, tools, browser_session, base_url):
|
||||
"""long_term_memory is set with element count summary."""
|
||||
await _navigate_and_wait(tools, browser_session, f'{base_url}/products')
|
||||
|
||||
result = await tools.find_elements(selector='tr.product-row', browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.long_term_memory is not None
|
||||
assert '4 element' in result.long_term_memory
|
||||
|
||||
async def test_empty_page(self, tools, browser_session, base_url):
|
||||
"""Works on a nearly empty page without errors."""
|
||||
await _navigate_and_wait(tools, browser_session, f'{base_url}/empty')
|
||||
|
||||
result = await tools.find_elements(selector='p', browser_session=browser_session)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is None
|
||||
assert result.extracted_content is not None
|
||||
assert 'No elements found' in result.extracted_content
|
||||
|
||||
async def test_img_src_attribute_resolved(self, tools, browser_session, base_url):
|
||||
"""find_elements with attributes=['src'] returns absolute URLs for img elements."""
|
||||
await _navigate_and_wait(tools, browser_session, f'{base_url}/images-page')
|
||||
|
||||
result = await tools.find_elements(
|
||||
selector='img',
|
||||
attributes=['src'],
|
||||
browser_session=browser_session,
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.error is None
|
||||
assert result.extracted_content is not None
|
||||
assert '1 element' in result.extracted_content
|
||||
assert 'src=' in result.extracted_content
|
||||
# The resolved DOM property should give the absolute URL (including the httpserver base URL)
|
||||
assert base_url in result.extracted_content
|
||||
assert 'product.jpg' in result.extracted_content
|
||||
|
||||
|
||||
# --- Registration tests ---
|
||||
|
||||
|
||||
class TestRegistration:
|
||||
"""Test that new actions are properly registered."""
|
||||
|
||||
async def test_search_page_registered(self, tools):
|
||||
"""search_page is in the default action registry."""
|
||||
assert 'search_page' in tools.registry.registry.actions
|
||||
|
||||
async def test_find_elements_registered(self, tools):
|
||||
"""find_elements is in the default action registry."""
|
||||
assert 'find_elements' in tools.registry.registry.actions
|
||||
|
||||
async def test_excluded_actions(self):
|
||||
"""New actions can be excluded via exclude_actions."""
|
||||
excluded_tools = Tools(exclude_actions=['search_page', 'find_elements'])
|
||||
assert 'search_page' not in excluded_tools.registry.registry.actions
|
||||
assert 'find_elements' not in excluded_tools.registry.registry.actions
|
||||
# Other actions still present
|
||||
assert 'navigate' in excluded_tools.registry.registry.actions
|
||||
@@ -0,0 +1,631 @@
|
||||
"""Tests for schema-enforced structured extraction."""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import tempfile
|
||||
from unittest.mock import AsyncMock
|
||||
|
||||
import pytest
|
||||
from pydantic import ValidationError
|
||||
from pytest_httpserver import HTTPServer
|
||||
|
||||
from browser_use.agent.views import ActionResult
|
||||
from browser_use.browser import BrowserProfile, BrowserSession
|
||||
from browser_use.filesystem.file_system import FileSystem
|
||||
from browser_use.llm.base import BaseChatModel
|
||||
from browser_use.llm.views import ChatInvokeCompletion
|
||||
from browser_use.tools.extraction.schema_utils import schema_dict_to_pydantic_model
|
||||
from browser_use.tools.extraction.views import ExtractionResult
|
||||
from browser_use.tools.service import Tools
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Unit tests: schema_dict_to_pydantic_model
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestSchemaDictToPydanticModel:
|
||||
"""Unit tests for the JSON-Schema → Pydantic model converter."""
|
||||
|
||||
def test_flat_object(self):
|
||||
schema = {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'name': {'type': 'string'},
|
||||
'age': {'type': 'integer'},
|
||||
},
|
||||
'required': ['name', 'age'],
|
||||
}
|
||||
Model = schema_dict_to_pydantic_model(schema)
|
||||
instance = Model(name='Alice', age=30)
|
||||
assert instance.name == 'Alice' # type: ignore[attr-defined]
|
||||
assert instance.age == 30 # type: ignore[attr-defined]
|
||||
|
||||
def test_nested_object(self):
|
||||
schema = {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'person': {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'first': {'type': 'string'},
|
||||
'last': {'type': 'string'},
|
||||
},
|
||||
'required': ['first'],
|
||||
},
|
||||
},
|
||||
'required': ['person'],
|
||||
}
|
||||
Model = schema_dict_to_pydantic_model(schema)
|
||||
instance = Model(person={'first': 'Bob', 'last': 'Smith'})
|
||||
assert instance.person.first == 'Bob' # type: ignore[attr-defined]
|
||||
|
||||
def test_array_of_objects(self):
|
||||
schema = {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'items': {
|
||||
'type': 'array',
|
||||
'items': {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'id': {'type': 'integer'},
|
||||
'label': {'type': 'string'},
|
||||
},
|
||||
'required': ['id', 'label'],
|
||||
},
|
||||
},
|
||||
},
|
||||
'required': ['items'],
|
||||
}
|
||||
Model = schema_dict_to_pydantic_model(schema)
|
||||
instance = Model(items=[{'id': 1, 'label': 'a'}, {'id': 2, 'label': 'b'}])
|
||||
assert len(instance.items) == 2 # type: ignore[attr-defined]
|
||||
assert instance.items[0].id == 1 # type: ignore[attr-defined]
|
||||
|
||||
def test_array_of_primitives(self):
|
||||
schema = {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'tags': {'type': 'array', 'items': {'type': 'string'}},
|
||||
},
|
||||
'required': ['tags'],
|
||||
}
|
||||
Model = schema_dict_to_pydantic_model(schema)
|
||||
instance = Model(tags=['a', 'b', 'c'])
|
||||
assert instance.tags == ['a', 'b', 'c'] # type: ignore[attr-defined]
|
||||
|
||||
def test_enum_field(self):
|
||||
schema = {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'status': {'type': 'string', 'enum': ['active', 'inactive']},
|
||||
},
|
||||
'required': ['status'],
|
||||
}
|
||||
Model = schema_dict_to_pydantic_model(schema)
|
||||
instance = Model(status='active')
|
||||
assert instance.status == 'active' # type: ignore[attr-defined]
|
||||
|
||||
def test_optional_enum_defaults_to_none(self):
|
||||
"""Non-required enum fields default to None, not an out-of-set empty string."""
|
||||
schema = {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'name': {'type': 'string'},
|
||||
'priority': {'type': 'string', 'enum': ['low', 'medium', 'high']},
|
||||
},
|
||||
'required': ['name'],
|
||||
}
|
||||
Model = schema_dict_to_pydantic_model(schema)
|
||||
instance = Model(name='task1')
|
||||
assert instance.priority is None # type: ignore[attr-defined]
|
||||
# Serialized output must not contain an out-of-set value
|
||||
dumped = instance.model_dump(mode='json')
|
||||
assert dumped['priority'] is None
|
||||
|
||||
# When provided, value still works
|
||||
instance2 = Model(name='task2', priority='high')
|
||||
assert instance2.priority == 'high' # type: ignore[attr-defined]
|
||||
|
||||
def test_optional_fields_get_type_appropriate_defaults(self):
|
||||
schema = {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'name': {'type': 'string'},
|
||||
'nickname': {'type': 'string'},
|
||||
'score': {'type': 'number'},
|
||||
'rank': {'type': 'integer'},
|
||||
'active': {'type': 'boolean'},
|
||||
'tags': {'type': 'array', 'items': {'type': 'string'}},
|
||||
},
|
||||
'required': ['name'],
|
||||
}
|
||||
Model = schema_dict_to_pydantic_model(schema)
|
||||
instance = Model(name='Alice')
|
||||
assert instance.name == 'Alice' # type: ignore[attr-defined]
|
||||
assert instance.nickname == '' # type: ignore[attr-defined]
|
||||
assert instance.score == 0.0 # type: ignore[attr-defined]
|
||||
assert instance.rank == 0 # type: ignore[attr-defined]
|
||||
assert instance.active is False # type: ignore[attr-defined]
|
||||
assert instance.tags == [] # type: ignore[attr-defined]
|
||||
|
||||
def test_optional_non_nullable_rejects_null(self):
|
||||
"""Non-required fields that aren't nullable must reject explicit null."""
|
||||
schema = {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'name': {'type': 'string'},
|
||||
'nickname': {'type': 'string'},
|
||||
},
|
||||
'required': ['name'],
|
||||
}
|
||||
Model = schema_dict_to_pydantic_model(schema)
|
||||
with pytest.raises(ValidationError):
|
||||
Model(name='Alice', nickname=None)
|
||||
|
||||
def test_optional_with_explicit_default(self):
|
||||
schema = {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'name': {'type': 'string'},
|
||||
'color': {'type': 'string', 'default': 'blue'},
|
||||
},
|
||||
'required': ['name'],
|
||||
}
|
||||
Model = schema_dict_to_pydantic_model(schema)
|
||||
instance = Model(name='Alice')
|
||||
assert instance.color == 'blue' # type: ignore[attr-defined]
|
||||
|
||||
def test_optional_nested_object_defaults_to_none(self):
|
||||
"""Non-required nested objects fall back to None since constructing a default is not feasible."""
|
||||
schema = {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'name': {'type': 'string'},
|
||||
'address': {
|
||||
'type': 'object',
|
||||
'properties': {'city': {'type': 'string'}},
|
||||
'required': ['city'],
|
||||
},
|
||||
},
|
||||
'required': ['name'],
|
||||
}
|
||||
Model = schema_dict_to_pydantic_model(schema)
|
||||
instance = Model(name='Alice')
|
||||
assert instance.address is None # type: ignore[attr-defined]
|
||||
|
||||
def test_model_name_from_title(self):
|
||||
schema = {
|
||||
'title': 'ProductInfo',
|
||||
'type': 'object',
|
||||
'properties': {'sku': {'type': 'string'}},
|
||||
'required': ['sku'],
|
||||
}
|
||||
Model = schema_dict_to_pydantic_model(schema)
|
||||
assert Model.__name__ == 'ProductInfo'
|
||||
|
||||
def test_model_validate_json_roundtrip(self):
|
||||
schema = {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'x': {'type': 'number'},
|
||||
'y': {'type': 'boolean'},
|
||||
},
|
||||
'required': ['x', 'y'],
|
||||
}
|
||||
Model = schema_dict_to_pydantic_model(schema)
|
||||
instance = Model(x=3.14, y=True)
|
||||
raw = instance.model_dump_json()
|
||||
restored = Model.model_validate_json(raw)
|
||||
assert restored.x == instance.x # type: ignore[attr-defined]
|
||||
assert restored.y == instance.y # type: ignore[attr-defined]
|
||||
|
||||
def test_rejects_ref(self):
|
||||
schema = {
|
||||
'type': 'object',
|
||||
'properties': {'item': {'$ref': '#/$defs/Item'}},
|
||||
'$defs': {'Item': {'type': 'object', 'properties': {'name': {'type': 'string'}}}},
|
||||
}
|
||||
with pytest.raises(ValueError, match='Unsupported JSON Schema keyword'):
|
||||
schema_dict_to_pydantic_model(schema)
|
||||
|
||||
def test_rejects_allOf(self):
|
||||
schema = {
|
||||
'type': 'object',
|
||||
'properties': {'x': {'allOf': [{'type': 'string'}]}},
|
||||
}
|
||||
with pytest.raises(ValueError, match='Unsupported JSON Schema keyword'):
|
||||
schema_dict_to_pydantic_model(schema)
|
||||
|
||||
def test_rejects_non_object_toplevel(self):
|
||||
with pytest.raises(ValueError, match='type "object"'):
|
||||
schema_dict_to_pydantic_model({'type': 'array', 'items': {'type': 'string'}})
|
||||
|
||||
def test_rejects_empty_properties(self):
|
||||
with pytest.raises(ValueError, match='at least one property'):
|
||||
schema_dict_to_pydantic_model({'type': 'object', 'properties': {}})
|
||||
|
||||
def test_extra_fields_forbidden(self):
|
||||
schema = {
|
||||
'type': 'object',
|
||||
'properties': {'name': {'type': 'string'}},
|
||||
'required': ['name'],
|
||||
}
|
||||
Model = schema_dict_to_pydantic_model(schema)
|
||||
with pytest.raises(ValidationError):
|
||||
Model(name='ok', bogus='nope')
|
||||
|
||||
def test_nullable_field(self):
|
||||
schema = {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'value': {'type': 'string', 'nullable': True},
|
||||
},
|
||||
'required': ['value'],
|
||||
}
|
||||
Model = schema_dict_to_pydantic_model(schema)
|
||||
instance = Model(value=None)
|
||||
assert instance.value is None # type: ignore[attr-defined]
|
||||
|
||||
def test_field_descriptions_preserved(self):
|
||||
schema = {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'price': {'type': 'number', 'description': 'The price in USD'},
|
||||
},
|
||||
'required': ['price'],
|
||||
}
|
||||
Model = schema_dict_to_pydantic_model(schema)
|
||||
field_info = Model.model_fields['price']
|
||||
assert field_info.description == 'The price in USD'
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Unit tests: ExtractionResult
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestExtractionResult:
|
||||
def test_construction(self):
|
||||
er = ExtractionResult(
|
||||
data={'name': 'Alice'},
|
||||
schema_used={'type': 'object', 'properties': {'name': {'type': 'string'}}},
|
||||
)
|
||||
assert er.data == {'name': 'Alice'}
|
||||
assert er.is_partial is False
|
||||
assert er.source_url is None
|
||||
|
||||
def test_serialization_roundtrip(self):
|
||||
er = ExtractionResult(
|
||||
data={'items': [1, 2]},
|
||||
schema_used={'type': 'object', 'properties': {'items': {'type': 'array'}}},
|
||||
is_partial=True,
|
||||
source_url='http://example.com',
|
||||
content_stats={'original_html_chars': 5000},
|
||||
)
|
||||
raw = er.model_dump_json()
|
||||
restored = ExtractionResult.model_validate_json(raw)
|
||||
assert restored == er
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Integration tests: extract action via Tools
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _make_extraction_llm(structured_response: dict | None = None, freetext_response: str = 'free text result') -> BaseChatModel:
|
||||
"""Create a mock LLM that handles both structured and freetext extraction calls."""
|
||||
llm = AsyncMock(spec=BaseChatModel)
|
||||
llm.model = 'mock-extraction-llm'
|
||||
llm._verified_api_keys = True
|
||||
llm.provider = 'mock'
|
||||
llm.name = 'mock-extraction-llm'
|
||||
llm.model_name = 'mock-extraction-llm'
|
||||
|
||||
async def mock_ainvoke(messages, output_format=None, **kwargs):
|
||||
if output_format is not None and structured_response is not None:
|
||||
# Structured path: parse the dict through the model
|
||||
instance = output_format.model_validate(structured_response)
|
||||
return ChatInvokeCompletion(completion=instance, usage=None)
|
||||
# Freetext path
|
||||
return ChatInvokeCompletion(completion=freetext_response, usage=None)
|
||||
|
||||
llm.ainvoke.side_effect = mock_ainvoke
|
||||
return llm
|
||||
|
||||
|
||||
@pytest.fixture(scope='module')
|
||||
async def browser_session():
|
||||
session = BrowserSession(browser_profile=BrowserProfile(headless=True, user_data_dir=None, keep_alive=True))
|
||||
await session.start()
|
||||
yield session
|
||||
await session.kill()
|
||||
await session.event_bus.stop(clear=True, timeout=5)
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def http_server():
|
||||
server = HTTPServer()
|
||||
server.start()
|
||||
server.expect_request('/products').respond_with_data(
|
||||
"""<html><body>
|
||||
<h1>Products</h1>
|
||||
<ul>
|
||||
<li>Widget A - $9.99</li>
|
||||
<li>Widget B - $19.99</li>
|
||||
</ul>
|
||||
</body></html>""",
|
||||
content_type='text/html',
|
||||
)
|
||||
yield server
|
||||
server.stop()
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def base_url(http_server):
|
||||
return f'http://{http_server.host}:{http_server.port}'
|
||||
|
||||
|
||||
class TestExtractStructured:
|
||||
"""Integration tests for the extract action's structured extraction path."""
|
||||
|
||||
async def test_structured_extraction_returns_json(self, browser_session, base_url):
|
||||
"""When output_schema is provided, extract returns structured JSON in <structured_result> tags."""
|
||||
tools = Tools()
|
||||
await tools.navigate(url=f'{base_url}/products', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
output_schema = {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'products': {
|
||||
'type': 'array',
|
||||
'items': {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'name': {'type': 'string'},
|
||||
'price': {'type': 'number'},
|
||||
},
|
||||
'required': ['name', 'price'],
|
||||
},
|
||||
},
|
||||
},
|
||||
'required': ['products'],
|
||||
}
|
||||
|
||||
mock_data = {'products': [{'name': 'Widget A', 'price': 9.99}, {'name': 'Widget B', 'price': 19.99}]}
|
||||
extraction_llm = _make_extraction_llm(structured_response=mock_data)
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
fs = FileSystem(tmp)
|
||||
result = await tools.extract(
|
||||
query='List all products with prices',
|
||||
output_schema=output_schema,
|
||||
browser_session=browser_session,
|
||||
page_extraction_llm=extraction_llm,
|
||||
file_system=fs,
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert '<structured_result>' in result.extracted_content
|
||||
assert '</structured_result>' in result.extracted_content
|
||||
|
||||
# Parse the JSON out of the tags
|
||||
start = result.extracted_content.index('<structured_result>') + len('<structured_result>')
|
||||
end = result.extracted_content.index('</structured_result>')
|
||||
parsed = json.loads(result.extracted_content[start:end].strip())
|
||||
assert parsed == mock_data
|
||||
|
||||
# Metadata
|
||||
assert result.metadata is not None
|
||||
assert result.metadata['structured_extraction'] is True
|
||||
meta = result.metadata['extraction_result']
|
||||
assert meta['data'] == mock_data
|
||||
assert meta['schema_used'] == output_schema
|
||||
|
||||
async def test_freetext_extraction_unchanged(self, browser_session, base_url):
|
||||
"""When output_schema is None, extract returns free-text in <result> tags (backward compat)."""
|
||||
tools = Tools()
|
||||
await tools.navigate(url=f'{base_url}/products', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
extraction_llm = _make_extraction_llm(freetext_response='Widget A costs $9.99, Widget B costs $19.99')
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
fs = FileSystem(tmp)
|
||||
result = await tools.extract(
|
||||
query='What products are listed?',
|
||||
browser_session=browser_session,
|
||||
page_extraction_llm=extraction_llm,
|
||||
file_system=fs,
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert '<result>' in result.extracted_content
|
||||
assert '</result>' in result.extracted_content
|
||||
assert '<structured_result>' not in result.extracted_content
|
||||
assert result.metadata is None
|
||||
|
||||
async def test_invalid_schema_falls_back_to_freetext(self, browser_session, base_url):
|
||||
"""When output_schema contains unsupported keywords, fall back to free-text gracefully."""
|
||||
tools = Tools()
|
||||
await tools.navigate(url=f'{base_url}/products', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
bad_schema = {
|
||||
'type': 'object',
|
||||
'properties': {'item': {'$ref': '#/$defs/Item'}},
|
||||
'$defs': {'Item': {'type': 'object', 'properties': {'name': {'type': 'string'}}}},
|
||||
}
|
||||
|
||||
extraction_llm = _make_extraction_llm(freetext_response='fallback text')
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
fs = FileSystem(tmp)
|
||||
result = await tools.extract(
|
||||
query='Get products',
|
||||
output_schema=bad_schema,
|
||||
browser_session=browser_session,
|
||||
page_extraction_llm=extraction_llm,
|
||||
file_system=fs,
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
# Should have used the free-text path
|
||||
assert '<result>' in result.extracted_content
|
||||
assert '<structured_result>' not in result.extracted_content
|
||||
assert result.metadata is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Integration tests: extraction_schema injection via special parameter
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
PRODUCT_SCHEMA = {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'products': {
|
||||
'type': 'array',
|
||||
'items': {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'name': {'type': 'string'},
|
||||
'price': {'type': 'number'},
|
||||
},
|
||||
'required': ['name', 'price'],
|
||||
},
|
||||
},
|
||||
},
|
||||
'required': ['products'],
|
||||
}
|
||||
|
||||
MOCK_PRODUCTS = {'products': [{'name': 'Widget A', 'price': 9.99}, {'name': 'Widget B', 'price': 19.99}]}
|
||||
|
||||
|
||||
class TestExtractionSchemaInjection:
|
||||
"""Tests that extraction_schema injected as a special parameter triggers structured extraction."""
|
||||
|
||||
async def test_injected_extraction_schema_triggers_structured_path(self, browser_session, base_url):
|
||||
"""extraction_schema passed via act() triggers structured extraction even without output_schema in params."""
|
||||
tools = Tools()
|
||||
await tools.navigate(url=f'{base_url}/products', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
extraction_llm = _make_extraction_llm(structured_response=MOCK_PRODUCTS)
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
fs = FileSystem(tmp)
|
||||
result = await tools.extract(
|
||||
query='List all products with prices',
|
||||
browser_session=browser_session,
|
||||
page_extraction_llm=extraction_llm,
|
||||
file_system=fs,
|
||||
extraction_schema=PRODUCT_SCHEMA,
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert '<structured_result>' in result.extracted_content
|
||||
|
||||
# Parse and verify JSON
|
||||
start = result.extracted_content.index('<structured_result>') + len('<structured_result>')
|
||||
end = result.extracted_content.index('</structured_result>')
|
||||
parsed = json.loads(result.extracted_content[start:end].strip())
|
||||
assert parsed == MOCK_PRODUCTS
|
||||
|
||||
assert result.metadata is not None
|
||||
assert result.metadata['structured_extraction'] is True
|
||||
|
||||
async def test_output_schema_takes_precedence_over_extraction_schema(self, browser_session, base_url):
|
||||
"""When the LLM provides output_schema in params, it should take precedence over injected extraction_schema."""
|
||||
tools = Tools()
|
||||
await tools.navigate(url=f'{base_url}/products', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
# Different schema than the injected one — just a name list
|
||||
param_schema = {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'names': {'type': 'array', 'items': {'type': 'string'}},
|
||||
},
|
||||
'required': ['names'],
|
||||
}
|
||||
param_response = {'names': ['Widget A', 'Widget B']}
|
||||
extraction_llm = _make_extraction_llm(structured_response=param_response)
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
fs = FileSystem(tmp)
|
||||
result = await tools.extract(
|
||||
query='List product names',
|
||||
output_schema=param_schema,
|
||||
browser_session=browser_session,
|
||||
page_extraction_llm=extraction_llm,
|
||||
file_system=fs,
|
||||
extraction_schema=PRODUCT_SCHEMA, # should be ignored
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert '<structured_result>' in result.extracted_content
|
||||
|
||||
start = result.extracted_content.index('<structured_result>') + len('<structured_result>')
|
||||
end = result.extracted_content.index('</structured_result>')
|
||||
parsed = json.loads(result.extracted_content[start:end].strip())
|
||||
# Should match param_schema response, NOT PRODUCT_SCHEMA
|
||||
assert parsed == param_response
|
||||
assert result.metadata is not None
|
||||
assert result.metadata['extraction_result']['schema_used'] == param_schema
|
||||
|
||||
async def test_no_schema_uses_freetext_path(self, browser_session, base_url):
|
||||
"""When neither output_schema nor extraction_schema is provided, free-text path is used (backward compat)."""
|
||||
tools = Tools()
|
||||
await tools.navigate(url=f'{base_url}/products', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
extraction_llm = _make_extraction_llm(freetext_response='Widget A costs $9.99')
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
fs = FileSystem(tmp)
|
||||
result = await tools.extract(
|
||||
query='What products are listed?',
|
||||
browser_session=browser_session,
|
||||
page_extraction_llm=extraction_llm,
|
||||
file_system=fs,
|
||||
# No extraction_schema, no output_schema
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert '<result>' in result.extracted_content
|
||||
assert '<structured_result>' not in result.extracted_content
|
||||
assert result.metadata is None
|
||||
|
||||
async def test_extraction_schema_threads_through_act(self, browser_session, base_url):
|
||||
"""extraction_schema passed to act() reaches extract() via the registry's special parameter injection."""
|
||||
tools = Tools()
|
||||
await tools.navigate(url=f'{base_url}/products', new_tab=False, browser_session=browser_session)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
extraction_llm = _make_extraction_llm(structured_response=MOCK_PRODUCTS)
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
fs = FileSystem(tmp)
|
||||
|
||||
# Build an ActionModel for the extract action
|
||||
action_model = tools.registry.create_action_model()
|
||||
action = action_model.model_validate({'extract': {'query': 'List products'}})
|
||||
|
||||
result = await tools.act(
|
||||
action=action,
|
||||
browser_session=browser_session,
|
||||
page_extraction_llm=extraction_llm,
|
||||
file_system=fs,
|
||||
extraction_schema=PRODUCT_SCHEMA,
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert '<structured_result>' in result.extracted_content
|
||||
@@ -0,0 +1,668 @@
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
import anyio
|
||||
import pytest
|
||||
from pydantic import BaseModel, Field
|
||||
from pytest_httpserver import HTTPServer
|
||||
|
||||
from browser_use.agent.views import ActionResult
|
||||
from browser_use.browser import BrowserSession
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
from browser_use.filesystem.file_system import FileSystem
|
||||
from browser_use.tools.service import Tools
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def http_server():
|
||||
"""Create and provide a test HTTP server that serves static content."""
|
||||
server = HTTPServer()
|
||||
server.start()
|
||||
|
||||
# Add routes for common test pages
|
||||
server.expect_request('/').respond_with_data(
|
||||
'<html><head><title>Test Home Page</title></head><body><h1>Test Home Page</h1><p>Welcome to the test site</p></body></html>',
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
server.expect_request('/page1').respond_with_data(
|
||||
'<html><head><title>Test Page 1</title></head><body><h1>Test Page 1</h1><p>This is test page 1</p></body></html>',
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
server.expect_request('/page2').respond_with_data(
|
||||
'<html><head><title>Test Page 2</title></head><body><h1>Test Page 2</h1><p>This is test page 2</p></body></html>',
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
server.expect_request('/search').respond_with_data(
|
||||
"""
|
||||
<html>
|
||||
<head><title>Search Results</title></head>
|
||||
<body>
|
||||
<h1>Search Results</h1>
|
||||
<div class="results">
|
||||
<div class="result">Result 1</div>
|
||||
<div class="result">Result 2</div>
|
||||
<div class="result">Result 3</div>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
yield server
|
||||
server.stop()
|
||||
|
||||
|
||||
@pytest.fixture(scope='session')
|
||||
def base_url(http_server):
|
||||
"""Return the base URL for the test HTTP server."""
|
||||
return f'http://{http_server.host}:{http_server.port}'
|
||||
|
||||
|
||||
@pytest.fixture(scope='module')
|
||||
async def browser_session():
|
||||
"""Create and provide a Browser instance with security disabled."""
|
||||
browser_session = BrowserSession(
|
||||
browser_profile=BrowserProfile(
|
||||
headless=True,
|
||||
user_data_dir=None,
|
||||
keep_alive=True,
|
||||
)
|
||||
)
|
||||
await browser_session.start()
|
||||
yield browser_session
|
||||
await browser_session.kill()
|
||||
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
def tools():
|
||||
"""Create and provide a Tools instance."""
|
||||
return Tools()
|
||||
|
||||
|
||||
class TestToolsIntegration:
|
||||
"""Integration tests for Tools using actual browser instances."""
|
||||
|
||||
async def test_registry_actions(self, tools, browser_session):
|
||||
"""Test that the registry contains the expected default actions."""
|
||||
# Check that common actions are registered
|
||||
common_actions = [
|
||||
'navigate',
|
||||
'search',
|
||||
'click',
|
||||
'input',
|
||||
'scroll',
|
||||
'go_back',
|
||||
'switch',
|
||||
'close',
|
||||
'wait',
|
||||
]
|
||||
|
||||
for action in common_actions:
|
||||
assert action in tools.registry.registry.actions
|
||||
assert tools.registry.registry.actions[action].function is not None
|
||||
assert tools.registry.registry.actions[action].description is not None
|
||||
|
||||
async def test_custom_action_registration(self, tools, browser_session, base_url):
|
||||
"""Test registering a custom action and executing it."""
|
||||
|
||||
# Define a custom action
|
||||
class CustomParams(BaseModel):
|
||||
text: str
|
||||
|
||||
@tools.action('Test custom action', param_model=CustomParams)
|
||||
async def custom_action(params: CustomParams, browser_session):
|
||||
current_url = await browser_session.get_current_page_url()
|
||||
return ActionResult(extracted_content=f'Custom action executed with: {params.text} on {current_url}')
|
||||
|
||||
# Navigate to a page first
|
||||
await tools.navigate(url=f'{base_url}/page1', new_tab=False, browser_session=browser_session)
|
||||
|
||||
# Execute the custom action directly
|
||||
result = await tools.custom_action(text='test_value', browser_session=browser_session)
|
||||
|
||||
# Verify the result
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'Custom action executed with: test_value on' in result.extracted_content
|
||||
assert f'{base_url}/page1' in result.extracted_content
|
||||
|
||||
async def test_wait_action(self, tools, browser_session):
|
||||
"""Test that the wait action correctly waits for the specified duration."""
|
||||
|
||||
# verify that it's in the default action set
|
||||
wait_action = None
|
||||
for action_name, action in tools.registry.registry.actions.items():
|
||||
if 'wait' in action_name.lower() and 'seconds' in str(action.param_model.model_fields):
|
||||
wait_action = action
|
||||
break
|
||||
assert wait_action is not None, 'Could not find wait action in tools'
|
||||
|
||||
# Check that it has seconds parameter with default
|
||||
assert 'seconds' in wait_action.param_model.model_fields
|
||||
schema = wait_action.param_model.model_json_schema()
|
||||
assert schema['properties']['seconds']['default'] == 3
|
||||
|
||||
# Record start time
|
||||
start_time = time.time()
|
||||
|
||||
# Execute wait action
|
||||
result = await tools.wait(seconds=3, browser_session=browser_session)
|
||||
|
||||
# Record end time
|
||||
end_time = time.time()
|
||||
|
||||
# Verify the result
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'Waited for' in result.extracted_content or 'Waiting for' in result.extracted_content
|
||||
|
||||
# Verify that approximately 1 second has passed (allowing some margin)
|
||||
assert end_time - start_time <= 2.5 # We wait 3-1 seconds for LLM call
|
||||
|
||||
# longer wait
|
||||
# Record start time
|
||||
start_time = time.time()
|
||||
|
||||
# Execute wait action
|
||||
result = await tools.wait(seconds=5, browser_session=browser_session)
|
||||
|
||||
# Record end time
|
||||
end_time = time.time()
|
||||
|
||||
# Verify the result
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'Waited for' in result.extracted_content or 'Waiting for' in result.extracted_content
|
||||
|
||||
assert 3.5 <= end_time - start_time <= 4.5 # We wait 5-1 seconds for LLM call
|
||||
|
||||
async def test_go_back_action(self, tools, browser_session, base_url):
|
||||
"""Test that go_back action navigates to the previous page."""
|
||||
# Navigate to first page
|
||||
await tools.navigate(url=f'{base_url}/page1', new_tab=False, browser_session=browser_session)
|
||||
|
||||
# Store the first page URL
|
||||
first_url = await browser_session.get_current_page_url()
|
||||
print(f'First page URL: {first_url}')
|
||||
|
||||
# Navigate to second page
|
||||
await tools.navigate(url=f'{base_url}/page2', new_tab=False, browser_session=browser_session)
|
||||
|
||||
# Verify we're on the second page
|
||||
second_url = await browser_session.get_current_page_url()
|
||||
print(f'Second page URL: {second_url}')
|
||||
assert f'{base_url}/page2' in second_url
|
||||
|
||||
# Execute go back action
|
||||
result = await tools.go_back(browser_session=browser_session)
|
||||
|
||||
# Verify the result
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'Navigated back' in result.extracted_content
|
||||
|
||||
# Add another delay to allow the navigation to complete
|
||||
await asyncio.sleep(1)
|
||||
|
||||
# Verify we're back on a different page than before
|
||||
final_url = await browser_session.get_current_page_url()
|
||||
print(f'Final page URL after going back: {final_url}')
|
||||
|
||||
# Try to verify we're back on the first page, but don't fail the test if not
|
||||
assert f'{base_url}/page1' in final_url, f'Expected to return to page1 but got {final_url}'
|
||||
|
||||
async def test_navigation_chain(self, tools, browser_session, base_url):
|
||||
"""Test navigating through multiple pages and back through history."""
|
||||
# Set up a chain of navigation: Home -> Page1 -> Page2
|
||||
urls = [f'{base_url}/', f'{base_url}/page1', f'{base_url}/page2']
|
||||
|
||||
# Navigate to each page in sequence
|
||||
for url in urls:
|
||||
await tools.navigate(url=url, new_tab=False, browser_session=browser_session)
|
||||
|
||||
# Verify current page
|
||||
current_url = await browser_session.get_current_page_url()
|
||||
assert url in current_url
|
||||
|
||||
# Go back twice and verify each step
|
||||
for expected_url in reversed(urls[:-1]):
|
||||
await tools.go_back(browser_session=browser_session)
|
||||
await asyncio.sleep(1) # Wait for navigation to complete
|
||||
|
||||
current_url = await browser_session.get_current_page_url()
|
||||
assert expected_url in current_url
|
||||
|
||||
async def test_excluded_actions(self, browser_session):
|
||||
"""Test that excluded actions are not registered."""
|
||||
# Create tools with excluded actions
|
||||
excluded_tools = Tools(exclude_actions=['search', 'scroll'])
|
||||
|
||||
# Verify excluded actions are not in the registry
|
||||
assert 'search' not in excluded_tools.registry.registry.actions
|
||||
assert 'scroll' not in excluded_tools.registry.registry.actions
|
||||
|
||||
# But other actions are still there
|
||||
assert 'navigate' in excluded_tools.registry.registry.actions
|
||||
assert 'click' in excluded_tools.registry.registry.actions
|
||||
|
||||
async def test_search_action(self, tools, browser_session, base_url):
|
||||
"""Test the search action."""
|
||||
|
||||
await browser_session.get_current_page_url()
|
||||
|
||||
# Execute search action - it will actually navigate to our search results page
|
||||
result = await tools.search(query='Python web automation', browser_session=browser_session)
|
||||
|
||||
# Verify the result
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert 'Searched' in result.extracted_content and 'Python web automation' in result.extracted_content
|
||||
|
||||
# For our test purposes, we just verify we're on some URL
|
||||
current_url = await browser_session.get_current_page_url()
|
||||
assert current_url is not None and 'Python' in current_url
|
||||
|
||||
async def test_done_action(self, tools, browser_session, base_url):
|
||||
"""Test that DoneAction completes a task and reports success or failure."""
|
||||
# Create a temporary directory for the file system
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
file_system = FileSystem(temp_dir)
|
||||
|
||||
# First navigate to a page
|
||||
await tools.navigate(url=f'{base_url}/page1', new_tab=False, browser_session=browser_session)
|
||||
|
||||
success_done_message = 'Successfully completed task'
|
||||
|
||||
# Execute done action with file_system
|
||||
result = await tools.done(
|
||||
text=success_done_message, success=True, browser_session=browser_session, file_system=file_system
|
||||
)
|
||||
|
||||
# Verify the result
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert success_done_message in result.extracted_content
|
||||
assert result.success is True
|
||||
assert result.is_done is True
|
||||
assert result.error is None
|
||||
|
||||
failed_done_message = 'Failed to complete task'
|
||||
|
||||
# Execute failed done action with file_system
|
||||
result = await tools.done(
|
||||
text=failed_done_message, success=False, browser_session=browser_session, file_system=file_system
|
||||
)
|
||||
|
||||
# Verify the result
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.extracted_content is not None
|
||||
assert failed_done_message in result.extracted_content
|
||||
assert result.success is False
|
||||
assert result.is_done is True
|
||||
assert result.error is None
|
||||
|
||||
async def test_get_dropdown_options(self, tools, browser_session, base_url, http_server):
|
||||
"""Test that get_dropdown_options correctly retrieves options from a dropdown."""
|
||||
# Add route for dropdown test page
|
||||
http_server.expect_request('/dropdown1').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Dropdown Test</title>
|
||||
</head>
|
||||
<body>
|
||||
<h1>Dropdown Test</h1>
|
||||
<select id="test-dropdown" name="test-dropdown">
|
||||
<option value="">Please select</option>
|
||||
<option value="option1">First Option</option>
|
||||
<option value="option2">Second Option</option>
|
||||
<option value="option3">Third Option</option>
|
||||
</select>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
# Navigate to the dropdown test page
|
||||
await tools.navigate(url=f'{base_url}/dropdown1', new_tab=False, browser_session=browser_session)
|
||||
|
||||
# Wait for the page to load using CDP
|
||||
cdp_session = await browser_session.get_or_create_cdp_session()
|
||||
assert cdp_session is not None, 'CDP session not initialized'
|
||||
|
||||
# Wait for page load by checking document ready state
|
||||
await asyncio.sleep(0.5) # Brief wait for navigation to start
|
||||
ready_state = await cdp_session.cdp_client.send.Runtime.evaluate(
|
||||
params={'expression': 'document.readyState'}, session_id=cdp_session.session_id
|
||||
)
|
||||
# If not complete, wait a bit more
|
||||
if ready_state.get('result', {}).get('value') != 'complete':
|
||||
await asyncio.sleep(1.0)
|
||||
|
||||
# Initialize the DOM state to populate the selector map
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
# Get the selector map
|
||||
selector_map = await browser_session.get_selector_map()
|
||||
|
||||
# Find the dropdown element in the selector map
|
||||
dropdown_index = None
|
||||
for idx, element in selector_map.items():
|
||||
if element.tag_name.lower() == 'select':
|
||||
dropdown_index = idx
|
||||
break
|
||||
|
||||
assert dropdown_index is not None, (
|
||||
f'Could not find select element in selector map. Available elements: {[f"{idx}: {element.tag_name}" for idx, element in selector_map.items()]}'
|
||||
)
|
||||
|
||||
# Execute the action with the dropdown index
|
||||
result = await tools.dropdown_options(index=dropdown_index, browser_session=browser_session)
|
||||
|
||||
expected_options = [
|
||||
{'index': 0, 'text': 'Please select', 'value': ''},
|
||||
{'index': 1, 'text': 'First Option', 'value': 'option1'},
|
||||
{'index': 2, 'text': 'Second Option', 'value': 'option2'},
|
||||
{'index': 3, 'text': 'Third Option', 'value': 'option3'},
|
||||
]
|
||||
|
||||
# Verify the result structure
|
||||
assert isinstance(result, ActionResult)
|
||||
|
||||
# Core logic validation: Verify all options are returned
|
||||
assert result.extracted_content is not None
|
||||
for option in expected_options[1:]: # Skip the placeholder option
|
||||
assert option['text'] in result.extracted_content, f"Option '{option['text']}' not found in result content"
|
||||
|
||||
# Verify the instruction for using the text in select_dropdown is included
|
||||
assert 'Use the exact text or value string' in result.extracted_content and 'select_dropdown' in result.extracted_content
|
||||
|
||||
# Verify the actual dropdown options in the DOM using CDP
|
||||
dropdown_options_result = await cdp_session.cdp_client.send.Runtime.evaluate(
|
||||
params={
|
||||
'expression': """
|
||||
JSON.stringify((() => {
|
||||
const select = document.getElementById('test-dropdown');
|
||||
return Array.from(select.options).map(opt => ({
|
||||
text: opt.text,
|
||||
value: opt.value
|
||||
}));
|
||||
})())
|
||||
""",
|
||||
'returnByValue': True,
|
||||
},
|
||||
session_id=cdp_session.session_id,
|
||||
)
|
||||
dropdown_options_json = dropdown_options_result.get('result', {}).get('value', '[]')
|
||||
import json
|
||||
|
||||
dropdown_options = json.loads(dropdown_options_json) if isinstance(dropdown_options_json, str) else dropdown_options_json
|
||||
|
||||
# Verify the dropdown has the expected options
|
||||
assert len(dropdown_options) == len(expected_options), (
|
||||
f'Expected {len(expected_options)} options, got {len(dropdown_options)}'
|
||||
)
|
||||
for i, expected in enumerate(expected_options):
|
||||
actual = dropdown_options[i]
|
||||
assert actual['text'] == expected['text'], (
|
||||
f"Option at index {i} has wrong text: expected '{expected['text']}', got '{actual['text']}'"
|
||||
)
|
||||
assert actual['value'] == expected['value'], (
|
||||
f"Option at index {i} has wrong value: expected '{expected['value']}', got '{actual['value']}'"
|
||||
)
|
||||
|
||||
async def test_select_dropdown_option(self, tools, browser_session, base_url, http_server):
|
||||
"""Test that select_dropdown_option correctly selects an option from a dropdown."""
|
||||
# Add route for dropdown test page
|
||||
http_server.expect_request('/dropdown2').respond_with_data(
|
||||
"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Dropdown Test</title>
|
||||
</head>
|
||||
<body>
|
||||
<h1>Dropdown Test</h1>
|
||||
<select id="test-dropdown" name="test-dropdown">
|
||||
<option value="">Please select</option>
|
||||
<option value="option1">First Option</option>
|
||||
<option value="option2">Second Option</option>
|
||||
<option value="option3">Third Option</option>
|
||||
</select>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
content_type='text/html',
|
||||
)
|
||||
|
||||
# Navigate to the dropdown test page
|
||||
await tools.navigate(url=f'{base_url}/dropdown2', new_tab=False, browser_session=browser_session)
|
||||
|
||||
# Wait for the page to load using CDP
|
||||
cdp_session = await browser_session.get_or_create_cdp_session()
|
||||
assert cdp_session is not None, 'CDP session not initialized'
|
||||
|
||||
# Wait for page load by checking document ready state
|
||||
await asyncio.sleep(0.5) # Brief wait for navigation to start
|
||||
ready_state = await cdp_session.cdp_client.send.Runtime.evaluate(
|
||||
params={'expression': 'document.readyState'}, session_id=cdp_session.session_id
|
||||
)
|
||||
# If not complete, wait a bit more
|
||||
if ready_state.get('result', {}).get('value') != 'complete':
|
||||
await asyncio.sleep(1.0)
|
||||
|
||||
# populate the selector map with highlight indices
|
||||
await browser_session.get_browser_state_summary()
|
||||
|
||||
# Now get the selector map which should contain our dropdown
|
||||
selector_map = await browser_session.get_selector_map()
|
||||
|
||||
# Find the dropdown element in the selector map
|
||||
dropdown_index = None
|
||||
for idx, element in selector_map.items():
|
||||
if element.tag_name.lower() == 'select':
|
||||
dropdown_index = idx
|
||||
break
|
||||
|
||||
assert dropdown_index is not None, (
|
||||
f'Could not find select element in selector map. Available elements: {[f"{idx}: {element.tag_name}" for idx, element in selector_map.items()]}'
|
||||
)
|
||||
|
||||
# Execute the action with the dropdown index
|
||||
result = await tools.select_dropdown(index=dropdown_index, text='Second Option', browser_session=browser_session)
|
||||
|
||||
# Verify the result structure
|
||||
assert isinstance(result, ActionResult)
|
||||
|
||||
# Core logic validation: Verify selection was successful
|
||||
assert result.extracted_content is not None
|
||||
assert 'selected option' in result.extracted_content.lower()
|
||||
assert 'Second Option' in result.extracted_content
|
||||
|
||||
# Verify the actual dropdown selection was made by checking the DOM using CDP
|
||||
selected_value_result = await cdp_session.cdp_client.send.Runtime.evaluate(
|
||||
params={'expression': "document.getElementById('test-dropdown').value"}, session_id=cdp_session.session_id
|
||||
)
|
||||
selected_value = selected_value_result.get('result', {}).get('value')
|
||||
assert selected_value == 'option2' # Second Option has value "option2"
|
||||
|
||||
|
||||
class TestStructuredOutputDoneWithFiles:
|
||||
"""Tests for file handling in structured output done action."""
|
||||
|
||||
async def test_structured_output_done_without_files(self, browser_session, base_url):
|
||||
"""Structured output done action works without files (backward compat)."""
|
||||
|
||||
class MyOutput(BaseModel):
|
||||
answer: str = Field(description='The answer')
|
||||
|
||||
tools = Tools(output_model=MyOutput)
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
file_system = FileSystem(temp_dir)
|
||||
|
||||
result = await tools.done(
|
||||
data={'answer': 'hello'},
|
||||
success=True,
|
||||
browser_session=browser_session,
|
||||
file_system=file_system,
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.is_done is True
|
||||
assert result.success is True
|
||||
assert result.extracted_content is not None
|
||||
output = json.loads(result.extracted_content)
|
||||
assert output == {'answer': 'hello'}
|
||||
assert result.attachments == []
|
||||
|
||||
async def test_structured_output_done_with_files_to_display(self, browser_session, base_url):
|
||||
"""Structured output done action resolves files_to_display into attachments."""
|
||||
|
||||
class MyOutput(BaseModel):
|
||||
summary: str
|
||||
|
||||
tools = Tools(output_model=MyOutput)
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
file_system = FileSystem(temp_dir)
|
||||
await file_system.write_file('report.txt', 'some report content')
|
||||
|
||||
result = await tools.done(
|
||||
data={'summary': 'done'},
|
||||
success=True,
|
||||
files_to_display=['report.txt'],
|
||||
browser_session=browser_session,
|
||||
file_system=file_system,
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.is_done is True
|
||||
assert result.success is True
|
||||
assert result.extracted_content is not None
|
||||
output = json.loads(result.extracted_content)
|
||||
assert output == {'summary': 'done'}
|
||||
assert result.attachments is not None
|
||||
assert len(result.attachments) == 1
|
||||
assert result.attachments[0].endswith('report.txt')
|
||||
|
||||
async def test_structured_output_done_auto_attaches_downloads(self, browser_session, base_url):
|
||||
"""Session downloads are auto-attached even without files_to_display."""
|
||||
|
||||
class MyOutput(BaseModel):
|
||||
url: str
|
||||
|
||||
tools = Tools(output_model=MyOutput)
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
file_system = FileSystem(temp_dir)
|
||||
|
||||
# Simulate a CDP-tracked browser download
|
||||
fake_download = os.path.join(temp_dir, 'tax-bill.pdf')
|
||||
await anyio.Path(fake_download).write_bytes(b'%PDF-1.4 fake pdf content')
|
||||
|
||||
saved_downloads = browser_session._downloaded_files.copy()
|
||||
browser_session._downloaded_files.append(fake_download)
|
||||
try:
|
||||
result = await tools.done(
|
||||
data={'url': f'{base_url}/bill.pdf'},
|
||||
success=True,
|
||||
browser_session=browser_session,
|
||||
file_system=file_system,
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.is_done is True
|
||||
assert result.extracted_content is not None
|
||||
output = json.loads(result.extracted_content)
|
||||
assert output == {'url': f'{base_url}/bill.pdf'}
|
||||
# The download should be auto-attached
|
||||
assert result.attachments is not None
|
||||
assert len(result.attachments) == 1
|
||||
assert result.attachments[0] == fake_download
|
||||
finally:
|
||||
browser_session._downloaded_files = saved_downloads
|
||||
|
||||
async def test_structured_output_done_deduplicates_attachments(self, browser_session):
|
||||
"""Downloads already covered by files_to_display are not duplicated."""
|
||||
|
||||
class MyOutput(BaseModel):
|
||||
status: str
|
||||
|
||||
tools = Tools(output_model=MyOutput)
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
file_system = FileSystem(temp_dir)
|
||||
await file_system.write_file('report.txt', 'content here')
|
||||
|
||||
# The same file appears in both files_to_display and session downloads
|
||||
fs_path = str(file_system.get_dir() / 'report.txt')
|
||||
|
||||
saved_downloads = browser_session._downloaded_files.copy()
|
||||
browser_session._downloaded_files.append(fs_path)
|
||||
try:
|
||||
result = await tools.done(
|
||||
data={'status': 'ok'},
|
||||
success=True,
|
||||
files_to_display=['report.txt'],
|
||||
browser_session=browser_session,
|
||||
file_system=file_system,
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
# Should have exactly 1 attachment, not 2
|
||||
assert result.attachments is not None
|
||||
assert len(result.attachments) == 1
|
||||
assert result.attachments[0] == fs_path
|
||||
finally:
|
||||
browser_session._downloaded_files = saved_downloads
|
||||
|
||||
async def test_structured_output_done_nonexistent_file_ignored(self, browser_session):
|
||||
"""Files that don't exist in FileSystem are not included via files_to_display."""
|
||||
|
||||
class MyOutput(BaseModel):
|
||||
value: int
|
||||
|
||||
tools = Tools(output_model=MyOutput)
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
file_system = FileSystem(temp_dir)
|
||||
|
||||
result = await tools.done(
|
||||
data={'value': 42},
|
||||
success=True,
|
||||
files_to_display=['nonexistent.txt'],
|
||||
browser_session=browser_session,
|
||||
file_system=file_system,
|
||||
)
|
||||
|
||||
assert isinstance(result, ActionResult)
|
||||
assert result.is_done is True
|
||||
assert result.extracted_content is not None
|
||||
output = json.loads(result.extracted_content)
|
||||
assert output == {'value': 42}
|
||||
# nonexistent file should not appear in attachments
|
||||
assert result.attachments == []
|
||||
|
||||
async def test_structured_output_schema_hides_internal_fields(self):
|
||||
"""The JSON schema for StructuredOutputAction hides success and files_to_display."""
|
||||
from browser_use.tools.views import StructuredOutputAction
|
||||
|
||||
class MyOutput(BaseModel):
|
||||
name: str
|
||||
|
||||
schema = StructuredOutputAction[MyOutput].model_json_schema()
|
||||
top_level_props = schema.get('properties', {})
|
||||
assert 'success' not in top_level_props
|
||||
assert 'files_to_display' not in top_level_props
|
||||
# data should still be present
|
||||
assert 'data' in top_level_props
|
||||
@@ -0,0 +1,408 @@
|
||||
"""Unit tests for variable detection in agent history"""
|
||||
|
||||
from browser_use.agent.variable_detector import (
|
||||
_detect_from_attributes,
|
||||
_detect_from_value_pattern,
|
||||
_detect_variable_type,
|
||||
_ensure_unique_name,
|
||||
detect_variables_in_history,
|
||||
)
|
||||
from browser_use.agent.views import DetectedVariable
|
||||
from browser_use.dom.views import DOMInteractedElement, NodeType
|
||||
|
||||
|
||||
def create_test_element(attributes: dict[str, str] | None = None) -> DOMInteractedElement:
|
||||
"""Helper to create a DOMInteractedElement for testing"""
|
||||
return DOMInteractedElement(
|
||||
node_id=1,
|
||||
backend_node_id=1,
|
||||
frame_id='frame1',
|
||||
node_type=NodeType.ELEMENT_NODE,
|
||||
node_value='',
|
||||
node_name='input',
|
||||
attributes=attributes or {},
|
||||
bounds=None,
|
||||
x_path='//*[@id="test"]',
|
||||
element_hash=12345,
|
||||
)
|
||||
|
||||
|
||||
def create_mock_history(actions_with_elements: list[tuple[dict, DOMInteractedElement | None]]):
|
||||
"""Helper to create mock history for testing"""
|
||||
from types import SimpleNamespace
|
||||
|
||||
history_items = []
|
||||
for action_dict, element in actions_with_elements:
|
||||
mock_action = SimpleNamespace(**action_dict)
|
||||
mock_output = SimpleNamespace(action=[mock_action])
|
||||
mock_state = SimpleNamespace(interacted_element=[element] if element else None)
|
||||
mock_history_item = SimpleNamespace(model_output=mock_output, state=mock_state)
|
||||
history_items.append(mock_history_item)
|
||||
|
||||
return SimpleNamespace(history=history_items)
|
||||
|
||||
|
||||
def test_detect_email_from_attributes():
|
||||
"""Test email detection via type='email' attribute"""
|
||||
attributes = {'type': 'email', 'id': 'email-input'}
|
||||
result = _detect_from_attributes(attributes)
|
||||
|
||||
assert result is not None
|
||||
var_name, var_format = result
|
||||
assert var_name == 'email'
|
||||
assert var_format == 'email'
|
||||
|
||||
|
||||
def test_detect_email_from_pattern():
|
||||
"""Test email detection via pattern matching"""
|
||||
result = _detect_from_value_pattern('test@example.com')
|
||||
|
||||
assert result is not None
|
||||
var_name, var_format = result
|
||||
assert var_name == 'email'
|
||||
assert var_format == 'email'
|
||||
|
||||
|
||||
def test_detect_phone_from_attributes():
|
||||
"""Test phone detection via type='tel' attribute"""
|
||||
attributes = {'type': 'tel', 'name': 'phone'}
|
||||
result = _detect_from_attributes(attributes)
|
||||
|
||||
assert result is not None
|
||||
var_name, var_format = result
|
||||
assert var_name == 'phone'
|
||||
assert var_format == 'phone'
|
||||
|
||||
|
||||
def test_detect_phone_from_pattern():
|
||||
"""Test phone detection via pattern matching"""
|
||||
result = _detect_from_value_pattern('+1 (555) 123-4567')
|
||||
|
||||
assert result is not None
|
||||
var_name, var_format = result
|
||||
assert var_name == 'phone'
|
||||
assert var_format == 'phone'
|
||||
|
||||
|
||||
def test_detect_date_from_attributes():
|
||||
"""Test date detection via type='date' attribute"""
|
||||
attributes = {'type': 'date', 'id': 'dob'}
|
||||
result = _detect_from_attributes(attributes)
|
||||
|
||||
assert result is not None
|
||||
var_name, var_format = result
|
||||
assert var_name == 'date'
|
||||
assert var_format == 'date'
|
||||
|
||||
|
||||
def test_detect_date_from_pattern():
|
||||
"""Test date detection via YYYY-MM-DD pattern"""
|
||||
result = _detect_from_value_pattern('1990-01-01')
|
||||
|
||||
assert result is not None
|
||||
var_name, var_format = result
|
||||
assert var_name == 'date'
|
||||
assert var_format == 'date'
|
||||
|
||||
|
||||
def test_detect_first_name_from_attributes():
|
||||
"""Test first name detection from element attributes"""
|
||||
attributes = {'name': 'first_name', 'placeholder': 'Enter your first name'}
|
||||
result = _detect_from_attributes(attributes)
|
||||
|
||||
assert result is not None
|
||||
var_name, var_format = result
|
||||
assert var_name == 'first_name'
|
||||
assert var_format is None
|
||||
|
||||
|
||||
def test_detect_first_name_from_pattern():
|
||||
"""Test first name detection from pattern (single capitalized word)"""
|
||||
result = _detect_from_value_pattern('John')
|
||||
|
||||
assert result is not None
|
||||
var_name, var_format = result
|
||||
assert var_name == 'first_name'
|
||||
assert var_format is None
|
||||
|
||||
|
||||
def test_detect_full_name_from_pattern():
|
||||
"""Test full name detection from pattern (two capitalized words)"""
|
||||
result = _detect_from_value_pattern('John Doe')
|
||||
|
||||
assert result is not None
|
||||
var_name, var_format = result
|
||||
assert var_name == 'full_name'
|
||||
assert var_format is None
|
||||
|
||||
|
||||
def test_detect_address_from_attributes():
|
||||
"""Test address detection from element attributes"""
|
||||
attributes = {'name': 'street_address', 'id': 'address-input'}
|
||||
result = _detect_from_attributes(attributes)
|
||||
|
||||
assert result is not None
|
||||
var_name, var_format = result
|
||||
assert var_name == 'address'
|
||||
assert var_format is None
|
||||
|
||||
|
||||
def test_detect_billing_address_from_attributes():
|
||||
"""Test billing address detection from element attributes"""
|
||||
attributes = {'name': 'billing_address', 'placeholder': 'Billing street address'}
|
||||
result = _detect_from_attributes(attributes)
|
||||
|
||||
assert result is not None
|
||||
var_name, var_format = result
|
||||
assert var_name == 'billing_address'
|
||||
assert var_format is None
|
||||
|
||||
|
||||
def test_detect_comment_from_attributes():
|
||||
"""Test comment detection from element attributes"""
|
||||
attributes = {'name': 'comment', 'placeholder': 'Enter your comment'}
|
||||
result = _detect_from_attributes(attributes)
|
||||
|
||||
assert result is not None
|
||||
var_name, var_format = result
|
||||
assert var_name == 'comment'
|
||||
assert var_format is None
|
||||
|
||||
|
||||
def test_detect_city_from_attributes():
|
||||
"""Test city detection from element attributes"""
|
||||
attributes = {'name': 'city', 'id': 'city-input'}
|
||||
result = _detect_from_attributes(attributes)
|
||||
|
||||
assert result is not None
|
||||
var_name, var_format = result
|
||||
assert var_name == 'city'
|
||||
assert var_format is None
|
||||
|
||||
|
||||
def test_detect_state_from_attributes():
|
||||
"""Test state detection from element attributes"""
|
||||
attributes = {'name': 'state', 'aria-label': 'State or Province'}
|
||||
result = _detect_from_attributes(attributes)
|
||||
|
||||
assert result is not None
|
||||
var_name, var_format = result
|
||||
assert var_name == 'state'
|
||||
assert var_format is None
|
||||
|
||||
|
||||
def test_detect_country_from_attributes():
|
||||
"""Test country detection from element attributes"""
|
||||
attributes = {'name': 'country', 'id': 'country-select'}
|
||||
result = _detect_from_attributes(attributes)
|
||||
|
||||
assert result is not None
|
||||
var_name, var_format = result
|
||||
assert var_name == 'country'
|
||||
assert var_format is None
|
||||
|
||||
|
||||
def test_detect_zip_code_from_attributes():
|
||||
"""Test zip code detection from element attributes"""
|
||||
attributes = {'name': 'zip_code', 'placeholder': 'Zip or postal code'}
|
||||
result = _detect_from_attributes(attributes)
|
||||
|
||||
assert result is not None
|
||||
var_name, var_format = result
|
||||
assert var_name == 'zip_code'
|
||||
assert var_format == 'postal_code'
|
||||
|
||||
|
||||
def test_detect_company_from_attributes():
|
||||
"""Test company detection from element attributes"""
|
||||
attributes = {'name': 'company', 'id': 'company-input'}
|
||||
result = _detect_from_attributes(attributes)
|
||||
|
||||
assert result is not None
|
||||
var_name, var_format = result
|
||||
assert var_name == 'company'
|
||||
assert var_format is None
|
||||
|
||||
|
||||
def test_detect_number_from_pattern():
|
||||
"""Test number detection from pattern (pure digits)"""
|
||||
result = _detect_from_value_pattern('12345')
|
||||
|
||||
assert result is not None
|
||||
var_name, var_format = result
|
||||
assert var_name == 'number'
|
||||
assert var_format == 'number'
|
||||
|
||||
|
||||
def test_no_detection_for_random_text():
|
||||
"""Test that random text is not detected as a variable"""
|
||||
result = _detect_from_value_pattern('some random text that is not a variable')
|
||||
assert result is None
|
||||
|
||||
|
||||
def test_no_detection_for_short_text():
|
||||
"""Test that very short text is not detected"""
|
||||
result = _detect_from_value_pattern('a')
|
||||
assert result is None
|
||||
|
||||
|
||||
def test_element_attributes_take_priority_over_pattern():
|
||||
"""Test that element attributes are checked before pattern matching"""
|
||||
# A value that could match pattern (capitalized name)
|
||||
value = 'Test'
|
||||
|
||||
# Element with explicit type="email"
|
||||
element = create_test_element(attributes={'type': 'email', 'id': 'email-input'})
|
||||
|
||||
result = _detect_variable_type(value, element)
|
||||
|
||||
assert result is not None
|
||||
var_name, var_format = result
|
||||
# Should detect as email (from attributes), not first_name (from pattern)
|
||||
assert var_name == 'email'
|
||||
assert var_format == 'email'
|
||||
|
||||
|
||||
def test_pattern_matching_used_when_no_element():
|
||||
"""Test that pattern matching is used when element context is missing"""
|
||||
value = 'test@example.com'
|
||||
|
||||
result = _detect_variable_type(value, element=None)
|
||||
|
||||
assert result is not None
|
||||
var_name, var_format = result
|
||||
assert var_name == 'email'
|
||||
assert var_format == 'email'
|
||||
|
||||
|
||||
def test_ensure_unique_name_no_conflict():
|
||||
"""Test unique name generation with no conflicts"""
|
||||
existing = {}
|
||||
result = _ensure_unique_name('email', existing)
|
||||
assert result == 'email'
|
||||
|
||||
|
||||
def test_ensure_unique_name_with_conflict():
|
||||
"""Test unique name generation with conflicts"""
|
||||
existing = {
|
||||
'email': DetectedVariable(name='email', original_value='test1@example.com'),
|
||||
}
|
||||
result = _ensure_unique_name('email', existing)
|
||||
assert result == 'email_2'
|
||||
|
||||
|
||||
def test_ensure_unique_name_with_multiple_conflicts():
|
||||
"""Test unique name generation with multiple conflicts"""
|
||||
existing = {
|
||||
'email': DetectedVariable(name='email', original_value='test1@example.com'),
|
||||
'email_2': DetectedVariable(name='email_2', original_value='test2@example.com'),
|
||||
}
|
||||
result = _ensure_unique_name('email', existing)
|
||||
assert result == 'email_3'
|
||||
|
||||
|
||||
def test_detect_variables_in_empty_history():
|
||||
"""Test variable detection in empty history"""
|
||||
from types import SimpleNamespace
|
||||
|
||||
history = SimpleNamespace(history=[])
|
||||
|
||||
result = detect_variables_in_history(history) # type: ignore[arg-type]
|
||||
|
||||
assert result == {}
|
||||
|
||||
|
||||
def test_detect_variables_in_history_with_input_action():
|
||||
"""Test variable detection in history with input action"""
|
||||
# Use mock objects to avoid Pydantic validation issues
|
||||
from types import SimpleNamespace
|
||||
|
||||
# Create mock history structure
|
||||
element = create_test_element(attributes={'type': 'email', 'id': 'email-input'})
|
||||
|
||||
mock_action = SimpleNamespace(**{'input': {'index': 1, 'text': 'test@example.com'}})
|
||||
mock_output = SimpleNamespace(action=[mock_action])
|
||||
mock_state = SimpleNamespace(interacted_element=[element])
|
||||
mock_history_item = SimpleNamespace(model_output=mock_output, state=mock_state)
|
||||
mock_history = SimpleNamespace(history=[mock_history_item])
|
||||
|
||||
result = detect_variables_in_history(mock_history) # type: ignore[arg-type]
|
||||
|
||||
assert len(result) == 1
|
||||
assert 'email' in result
|
||||
assert result['email'].original_value == 'test@example.com'
|
||||
assert result['email'].format == 'email'
|
||||
|
||||
|
||||
def test_detect_variables_skips_duplicate_values():
|
||||
"""Test that duplicate values are only detected once"""
|
||||
# Create history with same value entered twice
|
||||
element = create_test_element(attributes={'type': 'email'})
|
||||
history = create_mock_history(
|
||||
[
|
||||
({'input': {'index': 1, 'text': 'test@example.com'}}, element),
|
||||
({'input': {'index': 2, 'text': 'test@example.com'}}, element),
|
||||
]
|
||||
)
|
||||
|
||||
result = detect_variables_in_history(history) # type: ignore[arg-type]
|
||||
|
||||
# Should only detect one variable, not two
|
||||
assert len(result) == 1
|
||||
assert 'email' in result
|
||||
|
||||
|
||||
def test_detect_variables_handles_missing_state():
|
||||
"""Test that detection works when state is missing"""
|
||||
from types import SimpleNamespace
|
||||
|
||||
# Create history with None state
|
||||
mock_action = SimpleNamespace(**{'input': {'index': 1, 'text': 'test@example.com'}})
|
||||
mock_output = SimpleNamespace(action=[mock_action])
|
||||
mock_history_item = SimpleNamespace(model_output=mock_output, state=None)
|
||||
history = SimpleNamespace(history=[mock_history_item])
|
||||
|
||||
result = detect_variables_in_history(history) # type: ignore[arg-type]
|
||||
|
||||
# Should still detect via pattern matching
|
||||
assert len(result) == 1
|
||||
assert 'email' in result
|
||||
assert result['email'].original_value == 'test@example.com'
|
||||
|
||||
|
||||
def test_detect_variables_handles_missing_interacted_element():
|
||||
"""Test that detection works when interacted_element is missing"""
|
||||
# Use None as element to test when interacted_element is None
|
||||
history = create_mock_history(
|
||||
[
|
||||
({'input': {'index': 1, 'text': 'test@example.com'}}, None),
|
||||
]
|
||||
)
|
||||
|
||||
result = detect_variables_in_history(history) # type: ignore[arg-type]
|
||||
|
||||
# Should still detect via pattern matching
|
||||
assert len(result) == 1
|
||||
assert 'email' in result
|
||||
|
||||
|
||||
def test_detect_variables_multiple_types():
|
||||
"""Test detection of multiple variable types in one history"""
|
||||
history = create_mock_history(
|
||||
[
|
||||
({'input': {'index': 1, 'text': 'test@example.com'}}, create_test_element(attributes={'type': 'email'})),
|
||||
({'input': {'index': 2, 'text': 'John'}}, create_test_element(attributes={'name': 'first_name'})),
|
||||
({'input': {'index': 3, 'text': '1990-01-01'}}, create_test_element(attributes={'type': 'date'})),
|
||||
]
|
||||
)
|
||||
|
||||
result = detect_variables_in_history(history) # type: ignore[arg-type]
|
||||
|
||||
assert len(result) == 3
|
||||
assert 'email' in result
|
||||
assert 'first_name' in result
|
||||
assert 'date' in result
|
||||
|
||||
assert result['email'].original_value == 'test@example.com'
|
||||
assert result['first_name'].original_value == 'John'
|
||||
assert result['date'].original_value == '1990-01-01'
|
||||
@@ -0,0 +1,272 @@
|
||||
"""Unit tests for variable substitution in agent history"""
|
||||
|
||||
from types import SimpleNamespace
|
||||
|
||||
from browser_use.agent.service import Agent
|
||||
from browser_use.dom.views import DOMInteractedElement, NodeType
|
||||
|
||||
|
||||
def create_test_element(attributes: dict[str, str] | None = None) -> DOMInteractedElement:
|
||||
"""Helper to create a DOMInteractedElement for testing"""
|
||||
return DOMInteractedElement(
|
||||
node_id=1,
|
||||
backend_node_id=1,
|
||||
frame_id='frame1',
|
||||
node_type=NodeType.ELEMENT_NODE,
|
||||
node_value='',
|
||||
node_name='input',
|
||||
attributes=attributes or {},
|
||||
bounds=None,
|
||||
x_path='//*[@id="test"]',
|
||||
element_hash=12345,
|
||||
)
|
||||
|
||||
|
||||
def create_mock_history(actions_with_elements: list[tuple[dict, DOMInteractedElement | None]]):
|
||||
"""Helper to create mock history for testing"""
|
||||
history_items = []
|
||||
for action_dict, element in actions_with_elements:
|
||||
mock_action = SimpleNamespace(**action_dict)
|
||||
mock_output = SimpleNamespace(action=[mock_action])
|
||||
mock_state = SimpleNamespace(interacted_element=[element] if element else None)
|
||||
mock_history_item = SimpleNamespace(model_output=mock_output, state=mock_state)
|
||||
history_items.append(mock_history_item)
|
||||
|
||||
return SimpleNamespace(history=history_items)
|
||||
|
||||
|
||||
def test_substitute_single_variable(mock_llm):
|
||||
"""Test substitution of a single variable"""
|
||||
agent = Agent(task='test', llm=mock_llm)
|
||||
|
||||
# Create mock history with email
|
||||
element = create_test_element(attributes={'type': 'email'})
|
||||
history = create_mock_history(
|
||||
[
|
||||
({'input': {'index': 1, 'text': 'old@example.com'}}, element),
|
||||
]
|
||||
)
|
||||
|
||||
# Substitute the email
|
||||
modified_history = agent._substitute_variables_in_history(
|
||||
history, # type: ignore[arg-type]
|
||||
{'email': 'new@example.com'},
|
||||
)
|
||||
|
||||
# Check that the value was substituted
|
||||
action = modified_history.history[0].model_output.action[0] # type: ignore[attr-defined]
|
||||
action_dict = vars(action)
|
||||
assert action_dict['input']['text'] == 'new@example.com'
|
||||
|
||||
|
||||
def test_substitute_multiple_variables(mock_llm):
|
||||
"""Test substitution of multiple variables"""
|
||||
agent = Agent(task='test', llm=mock_llm)
|
||||
|
||||
# Create mock history with email and name
|
||||
history = create_mock_history(
|
||||
[
|
||||
({'input': {'index': 1, 'text': 'old@example.com'}}, create_test_element(attributes={'type': 'email'})),
|
||||
({'input': {'index': 2, 'text': 'John'}}, create_test_element(attributes={'name': 'first_name'})),
|
||||
({'input': {'index': 3, 'text': '1990-01-01'}}, create_test_element(attributes={'type': 'date'})),
|
||||
]
|
||||
)
|
||||
|
||||
# Substitute all variables
|
||||
modified_history = agent._substitute_variables_in_history(
|
||||
history, # type: ignore[arg-type]
|
||||
{
|
||||
'email': 'new@example.com',
|
||||
'first_name': 'Jane',
|
||||
'date': '1995-05-15',
|
||||
},
|
||||
)
|
||||
|
||||
# Check that all values were substituted
|
||||
action1 = modified_history.history[0].model_output.action[0] # type: ignore[attr-defined]
|
||||
action2 = modified_history.history[1].model_output.action[0] # type: ignore[attr-defined]
|
||||
action3 = modified_history.history[2].model_output.action[0] # type: ignore[attr-defined]
|
||||
|
||||
assert vars(action1)['input']['text'] == 'new@example.com'
|
||||
assert vars(action2)['input']['text'] == 'Jane'
|
||||
assert vars(action3)['input']['text'] == '1995-05-15'
|
||||
|
||||
|
||||
def test_substitute_partial_variables(mock_llm):
|
||||
"""Test substitution of only some variables"""
|
||||
agent = Agent(task='test', llm=mock_llm)
|
||||
|
||||
# Create mock history with email and name
|
||||
history = create_mock_history(
|
||||
[
|
||||
({'input': {'index': 1, 'text': 'old@example.com'}}, create_test_element(attributes={'type': 'email'})),
|
||||
({'input': {'index': 2, 'text': 'John'}}, create_test_element(attributes={'name': 'first_name'})),
|
||||
]
|
||||
)
|
||||
|
||||
# Substitute only email
|
||||
modified_history = agent._substitute_variables_in_history(
|
||||
history, # type: ignore[arg-type]
|
||||
{'email': 'new@example.com'},
|
||||
)
|
||||
|
||||
# Check that only email was substituted
|
||||
action1 = modified_history.history[0].model_output.action[0] # type: ignore[attr-defined]
|
||||
action2 = modified_history.history[1].model_output.action[0] # type: ignore[attr-defined]
|
||||
|
||||
assert vars(action1)['input']['text'] == 'new@example.com'
|
||||
assert vars(action2)['input']['text'] == 'John' # Unchanged
|
||||
|
||||
|
||||
def test_substitute_nonexistent_variable(mock_llm):
|
||||
"""Test that substituting a nonexistent variable doesn't break things"""
|
||||
agent = Agent(task='test', llm=mock_llm)
|
||||
|
||||
# Create mock history with email
|
||||
element = create_test_element(attributes={'type': 'email'})
|
||||
history = create_mock_history(
|
||||
[
|
||||
({'input': {'index': 1, 'text': 'old@example.com'}}, element),
|
||||
]
|
||||
)
|
||||
|
||||
# Try to substitute a variable that doesn't exist
|
||||
modified_history = agent._substitute_variables_in_history(
|
||||
history, # type: ignore[arg-type]
|
||||
{'nonexistent_var': 'some_value'},
|
||||
)
|
||||
|
||||
# Check that nothing changed
|
||||
action = modified_history.history[0].model_output.action[0] # type: ignore[attr-defined]
|
||||
action_dict = vars(action)
|
||||
assert action_dict['input']['text'] == 'old@example.com'
|
||||
|
||||
|
||||
def test_substitute_in_nested_dict(mock_llm):
|
||||
"""Test substitution in nested dictionary structures"""
|
||||
agent = Agent(task='test', llm=mock_llm)
|
||||
|
||||
# Create a more complex action with nested structure
|
||||
complex_action = {
|
||||
'search_google': {
|
||||
'query': 'test@example.com',
|
||||
'metadata': {'user': 'test@example.com'},
|
||||
}
|
||||
}
|
||||
|
||||
element = create_test_element(attributes={'type': 'email'})
|
||||
history = create_mock_history([(complex_action, element)])
|
||||
|
||||
# Substitute the email
|
||||
modified_history = agent._substitute_variables_in_history(
|
||||
history, # type: ignore[arg-type]
|
||||
{'email': 'new@example.com'},
|
||||
)
|
||||
|
||||
# Check that values in nested structures were substituted
|
||||
action = modified_history.history[0].model_output.action[0] # type: ignore[attr-defined]
|
||||
action_dict = vars(action)
|
||||
assert action_dict['search_google']['query'] == 'new@example.com'
|
||||
assert action_dict['search_google']['metadata']['user'] == 'new@example.com'
|
||||
|
||||
|
||||
def test_substitute_in_list(mock_llm):
|
||||
"""Test substitution in list structures"""
|
||||
agent = Agent(task='test', llm=mock_llm)
|
||||
|
||||
# Create history with an input action first (so email is detected)
|
||||
# Then an action with a list containing the same email
|
||||
history = create_mock_history(
|
||||
[
|
||||
({'input': {'index': 1, 'text': 'test@example.com'}}, create_test_element(attributes={'type': 'email'})),
|
||||
({'some_action': {'items': ['test@example.com', 'other_value', 'test@example.com']}}, None),
|
||||
]
|
||||
)
|
||||
|
||||
# Substitute the email
|
||||
modified_history = agent._substitute_variables_in_history(
|
||||
history, # type: ignore[arg-type]
|
||||
{'email': 'new@example.com'},
|
||||
)
|
||||
|
||||
# Check that values in the first action were substituted
|
||||
action1 = modified_history.history[0].model_output.action[0] # type: ignore[attr-defined]
|
||||
assert vars(action1)['input']['text'] == 'new@example.com'
|
||||
|
||||
# Check that values in lists were also substituted
|
||||
action2 = modified_history.history[1].model_output.action[0] # type: ignore[attr-defined]
|
||||
action_dict = vars(action2)
|
||||
assert action_dict['some_action']['items'] == ['new@example.com', 'other_value', 'new@example.com']
|
||||
|
||||
|
||||
def test_substitute_preserves_original_history(mock_llm):
|
||||
"""Test that substitution doesn't modify the original history"""
|
||||
agent = Agent(task='test', llm=mock_llm)
|
||||
|
||||
# Create mock history
|
||||
element = create_test_element(attributes={'type': 'email'})
|
||||
history = create_mock_history(
|
||||
[
|
||||
({'input': {'index': 1, 'text': 'old@example.com'}}, element),
|
||||
]
|
||||
)
|
||||
|
||||
# Get original value
|
||||
original_action = history.history[0].model_output.action[0]
|
||||
original_value = vars(original_action)['input']['text']
|
||||
|
||||
# Substitute
|
||||
agent._substitute_variables_in_history(history, {'email': 'new@example.com'}) # type: ignore[arg-type]
|
||||
|
||||
# Check that original history is unchanged
|
||||
current_value = vars(original_action)['input']['text']
|
||||
assert current_value == original_value
|
||||
assert current_value == 'old@example.com'
|
||||
|
||||
|
||||
def test_substitute_empty_variables(mock_llm):
|
||||
"""Test substitution with empty variables dict"""
|
||||
agent = Agent(task='test', llm=mock_llm)
|
||||
|
||||
# Create mock history
|
||||
element = create_test_element(attributes={'type': 'email'})
|
||||
history = create_mock_history(
|
||||
[
|
||||
({'input': {'index': 1, 'text': 'old@example.com'}}, element),
|
||||
]
|
||||
)
|
||||
|
||||
# Substitute with empty dict
|
||||
modified_history = agent._substitute_variables_in_history(history, {}) # type: ignore[arg-type]
|
||||
|
||||
# Check that nothing changed
|
||||
action = modified_history.history[0].model_output.action[0] # type: ignore[attr-defined]
|
||||
action_dict = vars(action)
|
||||
assert action_dict['input']['text'] == 'old@example.com'
|
||||
|
||||
|
||||
def test_substitute_same_value_multiple_times(mock_llm):
|
||||
"""Test that the same value is substituted across multiple actions"""
|
||||
agent = Agent(task='test', llm=mock_llm)
|
||||
|
||||
# Create history where same email appears twice
|
||||
element = create_test_element(attributes={'type': 'email'})
|
||||
history = create_mock_history(
|
||||
[
|
||||
({'input': {'index': 1, 'text': 'old@example.com'}}, element),
|
||||
({'input': {'index': 2, 'text': 'old@example.com'}}, element),
|
||||
]
|
||||
)
|
||||
|
||||
# Substitute the email
|
||||
modified_history = agent._substitute_variables_in_history(
|
||||
history, # type: ignore[arg-type]
|
||||
{'email': 'new@example.com'},
|
||||
)
|
||||
|
||||
# Check that both occurrences were substituted
|
||||
action1 = modified_history.history[0].model_output.action[0] # type: ignore[attr-defined]
|
||||
action2 = modified_history.history[1].model_output.action[0] # type: ignore[attr-defined]
|
||||
|
||||
assert vars(action1)['input']['text'] == 'new@example.com'
|
||||
assert vars(action2)['input']['text'] == 'new@example.com'
|
||||
@@ -0,0 +1,299 @@
|
||||
"""
|
||||
Debug test for iframe scrolling issue where DOM tree only shows top elements after scrolling.
|
||||
|
||||
This test verifies that after scrolling inside an iframe, the selector_map correctly
|
||||
contains lower input elements like City, State, Zip Code, etc.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Add parent directory to path to import browser_use modules
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent.parent))
|
||||
|
||||
from browser_use.agent.service import Agent
|
||||
from browser_use.agent.views import ActionModel
|
||||
from browser_use.browser import BrowserProfile, BrowserSession
|
||||
from browser_use.browser.events import BrowserStateRequestEvent
|
||||
|
||||
# Import the mock LLM helper from conftest
|
||||
from tests.ci.conftest import create_mock_llm
|
||||
|
||||
|
||||
async def debug_iframe_scrolling():
|
||||
"""Debug iframe scrolling and DOM visibility issue."""
|
||||
|
||||
print('Starting iframe scrolling debug test...')
|
||||
|
||||
# Create the sequence of actions for the mock LLM
|
||||
# We need to format these as the LLM would return them
|
||||
actions = [
|
||||
# First action: Navigate to the test URL
|
||||
"""
|
||||
{
|
||||
"thinking": "Navigating to the iframe test page",
|
||||
"evaluation_previous_goal": null,
|
||||
"memory": "Starting test",
|
||||
"next_goal": "Navigate to the iframe test page",
|
||||
"action": [
|
||||
{
|
||||
"navigate": {
|
||||
"url": "https://browser-use.github.io/stress-tests/challenges/iframe-inception-level1.html",
|
||||
"new_tab": false
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
# Second action: Input text in the first name field (to verify we can interact)
|
||||
"""
|
||||
{
|
||||
"thinking": "Inputting text in the first name field to test interaction",
|
||||
"evaluation_previous_goal": "Successfully navigated to the page",
|
||||
"memory": "Page loaded with nested iframes",
|
||||
"next_goal": "Type text in the first name field",
|
||||
"action": [
|
||||
{
|
||||
"input_text": {
|
||||
"index": 1,
|
||||
"text": "TestName"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
# Third action: Scroll the iframe (element_index=2 should be the iframe)
|
||||
"""
|
||||
{
|
||||
"thinking": "Scrolling inside the iframe to reveal lower form elements",
|
||||
"evaluation_previous_goal": "Successfully typed in first name field",
|
||||
"memory": "Typed TestName in first field",
|
||||
"next_goal": "Scroll inside the innermost iframe to see more form fields",
|
||||
"action": [
|
||||
{
|
||||
"scroll": {
|
||||
"down": true,
|
||||
"num_pages": 1.0,
|
||||
"index": 2
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
# Fourth action: Done
|
||||
"""
|
||||
{
|
||||
"thinking": "Completed scrolling, ready to inspect DOM",
|
||||
"evaluation_previous_goal": "Successfully scrolled inside iframe",
|
||||
"memory": "Scrolled to reveal lower form fields",
|
||||
"next_goal": "Task completed",
|
||||
"action": [
|
||||
{
|
||||
"done": {
|
||||
"text": "Scrolling completed",
|
||||
"success": true
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
""",
|
||||
]
|
||||
|
||||
# Create mock LLM with our action sequence
|
||||
mock_llm = create_mock_llm(actions=actions)
|
||||
|
||||
# Create browser session with headless=False so we can see what's happening
|
||||
browser_session = BrowserSession(
|
||||
browser_profile=BrowserProfile(
|
||||
headless=False, # Set to False to see the browser
|
||||
user_data_dir=None, # Use temporary directory
|
||||
keep_alive=True,
|
||||
enable_default_extensions=True,
|
||||
cross_origin_iframes=True, # Enable cross-origin iframe support
|
||||
)
|
||||
)
|
||||
|
||||
try:
|
||||
# Start the browser session
|
||||
await browser_session.start()
|
||||
print('Browser session started')
|
||||
|
||||
# Create an agent with the mock LLM
|
||||
agent = Agent(
|
||||
task='Navigate to the iframe test page and scroll inside the iframe',
|
||||
llm=mock_llm,
|
||||
browser_session=browser_session,
|
||||
)
|
||||
|
||||
# Helper function to capture and analyze DOM state
|
||||
async def capture_dom_state(label: str) -> dict:
|
||||
"""Capture DOM state and return analysis"""
|
||||
print(f'\n📸 Capturing DOM state: {label}')
|
||||
state_event = browser_session.event_bus.dispatch(
|
||||
BrowserStateRequestEvent(include_dom=True, include_screenshot=False, include_recent_events=False)
|
||||
)
|
||||
browser_state = await state_event.event_result()
|
||||
|
||||
if browser_state and browser_state.dom_state and browser_state.dom_state.selector_map:
|
||||
selector_map = browser_state.dom_state.selector_map
|
||||
element_count = len(selector_map)
|
||||
|
||||
# Check for specific elements
|
||||
found_elements = {}
|
||||
expected_checks = [
|
||||
('First Name', ['firstName', 'first name']),
|
||||
('Last Name', ['lastName', 'last name']),
|
||||
('Email', ['email']),
|
||||
('City', ['city']),
|
||||
('State', ['state']),
|
||||
('Zip', ['zip', 'zipCode']),
|
||||
]
|
||||
|
||||
for name, keywords in expected_checks:
|
||||
for index, element in selector_map.items():
|
||||
element_str = str(element).lower()
|
||||
if any(kw.lower() in element_str for kw in keywords):
|
||||
found_elements[name] = True
|
||||
break
|
||||
|
||||
return {
|
||||
'label': label,
|
||||
'total_elements': element_count,
|
||||
'found_elements': found_elements,
|
||||
'selector_map': selector_map,
|
||||
}
|
||||
return {'label': label, 'error': 'No DOM state available'}
|
||||
|
||||
# Capture initial state before any actions
|
||||
print('\n' + '=' * 80)
|
||||
print('PHASE 1: INITIAL PAGE LOAD')
|
||||
print('=' * 80)
|
||||
|
||||
# Navigate to the page first
|
||||
from browser_use.tools.service import Tools
|
||||
|
||||
tools = Tools()
|
||||
|
||||
# Create the action model for navigation
|
||||
goto_action = ActionModel.model_validate_json(actions[0])
|
||||
await tools.act(goto_action, browser_session)
|
||||
await asyncio.sleep(2) # Wait for page to fully load
|
||||
|
||||
initial_state = await capture_dom_state('INITIAL (after page load)')
|
||||
|
||||
# Now run the rest of the actions via the agent
|
||||
print('\n' + '=' * 80)
|
||||
print('PHASE 2: EXECUTING ACTIONS')
|
||||
print('=' * 80)
|
||||
|
||||
# Create new agent with remaining actions
|
||||
remaining_actions = actions[1:] # Skip the navigation we already did
|
||||
mock_llm_remaining = create_mock_llm(actions=remaining_actions)
|
||||
agent = Agent(
|
||||
task='Input text and scroll inside the iframe',
|
||||
llm=mock_llm_remaining,
|
||||
browser_session=browser_session,
|
||||
)
|
||||
|
||||
# Hook into agent actions to capture state after each one
|
||||
states = []
|
||||
original_act = tools.act
|
||||
|
||||
async def wrapped_act(action, session):
|
||||
result = await original_act(action, session)
|
||||
# Capture state after each action
|
||||
action_type = 'unknown'
|
||||
if hasattr(action, 'input_text') and action.input_text:
|
||||
action_type = 'input_text'
|
||||
await asyncio.sleep(1) # Give time for DOM to update
|
||||
state = await capture_dom_state('AFTER INPUT_TEXT')
|
||||
states.append(state)
|
||||
elif hasattr(action, 'scroll') and action.scroll:
|
||||
action_type = 'scroll'
|
||||
await asyncio.sleep(2) # Give more time after scroll
|
||||
state = await capture_dom_state('AFTER SCROLL')
|
||||
states.append(state)
|
||||
return result
|
||||
|
||||
tools.act = wrapped_act
|
||||
|
||||
# Run the agent with remaining actions
|
||||
result = await agent.run()
|
||||
print(f'\nAgent completed with result: {result}')
|
||||
|
||||
# Analyze all captured states
|
||||
print('\n' + '=' * 80)
|
||||
print('PHASE 3: ANALYSIS OF DOM STATES')
|
||||
print('=' * 80)
|
||||
|
||||
all_states = [initial_state] + states
|
||||
|
||||
for state in all_states:
|
||||
if 'error' in state:
|
||||
print(f'\n❌ {state["label"]}: {state["error"]}')
|
||||
else:
|
||||
print(f'\n📊 {state["label"]}:')
|
||||
print(f' Total elements: {state["total_elements"]}')
|
||||
print(' Found elements:')
|
||||
for elem_name, found in state['found_elements'].items():
|
||||
status = '✓' if found else '✗'
|
||||
print(f' {status} {elem_name}')
|
||||
|
||||
# Compare states
|
||||
print('\n' + '=' * 80)
|
||||
print('COMPARISON SUMMARY')
|
||||
print('=' * 80)
|
||||
|
||||
if len(all_states) >= 3:
|
||||
initial = all_states[0]
|
||||
after_input = all_states[1] if len(all_states) > 1 else None
|
||||
after_scroll = all_states[2] if len(all_states) > 2 else None
|
||||
|
||||
print('\nElement count changes:')
|
||||
print(f' Initial: {initial.get("total_elements", 0)} elements')
|
||||
if after_input:
|
||||
print(f' After input_text: {after_input.get("total_elements", 0)} elements')
|
||||
if after_scroll:
|
||||
print(f' After scroll: {after_scroll.get("total_elements", 0)} elements')
|
||||
|
||||
# Check if lower form fields appear after scroll
|
||||
if after_scroll and 'found_elements' in after_scroll:
|
||||
lower_fields = ['City', 'State', 'Zip']
|
||||
missing_fields = [f for f in lower_fields if not after_scroll['found_elements'].get(f, False)]
|
||||
|
||||
if missing_fields:
|
||||
print('\n⚠️ BUG CONFIRMED: Lower form fields missing after scroll:')
|
||||
for field in missing_fields:
|
||||
print(f' ✗ {field}')
|
||||
print('\nThis confirms that scrolling inside iframes does not update the DOM tree properly.')
|
||||
else:
|
||||
print('\n✅ SUCCESS: All lower form fields are visible after scrolling!')
|
||||
|
||||
# Show first few elements from final state for debugging
|
||||
if states and 'selector_map' in states[-1]:
|
||||
print('\n' + '=' * 80)
|
||||
print('DEBUG: First 5 elements in final selector_map')
|
||||
print('=' * 80)
|
||||
final_map = states[-1]['selector_map']
|
||||
for i, (index, element) in enumerate(list(final_map.items())[:5]):
|
||||
elem_preview = str(element)[:150]
|
||||
print(f'\n [{index}]: {elem_preview}...')
|
||||
|
||||
# Keep browser open for manual inspection if needed
|
||||
print('\n' + '=' * 80)
|
||||
print('Test complete. Browser will remain open for 10 seconds for inspection...')
|
||||
print('=' * 80)
|
||||
await asyncio.sleep(10)
|
||||
|
||||
finally:
|
||||
# Clean up
|
||||
print('\nCleaning up...')
|
||||
await browser_session.kill()
|
||||
await browser_session.event_bus.stop(clear=True, timeout=5)
|
||||
print('Browser session closed')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
# Run the debug test
|
||||
asyncio.run(debug_iframe_scrolling())
|
||||
Executable
+260
@@ -0,0 +1,260 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Test frame hierarchy for any URL passed as argument."""
|
||||
|
||||
import asyncio
|
||||
import sys
|
||||
|
||||
from browser_use.browser import BrowserSession
|
||||
from browser_use.browser.events import BrowserStartEvent
|
||||
from browser_use.browser.profile import BrowserProfile
|
||||
|
||||
|
||||
async def analyze_frame_hierarchy(url):
|
||||
"""Analyze and display complete frame hierarchy for a URL."""
|
||||
|
||||
profile = BrowserProfile(headless=True, user_data_dir=None)
|
||||
session = BrowserSession(browser_profile=profile)
|
||||
|
||||
try:
|
||||
print('🚀 Starting browser...')
|
||||
await session.on_BrowserStartEvent(BrowserStartEvent())
|
||||
|
||||
print(f'📍 Navigating to: {url}')
|
||||
await session._cdp_navigate(url)
|
||||
await asyncio.sleep(3)
|
||||
|
||||
print('\n' + '=' * 80)
|
||||
print('FRAME HIERARCHY ANALYSIS')
|
||||
print('=' * 80)
|
||||
|
||||
# Get all targets from SessionManager
|
||||
all_targets = session.session_manager.get_all_targets()
|
||||
|
||||
# Separate by type
|
||||
page_targets = [target for target in all_targets.values() if target.target_type == 'page']
|
||||
iframe_targets = [target for target in all_targets.values() if target.target_type == 'iframe']
|
||||
|
||||
print('\n📊 Target Summary:')
|
||||
print(f' Total targets: {len(all_targets)}')
|
||||
print(f' Page targets: {len(page_targets)}')
|
||||
print(f' Iframe targets (OOPIFs): {len(iframe_targets)}')
|
||||
|
||||
# Show all targets
|
||||
print('\n📋 All Targets:')
|
||||
for i, (target_id, target) in enumerate(all_targets.items()):
|
||||
if target.target_type in ['page', 'iframe']:
|
||||
print(f'\n [{i + 1}] Type: {target.target_type}')
|
||||
print(f' URL: {target.url}')
|
||||
print(f' Target ID: {target.target_id[:30]}...')
|
||||
|
||||
# Check if target has active sessions using the public API
|
||||
try:
|
||||
cdp_session = await session.get_or_create_cdp_session(target.target_id, focus=False)
|
||||
has_session = cdp_session is not None
|
||||
except Exception:
|
||||
has_session = False
|
||||
print(f' Has Session: {has_session}')
|
||||
|
||||
# Get main page frame tree
|
||||
main_target = next((t for t in page_targets if url in t.url), page_targets[0] if page_targets else None)
|
||||
|
||||
if main_target:
|
||||
print('\n📐 Main Page Frame Tree:')
|
||||
print(f' Target: {main_target.url}')
|
||||
print(f' Target ID: {main_target.target_id[:30]}...')
|
||||
|
||||
s = await session.cdp_client.send.Target.attachToTarget(params={'targetId': main_target.target_id, 'flatten': True})
|
||||
sid = s['sessionId']
|
||||
|
||||
try:
|
||||
await session.cdp_client.send.Page.enable(session_id=sid)
|
||||
tree = await session.cdp_client.send.Page.getFrameTree(session_id=sid)
|
||||
|
||||
print('\n Frame Tree Structure:')
|
||||
|
||||
def print_tree(node, indent=0, parent_id=None):
|
||||
frame = node['frame']
|
||||
frame_id = frame.get('id', 'unknown')
|
||||
frame_url = frame.get('url', 'none')
|
||||
|
||||
prefix = ' ' * indent + ('└─ ' if indent > 0 else '')
|
||||
print(f'{prefix}Frame: {frame_url}')
|
||||
print(f'{" " * (indent + 1)}ID: {frame_id[:30]}...')
|
||||
|
||||
if parent_id:
|
||||
print(f'{" " * (indent + 1)}Parent: {parent_id[:30]}...')
|
||||
|
||||
# Check cross-origin status
|
||||
cross_origin = frame.get('crossOriginIsolatedContextType', 'unknown')
|
||||
if cross_origin != 'NotIsolated':
|
||||
print(f'{" " * (indent + 1)}⚠️ Cross-Origin: {cross_origin}')
|
||||
|
||||
# Process children
|
||||
for child in node.get('childFrames', []):
|
||||
print_tree(child, indent + 1, frame_id)
|
||||
|
||||
print_tree(tree['frameTree'])
|
||||
|
||||
finally:
|
||||
await session.cdp_client.send.Target.detachFromTarget(params={'sessionId': sid})
|
||||
|
||||
# Show iframe target trees
|
||||
if iframe_targets:
|
||||
print('\n🔸 OOPIF Target Frame Trees:')
|
||||
|
||||
for iframe_target in iframe_targets:
|
||||
print(f'\n OOPIF Target: {iframe_target.url}')
|
||||
print(f' Target ID: {iframe_target.target_id[:30]}...')
|
||||
|
||||
s = await session.cdp_client.send.Target.attachToTarget(
|
||||
params={'targetId': iframe_target.target_id, 'flatten': True}
|
||||
)
|
||||
sid = s['sessionId']
|
||||
|
||||
try:
|
||||
await session.cdp_client.send.Page.enable(session_id=sid)
|
||||
tree = await session.cdp_client.send.Page.getFrameTree(session_id=sid)
|
||||
|
||||
frame = tree['frameTree']['frame']
|
||||
print(f' Frame ID: {frame.get("id", "unknown")[:30]}...')
|
||||
print(f' Frame URL: {frame.get("url", "none")}')
|
||||
print(' ⚠️ This frame runs in a separate process (OOPIF)')
|
||||
|
||||
except Exception as e:
|
||||
print(f' Error: {e}')
|
||||
finally:
|
||||
await session.cdp_client.send.Target.detachFromTarget(params={'sessionId': sid})
|
||||
|
||||
# Now show unified view from get_all_frames
|
||||
print('\n' + '=' * 80)
|
||||
print('UNIFIED FRAME HIERARCHY (get_all_frames method)')
|
||||
print('=' * 80)
|
||||
|
||||
all_frames, target_sessions = await session.get_all_frames()
|
||||
|
||||
# Clean up sessions
|
||||
for tid, sess_id in target_sessions.items():
|
||||
try:
|
||||
await session.cdp_client.send.Target.detachFromTarget(params={'sessionId': sess_id})
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
print('\n📊 Frame Statistics:')
|
||||
print(f' Total frames discovered: {len(all_frames)}')
|
||||
|
||||
# Separate root and child frames
|
||||
root_frames = []
|
||||
child_frames = []
|
||||
|
||||
for frame_id, frame_info in all_frames.items():
|
||||
if not frame_info.get('parentFrameId'):
|
||||
root_frames.append((frame_id, frame_info))
|
||||
else:
|
||||
child_frames.append((frame_id, frame_info))
|
||||
|
||||
print(f' Root frames: {len(root_frames)}')
|
||||
print(f' Child frames: {len(child_frames)}')
|
||||
|
||||
# Display all frames with details
|
||||
print('\n📋 All Frames:')
|
||||
|
||||
for i, (frame_id, frame_info) in enumerate(all_frames.items()):
|
||||
url = frame_info.get('url', 'none')
|
||||
parent = frame_info.get('parentFrameId')
|
||||
target_id = frame_info.get('frameTargetId', 'unknown')
|
||||
is_cross = frame_info.get('isCrossOrigin', False)
|
||||
|
||||
print(f'\n [{i + 1}] Frame URL: {url}')
|
||||
print(f' Frame ID: {frame_id[:30]}...')
|
||||
print(f' Parent Frame ID: {parent[:30] + "..." if parent else "None (ROOT)"}')
|
||||
print(f' Target ID: {target_id[:30]}...')
|
||||
print(f' Cross-Origin: {is_cross}')
|
||||
|
||||
# Highlight problems
|
||||
if not parent and 'v0-simple-landing' in url:
|
||||
print(' ❌ PROBLEM: Cross-origin frame incorrectly marked as root!')
|
||||
elif not parent and url != 'about:blank' and url not in ['chrome://newtab/', 'about:blank']:
|
||||
# Check if this should be the main frame
|
||||
if any(url in t.url for t in page_targets):
|
||||
print(' ✅ Correctly identified as root frame')
|
||||
|
||||
if is_cross:
|
||||
print(' 🔸 This is a cross-origin frame (OOPIF)')
|
||||
|
||||
# Show parent-child relationships
|
||||
print('\n🌳 Frame Relationships:')
|
||||
|
||||
# Build a tree structure
|
||||
def print_frame_tree(frame_id, frame_info, indent=0, visited=None):
|
||||
if visited is None:
|
||||
visited = set()
|
||||
|
||||
if frame_id in visited:
|
||||
return
|
||||
visited.add(frame_id)
|
||||
|
||||
url = frame_info.get('url', 'none')
|
||||
prefix = ' ' * indent + ('└─ ' if indent > 0 else '')
|
||||
|
||||
print(f'{prefix}{url[:60]}...')
|
||||
print(f'{" " * (indent + 1)}[{frame_id[:20]}...]')
|
||||
|
||||
# Find children
|
||||
for child_id, child_info in all_frames.items():
|
||||
if child_info.get('parentFrameId') == frame_id:
|
||||
print_frame_tree(child_id, child_info, indent + 1, visited)
|
||||
|
||||
# Print trees starting from roots
|
||||
for frame_id, frame_info in root_frames:
|
||||
print('\n Tree starting from root:')
|
||||
print_frame_tree(frame_id, frame_info)
|
||||
|
||||
print('\n' + '=' * 80)
|
||||
print('✅ Analysis complete!')
|
||||
print('=' * 80)
|
||||
|
||||
except Exception as e:
|
||||
print(f'❌ Error: {e}')
|
||||
import traceback
|
||||
|
||||
traceback.print_exc()
|
||||
finally:
|
||||
# Stop the CDP client first before killing the browser
|
||||
print('\n🛑 Shutting down...')
|
||||
|
||||
# Close CDP connection first while browser is still alive
|
||||
if session._cdp_client_root:
|
||||
try:
|
||||
await session._cdp_client_root.stop()
|
||||
except Exception:
|
||||
pass # Ignore errors if already disconnected
|
||||
|
||||
# Then stop the browser process
|
||||
from browser_use.browser.events import BrowserStopEvent
|
||||
|
||||
stop_event = session.event_bus.dispatch(BrowserStopEvent())
|
||||
try:
|
||||
await asyncio.wait_for(stop_event, timeout=2.0)
|
||||
except TimeoutError:
|
||||
print('⚠️ Browser stop timed out')
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) != 2:
|
||||
print('Usage: python test_frame_hierarchy.py <URL>')
|
||||
print('\nExample URLs to test:')
|
||||
print(' https://v0-website-with-clickable-elements.vercel.app/nested-iframe')
|
||||
print(' https://v0-website-with-clickable-elements.vercel.app/cross-origin')
|
||||
print(' https://v0-website-with-clickable-elements.vercel.app/shadow-dom')
|
||||
sys.exit(1)
|
||||
|
||||
url = sys.argv[1]
|
||||
asyncio.run(analyze_frame_hierarchy(url))
|
||||
|
||||
# Ensure clean exit
|
||||
print('✅ Script completed')
|
||||
sys.exit(0)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
Reference in New Issue
Block a user