chore: import upstream snapshot with attribution
Test Browser Use CLI Install / uv pip install (ubuntu-latest) (push) Failing after 1s
Test Browser Use CLI Install / uvx browser-use from local wheel (push) Failing after 1s
Test Browser Use CLI Install / uvx browser-use[cli] from PyPI (push) Failing after 1s
package / pip-install-on-macos-latest-py-3.11 (push) Has been skipped
package / pip-install-on-macos-latest-py-3.13 (push) Has been skipped
package / pip-install-on-ubuntu-latest-py-3.11 (push) Has been skipped
package / pip-install-on-windows-latest-py-3.13 (push) Has been skipped
cloud_evals / trigger_cloud_eval_image_build (push) Failing after 1s
docker / build_publish_image (push) Failing after 1s
Test Browser Use CLI Install / browser-use skill sync (push) Failing after 1s
lint / code-style (push) Failing after 0s
lint / type-checker (push) Failing after 1s
package / pip-build (push) Failing after 1s
lint / syntax-errors (push) Failing after 3s
package / pip-install-on-ubuntu-latest-py-3.13 (push) Has been skipped
package / pip-install-on-windows-latest-py-3.11 (push) Has been skipped
test / ${{ matrix.test_filename }} (push) Has been skipped
test / evaluate-tasks (push) Has been skipped
test / setup-chromium (push) Failing after 2s
test / find_tests (push) Failing after 2s
Test Browser Use CLI Install / uv pip install (windows-latest) (push) Has been cancelled
Test Browser Use CLI Install / uv pip install (macos-latest) (push) Has been cancelled
Test Browser Use CLI Install / uv pip install (ubuntu-latest) (push) Failing after 1s
Test Browser Use CLI Install / uvx browser-use from local wheel (push) Failing after 1s
Test Browser Use CLI Install / uvx browser-use[cli] from PyPI (push) Failing after 1s
package / pip-install-on-macos-latest-py-3.11 (push) Has been skipped
package / pip-install-on-macos-latest-py-3.13 (push) Has been skipped
package / pip-install-on-ubuntu-latest-py-3.11 (push) Has been skipped
package / pip-install-on-windows-latest-py-3.13 (push) Has been skipped
cloud_evals / trigger_cloud_eval_image_build (push) Failing after 1s
docker / build_publish_image (push) Failing after 1s
Test Browser Use CLI Install / browser-use skill sync (push) Failing after 1s
lint / code-style (push) Failing after 0s
lint / type-checker (push) Failing after 1s
package / pip-build (push) Failing after 1s
lint / syntax-errors (push) Failing after 3s
package / pip-install-on-ubuntu-latest-py-3.13 (push) Has been skipped
package / pip-install-on-windows-latest-py-3.11 (push) Has been skipped
test / ${{ matrix.test_filename }} (push) Has been skipped
test / evaluate-tasks (push) Has been skipped
test / setup-chromium (push) Failing after 2s
test / find_tests (push) Failing after 2s
Test Browser Use CLI Install / uv pip install (windows-latest) (push) Has been cancelled
Test Browser Use CLI Install / uv pip install (macos-latest) (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,87 @@
|
||||
# Ad-Use
|
||||
|
||||
Automatically generate Instagram image ads and TikTok video ads from any landing page using browser agents, Google's Nano Banana 🍌, and Veo3.
|
||||
|
||||
> [!WARNING]
|
||||
> This demo requires browser-use v0.7.7+.
|
||||
|
||||
https://github.com/user-attachments/assets/7fab54a9-b36b-4fba-ab98-a438f2b86b7e
|
||||
|
||||
## Features
|
||||
|
||||
1. Agent visits your target website
|
||||
2. Captures brand name, tagline, and key selling points
|
||||
3. Takes a clean screenshot for design reference
|
||||
4. Creates scroll-stopping Instagram image ads with 🍌
|
||||
5. Generates viral TikTok video ads with Veo3
|
||||
6. Supports parallel generation of multiple ads
|
||||
|
||||
## Setup
|
||||
|
||||
Make sure the newest version of browser-use is installed (with screenshot functionality):
|
||||
```bash
|
||||
pip install -U browser-use
|
||||
```
|
||||
|
||||
Export your Gemini API key, get it from: [Google AI Studio](https://makersuite.google.com/app/apikey)
|
||||
```
|
||||
export GOOGLE_API_KEY='your-google-api-key-here'
|
||||
```
|
||||
|
||||
Clone the repo and cd into the app folder
|
||||
```bash
|
||||
git clone https://github.com/browser-use/browser-use.git
|
||||
cd browser-use/examples/apps/ad-use
|
||||
```
|
||||
|
||||
## Normal Usage
|
||||
|
||||
```bash
|
||||
# Basic - Generate Instagram image ad (default)
|
||||
python ad_generator.py --url https://www.apple.com/iphone-17-pro/
|
||||
|
||||
# Generate TikTok video ad with Veo3
|
||||
python ad_generator.py --tiktok --url https://www.apple.com/iphone-17-pro/
|
||||
|
||||
# Generate multiple ads in parallel
|
||||
python ad_generator.py --instagram --count 3 --url https://www.apple.com/iphone-17-pro/
|
||||
python ad_generator.py --tiktok --count 2 --url https://www.apple.com/iphone-17-pro/
|
||||
|
||||
# Debug Mode - See the browser in action
|
||||
python ad_generator.py --url https://www.apple.com/iphone-17-pro/ --debug
|
||||
```
|
||||
|
||||
## Command Line Options
|
||||
|
||||
- `--url`: Landing page URL to analyze
|
||||
- `--instagram`: Generate Instagram image ad (default if no flag specified)
|
||||
- `--tiktok`: Generate TikTok video ad using Veo3
|
||||
- `--count N`: Generate N ads in parallel (default: 1)
|
||||
- `--debug`: Show browser window and enable verbose logging
|
||||
|
||||
## Programmatic Usage
|
||||
```python
|
||||
import asyncio
|
||||
from ad_generator import create_ad_from_landing_page
|
||||
|
||||
async def main():
|
||||
results = await create_ad_from_landing_page(
|
||||
url="https://your-landing-page.com",
|
||||
debug=False
|
||||
)
|
||||
print(f"Generated ads: {results}")
|
||||
|
||||
asyncio.run(main())
|
||||
```
|
||||
|
||||
## Output
|
||||
|
||||
Generated ads are saved in the `output/` directory with:
|
||||
- **PNG image files** (ad_timestamp.png) - Instagram ads generated with Gemini 2.5 Flash Image
|
||||
- **MP4 video files** (ad_timestamp.mp4) - TikTok ads generated with Veo3
|
||||
- **Analysis files** (analysis_timestamp.txt) - Browser agent analysis and prompts used
|
||||
- **Landing page screenshots** (landing_page_timestamp.png) - Reference screenshots
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
@@ -0,0 +1,417 @@
|
||||
import argparse
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
from browser_use.utils import create_task_with_error_handling
|
||||
|
||||
|
||||
def setup_environment(debug: bool):
|
||||
if not debug:
|
||||
os.environ['BROWSER_USE_SETUP_LOGGING'] = 'false'
|
||||
os.environ['BROWSER_USE_LOGGING_LEVEL'] = 'critical'
|
||||
logging.getLogger().setLevel(logging.CRITICAL)
|
||||
else:
|
||||
os.environ['BROWSER_USE_SETUP_LOGGING'] = 'true'
|
||||
os.environ['BROWSER_USE_LOGGING_LEVEL'] = 'info'
|
||||
|
||||
|
||||
parser = argparse.ArgumentParser(description='Generate ads from landing pages using browser-use + 🍌')
|
||||
parser.add_argument('--url', nargs='?', help='Landing page URL to analyze')
|
||||
parser.add_argument('--debug', action='store_true', default=False, help='Enable debug mode (show browser, verbose logs)')
|
||||
parser.add_argument('--count', type=int, default=1, help='Number of ads to generate in parallel (default: 1)')
|
||||
group = parser.add_mutually_exclusive_group()
|
||||
group.add_argument('--instagram', action='store_true', default=False, help='Generate Instagram image ad (default)')
|
||||
group.add_argument('--tiktok', action='store_true', default=False, help='Generate TikTok video ad using Veo3')
|
||||
args = parser.parse_args()
|
||||
if not args.instagram and not args.tiktok:
|
||||
args.instagram = True
|
||||
setup_environment(args.debug)
|
||||
|
||||
from typing import Any, cast
|
||||
|
||||
import aiofiles
|
||||
from google import genai
|
||||
from PIL import Image
|
||||
|
||||
from browser_use import Agent, BrowserSession
|
||||
from browser_use.llm.google import ChatGoogle
|
||||
|
||||
GOOGLE_API_KEY = os.getenv('GOOGLE_API_KEY')
|
||||
|
||||
|
||||
class LandingPageAnalyzer:
|
||||
def __init__(self, debug: bool = False):
|
||||
self.debug = debug
|
||||
self.llm = ChatGoogle(model='gemini-2.0-flash-exp', api_key=GOOGLE_API_KEY)
|
||||
self.output_dir = Path('output')
|
||||
self.output_dir.mkdir(exist_ok=True)
|
||||
|
||||
async def analyze_landing_page(self, url: str, mode: str = 'instagram') -> dict:
|
||||
browser_session = BrowserSession(
|
||||
headless=not self.debug,
|
||||
)
|
||||
|
||||
agent = Agent(
|
||||
task=f"""Go to {url} and quickly extract key brand information for Instagram ad creation.
|
||||
|
||||
Steps:
|
||||
1. Navigate to the website
|
||||
2. From the initial view, extract ONLY these essentials:
|
||||
- Brand/Product name
|
||||
- Main tagline or value proposition (one sentence)
|
||||
- Primary call-to-action text
|
||||
- Any visible pricing or special offer
|
||||
3. Scroll down half a page, twice (0.5 pages each) to check for any key info
|
||||
4. Done - keep it simple and focused on the brand
|
||||
|
||||
Return ONLY the key brand info, not page structure details.""",
|
||||
llm=self.llm,
|
||||
browser_session=browser_session,
|
||||
max_actions_per_step=2,
|
||||
step_timeout=30,
|
||||
use_thinking=False,
|
||||
vision_detail_level='high',
|
||||
)
|
||||
|
||||
screenshot_path = None
|
||||
timestamp = datetime.now().strftime('%Y%m%d_%H%M%S')
|
||||
|
||||
async def screenshot_callback(agent_instance):
|
||||
nonlocal screenshot_path
|
||||
await asyncio.sleep(4)
|
||||
screenshot_path = self.output_dir / f'landing_page_{timestamp}.png'
|
||||
await agent_instance.browser_session.take_screenshot(path=str(screenshot_path), full_page=False)
|
||||
|
||||
screenshot_task = create_task_with_error_handling(
|
||||
screenshot_callback(agent), name='screenshot_callback', suppress_exceptions=True
|
||||
)
|
||||
history = await agent.run()
|
||||
try:
|
||||
await screenshot_task
|
||||
except Exception as e:
|
||||
print(f'Screenshot task failed: {e}')
|
||||
|
||||
analysis = history.final_result() or 'No analysis content extracted'
|
||||
return {'url': url, 'analysis': analysis, 'screenshot_path': screenshot_path, 'timestamp': timestamp}
|
||||
|
||||
|
||||
class AdGenerator:
|
||||
def __init__(self, api_key: str | None = GOOGLE_API_KEY, mode: str = 'instagram'):
|
||||
if not api_key:
|
||||
raise ValueError('GOOGLE_API_KEY is missing or empty – set the environment variable or pass api_key explicitly')
|
||||
|
||||
self.client = genai.Client(api_key=api_key)
|
||||
self.output_dir = Path('output')
|
||||
self.output_dir.mkdir(exist_ok=True)
|
||||
self.mode = mode
|
||||
|
||||
async def create_video_concept(self, browser_analysis: str, ad_id: int) -> str:
|
||||
"""Generate a unique creative concept for each video ad"""
|
||||
if self.mode != 'tiktok':
|
||||
return ''
|
||||
|
||||
concept_prompt = f"""Based on this brand analysis:
|
||||
{browser_analysis}
|
||||
|
||||
Create a UNIQUE and SPECIFIC TikTok video concept #{ad_id}.
|
||||
|
||||
Be creative and different! Consider various approaches like:
|
||||
- Different visual metaphors and storytelling angles
|
||||
- Various trending TikTok formats (transitions, reveals, transformations)
|
||||
- Different emotional appeals (funny, inspiring, surprising, relatable)
|
||||
- Unique visual styles (neon, retro, minimalist, maximalist, surreal)
|
||||
- Different perspectives (first-person, aerial, macro, time-lapse)
|
||||
|
||||
Return a 2-3 sentence description of a specific, unique video concept that would work for this brand.
|
||||
Make it visually interesting and different from typical ads. Be specific about visual elements, transitions, and mood."""
|
||||
|
||||
response = self.client.models.generate_content(model='gemini-2.0-flash-exp', contents=concept_prompt)
|
||||
return response.text if response and response.text else ''
|
||||
|
||||
def create_ad_prompt(self, browser_analysis: str, video_concept: str = '') -> str:
|
||||
if self.mode == 'instagram':
|
||||
prompt = f"""Create an Instagram ad for this brand:
|
||||
|
||||
{browser_analysis}
|
||||
|
||||
Create a vibrant, eye-catching Instagram ad image with:
|
||||
- Try to use the colors and style of the logo or brand, else:
|
||||
- Bold, modern gradient background with bright colors
|
||||
- Large, playful sans-serif text with the product/service name from the analysis
|
||||
- Trendy design elements: geometric shapes, sparkles, emojis
|
||||
- Fun bubbles or badges for any pricing or special offers mentioned
|
||||
- Call-to-action button with text from the analysis
|
||||
- Emphasizes the key value proposition from the analysis
|
||||
- Uses visual elements that match the brand personality
|
||||
- Square format (1:1 ratio)
|
||||
- Use color psychology to drive action
|
||||
|
||||
Style: Modern Instagram advertisement, (1:1), scroll-stopping, professional but playful, conversion-focused"""
|
||||
else: # tiktok
|
||||
if video_concept:
|
||||
prompt = f"""Create a TikTok video ad based on this specific concept:
|
||||
|
||||
{video_concept}
|
||||
|
||||
Brand context: {browser_analysis}
|
||||
|
||||
Requirements:
|
||||
- Vertical 9:16 format
|
||||
- High quality, professional execution
|
||||
- Bring the concept to life exactly as described
|
||||
- No text overlays, pure visual storytelling"""
|
||||
else:
|
||||
prompt = f"""Create a viral TikTok video ad for this brand:
|
||||
|
||||
{browser_analysis}
|
||||
|
||||
Create a dynamic, engaging vertical video with:
|
||||
- Quick hook opening that grabs attention immediately
|
||||
- Minimal text overlays (focus on visual storytelling)
|
||||
- Fast-paced but not overwhelming editing
|
||||
- Authentic, relatable energy that appeals to Gen Z
|
||||
- Vertical 9:16 format optimized for mobile
|
||||
- High energy but professional execution
|
||||
|
||||
Style: Modern TikTok advertisement, viral potential, authentic energy, minimal text, maximum visual impact"""
|
||||
return prompt
|
||||
|
||||
async def generate_ad_image(self, prompt: str, screenshot_path: Path | None = None) -> bytes | None:
|
||||
"""Generate ad image bytes using Gemini. Returns None on failure."""
|
||||
try:
|
||||
from typing import Any
|
||||
|
||||
contents: list[Any] = [prompt]
|
||||
|
||||
if screenshot_path and screenshot_path.exists():
|
||||
img = Image.open(screenshot_path)
|
||||
w, h = img.size
|
||||
side = min(w, h)
|
||||
img = img.crop(((w - side) // 2, (h - side) // 2, (w + side) // 2, (h + side) // 2))
|
||||
contents = [prompt + '\n\nHere is the actual landing page screenshot to reference for design inspiration:', img]
|
||||
|
||||
response = await self.client.aio.models.generate_content(
|
||||
model='gemini-2.5-flash-image-preview',
|
||||
contents=contents,
|
||||
)
|
||||
|
||||
cand = getattr(response, 'candidates', None)
|
||||
if cand:
|
||||
for part in getattr(cand[0].content, 'parts', []):
|
||||
inline = getattr(part, 'inline_data', None)
|
||||
if inline:
|
||||
return inline.data
|
||||
except Exception as e:
|
||||
print(f'❌ Image generation failed: {e}')
|
||||
return None
|
||||
|
||||
async def generate_ad_video(self, prompt: str, screenshot_path: Path | None = None, ad_id: int = 1) -> bytes:
|
||||
"""Generate ad video using Veo3."""
|
||||
sync_client = genai.Client(api_key=GOOGLE_API_KEY)
|
||||
|
||||
# Commented out image input for now - it was using the screenshot as first frame
|
||||
# if screenshot_path and screenshot_path.exists():
|
||||
# import base64
|
||||
# import io
|
||||
|
||||
# img = Image.open(screenshot_path)
|
||||
# img_buffer = io.BytesIO()
|
||||
# img.save(img_buffer, format='PNG')
|
||||
# img_bytes = img_buffer.getvalue()
|
||||
|
||||
# operation = sync_client.models.generate_videos(
|
||||
# model='veo-3.0-generate-001',
|
||||
# prompt=prompt,
|
||||
# image=cast(Any, {
|
||||
# 'imageBytes': base64.b64encode(img_bytes).decode('utf-8'),
|
||||
# 'mimeType': 'image/png'
|
||||
# }),
|
||||
# config=cast(Any, {'aspectRatio': '9:16', 'resolution': '720p'}),
|
||||
# )
|
||||
# else:
|
||||
operation = sync_client.models.generate_videos(
|
||||
model='veo-3.0-generate-001',
|
||||
prompt=prompt,
|
||||
config=cast(Any, {'aspectRatio': '9:16', 'resolution': '720p'}),
|
||||
)
|
||||
|
||||
while not operation.done:
|
||||
await asyncio.sleep(10)
|
||||
operation = sync_client.operations.get(operation)
|
||||
|
||||
if not operation.response or not operation.response.generated_videos:
|
||||
raise RuntimeError('No videos generated')
|
||||
videos = operation.response.generated_videos
|
||||
video = videos[0]
|
||||
video_file = getattr(video, 'video', None)
|
||||
if not video_file:
|
||||
raise RuntimeError('No video file in response')
|
||||
sync_client.files.download(file=video_file)
|
||||
video_bytes = getattr(video_file, 'video_bytes', None)
|
||||
if not video_bytes:
|
||||
raise RuntimeError('No video bytes in response')
|
||||
return video_bytes
|
||||
|
||||
async def save_results(self, ad_content: bytes, prompt: str, analysis: str, url: str, timestamp: str) -> str:
|
||||
if self.mode == 'instagram':
|
||||
content_path = self.output_dir / f'ad_{timestamp}.png'
|
||||
else: # tiktok
|
||||
content_path = self.output_dir / f'ad_{timestamp}.mp4'
|
||||
|
||||
async with aiofiles.open(content_path, 'wb') as f:
|
||||
await f.write(ad_content)
|
||||
|
||||
analysis_path = self.output_dir / f'analysis_{timestamp}.txt'
|
||||
async with aiofiles.open(analysis_path, 'w', encoding='utf-8') as f:
|
||||
await f.write(f'URL: {url}\n\n')
|
||||
await f.write('BROWSER-USE ANALYSIS:\n')
|
||||
await f.write(analysis)
|
||||
await f.write('\n\nGENERATED PROMPT:\n')
|
||||
await f.write(prompt)
|
||||
|
||||
return str(content_path)
|
||||
|
||||
|
||||
def open_file(file_path: str):
|
||||
"""Open file with default system viewer"""
|
||||
try:
|
||||
if sys.platform.startswith('darwin'):
|
||||
subprocess.run(['open', file_path], check=True)
|
||||
elif sys.platform.startswith('win'):
|
||||
subprocess.run(['cmd', '/c', 'start', '', file_path], check=True)
|
||||
else:
|
||||
subprocess.run(['xdg-open', file_path], check=True)
|
||||
except Exception as e:
|
||||
print(f'❌ Could not open file: {e}')
|
||||
|
||||
|
||||
async def create_ad_from_landing_page(url: str, debug: bool = False, mode: str = 'instagram', ad_id: int = 1):
|
||||
analyzer = LandingPageAnalyzer(debug=debug)
|
||||
|
||||
try:
|
||||
if ad_id == 1:
|
||||
print(f'🚀 Analyzing {url} for {mode.capitalize()} ad...')
|
||||
page_data = await analyzer.analyze_landing_page(url, mode=mode)
|
||||
else:
|
||||
analyzer_temp = LandingPageAnalyzer(debug=debug)
|
||||
page_data = await analyzer_temp.analyze_landing_page(url, mode=mode)
|
||||
|
||||
generator = AdGenerator(mode=mode)
|
||||
|
||||
if mode == 'instagram':
|
||||
prompt = generator.create_ad_prompt(page_data['analysis'])
|
||||
ad_content = await generator.generate_ad_image(prompt, page_data.get('screenshot_path'))
|
||||
if ad_content is None:
|
||||
raise RuntimeError(f'Ad image generation failed for ad #{ad_id}')
|
||||
else: # tiktok
|
||||
video_concept = await generator.create_video_concept(page_data['analysis'], ad_id)
|
||||
prompt = generator.create_ad_prompt(page_data['analysis'], video_concept)
|
||||
ad_content = await generator.generate_ad_video(prompt, page_data.get('screenshot_path'), ad_id)
|
||||
|
||||
result_path = await generator.save_results(ad_content, prompt, page_data['analysis'], url, page_data['timestamp'])
|
||||
|
||||
if mode == 'instagram':
|
||||
print(f'🎨 Generated image ad #{ad_id}: {result_path}')
|
||||
else:
|
||||
print(f'🎬 Generated video ad #{ad_id}: {result_path}')
|
||||
|
||||
open_file(result_path)
|
||||
|
||||
return result_path
|
||||
|
||||
except Exception as e:
|
||||
print(f'❌ Error for ad #{ad_id}: {e}')
|
||||
raise
|
||||
finally:
|
||||
if ad_id == 1 and page_data.get('screenshot_path'):
|
||||
print(f'📸 Page screenshot: {page_data["screenshot_path"]}')
|
||||
|
||||
|
||||
async def generate_single_ad(page_data: dict, mode: str, ad_id: int):
|
||||
"""Generate a single ad using pre-analyzed page data"""
|
||||
generator = AdGenerator(mode=mode)
|
||||
|
||||
try:
|
||||
if mode == 'instagram':
|
||||
prompt = generator.create_ad_prompt(page_data['analysis'])
|
||||
ad_content = await generator.generate_ad_image(prompt, page_data.get('screenshot_path'))
|
||||
if ad_content is None:
|
||||
raise RuntimeError(f'Ad image generation failed for ad #{ad_id}')
|
||||
else: # tiktok
|
||||
video_concept = await generator.create_video_concept(page_data['analysis'], ad_id)
|
||||
prompt = generator.create_ad_prompt(page_data['analysis'], video_concept)
|
||||
ad_content = await generator.generate_ad_video(prompt, page_data.get('screenshot_path'), ad_id)
|
||||
|
||||
# Create unique timestamp for each ad
|
||||
timestamp = datetime.now().strftime('%Y%m%d_%H%M%S') + f'_{ad_id}'
|
||||
result_path = await generator.save_results(ad_content, prompt, page_data['analysis'], page_data['url'], timestamp)
|
||||
|
||||
if mode == 'instagram':
|
||||
print(f'🎨 Generated image ad #{ad_id}: {result_path}')
|
||||
else:
|
||||
print(f'🎬 Generated video ad #{ad_id}: {result_path}')
|
||||
|
||||
return result_path
|
||||
|
||||
except Exception as e:
|
||||
print(f'❌ Error for ad #{ad_id}: {e}')
|
||||
raise
|
||||
|
||||
|
||||
async def create_multiple_ads(url: str, debug: bool = False, mode: str = 'instagram', count: int = 1):
|
||||
"""Generate multiple ads in parallel using asyncio concurrency"""
|
||||
if count == 1:
|
||||
return await create_ad_from_landing_page(url, debug, mode, 1)
|
||||
|
||||
print(f'🚀 Analyzing {url} for {count} {mode} ads...')
|
||||
|
||||
analyzer = LandingPageAnalyzer(debug=debug)
|
||||
page_data = await analyzer.analyze_landing_page(url, mode=mode)
|
||||
|
||||
print(f'🎯 Generating {count} {mode} ads in parallel...')
|
||||
|
||||
tasks = []
|
||||
for i in range(count):
|
||||
task = create_task_with_error_handling(generate_single_ad(page_data, mode, i + 1), name=f'generate_ad_{i + 1}')
|
||||
tasks.append(task)
|
||||
|
||||
results = await asyncio.gather(*tasks, return_exceptions=True)
|
||||
|
||||
successful = []
|
||||
failed = []
|
||||
|
||||
for i, result in enumerate(results):
|
||||
if isinstance(result, Exception):
|
||||
failed.append(i + 1)
|
||||
else:
|
||||
successful.append(result)
|
||||
|
||||
print(f'\n✅ Successfully generated {len(successful)}/{count} ads')
|
||||
if failed:
|
||||
print(f'❌ Failed ads: {failed}')
|
||||
|
||||
if page_data.get('screenshot_path'):
|
||||
print(f'📸 Page screenshot: {page_data["screenshot_path"]}')
|
||||
|
||||
for ad_path in successful:
|
||||
open_file(ad_path)
|
||||
|
||||
return successful
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
url = args.url
|
||||
if not url:
|
||||
url = input('🔗 Enter URL: ').strip() or 'https://www.apple.com/iphone-17-pro/'
|
||||
|
||||
if args.tiktok:
|
||||
mode = 'tiktok'
|
||||
else:
|
||||
mode = 'instagram'
|
||||
|
||||
asyncio.run(create_multiple_ads(url, debug=args.debug, mode=mode, count=args.count))
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 1.4 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.5 MiB |
@@ -0,0 +1,114 @@
|
||||
# Msg-Use
|
||||
|
||||
AI-powered message scheduler using browser agents and Gemini. Schedule personalized messages in natural language and let AI compose them intelligently.
|
||||
|
||||
[!WARNING]
|
||||
This demo requires browser-use v0.7.7+.
|
||||
|
||||
https://browser-use.github.io/media/demos/msg_use.mp4
|
||||
|
||||
## Features
|
||||
|
||||
1. Agent logs into WhatsApp Web automatically
|
||||
2. Parses natural language scheduling instructions
|
||||
3. Composes personalized messages using AI
|
||||
4. Schedules messages for future delivery or sends immediately
|
||||
5. Persistent session (no repeated QR scanning)
|
||||
|
||||
## Setup
|
||||
|
||||
Make sure the newest version of browser-use is installed:
|
||||
```bash
|
||||
pip install -U browser-use
|
||||
```
|
||||
|
||||
Export your Gemini API key, get it from: [Google AI Studio](https://makersuite.google.com/app/apikey)
|
||||
```
|
||||
export GOOGLE_API_KEY='your-gemini-api-key-here'
|
||||
```
|
||||
|
||||
Clone the repo and cd into the app folder
|
||||
```bash
|
||||
git clone https://github.com/browser-use/browser-use.git
|
||||
cd browser-use/examples/apps/msg-use
|
||||
```
|
||||
|
||||
## Initial Login
|
||||
|
||||
First-time setup requires QR code scanning:
|
||||
```bash
|
||||
python login.py
|
||||
```
|
||||
- Scan QR code when browser opens
|
||||
- Session will be saved for future use
|
||||
|
||||
## Normal Usage
|
||||
|
||||
1. **Edit your schedule** in `messages.txt`:
|
||||
```
|
||||
- Send "Hi" to Magnus on the 09.09 at 18:15
|
||||
- Tell hinge date (Camila) at 20:00 that I miss her
|
||||
- Remind mom to pick up the car next tuesday
|
||||
```
|
||||
|
||||
2. **Test mode** - See what will be sent:
|
||||
```bash
|
||||
python scheduler.py --test
|
||||
```
|
||||
|
||||
3. **Run scheduler**:
|
||||
```bash
|
||||
python scheduler.py
|
||||
|
||||
# Debug Mode - See the browser in action
|
||||
python scheduler.py --debug
|
||||
|
||||
# Auto Mode - Respond to unread messages every ~30 minutes
|
||||
python scheduler.py --auto
|
||||
```
|
||||
|
||||
## Programmatic Usage
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
from scheduler import schedule_messages
|
||||
|
||||
async def main():
|
||||
messages = [
|
||||
"Send hello to John at 15:30",
|
||||
"Remind Sarah about meeting tomorrow at 9am"
|
||||
]
|
||||
await schedule_messages(messages, debug=False)
|
||||
|
||||
asyncio.run(main())
|
||||
```
|
||||
|
||||
## Output
|
||||
|
||||
Example scheduling output:
|
||||
```json
|
||||
[
|
||||
{
|
||||
"contact": "Magnus",
|
||||
"original_message": "Hi",
|
||||
"composed_message": "Hi",
|
||||
"scheduled_time": "2025-06-13 18:15"
|
||||
},
|
||||
{
|
||||
"contact": "Camila",
|
||||
"original_message": "I miss her",
|
||||
"composed_message": "I miss you ❤️",
|
||||
"scheduled_time": "2025-06-14 20:00"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
## Files
|
||||
|
||||
- `scheduler.py` - Main scheduler script
|
||||
- `login.py` - One-time login setup
|
||||
- `messages.txt` - Your message schedule in natural language
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
@@ -0,0 +1,71 @@
|
||||
import asyncio
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from browser_use import Agent, BrowserSession
|
||||
from browser_use.llm.google import ChatGoogle
|
||||
|
||||
GOOGLE_API_KEY = os.getenv('GOOGLE_API_KEY')
|
||||
|
||||
# Browser profile directory for persistence (same as main script)
|
||||
USER_DATA_DIR = Path.home() / '.config' / 'whatsapp_scheduler' / 'browser_profile'
|
||||
USER_DATA_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Storage state file for cookies
|
||||
STORAGE_STATE_FILE = USER_DATA_DIR / 'storage_state.json'
|
||||
|
||||
|
||||
async def login_to_whatsapp():
|
||||
"""Open WhatsApp Web and wait for user to scan QR code"""
|
||||
if not GOOGLE_API_KEY:
|
||||
print('❌ Error: GOOGLE_API_KEY environment variable is required')
|
||||
print("Please set it with: export GOOGLE_API_KEY='your-api-key-here'")
|
||||
return
|
||||
|
||||
print('WhatsApp Login Setup')
|
||||
print('=' * 50)
|
||||
print(f'Browser profile directory: {USER_DATA_DIR}')
|
||||
print(f'Storage state file: {STORAGE_STATE_FILE}')
|
||||
print('=' * 50)
|
||||
|
||||
try:
|
||||
llm = ChatGoogle(model='gemini-2.0-flash-exp', temperature=0.3, api_key=GOOGLE_API_KEY)
|
||||
|
||||
task = """
|
||||
You are helping a user log into WhatsApp Web. Follow these steps:
|
||||
|
||||
1. Navigate to https://web.whatsapp.com
|
||||
2. Wait for the page to load completely
|
||||
3. If you see a QR code, tell the user to scan it with their phone
|
||||
4. Wait patiently for the login to complete
|
||||
5. Once you see the WhatsApp chat interface, confirm successful login
|
||||
|
||||
Take your time and be patient with page loads.
|
||||
"""
|
||||
|
||||
print('\nOpening WhatsApp Web...')
|
||||
print('Please scan the QR code when it appears.\n')
|
||||
|
||||
browser_session = BrowserSession(
|
||||
headless=False, # Show browser
|
||||
user_data_dir=str(USER_DATA_DIR), # Use persistent profile directory
|
||||
storage_state=str(STORAGE_STATE_FILE) if STORAGE_STATE_FILE.exists() else None, # Use saved cookies/session
|
||||
)
|
||||
|
||||
agent = Agent(task=task, llm=llm, browser_session=browser_session)
|
||||
|
||||
result = await agent.run()
|
||||
|
||||
print('\n✅ Login completed!')
|
||||
print("Note: For now, you'll need to scan the QR code each time.")
|
||||
print("We'll improve session persistence in a future update.")
|
||||
print('\nPress Enter to close the browser...')
|
||||
input()
|
||||
|
||||
except Exception as e:
|
||||
print(f'\n❌ Error during login: {str(e)}')
|
||||
print('Please try again.')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
asyncio.run(login_to_whatsapp())
|
||||
Executable
+286
@@ -0,0 +1,286 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
WhatsApp Message Scheduler - Send scheduled messages via WhatsApp Web
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import random
|
||||
import re
|
||||
from datetime import datetime, timedelta
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def setup_environment(debug: bool):
|
||||
if not debug:
|
||||
os.environ['BROWSER_USE_SETUP_LOGGING'] = 'false'
|
||||
os.environ['BROWSER_USE_LOGGING_LEVEL'] = 'critical'
|
||||
logging.getLogger().setLevel(logging.CRITICAL)
|
||||
else:
|
||||
os.environ['BROWSER_USE_SETUP_LOGGING'] = 'true'
|
||||
os.environ['BROWSER_USE_LOGGING_LEVEL'] = 'info'
|
||||
|
||||
|
||||
parser = argparse.ArgumentParser(description='WhatsApp Scheduler - Send scheduled messages via WhatsApp Web')
|
||||
parser.add_argument('--debug', action='store_true', help='Debug mode: show browser and verbose logs')
|
||||
parser.add_argument('--test', action='store_true', help='Test mode: show what messages would be sent without sending them')
|
||||
parser.add_argument('--auto', action='store_true', help='Auto mode: respond to unread messages every 30 minutes')
|
||||
args = parser.parse_args()
|
||||
setup_environment(args.debug)
|
||||
|
||||
from browser_use import Agent, BrowserSession
|
||||
from browser_use.llm.google import ChatGoogle
|
||||
|
||||
GOOGLE_API_KEY = os.getenv('GOOGLE_API_KEY') or os.getenv('GEMINI_API_KEY')
|
||||
|
||||
USER_DATA_DIR = Path.home() / '.config' / 'whatsapp_scheduler' / 'browser_profile'
|
||||
USER_DATA_DIR.mkdir(parents=True, exist_ok=True)
|
||||
STORAGE_STATE_FILE = USER_DATA_DIR / 'storage_state.json'
|
||||
|
||||
|
||||
async def parse_messages():
|
||||
"""Parse messages.txt and extract scheduling info"""
|
||||
messages_file = Path('messages.txt')
|
||||
if not messages_file.exists():
|
||||
print('❌ messages.txt not found!')
|
||||
return []
|
||||
|
||||
import aiofiles
|
||||
|
||||
async with aiofiles.open(messages_file) as f:
|
||||
content = await f.read()
|
||||
|
||||
llm = ChatGoogle(model='gemini-2.0-flash-exp', temperature=0.1, api_key=GOOGLE_API_KEY)
|
||||
|
||||
now = datetime.now()
|
||||
prompt = f"""
|
||||
Parse these WhatsApp message instructions and extract:
|
||||
1. Contact name (extract just the name, not descriptions)
|
||||
2. Message content (what to send)
|
||||
3. Date and time (when to send)
|
||||
|
||||
Current date/time: {now.strftime('%Y-%m-%d %H:%M')}
|
||||
Today is: {now.strftime('%Y-%m-%d')}
|
||||
Current time is: {now.strftime('%H:%M')}
|
||||
|
||||
Instructions:
|
||||
{content}
|
||||
|
||||
Return ONLY a JSON array with format:
|
||||
[{{"contact": "name", "message": "text", "datetime": "YYYY-MM-DD HH:MM"}}]
|
||||
|
||||
CRITICAL: Transform instructions into actual messages:
|
||||
|
||||
QUOTED TEXT → Use exactly as-is:
|
||||
- Text in "quotes" becomes the exact message
|
||||
|
||||
UNQUOTED INSTRUCTIONS → Generate actual content:
|
||||
- If it's an instruction to write something → write the actual thing
|
||||
- If it's an instruction to tell someone something → write what to tell them
|
||||
- If it's an instruction to remind someone → write the actual reminder
|
||||
- For multi-line content like poems: use single line with spacing, not line breaks
|
||||
|
||||
DO NOT copy the instruction - create the actual message content!
|
||||
|
||||
Time Rules:
|
||||
- If only time given (like "at 15:30"), use TODAY
|
||||
- If no date specified, assume TODAY
|
||||
- If no year given, use current year
|
||||
- Default time is 9:00 if not specified
|
||||
- Extract names from parentheses: "hinge date (Camila)" → "Camila"
|
||||
- "tomorrow" means {(now + timedelta(days=1)).strftime('%Y-%m-%d')}
|
||||
- "next tuesday" or similar means the next occurrence of that day
|
||||
"""
|
||||
|
||||
from browser_use.llm.messages import UserMessage
|
||||
|
||||
response = await llm.ainvoke([UserMessage(content=prompt)])
|
||||
response_text = response.completion if hasattr(response, 'completion') else str(response)
|
||||
|
||||
# Extract JSON
|
||||
json_match = re.search(r'\[.*?\]', response_text, re.DOTALL)
|
||||
if json_match:
|
||||
try:
|
||||
messages = json.loads(json_match.group())
|
||||
for msg in messages:
|
||||
if 'message' in msg:
|
||||
msg['message'] = re.sub(r'\n+', ' • ', msg['message'])
|
||||
msg['message'] = re.sub(r'\s+', ' ', msg['message']).strip()
|
||||
return messages
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
return []
|
||||
|
||||
|
||||
async def send_message(contact, message):
|
||||
"""Send a WhatsApp message"""
|
||||
print(f'\n📱 Sending to {contact}: {message}')
|
||||
|
||||
llm = ChatGoogle(model='gemini-2.0-flash-exp', temperature=0.3, api_key=GOOGLE_API_KEY)
|
||||
|
||||
task = f"""
|
||||
Send WhatsApp message:
|
||||
1. Go to https://web.whatsapp.com
|
||||
2. Search for contact: {contact}
|
||||
3. Click on the contact
|
||||
4. Type message: {message}
|
||||
5. Press Enter to send
|
||||
6. Confirm sent
|
||||
"""
|
||||
|
||||
browser = BrowserSession(
|
||||
headless=not args.debug, # headless=False only when debug=True
|
||||
user_data_dir=str(USER_DATA_DIR),
|
||||
storage_state=str(STORAGE_STATE_FILE) if STORAGE_STATE_FILE.exists() else None,
|
||||
)
|
||||
|
||||
agent = Agent(task=task, llm=llm, browser_session=browser)
|
||||
await agent.run()
|
||||
print(f'✅ Sent to {contact}')
|
||||
|
||||
|
||||
async def auto_respond_to_unread():
|
||||
"""Click unread tab and respond to messages"""
|
||||
print('\nAuto-responding to unread messages...')
|
||||
|
||||
llm = ChatGoogle(model='gemini-2.0-flash-exp', temperature=0.3, api_key=GOOGLE_API_KEY)
|
||||
|
||||
task = """
|
||||
1. Go to https://web.whatsapp.com
|
||||
2. Wait for page to load
|
||||
3. Click on the "Unread" filter tab
|
||||
4. If there are unread messages:
|
||||
- Click on each unread chat
|
||||
- Read the last message
|
||||
- Generate and send a friendly, contextual response
|
||||
- Move to next unread chat
|
||||
5. Report how many messages were responded to
|
||||
"""
|
||||
|
||||
browser = BrowserSession(
|
||||
headless=not args.debug,
|
||||
user_data_dir=str(USER_DATA_DIR),
|
||||
storage_state=str(STORAGE_STATE_FILE) if STORAGE_STATE_FILE.exists() else None,
|
||||
)
|
||||
|
||||
agent = Agent(task=task, llm=llm, browser_session=browser)
|
||||
result = await agent.run()
|
||||
print('✅ Auto-response complete')
|
||||
return result
|
||||
|
||||
|
||||
async def main():
|
||||
if not GOOGLE_API_KEY:
|
||||
print('❌ Set GOOGLE_API_KEY or GEMINI_API_KEY environment variable')
|
||||
return
|
||||
|
||||
print('WhatsApp Scheduler')
|
||||
print(f'Profile: {USER_DATA_DIR}')
|
||||
print()
|
||||
|
||||
# Auto mode - respond to unread messages periodically
|
||||
if args.auto:
|
||||
print('AUTO MODE - Responding to unread messages every ~30 minutes')
|
||||
print('Press Ctrl+C to stop.\n')
|
||||
|
||||
while True:
|
||||
try:
|
||||
await auto_respond_to_unread()
|
||||
|
||||
# Wait 30 minutes +/- 5 minutes randomly
|
||||
wait_minutes = 30 + random.randint(-5, 5)
|
||||
print(f'\n⏰ Next check in {wait_minutes} minutes...')
|
||||
await asyncio.sleep(wait_minutes * 60)
|
||||
|
||||
except KeyboardInterrupt:
|
||||
print('\n\nAuto mode stopped by user')
|
||||
break
|
||||
except Exception as e:
|
||||
print(f'\n❌ Error in auto mode: {e}')
|
||||
print('Waiting 5 minutes before retry...')
|
||||
await asyncio.sleep(300)
|
||||
return
|
||||
|
||||
# Parse messages
|
||||
print('Parsing messages.txt...')
|
||||
messages = await parse_messages()
|
||||
|
||||
if not messages:
|
||||
print('No messages found')
|
||||
return
|
||||
|
||||
print(f'\nFound {len(messages)} messages:')
|
||||
for msg in messages:
|
||||
print(f' • {msg["datetime"]}: {msg["message"][:30]}... to {msg["contact"]}')
|
||||
|
||||
now = datetime.now()
|
||||
immediate = []
|
||||
future = []
|
||||
|
||||
for msg in messages:
|
||||
msg_time = datetime.strptime(msg['datetime'], '%Y-%m-%d %H:%M')
|
||||
if msg_time <= now:
|
||||
immediate.append(msg)
|
||||
else:
|
||||
future.append(msg)
|
||||
|
||||
if args.test:
|
||||
print('\n=== TEST MODE - Preview ===')
|
||||
if immediate:
|
||||
print(f'\nWould send {len(immediate)} past-due messages NOW:')
|
||||
for msg in immediate:
|
||||
print(f' 📱 To {msg["contact"]}: {msg["message"]}')
|
||||
if future:
|
||||
print(f'\nWould monitor {len(future)} future messages:')
|
||||
for msg in future:
|
||||
print(f' ⏰ {msg["datetime"]}: To {msg["contact"]}: {msg["message"]}')
|
||||
print('\nTest mode complete. No messages sent.')
|
||||
return
|
||||
|
||||
if immediate:
|
||||
print(f'\nSending {len(immediate)} past-due messages NOW...')
|
||||
for msg in immediate:
|
||||
await send_message(msg['contact'], msg['message'])
|
||||
|
||||
if future:
|
||||
print(f'\n⏰ Monitoring {len(future)} future messages...')
|
||||
print('Press Ctrl+C to stop.\n')
|
||||
|
||||
last_status = None
|
||||
|
||||
while future:
|
||||
now = datetime.now()
|
||||
due = []
|
||||
remaining = []
|
||||
|
||||
for msg in future:
|
||||
msg_time = datetime.strptime(msg['datetime'], '%Y-%m-%d %H:%M')
|
||||
if msg_time <= now:
|
||||
due.append(msg)
|
||||
else:
|
||||
remaining.append(msg)
|
||||
|
||||
for msg in due:
|
||||
print(f'\n⏰ Time reached for {msg["contact"]}')
|
||||
await send_message(msg['contact'], msg['message'])
|
||||
|
||||
future = remaining
|
||||
|
||||
if future:
|
||||
next_msg = min(future, key=lambda x: datetime.strptime(x['datetime'], '%Y-%m-%d %H:%M'))
|
||||
current_status = f'Next: {next_msg["datetime"]} to {next_msg["contact"]}'
|
||||
|
||||
if current_status != last_status:
|
||||
print(current_status)
|
||||
last_status = current_status
|
||||
|
||||
await asyncio.sleep(30) # Check every 30 seconds
|
||||
|
||||
print('\n✅ All messages processed!')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,87 @@
|
||||
# News-Use
|
||||
|
||||
Automatically monitor news websites and extract the latest articles with sentiment analysis using browser agents and Google Gemini.
|
||||
|
||||
> [!IMPORTANT]
|
||||
> This demo requires browser-use v0.7.7+.
|
||||
|
||||
https://github.com/user-attachments/assets/698757ca-8827-41f3-98e5-c235d6eef69f
|
||||
|
||||
## Features
|
||||
|
||||
1. Agent visits any news website
|
||||
2. Finds and clicks the most recent headline article
|
||||
3. Extracts title, URL, posting time, and content
|
||||
4. Generates short/long summaries with sentiment analysis
|
||||
5. Persistent deduplication across restarts
|
||||
|
||||
## Setup
|
||||
|
||||
Make sure the newest version of browser-use is installed:
|
||||
```bash
|
||||
pip install -U browser-use
|
||||
```
|
||||
|
||||
Export your Gemini API key, get it from: [Google AI Studio](https://makersuite.google.com/app/apikey)
|
||||
```
|
||||
export GEMINI_API_KEY='your-google-api-key-here'
|
||||
```
|
||||
|
||||
Clone the repo and cd into the app folder
|
||||
```bash
|
||||
git clone https://github.com/browser-use/browser-use.git
|
||||
cd browser-use/examples/apps/news-use
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
# One-time extraction - Get the latest article and exit
|
||||
python news_monitor.py --once
|
||||
|
||||
# Continuous monitoring - Check every 5 minutes (default)
|
||||
python news_monitor.py
|
||||
|
||||
# Custom interval - Check every 60 seconds
|
||||
python news_monitor.py --interval 60
|
||||
|
||||
# Different news site
|
||||
python news_monitor.py --url https://techcrunch.com
|
||||
|
||||
# Debug mode - See browser in action with verbose output
|
||||
python news_monitor.py --once --debug
|
||||
```
|
||||
|
||||
## Output Format
|
||||
|
||||
Articles are displayed with timestamp, sentiment emoji, and summary:
|
||||
```
|
||||
[2025-09-11 02:49:21] - 🟢 - Klarna's IPO raises $1.4B, benefiting existing investors
|
||||
```
|
||||
|
||||
Sentiment indicators:
|
||||
- 🟢 Positive
|
||||
- 🟡 Neutral
|
||||
- 🔴 Negative
|
||||
|
||||
## Programmatic Usage
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
from news_monitor import extract_latest_article
|
||||
|
||||
async def main():
|
||||
result = await extract_latest_article(
|
||||
site_url="https://techcrunch.com",
|
||||
debug=False
|
||||
)
|
||||
if result["status"] == "success":
|
||||
article = result["data"]
|
||||
print(f"Latest: {article['title']}")
|
||||
|
||||
asyncio.run(main())
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
Executable
+303
@@ -0,0 +1,303 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
News monitoring agent with browser-use + Gemini Flash.
|
||||
Automatically extracts and analyzes the latest articles from any news website.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import hashlib
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import time
|
||||
from datetime import datetime
|
||||
from typing import Literal
|
||||
|
||||
from dateutil import parser as dtparser
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
def setup_environment(debug: bool):
|
||||
if not debug:
|
||||
os.environ['BROWSER_USE_SETUP_LOGGING'] = 'false'
|
||||
os.environ['BROWSER_USE_LOGGING_LEVEL'] = 'critical'
|
||||
logging.getLogger().setLevel(logging.CRITICAL)
|
||||
else:
|
||||
os.environ['BROWSER_USE_SETUP_LOGGING'] = 'true'
|
||||
os.environ['BROWSER_USE_LOGGING_LEVEL'] = 'info'
|
||||
|
||||
|
||||
parser = argparse.ArgumentParser(description='News extractor using Browser-Use + Gemini')
|
||||
parser.add_argument('--url', default='https://www.techcrunch.com', help='News site root URL')
|
||||
parser.add_argument('--interval', type=int, default=300, help='Seconds between checks in monitor mode')
|
||||
parser.add_argument('--once', action='store_true', help='Run a single extraction and exit')
|
||||
parser.add_argument('--output', default='news_data.json', help='Path to JSON file where articles are stored')
|
||||
parser.add_argument('--debug', action='store_true', help='Verbose console output and non-headless browser')
|
||||
args = parser.parse_args()
|
||||
|
||||
setup_environment(args.debug)
|
||||
|
||||
from browser_use import Agent, BrowserSession, ChatGoogle
|
||||
|
||||
GEMINI_API_KEY = os.getenv('GOOGLE_API_KEY') or 'xxxx'
|
||||
|
||||
if GEMINI_API_KEY == 'xxxx':
|
||||
print('⚠️ WARNING: Please set GOOGLE_API_KEY environment variable')
|
||||
print(' You can get an API key at: https://makersuite.google.com/app/apikey')
|
||||
print(" Then run: export GEMINI_API_KEY='your-api-key-here'")
|
||||
print()
|
||||
|
||||
|
||||
class NewsArticle(BaseModel):
|
||||
title: str
|
||||
url: str
|
||||
posting_time: str
|
||||
short_summary: str
|
||||
long_summary: str
|
||||
sentiment: Literal['positive', 'neutral', 'negative']
|
||||
|
||||
|
||||
# ---------------------------------------------------------
|
||||
# Core extractor
|
||||
# ---------------------------------------------------------
|
||||
|
||||
|
||||
async def extract_latest_article(site_url: str, debug: bool = False) -> dict:
|
||||
"""Open site_url, navigate to the newest article and return structured JSON."""
|
||||
|
||||
prompt = (
|
||||
f'Navigate to {site_url} and find the most recent headline article (usually at the top). '
|
||||
f'Click on it to open the full article page. Once loaded, scroll & extract ALL required information: '
|
||||
f'1. title: The article headline '
|
||||
f'2. url: The full URL of the article page '
|
||||
f'3. posting_time: The publication date/time as shown on the page '
|
||||
f"4. short_summary: A 10-word overview of the article's content "
|
||||
f'5. long_summary: A 100-word detailed summary of the article '
|
||||
f"6. sentiment: Classify as 'positive', 'neutral', or 'negative' based on the article tone. "
|
||||
f'When done, call the done action with success=True and put ALL extracted data in the text field '
|
||||
f'as valid JSON in this exact format: '
|
||||
f'{{"title": "...", "url": "...", "posting_time": "...", "short_summary": "...", "long_summary": "...", "sentiment": "positive|neutral|negative"}}'
|
||||
)
|
||||
|
||||
llm = ChatGoogle(model='gemini-2.0-flash', temperature=0.1, api_key=GEMINI_API_KEY)
|
||||
browser_session = BrowserSession(headless=not debug)
|
||||
|
||||
agent = Agent(task=prompt, llm=llm, browser_session=browser_session, use_vision=False)
|
||||
|
||||
if debug:
|
||||
print(f'[DEBUG] Starting extraction from {site_url}')
|
||||
start = time.time()
|
||||
|
||||
result = await agent.run(max_steps=25)
|
||||
|
||||
raw = result.final_result() if result else None
|
||||
if debug:
|
||||
print(f'[DEBUG] Raw result type: {type(raw)}')
|
||||
print(f'[DEBUG] Raw result: {raw[:500] if isinstance(raw, str) else raw}')
|
||||
print(f'[DEBUG] Extraction time: {time.time() - start:.2f}s')
|
||||
|
||||
if isinstance(raw, dict):
|
||||
return {'status': 'success', 'data': raw}
|
||||
|
||||
text = str(raw).strip() if raw else ''
|
||||
|
||||
if '<json>' in text and '</json>' in text:
|
||||
text = text.split('<json>', 1)[1].split('</json>', 1)[0].strip()
|
||||
|
||||
if text.lower().startswith('here is'):
|
||||
brace = text.find('{')
|
||||
if brace != -1:
|
||||
text = text[brace:]
|
||||
|
||||
if text.startswith('```'):
|
||||
text = text.lstrip('`\n ')
|
||||
if text.lower().startswith('json'):
|
||||
text = text[4:].lstrip()
|
||||
|
||||
def _escape_newlines(src: str) -> str:
|
||||
out, in_str, esc = [], False, False
|
||||
for ch in src:
|
||||
if in_str:
|
||||
if esc:
|
||||
esc = False
|
||||
elif ch == '\\':
|
||||
esc = True
|
||||
elif ch == '"':
|
||||
in_str = False
|
||||
elif ch == '\n':
|
||||
out.append('\\n')
|
||||
continue
|
||||
elif ch == '\r':
|
||||
continue
|
||||
else:
|
||||
if ch == '"':
|
||||
in_str = True
|
||||
out.append(ch)
|
||||
return ''.join(out)
|
||||
|
||||
cleaned = _escape_newlines(text)
|
||||
|
||||
def _try_parse(txt: str):
|
||||
try:
|
||||
return json.loads(txt)
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
data = _try_parse(cleaned)
|
||||
|
||||
# Fallback: grab first balanced JSON object
|
||||
if data is None:
|
||||
brace = 0
|
||||
start = None
|
||||
for i, ch in enumerate(text):
|
||||
if ch == '{':
|
||||
if brace == 0:
|
||||
start = i
|
||||
brace += 1
|
||||
elif ch == '}':
|
||||
brace -= 1
|
||||
if brace == 0 and start is not None:
|
||||
candidate = _escape_newlines(text[start : i + 1])
|
||||
data = _try_parse(candidate)
|
||||
if data is not None:
|
||||
break
|
||||
|
||||
if isinstance(data, dict):
|
||||
return {'status': 'success', 'data': data}
|
||||
return {'status': 'error', 'error': f'JSON parse failed. Raw head: {text[:200]}'}
|
||||
|
||||
|
||||
# ---------------------------------------------------------
|
||||
# Persistence helpers
|
||||
# ---------------------------------------------------------
|
||||
|
||||
|
||||
def load_seen_hashes(file_path: str = 'news_data.json') -> set:
|
||||
"""Load already-saved article URL hashes from disk for dedup across restarts."""
|
||||
if not os.path.exists(file_path):
|
||||
return set()
|
||||
try:
|
||||
with open(file_path) as f:
|
||||
items = json.load(f)
|
||||
return {entry['hash'] for entry in items if 'hash' in entry}
|
||||
except Exception:
|
||||
return set()
|
||||
|
||||
|
||||
def save_article(article: dict, file_path: str = 'news_data.json'):
|
||||
"""Append article to disk with a hash for future dedup."""
|
||||
payload = {
|
||||
'hash': hashlib.md5(article['url'].encode()).hexdigest(),
|
||||
'pulled_at': time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime()),
|
||||
'data': article,
|
||||
}
|
||||
|
||||
existing = []
|
||||
if os.path.exists(file_path):
|
||||
try:
|
||||
with open(file_path) as f:
|
||||
existing = json.load(f)
|
||||
except Exception:
|
||||
existing = []
|
||||
|
||||
existing.append(payload)
|
||||
# Keep last 100
|
||||
existing = existing[-100:]
|
||||
|
||||
with open(file_path, 'w') as f:
|
||||
json.dump(existing, f, ensure_ascii=False, indent=2)
|
||||
|
||||
|
||||
# ---------------------------------------------------------
|
||||
# CLI functions
|
||||
# ---------------------------------------------------------
|
||||
|
||||
|
||||
def _fmt(ts_raw: str) -> str:
|
||||
"""Format timestamp string"""
|
||||
try:
|
||||
return dtparser.parse(ts_raw).strftime('%Y-%m-%d %H:%M:%S')
|
||||
except Exception:
|
||||
return datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S')
|
||||
|
||||
|
||||
async def run_once(url: str, output_path: str, debug: bool):
|
||||
"""Run a single extraction and exit"""
|
||||
res = await extract_latest_article(url, debug)
|
||||
|
||||
if res['status'] == 'success':
|
||||
art = res['data']
|
||||
url_val = art.get('url', '')
|
||||
hash_ = hashlib.md5(url_val.encode()).hexdigest() if url_val else None
|
||||
if url_val:
|
||||
save_article(art, output_path)
|
||||
ts = _fmt(art.get('posting_time', ''))
|
||||
sentiment = art.get('sentiment', 'neutral')
|
||||
emoji = {'positive': '🟢', 'negative': '🔴', 'neutral': '🟡'}.get(sentiment, '🟡')
|
||||
summary = art.get('short_summary', art.get('summary', art.get('title', '')))
|
||||
if debug:
|
||||
print(json.dumps(art, ensure_ascii=False, indent=2))
|
||||
print()
|
||||
print(f'[{ts}] - {emoji} - {summary}')
|
||||
if not debug:
|
||||
print() # Only add spacing in non-debug mode
|
||||
return hash_
|
||||
else:
|
||||
print(f'Error: {res["error"]}')
|
||||
return None
|
||||
|
||||
|
||||
async def monitor(url: str, interval: int, output_path: str, debug: bool):
|
||||
"""Continuous monitoring mode"""
|
||||
seen = load_seen_hashes(output_path)
|
||||
print(f'Monitoring {url} every {interval}s')
|
||||
print()
|
||||
|
||||
while True:
|
||||
try:
|
||||
res = await extract_latest_article(url, debug)
|
||||
|
||||
if res['status'] == 'success':
|
||||
art = res['data']
|
||||
url_val = art.get('url', '')
|
||||
hash_ = hashlib.md5(url_val.encode()).hexdigest() if url_val else None
|
||||
if hash_ and hash_ not in seen:
|
||||
seen.add(hash_)
|
||||
ts = _fmt(art.get('posting_time', ''))
|
||||
sentiment = art.get('sentiment', 'neutral')
|
||||
emoji = {'positive': '🟢', 'negative': '🔴', 'neutral': '🟡'}.get(sentiment, '🟡')
|
||||
summary = art.get('short_summary', art.get('title', ''))
|
||||
save_article(art, output_path)
|
||||
if debug:
|
||||
print(json.dumps(art, ensure_ascii=False, indent=2))
|
||||
print(f'[{ts}] - {emoji} - {summary}')
|
||||
if not debug:
|
||||
print() # Add spacing between articles in non-debug mode
|
||||
elif debug:
|
||||
print(f'Error: {res["error"]}')
|
||||
|
||||
except Exception as e:
|
||||
if debug:
|
||||
import traceback
|
||||
|
||||
traceback.print_exc()
|
||||
else:
|
||||
print(f'Unhandled error: {e}')
|
||||
|
||||
await asyncio.sleep(interval)
|
||||
|
||||
|
||||
def main():
|
||||
"""Main entry point"""
|
||||
if args.once:
|
||||
asyncio.run(run_once(args.url, args.output, args.debug))
|
||||
else:
|
||||
try:
|
||||
asyncio.run(monitor(args.url, args.interval, args.output, args.debug))
|
||||
except KeyboardInterrupt:
|
||||
print('\nStopped by user')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
Reference in New Issue
Block a user