Files
wehub-resource-sync db620d33df
CodeQL / Analyze (csharp) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
dotnet-build-and-test / dotnet-test-functions (push) Has been cancelled
dotnet-build-and-test / paths-filter (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Debug, windows-latest, net9.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net8.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-foundry-hosted-it (push) Has been cancelled
dotnet-build-and-test / dotnet-build-and-test-check (push) Has been cancelled
dotnet-build-and-test / Integration Test Report (push) Has been cancelled
chore: import upstream snapshot with attribution
2026-07-13 13:39:25 +08:00
..

DevUI Multi-Modal Agent

Interactive web UI for uploading and chatting with documents, images, audio, and video using Azure Content Understanding.

Setup

  1. Set environment variables (or create a .env file in python/):

    FOUNDRY_PROJECT_ENDPOINT=https://your-project.api.azureml.ms
    AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME=gpt-4.1
    AZURE_CONTENTUNDERSTANDING_ENDPOINT=https://your-cu-resource.cognitiveservices.azure.com/
    
  2. Log in with Azure CLI:

    az login
    
  3. Run with DevUI:

    uv run poe devui --agent packages/azure-contentunderstanding/samples/devui_multimodal_agent
    
  4. Open the DevUI URL in your browser and start uploading files.

What You Can Do

  • Upload PDFs — including scanned/image-based PDFs that LLM vision struggles with
  • Upload images — handwritten notes, infographics, charts
  • Upload audio — meeting recordings, call center calls (transcription with speaker ID)
  • Upload video — product demos, training videos (frame extraction + transcription)
  • Ask questions across all uploaded documents
  • Check status — "which documents are ready?" uses the auto-registered list_documents() tool