chore: import upstream snapshot with attribution
CI / unit-test (push) Has been cancelled
CI / detect-changes (push) Has been cancelled
CI / build (push) Has been cancelled
Publish docs via GitHub Pages / Deploy docs (push) Has been cancelled
CI / test-harness (push) Has been cancelled
CI / generate-e2e-matrix (push) Has been cancelled
CI / e2e (push) Has been cancelled
CI / build-ui (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
UI v2 Integration CI / E2E (Integration) (push) Has been cancelled
UI v2 CI / Lint, Format & Test (push) Has been cancelled
UI v2 CI / E2E (Mocked) (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 12:37:56 +08:00
commit a9cd7750f4
3727 changed files with 627706 additions and 0 deletions
+68
View File
@@ -0,0 +1,68 @@
---
name: docs-writer
description: Technical documentation specialist for Conductor workflow orchestration features. Creates clear, comprehensive documentation for APIs, workflows, tasks, and system architecture.
tools: Read, Grep, Glob, Write, Edit, Bash
model: inherit
---
You are a technical documentation specialist for Conductor, an open-source workflow orchestration engine built at Netflix.
## Your Role
Create clear, comprehensive, and accurate documentation for Conductor features, including:
- Workflow definitions and task types
- REST API endpoints and payloads
- System architecture and components
- Configuration options and database integrations
- SDK usage examples (Java, Python, JavaScript, Go, C#)
- Developer guides and tutorials
## Documentation Process
1. **Understand the Feature**
- Read relevant source code to understand implementation
- Identify key classes, methods, and APIs
- Test functionality if possible
- Review existing related documentation
2. **Structure Documentation**
- Start with a clear overview/summary
- Include purpose and use cases
- Provide syntax and parameters
- Add practical examples
- Document edge cases and limitations
- Link to related documentation
3. **Follow Conductor Style**
- Use clear, concise language
- Include code examples in relevant languages
- Use Markdown formatting consistently
- Add diagrams or JSON examples for workflows
- Follow existing documentation patterns in `/docs`
4. **Quality Standards**
- Ensure technical accuracy
- Test all code examples
- Use proper terminology (workflows, tasks, workers, etc.)
- Include error handling examples
- Add troubleshooting sections when relevant
## Key Conductor Concepts to Reference
- **Workflows**: JSON-based orchestration definitions
- **Tasks**: Units of work (HTTP, Lambda, Sub-workflow, etc.)
- **Workers**: Services that execute tasks
- **Task Definitions**: Reusable task configurations
- **System Tasks**: Built-in task types
- **Event Handlers**: Trigger workflows from events
## Output Format
Provide documentation in Markdown format suitable for the `/docs` directory, with:
- Clear headings and sections
- Code blocks with proper syntax highlighting
- Tables for parameters and options
- Links to related documentation
- Version information when relevant
Always prioritize clarity and practical usefulness for developers using Conductor.
+47
View File
@@ -0,0 +1,47 @@
# Git/VCS
.git
.gitignore
.gitattributes
.github
# IDE/editor
.idea
.vscode
.classpath
.project
.settings
*.iml
# OS/filesystem noise
.DS_Store
# Caches & temp
**/.gradle
**/.cache
**/tmp
**/logs
**/*.log
# Build outputs (keep source in docker build)
**/build
!ui/build
**/out
**/target
**/dist
!ui-next/dist
**/coverage
# Python
venv
**/__pycache__/
**/.pytest_cache/
# JS tooling
**/node_modules
**/.npm
**/.yarn
**/.pnpm-store
**/.eslintcache
**/.parcel-cache
**/.next
+11
View File
@@ -0,0 +1,11 @@
gradlew eol=lf
*.gradle eol=lf
*.java eol=lf
*.groovy eol=lf
spring.factories eol=lf
*.sh eol=lf
docs/* linguist-documentation
server/src/main/resources/swagger-ui/* linguist-vendored
+42
View File
@@ -0,0 +1,42 @@
---
name: Bug report
about: Create a report to help us improve
title: ""
labels: 'type: bug'
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**Details**
Conductor version:
Persistence implementation: Cassandra, Postgres, MySQL, Dynomite etc
Queue implementation: Postgres, MySQL, Dynoqueues etc
Lock: Redis or Zookeeper?
Workflow definition:
Task definition:
Event handler definition:
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Code to reproduce (paste relevant snippet):**
```
[paste the minimal code that triggers the bug]
```
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Additional context**
Add any other context about the problem here.
+43
View File
@@ -0,0 +1,43 @@
name: Bug Report
description: Create a report to help us reproduce and fix the bug
body:
- type: markdown
attributes:
value: >
#### Before submitting a bug, please make sure the issue hasn't been already addressed by searching through [the existing and past issues](https://github.com/conductor-oss/conductor/issues?q=is%3Aissue%20label%3Abug).
- type: textarea
attributes:
label: Describe the bug
description: |
Please provide a clear and concise description of what the bug is.
If relevant, add a minimal example so that we can reproduce the error by running the code. It is very important for the snippet to be as succinct (minimal) as possible, so please take time to trim down any irrelevant code to help us debug efficiently.
Your example should be fully self-contained and not rely on any artifact that should be downloaded.
For example:
```
# A succinct reproducing example trimmed down to the essential parts
# (any language is fine — Java, Python, Go, JavaScript, etc.)
```
If the code is too long (hopefully, it isn't), feel free to put it in a public gist and link it in the issue: https://gist.github.com.
Please also paste or describe the results you observe instead of the expected results. If you observe an error, please paste the error message including the **full** traceback of the exception. It may be relevant to wrap error messages in ```` ```triple quotes blocks``` ````.
placeholder: |
A clear and concise description of what the bug is.
```
# Sample code to reproduce the problem (any language)
```
```
The error message you got, with the full traceback.
```
validations:
required: true
- type: markdown
attributes:
value: >
Thanks for contributing!
+5
View File
@@ -0,0 +1,5 @@
blank_issues_enabled: true
contact_links:
- name: Questions
url: https://orkes-conductor.slack.com/join/shared_invite/zt-2vdbx239s-Eacdyqya9giNLHfrCavfaA#/shared-invite/email
about: Ask questions and discuss with other Conductor community members
+12
View File
@@ -0,0 +1,12 @@
---
name: Documentation
about: Something in the documentation that needs improvement
title: "[DOC]: "
labels: 'type: docs'
assignees: ''
---
## What are you missing in the docs
## Proposed text
+24
View File
@@ -0,0 +1,24 @@
name: Documentation
description: Report an issue related to https://docs.conductor-oss.org/index.html
body:
- type: markdown
attributes:
value: >
#### Note: Please report your documentation issue in English to ensure it can be understood and addressed by the development team.
- type: textarea
attributes:
label: The doc issue
description: >
A clear and concise description of what content on https://docs.conductor-oss.org/index.html is an issue.
validations:
required: true
- type: textarea
attributes:
label: Suggest a potential alternative/fix
description: >
Tell us how we could improve the documentation in this regard.
- type: markdown
attributes:
value: >
Thanks for contributing!
+20
View File
@@ -0,0 +1,20 @@
---
name: Feature request
about: Propose a new feature
title: "[FEATURE]: "
labels: 'type: feature'
assignees: ''
---
Please read our [contributor guide](https://github.com/conductor-oss/conductor/blob/main/CONTRIBUTING.md) before creating an issue.
Also consider discussing your idea on the [discussion forum](https://github.com/conductor-oss/conductor/discussions) first.
## Describe the Feature Request
_A clear and concise description of what the feature request is._
## Describe Preferred Solution
_A clear and concise description of what you want to happen._
## Describe Alternatives
_A clear and concise description of any alternative solutions or features you've considered._
@@ -0,0 +1,29 @@
name: Feature request
description: Submit a proposal/request for a new Conductor feature
body:
- type: markdown
attributes:
value: >
#### Note: Please write your feature request in English to ensure it can be understood and addressed by the development team.
- type: textarea
attributes:
label: The feature, motivation and pitch
description: >
A clear and concise description of the feature proposal. Please outline the motivation for the proposal. Is your feature request related to a specific problem? e.g., *"I'm working on X and would like Y to be possible"*. If this is related to another GitHub issue, please link here too.
validations:
required: true
- type: textarea
attributes:
label: Alternatives
description: >
A description of any alternative solutions or features you've considered, if any.
- type: textarea
attributes:
label: Additional context
description: >
Add any other context or screenshots about the feature request.
- type: markdown
attributes:
value: >
Thanks for contributing!
+21
View File
@@ -0,0 +1,21 @@
Pull Request type
----
- [ ] Bugfix
- [ ] Feature
- [ ] Refactoring (no functional changes, no api changes)
- [ ] Build related changes
- [ ] WHOSUSING.md
- [ ] Other (please describe):
**NOTE**: Please remember to run `./gradlew spotlessApply` to fix any format violations.
Changes in this PR
----
_Describe the new behavior from this PR, and why it's needed_
Issue #
Alternatives considered
----
_Describe alternative implementation you have considered_
+39
View File
@@ -0,0 +1,39 @@
template: |
## Whats Changed
$CHANGES
name-template: 'v$RESOLVED_VERSION'
tag-template: 'v$RESOLVED_VERSION'
categories:
- title: 'IMPORTANT'
label: 'type: important'
- title: 'New'
label: 'type: feature'
- title: 'Bug Fixes'
label: 'type: bug'
- title: 'Refactor'
label: 'type: maintenance'
- title: 'Documentation'
label: 'type: docs'
- title: 'Dependency Updates'
label: 'type: dependencies'
version-resolver:
minor:
labels:
- 'type: important'
patch:
labels:
- 'type: bug'
- 'type: maintenance'
- 'type: docs'
- 'type: dependencies'
- 'type: feature'
exclude-labels:
- 'skip-changelog'
- 'gradle-wrapper'
- 'github_actions'
+419
View File
@@ -0,0 +1,419 @@
name: CI
on:
push:
branches:
- main
paths-ignore:
- "conductor-clients/**"
pull_request:
paths-ignore:
- "conductor-clients/**"
workflow_dispatch:
inputs:
redis_es8:
description: "Redis + Elasticsearch 8"
type: boolean
default: true
postgres:
description: "PostgreSQL"
type: boolean
default: false
mysql:
description: "MySQL"
type: boolean
default: false
redis_os3:
description: "Redis + OpenSearch 3"
type: boolean
default: false
redis_es7:
description: "Redis + Elasticsearch 7"
type: boolean
default: false
cassandra_es7:
description: "Cassandra + Elasticsearch 7"
type: boolean
default: false
jobs:
detect-changes:
runs-on: ubuntu-latest
outputs:
heavy-persistence: ${{ steps.filter.outputs.heavy-persistence }}
steps:
- uses: actions/checkout@v7
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 0
- uses: dorny/paths-filter@v3
id: filter
with:
filters: |
heavy-persistence:
- 'cassandra-persistence/**'
- 'es6-persistence/**'
- 'es7-persistence/**'
- 'es8-persistence/**'
- 'mysql-persistence/**'
- 'os-persistence/**'
- 'os-persistence-v2/**'
- 'os-persistence-v3/**'
- 'scheduler/cassandra-persistence/**'
- 'scheduler/mysql-persistence/**'
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 0
- name: Free disk space
run: |
sudo rm -rf /usr/share/dotnet
sudo rm -rf /usr/local/lib/android
sudo rm -rf /opt/ghc
- name: Gradle wrapper validation
uses: gradle/wrapper-validation-action@v3
- name: Set up Zulu JDK 21
uses: actions/setup-java@v5
with:
distribution: "zulu"
java-version: "21"
- name: Cache SonarCloud packages
uses: actions/cache@v5
with:
path: ~/.sonar/cache
key: ${{ runner.os }}-sonar
restore-keys: ${{ runner.os }}-sonar
- name: Cache Gradle packages
uses: actions/cache@v5
with:
path: |
~/.gradle/caches
~/.gradle/wrapper
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
restore-keys: ${{ runner.os }}-gradle-
- name: Build with Gradle
if: github.ref != 'refs/heads/main'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
run: |
./gradlew build -x :conductor-test-harness:test -x test
- name: Build and Publish snapshot
if: github.event_name != 'pull_request' && github.ref == 'refs/heads/main'
run: |
echo "Running build for commit ${{ github.sha }}"
./gradlew build -x :conductor-test-harness:test -x test
- name: Generate aggregated coverage report
if: always()
run: ./gradlew jacocoAggregatedReport -x test || true
- name: Publish Test Report
uses: mikepenz/action-junit-report@v6
if: always()
with:
report_paths: "**/build/test-results/test/TEST-*.xml"
- name: Upload build artifacts
uses: actions/upload-artifact@v7
with:
name: build-artifacts
path: "**/build/reports"
- name: Upload coverage report
uses: actions/upload-artifact@v7
if: always()
with:
name: coverage-report
path: build/reports/jacoco/aggregated
unit-test:
needs: detect-changes
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 0
- name: Free disk space
run: |
sudo rm -rf /usr/share/dotnet
sudo rm -rf /usr/local/lib/android
sudo rm -rf /opt/ghc
- name: Set up Zulu JDK 21
uses: actions/setup-java@v5
with:
distribution: "zulu"
java-version: "21"
- name: Cache Gradle packages
uses: actions/cache@v5
with:
path: |
~/.gradle/caches
~/.gradle/wrapper
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
restore-keys: ${{ runner.os }}-gradle-
- name: Force Docker API Version
run: echo 'api.version=1.44' > ~/.docker-java.properties
- name: Run unit tests
run: |
# On main (post-merge): run everything
if [ "${{ github.event_name }}" != "pull_request" ]; then
./gradlew test -x :conductor-test-harness:test
exit 0
fi
# On PRs: skip heavy container tests (cassandra, es, mysql, opensearch)
# unless their code changed. Redis, postgres, sqlite always run.
if [ "${{ needs.detect-changes.outputs.heavy-persistence }}" == "true" ]; then
./gradlew test -x :conductor-test-harness:test
else
./gradlew test \
-x :conductor-test-harness:test \
-x :conductor-cassandra-persistence:test \
-x :conductor-scheduler-cassandra-persistence:test \
-x :conductor-es6-persistence:test \
-x :conductor-es7-persistence:test \
-x :conductor-es8-persistence:test \
-x :conductor-mysql-persistence:test \
-x :conductor-scheduler-mysql-persistence:test \
-x :conductor-os-persistence:test \
-x :conductor-os-persistence-v2:test \
-x :conductor-os-persistence-v3:test
fi
- name: Publish Test Report
uses: mikepenz/action-junit-report@v6
if: always()
with:
report_paths: "**/build/test-results/test/TEST-*.xml"
check_name: Unit Test Report
- name: Upload unit-test reports
uses: actions/upload-artifact@v7
if: always()
with:
name: unit-test-reports
path: "**/build/reports/tests"
test-harness:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 0
- name: Free disk space
run: |
sudo rm -rf /usr/share/dotnet
sudo rm -rf /usr/local/lib/android
sudo rm -rf /opt/ghc
- name: Set up Zulu JDK 21
uses: actions/setup-java@v5
with:
distribution: "zulu"
java-version: "21"
- name: Cache Gradle packages
uses: actions/cache@v5
with:
path: |
~/.gradle/caches
~/.gradle/wrapper
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
restore-keys: ${{ runner.os }}-gradle-
- name: Force Docker API Version
run: echo 'api.version=1.44' > ~/.docker-java.properties
- name: Cache Docker images
uses: actions/cache@v5
id: docker-cache
with:
path: /tmp/docker-images-test-harness.tar
key: docker-test-harness-v2
- name: Load cached Docker images
if: steps.docker-cache.outputs.cache-hit == 'true'
run: docker load -i /tmp/docker-images-test-harness.tar || true
- name: Run test-harness tests
run: |
./gradlew :conductor-test-harness:test
- name: Save Docker images for cache
if: steps.docker-cache.outputs.cache-hit != 'true'
run: |
set -euo pipefail
docker image prune -f
mapfile -t images < <((docker images --format '{{.Repository}}:{{.Tag}}' \
| grep -v '<none>' \
| grep -E '(^|/)(elasticsearch|redis|postgres|mysql|mongo|cassandra)(:|/)|mockserver/mockserver|opensearchproject/opensearch|testcontainers/|orkesio/') || true)
if [ "${#images[@]}" -eq 0 ]; then
echo "No Testcontainers-related images to cache; writing empty tar for cache action."
tar -cf /tmp/docker-images-test-harness.tar --files-from /dev/null
exit 0
fi
printf '%s\n' "${images[@]}"
docker save -o /tmp/docker-images-test-harness.tar "${images[@]}"
- name: Publish Test Report
uses: mikepenz/action-junit-report@v6
if: always()
with:
report_paths: "test-harness/build/test-results/test/TEST-*.xml"
- name: Upload test-harness reports
uses: actions/upload-artifact@v7
if: always()
with:
name: test-harness-reports
path: "test-harness/build/reports"
- name: Upload test-harness coverage report
uses: actions/upload-artifact@v7
if: always()
with:
name: test-harness-coverage-report
path: "test-harness/build/reports/jacoco"
generate-e2e-matrix:
runs-on: ubuntu-latest
outputs:
matrix: ${{ steps.set-matrix.outputs.matrix }}
steps:
- id: set-matrix
shell: bash
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
items=""
[ "${{ inputs.redis_es8 }}" = "true" ] && items="${items}{\"name\":\"redis-es8\",\"script\":\"./e2e/run_tests-es8.sh\"},"
[ "${{ inputs.postgres }}" = "true" ] && items="${items}{\"name\":\"postgres\",\"script\":\"./e2e/run_tests-postgres.sh\"},"
[ "${{ inputs.mysql }}" = "true" ] && items="${items}{\"name\":\"mysql\",\"script\":\"./e2e/run_tests-mysql.sh\"},"
[ "${{ inputs.redis_os3 }}" = "true" ] && items="${items}{\"name\":\"redis-os3\",\"script\":\"./e2e/run_tests-redis-os3.sh\"},"
[ "${{ inputs.redis_es7 }}" = "true" ] && items="${items}{\"name\":\"redis-es7\",\"script\":\"./e2e/run_tests-redis-es7.sh\"},"
[ "${{ inputs.cassandra_es7 }}" = "true" ] && items="${items}{\"name\":\"cassandra-es7\",\"script\":\"./e2e/run_tests-cassandra-es7.sh\"},"
items="${items%,}"
echo "matrix={\"include\":[${items}]}" >> "$GITHUB_OUTPUT"
else
echo 'matrix={"include":[{"name":"redis-es8","script":"./e2e/run_tests-es8.sh"}]}' >> "$GITHUB_OUTPUT"
fi
e2e:
needs: generate-e2e-matrix
if: ${{ needs.generate-e2e-matrix.outputs.matrix != '{"include":[]}' }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix: ${{ fromJson(needs.generate-e2e-matrix.outputs.matrix) }}
steps:
- uses: actions/checkout@v7
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 0
- name: Free disk space
run: |
sudo rm -rf /usr/share/dotnet
sudo rm -rf /usr/local/lib/android
sudo rm -rf /opt/ghc
- name: Set up Zulu JDK 21
uses: actions/setup-java@v5
with:
distribution: "zulu"
java-version: "21"
- name: Cache Gradle packages
uses: actions/cache@v5
with:
path: |
~/.gradle/caches
~/.gradle/wrapper
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
restore-keys: ${{ runner.os }}-gradle-
- name: Run E2E tests (${{ matrix.name }})
run: ${{ matrix.script }}
- name: Publish Test Report
uses: mikepenz/action-junit-report@v6
if: always()
with:
report_paths: "e2e/build/test-results/test/TEST-*.xml"
- name: Generate test summary table
if: always()
shell: python3 {0}
run: |
import os, glob, xml.etree.ElementTree as ET
xml_files = glob.glob("e2e/build/test-results/test/TEST-*.xml")
rows = []
totals = {"passed": 0, "failed": 0, "skipped": 0}
for f in sorted(xml_files):
try:
root = ET.parse(f).getroot()
except ET.ParseError:
continue
for tc in root.iter("testcase"):
name = tc.get("name", "?")
classname = tc.get("classname", "").split(".")[-1]
duration = float(tc.get("time", 0))
if tc.find("skipped") is not None:
status, totals["skipped"] = "⏭ skip", totals["skipped"] + 1
elif tc.find("failure") is not None or tc.find("error") is not None:
node = tc.find("failure") if tc.find("failure") is not None else tc.find("error")
msg = (node.get("message") or "")[:120]
status, totals["failed"] = f"❌ `{msg}`", totals["failed"] + 1
else:
status, totals["passed"] = "✅", totals["passed"] + 1
rows.append((classname, name, f"{duration:.1f}s", status))
backend = "${{ matrix.name }}"
lines = [
f"## E2E results — {backend}",
f"**✅ {totals['passed']} passed · ❌ {totals['failed']} failed · ⏭ {totals['skipped']} skipped**",
"",
"| Class | Test | Duration | Result |",
"|-------|------|----------|--------|",
]
for cls, name, dur, status in rows:
lines.append(f"| {cls} | {name} | {dur} | {status} |")
summary = os.environ.get("GITHUB_STEP_SUMMARY", "/dev/null")
with open(summary, "a") as fh:
fh.write("\n".join(lines) + "\n")
- name: Upload E2E reports
uses: actions/upload-artifact@v7
if: always()
with:
name: e2e-reports-${{ matrix.name }}
path: "e2e/build/reports"
build-ui:
runs-on: ubuntu-latest
defaults:
run:
working-directory: ui
steps:
- uses: actions/checkout@v7
- name: Set up Node
uses: actions/setup-node@v6
with:
node-version: "22"
cache: "yarn"
cache-dependency-path: ui/yarn.lock
- name: Install Dependencies
run: yarn install
- name: Build UI
run: yarn run build
- name: Cache Playwright browsers
uses: actions/cache@v5
id: playwright-cache
with:
path: ~/.cache/ms-playwright
key: playwright-chromium-${{ hashFiles('ui/yarn.lock') }}
- name: Install Playwright browsers
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: npx playwright install --with-deps chromium
- name: Install Playwright browser deps (cached)
if: steps.playwright-cache.outputs.cache-hit == 'true'
run: npx playwright install-deps chromium
- name: Run Playwright E2E Tests
run: yarn test:e2e
- name: Upload Playwright report
uses: actions/upload-artifact@v7
if: failure()
with:
name: playwright-report
path: ui/playwright-report
retention-days: 7
@@ -0,0 +1,34 @@
name: Debug Docker Credentials
on:
workflow_dispatch:
jobs:
check-docker-user:
runs-on: ubuntu-latest
steps:
- name: Check Docker Hub user for API key
env:
DOCKER_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
DOCKER_PASSWORD: ${{ secrets.DOCKERHUB_TOKEN }}
run: |
echo "Fetching JWT from Docker Hub..."
RESPONSE=$(curl -s -X POST \
-H "Content-Type: application/json" \
-d "{\"username\": \"${DOCKER_USERNAME}\", \"password\": \"${DOCKER_PASSWORD}\"}" \
https://hub.docker.com/v2/users/login)
JWT=$(echo "$RESPONSE" | jq -r '.token // empty')
if [ -z "$JWT" ]; then
echo "Login failed. Response:"
echo "$RESPONSE" | jq .
exit 1
fi
echo "Login successful. Fetching user info..."
USER_INFO=$(curl -s -H "Authorization: Bearer $JWT" https://hub.docker.com/v2/user/)
echo "Raw response:"
echo "$USER_INFO" | jq .
echo "Docker Hub account info:"
echo "$USER_INFO" | jq '{username: .username, full_name: .full_name, email: .email, company: .company, date_joined: .date_joined}'
@@ -0,0 +1,44 @@
name: Deprecate conductor-standalone on Docker Hub
on:
workflow_dispatch:
permissions:
contents: read
jobs:
deprecate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: Login to Docker Hub
uses: docker/login-action@v4
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Build and push :deprecated tag
uses: docker/build-push-action@v7
with:
context: docker/conductor-standalone-deprecation
platforms: linux/amd64,linux/arm64
push: true
tags: conductoross/conductor-standalone:deprecated
- name: Update Docker Hub description
run: |
TOKEN=$(curl -s -X POST "https://hub.docker.com/v2/users/login" \
-H "Content-Type: application/json" \
-d "{\"username\":\"${{ secrets.DOCKERHUB_USERNAME }}\",\"password\":\"${{ secrets.DOCKERHUB_TOKEN }}\"}" \
| jq -r '.token')
curl -s -X PATCH "https://hub.docker.com/v2/repositories/conductoross/conductor-standalone/" \
-H "Authorization: Bearer ${TOKEN}" \
-H "Content-Type: application/json" \
-d "{\"full_description\":$(jq -Rs . < docker/conductor-standalone-deprecation/dockerhub-description.md)}"
+25
View File
@@ -0,0 +1,25 @@
name: Publish docs via GitHub Pages
permissions:
contents: write
on:
push:
branches:
- main
workflow_dispatch:
jobs:
build:
name: Deploy docs
runs-on: ubuntu-latest
steps:
- name: Checkout main
uses: actions/checkout@v7
- name: Deploy docs
uses: mhausenblas/mkdocs-deploy-gh-pages@master
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
CONFIG_FILE: mkdocs.yml
REQUIREMENTS: requirements.txt
+113
View File
@@ -0,0 +1,113 @@
name: Publish Next (ui-next)
# Publishes the server with ui-next as a parallel track alongside :latest.
# Does NOT touch :latest, any versioned release tags, or the existing S3 JARs.
#
# Resulting artifacts:
# Docker: conductoross/conductor:next
# S3 JAR: conductor-server-next.jar
#
# Test the Docker image:
# docker run -p 8080:8080 -p 5000:5000 conductoross/conductor:next
# → API / Swagger: http://localhost:8080
# → ui-next: http://localhost:5000
#
# Test via the CLI:
# conductor server start --version next
#
on:
workflow_dispatch:
permissions:
contents: read
jobs:
publish-next:
runs-on: ubuntu-latest
name: Build and publish ui-next artifacts
steps:
- uses: actions/checkout@v7
# ── Java ─────────────────────────────────────────────────────────────────
- name: Set up Zulu JDK 21
uses: actions/setup-java@v5
with:
distribution: 'zulu'
java-version: '21'
- name: Cache Gradle packages
uses: actions/cache@v5
with:
path: |
~/.gradle/caches
~/.gradle/wrapper
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
restore-keys: |
${{ runner.os }}-gradle-
- name: Build Server JAR
run: |
./gradlew :conductor-server:build -x test -x spotlessCheck -x shadowJar \
-Dorg.gradle.jvmargs=-Xmx2g --no-daemon
# ── UI (ui-next) ─────────────────────────────────────────────────────────
- name: Set up Node.js
uses: actions/setup-node@v6
with:
node-version: '22'
- name: Set up pnpm
uses: pnpm/action-setup@v6
with:
version: "10.32.0"
- name: Build ui-next
run: |
cd ui-next
pnpm install && pnpm build
# ── Stage pre-built artifacts for PREBUILT=true Docker build ─────────────
- name: Stage artifacts
run: |
mkdir -p docker/server/libs
cp server/build/libs/*boot*.jar docker/server/libs/conductor-server.jar
# ── Docker ───────────────────────────────────────────────────────────────
- name: Login to Docker Hub
uses: docker/login-action@v4
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Build and push :next image
uses: docker/build-push-action@v7
with:
context: .
file: docker/server/Dockerfile.next
platforms: linux/arm64,linux/amd64
push: true
build-args: |
PREBUILT=true
tags: |
conductoross/conductor:next
# ── S3 JAR ───────────────────────────────────────────────────────────────
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v6
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ secrets.AWS_REGION }}
- name: Upload :next JAR to S3
run: |
aws s3 cp server/build/libs/*boot*.jar \
s3://${{ secrets.AWS_S3_BUCKET }}/conductor-server-next.jar
echo "Published: conductor-server-next.jar"
echo "Test with: conductor server start --version next"
+114
View File
@@ -0,0 +1,114 @@
name: Publish Conductor OSS toMaven Central
on:
release:
types:
- released
- prereleased
workflow_dispatch:
inputs:
version:
description: 'Version to publish (e.g., v1.0.0)'
required: true
permissions:
contents: read
jobs:
publish:
runs-on: ubuntu-latest
name: Gradle Build and Publish
steps:
- uses: actions/checkout@v7
- name: Set up Zulu JDK 21
uses: actions/setup-java@v5
with:
distribution: 'zulu'
java-version: '21'
- name: Cache Gradle packages
uses: actions/cache@v5
with:
path: |
~/.gradle/caches
~/.gradle/wrapper
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
restore-keys: |
${{ runner.os }}-gradle-
- name: Publish release
run: |
export VERSION="${{github.ref_name}}"
export PUBLISH_VERSION=`echo ${VERSION:1}`
echo Publishing version $PUBLISH_VERSION
./gradlew publish -PmavenCentral -Pversion=$PUBLISH_VERSION -Pusername=${{ secrets.SONATYPE_USERNAME }} -Ppassword=${{ secrets.SONATYPE_PASSWORD }}
env:
ORG_GRADLE_PROJECT_signingKeyId: ${{ secrets.SIGNING_KEY_ID }}
ORG_GRADLE_PROJECT_signingKey: ${{ secrets.SIGNING_KEY }}
ORG_GRADLE_PROJECT_signingPassword: ${{ secrets.SIGNING_PASSWORD }}
publish-docker:
runs-on: ubuntu-latest
name: Gradle Build and Publish to Container Registry
steps:
- uses: actions/checkout@v7
- name: Set up Zulu JDK 21
uses: actions/setup-java@v5
with:
distribution: 'zulu'
java-version: '21'
- name: Login to Docker Hub Container Registry
uses: docker/login-action@v4
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up Node.js
uses: actions/setup-node@v6
with:
node-version: 'lts/*'
- name: Build ui-next and embed into server static resources
run: |
corepack enable
cd ui-next
pnpm install
NODE_OPTIONS=--max-old-space-size=4096 pnpm build
cd ..
mkdir -p server/src/main/resources/static
rm -rf server/src/main/resources/static/*
cp -r ui-next/dist/. server/src/main/resources/static/
- name: Build Server JAR
run: |
export VERSION="${{github.ref_name}}"
export PUBLISH_VERSION=`echo ${VERSION:1}`
echo "RELEASE_VERSION=$PUBLISH_VERSION" >> $GITHUB_ENV
echo Publishing version $PUBLISH_VERSION
./gradlew :conductor-server:build -x test -x spotlessCheck -x shadowJar -x :conductor-os-persistence-v3:build \
-Pversion=$PUBLISH_VERSION
- name: Stage artifacts for Docker build
run: |
mkdir -p docker/server/libs
cp server/build/libs/*boot*.jar docker/server/libs/conductor-server.jar
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Build and push Server
uses: docker/build-push-action@v7
with:
context: .
file: docker/server/Dockerfile
platforms: linux/arm64,linux/amd64
push: true
build-args: |
PREBUILT=true
tags: |
conductoross/conductor:${{ env.RELEASE_VERSION }}
orkesio/orkes-conductor-community-standalone:${{ env.RELEASE_VERSION }}
orkesio/orkes-conductor-community:${{ env.RELEASE_VERSION }}
${{ (github.event_name == 'release' && !github.event.release.prerelease && !contains(github.ref_name, 'rc') && !contains(github.ref_name, 'beta') && !contains(github.ref_name, 'alpha')) && 'conductoross/conductor:latest' || '' }}
${{ (github.event_name == 'release' && !github.event.release.prerelease && !contains(github.ref_name, 'rc') && !contains(github.ref_name, 'beta') && !contains(github.ref_name, 'alpha')) && 'orkesio/orkes-conductor-community-standalone:latest' || '' }}
${{ (github.event_name == 'release' && !github.event.release.prerelease && !contains(github.ref_name, 'rc') && !contains(github.ref_name, 'beta') && !contains(github.ref_name, 'alpha')) && 'orkesio/orkes-conductor-community:latest' || '' }}
+100
View File
@@ -0,0 +1,100 @@
name: Publish Conductor OSS Server
on:
release:
types:
- released
- prereleased
workflow_dispatch:
inputs:
version:
description: 'Version to publish (e.g., v1.0.0)'
required: true
permissions:
contents: read
jobs:
publish:
runs-on: ubuntu-latest
name: Build and Publish the server
permissions:
contents: write
steps:
- uses: actions/checkout@v7
- name: Set up Zulu JDK 21
uses: actions/setup-java@v5
with:
distribution: 'zulu'
java-version: '21'
- name: Cache Gradle packages
uses: actions/cache@v5
with:
path: |
~/.gradle/caches
~/.gradle/wrapper
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
restore-keys: |
${{ runner.os }}-gradle-
- name: Set version
id: version
run: |
# Use inputs.version for workflow_dispatch, or ref_name for release events
VERSION="${{ github.event.inputs.version || github.ref_name }}"
# Strip the 'v' prefix if present
PUBLISH_VERSION="${VERSION#v}"
echo "version=$PUBLISH_VERSION" >> $GITHUB_OUTPUT
echo "RELEASE_VERSION=$PUBLISH_VERSION" >> $GITHUB_ENV
echo "Publishing version: $PUBLISH_VERSION"
- name: Build Server JAR
run: |
./gradlew :conductor-server:build -x test -x spotlessCheck -x shadowJar -x :conductor-os-persistence-v3:build \
-Pversion=${{ steps.version.outputs.version }}
- name: Set up Node.js
uses: actions/setup-node@v6
with:
node-version: 'lts/*'
- name: Build ui-next and embed into server static resources
run: |
corepack enable
cd ui-next
pnpm install
NODE_OPTIONS=--max-old-space-size=4096 pnpm build
cd ..
mkdir -p server/src/main/resources/static
rm -rf server/src/main/resources/static/*
cp -r ui-next/dist/. server/src/main/resources/static/
- name: Stage artifacts for Docker build
run: |
mkdir -p docker/server/libs
cp server/build/libs/*boot*.jar docker/server/libs/conductor-server.jar
- name: Login to Docker Hub Container Registry
uses: docker/login-action@v4
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Build and push Server
uses: docker/build-push-action@v7
with:
context: .
file: docker/server/Dockerfile
platforms: linux/arm64,linux/amd64
push: true
build-args: |
PREBUILT=true
tags: |
conductoross/conductor:${{ env.RELEASE_VERSION }}
orkesio/orkes-conductor-community-standalone:${{ env.RELEASE_VERSION }}
orkesio/orkes-conductor-community:${{ env.RELEASE_VERSION }}
${{ (github.event_name == 'release' && !github.event.release.prerelease) && 'conductoross/conductor:latest' || '' }}
${{ (github.event_name == 'release' && !github.event.release.prerelease) && 'orkesio/orkes-conductor-community-standalone:latest' || '' }}
${{ (github.event_name == 'release' && !github.event.release.prerelease) && 'orkesio/orkes-conductor-community:latest' || '' }}
+84
View File
@@ -0,0 +1,84 @@
name: Publish Conductor Server to S3
on:
push:
branches:
- docker_build # TEMPORARY - remove before merging to main
release:
types:
- released
- prereleased
workflow_dispatch:
inputs:
version:
description: 'Version to publish (e.g., v1.0.0)'
required: true
permissions:
contents: read
jobs:
publish-s3:
runs-on: ubuntu-latest
name: Build and Publish Server JAR to S3
steps:
- uses: actions/checkout@v7
- name: Set up Zulu JDK 21
uses: actions/setup-java@v5
with:
distribution: 'zulu'
java-version: '21'
- name: Cache Gradle packages
uses: actions/cache@v5
with:
path: |
~/.gradle/caches
~/.gradle/wrapper
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
restore-keys: |
${{ runner.os }}-gradle-
- name: Set version
id: version
run: |
# Use inputs.version for workflow_dispatch, or ref_name for release events
VERSION="${{ github.event.inputs.version || github.ref_name }}"
# Strip the 'v' prefix if present
PUBLISH_VERSION="${VERSION#v}"
echo "version=$PUBLISH_VERSION" >> $GITHUB_OUTPUT
echo "Publishing version: $PUBLISH_VERSION"
- name: Set up Node.js
uses: actions/setup-node@v6
with:
node-version: 'lts/*'
- name: Build Server module
run: |
corepack enable
./build_ui_next.sh
./gradlew :conductor-server:bootJar -x test -Pversion=${{ steps.version.outputs.version }}
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v6
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ secrets.AWS_REGION }}
- name: Upload Server JAR to S3 (Versioned)
run: |
aws s3 cp server/build/libs/conductor-server-${{ steps.version.outputs.version }}-boot.jar \
s3://${{ secrets.AWS_S3_BUCKET }}/conductor-server-${{ steps.version.outputs.version }}.jar
- name: Upload Server JAR to S3 (Latest)
if: github.event_name == 'release' && !github.event.release.prerelease
run: |
aws s3 cp server/build/libs/conductor-server-${{ steps.version.outputs.version }}-boot.jar \
s3://${{ secrets.AWS_S3_BUCKET }}/conductor-server-latest.jar
- name: Verify uploads
run: |
echo "Uploaded conductor-server-${{ steps.version.outputs.version }}.jar"
echo "S3 bucket: ${{ secrets.AWS_S3_BUCKET }}"
+22
View File
@@ -0,0 +1,22 @@
name: Release Drafter
on:
push:
branches:
- main
paths-ignore:
- 'conductor-clients/**'
permissions:
contents: read
jobs:
update_release_draft:
permissions:
contents: write # for release-drafter/release-drafter to create a github release
pull-requests: write # for release-drafter/release-drafter to add label to PR
runs-on: ubuntu-latest
steps:
- uses: release-drafter/release-drafter@v7
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+81
View File
@@ -0,0 +1,81 @@
name: UI v2 CI
on:
pull_request:
branches:
- main
paths:
- "ui-next/**"
push:
branches:
- main
paths:
- "ui-next/**"
permissions:
contents: read
jobs:
lint-format-test:
name: Lint, Format & Test
runs-on: ubuntu-latest
defaults:
run:
working-directory: ui-next
steps:
- uses: actions/checkout@v7
- name: Setup pnpm
uses: pnpm/action-setup@v6
with:
version: 10.32.0
# setup-node must come after pnpm/action-setup so it can locate the store.
- name: Setup Node.js
uses: actions/setup-node@v6
with:
node-version: 22
cache: pnpm
cache-dependency-path: ui-next/pnpm-lock.yaml
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Prettier check
run: pnpm prettier:check
- name: Lint
run: pnpm lint
- name: Type check
run: pnpm typecheck
- name: Test
run: pnpm test
- name: Build
run: pnpm build
e2e-mocked:
name: E2E (Mocked)
runs-on: ubuntu-latest
needs: lint-format-test
steps:
- uses: actions/checkout@v7
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Run E2E tests in Docker
run: docker compose -f docker-compose.snapshots.yml run --rm playwright
working-directory: ui-next
- name: Upload Playwright report
uses: actions/upload-artifact@v7
if: failure()
with:
name: playwright-report
path: ui-next/playwright-snapshots-report/
retention-days: 7
@@ -0,0 +1,103 @@
name: UI v2 Integration CI
on:
pull_request:
branches:
- main
paths:
- "ui-next/**"
push:
branches:
- main
paths:
- "ui-next/**"
permissions:
contents: read
jobs:
e2e-integration:
name: E2E (Integration)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: Free disk space
run: |
sudo rm -rf /usr/share/dotnet
sudo rm -rf /usr/local/lib/android
sudo rm -rf /opt/ghc
sudo rm -rf /usr/local/share/boost
sudo apt-get clean
- name: Setup pnpm
uses: pnpm/action-setup@v6
with:
version: 10.32.0
# setup-node must come after pnpm/action-setup so it can locate the store.
- name: Setup Node.js
uses: actions/setup-node@v6
with:
node-version: 22
cache: pnpm
cache-dependency-path: ui-next/pnpm-lock.yaml
- name: Install dependencies
run: pnpm install --frozen-lockfile
working-directory: ui-next
- name: Cache Playwright browsers
id: playwright-cache
uses: actions/cache@v5
with:
path: ~/.cache/ms-playwright
key: ${{ runner.os }}-playwright-chromium-${{ hashFiles('ui-next/pnpm-lock.yaml') }}
- name: Install Playwright browsers
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: pnpm exec playwright install chromium
working-directory: ui-next
# OS-level dependencies (apt packages) cannot be cached — always install.
- name: Install Playwright OS dependencies
run: pnpm exec playwright install-deps chromium
working-directory: ui-next
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Cache Docker layers for conductor:server
uses: actions/cache@v5
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-conductor-server-${{ hashFiles('docker/server/Dockerfile', '**/build.gradle', 'settings.gradle') }}
restore-keys: ${{ runner.os }}-conductor-server-
- name: Build conductor:server Docker image
uses: docker/build-push-action@v7
with:
context: .
file: docker/server/Dockerfile
tags: conductor:server
load: true
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache-new,mode=min
- name: Move Docker layer cache
run: |
rm -rf /tmp/.buildx-cache
mv /tmp/.buildx-cache-new /tmp/.buildx-cache
- name: Run integration E2E tests
run: pnpm test:e2e:integration
working-directory: ui-next
- name: Upload Playwright integration report
uses: actions/upload-artifact@v7
if: failure()
with:
name: playwright-integration-report
path: ui-next/playwright-integration-report/
retention-days: 7
+58
View File
@@ -0,0 +1,58 @@
# Worktrees
.worktrees/
# Java Build
.gradle
.classpath
dump.rdb
out
bin
target
buildscan.log
/docs/site
# Python
/polyglot-clients/python/conductor.egg-info
*.pyc
# OS & IDE
.DS_Store
.settings
.vscode
.idea
.project
*.iml
# JS & UI Related
node_modules
/ui/build
/ui/public/monaco-editor
# publishing secrets
secrets/signing-key
# local builds
lib/
build/
*/build/
# asdf version file
.tool-versions
# jenv version file
.java-version
.qodo
conductorosstest.db
conductorosstest.db
yarn.lock
.java-version
/server/src/main/resources/static
*.factorypath
server/*.db*
/site
*.db
*.db-shm
*.db-wal
docs/superpowers
+178
View File
@@ -0,0 +1,178 @@
# AGENTS.md
Instructions for AI coding agents working on the Conductor codebase.
## Project Overview
Conductor is an open-source, distributed workflow orchestration engine designed for microservices.
It uses a pluggable architecture with interface-based abstractions for persistence, queuing, and indexing.
The project is built with Java 21 and uses Gradle as the build system.
## Setup Commands
| Command | Description |
|---------|-------------|
| `./gradlew build` | Build the entire project |
| `./gradlew test` | Run all tests |
| `./gradlew :module-name:test` | Run tests for a specific module |
| `./gradlew spotlessApply` | Apply code formatting |
| `./gradlew clean build` | Clean and rebuild |
> **Important**: Always run `./gradlew spotlessApply` after making code changes to ensure consistent formatting.
## Java Version References
**Never link to a specific Java distribution** (e.g., Adoptium, Temurin, OpenJDK.org, Amazon Corretto) in docs, READMEs, or comments. Just say "Java 21+" and let users install it however they prefer.
## Code Style
- Use the Spotless plugin for uniform code formatting—always run before committing
- Conductor is pluggable: when introducing new concepts, always use an **interface-based approach**
- DAO interfaces **MUST** be defined in the `core` module
- Implementation classes go in their respective persistence modules (e.g., `postgres-persistence`, `redis-persistence`)
- Follow existing patterns in the codebase for consistency
- Do not use emojis such as ✅ in the code, logs, or comments. Keep comments professionals
- When adding new logic, comment the algorithm, design etc.
## Architecture Guidelines
### Module Structure
- **core**: Contains interfaces, domain models, and core business logic
- **persistence modules**: Implementations of DAO interfaces (postgres, redis, mysql, etc.)
- **server**: Spring Boot application that brings everything together
- **client**: SDK for interacting with Conductor
- **ui**: React-based user interface
### Key Patterns
- DAOs are defined as interfaces in `core` and implemented in persistence modules
- System tasks extend `WorkflowSystemTask` and are registered via Spring
- Worker tasks use the `@WorkerTask` annotation for automatic discovery
- Configuration is primarily done through Spring properties
## Testing
- **Avoid mocks**: Use real implementations whenever possible
- **Test actual behavior**: Tests must verify real implementation logic, not duplicate it
- **Use Testcontainers**: For database, cache, and other external dependencies
- **Cover concurrency**: Ensure multi-threading scenarios are tested
- **Run tests before submitting**: `./gradlew test` must pass
### Test Locations
- Unit tests: `src/test/java` in each module
- Integration tests: `test-harness` module and `*-integration-test` modules
- E2E tests: `e2e` module
## PR Guidelines
- Submit PRs against the `main` branch
- Use clear, descriptive commit messages
- Run `./gradlew spotlessApply` and `./gradlew test` before pushing
- Add or update tests for any code changes
- Keep PRs focused—one logical change per PR
## Dependency Pinning
Some dependencies have hard version constraints that **must not be auto-bumped**. These are marked with:
```groovy
// PINNED (#964): <reason>
```
The issue number links back to https://github.com/conductor-oss/conductor/issues/964, which documents the full audit and upgrade path for each constraint.
### What PINNED means
`// PINNED (#964):` means the version is intentionally locked and upgrading it without understanding the constraint will break the build or cause a runtime failure. Do not bump a PINNED dependency as part of routine dependency updates or refactoring.
### Current hard pins
| Dependency | Pinned at | Why |
|---|---|---|
| `com.google.protobuf:protobuf-java` | `3.x` | 4.x + GraalVM polyglot 25.x causes Gradle to require `polyglot4`, which does not exist on Maven Central |
| `com.google.protobuf:protoc` | `3.25.5` | Must match `grpc-protobuf:1.73.0`, which depends on protobuf-java 3.x |
| `org.graalvm.*` (all 5 artifacts) | same version | All must share one version — mixing causes a `"polyglot version X not compatible with Truffle Y"` runtime error |
| `redis.clients:jedis` in `redis-concurrency-limit` | `3.6.0` | `revJedis` (6.0.0) does not work with Spring Data Redis in that module |
| `org.codehaus.jettison:jettison` | `strictly 1.5.4` | Gradle `strictly` constraint — no higher version has been validated |
| `org.conductoross:conductor-client` in `test-harness` | `5.0.1` | Fat JAR classpath conflict with conductor-common; resolved via a stripped JAR task |
| `org.awaitility:awaitility` in functional tests | `4.x` | e2e tests call `pollInterval(Duration)` added in Awaitility 4.0 |
### Before bumping a PINNED dependency
1. Read the comment carefully — it will name the incompatibility and often link to an upstream issue.
2. Check whether the upstream blocker has been resolved (e.g., new grpc-java release, new GraalVM release).
3. Test locally: `./gradlew clean build` plus `./gradlew test` in the affected modules.
4. If bumping GraalVM, bump **all five** `org.graalvm.*` artifacts together using `revGraalVM` in `dependencies.gradle`.
5. Update or remove the `// PINNED` comment once the constraint is lifted.
### PINNED vs. version floors
Hard caps use `// PINNED (#964):`. Version floors — where a minimum is enforced but higher versions are always welcome — use one of two lowercase prefixes instead:
```groovy
// Security: CVE-2025-12183 — lz4-java minimum patched version
// Compat: commons-lang3 3.18.0+ required by Testcontainers/commons-compress
```
- `// Security:` — minimum set to address a CVE or known vulnerability
- `// Compat:` — minimum set for compatibility with another library or framework
These are grep-able (`grep "// Security:" **/*.gradle`, `grep "// Compat:" **/*.gradle`) but read as normal developer comments. Dependabot may raise these freely; no special review needed beyond the usual.
## Security Considerations
- Never commit secrets, API keys, or credentials
- Be cautious with external dependencies—prefer well-maintained libraries
- Follow secure coding practices for input validation and error handling
- Review [SECURITY.md](SECURITY.md) for vulnerability reporting procedures
## Writing Documentation
Documentation in this project is **derived from source**, not composed from memory. Open the source first, read what's there, then write the doc from what you find. The source is the spec; the doc is a rendering of it.
This matters because plausible-looking docs can be silently wrong. Concretely: a curl equivalent for `conductor workflow start --sync` was once written as `POST /api/workflow/{name}/run` — an endpoint that does not exist. Reading the controller first would have given the correct path immediately.
### Workflow for each content type
**REST API endpoint or curl example**
1. Open the relevant controller: `rest/src/main/java/com/netflix/conductor/rest/controllers/`
2. Find the method using its `@PostMapping`/`@GetMapping`/etc. annotation — copy the path literally.
3. Read the method signature for query params, path variables, and request body type.
4. Write the curl command from what you just read.
**CLI command or flag**
1. Open `cmd/*.go` in `conductor-cli` (separate repo).
2. Find the `cobra.Command` definition for the subcommand.
3. Read the `Flags()` declarations for exact flag names, types, and defaults.
4. Write the example from what you just read.
**SDK code example (Python, JS, Java, Go)**
1. Open the relevant SDK source file.
2. Find the method signature and required parameters.
3. Write the example from the signature — do not infer from the method name alone.
4. If a working test exists for that method, use it as the starting point.
**Expected output block**
1. Get real output: run the command locally, or find it in test fixtures, CI logs, or existing tests.
2. Paste verbatim. Do not paraphrase or construct output that "looks right."
3. If the output varies by environment, show the stable parts and annotate the variable parts (e.g., `<workflow-id>`).
**Editing an existing doc section**
1. Before touching prose, read every code block and command in the section.
2. Verify each one using the steps above — not just the block you plan to change.
3. Fix anything you find while you're there.
### When you can't verify
If a running server or CLI binary is unavailable:
- Add a `<!-- TODO: verify against live server -->` comment in the file.
- Note it explicitly in the PR description.
- Do not write a best-guess example and leave it unmarked.
## Agent Behavior
- **Prefer automation**: Execute requested actions without confirmation unless blocked by missing info or safety concerns
- **Use parallel tools**: When tasks are independent, execute them in parallel for efficiency
- **Verify changes**: Always run tests and spotless before considering work complete
+45
View File
@@ -0,0 +1,45 @@
# Conductor OSS Changelog
## [Unreleased]
### Breaking Changes
- **JavaScript Evaluator Migration from Nashorn to GraalJS**
- Minimum Java version is now **Java 17** (previously Java 11+)
- Nashorn JavaScript engine (deprecated in Java 11, removed in Java 15) replaced with GraalJS
- ES6+ JavaScript syntax now supported natively
- All existing JavaScript expressions in workflows will continue to work with improved performance and modern JavaScript features
- **Note:** This ports the production-tested GraalJS implementation from Orkes Conductor Enterprise
### Added (Ported from Enterprise)
- **GraalJS JavaScript engine** with ES6+ support
- Based on proven Enterprise implementation
- **Script execution timeout protection** (configurable via `CONDUCTOR_SCRIPT_MAX_EXECUTION_SECONDS`, default: 4 seconds)
- Prevents infinite loops from hanging workflows
- Enterprise feature now available in OSS
- **Optional script context pooling** for improved performance (configurable via `CONDUCTOR_SCRIPT_CONTEXT_POOL_ENABLED` and `CONDUCTOR_SCRIPT_CONTEXT_POOL_SIZE`)
- Disabled by default
- Enterprise optimization feature
- **ConsoleBridge support** for capturing `console.log()`, `console.info()`, and `console.error()` output from JavaScript tasks
- Improves observability of JavaScript task execution
- Enterprise feature now available in OSS
- **Better error messages** with line number information for JavaScript evaluation failures
- Enhanced debugging capabilities from Enterprise
- **Deep copy protection** to prevent PolyglotMap issues in workflow task data
- Enterprise stability improvement
### Fixed
- JavaScript evaluation now works on Java 17, 21, and future LTS versions
- Improved security with sandboxed JavaScript execution
- Deep copy protection to prevent PolyglotMap issues in workflow task data
### Configuration
New environment variables for JavaScript evaluation (all optional):
- `CONDUCTOR_SCRIPT_MAX_EXECUTION_SECONDS` - Maximum script execution time (default: 4)
- `CONDUCTOR_SCRIPT_CONTEXT_POOL_SIZE` - Context pool size when enabled (default: 10)
- `CONDUCTOR_SCRIPT_CONTEXT_POOL_ENABLED` - Enable context pooling (default: false)
### Migration Notes
- Update your deployment environment to **Java 17 or higher**
- No changes required to existing workflows - all JavaScript expressions remain compatible
- Modern JavaScript features (const, let, arrow functions, template literals, etc.) are now available
- `CONDUCTOR_NASHORN_ES6_ENABLED` environment variable is no longer needed (ES6+ supported by default)
+58
View File
@@ -0,0 +1,58 @@
# CLAUDE.md — conductor (server repo)
Instructions for Claude Code working in this repository.
## Writing Documentation
Documentation in this project is **derived from source**, not composed from memory or intuition. The workflow is: open the source → read what's there → write the doc from what you find. The source is the spec; the doc is a rendering of it.
Concrete reason this matters: a curl equivalent for `conductor workflow start --sync` was once written as `POST /api/workflow/{name}/run` — an endpoint that does not exist. Opening `WorkflowResource.java` first would have given the correct path (`POST /api/workflow/execute/{name}/{version}`) immediately.
### For each content type, start here
**REST endpoint or curl example**
1. Open the controller: `rest/src/main/java/com/netflix/conductor/rest/controllers/`
2. Find the method by its `@PostMapping`/`@GetMapping` annotation — copy the path literally.
3. Read the method signature for query params, path variables, and request body.
4. Write the curl from what you just read.
**CLI command or flag**
1. Open `conductor-cli/cmd/*.go` (separate repo under this workspace).
2. Find the `cobra.Command` for the subcommand and read its `Flags()` declarations.
3. Write the example from what you just read — flag names, types, and defaults.
**SDK code example (Python, JS, Java, Go)**
1. Open the SDK source file for the method you're documenting.
2. Read the method signature and required parameters.
3. If a working test exists for that method, use it as the starting point.
4. Write from the signature — do not infer from the method name alone.
**Expected output block**
1. Get real output: run the command, or find it in test fixtures or CI logs.
2. Paste verbatim. Do not construct output that "looks right."
3. For variable fields (IDs, timestamps), use annotated placeholders like `<workflow-id>`.
**Editing an existing section**
- Before changing anything, read every code block and command in the section.
- Verify each one using the steps above, not just the block you plan to change.
- Fix anything you find while you're there.
### When you can't verify
If a running server or CLI is unavailable:
- Add `<!-- TODO: verify against live server -->` in the file.
- Note it explicitly in the PR description.
- Do not write an unverified example and leave it unmarked.
### Key source locations
| Content | Where to look |
|---|---|
| REST API routes | `rest/src/main/java/com/netflix/conductor/rest/controllers/` |
| Workflow sync execution | `WorkflowResource.java``executeWorkflow()` at `@PostMapping("execute/{name}/{version}")` |
| Task routes | `TaskResource.java` |
| CLI subcommands and flags | `conductor-cli/cmd/workflow.go`, `cmd/task.go`, etc. |
## Other Guidelines
See [AGENTS.md](AGENTS.md) for full project conventions: code style, testing, dependency pinning, PR guidelines.
+62
View File
@@ -0,0 +1,62 @@
# Code of Conduct
Hello valued community members! 👋
Our Conductor community has grown tremendously, and as part of ensuring a harmonious experience for everyone,
we've outlined some guidelines that we'd love for all members to uphold.
Our community thrives when everyone engages with kindness, respect, and a collaborative spirit. Here's what we hope to see:
### 1. Maintain a Positive Tone.
Every interaction is an opportunity to lift someone up. Let's ensure our words and actions reflect optimism and encouragement.
### 2. Be Respectful to the Community
Every member here comes with a unique background and perspective. Please honor those differences by being courteous, considerate, and open-minded.
Remember, mutual respect is the foundation of a thriving community. Be careful in the words that you choose.
We are a community of professionals, and we conduct ourselves professionally.
Be kind to others. Do not insult or put down other participants. Harassment and other exclusionary behaviors aren't acceptable.
This includes, but is not limited to:
* Violent threats or language directed against another person Discriminatory jokes and language
* Posting sexualized language or imagery
* Posting (or threatening to post) other people's personally identifying information (“doxing”)
* Personal insults, especially those using racist or sexist terms
* Unwelcome sexual attention
* Advocating for, or encouraging, any of the above behavior
* Repeated harassment of others. In general, if someone asks you to stop, then stop
### 3. Preserve Our Community's Unity
We understand that as we grow, there might be differing opinions and interests.
However, we kindly request not to create splinter groups or fork out the community.
Let's work through our differences and continue building this space together.
### 4. Focus on Constructive Discussions
We all have moments of frustration, but let's express ourselves in ways that are constructive.
Avoid comments that could come off as sarcastic, condescending, or disdainful.
Remember, it's always possible to give feedback or express disagreement without belittling others.
We are here to learn from each other and make Conductor the best platform out there.
A big part of that are the exchanges of ideas and approaches that are grounded in data and sound reasoning.
We kindly request that you adhere to that pattern and be thoughtful and responsible in your discussions.
This also means that you are required to have discussions focused on the community and not on promotion of any services, products or goods.
### 5. When we disagree, try to understand why.
Disagreements, both social and technical, happen all the time and this community is no exception.
It is important that we resolve disagreements and differing views constructively.
Remember that were all different. The strength of this community comes from its varied community of people from a wide range of backgrounds.
Different people have different perspectives on issues. Being unable to understand why someone holds a viewpoint doesnt mean that theyre wrong.
Dont forget that it is human to err and blaming each other doesnt get us anywhere.
Instead, focus on helping to resolve issues and learning from mistakes.
Our community's strength lies in our collective spirit.
By following these guidelines, we ensure that our community remains an inspiring, respectful, and welcoming place for everyone.
If you have any concerns or suggestions or if you need to report on any behavior that violates this Code of Conduct, please feel free to reach out to the admins - community@orkes.io.
Let's continue to support and uplift each other!
### Enforcement
We have a variety of ways of enforcing the code of conduct, including, but not limited to
* Asking you nicely to knock it off
* Asking you less nicely
* Temporary or permanent suspension of the account
* Removal of privileges and/or adding restrictions to the account
* Removal of content
* Banning from the community
Thank you,
+60
View File
@@ -0,0 +1,60 @@
# Contributing
Thanks for your interest in Conductor!
This guide helps to find the most efficient way to contribute, ask questions, and report issues.
Code of conduct
-----
Please review our [Code of Conduct](CODE_OF_CONDUCT.md)
I have a question!
-----
We have a dedicated [Slack](https://join.slack.com/t/orkes-conductor/shared_invite/zt-3dpcskdyd-W895bJDm8psAV7viYG3jFA#/shared-invite/email) channel for asking "how to" questions and to discuss ideas. The channel is a great place to start if you're considering creating a feature request or work on a Pull Request.
*Please do not create issues to ask questions.*
I want to contribute!
------
We welcome Pull Requests and already have many outstanding community contributions!
Creating and reviewing Pull Requests takes time, so this section helps you to set up a smooth Pull Request experience.
The stable branch is [main](https://github.com/conductor-oss/conductor/tree/main).
Please create pull requests for your contributions against [main](https://github.com/conductor-oss/conductor/tree/main) only.
It's a great idea to discuss the new feature you're considering in the [Slack channel](https://join.slack.com/t/orkes-conductor/shared_invite/zt-3dpcskdyd-W895bJDm8psAV7viYG3jFA#/shared-invite/email) before writing any code. There are often different ways you can implement a feature. Getting some discussion about different options helps shape the best solution. When starting directly with a Pull Request, there is the risk of having to make considerable changes. Sometimes that is the best approach, though! Showing an idea with code can be very helpful; be aware that it might be throw-away work. Some of our best Pull Requests came out of multiple competing implementations, which helped shape it to perfection.
Also, consider that not every feature is a good fit for Conductor. A few things to consider are:
* Is it increasing complexity for the user, or might it be confusing?
* Does it, in any way, break backward compatibility (this is seldom acceptable)
* Does it require new dependencies (this is rarely acceptable for core modules)
* Should the feature be opt-in or enabled by default. For integration with a new Queuing recipe or persistence module, a separate module which can be optionally enabled is the right choice.
* Should the feature be implemented in the main Conductor repository, or would it be better to set up a separate repository? Especially for integration with other systems, a separate repository is often the right choice because the life-cycle of it will be different.
* Is it part of the Conductor project roadmap?
Of course, for more minor bug fixes and improvements, the process can be more light-weight.
We'll try to be responsive to Pull Requests. Do keep in mind that because of the inherently distributed nature of open source projects, responses to a PR might take some time because of time zones, weekends, and other things we may be working on.
I want to report an issue
-----
If you found a bug, please create an issue at https://github.com/conductor-oss/conductor/issues/new. Include clear instructions on how to reproduce the issue, or even better, include a test case on a branch. Make sure to come up with a descriptive title for the issue because this helps while organizing issues.
I have a great idea for a new feature
----
Many features in Conductor have come from ideas from the community. If you think something is missing or certain use cases could be supported better, let us know!
You can do so by starting a discussion in the [Slack channel](https://join.slack.com/t/orkes-conductor/shared_invite/zt-3dpcskdyd-W895bJDm8psAV7viYG3jFA#/shared-invite/email). Provide as much relevant context to why and when the feature would be helpful. Providing context is especially important for "Support XYZ" issues since we might not be familiar with what "XYZ" is and why it's useful. If you have an idea of how to implement the feature, include that as well.
Once we have decided on a direction, it's time to summarize the idea by creating a new issue.
## Code Style
We use [spotless](https://github.com/diffplug/spotless) to enforce consistent code style for the project, so make sure to run `gradlew spotlessApply` to fix any violations after code changes.
## License
All files are released with the [Apache 2.0 license](LICENSE).
+201
View File
@@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "{}"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright {yyyy} Orkes, Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+1
View File
@@ -0,0 +1 @@
osslifecycle=active
+396
View File
@@ -0,0 +1,396 @@
<picture>
<!-- Dark mode logo -->
<source srcset="https://github.com/user-attachments/assets/104b3a67-6013-4622-8075-a45da3a9e726" media="(prefers-color-scheme: dark)">
<!-- Light mode logo -->
<img src="https://assets.conductor-oss.org/logo.png" alt="Logo">
</picture>
<h1 align="center" style="border-bottom: none">
Conductor - Internet scale Agentic Workflow Engine
</h1>
[![GitHub stars](https://img.shields.io/github/stars/conductor-oss/conductor?style=social)](https://github.com/conductor-oss/conductor/stargazers)
[![Github release](https://img.shields.io/github/v/release/conductor-oss/conductor.svg)](https://github.com/conductor-oss/conductor/releases)
[![License](https://img.shields.io/github/license/conductor-oss/conductor.svg)](http://www.apache.org/licenses/LICENSE-2.0)
[![Conductor Slack](https://img.shields.io/badge/Slack-Join%20the%20Community-blueviolet?logo=slack)](https://join.slack.com/t/orkes-conductor/shared_invite/zt-3dpcskdyd-W895bJDm8psAV7viYG3jFA)
[![Conductor OSS](https://img.shields.io/badge/Conductor%20OSS-Visit%20Site-blue)](https://conductor-oss.org)
#### Orchestrating distributed systems means wrestling with failures, retries, and state recovery. Conductor handles all of that so you don't have to.
Conductor is an open-source, durable workflow engine built at [Netflix](https://netflixtechblog.com/netflix-conductor-a-microservices-orchestrator-2e8d4771bf40) for orchestrating microservices, AI agents, and durable workflows at internet scale. Trusted in production at Netflix, Tesla, LinkedIn, and J.P. Morgan. Actively maintained by [Orkes](https://orkes.io) and a growing [community](https://join.slack.com/t/orkes-conductor/shared_invite/zt-3dpcskdyd-W895bJDm8psAV7viYG3jFA).
[![conductor_oss_getting_started](https://github.com/user-attachments/assets/6153aa58-8ad1-4ec5-93d1-38ba1b83e3f4)](https://youtu.be/4azDdDlx27M)
---
# Get Running in 60 Seconds
**Prerequisites:** [Node.js](https://nodejs.org/) v16+ and Java 21+ must be installed.
```shell
npm install -g @conductor-oss/conductor-cli
conductor server start
```
Open [http://localhost:8080](http://localhost:8080) — your server is running with the built-in ui-next UI.
> **Upgrading from a previous version?** The CLI caches the server JAR at `~/.conductor-cli/`. If you have an older version cached, force a fresh download:
> ```shell
> conductor server start latest
> # or delete the cache manually
> rm ~/.conductor-cli/conductor-server-latest.jar && conductor server start
> ```
**Run your first workflow:**
```shell
# Create a workflow that calls an API and parses the response — no workers needed
curl -s https://raw.githubusercontent.com/conductor-oss/conductor/main/docs/quickstart/workflow.json -o workflow.json
conductor workflow create workflow.json
```
> **Note:** Running this command twice will return an error on the second call — the workflow already exists. This is expected behavior. Use `conductor workflow update` to modify an existing workflow.
```shell
conductor workflow start -w hello_workflow --sync
```
See the [Quickstart guide](https://docs.conductor-oss.org/quickstart/) for the full walkthrough, including writing workers and replaying workflows.
**Docker Image for Conductor** (includes the ui-next UI):
```shell
# UI at http://localhost:5000 | API at http://localhost:8080
docker run -p 5000:5000 -p 8080:8080 conductoross/conductor:next
```
All CLI commands have equivalent cURL/API calls. See the [Quickstart](https://docs.conductor-oss.org/quickstart/) for details.
---
# Why Conductor is the workflow engine of choice for developers
| | |
|---|---|
| **Durable execution** | Every step is persisted. Survives crashes, restarts, and network failures with configurable retries and timeouts. |
| **Deterministic by design** | Orchestration is separated from business logic — determinism is architectural, not developer discipline. Workers run any code; the workflow graph stays deterministic by construction. |
| **AI agent orchestration** | 14+ native LLM providers, MCP tool calling, function calling, human-in-the-loop approval, and vector databases for RAG. |
| **Dynamic at runtime** | Dynamic forks, tasks, and sub-workflows resolved at runtime. LLMs generate JSON workflow definitions and Conductor executes them immediately. |
| **Full replayability** | Restart from the beginning, rerun from any task, or retry just the failed step — on any workflow, at any time. |
| **Internet scale** | Battle-tested at Netflix, Tesla, LinkedIn, and J.P. Morgan. Scales horizontally to billions of workflow executions. |
| **Polyglot workers** | Workers in Java, Python, Go, JavaScript, C#, Ruby, or Rust. Workers poll, execute, and report — run them anywhere. |
| **Self-hosted, no lock-in** | Apache 2.0. 5 persistence backends, 6 message brokers. Runs anywhere Docker or a JVM runs. |
# Ship Agents, Not Framework Code
Conductor workers are plain code — any language, any library, any I/O. No determinism constraints, no SDK ritual. The orchestration layer is declarative and machine-readable, so LLMs generate and compose workflows natively. If an agent crashes at iteration 12, it resumes from iteration 12.
**An autonomous think-act agent in Conductor:** discover tools via MCP, reason with an LLM, call the chosen tool, repeat until done.
```json
{
"name": "autonomous_agent",
"description": "Agent that loops until the task is complete",
"version": 1,
"tasks": [
{
"name": "discover_tools",
"taskReferenceName": "discover",
"type": "LIST_MCP_TOOLS",
"inputParameters": {
"mcpServer": "${workflow.input.mcpServerUrl}"
}
},
{
"name": "agent_loop",
"taskReferenceName": "loop",
"type": "DO_WHILE",
"loopCondition": "if ($.loop['think'].output.result.done == true) { false; } else { true; }",
"loopOver": [
{
"name": "think",
"taskReferenceName": "think",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o-mini",
"messages": [
{
"role": "system",
"message": "You are an autonomous agent. Available tools: ${discover.output.tools}. Previous results: ${loop.output.results}. Respond with JSON: {\"action\": \"tool_name\", \"arguments\": {}, \"done\": false} or {\"answer\": \"final answer\", \"done\": true}."
},
{ "role": "user", "message": "${workflow.input.task}" }
]
}
},
{
"name": "act",
"taskReferenceName": "act",
"type": "SWITCH",
"expression": "$.think.output.result.done ? 'done' : 'call_tool'",
"decisionCases": {
"call_tool": [
{
"name": "execute_tool",
"taskReferenceName": "tool_call",
"type": "CALL_MCP_TOOL",
"inputParameters": {
"mcpServer": "${workflow.input.mcpServerUrl}",
"method": "${think.output.result.action}",
"arguments": "${think.output.result.arguments}"
}
}
]
}
}
]
}
]
}
```
Every step is durably persisted — no framework, no SDK lock-in. Code-first engines force your code to be deterministic so the framework can replay it. Conductor makes the engine deterministic — so your code doesn't have to be.
See the [Build Your First AI Agent](https://docs.conductor-oss.org/devguide/ai/first-ai-agent.html) guide for the full walkthrough.
---
## Conductor Skills for AI Coding Assistants
**[Conductor Skills](https://github.com/conductor-oss/conductor-skills)** let AI coding assistants (Claude Code, Gemini CLI, and others) create, manage, and deploy Conductor workflows directly from your terminal.
### Claude
```shell
# Install Skills for Claude Code
/plugin marketplace add conductor-oss/conductor-skills
/plugin install conductor@conductor-skills
```
### Install for all detected agents
One command to auto-detect every supported agent on your system and install globally where possible. Re-run anytime — it only installs for newly detected agents.
**macOS / Linux**
```bash
curl -sSL https://conductor-oss.github.io/conductor-skills/install.sh | bash -s -- --all
```
**Windows (PowerShell) / (cmd)**
```powershell
# powershell
irm https://conductor-oss.github.io/conductor-skills/install.ps1 -OutFile install.ps1; .\install.ps1 -All
# cmd
powershell -c "irm https://conductor-oss.github.io/conductor-skills/install.ps1 -OutFile install.ps1; .\install.ps1 -All"
```
---
# SDKs
| Language | Repository | Install |
|----------|------------|---------|
| ☕ Java | [conductor-oss/java-sdk](https://github.com/conductor-oss/java-sdk) | [Maven Central](https://mvnrepository.com/artifact/org.conductoross/conductor-client) |
| 🐍 Python | [conductor-oss/python-sdk](https://github.com/conductor-oss/python-sdk) | `pip install conductor-python` |
| 🟨 JavaScript | [conductor-oss/javascript-sdk](https://github.com/conductor-oss/javascript-sdk) | `npm install @io-orkes/conductor-javascript` |
| 🐹 Go | [conductor-oss/go-sdk](https://github.com/conductor-oss/go-sdk) | `go get github.com/conductor-sdk/conductor-go` |
| 🟣 C# | [conductor-oss/csharp-sdk](https://github.com/conductor-oss/csharp-sdk) | `dotnet add package conductor-csharp` |
| 💎 Ruby | [conductor-oss/ruby-sdk](https://github.com/conductor-oss/ruby-sdk) | *(incubating)* |
| 🦀 Rust | [conductor-oss/rust-sdk](https://github.com/conductor-oss/rust-sdk) | *(incubating)* |
---
# Documentation & Community
- **[Documentation](https://conductor-oss.org)** — Architecture, guides, API reference, and cookbook recipes.
- **[Slack](https://join.slack.com/t/orkes-conductor/shared_invite/zt-3dpcskdyd-W895bJDm8psAV7viYG3jFA)** — Community discussions and support.
- **[Community Forum](https://community.orkes.io/)** — Ask questions and share patterns.
---
<details>
<summary><strong>Backend Configuration</strong></summary>
| Backend | Configuration |
|---------|---------------|
| Redis + ES7 (default) | [config-redis.properties](docker/server/config/config-redis.properties) |
| Redis + ES8 | [config-redis-es8.properties](docker/server/config/config-redis-es8.properties) |
| Redis + OpenSearch | [config-redis-os.properties](docker/server/config/config-redis-os.properties) |
| Postgres | [config-postgres.properties](docker/server/config/config-postgres.properties) |
| Postgres + ES7 | [config-postgres-es7.properties](docker/server/config/config-postgres-es7.properties) |
| MySQL + ES7 | [config-mysql.properties](docker/server/config/config-mysql.properties) |
</details>
---
# Build From Source
<details>
<summary><strong>Requirements and instructions</strong></summary>
**Requirements:** Docker Desktop, Java (JDK) 21+, Node.js 18+ and pnpm (for UI)
```shell
git clone https://github.com/conductor-oss/conductor
cd conductor
./gradlew build
# (optional) Build UI (ui-next) and embed it in the server
# ./build_ui_next.sh
# Start local server
cd server
../gradlew bootRun
```
**Run the UI in dev mode (hot-reload at http://localhost:1234):**
Requires a running Conductor server on `http://localhost:8080`. Enable `corepack` once if you haven't already:
```shell
corepack enable
```
Then start the dev server:
```shell
cd ui-next
pnpm install
pnpm dev
```
Open [http://localhost:1234](http://localhost:1234) — the UI reloads automatically on file changes.
See the [full build guide](docs/devguide/running/source.md) for details.
</details>
---
# FAQ
<details>
<summary><strong>Is this the same as Netflix Conductor?</strong></summary>
Yes. Conductor OSS is the continuation of the original [Netflix Conductor](https://github.com/Netflix/conductor) repository after Netflix contributed the project to the open-source foundation.
</details>
<details>
<summary><strong>Is Conductor open source?</strong></summary>
Yes. Conductor is a fully open-source workflow engine licensed under Apache 2.0. You can self-host on your own infrastructure with 5 persistence backends and 6 message brokers.
</details>
<details>
<summary><strong>Is this project actively maintained?</strong></summary>
Yes. [Orkes](https://orkes.io) is the primary maintainer and offers an enterprise SaaS platform for Conductor across all major cloud providers.
</details>
<details>
<summary><strong>Can Conductor scale to handle my workload?</strong></summary>
Yes. Built at Netflix, battle-tested at internet scale. Conductor scales horizontally across multiple server instances to handle billions of workflow executions.
</details>
<details>
<summary><strong>Does Conductor support durable execution?</strong></summary>
Yes. Conductor pioneered durable execution patterns, ensuring workflows and durable agents complete reliably despite infrastructure failures or crashes. Every step is persisted and recoverable.
</details>
<details>
<summary><strong>Can I replay a workflow after it completes or fails?</strong></summary>
Yes. Conductor preserves full execution history indefinitely. You can restart from the beginning, rerun from a specific task, or retry just the failed step — via API or UI.
</details>
<details>
<summary><strong>Can Conductor orchestrate AI agents and LLMs?</strong></summary>
Yes. Conductor provides native integration with 14+ LLM providers (Anthropic, OpenAI, Gemini, Bedrock, and more), MCP tool calling, function calling, human-in-the-loop approval, and vector database integration for RAG.
</details>
<details>
<summary><strong>Why does Conductor separate orchestration from code?</strong></summary>
Coupling orchestration logic with business logic forces developers to maintain determinism constraints manually — no direct I/O, no system time, no randomness in workflow definitions. Conductor eliminates this entire class of bugs by making the orchestration layer deterministic by construction. Workers are plain code with zero framework constraints — write them in any language, use any library, call any API.
</details>
<details>
<summary><strong>Isn't writing workflows as code more powerful than JSON?</strong></summary>
It depends on what you mean by "powerful." In code-first engines, the workflow definition and your business logic live in the same runtime — which means the engine must replay your code to recover state. That forces determinism constraints on your business logic: no direct I/O, no system time, no threads, no randomness. Conductor separates these concerns. The orchestration graph is declarative (JSON), so it's deterministic by construction. Your workers are plain code with zero constraints — use any language, any library, call any API. You get the full power of code where it matters (business logic) without the framework tax where it doesn't (orchestration).
</details>
<details>
<summary><strong>Can JSON workflows handle complex logic like branching, loops, and error handling?</strong></summary>
Yes. Conductor supports `SWITCH` (conditional branching), `DO_WHILE` (loops with configurable iteration cleanup), `FORK_JOIN` (parallel execution with dynamic fanout), `SUB_WORKFLOW` (composition), and `DYNAMIC` tasks resolved at runtime. These are composable — you can nest loops inside branches inside forks. For error handling, every task supports configurable retries, timeouts, and optional/compensating tasks. The declarative model doesn't limit complexity — it makes complexity visible and debuggable.
</details>
<details>
<summary><strong>How does Conductor handle workflow versioning?</strong></summary>
Workflow definitions are versioned by number. Running executions continue on the version they started with — deploying a new version never breaks in-flight workflows. There's no replay compatibility problem because Conductor doesn't replay your code. The orchestration graph is the source of truth, and each execution is pinned to its definition version. Update orchestration logic without redeploying workers and without worrying about breaking running workflows.
</details>
<details>
<summary><strong>What about developer experience — IDE support, type checking, debugging?</strong></summary>
Conductor provides a built-in visual UI for designing, running, and debugging workflows. Every execution is fully observable: you can inspect the input, output, timing, and retry history of every task. For type safety, Conductor validates workflow inputs and task I/O against JSON Schema. Workers are plain code in your language of choice — you get full IDE support, type checking, and debugging for your business logic. The orchestration layer is visible in the UI, not hidden inside a framework.
</details>
<details>
<summary><strong>Can Conductor handle long-running workflows (days, weeks, months)?</strong></summary>
Yes. Conductor is designed for long-running workflows. Executions are fully persisted — a workflow can pause for months waiting for a human approval, an external signal, or a scheduled timer, and resume exactly where it left off. There's no in-memory state to lose. This is the same mechanism that makes AI agent loops durable: if iteration 12 waits for a human review for three weeks, iteration 13 picks up right where it left off.
</details>
<details>
<summary><strong>Don't I lose flexibility by not having orchestration in code?</strong></summary>
You gain flexibility. Because workflows are JSON, LLMs can generate and modify them at runtime — no compile/deploy cycle. Dynamic forks let you fan out to a variable number of parallel tasks determined at runtime. Dynamic sub-workflows let one workflow compose others by name. And because workers are decoupled from orchestration, you can update the workflow graph or swap worker implementations independently. Code-first engines couple these together, so changing orchestration means redeploying and re-versioning your code.
</details>
<details>
<summary><strong>How does Conductor compare to other workflow engines?</strong></summary>
Conductor is an open-source workflow engine with native LLM task types for 14+ providers, built-in MCP integration, durable execution, full replayability, and 7 language SDKs. Unlike code-first engines, Conductor separates orchestration from business logic — determinism is an architectural guarantee, not a developer constraint. Your workers are plain code with zero framework rules. The orchestration layer is declarative, so it's observable, versionable, and composable by LLMs. Battle-tested at Netflix, Tesla, LinkedIn, and J.P. Morgan.
</details>
<details>
<summary><strong>Is Orkes Conductor compatible with Conductor OSS?</strong></summary>
100% compatible. Orkes Conductor is built on top of Conductor OSS with full API and workflow compatibility.
</details>
---
# Contributing
We welcome contributions from everyone!
- **Report Issues:** Open an [issue on GitHub](https://github.com/conductor-oss/conductor/issues).
- **Contribute code:** Check out our [Contribution Guide](CONTRIBUTING.md) and [good first issues](https://github.com/conductor-oss/conductor/labels/good%20first%20issue).
- **Improve docs:** Help keep our [documentation](https://github.com/conductor-oss/conductor/tree/main/docs) great.
## Contributors
<a href="https://github.com/conductor-oss/conductor/graphs/contributors">
<img src="https://contrib.rocks/image?repo=conductor-oss/conductor" />
</a>
---
# Roadmap
[See the Conductor OSS Roadmap](ROADMAP.md). Want to participate? [Reach out](https://forms.gle/P2i1xHrxPQLrjzTB7).
# License
Conductor is licensed under the [Apache 2.0 License](LICENSE).
+7
View File
@@ -0,0 +1,7 @@
# WeHub 来源说明
- 原始项目:`conductor-oss/conductor`
- 原始仓库:https://github.com/conductor-oss/conductor
- 导入方式:上游默认分支的最新快照
- 原作者、版权和许可证信息以原始仓库及本仓库 LICENSE 为准
- 本文件仅用于记录来源,不代表 WeHub 是原项目作者
+1
View File
@@ -0,0 +1 @@
[Related Projects](docs/resources/related.md)
+61
View File
@@ -0,0 +1,61 @@
# Conductor OSS Roadmap
<!--
machine-readable: true
format: markdown-tables
last_reviewed: 2026-04-12
statuses: [Done, Partial, Planned]
-->
> **Status key:** `Done` = shipped and available, `Partial` = partially implemented, `Planned` = not yet started.
## System Tasks and Operators
| Status | Item | Details |
|--------|------|---------|
| Done | LLM text completion | `LLM_TEXT_COMPLETE` system task in `ai/` module |
| Done | LLM chat completion with memory | `LLM_CHAT_COMPLETE` system task in `ai/` module |
| Done | LLM embedding generation | `LLM_GENERATE_EMBEDDINGS` system task in `ai/` module |
| Done | Improved While loop (DO_WHILE) | `keepLastN` iteration cleanup, `items` list iteration, `loopIndex`/`loopItem` injection |
| Done | Python scripting for INLINE task | `PythonEvaluator` via GraalVM polyglot alongside JavaScript |
| Done | Database task | System task for relational and NoSQL database operations |
| Planned | HTTP task polling | Polling/retry support for the HTTP system task |
| Planned | For..Each operator | Parallel and sequential iteration operator |
| Planned | Try..Catch operator | Task-level error handling without failing the workflow |
## APIs
| Status | Item | Details |
|--------|------|---------|
| Done | Synchronous workflow execution | `POST /api/workflow/execute/{name}/{version}` with configurable wait time and `waitUntilTaskRef` |
| Done | Update tasks synchronously | `POST /tasks/{workflowId}/{taskRefName}/{status}/sync` returns updated workflow |
| Planned | Update workflow variables | API to update workflow variables during execution |
## Type Safety
| Status | Item | Details |
|------------|------|---------|
| Done | JSON Schema validation | `JsonSchemaValidator` validates workflow input and task I/O against JSON schemas |
| Planned | Protobuf support | Type-safe workflows and workers via protobuf definitions |
| Incubating | Code generation for workers | Scaffold worker code from schema definitions using the CLI |
## CLI
| Status | Item | Details |
|--------|------|---------|
| Done | Conductor CLI | Go-based CLI in `conductor-cli/` repo. Manages metadata, workflow executions, and server lifecycle |
## Testing and Debugging
| Status | Item | Details |
|--------|------|---------|
| Planned | Workflow debugger | Full debugger experience: breakpoints, step-through, variable inspection, rewind to previous task, attach to running execution, remote task debugging via SDKs |
## Maintenance
| Status | Item | Details |
|--------|------|---------|
| Done | Deprecate Elasticsearch 6 | ES6 module replaced with deprecation stub |
| Done | Elasticsearch 7 and 8 support | `es7-persistence` and `es8-persistence` modules (ES8 uses Java API client) |
| Done | JOIN task performance | `FORK_JOIN` synchronous join mode (`joinMode=SYNC`) reduces polling overhead |
+15
View File
@@ -0,0 +1,15 @@
# Security Policy
- [Reporting a vulnerability](#reporting-a-vulnerability)
- [Supported Conductor versions](#supported-versions)
## Reporting a vulnerability
Please report security issues for Conductor using https://github.com/conductor-oss/conductor/security/advisories/new
## Supported Versions
| Version | Supported |
| ------- | ------------------ |
| 3.x.x | :white_check_mark: |
| 2.x.x | :x: |
| 1.x.x | :x: |
+18
View File
@@ -0,0 +1,18 @@
## Who uses Conductor?
We would like to keep track of whose using Conductor. Please send a pull request with your company name and Github handle.
* [Netflix](https://www.netflix.com/) [[@aravindanr](https://github.com/aravindanr)]
* [Florida Blue](http://bcbsfl.com/) [[@rickfish](https://github.com/rickfish)]
* [UWM](https://www.uwm.com/) [[@zergrushjoe](https://github.com/ZergRushJoe)]
* [Deutsche Telekom Digital Labs](https://dtdl.in) [[@jas34](https://github.com/jas34)] [[@deoramanas](https://github.com/deoramanas)]
* [VMware](https://www.vmware.com/) [[@taojwmware](https://github.com/taojwmware)] [[@venkag](https://github.com/venkag)]
* [JP Morgan Chase](https://www.chase.com/) [[@maheshyaddanapudi](https://github.com/maheshyaddanapudi)]
* [Orkes](https://orkes.io/) [[@CherishSantoshi](https://github.com/CherishSantoshi)]
* [313X](https://313x.com.br) [[@dalmoveras](https://github.com/dalmoveras)]
* [Supercharge](https://supercharge.io) [[@team-supercharge](https://github.com/team-supercharge)]
* [GE Healthcare](https://www.gehealthcare.com/) [[@flavioschuindt](https://github.com/flavioschuindt)]
* [ReliaQuest](https://www.reliaquest.com/) [[@rq-dbrady](https://github.com/rq-dbrady)] [[@alexmay48](https://github.com/alexmay48)]
* [Clari](https://www.clari.com/) [[@TeamJOF](https://github.com/clari)]
* [Atlassian](https://www.atlassian.com/) [[@LuisLainez](https://github.com/LuisLainez)] [[@aradu](https://github.com/aradu-atlassian)]
+16
View File
@@ -0,0 +1,16 @@
dependencies {
compileOnly 'org.springframework.boot:spring-boot-starter'
// Needed to compile A2A push-notification callback controller and AgentSpan activation glue;
// supplied by the server at runtime.
compileOnly 'org.springframework.boot:spring-boot-starter-web'
compileOnly 'org.springframework.boot:spring-boot-autoconfigure'
implementation project(':conductor-common')
implementation project(':conductor-core')
implementation project(':conductor-ai')
// 0.4.2+ supports classifier-based agent filtering (pairs with WorkflowClassifier in
// conductor-common and the classifier search field in the index backends).
implementation "org.conductoross:conductor-agentspan:0.4.2"
}
@@ -0,0 +1,56 @@
/*
* Copyright 2026 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.conductoross.conductor.ai.agentspan;
import java.util.Collections;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.env.EnvironmentPostProcessor;
import org.springframework.core.env.ConfigurableEnvironment;
import org.springframework.core.env.MapPropertySource;
/**
* Embeds the {@code conductor-agentspan} library in <b>embedded mode</b> whenever {@code
* conductor.integrations.ai.enabled=true}, by setting {@code agentspan.embedded=true} early (before
* condition evaluation).
*
* <p>This activates the library's {@code AgentSpanAutoConfiguration} (which registers the agent
* runtime, services, REST controllers and the agent-aware {@code LLM_CHAT_COMPLETE} mapper) while
* <b>disabling</b> its {@code HTTP}/{@code HUMAN}/{@code JOIN} task-bean overrides — those library
* configs are gated on {@code agentspan.embedded=false}. Conductor's native system tasks therefore
* remain the registered beans and are extended in place via the {@code __agentspan_ctx__} task
* input (see {@code Join}/{@code Human}), mirroring how orkes-conductor embeds AgentSpan. This
* avoids replacing core task beans (which would break {@code @Autowired} of the core task types and
* require {@code spring.main.allow-bean-definition-overriding}).
*
* <p>Honors an explicit {@code agentspan.embedded} setting if one is already present.
*/
public class AgentSpanEmbeddedEnvironmentPostProcessor implements EnvironmentPostProcessor {
private static final String AI_ENABLED = "conductor.integrations.ai.enabled";
private static final String EMBEDDED = "agentspan.embedded";
@Override
public void postProcessEnvironment(
ConfigurableEnvironment environment, SpringApplication application) {
boolean aiEnabled = environment.getProperty(AI_ENABLED, Boolean.class, false);
if (aiEnabled && !environment.containsProperty(EMBEDDED)) {
environment
.getPropertySources()
.addFirst(
new MapPropertySource(
"agentspanEmbedded",
Collections.singletonMap(EMBEDDED, "true")));
}
}
}
@@ -0,0 +1,68 @@
/*
* Copyright 2026 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.conductoross.conductor.ai.agentspan;
import java.io.IOException;
import java.time.Instant;
import java.util.UUID;
import org.springframework.web.filter.OncePerRequestFilter;
import dev.agentspan.runtime.context.RequestContext;
import dev.agentspan.runtime.context.RequestContextHolder;
import jakarta.servlet.FilterChain;
import jakarta.servlet.ServletException;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
/**
* Populates AgentSpan's {@link RequestContextHolder} with a default principal id for the duration
* of each request, so the AgentSpan controllers and services (which call {@link
* RequestContextHolder#getRequiredUserId()}) work on an OSS server that has no per-user
* authentication.
*
* <p>If a context is already present on the thread (e.g. a host security adapter set the real
* principal), this filter leaves it untouched. Otherwise it sets the configured default id and
* clears it in a finally block to avoid leaking across pooled threads.
*/
public class AgentSpanPrincipalFilter extends OncePerRequestFilter {
private final String defaultUserId;
public AgentSpanPrincipalFilter(String defaultUserId) {
this.defaultUserId = defaultUserId;
}
@Override
protected void doFilterInternal(
HttpServletRequest request, HttpServletResponse response, FilterChain filterChain)
throws ServletException, IOException {
boolean contextSetHere = false;
if (RequestContextHolder.get().isEmpty()) {
RequestContextHolder.set(
RequestContext.builder()
.requestId(UUID.randomUUID().toString())
.userId(defaultUserId)
.createdAt(Instant.now())
.build());
contextSetHere = true;
}
try {
filterChain.doFilter(request, response);
} finally {
if (contextSetHere) {
RequestContextHolder.clear();
}
}
}
}
@@ -0,0 +1,142 @@
/*
* Copyright 2026 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.conductoross.conductor.ai.agentspan;
import java.nio.charset.StandardCharsets;
import java.security.SecureRandom;
import java.util.Arrays;
import java.util.Base64;
import java.util.List;
import org.conductoross.conductor.config.AIIntegrationEnabledCondition;
import org.conductoross.conductor.dao.SkillPackageDAO;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.web.servlet.FilterRegistrationBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Conditional;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.Ordered;
import com.fasterxml.jackson.databind.ObjectMapper;
import dev.agentspan.runtime.spi.CredentialStoreProvider;
import dev.agentspan.runtime.spi.SecretOutputMasker;
/**
* Supplies the host SPI beans the embedded {@code conductor-agentspan} library requires, when
* {@code conductor.integrations.ai.enabled=true}.
*
* <p>AgentSpan itself is activated in <b>embedded mode</b> by {@link
* AgentSpanEmbeddedEnvironmentPostProcessor} (which sets {@code agentspan.embedded=true}), so the
* library's own auto-configuration component-scans and registers the agent runtime, services, REST
* controllers and the agent-aware {@code LLM_CHAT_COMPLETE} mapper. Crucially, in embedded mode the
* library does <b>not</b> override Conductor's {@code HTTP}/{@code HUMAN}/{@code JOIN} system-task
* beans; those native tasks are extended in place via the {@code __agentspan_ctx__} task input.
* This class therefore only contributes the host-provided SPI implementations:
*
* <ul>
* <li>{@link CredentialStoreProvider} — environment-seeded, read-only ({@link
* EnvBackedCredentialStore}).
* <li>{@link SecretOutputMasker} — no-op (OSS parity).
* <li>{@code credentialMasterKey} — HMAC signing key for worker execution tokens.
* <li>Skill metadata/package SPI adapters onto Conductor's per-backend DAOs.
* <li>A request-principal filter populating AgentSpan's {@code RequestContextHolder}.
* </ul>
*/
@Configuration(proxyBeanMethods = false)
@Conditional(AIIntegrationEnabledCondition.class)
public class ConductorAgentSpanConfiguration {
private static final Logger log =
LoggerFactory.getLogger(ConductorAgentSpanConfiguration.class);
private static final int MASTER_KEY_BYTES = 32;
/** OSS parity: no per-execution disclosure tracking, so nothing to redact. */
@Bean
public SecretOutputMasker agentSpanSecretOutputMasker() {
return (executionId, userId, payload) -> payload;
}
/**
* HMAC signing key for AgentSpan worker execution tokens. Sourced from {@code
* AGENTSPAN_MASTER_KEY} (base64 or raw), else a random key generated at boot — tokens are
* short-lived and per-run, so a fresh key on restart is acceptable.
*/
@Bean("credentialMasterKey")
public byte[] agentSpanCredentialMasterKey(
@Value("${AGENTSPAN_MASTER_KEY:}") String configured) {
if (configured != null && !configured.isBlank()) {
try {
return Base64.getDecoder().decode(configured.trim());
} catch (IllegalArgumentException notBase64) {
return configured.getBytes(StandardCharsets.UTF_8);
}
}
byte[] key = new byte[MASTER_KEY_BYTES];
new SecureRandom().nextBytes(key);
log.info(
"AGENTSPAN_MASTER_KEY not set — generated an ephemeral worker-token signing key for this run");
return key;
}
/**
* Read-only, environment-seeded credential store. Secrets must be supplied as environment
* variables and injected at runtime; the API cannot write them (see {@link
* EnvBackedCredentialStore}).
*/
@Bean
public CredentialStoreProvider agentSpanCredentialStoreProvider(
@Value("${conductor.integrations.ai.secrets.env-names:}") String extraNamesCsv) {
List<String> extraNames =
Arrays.stream(extraNamesCsv.split(","))
.map(String::trim)
.filter(s -> !s.isBlank())
.toList();
return new EnvBackedCredentialStore(extraNames, System::getenv);
}
/**
* Bridges AgentSpan's skill-metadata SPI to Conductor's per-backend {@code SkillMetadataDAO}.
*/
@Bean
public dev.agentspan.runtime.spi.SkillMetadataDAO agentSpanSkillMetadataDAO(
org.conductoross.conductor.dao.SkillMetadataDAO skillMetadataDAO,
ObjectMapper objectMapper) {
return new SkillMetadataDaoAdapter(skillMetadataDAO, objectMapper);
}
/** Bridges AgentSpan's skill-package SPI to Conductor's per-backend {@code SkillPackageDAO}. */
@Bean
public dev.agentspan.runtime.spi.SkillPackageStore agentSpanSkillPackageStore(
SkillPackageDAO skillPackageDAO) {
return new SkillPackageStoreAdapter(skillPackageDAO);
}
/**
* Populates AgentSpan's request principal for {@code /api/*}. Runs late so a host security
* adapter (if any) can set the real principal first.
*/
@Bean
public FilterRegistrationBean<AgentSpanPrincipalFilter> agentSpanPrincipalFilter(
@Value("${conductor.integrations.ai.agentspan.default-user-id:agentspan-system}")
String defaultUserId) {
FilterRegistrationBean<AgentSpanPrincipalFilter> registration =
new FilterRegistrationBean<>(new AgentSpanPrincipalFilter(defaultUserId));
registration.addUrlPatterns("/api/*");
registration.setOrder(Ordered.LOWEST_PRECEDENCE);
registration.setName("agentSpanPrincipalFilter");
return registration;
}
}
@@ -0,0 +1,114 @@
/*
* Copyright 2026 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.conductoross.conductor.ai.agentspan;
import java.util.ArrayList;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import java.util.function.UnaryOperator;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import dev.agentspan.runtime.credentials.KnownProviderEnvVars;
import dev.agentspan.runtime.model.credentials.CredentialMeta;
import dev.agentspan.runtime.spi.CredentialStoreProvider;
/**
* Read-only, environment-seeded {@link CredentialStoreProvider} for OSS Conductor.
*
* <p>On construction it scans the process environment for the well-known provider variables ({@link
* KnownProviderEnvVars#NAMES}) plus any additional names configured via {@code
* conductor.integrations.ai.secrets.env-names}, and populates an in-memory map. Agents resolve
* credentials from this map at runtime.
*
* <p>Secrets are intentionally <b>not</b> writable through the API: {@link #set} throws, with a
* message directing operators to inject secrets as environment variables (e.g. via Kubernetes
* secrets) and restart. The AgentSpan {@code AgentExceptionHandler} renders that {@link
* IllegalStateException} as an HTTP 400. {@link #get}, {@link #list} and {@link #delete} operate on
* the in-memory map; deletions are not persisted and are re-seeded from the environment on the next
* restart.
*
* <p>This store is global (not per-user): the {@code userId} argument is accepted for SPI
* compatibility but ignored, because environment-injected secrets are shared by the process.
*/
public class EnvBackedCredentialStore implements CredentialStoreProvider {
private static final Logger log = LoggerFactory.getLogger(EnvBackedCredentialStore.class);
static final String READ_ONLY_MESSAGE =
"Secrets are read-only via the Conductor API. Provide them as environment variables "
+ "(for example OPENAI_API_KEY) so they are injected at runtime, then restart the server.";
private final Map<String, String> values = new ConcurrentHashMap<>();
public EnvBackedCredentialStore(List<String> extraNames, UnaryOperator<String> envLookup) {
Set<String> names = new LinkedHashSet<>(KnownProviderEnvVars.NAMES);
if (extraNames != null) {
names.addAll(extraNames);
}
int seeded = 0;
for (String name : names) {
String value = envLookup.apply(name);
if (value != null && !value.isBlank()) {
values.put(name, value);
seeded++;
}
}
log.info(
"AgentSpan EnvBackedCredentialStore seeded {} secret(s) from {} known environment variable name(s)",
seeded,
names.size());
}
@Override
public String get(String userId, String name) {
return values.get(name);
}
@Override
public void set(String userId, String name, String value) {
// IllegalArgumentException maps to HTTP 400 via both Conductor's ApplicationExceptionMapper
// and AgentSpan's AgentExceptionHandler (a read-only store is a client error, not 500).
throw new IllegalArgumentException(READ_ONLY_MESSAGE);
}
@Override
public void delete(String userId, String name) {
values.remove(name);
}
@Override
public List<CredentialMeta> list(String userId) {
List<CredentialMeta> out = new ArrayList<>();
for (Map.Entry<String, String> entry : values.entrySet()) {
out.add(
CredentialMeta.builder()
.name(entry.getKey())
.partial(toPartial(entry.getValue()))
.build());
}
return out;
}
/** First 4 + "..." + last 4 characters, matching common API-key display conventions. */
static String toPartial(String value) {
if (value == null || value.length() < 8) {
return "****...****";
}
return value.substring(0, 4) + "..." + value.substring(value.length() - 4);
}
}
@@ -0,0 +1,91 @@
/*
* Copyright 2026 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.conductoross.conductor.ai.agentspan;
import java.util.List;
import java.util.Optional;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import dev.agentspan.runtime.model.skill.SkillDetail;
/**
* Bridges AgentSpan's {@link dev.agentspan.runtime.spi.SkillMetadataDAO} SPI onto Conductor's
* backend-agnostic {@link org.conductoross.conductor.dao.SkillMetadataDAO}. The {@link SkillDetail}
* manifest is serialized to JSON for storage so the persistence layer carries no dependency on
* AgentSpan model types; it is rehydrated on read.
*/
public class SkillMetadataDaoAdapter implements dev.agentspan.runtime.spi.SkillMetadataDAO {
private final org.conductoross.conductor.dao.SkillMetadataDAO delegate;
private final ObjectMapper objectMapper;
public SkillMetadataDaoAdapter(
org.conductoross.conductor.dao.SkillMetadataDAO delegate, ObjectMapper objectMapper) {
this.delegate = delegate;
this.objectMapper = objectMapper;
}
@Override
public void save(SkillDetail detail, boolean makeLatest) {
delegate.save(
detail.getOwnerId(),
detail.getName(),
detail.getVersion(),
makeLatest,
toJson(detail),
detail.getCreatedAt(),
detail.getUpdatedAt());
}
@Override
public Optional<SkillDetail> find(String ownerId, String name, String version) {
return delegate.find(ownerId, name, version).map(this::fromJson);
}
@Override
public Optional<String> latestVersion(String ownerId, String name) {
return delegate.latestVersion(ownerId, name);
}
@Override
public List<SkillDetail> listVersions(String ownerId, String name) {
return delegate.listVersions(ownerId, name).stream().map(this::fromJson).toList();
}
@Override
public List<SkillDetail> list(String ownerId, boolean allVersions) {
return delegate.list(ownerId, allVersions).stream().map(this::fromJson).toList();
}
@Override
public void delete(String ownerId, String name, String version) {
delegate.delete(ownerId, name, version);
}
private String toJson(SkillDetail detail) {
try {
return objectMapper.writeValueAsString(detail);
} catch (JsonProcessingException e) {
throw new IllegalStateException("Failed to serialize skill metadata", e);
}
}
private SkillDetail fromJson(String json) {
try {
return objectMapper.readValue(json, SkillDetail.class);
} catch (JsonProcessingException e) {
throw new IllegalStateException("Failed to deserialize skill metadata", e);
}
}
}
@@ -0,0 +1,69 @@
/*
* Copyright 2026 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.conductoross.conductor.ai.agentspan;
import org.conductoross.conductor.dao.SkillPackageDAO;
import dev.agentspan.runtime.spi.SkillPackageStore;
import dev.agentspan.runtime.spi.StoredSkillPackage;
/**
* Bridges AgentSpan's {@link SkillPackageStore} SPI onto Conductor's backend-agnostic {@link
* SkillPackageDAO}, so skill package bytes persist through whichever Conductor backend is
* configured (Postgres, MySQL, SQLite, Redis).
*
* <p>The opaque handle is content-addressed ({@code conductor-db://<checksum>}); identical package
* bytes therefore deduplicate to one stored row.
*/
public class SkillPackageStoreAdapter implements SkillPackageStore {
static final String STORAGE_TYPE = "conductor-db";
private static final String HANDLE_PREFIX = "conductor-db://";
private final SkillPackageDAO delegate;
public SkillPackageStoreAdapter(SkillPackageDAO delegate) {
this.delegate = delegate;
}
@Override
public String storageType() {
return STORAGE_TYPE;
}
@Override
public StoredSkillPackage store(String name, String version, String checksum, byte[] bytes) {
String handle = HANDLE_PREFIX + checksum;
delegate.put(handle, bytes);
return new StoredSkillPackage(handle, STORAGE_TYPE, bytes.length);
}
@Override
public byte[] read(String handle) {
byte[] bytes = delegate.get(handle);
if (bytes == null) {
throw new IllegalArgumentException("Skill package not found: " + handle);
}
return bytes;
}
@Override
public boolean exists(String handle) {
return delegate.exists(handle);
}
@Override
public void delete(String handle) {
delegate.delete(handle);
}
}
@@ -0,0 +1,166 @@
/*
* Copyright 2026 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.conductoross.conductor.ai.agentspan.listener;
import java.util.function.Consumer;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.ObjectProvider;
import com.netflix.conductor.core.listener.TaskStatusListener;
import com.netflix.conductor.model.TaskModel;
/**
* A {@link TaskStatusListener} that fans every callback out to all other {@code TaskStatusListener}
* beans in the context.
*
* <p>The motivation mirrors {@link AgentSpanCompositeWorkflowStatusListener}: Conductor injects a
* <b>single</b> {@code TaskStatusListener} bean, so the embedded {@code conductor-agentspan}
* library's {@code @Primary AgentEventListener} would otherwise shadow an operator-configured
* {@code task_publisher}. Registering this composite as the sole {@code @Primary} listener and
* re-broadcasting to every registered listener lets agentspan's SSE listener and the configured
* publisher coexist.
*
* <p>Each callback delegates to the <b>same</b> method on every other listener so each delegate
* keeps its own gating semantics. The composite excludes itself to avoid recursion and isolates
* delegates so one listener's failure cannot suppress the others. Delegates are resolved lazily via
* {@link ObjectProvider} to avoid a self-referential construction cycle for this {@code @Primary}
* bean.
*/
public class AgentSpanCompositeTaskStatusListener implements TaskStatusListener {
private static final Logger LOGGER =
LoggerFactory.getLogger(AgentSpanCompositeTaskStatusListener.class);
private final ObjectProvider<TaskStatusListener> delegates;
public AgentSpanCompositeTaskStatusListener(ObjectProvider<TaskStatusListener> delegates) {
this.delegates = delegates;
}
/**
* Invoke {@code action} on every registered listener except this composite, isolating each
* delegate so one listener's failure does not stop the rest.
*/
private void fanOut(Consumer<TaskStatusListener> action) {
delegates.stream()
.filter(listener -> listener != this)
.forEach(
listener -> {
try {
action.accept(listener);
} catch (Exception e) {
LOGGER.warn(
"TaskStatusListener {} threw during fan-out: {}",
listener.getClass().getName(),
e.getMessage(),
e);
}
});
}
// ── *IfEnabled variants — the primary callback path used by WorkflowExecutorOps ──
@Override
public void onTaskScheduledIfEnabled(TaskModel task) {
fanOut(listener -> listener.onTaskScheduledIfEnabled(task));
}
@Override
public void onTaskInProgressIfEnabled(TaskModel task) {
fanOut(listener -> listener.onTaskInProgressIfEnabled(task));
}
@Override
public void onTaskCanceledIfEnabled(TaskModel task) {
fanOut(listener -> listener.onTaskCanceledIfEnabled(task));
}
@Override
public void onTaskFailedIfEnabled(TaskModel task) {
fanOut(listener -> listener.onTaskFailedIfEnabled(task));
}
@Override
public void onTaskFailedWithTerminalErrorIfEnabled(TaskModel task) {
fanOut(listener -> listener.onTaskFailedWithTerminalErrorIfEnabled(task));
}
@Override
public void onTaskCompletedIfEnabled(TaskModel task) {
fanOut(listener -> listener.onTaskCompletedIfEnabled(task));
}
@Override
public void onTaskCompletedWithErrorsIfEnabled(TaskModel task) {
fanOut(listener -> listener.onTaskCompletedWithErrorsIfEnabled(task));
}
@Override
public void onTaskTimedOutIfEnabled(TaskModel task) {
fanOut(listener -> listener.onTaskTimedOutIfEnabled(task));
}
@Override
public void onTaskSkippedIfEnabled(TaskModel task) {
fanOut(listener -> listener.onTaskSkippedIfEnabled(task));
}
// ── Direct variants — fan out for any caller that bypasses the *IfEnabled path ──
@Override
public void onTaskScheduled(TaskModel task) {
fanOut(listener -> listener.onTaskScheduled(task));
}
@Override
public void onTaskInProgress(TaskModel task) {
fanOut(listener -> listener.onTaskInProgress(task));
}
@Override
public void onTaskCanceled(TaskModel task) {
fanOut(listener -> listener.onTaskCanceled(task));
}
@Override
public void onTaskFailed(TaskModel task) {
fanOut(listener -> listener.onTaskFailed(task));
}
@Override
public void onTaskFailedWithTerminalError(TaskModel task) {
fanOut(listener -> listener.onTaskFailedWithTerminalError(task));
}
@Override
public void onTaskCompleted(TaskModel task) {
fanOut(listener -> listener.onTaskCompleted(task));
}
@Override
public void onTaskCompletedWithErrors(TaskModel task) {
fanOut(listener -> listener.onTaskCompletedWithErrors(task));
}
@Override
public void onTaskTimedOut(TaskModel task) {
fanOut(listener -> listener.onTaskTimedOut(task));
}
@Override
public void onTaskSkipped(TaskModel task) {
fanOut(listener -> listener.onTaskSkipped(task));
}
}
@@ -0,0 +1,173 @@
/*
* Copyright 2026 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.conductoross.conductor.ai.agentspan.listener;
import java.util.function.Consumer;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.ObjectProvider;
import com.netflix.conductor.core.listener.WorkflowStatusListener;
import com.netflix.conductor.model.WorkflowModel;
/**
* A {@link WorkflowStatusListener} that fans every callback out to all other {@code
* WorkflowStatusListener} beans in the context.
*
* <p>Conductor injects a <b>single</b> {@code WorkflowStatusListener} bean (see {@code
* WorkflowExecutorOps}/{@code ExecutionService}), so when the embedded {@code conductor-agentspan}
* library contributes its {@code @Primary AgentEventListener}, that one bean would otherwise shadow
* any operator-configured publisher ({@code queue_publisher}, {@code kafka}, {@code archive},
* {@code workflow_publisher}). To let agentspan's SSE listener and the configured publisher
* coexist, this composite is registered as the sole {@code @Primary} listener and re-broadcasts
* each event to every registered listener.
*
* <p>Each callback delegates to the <b>same</b> method on every other listener so that each
* delegate applies its own gating semantics: the {@code *IfEnabled} variants honour each listener's
* choice to gate on {@code workflowStatusListenerEnabled}, while agentspan's listener (which
* overrides the {@code *IfEnabled} methods directly) fires unconditionally as it intends. The
* composite excludes itself from the fan-out to avoid infinite recursion, and isolates delegates so
* a failure in one listener cannot suppress the others.
*
* <p>Delegates are resolved lazily through an {@link ObjectProvider} so that wiring this
* {@code @Primary} bean (which is itself a {@code WorkflowStatusListener}) does not create a
* self-referential construction cycle.
*/
public class AgentSpanCompositeWorkflowStatusListener implements WorkflowStatusListener {
private static final Logger LOGGER =
LoggerFactory.getLogger(AgentSpanCompositeWorkflowStatusListener.class);
private final ObjectProvider<WorkflowStatusListener> delegates;
public AgentSpanCompositeWorkflowStatusListener(
ObjectProvider<WorkflowStatusListener> delegates) {
this.delegates = delegates;
}
/**
* Invoke {@code action} on every registered listener except this composite, isolating each
* delegate so one listener's failure does not stop the rest.
*/
private void fanOut(Consumer<WorkflowStatusListener> action) {
delegates.stream()
.filter(listener -> listener != this)
.forEach(
listener -> {
try {
action.accept(listener);
} catch (Exception e) {
LOGGER.warn(
"WorkflowStatusListener {} threw during fan-out: {}",
listener.getClass().getName(),
e.getMessage(),
e);
}
});
}
// ── *IfEnabled variants — the primary callback path used by WorkflowExecutorOps ──
@Override
public void onWorkflowCompletedIfEnabled(WorkflowModel workflow) {
fanOut(listener -> listener.onWorkflowCompletedIfEnabled(workflow));
}
@Override
public void onWorkflowTerminatedIfEnabled(WorkflowModel workflow) {
fanOut(listener -> listener.onWorkflowTerminatedIfEnabled(workflow));
}
@Override
public void onWorkflowFinalizedIfEnabled(WorkflowModel workflow) {
fanOut(listener -> listener.onWorkflowFinalizedIfEnabled(workflow));
}
@Override
public void onWorkflowStartedIfEnabled(WorkflowModel workflow) {
fanOut(listener -> listener.onWorkflowStartedIfEnabled(workflow));
}
@Override
public void onWorkflowRestartedIfEnabled(WorkflowModel workflow) {
fanOut(listener -> listener.onWorkflowRestartedIfEnabled(workflow));
}
@Override
public void onWorkflowRerunIfEnabled(WorkflowModel workflow) {
fanOut(listener -> listener.onWorkflowRerunIfEnabled(workflow));
}
@Override
public void onWorkflowRetriedIfEnabled(WorkflowModel workflow) {
fanOut(listener -> listener.onWorkflowRetriedIfEnabled(workflow));
}
@Override
public void onWorkflowPausedIfEnabled(WorkflowModel workflow) {
fanOut(listener -> listener.onWorkflowPausedIfEnabled(workflow));
}
@Override
public void onWorkflowResumedIfEnabled(WorkflowModel workflow) {
fanOut(listener -> listener.onWorkflowResumedIfEnabled(workflow));
}
// ── Direct variants — fan out for any caller that bypasses the *IfEnabled path ──
@Override
public void onWorkflowCompleted(WorkflowModel workflow) {
fanOut(listener -> listener.onWorkflowCompleted(workflow));
}
@Override
public void onWorkflowTerminated(WorkflowModel workflow) {
fanOut(listener -> listener.onWorkflowTerminated(workflow));
}
@Override
public void onWorkflowFinalized(WorkflowModel workflow) {
fanOut(listener -> listener.onWorkflowFinalized(workflow));
}
@Override
public void onWorkflowStarted(WorkflowModel workflow) {
fanOut(listener -> listener.onWorkflowStarted(workflow));
}
@Override
public void onWorkflowRestarted(WorkflowModel workflow) {
fanOut(listener -> listener.onWorkflowRestarted(workflow));
}
@Override
public void onWorkflowRerun(WorkflowModel workflow) {
fanOut(listener -> listener.onWorkflowRerun(workflow));
}
@Override
public void onWorkflowPaused(WorkflowModel workflow) {
fanOut(listener -> listener.onWorkflowPaused(workflow));
}
@Override
public void onWorkflowResumed(WorkflowModel workflow) {
fanOut(listener -> listener.onWorkflowResumed(workflow));
}
@Override
public void onWorkflowRetried(WorkflowModel workflow) {
fanOut(listener -> listener.onWorkflowRetried(workflow));
}
}
@@ -0,0 +1,111 @@
/*
* Copyright 2026 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.conductoross.conductor.ai.agentspan.listener;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.ObjectProvider;
import org.springframework.beans.factory.config.BeanDefinition;
import org.springframework.beans.factory.config.BeanFactoryPostProcessor;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import com.netflix.conductor.core.listener.TaskStatusListener;
import com.netflix.conductor.core.listener.WorkflowStatusListener;
/**
* Makes the embedded {@code conductor-agentspan} status listener coexist with any
* operator-configured status publisher.
*
* <p>The agentspan library registers {@code AgentEventListener} as a {@code @Primary} bean for both
* {@link WorkflowStatusListener} and {@link TaskStatusListener} (it streams agent events over SSE).
* Conductor, however, injects a <b>single</b> listener bean of each type, so {@code @Primary} alone
* would make agentspan's listener silently shadow a configured {@code queue_publisher} / {@code
* kafka} / {@code archive} / {@code workflow_publisher} (or {@code task_publisher}) — and {@code
* conductor.integrations.ai.enabled} defaults to {@code true}, so this would affect default
* servers.
*
* <p>Active only in embedded mode ({@code agentspan.embedded=true}, set by {@link
* org.conductoross.conductor.ai.agentspan.AgentSpanEmbeddedEnvironmentPostProcessor}), this
* configuration:
*
* <ol>
* <li>demotes agentspan's {@code AgentEventListener} from {@code @Primary} (via a {@link
* BeanFactoryPostProcessor} that edits the bean definition before instantiation), and
* <li>registers composite {@code @Primary} listeners ({@link
* AgentSpanCompositeWorkflowStatusListener}, {@link AgentSpanCompositeTaskStatusListener})
* that fan every callback out to <i>all</i> registered listeners — agentspan's listener and
* the configured publisher alike.
* </ol>
*
* <p>The composite becomes the sole {@code @Primary} candidate, so the single-bean injection points
* in {@code WorkflowExecutorOps}/{@code ExecutionService} resolve to it, and every listener
* receives every event.
*/
@Configuration(proxyBeanMethods = false)
@ConditionalOnProperty(name = "agentspan.embedded", havingValue = "true")
public class AgentSpanListenerCoexistenceConfiguration {
private static final Logger LOGGER =
LoggerFactory.getLogger(AgentSpanListenerCoexistenceConfiguration.class);
/**
* Fully-qualified name of the agentspan listener. Matched by name rather than imported type to
* avoid coupling to a library-internal class and to keep the post-processor free of side
* effects (no premature class loading).
*/
private static final String AGENT_EVENT_LISTENER_CLASS =
"dev.agentspan.runtime.service.AgentEventListener";
/**
* Demotes agentspan's {@code @Primary AgentEventListener} to an ordinary candidate so the
* composite beans below can become the sole {@code @Primary} listeners. Declared {@code static}
* so the post-processor is instantiated early (before regular bean instantiation) without
* forcing the enclosing configuration to initialise prematurely. Runs after {@code
* ConfigurationClassPostProcessor} (which registers agentspan's component-scanned definition),
* so the target definition is present; it is a no-op if agentspan is absent or already
* non-primary.
*/
@Bean
public static BeanFactoryPostProcessor agentSpanListenerPrimaryDemoter() {
return beanFactory -> {
for (String name : beanFactory.getBeanDefinitionNames()) {
BeanDefinition definition = beanFactory.getBeanDefinition(name);
if (AGENT_EVENT_LISTENER_CLASS.equals(definition.getBeanClassName())
&& definition.isPrimary()) {
definition.setPrimary(false);
LOGGER.info(
"Demoted agentspan listener bean '{}' from @Primary so workflow/task "
+ "status listeners can coexist via the composite",
name);
}
}
};
}
@Bean
@Primary
public WorkflowStatusListener agentSpanCompositeWorkflowStatusListener(
ObjectProvider<WorkflowStatusListener> workflowStatusListeners) {
return new AgentSpanCompositeWorkflowStatusListener(workflowStatusListeners);
}
@Bean
@Primary
public TaskStatusListener agentSpanCompositeTaskStatusListener(
ObjectProvider<TaskStatusListener> taskStatusListeners) {
return new AgentSpanCompositeTaskStatusListener(taskStatusListeners);
}
}
@@ -0,0 +1,2 @@
org.springframework.boot.env.EnvironmentPostProcessor=\
org.conductoross.conductor.ai.agentspan.AgentSpanEmbeddedEnvironmentPostProcessor
@@ -0,0 +1,92 @@
/*
* Copyright 2026 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.conductoross.conductor.ai.agentspan;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.Test;
import dev.agentspan.runtime.model.credentials.CredentialMeta;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
class EnvBackedCredentialStoreTest {
private static final String USER = "agentspan-system";
private EnvBackedCredentialStore store(Map<String, String> env, List<String> extra) {
return new EnvBackedCredentialStore(extra, env::get);
}
@Test
void seedsKnownProviderVarsFromEnvironment() {
EnvBackedCredentialStore store =
store(
Map.of("OPENAI_API_KEY", "sk-abcdefgh", "UNRELATED_VAR", "ignored"),
List.of());
assertEquals("sk-abcdefgh", store.get(USER, "OPENAI_API_KEY"));
// Not a known provider var and not in the extra list -> not seeded.
assertNull(store.get(USER, "UNRELATED_VAR"));
}
@Test
void seedsConfiguredExtraNames() {
EnvBackedCredentialStore store =
store(Map.of("MY_CUSTOM_SECRET", "value-1234"), List.of("MY_CUSTOM_SECRET"));
assertEquals("value-1234", store.get(USER, "MY_CUSTOM_SECRET"));
}
@Test
void listReturnsMaskedMetadataNeverPlaintext() {
EnvBackedCredentialStore store =
store(Map.of("OPENAI_API_KEY", "sk-test-1234567890abcdef"), List.of());
List<CredentialMeta> list = store.list(USER);
assertEquals(1, list.size());
CredentialMeta meta = list.get(0);
assertEquals("OPENAI_API_KEY", meta.getName());
assertEquals("sk-t...cdef", meta.getPartial());
assertTrue(!meta.getPartial().contains("567890"), "partial must not leak the full value");
}
@Test
void setIsRejectedWithGuidance() {
EnvBackedCredentialStore store = store(Map.of(), List.of());
IllegalArgumentException ex =
assertThrows(
IllegalArgumentException.class, () -> store.set(USER, "ANYTHING", "v"));
assertTrue(ex.getMessage().contains("environment variables"));
}
@Test
void deleteRemovesFromInMemoryView() {
EnvBackedCredentialStore store = store(Map.of("OPENAI_API_KEY", "sk-abcdefgh"), List.of());
store.delete(USER, "OPENAI_API_KEY");
assertNull(store.get(USER, "OPENAI_API_KEY"));
assertTrue(store.list(USER).isEmpty());
}
@Test
void shortValuesAreFullyMasked() {
assertEquals("****...****", EnvBackedCredentialStore.toPartial("short"));
assertEquals("****...****", EnvBackedCredentialStore.toPartial(null));
}
}
@@ -0,0 +1,169 @@
/*
* Copyright 2026 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.conductoross.conductor.ai.agentspan.listener;
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Stream;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.ObjectProvider;
import com.netflix.conductor.common.metadata.workflow.WorkflowDef;
import com.netflix.conductor.core.listener.WorkflowStatusListener;
import com.netflix.conductor.model.WorkflowModel;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
/**
* Verifies the fan-out semantics that let agentspan's listener coexist with a configured publisher:
* the composite broadcasts to every other listener, excludes itself (no recursion), isolates
* failures, and preserves each delegate's own {@code *IfEnabled} gating.
*/
class AgentSpanCompositeWorkflowStatusListenerTest {
/** Real listener that records which callbacks it received. */
private static final class RecordingListener implements WorkflowStatusListener {
final List<String> events = new ArrayList<>();
@Override
public void onWorkflowCompleted(WorkflowModel workflow) {
events.add("completed:" + workflow.getWorkflowId());
}
@Override
public void onWorkflowTerminated(WorkflowModel workflow) {
events.add("terminated:" + workflow.getWorkflowId());
}
}
/** Listener that always throws, to prove failures are isolated from siblings. */
private static final class ThrowingListener implements WorkflowStatusListener {
boolean invoked = false;
@Override
public void onWorkflowCompleted(WorkflowModel workflow) {
invoked = true;
throw new RuntimeException("boom");
}
@Override
public void onWorkflowTerminated(WorkflowModel workflow) {
throw new RuntimeException("boom");
}
}
/**
* Minimal {@link ObjectProvider} backed by a mutable list. The composite only ever calls {@link
* ObjectProvider#stream()}, so the remaining methods are intentionally unsupported.
*/
private static ObjectProvider<WorkflowStatusListener> providerOf(
List<WorkflowStatusListener> registry) {
return new ObjectProvider<>() {
@Override
public Stream<WorkflowStatusListener> stream() {
return registry.stream();
}
@Override
public WorkflowStatusListener getObject(Object... args) {
throw new UnsupportedOperationException();
}
@Override
public WorkflowStatusListener getObject() {
throw new UnsupportedOperationException();
}
@Override
public WorkflowStatusListener getIfAvailable() {
throw new UnsupportedOperationException();
}
@Override
public WorkflowStatusListener getIfUnique() {
throw new UnsupportedOperationException();
}
};
}
private static WorkflowModel workflow(String id, boolean listenerEnabled) {
WorkflowModel workflow = new WorkflowModel();
workflow.setWorkflowId(id);
WorkflowDef def = new WorkflowDef();
def.setName("test_wf");
def.setWorkflowStatusListenerEnabled(listenerEnabled);
workflow.setWorkflowDefinition(def);
return workflow;
}
@Test
void fansOutToAllDelegatesAndExcludesItself() {
List<WorkflowStatusListener> registry = new ArrayList<>();
AgentSpanCompositeWorkflowStatusListener composite =
new AgentSpanCompositeWorkflowStatusListener(providerOf(registry));
RecordingListener first = new RecordingListener();
RecordingListener second = new RecordingListener();
// Include the composite itself to prove it is filtered out (otherwise this would recurse
// infinitely and overflow the stack).
registry.add(composite);
registry.add(first);
registry.add(second);
composite.onWorkflowCompleted(workflow("wf-1", true));
assertEquals(List.of("completed:wf-1"), first.events);
assertEquals(List.of("completed:wf-1"), second.events);
}
@Test
void isolatesFailingDelegateFromTheRest() {
List<WorkflowStatusListener> registry = new ArrayList<>();
AgentSpanCompositeWorkflowStatusListener composite =
new AgentSpanCompositeWorkflowStatusListener(providerOf(registry));
RecordingListener before = new RecordingListener();
ThrowingListener throwing = new ThrowingListener();
RecordingListener after = new RecordingListener();
registry.add(composite);
registry.add(before);
registry.add(throwing);
registry.add(after);
// Must not propagate the delegate's exception.
composite.onWorkflowCompleted(workflow("wf-2", true));
assertTrue(throwing.invoked);
assertEquals(List.of("completed:wf-2"), before.events);
assertEquals(List.of("completed:wf-2"), after.events);
}
@Test
void ifEnabledPathHonoursEachDelegateGating() {
List<WorkflowStatusListener> registry = new ArrayList<>();
AgentSpanCompositeWorkflowStatusListener composite =
new AgentSpanCompositeWorkflowStatusListener(providerOf(registry));
RecordingListener listener = new RecordingListener();
registry.add(composite);
registry.add(listener);
// Delegate uses the interface default *IfEnabled gating: enabled -> delivered.
composite.onWorkflowTerminatedIfEnabled(workflow("wf-on", true));
// Disabled -> the delegate's own gating suppresses delivery.
composite.onWorkflowTerminatedIfEnabled(workflow("wf-off", false));
assertEquals(List.of("terminated:wf-on"), listener.events);
assertFalse(listener.events.contains("terminated:wf-off"));
}
}
+627
View File
@@ -0,0 +1,627 @@
# Contributing to Conductor AI Module
Thank you for your interest in contributing to the Conductor AI module! This guide will help you add new LLM providers, vector database integrations, workers, and other enhancements.
## Table of Contents
- [Architecture Overview](#architecture-overview)
- [Adding a New LLM Provider](#adding-a-new-llm-provider)
- [Adding a Vector Database Integration](#adding-a-vector-database-integration)
- [Adding New Workers/Tasks](#adding-new-workerstasks)
- [Adding MCP Tools](#adding-mcp-tools)
- [Testing Guidelines](#testing-guidelines)
- [Code Style and Best Practices](#code-style-and-best-practices)
---
## Architecture Overview
The AI module is organized into several key packages:
```
org.conductoross.conductor.ai/
├── providers/ # LLM provider implementations (OpenAI, Anthropic, etc.)
├── vectordb/ # Vector database integrations (Pinecone, MongoDB, etc.)
├── video/ # Video generation abstractions (VideoModel, AsyncVideoModel, etc.)
├── tasks/ # Worker task definitions
│ ├── mapper/ # Input/output parameter mappers
│ └── worker/ # Worker implementations
├── mcp/ # Model Context Protocol implementation
├── models/ # Request/response models
└── document/ # Document readers and parsers
```
Key interfaces:
- **`AIModel`**: Base interface for LLM providers
- **`VideoModel`**: Functional interface for synchronous video generation (mirrors Spring AI's `ImageModel`)
- **`AsyncVideoModel`**: Extends `VideoModel` with async polling via `checkStatus(String jobId)`
- **`VectorDBProvider`**: Base interface for vector databases
- **`@WorkerTask`**: Annotation for defining worker tasks
---
## Adding a New LLM Provider
### Step 1: Create Provider Package
Create a new package under `providers/`:
```
org.conductoross.conductor.ai.providers.yourprovider/
├── YourProvider.java # Main provider implementation
└── YourProviderConfiguration.java # Spring configuration
```
### Step 2: Implement AIModel Interface
Create your provider class implementing `AIModel`:
```java
package org.conductoross.conductor.ai.providers.yourprovider;
import org.conductoross.conductor.ai.AIModel;
import org.springframework.ai.chat.model.ChatModel;
import org.springframework.ai.embedding.EmbeddingModel;
public class YourProvider implements AIModel {
private final ChatModel chatModel;
private final EmbeddingModel embeddingModel;
public YourProvider(ChatModel chatModel, EmbeddingModel embeddingModel) {
this.chatModel = chatModel;
this.embeddingModel = embeddingModel;
}
@Override
public String getModelProvider() {
return "your_provider_name"; // Used in workflow definitions
}
@Override
public ChatModel getChatModel() {
return chatModel;
}
@Override
public EmbeddingModel getEmbeddingModel() {
return embeddingModel;
}
}
```
### Step 3: Create Configuration Class
Use `@ConditionalOnProperty` to ensure the provider only loads when configured:
```java
package org.conductoross.conductor.ai.providers.yourprovider;
import org.conductoross.conductor.ai.ModelConfiguration;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
@EnableConfigurationProperties(YourProviderProperties.class)
@ConditionalOnProperty(prefix = "conductor.ai.your-provider", name = "api-key")
public class YourProviderConfiguration {
@Bean
public ModelConfiguration<YourProvider> yourProviderConfiguration(
YourProviderProperties properties) {
return () -> {
// Initialize chat and embedding models
ChatModel chatModel = // ... create from properties
EmbeddingModel embeddingModel = // ... create from properties
return new YourProvider(chatModel, embeddingModel);
};
}
}
```
### Step 4: Create Properties Class
```java
package org.conductoross.conductor.ai.providers.yourprovider;
import org.springframework.boot.context.properties.ConfigurationProperties;
import lombok.Data;
@Data
@ConfigurationProperties(prefix = "conductor.ai.your-provider")
public class YourProviderProperties {
private String apiKey;
private String baseUrl = "https://api.yourprovider.com";
private String model = "default-model";
// Add other configuration properties
}
```
### Step 5: Add Tests
Create `YourProviderConfigurationTest.java`:
```java
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class YourProviderConfigurationTest {
@Test
void testProviderLoadsWhenConfigured() {
ApplicationContextRunner contextRunner =
new ApplicationContextRunner()
.withConfiguration(
AutoConfigurations.of(YourProviderConfiguration.class))
.withPropertyValues(
"conductor.ai.your-provider.api-key=test-key");
contextRunner.run(
context -> {
assertThat(context).hasSingleBean(ModelConfiguration.class);
});
}
@Test
void testProviderDoesNotLoadWithoutApiKey() {
ApplicationContextRunner contextRunner =
new ApplicationContextRunner()
.withConfiguration(
AutoConfigurations.of(YourProviderConfiguration.class));
contextRunner.run(
context -> {
assertThat(context).doesNotHaveBean(ModelConfiguration.class);
});
}
}
```
### Step 6: Add Video Generation Support (Optional)
If your provider supports video generation, implement video model support using the `video/` package abstractions. Video generation is async by nature (submit a job, poll for results), so most providers will implement `AsyncVideoModel`.
#### 6a. Create a Video Model Class
```java
package org.conductoross.conductor.ai.providers.yourprovider;
import org.conductoross.conductor.ai.video.*;
public class YourVideoModel implements AsyncVideoModel {
private final String apiKey;
public YourVideoModel(String apiKey) {
this.apiKey = apiKey;
}
@Override
public VideoResponse call(VideoPrompt prompt) {
// Submit video generation job to provider API
// Return a VideoResponse with jobId in metadata
VideoResponseMetadata metadata = new VideoResponseMetadata();
metadata.put("jobId", submittedJobId);
metadata.put("status", "PENDING");
return new VideoResponse(List.of(), metadata);
}
@Override
public VideoResponse checkStatus(String jobId) {
// Poll provider API for job status
// When complete, download video bytes and return Video objects
// Set mimeType on each Video (e.g., "video/mp4", "image/webp" for thumbnails)
Video video = new Video(videoUrl, null, "video/mp4");
VideoGeneration generation = new VideoGeneration(video);
VideoResponseMetadata metadata = new VideoResponseMetadata();
metadata.put("jobId", jobId);
metadata.put("status", "COMPLETED");
return new VideoResponse(List.of(generation), metadata);
}
}
```
#### 6b. Wire Video Model into Your Provider
Override the video-related methods in your `AIModel` implementation:
```java
@Override
public VideoModel getVideoModel() {
if (videoModel == null) {
videoModel = new YourVideoModel(apiKey);
}
return videoModel;
}
@Override
public LLMResponse generateVideo(VideoGenRequest request) {
VideoOptions options = getVideoOptions(request);
VideoPrompt prompt = new VideoPrompt(
List.of(new VideoMessage(request.getPrompt())), options);
VideoResponse response = getVideoModel().call(prompt);
// Convert to LLMResponse with jobId
}
@Override
public LLMResponse checkVideoStatus(VideoGenRequest request) {
AsyncVideoModel asyncModel = (AsyncVideoModel) getVideoModel();
VideoResponse response = asyncModel.checkStatus(request.getJobId());
// Convert to LLMResponse with media list
}
```
The `video/` package mirrors Spring AI's `Image*` abstraction pattern:
- `VideoPrompt` -> `ImagePrompt` (request wrapper)
- `VideoResponse` -> `ImageResponse` (response wrapper)
- `VideoGeneration` -> `ImageGeneration` (individual result)
- `Video` -> `Image` (the actual media, with url, b64Json, and mimeType fields)
- `VideoOptions` -> `ImageOptions` (generation parameters)
### Step 7: Update Documentation
Add your provider to `README.md` under the supported providers section with configuration examples.
---
## Adding a Vector Database Integration
### Step 1: Create Config Class
Create a new configuration class in the database package (e.g., `org.conductoross.conductor.ai.vectordb.yourdb`):
```java
@Data
@NoArgsConstructor
@AllArgsConstructor
public class YourDBConfig implements VectorDBConfig<YourVectorDB> {
private String connectionString;
// other properties
@Override
public YourVectorDB get() {
throw new UnsupportedOperationException("Use get(String name) instead");
}
public YourVectorDB get(String name) {
return new YourVectorDB(name, this);
}
}
```
### Step 2: Implement VectorDB Class
Extend the `VectorDB` abstract class:
```java
public class YourVectorDB extends VectorDB {
public static final String TYPE = "yourdb";
private final YourDBConfig config;
public YourVectorDB(String name, YourDBConfig config) {
super(name, TYPE);
this.config = config;
}
@Override
public int updateEmbeddings(String indexName, String namespace, String doc, String parentDocId, String id, List<Float> embeddings, Map<String, Object> metadata) {
// Implement logic to store embeddings
}
@Override
public List<IndexedDoc> search(String indexName, String namespace, List<Float> embeddings, int maxResults) {
// Implement logic to search embeddings
}
}
```
### Step 3: Register in VectorDBInstanceConfig
Add your database type to the `createVectorDB` method and the `VectorDBInstance` inner class in `org.conductoross.conductor.ai.vectordb.VectorDBInstanceConfig`.
### Step 4: Add Integration Tests
Use Testcontainers for integration testing:
```java
@Testcontainers
class YourVectorDBTest {
@Container
static GenericContainer<?> yourdb =
new GenericContainer<>("yourdb:latest")
.withExposedPorts(1234);
@Test
void testStoreAndSearch() {
// Test vector storage and similarity search
}
}
```
---
## Adding New Workers/Tasks
### Step 1: Create Request Model
```java
package org.conductoross.conductor.ai.model;
import lombok.Data;
import lombok.EqualsAndHashCode;
@Data
@EqualsAndHashCode(callSuper = false)
public class YourTaskRequest extends LLMWorkerInput {
private String parameter1;
private String parameter2;
// Add task-specific parameters
}
```
### Step 2: Create Worker Class
```java
package org.conductoross.conductor.ai.tasks.worker;
import com.netflix.conductor.sdk.workflow.annotations.WorkerTask;
import org.conductoross.conductor.ai.model.YourTaskRequest;
@Component
public class YourWorker {
private final YourService yourService;
public YourWorker(YourService yourService) {
this.yourService = yourService;
}
@WorkerTask("YOUR_TASK_NAME")
public @OutputParam("result") YourTaskResult executeTask(YourTaskRequest request) {
// Implement task logic
return yourService.processRequest(request);
}
}
```
### Step 3: Add Task Tests
```java
class YourWorkerTest {
@Test
void testTaskExecution() {
YourWorker worker = new YourWorker(mockService);
YourTaskRequest request = new YourTaskRequest();
request.setParameter1("test");
YourTaskResult result = worker.executeTask(request);
assertNotNull(result);
// Add assertions
}
}
```
---
## Adding MCP Tools
Model Context Protocol (MCP) allows external tools to be called from workflows.
### Adding MCP Server Support
The `MCPService` already supports:
- HTTP/SSE transports
- stdio (local process) transports
- Direct JSON-RPC fallback
To add a new MCP server:
1. **Deploy your MCP server** (HTTP or local script)
2. **Use existing `CALL_MCP_TOOL` task** in workflows:
```json
{
"name": "call_your_tool",
"taskReferenceName": "your_tool",
"type": "CALL_MCP_TOOL",
"inputParameters": {
"mcpServer": "http://localhost:3000",
"methodName": "your_tool_name",
"param1": "value1",
"param2": "value2"
}
}
```
### Extending MCP Capabilities
To add new MCP-related features, modify:
- `MCPService.java` - Core MCP communication logic
- `MCPWorkers.java` - Worker task definitions
- `models/MCP*.java` - Request/response models
---
## Testing Guidelines
### Unit Tests
- Place in `src/test/java` mirroring the source structure
- Use MockBean for Spring dependencies
- Test individual methods and edge cases
- Aim for 80%+ code coverage
### Integration Tests
- Use `@SpringBootTest` for full context testing
- Use Testcontainers for external dependencies (databases, servers)
- Test real interactions between components
### Test Naming Convention
```java
// Unit test method format
void test<MethodName>_<Scenario>_<ExpectedResult>()
// Examples:
void testGetModel_WithValidProvider_ReturnsModel()
void testGetModel_WithInvalidProvider_ThrowsException()
```
### Running Tests
```bash
# Run all tests
./gradlew :conductor-ai:test
# Run specific test class
./gradlew :conductor-ai:test --tests YourProviderTest
# Run with coverage
./gradlew :conductor-ai:test jacocoTestReport
```
---
## Code Style and Best Practices
### Lombok Usage
Use Lombok annotations consistently:
- `@Data` for simple POJOs
- `@Builder` for complex object construction
- `@Slf4j` for logging
- `@AllArgsConstructor` / `@NoArgsConstructor` for constructors
### Logging
- Use SLF4J via `@Slf4j`
- Log levels:
- `log.debug()` - Detailed diagnostic information
- `log.info()` - Important business events
- `log.warn()` - Recoverable issues
- `log.error()` - Errors requiring attention
### Error Handling
- Throw descriptive exceptions
- Include context in error messages
- Use try-catch for recoverable errors
- Let unchecked exceptions propagate for programming errors
### Configuration Properties
- Use `@ConfigurationProperties` for type-safe configuration
- Provide sensible defaults
- Document all properties in javadoc
- Use `@ConditionalOnProperty` to make features optional
### Spring Beans
- Prefer constructor injection over field injection
- Use `@Component` for auto-detected beans
- Use `@Configuration` for explicit bean definitions
- Apply `@ConditionalOnProperty` for optional features
### Documentation
- Add Javadoc to all public classes and methods
- Include usage examples in class-level Javadoc
- Update `README.md` with new features
- Provide workflow examples for new tasks
---
## Development Workflow
### 1. Create a Feature Branch
```bash
git checkout -b feature/add-your-provider
```
### 2. Implement Your Changes
Follow the patterns above for your contribution type.
### 3. Write Tests
Ensure your code has comprehensive test coverage.
### 4. Run Tests and Checks
```bash
./gradlew :conductor-ai:test
./gradlew :conductor-ai:compileJava
```
### 5. Update Documentation
- Update `README.md` with examples
- Add Javadoc to new classes
- Update this CONTRIBUTING.md if adding new patterns
### 6. Submit Pull Request
- Provide clear description of changes
- Reference any related issues
- Include test results
- Update changelog if applicable
---
## Common Patterns
### Conditional Bean Creation
Always use `@ConditionalOnProperty` for optional integrations:
```java
@ConditionalOnProperty(
prefix = "conductor.ai.your-feature",
name = "enabled",
havingValue = "true"
)
```
### Parameter Mapping
For workers with dynamic parameters, use `@JsonAnySetter`:
```java
@JsonAnySetter
public void setAdditionalProperty(String key, Object value) {
additionalProperties.put(key, value);
}
```
### Resource Cleanup
Implement `DisposableBean` for cleanup:
```java
@Override
public void destroy() throws Exception {
// Clean up resources
}
```
---
## Getting Help
- Check existing implementations in `providers/` for examples
- Review `README.md` for usage patterns
- Look at test files for testing patterns
- Open a GitHub issue for questions
## License
By contributing, you agree that your contributions will be licensed under the Apache License 2.0.
+257
View File
@@ -0,0 +1,257 @@
# JDBC Configuration
This document describes the configuration format for JDBC database connections in Conductor.
## Overview
Conductor supports configuring **multiple named JDBC instances** for use by the `JDBC` worker task. This allows you to:
- Connect to multiple databases (MySQL, PostgreSQL, Oracle, etc.)
- Separate environments (prod, dev, staging)
- Use different connection pool settings per use case (read-heavy vs write-heavy)
## Configuration Format
JDBC instances are configured using a list-based approach under `conductor.jdbc.instances`:
```yaml
conductor:
jdbc:
instances:
- name: "instance-name" # Unique identifier for this instance
connection: # Connection configuration
datasourceURL: "jdbc:..." # JDBC connection URL
jdbcDriver: "..." # JDBC driver class (optional, auto-detected from URL)
user: "..." # Database username
password: "..." # Database password
# ... pool settings
```
## Configuration Examples
### Single MySQL Instance
```yaml
conductor:
jdbc:
instances:
- name: "mysql-prod"
connection:
datasourceURL: "jdbc:mysql://prod-db:3306/myapp"
jdbcDriver: "com.mysql.cj.jdbc.Driver"
user: "conductor"
password: "secret"
maximumPoolSize: 20
minimumIdle: 5
```
### Multiple Instances
```yaml
conductor:
jdbc:
instances:
- name: "mysql-prod"
connection:
datasourceURL: "jdbc:mysql://prod-db:3306/myapp"
jdbcDriver: "com.mysql.cj.jdbc.Driver"
user: "conductor"
password: "prod-secret"
maximumPoolSize: 20
- name: "postgres-analytics"
connection:
datasourceURL: "jdbc:postgresql://analytics-db:5432/warehouse"
user: "analyst"
password: "analytics-secret"
maximumPoolSize: 10
- name: "mysql-staging"
connection:
datasourceURL: "jdbc:mysql://staging-db:3306/myapp"
jdbcDriver: "com.mysql.cj.jdbc.Driver"
user: "conductor"
password: "staging-secret"
maximumPoolSize: 5
minimumIdle: 1
```
## Usage in Workflows
When using the JDBC task in your workflows, reference the instance by its configured name using `connectionId`:
```json
{
"name": "query_users",
"taskReferenceName": "query_users_ref",
"type": "JDBC",
"inputParameters": {
"connectionId": "mysql-prod",
"type": "SELECT",
"statement": "SELECT id, name, email FROM users WHERE status = ?",
"parameters": ["active"]
}
}
```
### SELECT Example
```json
{
"name": "find_orders",
"taskReferenceName": "find_orders_ref",
"type": "JDBC",
"inputParameters": {
"connectionId": "postgres-analytics",
"type": "SELECT",
"statement": "SELECT order_id, total FROM orders WHERE customer_id = ?",
"parameters": ["${workflow.input.customerId}"]
}
}
```
Output:
```json
{
"result": [
{"order_id": 101, "total": 49.99},
{"order_id": 205, "total": 129.50}
]
}
```
### UPDATE Example
```json
{
"name": "update_status",
"taskReferenceName": "update_status_ref",
"type": "JDBC",
"inputParameters": {
"connectionId": "mysql-prod",
"type": "UPDATE",
"statement": "UPDATE orders SET status = ? WHERE order_id = ?",
"parameters": ["shipped", "${workflow.input.orderId}"],
"expectedUpdateCount": 1
}
}
```
Output:
```json
{
"update_count": 1
}
```
If the actual update count does not match `expectedUpdateCount`, the transaction is rolled back and the task fails.
## Connection Configuration Options
| Property | Type | Default | Description |
|----------|------|---------|-------------|
| `datasourceURL` | String | Required | JDBC connection URL |
| `jdbcDriver` | String | Auto-detected | JDBC driver class name |
| `user` | String | Optional | Database username |
| `password` | String | Optional | Database password |
| `maximumPoolSize` | Integer | 32 | Maximum connections in the pool |
| `minimumIdle` | Integer | 2 | Minimum idle connections |
| `idleTimeoutMs` | Long | 30000 | Idle connection timeout (ms) |
| `connectionTimeout` | Long | 30000 | Connection acquisition timeout (ms) |
| `leakDetectionThreshold` | Long | 60000 | Leak detection threshold (ms) |
| `maxLifetime` | Long | 1800000 | Maximum connection lifetime (ms) |
## Migration from Old Configuration
### Old Format
```properties
conductor.worker.jdbc.connectionIds=mysql,postgres
conductor.worker.jdbc.mysql.connectionURL=jdbc:mysql://localhost:3306/db
conductor.worker.jdbc.mysql.driverClassName=com.mysql.cj.jdbc.Driver
conductor.worker.jdbc.mysql.username=root
conductor.worker.jdbc.mysql.password=secret
conductor.worker.jdbc.mysql.maximum-pool-size=10
conductor.worker.jdbc.postgres.connectionURL=jdbc:postgresql://localhost:5432/db
conductor.worker.jdbc.postgres.driverClassName=org.postgresql.Driver
conductor.worker.jdbc.postgres.username=pguser
conductor.worker.jdbc.postgres.password=pgpass
```
### New Format
```yaml
conductor:
jdbc:
instances:
- name: "mysql"
connection:
datasourceURL: "jdbc:mysql://localhost:3306/db"
jdbcDriver: "com.mysql.cj.jdbc.Driver"
user: "root"
password: "secret"
maximumPoolSize: 10
- name: "postgres"
connection:
datasourceURL: "jdbc:postgresql://localhost:5432/db"
jdbcDriver: "org.postgresql.Driver"
user: "pguser"
password: "pgpass"
```
**Note:** The old `conductor.worker.jdbc.*` format is still supported for backwards compatibility. If no `conductor.jdbc.instances` are configured, the system automatically falls back to reading the legacy format. The old and new formats are mutually exclusive -- if new-format instances are found, the legacy format is ignored.
### Property Name Mapping
| Old Property | New Property |
|---|---|
| `connectionURL` | `datasourceURL` |
| `driverClassName` | `jdbcDriver` |
| `username` | `user` |
| `password` | `password` |
| `maximum-pool-size` | `maximumPoolSize` |
| `idle-timeout-ms` | `idleTimeoutMs` |
| `minimum-idle` | `minimumIdle` |
## Best Practices
1. **Use descriptive names**: Choose instance names that clearly indicate their purpose (e.g., `mysql-prod`, `postgres-analytics`, `oracle-reporting`)
2. **Separate read/write pools**: For high-throughput systems, configure separate instances for read and write operations with appropriate pool sizes
3. **Right-size connection pools**: Set `maximumPoolSize` based on your database capacity and workload. A common formula is `connections = (core_count * 2) + effective_spindle_count`
4. **Enable leak detection**: The default `leakDetectionThreshold` of 60 seconds logs warnings for connections held longer than expected
5. **Use parameterized queries**: Always use `?` placeholders with the `parameters` list instead of string concatenation to prevent SQL injection
6. **Set expectedUpdateCount**: For critical UPDATE/INSERT/DELETE operations, set `expectedUpdateCount` to automatically rollback if the affected row count doesn't match
## Troubleshooting
### Instance Not Found
If you see "JDBC instance not found: xyz", check:
1. The `connectionId` in your workflow matches the configured `name` exactly
2. The instance is properly configured in your application.yml/properties
3. The application has been restarted after configuration changes
### Connection Timeout
If connections are timing out:
1. Verify network connectivity to the database
2. Check `connectionTimeout` value (default 30 seconds)
3. Ensure the connection pool is not exhausted (increase `maximumPoolSize` if needed)
4. Check database max connections limit
### Connection Leaks
If you see leak detection warnings:
1. Ensure all connections are properly closed (the JDBC worker handles this automatically)
2. If using custom integrations, wrap connection usage in try-with-resources
3. Review `leakDetectionThreshold` setting
+1699
View File
File diff suppressed because it is too large Load Diff
+331
View File
@@ -0,0 +1,331 @@
# Vector Database Configuration
This document describes the configuration format for vector databases in Conductor.
## Overview
Conductor supports multiple vector database providers with the ability to configure **multiple named instances** of each type. This allows you to:
- Use multiple databases of the same type (e.g., multiple PostgreSQL instances)
- Connect to different environments (prod, dev, staging)
- Separate concerns by use case (embeddings, search, recommendations)
## Supported Vector Databases
- **PostgreSQL** (with pgvector extension)
- **MongoDB** (with Atlas Vector Search)
- **Pinecone**
- **SQLite** (with the [sqlite-vec](https://github.com/asg017/sqlite-vec) extension) — embedded, zero-infrastructure backend for local development, demos and small deployments
## Configuration Format
Vector databases are configured using a list-based approach under `conductor.vectordb.instances`:
```yaml
conductor:
vectordb:
instances:
- name: "instance-name" # Unique identifier for this instance
type: "database-type" # Type: postgres, mongodb, pinecone, or sqlite
<type-specific-config>: # Configuration block for the database type
# ... type-specific properties
```
## Configuration Examples
### Single PostgreSQL Instance
```yaml
conductor:
vectordb:
instances:
- name: "postgres-main"
type: "postgres"
postgres:
datasourceURL: "jdbc:postgresql://localhost:5432/vectors"
user: "conductor"
password: "secret"
dimensions: 1536
connectionPoolSize: 10
indexingMethod: "hnsw" # Options: hnsw, ivfflat
distanceMetric: "cosine" # Options: l2, cosine, inner_product
tablePrefix: "conductor"
```
### Multiple PostgreSQL Instances
```yaml
conductor:
vectordb:
instances:
- name: "postgres-prod"
type: "postgres"
postgres:
datasourceURL: "jdbc:postgresql://prod-db:5432/vectors"
user: "conductor"
password: "prod-secret"
dimensions: 1536
- name: "postgres-dev"
type: "postgres"
postgres:
datasourceURL: "jdbc:postgresql://dev-db:5432/vectors"
user: "conductor"
password: "dev-secret"
dimensions: 768
```
### MongoDB Atlas Vector Search
```yaml
conductor:
vectordb:
instances:
- name: "mongodb-embeddings"
type: "mongodb"
mongodb:
connectionString: "mongodb+srv://user:pass@cluster.mongodb.net/"
database: "conductor"
collection: "embeddings"
numCandidates: 100
```
### Pinecone
```yaml
conductor:
vectordb:
instances:
- name: "pinecone-search"
type: "pinecone"
pinecone:
apiKey: "your-pinecone-api-key"
```
### SQLite (sqlite-vec)
```yaml
conductor:
vectordb:
instances:
- name: "sqlite-local"
type: "sqlite"
sqlite:
dbPath: "/var/lib/conductor/vectordb.db" # use ":memory:" for an ephemeral DB
dimensions: 1536
distanceMetric: "cosine" # Options: l2, cosine, l1
connectionPoolSize: 5
tablePrefix: "conductor"
# extensionPath is optional — the native vec0 binary is bundled in the jar.
# Set it only to override with a custom build:
# extensionPath: "/opt/sqlite-vec/vec0"
```
> **The native `vec0` extension is bundled.** Conductor ships the official, checksum-pinned sqlite-vec
> loadable binaries for linux (x86_64/aarch64), macOS (x86_64/aarch64) and windows (x86_64) inside the
> AI jar, and extracts the right one at runtime — so `extensionPath` is normally unnecessary. Provide
> `extensionPath` (the file name without its platform suffix, e.g. `/opt/sqlite-vec/vec0`) only to
> override the bundled binary or to support a platform that is not bundled. sqlite-vec is pre-v1 and
> performs exact (brute-force) KNN with no ANN index, making it suitable for thousands to low-millions
> of vectors.
### Zero-config default (SQLite persistence + AI)
When the server runs with **both** `conductor.db.type=sqlite` and `conductor.integrations.ai.enabled=true`,
Conductor automatically registers a vector DB instance named **`default`** backed by the bundled
sqlite-vec extension — no `conductor.vectordb.instances` entry required. Workflows can target it with
`"vectorDB": "default"`. The defaults below can be overridden:
```yaml
conductor:
vectordb:
sqlite-default:
name: "default" # instance name workflows reference
db-path: "" # default: a *_vectordb.db file next to the persistence DB
dimensions: 256 # must match the embedding model's output dimensions
distance-metric: "cosine" # l2, cosine or l1
extension-path: "" # default: the bundled vec0 binary
```
An explicitly configured instance named `default` takes precedence over the auto-registered one.
### Mixed Configuration (Multiple Types)
```yaml
conductor:
vectordb:
instances:
- name: "postgres-prod"
type: "postgres"
postgres:
datasourceURL: "jdbc:postgresql://prod:5432/vectors"
user: "conductor"
password: "secret"
dimensions: 1536
- name: "pinecone-embeddings"
type: "pinecone"
pinecone:
apiKey: "pk-xxx"
- name: "mongodb-cache"
type: "mongodb"
mongodb:
connectionString: "mongodb://localhost:27017"
database: "conductor"
```
## Usage in Workflows
When using vector database tasks in your workflows, reference the instance by its configured name:
```json
{
"name": "store_embeddings",
"taskReferenceName": "store_embeddings_ref",
"type": "LLM_STORE_EMBEDDINGS",
"inputParameters": {
"vectorDB": "postgres-prod",
"index": "documents",
"namespace": "my_namespace",
"embeddings": "${embedding_task.output.embeddings}",
"metadata": {
"documentId": "${workflow.input.docId}"
}
}
}
```
## PostgreSQL Configuration Options
| Property | Type | Default | Description |
|----------|------|---------|-------------|
| `datasourceURL` | String | Required | JDBC connection URL |
| `user` | String | Required | Database username |
| `password` | String | Required | Database password |
| `dimensions` | Integer | 256 | Vector dimensions |
| `connectionPoolSize` | Integer | 5 | Connection pool size |
| `indexingMethod` | String | "hnsw" | Index method (hnsw or ivfflat) |
| `distanceMetric` | String | "l2" | Distance metric (l2, cosine, inner_product) |
| `invertedListCount` | Integer | 100 | IVFFlat index parameter |
| `tablePrefix` | String | null | Prefix for table names |
## MongoDB Configuration Options
| Property | Type | Default | Description |
|----------|------|---------|-------------|
| `connectionString` | String | Required | MongoDB connection string |
| `database` | String | Required | Database name |
| `collection` | String | Optional | Collection name |
| `numCandidates` | Integer | Optional | Vector search parameter |
## Pinecone Configuration Options
| Property | Type | Default | Description |
|----------|------|---------|-------------|
| `apiKey` | String | Required | Pinecone API key |
## SQLite Configuration Options
| Property | Type | Default | Description |
|----------|------|---------|-------------|
| `dbPath` | String | Required | Path to the SQLite database file, or `:memory:` for an in-memory DB |
| `extensionPath` | String | bundled | Override path to the `vec0` extension (without platform suffix); defaults to the binary bundled in the jar |
| `dimensions` | Integer | 256 | Vector dimensions |
| `distanceMetric` | String | "l2" | Distance metric (l2, cosine, l1) |
| `connectionPoolSize` | Integer | 5 | Connection pool size (forced to 1 for `:memory:`) |
| `tablePrefix` | String | null | Prefix for table names |
## Migration from Old Configuration
### Old Format (Single Instance Per Type)
```yaml
conductor:
vectordb:
postgres:
datasourceURL: "jdbc:postgresql://localhost:5432/vectors"
user: "conductor"
password: "secret"
```
### New Format (Named Instances)
```yaml
conductor:
vectordb:
instances:
- name: "pgvectordb" # Use old type name for backward compatibility
type: "postgres"
postgres:
datasourceURL: "jdbc:postgresql://localhost:5432/vectors"
user: "conductor"
password: "secret"
```
**Note:** The type identifiers have been simplified:
- `pgvectordb``postgres`
- `mongovectordb``mongodb`
- `pineconedb``pinecone`
However, for backward compatibility, you can still reference instances using the old type names if you name your instance accordingly.
## Best Practices
1. **Use descriptive names**: Choose instance names that clearly indicate their purpose (e.g., `postgres-prod`, `pinecone-embeddings-search`)
2. **Separate environments**: Use different instances for different environments to avoid accidental data mixing
3. **Optimize dimensions**: Configure `dimensions` to match your embedding model to avoid runtime errors
4. **Connection pooling**: Adjust `connectionPoolSize` based on your workload and database capacity
5. **Index selection**:
- Use `hnsw` for better query performance (default)
- Use `ivfflat` for faster indexing with slightly lower query performance
6. **Distance metrics**:
- Use `cosine` for normalized embeddings
- Use `l2` (Euclidean) for absolute distances
- Use `inner_product` for dot product similarity
## Troubleshooting
### Instance Not Found
If you see an error like "Vector DB instance not found: xyz", check:
1. The instance name in your workflow matches the configured name exactly
2. The instance is properly configured in your application.yml/properties
3. The application has been restarted after configuration changes
### PostgreSQL Connection Issues
- Ensure pgvector extension is installed: `CREATE EXTENSION vector;`
- Verify JDBC URL format and network connectivity
- Check database user permissions
### MongoDB Vector Search
- Vector search requires MongoDB Atlas or MongoDB 6.0+ with Atlas Search
- Ensure vector search index is created on your collection
- Local MongoDB containers don't support vector search
### Pinecone
- Verify API key is valid and has necessary permissions
- Ensure index exists in your Pinecone account before using it
### SQLite (sqlite-vec)
- "no such function: load_extension" — extension loading is disabled; the backend enables it at the
driver level, so this usually means the driver in use does not permit it
- "no such module: vec0" — the `vec0` extension could not be loaded. The binary is bundled for common
platforms; if your platform is not bundled (the log shows "No bundled sqlite-vec extension for ..."),
install vec0 and set `extensionPath` to the compiled extension (Linux `.so`, macOS `.dylib`, Windows
`.dll`), ensuring it is readable by the server process
- "Embeddings must be of dimensions : N" — the instance's `dimensions` must equal the embedding model's
output size; for the auto-registered `default` instance set `conductor.vectordb.sqlite-default.dimensions`
or request that many dimensions from the embedding model
- Local-only: each instance maps to a single SQLite file on the server host and is not shared across
a cluster — use pgvector/Pinecone/MongoDB for distributed deployments
+119
View File
@@ -0,0 +1,119 @@
import java.security.MessageDigest
plugins {
id 'java'
}
repositories {
maven { url 'https://repo.spring.io/snapshot' }
maven { url 'https://repo.spring.io/milestone' }
}
// --- sqlite-vec loadable extensions -------------------------------------------------------------
// sqlite-vec has no Maven artifact and no JVM binding, so we download the official, checksum-pinned
// loadable binaries at build time and package them as jar resources under /sqlite-vec/<platform>/.
// SqliteVecExtensions extracts the right one at runtime. Apache-2.0/MIT licensed, redistributable.
ext.sqliteVecVersion = '0.1.9'
ext.sqliteVecArtifacts = [
'linux-x86_64' : 'b959baa1d8dc88861b1edb337b8587178cdcb12d60b4998f9d10b6a82052d5d7',
'linux-aarch64' : 'ea03d39541e478fab5974253c461e1cb5d77742f69e40cf96e3fad5bc309a37c',
'macos-aarch64' : '8282126333399ddfe98bbbcc7a1936e7252625aac49df056a98be602e46bfd29',
'macos-x86_64' : '53ad76e400786515e2edcaed2f01271dda846316390b761fadbd2dcf56aa4713',
'windows-x86_64': '51581189d52066b4dfc6631f6d7a3eab7dedc2260656ab09ca97ab3fb8165983',
]
def sqliteVecResourceDir = layout.buildDirectory.dir('generated/resources/sqlite-vec')
def downloadSqliteVec = tasks.register('downloadSqliteVec') {
description = 'Downloads, verifies and stages the sqlite-vec loadable extensions as jar resources.'
group = 'build'
inputs.property('version', sqliteVecVersion)
inputs.property('artifacts', sqliteVecArtifacts)
outputs.dir(sqliteVecResourceDir)
doLast {
def base = sqliteVecResourceDir.get().asFile
def cache = new File(temporaryDir, 'archives')
cache.mkdirs()
sqliteVecArtifacts.each { platform, expectedSha ->
def archive = new File(cache, "sqlite-vec-${platform}.tar.gz")
if (!archive.exists()) {
def url = "https://github.com/asg017/sqlite-vec/releases/download/v${sqliteVecVersion}/sqlite-vec-${sqliteVecVersion}-loadable-${platform}.tar.gz"
logger.lifecycle("Downloading sqlite-vec ${platform} from ${url}")
ant.get(src: url, dest: archive, verbose: false)
}
def digest = MessageDigest.getInstance('SHA-256')
archive.eachByte(64 * 1024) { buf, len -> digest.update(buf, 0, len) }
def actualSha = digest.digest().encodeHex().toString()
if (actualSha != expectedSha) {
throw new GradleException("Checksum mismatch for sqlite-vec ${platform}: expected ${expectedSha}, got ${actualSha}")
}
def platformDir = new File(base, "sqlite-vec/${platform}")
platformDir.mkdirs()
copy {
from tarTree(resources.gzip(archive))
into platformDir
}
}
}
}
// Register the task provider (not the bare directory) as a resource source dir so that every
// consumer of the resources — processResources, sourcesJar, etc. — automatically depends on
// downloadSqliteVec instead of racing it.
sourceSets.main.resources.srcDir(downloadSqliteVec)
dependencies {
compileOnly 'org.springframework.boot:spring-boot-starter'
// Needed to compile A2A push-notification callback controller and AgentSpan activation glue;
// supplied by the server at runtime.
compileOnly 'org.springframework.boot:spring-boot-starter-web'
compileOnly 'org.springframework.boot:spring-boot-autoconfigure'
implementation project(':conductor-common')
implementation project(':conductor-core')
api "org.springframework.ai:spring-ai-model:${revSpringAI}"
api "org.springframework.ai:spring-ai-client-chat:${revSpringAI}"
api "org.springframework.ai:spring-ai-mistral-ai:${revSpringAI}"
api "org.springframework.ai:spring-ai-bedrock-converse:${revSpringAI}"
api "org.springframework.ai:spring-ai-ollama:${revSpringAI}"
api "com.google.auth:google-auth-library-oauth2-http:1.33.0"
api "com.networknt:json-schema-validator:${revJSonSchemaValidator}"
api("io.modelcontextprotocol.sdk:mcp-core:${revMCP}")
api "com.squareup.okhttp3:okhttp:4.12.0"
api "org.apache.commons:commons-lang3"
// Markdown parsing and PDF generation
// Exclude openhtmltopdf-pdfbox (pulled in by flexmark-pdf-converter inside flexmark-all):
// it was compiled against fontbox 2.x and calls TTFParser.parse(InputStream) which was
// removed in fontbox 3.x, causing a NoSuchMethodError at runtime in GENERATE_PDF tasks.
// The custom MarkdownToPdfConverter uses PDFBox 3.x directly and does not need openhtmltopdf.
implementation("com.vladsch.flexmark:flexmark-all:${revFlexmark}") {
exclude group: "com.openhtmltopdf", module: "openhtmltopdf-pdfbox"
exclude group: "de.rototor.pdfbox", module: "graphics2d"
}
implementation "org.apache.pdfbox:pdfbox:3.0.5"
//Vector Databases
api "org.mongodb:mongodb-driver-sync:${mongodb}"
api "org.mongodb:mongodb-driver-core:${mongodb}"
api "org.mongodb:bson:${mongodb}"
api "io.pinecone:pinecone-client:${pinecone}"
api "com.pgvector:pgvector:${pgVector}"
// SQLite + sqlite-vec backend. The native vec0 extension is supplied by the operator at
// runtime (no JVM binding / Maven artifact exists); see SqliteConfig.
api "org.xerial:sqlite-jdbc:${sqliteJdbc}"
api "org.springframework.boot:spring-boot-starter-jdbc"
testImplementation 'org.springframework.boot:spring-boot-starter-web'
testImplementation "com.squareup.okhttp3:mockwebserver:4.12.0"
testImplementation "org.testcontainers:mongodb:${revTestContainer}"
testImplementation "org.testcontainers:postgresql:${revTestContainer}"
testImplementation "org.postgresql:postgresql:${revPostgres}"
testImplementation "org.apache.commons:commons-compress:${revCommonsCompress}"
testImplementation "com.h2database:h2:2.4.240"
}
+28
View File
@@ -0,0 +1,28 @@
{
"name": "chat_workflow",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "chat_task",
"taskReferenceName": "chat",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o-mini",
"messages": [
{
"role": "system",
"message": "You are a helpful assistant."
},
{
"role": "user",
"message": "What is the capital of France?"
}
],
"temperature": 0.7,
"maxTokens": 500
}
}
]
}
+17
View File
@@ -0,0 +1,17 @@
{
"name": "embedding_workflow",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "generate_embeddings",
"taskReferenceName": "embeddings",
"type": "LLM_GENERATE_EMBEDDINGS",
"inputParameters": {
"llmProvider": "openai",
"model": "text-embedding-3-small",
"text": "Conductor is an orchestration platform"
}
}
]
}
+21
View File
@@ -0,0 +1,21 @@
{
"name": "image_gen_workflow",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "generate_image",
"taskReferenceName": "image",
"type": "GENERATE_IMAGE",
"inputParameters": {
"llmProvider": "openai",
"model": "dall-e-3",
"prompt": "A futuristic cityscape at sunset",
"width": 1024,
"height": 1024,
"n": 1,
"style": "vivid"
}
}
]
}
+18
View File
@@ -0,0 +1,18 @@
{
"name": "tts_workflow",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "generate_audio",
"taskReferenceName": "audio",
"type": "GENERATE_AUDIO",
"inputParameters": {
"llmProvider": "openai",
"model": "tts-1",
"text": "Hello, this is a test of text to speech.",
"voice": "alloy"
}
}
]
}
+35
View File
@@ -0,0 +1,35 @@
{
"name": "semantic_search_workflow",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "index_documents",
"taskReferenceName": "index",
"type": "LLM_INDEX_TEXT",
"inputParameters": {
"vectorDB": "postgres-prod",
"namespace": "documentation",
"index": "tech_docs",
"embeddingModelProvider": "openai",
"embeddingModel": "text-embedding-3-small",
"text": "Conductor is a workflow orchestration platform",
"docId": "doc_001"
}
},
{
"name": "search_documents",
"taskReferenceName": "search",
"type": "LLM_SEARCH_INDEX",
"inputParameters": {
"vectorDB": "postgres-prod",
"namespace": "documentation",
"index": "tech_docs",
"embeddingModelProvider": "openai",
"embeddingModel": "text-embedding-3-small",
"query": "workflow orchestration",
"llmMaxResults": 5
}
}
]
}
+41
View File
@@ -0,0 +1,41 @@
{
"name": "rag_workflow",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "search_knowledge_base",
"taskReferenceName": "search",
"type": "LLM_SEARCH_INDEX",
"inputParameters": {
"vectorDB": "postgres-prod",
"namespace": "kb",
"index": "articles",
"embeddingModelProvider": "openai",
"embeddingModel": "text-embedding-3-small",
"query": "${workflow.input.question}",
"llmMaxResults": 3
}
},
{
"name": "generate_answer",
"taskReferenceName": "answer",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "anthropic",
"model": "claude-3-5-sonnet-20241022",
"messages": [
{
"role": "system",
"message": "Answer based on the following context: ${search.output.result}"
},
{
"role": "user",
"message": "${workflow.input.question}"
}
],
"temperature": 0.3
}
}
]
}
+96
View File
@@ -0,0 +1,96 @@
{
"name": "complete_rag_demo",
"description": "Index documents, search, and generate RAG answer",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "index_doc_1",
"taskReferenceName": "index_doc_1_ref",
"type": "LLM_INDEX_TEXT",
"inputParameters": {
"vectorDB": "postgres-prod",
"index": "demo_index",
"namespace": "demo_docs",
"docId": "intro-001",
"text": "Conductor is a distributed workflow orchestration engine that runs in the cloud. It allows developers to build complex stateful applications by orchestrating microservices.",
"embeddingModelProvider": "openai",
"embeddingModel": "text-embedding-3-small",
"dimensions": 1536,
"metadata": { "category": "introduction" }
}
},
{
"name": "index_doc_2",
"taskReferenceName": "index_doc_2_ref",
"type": "LLM_INDEX_TEXT",
"inputParameters": {
"vectorDB": "postgres-prod",
"index": "demo_index",
"namespace": "demo_docs",
"docId": "features-002",
"text": "Conductor supports multiple vector databases including PostgreSQL (pgvector), MongoDB Atlas, and Pinecone. It also integrates with LLM providers like OpenAI, Anthropic, and Azure OpenAI.",
"embeddingModelProvider": "openai",
"embeddingModel": "text-embedding-3-small",
"dimensions": 1536,
"metadata": { "category": "features" }
}
},
{
"name": "index_doc_3",
"taskReferenceName": "index_doc_3_ref",
"type": "LLM_INDEX_TEXT",
"inputParameters": {
"vectorDB": "postgres-prod",
"index": "demo_index",
"namespace": "demo_docs",
"docId": "config-003",
"text": "You can configure multiple named instances of the same vector database type for different environments like production, development, and staging.",
"embeddingModelProvider": "openai",
"embeddingModel": "text-embedding-3-small",
"dimensions": 1536,
"metadata": { "category": "configuration" }
}
},
{
"name": "search_index",
"taskReferenceName": "search_ref",
"type": "LLM_SEARCH_INDEX",
"inputParameters": {
"vectorDB": "postgres-prod",
"index": "demo_index",
"namespace": "demo_docs",
"query": "What vector databases does Conductor support?",
"embeddingModelProvider": "openai",
"embeddingModel": "text-embedding-3-small",
"dimensions": 1536,
"maxResults": 3
}
},
{
"name": "generate_rag_answer",
"taskReferenceName": "answer_ref",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o-mini",
"messages": [
{
"role": "system",
"message": "You are a technical expert. Answer the question using only the provided context."
},
{
"role": "user",
"message": "Context:\n${search_ref.output.result}\n\nQuestion: What vector databases does Conductor support?"
}
],
"temperature": 0.2
}
}
],
"outputParameters": {
"indexed_docs": ["${index_doc_1_ref.output}", "${index_doc_2_ref.output}", "${index_doc_3_ref.output}"],
"search_results": "${search_ref.output.result}",
"answer": "${answer_ref.output.result}"
}
}
+15
View File
@@ -0,0 +1,15 @@
{
"name": "mcp_list_tools_workflow",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "list_mcp_tools",
"taskReferenceName": "list_tools",
"type": "LIST_MCP_TOOLS",
"inputParameters": {
"mcpServer": "http://localhost:3001/mcp"
}
}
]
}
+18
View File
@@ -0,0 +1,18 @@
{
"name": "mcp_weather_workflow",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "get_weather",
"taskReferenceName": "weather",
"type": "CALL_MCP_TOOL",
"inputParameters": {
"mcpServer": "http://localhost:3001/mcp",
"method": "get_weather",
"location": "New York",
"units": "fahrenheit"
}
}
]
}
+21
View File
@@ -0,0 +1,21 @@
{
"name": "a2a_agent",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "call_currency_agent",
"taskReferenceName": "agent",
"type": "AGENT",
"inputParameters": {
"agentType": "a2a",
"agentUrl": "http://localhost:9999",
"text": "convert 100 USD to EUR",
"pollIntervalSeconds": 5,
"headers": {
"Authorization": "Bearer ${workflow.input.agentToken}"
}
}
}
]
}
+62
View File
@@ -0,0 +1,62 @@
{
"name": "mcp_ai_agent_workflow",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "list_available_tools",
"taskReferenceName": "discover_tools",
"type": "LIST_MCP_TOOLS",
"inputParameters": {
"mcpServer": "http://localhost:3001/mcp"
}
},
{
"name": "decide_which_tools_to_use",
"taskReferenceName": "plan",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "anthropic",
"model": "claude-3-5-sonnet-20241022",
"messages": [
{
"role": "system",
"message": "You are an AI agent. Available tools: ${discover_tools.output.tools}. User wants to: ${workflow.input.task}"
},
{
"role": "user",
"message": "Which tool should I use and what parameters? Respond with JSON: {method: string, arguments: object}"
}
],
"temperature": 0.1,
"maxTokens": 500
}
},
{
"name": "execute_tool",
"taskReferenceName": "execute",
"type": "CALL_MCP_TOOL",
"inputParameters": {
"mcpServer": "http://localhost:3001/mcp",
"method": "${plan.output.result.method}",
"arguments": "${plan.output.result.arguments}"
}
},
{
"name": "summarize_result",
"taskReferenceName": "summarize",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o-mini",
"messages": [
{
"role": "user",
"message": "Summarize this result for the user: ${execute.output.content}"
}
],
"maxTokens": 200
}
}
]
}
+15
View File
@@ -0,0 +1,15 @@
{
"name": "a2a_get_agent_card_workflow",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "discover_agent",
"taskReferenceName": "discover",
"type": "GET_AGENT_CARD",
"inputParameters": {
"agentUrl": "http://localhost:9999"
}
}
]
}
+22
View File
@@ -0,0 +1,22 @@
{
"name": "video_gen_openai_sora",
"description": "Generates a video using OpenAI Sora. The task is async -- it submits a job and polls until the video is ready.",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "generate_video",
"taskReferenceName": "sora_video",
"type": "GENERATE_VIDEO",
"inputParameters": {
"llmProvider": "openai",
"model": "sora-2",
"prompt": "A slow cinematic aerial shot of a coastal city at golden hour, waves crashing against cliffs",
"duration": 8,
"size": "1280x720",
"n": 1,
"style": "cinematic"
}
}
]
}
+23
View File
@@ -0,0 +1,23 @@
{
"name": "order_pizza",
"version": 1,
"schemaVersion": 2,
"description": "Takes a pizza order — exposed as an A2A agent via metadata.a2a.enabled",
"ownerEmail": "a2a@example.com",
"metadata": {
"a2a.enabled": true,
"a2a.tags": ["ordering", "demo"]
},
"tasks": [
{
"name": "confirm_order",
"taskReferenceName": "confirm",
"type": "INLINE",
"inputParameters": {
"evaluatorType": "graaljs",
"expression": "({ orderId: 'ORD-' + ($.text || 'unknown') })",
"text": "${workflow.input._a2a_text}"
}
}
]
}
+25
View File
@@ -0,0 +1,25 @@
{
"name": "video_gen_gemini_veo",
"description": "Generates a video using Google Gemini Veo. Supports negative prompts, person generation controls, and optional audio generation.",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "generate_video",
"taskReferenceName": "veo_video",
"type": "GENERATE_VIDEO",
"inputParameters": {
"llmProvider": "vertex_ai",
"model": "veo-3",
"prompt": "A time-lapse of a blooming flower in a sunlit garden, soft bokeh background",
"duration": 8,
"aspectRatio": "16:9",
"resolution": "720p",
"personGeneration": "dont_allow",
"generateAudio": true,
"negativePrompt": "blurry, low quality, text overlay",
"n": 1
}
}
]
}
@@ -0,0 +1,34 @@
{
"name": "image_to_video_pipeline",
"description": "A two-step creative pipeline: generates a still image with DALL-E, then creates a video continuation using OpenAI Sora with a prompt inspired by the scene.",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "generate_image",
"taskReferenceName": "source_image",
"type": "GENERATE_IMAGE",
"inputParameters": {
"llmProvider": "openai",
"model": "dall-e-3",
"prompt": "A serene mountain lake at dawn with mist rising from the water, photorealistic, wide landscape",
"width": 1792,
"height": 1024,
"n": 1
}
},
{
"name": "generate_video",
"taskReferenceName": "animated_video",
"type": "GENERATE_VIDEO",
"inputParameters": {
"llmProvider": "openai",
"model": "sora-2",
"prompt": "A serene mountain lake at dawn, gentle ripples spread across the water as mist slowly drifts, birds flying in the distance, cinematic slow motion",
"duration": 8,
"size": "1280x720",
"style": "cinematic"
}
}
]
}
+22
View File
@@ -0,0 +1,22 @@
{
"name": "image_gen_stabilityai",
"description": "Generates an image using Stability AI's SD3.5 Large model via the v2beta API.",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "generate_image",
"taskReferenceName": "stable_diffusion_image",
"type": "GENERATE_IMAGE",
"inputParameters": {
"llmProvider": "stabilityai",
"model": "sd3.5-large",
"prompt": "A fantasy castle perched on a floating island surrounded by clouds, digital art, highly detailed",
"width": 1024,
"height": 1024,
"n": 1,
"style": "cinematic"
}
}
]
}
+24
View File
@@ -0,0 +1,24 @@
{
"name": "pdf_generation_workflow",
"description": "Generate a PDF document from markdown content with custom layout options",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "generate_pdf",
"taskReferenceName": "pdf",
"type": "GENERATE_PDF",
"inputParameters": {
"markdown": "# Monthly Sales Report\n\n## Executive Summary\n\nThis report covers sales performance for **Q4 2025**.\n\n| Region | Revenue | Growth |\n|--------|---------|--------|\n| North America | $2.4M | +12% |\n| Europe | $1.8M | +8% |\n| Asia Pacific | $1.2M | +15% |\n\n## Key Highlights\n\n- Total revenue reached **$5.4M**, exceeding target by 10%\n- Customer acquisition increased by *23%* across all regions\n- Product satisfaction score: **4.7/5.0**\n\n## Action Items\n\n1. Expand APAC sales team by Q1 2026\n2. Launch enterprise tier in European market\n3. Increase marketing budget for North America\n\n> *\"Our best quarter yet -- the team delivered exceptional results across every metric.\"* -- VP of Sales\n\n---\n\nGenerated by Conductor Workflow Engine",
"pageSize": "LETTER",
"theme": "default",
"baseFontSize": 11,
"pdfMetadata": {
"title": "Monthly Sales Report - Q4 2025",
"author": "Conductor Workflow",
"subject": "Quarterly Sales Performance"
}
}
}
]
}
+51
View File
@@ -0,0 +1,51 @@
{
"name": "llm_to_pdf_pipeline",
"description": "End-to-end pipeline: LLM generates a markdown report from user input, then converts it to a PDF document",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["topic", "audience"],
"tasks": [
{
"name": "generate_report_markdown",
"taskReferenceName": "llm_report",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o-mini",
"messages": [
{
"role": "system",
"message": "You are a professional report writer. Generate well-structured markdown reports with headings, tables, bullet points, and bold/italic emphasis. Always include an executive summary, key findings, and recommendations sections."
},
{
"role": "user",
"message": "Write a detailed report about: ${workflow.input.topic}\n\nTarget audience: ${workflow.input.audience}\n\nUse markdown formatting with:\n- A clear title (# heading)\n- Executive summary section\n- Key findings with a data table\n- Bullet-point recommendations\n- A blockquote conclusion"
}
],
"temperature": 0.7,
"maxTokens": 2000
}
},
{
"name": "convert_to_pdf",
"taskReferenceName": "pdf_output",
"type": "GENERATE_PDF",
"inputParameters": {
"markdown": "${llm_report.output.result}",
"pageSize": "A4",
"theme": "default",
"baseFontSize": 11,
"pdfMetadata": {
"title": "${workflow.input.topic}",
"author": "Conductor AI Pipeline",
"subject": "Auto-generated report"
}
}
}
],
"outputParameters": {
"reportMarkdown": "${llm_report.output.result}",
"pdfLocation": "${pdf_output.output.result.location}",
"pdfSizeBytes": "${pdf_output.output.result.sizeBytes}"
}
}
+34
View File
@@ -0,0 +1,34 @@
{
"name": "web_search_workflow",
"description": "Chat completion with built-in web search — the LLM can search the web for real-time information",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["question"],
"tasks": [
{
"name": "web_search_chat",
"taskReferenceName": "chat",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o-mini",
"messages": [
{
"role": "system",
"message": "You are a helpful research assistant. Use web search to find current information."
},
{
"role": "user",
"message": "${workflow.input.question}"
}
],
"webSearch": true,
"temperature": 0.3,
"maxTokens": 1000
}
}
],
"outputParameters": {
"answer": "${chat.output.result}"
}
}
+34
View File
@@ -0,0 +1,34 @@
{
"name": "code_execution_workflow",
"description": "Chat completion with built-in code execution — the LLM can write and run code in a sandbox",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["task"],
"tasks": [
{
"name": "code_execution_chat",
"taskReferenceName": "chat",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "google_gemini",
"model": "gemini-2.5-flash",
"messages": [
{
"role": "system",
"message": "You are a data analyst. Use code execution to compute results, generate charts, and analyze data."
},
{
"role": "user",
"message": "${workflow.input.task}"
}
],
"codeInterpreter": true,
"temperature": 0.2,
"maxTokens": 2000
}
}
],
"outputParameters": {
"result": "${chat.output.result}"
}
}
+78
View File
@@ -0,0 +1,78 @@
{
"name": "coding_agent",
"description": "A coding agent that uses OpenAI code_interpreter to write, run, and iterate on code",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["task"],
"tasks": [
{
"name": "plan_implementation",
"taskReferenceName": "plan",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o",
"messages": [
{
"role": "system",
"message": "You are a senior software engineer. Break down the coding task into clear steps. Output a numbered plan."
},
{
"role": "user",
"message": "${workflow.input.task}"
}
],
"temperature": 0.2,
"maxTokens": 1000
}
},
{
"name": "write_and_run_code",
"taskReferenceName": "code",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o",
"messages": [
{
"role": "system",
"message": "You are a coding assistant with access to a code execution sandbox. Write the code, run it, verify the output, and fix any errors. Return the final working code and its output."
},
{
"role": "user",
"message": "Implementation plan:\n${plan.output.result}\n\nOriginal task: ${workflow.input.task}\n\nWrite the code, execute it, and return the working result."
}
],
"codeInterpreter": true,
"temperature": 0.1,
"maxTokens": 4000
}
},
{
"name": "review_code",
"taskReferenceName": "review",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o-mini",
"messages": [
{
"role": "system",
"message": "You are a code reviewer. Review the implementation for correctness, edge cases, and code quality. Provide a brief summary of what was built and any suggestions."
},
{
"role": "user",
"message": "Task: ${workflow.input.task}\n\nImplementation:\n${code.output.result}"
}
],
"temperature": 0.3,
"maxTokens": 1000
}
}
],
"outputParameters": {
"plan": "${plan.output.result}",
"code": "${code.output.result}",
"review": "${review.output.result}"
}
}
+29
View File
@@ -0,0 +1,29 @@
{
"name": "extended_thinking_workflow",
"description": "Chat completion with extended thinking — gives the LLM a token budget for step-by-step reasoning",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["problem"],
"tasks": [
{
"name": "think_deeply",
"taskReferenceName": "think",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "anthropic",
"model": "claude-sonnet-4-20250514",
"messages": [
{
"role": "user",
"message": "${workflow.input.problem}"
}
],
"thinkingTokenLimit": 10000,
"maxTokens": 16000
}
}
],
"outputParameters": {
"answer": "${think.output.result}"
}
}
@@ -0,0 +1,70 @@
{
"name": "web_research_agent",
"description": "An autonomous research agent that uses web search to gather information and synthesize a report",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["topic"],
"tasks": [
{
"name": "gather_information",
"taskReferenceName": "research",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o",
"messages": [
{
"role": "system",
"message": "You are a research analyst. Use web search to find comprehensive, current information about the topic. Search for multiple perspectives and recent developments."
},
{
"role": "user",
"message": "Research this topic thoroughly: ${workflow.input.topic}"
}
],
"webSearch": true,
"temperature": 0.3,
"maxTokens": 3000
}
},
{
"name": "synthesize_report",
"taskReferenceName": "report",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "anthropic",
"model": "claude-sonnet-4-20250514",
"messages": [
{
"role": "system",
"message": "You are a technical writer. Synthesize the research into a well-structured markdown report with sections, key findings, and citations."
},
{
"role": "user",
"message": "Topic: ${workflow.input.topic}\n\nResearch findings:\n${research.output.result}\n\nWrite a comprehensive report in markdown format."
}
],
"thinkingTokenLimit": 5000,
"maxTokens": 8000
}
},
{
"name": "convert_to_pdf",
"taskReferenceName": "pdf",
"type": "GENERATE_PDF",
"inputParameters": {
"markdown": "${report.output.result}",
"pageSize": "A4",
"theme": "default",
"pdfMetadata": {
"title": "${workflow.input.topic}",
"author": "Conductor Research Agent"
}
}
}
],
"outputParameters": {
"report": "${report.output.result}",
"pdf": "${pdf.output.result.location}"
}
}
+52
View File
@@ -0,0 +1,52 @@
{
"name": "multi_turn_chain",
"description": "Two-step conversation using previousResponseId to avoid resending history",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["topic"],
"tasks": [
{
"name": "first_turn",
"taskReferenceName": "turn1",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o",
"messages": [
{
"role": "system",
"message": "You are a technical architect. Be concise."
},
{
"role": "user",
"message": "Design a high-level architecture for: ${workflow.input.topic}"
}
],
"temperature": 0.3,
"maxTokens": 2000
}
},
{
"name": "follow_up",
"taskReferenceName": "turn2",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o",
"messages": [
{
"role": "user",
"message": "Now list the key risks and mitigations for this architecture."
}
],
"previousResponseId": "${turn1.output.responseId}",
"temperature": 0.3,
"maxTokens": 2000
}
}
],
"outputParameters": {
"architecture": "${turn1.output.result}",
"risks": "${turn2.output.result}"
}
}
+19
View File
@@ -0,0 +1,19 @@
{
"name": "a2a_agent_streaming",
"version": 1,
"schemaVersion": 2,
"description": "Calls a remote agent in streaming mode — consumes the agent's message/stream (SSE) and aggregates events to completion. Requires the agent card to advertise capabilities.streaming=true.",
"ownerEmail": "a2a@example.com",
"tasks": [
{
"name": "stream_from_agent",
"taskReferenceName": "agent",
"type": "AGENT",
"inputParameters": {
"agentUrl": "http://localhost:9999",
"text": "summarize the attached document",
"streaming": true
}
}
]
}
+21
View File
@@ -0,0 +1,21 @@
{
"name": "a2a_agent_push",
"version": 1,
"schemaVersion": 2,
"description": "Calls a remote agent in push mode — the agent calls Conductor's webhook when the task reaches a terminal state, so no worker thread polls. Requires conductor.a2a.callback.url to be set (otherwise it falls back to polling). A slow backstop poll still runs so a lost webhook can't hang the task.",
"ownerEmail": "a2a@example.com",
"tasks": [
{
"name": "call_research_agent",
"taskReferenceName": "agent",
"type": "AGENT",
"inputParameters": {
"agentUrl": "http://localhost:9999",
"text": "research the latest on durable agent protocols",
"pushNotification": true,
"pushBackstopPollSeconds": 300,
"maxDurationSeconds": 3600
}
}
]
}
+28
View File
@@ -0,0 +1,28 @@
{
"name": "book_appointment",
"version": 1,
"schemaVersion": 2,
"description": "Multi-turn A2A agent: asks a clarifying question (HUMAN task → A2A 'input-required'), then confirms. Exposed via metadata.a2a.enabled. A client resumes it by sending another message/send carrying the returned task id; that follow-up completes the HUMAN task and the workflow continues.",
"ownerEmail": "a2a@example.com",
"metadata": {
"a2a.enabled": true,
"a2a.tags": ["scheduling", "multi-turn"]
},
"tasks": [
{
"name": "ask_preferred_time",
"taskReferenceName": "ask",
"type": "HUMAN"
},
{
"name": "confirm_appointment",
"taskReferenceName": "confirm",
"type": "INLINE",
"inputParameters": {
"evaluatorType": "graaljs",
"expression": "({ status: 'confirmed', when: $.when })",
"when": "${ask.output._a2a_text}"
}
}
]
}
+28
View File
@@ -0,0 +1,28 @@
{
"name": "a2a_cancel_agent",
"version": 1,
"schemaVersion": 2,
"description": "Starts work on a remote agent (AGENT) then cancels it (CANCEL_AGENT → A2A tasks/cancel), passing the remote task id from the first task's output. Illustrative — cancel is typically wired into fork/timeout or compensation flows.",
"ownerEmail": "a2a@example.com",
"tasks": [
{
"name": "start_agent_task",
"taskReferenceName": "start",
"type": "AGENT",
"inputParameters": {
"agentUrl": "http://localhost:9999",
"text": "begin a long-running job",
"maxDurationSeconds": 60
}
},
{
"name": "cancel_agent_task",
"taskReferenceName": "cancel",
"type": "CANCEL_AGENT",
"inputParameters": {
"agentUrl": "http://localhost:9999",
"taskId": "${start.output.taskId}"
}
}
]
}
+44
View File
@@ -0,0 +1,44 @@
{
"name": "a2a_multi_agent_orchestration",
"version": 1,
"schemaVersion": 2,
"description": "Durable multi-agent orchestration: call several remote A2A agents in parallel with FORK_JOIN, then JOIN their results. Each branch is an independent, crash-safe AGENT — if Conductor restarts mid-flight, every in-flight agent call resumes.",
"ownerEmail": "a2a@example.com",
"tasks": [
{
"name": "fork_agents",
"taskReferenceName": "fork",
"type": "FORK_JOIN",
"forkTasks": [
[
{
"name": "call_flights_agent",
"taskReferenceName": "flights",
"type": "AGENT",
"inputParameters": {
"agentUrl": "${workflow.input.flightsAgentUrl}",
"text": "Find flights for: ${workflow.input.request}"
}
}
],
[
{
"name": "call_hotels_agent",
"taskReferenceName": "hotels",
"type": "AGENT",
"inputParameters": {
"agentUrl": "${workflow.input.hotelsAgentUrl}",
"text": "Find hotels for: ${workflow.input.request}"
}
}
]
]
},
{
"name": "join_agents",
"taskReferenceName": "join",
"type": "JOIN",
"joinOn": ["flights", "hotels"]
}
]
}
+35
View File
@@ -0,0 +1,35 @@
{
"name": "a2a_llm_pick_skill",
"version": 1,
"schemaVersion": 2,
"description": "Agentic A2A: discover a remote agent's Agent Card (GET_AGENT_CARD), let an LLM choose the best prompt from the advertised skills (LLM_CHAT_COMPLETE), then call the agent (AGENT). Discovery + reasoning + invocation, all durable.",
"ownerEmail": "a2a@example.com",
"tasks": [
{
"name": "discover_agent",
"taskReferenceName": "discover",
"type": "GET_AGENT_CARD",
"inputParameters": { "agentUrl": "${workflow.input.agentUrl}" }
},
{
"name": "choose_request",
"taskReferenceName": "choose",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "${workflow.input.llmProvider}",
"model": "${workflow.input.model}",
"instructions": "You are given an A2A agent's Agent Card (its skills) and a user goal. Reply with ONLY the single best prompt to send that agent.",
"userInput": "Agent card: ${discover.output.agentCard}\nUser goal: ${workflow.input.goal}"
}
},
{
"name": "call_agent",
"taskReferenceName": "call",
"type": "AGENT",
"inputParameters": {
"agentUrl": "${workflow.input.agentUrl}",
"text": "${choose.output.result}"
}
}
]
}
+42
View File
@@ -0,0 +1,42 @@
{
"name": "a2a_client_multi_turn",
"version": 1,
"schemaVersion": 2,
"description": "Client-side multi-turn conversation: call a remote agent; if it returns input-required, branch (SWITCH on the agent's state) and call again with the SAME contextId/taskId carrying the answer — resuming the same remote task.",
"ownerEmail": "a2a@example.com",
"tasks": [
{
"name": "ask_agent",
"taskReferenceName": "ask",
"type": "AGENT",
"inputParameters": {
"agentUrl": "${workflow.input.agentUrl}",
"text": "${workflow.input.prompt}"
}
},
{
"name": "branch_on_state",
"taskReferenceName": "branch",
"type": "SWITCH",
"evaluatorType": "value-param",
"expression": "state",
"inputParameters": { "state": "${ask.output.state}" },
"decisionCases": {
"input-required": [
{
"name": "answer_agent",
"taskReferenceName": "answer",
"type": "AGENT",
"inputParameters": {
"agentUrl": "${workflow.input.agentUrl}",
"text": "${workflow.input.answer}",
"contextId": "${ask.output.contextId}",
"taskId": "${ask.output.taskId}"
}
}
]
},
"defaultCase": []
}
]
}
+96
View File
@@ -0,0 +1,96 @@
{
"name": "rag_sqlite_vec_demo",
"description": "Zero-infrastructure RAG using the bundled SQLite + sqlite-vec vector store. Requires conductor.db.type=sqlite and conductor.integrations.ai.enabled=true, which auto-registers the 'default' vector DB instance. Embeddings are requested at 256 dimensions to match the default instance.",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "index_doc_1",
"taskReferenceName": "index_doc_1_ref",
"type": "LLM_INDEX_TEXT",
"inputParameters": {
"vectorDB": "default",
"index": "demo_index",
"namespace": "demo_docs",
"docId": "intro-001",
"text": "Conductor is a distributed workflow orchestration engine. It lets developers build complex stateful applications by orchestrating microservices and AI agents.",
"embeddingModelProvider": "openai",
"embeddingModel": "text-embedding-3-small",
"dimensions": 256,
"metadata": { "category": "introduction" }
}
},
{
"name": "index_doc_2",
"taskReferenceName": "index_doc_2_ref",
"type": "LLM_INDEX_TEXT",
"inputParameters": {
"vectorDB": "default",
"index": "demo_index",
"namespace": "demo_docs",
"docId": "vectordb-002",
"text": "Conductor supports several vector databases: PostgreSQL (pgvector), MongoDB Atlas, Pinecone, and an embedded SQLite backend powered by the sqlite-vec extension that needs no external server.",
"embeddingModelProvider": "openai",
"embeddingModel": "text-embedding-3-small",
"dimensions": 256,
"metadata": { "category": "features" }
}
},
{
"name": "index_doc_3",
"taskReferenceName": "index_doc_3_ref",
"type": "LLM_INDEX_TEXT",
"inputParameters": {
"vectorDB": "default",
"index": "demo_index",
"namespace": "demo_docs",
"docId": "sqlite-003",
"text": "When SQLite persistence and the AI integration are both enabled, Conductor bundles the sqlite-vec native extension and registers a default vector store automatically, so semantic search works out of the box.",
"embeddingModelProvider": "openai",
"embeddingModel": "text-embedding-3-small",
"dimensions": 256,
"metadata": { "category": "configuration" }
}
},
{
"name": "search_index",
"taskReferenceName": "search_ref",
"type": "LLM_SEARCH_INDEX",
"inputParameters": {
"vectorDB": "default",
"index": "demo_index",
"namespace": "demo_docs",
"query": "${workflow.input.question}",
"embeddingModelProvider": "openai",
"embeddingModel": "text-embedding-3-small",
"dimensions": 256,
"maxResults": 3
}
},
{
"name": "generate_rag_answer",
"taskReferenceName": "answer_ref",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o-mini",
"messages": [
{
"role": "system",
"message": "You are a technical expert. Answer the question using only the provided context."
},
{
"role": "user",
"message": "Context:\n${search_ref.output.result}\n\nQuestion: ${workflow.input.question}"
}
],
"temperature": 0.2
}
}
],
"inputParameters": ["question"],
"outputParameters": {
"search_results": "${search_ref.output.result}",
"answer": "${answer_ref.output.result}"
}
}
+545
View File
@@ -0,0 +1,545 @@
# Conductor AI Workflow Examples
This folder contains ready-to-use workflow examples demonstrating the AI capabilities of Conductor.
## Prerequisites
### 1. Start Conductor Server
Ensure Conductor is running with AI integrations enabled:
```bash
# From the conductor root directory
./gradlew bootRun
```
### 2. Configure AI Providers
Set environment variables before starting the server:
```bash
# OpenAI (required for most examples)
export OPENAI_API_KEY=sk-your-openai-api-key
# Anthropic (optional, for RAG examples)
export ANTHROPIC_API_KEY=sk-ant-your-anthropic-key
# Google Gemini (optional, for Gemini/Veo examples)
# Option 1: API key (simplest)
export GEMINI_API_KEY=your-gemini-api-key
# Option 2: Vertex AI — set project and location in application.properties
```
For vector database examples, add to `application.properties`:
```properties
# PostgreSQL Vector DB (for RAG/embedding examples)
conductor.vectordb.instances[0].name=postgres-prod
conductor.vectordb.instances[0].type=postgres
conductor.vectordb.instances[0].postgres.datasourceURL=jdbc:postgresql://localhost:5432/vectors
conductor.vectordb.instances[0].postgres.user=conductor
conductor.vectordb.instances[0].postgres.password=secret
conductor.vectordb.instances[0].postgres.dimensions=1536
```
### 3. MCP Test Server (for MCP examples)
Install and start the MCP test server:
```bash
# Install mcp-testkit — a test MCP server with 65 deterministic tools
pip install mcp-testkit
# Start the server in HTTP mode
mcp-testkit --transport http
```
The server will be available at `http://localhost:3001/mcp`.
---
## Available Examples
| File | Description | Requirements |
|------|-------------|--------------|
| `01-chat-completion.json` | Basic chat with GPT-4o-mini | OpenAI |
| `02-generate-embeddings.json` | Generate text embeddings | OpenAI |
| `03-image-generation.json` | Generate images with DALL-E 3 | OpenAI |
| `04-audio-generation.json` | Text-to-speech with OpenAI TTS | OpenAI |
| `05-semantic-search.json` | Index and search documents | OpenAI, PostgreSQL |
| `06-rag-basic.json` | Basic RAG with search + answer | OpenAI/Anthropic, PostgreSQL |
| `07-rag-complete.json` | Full RAG demo (index + search + answer) | OpenAI, PostgreSQL |
| `08-mcp-list-tools.json` | List tools from MCP server | MCP Server |
| `09-mcp-call-tool.json` | Call MCP tool (weather) | MCP Server |
| `10-mcp-ai-agent.json` | AI agent with MCP tools | OpenAI/Anthropic, MCP Server |
| `11-video-openai-sora.json` | Generate video with OpenAI Sora-2 (async) | OpenAI |
| `12-video-gemini-veo.json` | Generate video with Google Veo-3 (async) | Google Vertex AI |
| `13-image-to-video-pipeline.json` | Image + video generation pipeline | OpenAI |
| `14-stabilityai-image.json` | Image generation with Stability AI (SD3.5) | Stability AI |
| `15-pdf-generation.json` | Generate PDF from markdown content | None (built-in) |
| `16-llm-to-pdf-pipeline.json` | LLM generates report → convert to PDF | OpenAI |
| `17-web-search.json` | Chat with built-in web search for real-time info | OpenAI |
| `18-code-execution.json` | Chat with built-in code execution sandbox | Google Gemini |
| `19-coding-agent.json` | Coding agent: plan → write & run code → review | OpenAI |
| `20-extended-thinking.json` | Extended thinking with token budget for reasoning | Anthropic |
| `21-web-search-research-agent.json` | Research agent: web search → synthesize → PDF | OpenAI, Anthropic |
| `22-multi-turn-chain.json` | Multi-turn conversation chaining with previousResponseId | OpenAI |
| `30-rag-sqlite-vec.json` | Zero-infra RAG on the bundled SQLite + sqlite-vec store | OpenAI, SQLite (built-in) |
### A2A (Agent2Agent) examples
Conductor as an A2A **client** (calling remote agents) and **server** (exposing a workflow as an
agent). The client tasks (`AGENT`, `GET_AGENT_CARD`, `CANCEL_AGENT`) need a reachable A2A
agent — see `ai/src/test/resources/a2a/` for a runnable test agent. The server examples are exposed
by registering them with `metadata.a2a.enabled=true` and `conductor.a2a.server.enabled=true`.
| File | Description | Requirements |
|------|-------------|--------------|
| `10-a2a-call-agent.json` | Call a remote agent (poll mode) | A2A agent |
| `11-a2a-get-agent-card.json` | Discover an agent's skills/capabilities | A2A agent |
| `12-a2a-server-workflow.json` | Expose a workflow as an A2A agent (server) | `conductor.a2a.server.enabled=true` |
| `23-a2a-streaming.json` | Call an agent in streaming (SSE) mode | A2A agent (`capabilities.streaming=true`) |
| `24-a2a-push.json` | Call an agent in push-notification mode | A2A agent, `conductor.a2a.callback.url` |
| `25-a2a-server-multi-turn.json` | Multi-turn server agent (HUMAN task → input-required → resume) | `conductor.a2a.server.enabled=true` |
| `26-a2a-cancel.json` | Start then cancel a remote agent task | A2A agent |
| `27-a2a-multi-agent.json` | Call multiple agents in parallel (FORK_JOIN → JOIN) | A2A agents |
| `28-a2a-llm-pick-skill.json` | Discover an agent, let an LLM pick the prompt, then call it | A2A agent, OpenAI/Anthropic |
| `29-a2a-client-multi-turn.json` | Client multi-turn: branch on input-required, re-call with the same context | A2A agent |
---
## Quick Start
### Step 1: Register a Workflow
```bash
# Register the chat completion workflow
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @01-chat-completion.json
```
### Step 2: Execute the Workflow
```bash
# Run the workflow (no input needed for hardcoded examples)
curl -X POST 'http://localhost:8080/api/workflow/chat_workflow' \
-H 'Content-Type: application/json' \
-d '{}'
```
### Step 3: Check the Result
```bash
# Get workflow execution status (replace {workflowId} with the returned ID)
curl -X GET 'http://localhost:8080/api/workflow/{workflowId}'
```
---
## Example Commands
### 1. Chat Completion
```bash
# Register
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @01-chat-completion.json
# Execute
curl -X POST 'http://localhost:8080/api/workflow/chat_workflow' \
-H 'Content-Type: application/json' \
-d '{}'
```
### 2. Generate Embeddings
```bash
# Register
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @02-generate-embeddings.json
# Execute
curl -X POST 'http://localhost:8080/api/workflow/embedding_workflow' \
-H 'Content-Type: application/json' \
-d '{}'
```
### 3. Image Generation
```bash
# Register
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @03-image-generation.json
# Execute
curl -X POST 'http://localhost:8080/api/workflow/image_gen_workflow' \
-H 'Content-Type: application/json' \
-d '{}'
```
### 4. Audio Generation (TTS)
```bash
# Register
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @04-audio-generation.json
# Execute
curl -X POST 'http://localhost:8080/api/workflow/tts_workflow' \
-H 'Content-Type: application/json' \
-d '{}'
```
### 5. Semantic Search
```bash
# Register
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @05-semantic-search.json
# Execute
curl -X POST 'http://localhost:8080/api/workflow/semantic_search_workflow' \
-H 'Content-Type: application/json' \
-d '{}'
```
### 6. RAG (Basic)
```bash
# Register
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @06-rag-basic.json
# Execute with a question
curl -X POST 'http://localhost:8080/api/workflow/rag_workflow' \
-H 'Content-Type: application/json' \
-d '{"question": "What is Conductor?"}'
```
### 7. RAG (Complete Demo)
```bash
# Register
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @07-rag-complete.json
# Execute (no input needed - fully self-contained)
curl -X POST 'http://localhost:8080/api/workflow/complete_rag_demo' \
-H 'Content-Type: application/json' \
-d '{}'
```
### 30. RAG on SQLite (sqlite-vec, zero infrastructure)
Runs the full index → search → answer RAG loop against the **embedded** SQLite + sqlite-vec vector
store — no PostgreSQL, MongoDB or Pinecone required. When the server runs with `conductor.db.type=sqlite`
and `conductor.integrations.ai.enabled=true`, Conductor bundles the native `vec0` extension and
auto-registers a vector DB instance named `default`, which this workflow targets. Embeddings are
requested at 256 dimensions to match that default instance.
```bash
# Register
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @30-rag-sqlite-vec.json
# Execute with a question
curl -X POST 'http://localhost:8080/api/workflow/rag_sqlite_vec_demo' \
-H 'Content-Type: application/json' \
-d '{"question": "What vector databases does Conductor support?"}'
```
### 8. MCP List Tools
```bash
# Start MCP server first (see Prerequisites)
# Register
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @08-mcp-list-tools.json
# Execute
curl -X POST 'http://localhost:8080/api/workflow/mcp_list_tools_workflow' \
-H 'Content-Type: application/json' \
-d '{}'
```
### 9. MCP Call Tool (Weather)
```bash
# Start MCP server first (see Prerequisites)
# Register
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @09-mcp-call-tool.json
# Execute
curl -X POST 'http://localhost:8080/api/workflow/mcp_weather_workflow' \
-H 'Content-Type: application/json' \
-d '{}'
```
### 10. MCP AI Agent
```bash
# Start MCP server first (see Prerequisites)
# Register
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @10-mcp-ai-agent.json
# Execute with a task
curl -X POST 'http://localhost:8080/api/workflow/mcp_ai_agent_workflow' \
-H 'Content-Type: application/json' \
-d '{"task": "Get the current weather in San Francisco"}'
```
### 11. Video Generation (OpenAI Sora)
```bash
# Register
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @11-video-openai-sora.json
# Execute (async -- returns workflowId immediately, polls internally until video is ready)
curl -X POST 'http://localhost:8080/api/workflow/video_gen_openai_sora' \
-H 'Content-Type: application/json' \
-d '{}'
```
### 12. Video Generation (Google Gemini Veo)
```bash
# Requires Google Vertex AI credentials (see Prerequisites)
# Register
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @12-video-gemini-veo.json
# Execute
curl -X POST 'http://localhost:8080/api/workflow/video_gen_gemini_veo' \
-H 'Content-Type: application/json' \
-d '{}'
```
### 13. Image-to-Video Pipeline
```bash
# Register
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @13-image-to-video-pipeline.json
# Execute (generates a DALL-E image first, then a Sora video)
curl -X POST 'http://localhost:8080/api/workflow/image_to_video_pipeline' \
-H 'Content-Type: application/json' \
-d '{}'
```
### 14. Image Generation (Stability AI)
```bash
# Requires STABILITY_API_KEY environment variable
# Register
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @14-stabilityai-image.json
# Execute
curl -X POST 'http://localhost:8080/api/workflow/image_gen_stabilityai' \
-H 'Content-Type: application/json' \
-d '{}'
```
### 15. PDF Generation (Markdown to PDF)
```bash
# No external API keys required -- uses built-in PDFBox renderer
# Register
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @15-pdf-generation.json
# Execute
curl -X POST 'http://localhost:8080/api/workflow/pdf_generation_workflow' \
-H 'Content-Type: application/json' \
-d '{}'
```
### 16. LLM-to-PDF Pipeline (Report Generation)
```bash
# Register
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @16-llm-to-pdf-pipeline.json
# Execute with a topic and audience
curl -X POST 'http://localhost:8080/api/workflow/llm_to_pdf_pipeline' \
-H 'Content-Type: application/json' \
-d '{"topic": "Cloud Migration Best Practices", "audience": "CTO and engineering leadership"}'
```
### 17. Web Search
```bash
# Register
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @17-web-search.json
# Execute with a question about current events
curl -X POST 'http://localhost:8080/api/workflow/web_search_workflow' \
-H 'Content-Type: application/json' \
-d '{"question": "What are the latest developments in AI regulation?"}'
```
### 18. Code Execution
```bash
# Register
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @18-code-execution.json
# Execute with a data analysis task
curl -X POST 'http://localhost:8080/api/workflow/code_execution_workflow' \
-H 'Content-Type: application/json' \
-d '{"task": "Generate the first 50 Fibonacci numbers and calculate the golden ratio convergence"}'
```
### 19. Coding Agent
```bash
# Register
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @19-coding-agent.json
# Execute — the agent plans, writes code, executes, and reviews
curl -X POST 'http://localhost:8080/api/workflow/coding_agent' \
-H 'Content-Type: application/json' \
-d '{"task": "Write a Python function that converts Roman numerals to integers, with unit tests"}'
```
### 20. Extended Thinking
```bash
# Register
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @20-extended-thinking.json
# Execute with a complex reasoning problem
curl -X POST 'http://localhost:8080/api/workflow/extended_thinking_workflow' \
-H 'Content-Type: application/json' \
-d '{"problem": "Design a distributed consensus algorithm for a system with up to 3 Byzantine nodes out of 10 total. Explain the correctness proof."}'
```
### 21. Web Research Agent
```bash
# Register
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @21-web-search-research-agent.json
# Execute — researches the topic, writes a report, converts to PDF
curl -X POST 'http://localhost:8080/api/workflow/web_research_agent' \
-H 'Content-Type: application/json' \
-d '{"topic": "The state of WebAssembly in 2026"}'
```
### 22. Multi-Turn Conversation Chain
```bash
# Register
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @22-multi-turn-chain.json
# Execute — second turn uses previousResponseId to continue the conversation without resending history
curl -X POST 'http://localhost:8080/api/workflow/multi_turn_chain' \
-H 'Content-Type: application/json' \
-d '{"topic": "Real-time collaborative document editor"}'
```
---
## Register All Workflows at Once
```bash
# Register all example workflows
for f in *.json; do
echo "Registering $f..."
curl -s -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @"$f"
echo ""
done
```
---
## Troubleshooting
### "VectorDB not found: postgres-prod"
Ensure you have configured the PostgreSQL vector database in your `application.properties`:
```properties
conductor.vectordb.instances[0].name=postgres-prod
conductor.vectordb.instances[0].type=postgres
conductor.vectordb.instances[0].postgres.datasourceURL=jdbc:postgresql://localhost:5432/vectors
conductor.vectordb.instances[0].postgres.user=conductor
conductor.vectordb.instances[0].postgres.password=secret
conductor.vectordb.instances[0].postgres.dimensions=1536
```
### "No configuration found for: openai"
Ensure you have set the OpenAI API key environment variable:
```bash
export OPENAI_API_KEY=sk-your-openai-api-key
```
### MCP Server Connection Refused
1. Verify the MCP server is running:
```bash
curl http://localhost:3001/mcp
```
2. Check the server logs for errors
3. Ensure you're using the correct port in the workflow (default: 3001)
### PostgreSQL Vector Extension Not Found
Ensure the `pgvector` extension is installed in your PostgreSQL database:
```sql
CREATE EXTENSION IF NOT EXISTS vector;
```
---
## License
Copyright 2026 Conductor Authors. Licensed under the Apache License 2.0.
@@ -0,0 +1,235 @@
/*
* Copyright 2025 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.conductoross.conductor.ai;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.function.Function;
import org.conductoross.conductor.ai.model.AudioGenRequest;
import org.conductoross.conductor.ai.model.ChatCompletion;
import org.conductoross.conductor.ai.model.EmbeddingGenRequest;
import org.conductoross.conductor.ai.model.ImageGenRequest;
import org.conductoross.conductor.ai.model.LLMResponse;
import org.conductoross.conductor.ai.model.ToolSpec;
import org.conductoross.conductor.ai.model.VideoGenRequest;
import org.conductoross.conductor.ai.video.VideoModel;
import org.conductoross.conductor.ai.video.VideoOptions;
import org.conductoross.conductor.ai.video.VideoOptionsBuilder;
import org.springframework.ai.chat.model.ChatModel;
import org.springframework.ai.chat.prompt.ChatOptions;
import org.springframework.ai.image.ImageModel;
import org.springframework.ai.image.ImageOptions;
import org.springframework.ai.image.ImageOptionsBuilder;
import org.springframework.ai.model.tool.ToolCallingChatOptions;
import org.springframework.ai.tool.ToolCallback;
import org.springframework.ai.tool.function.FunctionToolCallback;
import com.netflix.conductor.common.config.ObjectMapperProvider;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
/** Interface for LLM implementations. */
public interface AIModel {
enum ConductorTask {
CHAT_COMPLETE,
GENERATE_IMAGE,
GENERATE_VIDEO,
TEXT_TO_SPEECH
}
ObjectMapper objectMapper = new ObjectMapperProvider().getObjectMapper();
/**
* @return name of the foundation model provider. e.g. openai, anthropic etc.
*/
String getModelProvider();
/**
* @return alternative provider names that resolve to this same provider
*/
default List<String> getProviderAliases() {
return List.of();
}
/**
* Whether this provider accepts a chat-completion request whose last message has the {@code
* assistant} role — i.e. assistant-message prefill, used to nudge the model's response to start
* a certain way (and, in this codebase, the format in which {@link
* org.conductoross.conductor.ai.tasks.mapper.ChatCompleteTaskMapper}'s loop-history injection
* carries prior DO_WHILE iteration outputs back into the next iteration).
*
* <p>When this returns {@code false}, the mapper suppresses the same-refName loop-iteration
* assistant injection: those auto-attached assistant messages would either be rejected outright
* by the provider's API (e.g. Anthropic Claude Sonnet 4.6+ returns {@code 400 "This model does
* not support assistant message prefill. The conversation must end with a user message."}) or
* be silently mis-handled. Participants, tool calls, and sub-workflow context are unaffected by
* this flag.
*
* <p>Default is {@code true} to preserve historical behavior for providers (OpenAI Responses
* API with {@code previousResponseId}, Gemini, etc.) that accept trailing assistant messages.
*/
default boolean supportsAssistantPrefill() {
return true;
}
/**
* Embedding generation
*
* @param embeddingGenRequest request
* @return embeddings
*/
List<Float> generateEmbeddings(EmbeddingGenRequest embeddingGenRequest);
/**
* @return Chat Completion model
*/
ChatModel getChatModel();
/**
* @param input request to do chat completion
* @return Options
*/
default ChatOptions getChatOptions(ChatCompletion input) {
return ToolCallingChatOptions.builder()
.model(input.getModel())
.maxTokens(input.getMaxTokens())
.topP(input.getTopP())
.temperature(input.getTemperature())
.toolCallbacks(getToolCallback(input))
.stopSequences(input.getStopWords())
.frequencyPenalty(input.getFrequencyPenalty())
.topK(input.getTopK())
.internalToolExecutionEnabled(false)
.presencePenalty(input.getPresencePenalty())
.build();
}
/**
* @param input Image gen request
* @return Options
*/
default ImageOptions getImageOptions(ImageGenRequest input) {
return ImageOptionsBuilder.builder()
.model(input.getModel())
.height(input.getHeight())
.width(input.getWidth())
.N(input.getN())
.responseFormat("b64_json")
.style(input.getStyle())
.build();
}
/**
* @return Model to generate images
*/
ImageModel getImageModel();
/**
* @param input Video gen request
* @return Options
*/
default VideoOptions getVideoOptions(VideoGenRequest input) {
return VideoOptionsBuilder.builder()
.model(input.getModel())
.duration(input.getDuration())
.width(input.getWidth())
.height(input.getHeight())
.fps(input.getFps())
.outputFormat(input.getOutputFormat())
.n(input.getN())
.style(input.getStyle())
.motion(input.getMotion())
.seed(input.getSeed())
.guidanceScale(input.getGuidanceScale())
.aspectRatio(input.getAspectRatio())
.generateThumbnail(input.getGenerateThumbnail())
.thumbnailTimestamp(input.getThumbnailTimestamp())
.inputImage(input.getInputImage())
.negativePrompt(input.getNegativePrompt())
.personGeneration(input.getPersonGeneration())
.resolution(input.getResolution())
.generateAudio(input.getGenerateAudio())
.size(input.getSize())
.build();
}
/**
* @return Model to generate videos
*/
default VideoModel getVideoModel() {
return null; // Default: video generation not supported
}
/**
* Generate video (async job submission)
*
* @param request Video generation request
* @return Response with job ID
*/
default LLMResponse generateVideo(VideoGenRequest request) {
throw new UnsupportedOperationException("Video generation not supported by this provider");
}
/**
* Check video generation job status (polling)
*
* @param request Video generation request with jobId
* @return Response with current status or completed video
*/
default LLMResponse checkVideoStatus(VideoGenRequest request) {
throw new UnsupportedOperationException("Video generation not supported by this provider");
}
default LLMResponse generateAudio(AudioGenRequest request) {
throw new UnsupportedOperationException();
}
default List<ToolCallback> getToolCallback(ChatCompletion input) {
if (input.getTools() == null || input.getTools().isEmpty()) {
return List.of();
}
List<ToolCallback> functions = new ArrayList<>();
try {
for (ToolSpec tool : input.getTools()) {
FunctionToolCallback<Object, Object> function =
FunctionToolCallback.builder(tool.getName(), Function.identity())
.description(tool.getDescription())
.inputSchema(objectMapper.writeValueAsString(tool.getInputSchema()))
.inputType(Map.class) // does not matter, we are not doing
// internal tool calling!
.build();
functions.add(function);
}
} catch (JsonProcessingException jpe) {
throw new RuntimeException(jpe);
}
return functions;
}
static URI getURI(String input) {
if (input == null || input.isBlank()) {
return null;
}
try {
return new URI(input);
} catch (URISyntaxException e) {
return null;
}
}
}
@@ -0,0 +1,77 @@
/*
* Copyright 2025 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.conductoross.conductor.ai;
import java.io.File;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.function.Consumer;
import org.conductoross.conductor.ai.model.LLMWorkerInput;
import org.springframework.core.env.Environment;
import org.springframework.stereotype.Component;
import lombok.Getter;
import lombok.extern.slf4j.Slf4j;
@Getter
@Component
@Slf4j
public class AIModelProvider {
private final Map<String, AIModel> providerToLLM = new HashMap<>();
private String payloadStoreLocation;
public AIModelProvider(
List<ModelConfiguration<? extends AIModel>> modelConfigurations, Environment env) {
String defaultPayloadStoreLocation = System.getProperty("user.home") + "/worker-payload/";
for (ModelConfiguration<? extends AIModel> modelConfiguration : modelConfigurations) {
try {
AIModel llm = modelConfiguration.get();
payloadStoreLocation =
env.getProperty(
"conductor.file-storage.parentDir", defaultPayloadStoreLocation);
boolean result = new File(payloadStoreLocation).mkdirs();
log.info(
"Created directory {} ? {} for storing worker payload data",
payloadStoreLocation,
result);
providerToLLM.put(llm.getModelProvider(), llm);
for (String alias : llm.getProviderAliases()) {
providerToLLM.put(alias, llm);
}
} catch (Throwable t) {
log.error("cannot init {} model, reason: {}", modelConfiguration, t.getMessage());
}
}
}
public AIModel getModel(LLMWorkerInput input) {
String name = input.getLlmProvider();
if (name == null) {
throw new RuntimeException("llmProvider not specified: " + name);
}
AIModel model = providerToLLM.get(name);
if (model == null) {
throw new RuntimeException("no configuration found for: " + name);
}
return model;
}
public Consumer<TokenUsageLog> getTokenUsageLogger() {
return usageLog -> log.info("{}", usageLog);
}
}
@@ -0,0 +1,909 @@
/*
* Copyright 2025 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.conductoross.conductor.ai;
import java.net.URI;
import java.util.ArrayList;
import java.util.Base64;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import java.util.UUID;
import java.util.function.Consumer;
import java.util.stream.Collectors;
import org.apache.commons.lang3.StringUtils;
import org.conductoross.conductor.ai.document.DocumentLoader;
import org.conductoross.conductor.ai.http.AIHttpClients;
import org.conductoross.conductor.ai.model.AudioGenRequest;
import org.conductoross.conductor.ai.model.ChatCompletion;
import org.conductoross.conductor.ai.model.ChatMessage;
import org.conductoross.conductor.ai.model.EmbeddingGenRequest;
import org.conductoross.conductor.ai.model.ImageGenRequest;
import org.conductoross.conductor.ai.model.LLMResponse;
import org.conductoross.conductor.ai.model.ToolCall;
import org.conductoross.conductor.ai.model.ToolSpec;
import org.conductoross.conductor.ai.model.VideoGenRequest;
import org.conductoross.conductor.common.JsonSchemaValidator;
import org.conductoross.conductor.common.utils.StringTemplate;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.messages.AssistantMessage;
import org.springframework.ai.chat.messages.Message;
import org.springframework.ai.chat.messages.SystemMessage;
import org.springframework.ai.chat.messages.ToolResponseMessage;
import org.springframework.ai.chat.messages.UserMessage;
import org.springframework.ai.chat.model.ChatModel;
import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.ai.chat.model.Generation;
import org.springframework.ai.chat.prompt.ChatOptions;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.ai.content.Media;
import org.springframework.ai.image.ImageGeneration;
import org.springframework.ai.image.ImageMessage;
import org.springframework.ai.image.ImageModel;
import org.springframework.ai.image.ImageOptions;
import org.springframework.ai.image.ImagePrompt;
import org.springframework.ai.image.ImageResponse;
import org.springframework.util.MimeType;
import com.netflix.conductor.common.config.ObjectMapperProvider;
import com.netflix.conductor.common.metadata.SchemaDef;
import com.netflix.conductor.common.metadata.tasks.Task;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.google.common.annotations.VisibleForTesting;
import com.networknt.schema.JsonSchemaException;
import com.networknt.schema.ValidationMessage;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.Response;
import okhttp3.ResponseBody;
import static com.netflix.conductor.common.metadata.tasks.TaskType.TASK_TYPE_SIMPLE;
import static org.conductoross.conductor.ai.MimeExtensionResolver.getExtension;
import static org.conductoross.conductor.ai.MimeExtensionResolver.getMimeTypeFromUrl;
@Slf4j
public class LLMHelper {
private static final TypeReference<Map<String, Object>> MAP_OF_STRING_TO_OBJ =
new TypeReference<>() {};
private static final Map<String, String> finishReasonMap =
Map.of("end_turn", "STOP", "tool_use", "TOOL_CALLS", "refusal", "CONTENT_FILTER");
private final ObjectMapper objectMapper = new ObjectMapperProvider().getObjectMapper();
private final JsonSchemaValidator jsonSchemaValidator;
private final List<DocumentLoader> documentLoaders;
private final OkHttpClient httpClient;
public LLMHelper(
JsonSchemaValidator jsonSchemaValidator, List<DocumentLoader> documentLoaders) {
this(jsonSchemaValidator, documentLoaders, AIHttpClients.defaultClient());
}
public LLMHelper(
JsonSchemaValidator jsonSchemaValidator,
List<DocumentLoader> documentLoaders,
OkHttpClient httpClient) {
this.jsonSchemaValidator = jsonSchemaValidator;
this.documentLoaders = documentLoaders;
this.httpClient = httpClient;
}
public LLMResponse chatComplete(
Task task,
AIModel llm,
ChatCompletion chatCompletion,
String payloadStoreLocation,
Consumer<TokenUsageLog> tokenUsageLogger) {
ChatModel chatModel = llm.getChatModel();
ChatOptions chatOptions = llm.getChatOptions(chatCompletion);
LLMResponse response = chatComplete(chatModel, chatOptions, chatCompletion);
String prompt =
replacePromptVariables(
chatCompletion.getInstructions(), chatCompletion.getPromptVariables());
chatCompletion.setPrompt(prompt);
extractResponse(response, chatCompletion);
storeMedia(payloadStoreLocation, response.getMedia());
TokenUsageLog usage =
TokenUsageLog.builder()
.taskId(task.getTaskId())
.api(chatCompletion.getModel())
.integrationName(chatCompletion.getLlmProvider())
.completionTokens(response.getCompletionTokens())
.promptTokens(response.getPromptTokens())
.totalTokens(response.getTokenUsed())
.build();
tokenUsageLogger.accept(usage);
return response;
}
public LLMResponse generateImage(
Task task,
AIModel llm,
ImageGenRequest imageGenRequest,
String payloadStoreLocation,
Consumer<TokenUsageLog> tokenUsageLogger) {
String prompt =
replacePromptVariables(
imageGenRequest.getPrompt(), imageGenRequest.getPromptVariables());
imageGenRequest.setPrompt(prompt);
ImageOptions options = llm.getImageOptions(imageGenRequest);
ImageModel model = llm.getImageModel();
LLMResponse response = generateImage(model, options, imageGenRequest);
storeMedia(payloadStoreLocation, response.getMedia());
TokenUsageLog usage =
TokenUsageLog.builder()
.taskId(task.getTaskId())
.api(imageGenRequest.getModel())
.integrationName(imageGenRequest.getLlmProvider())
.completionTokens(response.getCompletionTokens())
.promptTokens(response.getPromptTokens())
.totalTokens(response.getTokenUsed())
.build();
tokenUsageLogger.accept(usage);
return response;
}
public List<Float> generateEmbeddings(
Task task,
AIModel llm,
EmbeddingGenRequest embeddingGenRequest,
Consumer<TokenUsageLog> tokenUsageLogger) {
return llm.generateEmbeddings(embeddingGenRequest);
}
public LLMResponse generateAudio(
Task task,
AIModel llm,
AudioGenRequest request,
String payloadStoreLocation,
Consumer<TokenUsageLog> tokenUsageLogger) {
LLMResponse response = llm.generateAudio(request);
storeMedia(payloadStoreLocation, response.getMedia());
TokenUsageLog usage =
TokenUsageLog.builder()
.taskId(task.getTaskId())
.api(request.getModel())
.integrationName(request.getLlmProvider())
.completionTokens(response.getCompletionTokens())
.promptTokens(response.getPromptTokens())
.totalTokens(response.getTokenUsed())
.build();
tokenUsageLogger.accept(usage);
return response;
}
public LLMResponse generateVideo(
Task task,
AIModel llm,
VideoGenRequest videoGenRequest,
String payloadStoreLocation,
Consumer<TokenUsageLog> tokenUsageLogger) {
return llm.generateVideo(videoGenRequest);
}
public LLMResponse checkVideoStatus(
Task task, AIModel llm, VideoGenRequest videoGenRequest, String payloadStoreLocation) {
LLMResponse response = llm.checkVideoStatus(videoGenRequest);
// If completed, download and store media
if ("COMPLETED".equals(response.getFinishReason())) {
storeMedia(payloadStoreLocation, response.getMedia());
}
return response;
}
// Helper methods
private String replacePromptVariables(String prompt, Map<String, Object> paramReplacement) {
if (StringUtils.isBlank(prompt)) {
return prompt;
}
if (paramReplacement != null) {
prompt = StringTemplate.fString(prompt, paramReplacement);
}
return prompt;
}
@SneakyThrows
@SuppressWarnings({"raw", "unchecked"})
private void extractResponse(LLMResponse llmResponse, ChatCompletion input) {
if (llmResponse.getResult() == null || llmResponse.getResult().toString().isEmpty()) {
// empty response
log.debug("empty response: finishReason: {}", llmResponse.getFinishReason());
return;
}
Object result = llmResponse.getResult();
switch (result) {
case null -> llmResponse.setResult(Map.of());
case String ignored ->
llmResponse.setResult(
tryToConvertToJSON(llmResponse.getResult().toString(), input));
case List<?> resultList -> {
List<String> errors = new ArrayList<>();
List<Object> output = new ArrayList<>();
boolean hasJsonOutput = false;
for (Object o : resultList) {
String responseText = o.toString();
var responseObj = tryToConvertToJSON(responseText, input);
hasJsonOutput = true;
if (input.getOutputSchema() != null) {
String error = null;
if (!(responseObj instanceof Map)) {
error = "not a JSON response: %s".formatted(responseObj);
} else {
error =
validateJsonSchema(
input.getInputSchema(),
(Map<String, Object>) responseObj);
}
if (error != null) {
errors.add(
String.format(
"Output does not confirm to the schema. errors: %s",
error));
}
}
output.add(responseObj);
}
llmResponse.setResult(output);
if (input.isJsonOutput() && !hasJsonOutput && !llmResponse.hasToolCalls()) {
Map<String, Object> outputErrors = Map.of("error", errors, "response", output);
throw new RuntimeException(objectMapper.writeValueAsString(outputErrors));
}
}
default -> llmResponse.setResult(result.toString());
}
}
@SneakyThrows
private Object tryToConvertToJSON(String responseText, ChatCompletion chatCompletion) {
try {
responseText = responseText.trim();
if (responseText.startsWith("```json")) {
responseText = responseText.substring("```json".length());
responseText =
responseText.substring(0, responseText.length() - "```".length() - 1);
}
Map<String, Object> map = objectMapper.readValue(responseText, MAP_OF_STRING_TO_OBJ);
if (chatCompletion.getOutputSchema() != null) {
String error = validateJsonSchema(chatCompletion.getInputSchema(), map);
if (error != null) {
throw new RuntimeException(
String.format(
"Output does not confirm to the schema. errors: %s", error));
}
}
// llmResponse.setResult(map);
return map;
} catch (JsonProcessingException e) {
if (chatCompletion.isJsonOutput()) {
log.error(
"error converting to json, response: {}, error: {}",
responseText,
e.getMessage(),
e);
Map<String, Object> outputErrors =
Map.of("error", e.getMessage(), "response", responseText);
throw new RuntimeException(objectMapper.writeValueAsString(outputErrors));
}
return responseText;
}
}
private String validateJsonSchema(final SchemaDef schema, Map<String, Object> data) {
try {
// Order in which we use the schema
// 1. If there is data -- inline schema def, we use that
// 2. Else use name + version to lookup
// 3. externalRef if present, in future we will use it -- currently not supported
String schemaContent = objectMapper.writeValueAsString(schema.getData());
if (schemaContent == null) {
return null;
}
Set<ValidationMessage> validationMessages =
jsonSchemaValidator.validate(schemaContent, data);
if (validationMessages != null && !validationMessages.isEmpty()) {
return String.format(
"Schema validation failed %s",
validationMessages.stream()
.map(ValidationMessage::getMessage)
.collect(Collectors.joining(", ")));
}
return null;
} catch (JsonSchemaException jpe) {
throw new RuntimeException(
"Bad/Unsupported schema? : " + jpe.getValidationMessages().toString());
} catch (JsonProcessingException jpe) {
throw new RuntimeException("Error parsing the json schema : " + jpe.getMessage(), jpe);
}
}
@SneakyThrows
private LLMResponse chatComplete(
ChatModel chatModel, ChatOptions chatOptions, ChatCompletion input) {
ChatClient chatClient = ChatClient.create(chatModel);
if (StringUtils.isNotBlank(input.getInstructions())) {
input.getMessages()
.addFirst(new ChatMessage(ChatMessage.Role.system, input.getInstructions()));
}
List<Message> messages =
new ArrayList<>(input.getMessages().stream().map(this::constructMessage).toList());
ensureLastMessageIsFromUser(messages);
Prompt prompt = new Prompt(messages, chatOptions);
ChatResponse chatResponse = chatClient.prompt(prompt).call().chatResponse();
if (chatResponse == null) {
throw new RuntimeException("No response generated");
}
if (chatResponse.getResults().isEmpty()) {
String result = objectMapper.writeValueAsString(chatResponse);
return LLMResponse.builder()
.result(result)
.completionTokens(chatResponse.getMetadata().getUsage().getCompletionTokens())
.promptTokens(chatResponse.getMetadata().getUsage().getPromptTokens())
.tokenUsed(chatResponse.getMetadata().getUsage().getTotalTokens())
.build();
}
List<ToolCall> tools = null;
String finishReason = null;
List<String> responses = new ArrayList<>();
List<org.conductoross.conductor.ai.model.Media> media = new ArrayList<>();
for (Generation result : chatResponse.getResults()) {
if (result.getOutput().hasToolCalls()) {
List<AssistantMessage.ToolCall> toolCalls = result.getOutput().getToolCalls();
tools = new ArrayList<>();
for (AssistantMessage.ToolCall toolCall : toolCalls) {
String name = toolCall.name();
String id = toolCall.id();
if (id == null || id.isBlank()) {
id = UUID.randomUUID().toString();
}
String argsAsString = toolCall.arguments();
Map<String, Object> args = new HashMap<>();
try {
@SuppressWarnings("unchecked")
Map<String, Object> parsedArgs =
objectMapper.readValue(argsAsString, Map.class);
// Recursively parse any nested JSON strings
args = parseNestedJsonStrings(parsedArgs);
} catch (JsonProcessingException ignored) {
log.warn(ignored.getMessage(), ignored);
}
args.put("method", name);
Optional<ToolSpec> matched =
input.getTools().stream()
.filter(toolSpec -> toolSpec.getName().equals(name))
.findFirst();
String type = matched.map(ToolSpec::getType).orElse(TASK_TYPE_SIMPLE);
tools.add(
ToolCall.builder()
.taskReferenceName(id)
.name(name)
.inputParameters(args)
.integrationNames(getIntegrationNames(name, input.getTools()))
.type(type)
.build());
}
finishReason = result.getMetadata().getFinishReason();
} else {
responses.add(result.getOutput().getText());
result.getOutput()
.getMedia()
.forEach(
m ->
media.add(
org.conductoross.conductor.ai.model.Media.builder()
.data(m.getDataAsByteArray())
.mimeType(m.getMimeType().toString())
.build()));
// storeMedia(outputLocation, result.getOutput().getMedia());
if (finishReason == null) {
finishReason = result.getMetadata().getFinishReason();
}
}
}
Object result = responses;
if (responses.size() == 1) {
result = responses.getFirst();
}
finishReason = finishReasonMap.getOrDefault(finishReason, finishReason).toUpperCase();
// Extract response_id if present (set by OpenAI Responses API for chaining)
String responseId = null;
Object respIdObj = chatResponse.getMetadata().get("response_id");
if (respIdObj instanceof String rid) {
responseId = rid;
}
// Reasoning summary + reasoning token count. Surfaced by the OpenAI
// Responses API, Anthropic extended thinking, and Gemini thought
// summaries — the chat-model adapters normalize all three onto these
// two metadata keys before we read them here.
String reasoning = null;
Object reasoningObj = chatResponse.getMetadata().get("reasoning");
if (reasoningObj instanceof String r && !r.isBlank()) {
reasoning = r;
}
Integer reasoningTokens = null;
Object rtObj = chatResponse.getMetadata().get("reasoning_tokens");
// ``Number`` rather than ``Integer`` so a Long survives the Jackson
// round-trip ChatResponseMetadata may go through. Token counts won't
// overflow int — convert and move on.
if (rtObj instanceof Number rt) {
reasoningTokens = rt.intValue();
}
return LLMResponse.builder()
.result(result)
.media(media)
.toolCalls(tools)
.finishReason(finishReason)
.responseId(responseId)
.reasoning(reasoning)
.reasoningTokens(reasoningTokens)
.completionTokens(chatResponse.getMetadata().getUsage().getCompletionTokens())
.promptTokens(chatResponse.getMetadata().getUsage().getPromptTokens())
.tokenUsed(chatResponse.getMetadata().getUsage().getTotalTokens())
.build();
}
private LLMResponse generateImage(
ImageModel imageModel, ImageOptions options, ImageGenRequest request) {
ImageMessage imageMessage = new ImageMessage(request.getPrompt(), request.getWeight());
ImagePrompt prompt = new ImagePrompt(List.of(imageMessage), options);
ImageResponse response = imageModel.call(prompt);
LLMResponse mediaGenResponse = new LLMResponse();
List<org.conductoross.conductor.ai.model.Media> mediaList = new ArrayList<>();
for (ImageGeneration result : response.getResults()) {
var image = result.getOutput();
String url = image.getUrl();
String base64 = image.getB64Json();
// Determine the mime type from the output format
String mimeType =
"image/"
+ (request.getOutputFormat() != null
? request.getOutputFormat()
: "png");
if (base64 != null) {
// Base64 data provided - decode and store
mediaList.add(
org.conductoross.conductor.ai.model.Media.builder()
.data(Base64.getDecoder().decode(base64))
.mimeType(mimeType)
.build());
} else if (url != null) {
// URL provided - download the image bytes so we can store locally
// This ensures the image is persisted even after the provider's URL expires
byte[] imageBytes = downloadImageFromUrl(url);
if (imageBytes != null) {
// Detect mime type from URL extension if possible
String detectedMimeType = getMimeTypeFromUrl(url, mimeType);
mediaList.add(
org.conductoross.conductor.ai.model.Media.builder()
.data(imageBytes)
.mimeType(detectedMimeType)
.build());
} else {
// Fallback: if download fails, keep the URL (will expire but better than
// nothing)
log.warn("Failed to download image from URL, keeping external URL: {}", url);
mediaList.add(
org.conductoross.conductor.ai.model.Media.builder()
.location(url)
.mimeType(mimeType)
.build());
}
}
}
mediaGenResponse.setMedia(mediaList);
return mediaGenResponse;
}
/**
* Downloads image bytes from a URL. Used to persist images locally instead of relying on
* provider-hosted URLs that may expire.
*
* @param url The image URL to download from
* @return The image bytes, or null if download failed
*/
private byte[] downloadImageFromUrl(String url) {
try {
Request request = new Request.Builder().url(url).get().build();
try (Response response = httpClient.newCall(request).execute()) {
if (!response.isSuccessful()) {
log.error(
"Failed to download image from URL {}: HTTP {}", url, response.code());
return null;
}
ResponseBody body = response.body();
if (body == null) {
log.error("Empty response body when downloading image from URL: {}", url);
return null;
}
byte[] bytes = body.bytes();
log.debug("Downloaded {} bytes from image URL: {}", bytes.length, url);
return bytes;
}
} catch (Exception e) {
log.error("Exception downloading image from URL {}: {}", url, e.getMessage());
return null;
}
}
@SneakyThrows
private Message constructMessage(ChatMessage chatMessage) {
return switch (chatMessage.getRole()) {
case user -> getMessage(chatMessage);
case assistant -> new AssistantMessage(chatMessage.getMessage());
case system -> new SystemMessage(chatMessage.getMessage());
case tool_call ->
AssistantMessage.builder()
.content("{}")
.toolCalls(
chatMessage.getToolCalls().stream()
.map(
tc -> {
var name =
extractMethodFromInputParameters(
tc.getInputParameters());
return new AssistantMessage.ToolCall(
tc.getTaskReferenceName(),
"function",
name == null ? tc.getName() : name,
toJSON(tc.getInputParameters()));
})
.toList())
.build();
case tool -> {
List<ToolCall> toolCalls = chatMessage.getToolCalls();
if (toolCalls == null) {
log.warn("chat message role: {}, but toolCalls is null", chatMessage.getRole());
toolCalls = new ArrayList<>();
}
try {
List<ToolResponseMessage.ToolResponse> responses = new ArrayList<>();
if (toolCalls.isEmpty()) {
log.info("toolCalls is empty for {}", chatMessage);
}
for (ToolCall toolCall : toolCalls) {
Map<String, Object> inputJson = toolCall.getInputParameters();
var name = extractMethodFromInputParameters(inputJson);
String outputJSON = objectMapper.writeValueAsString(toolCall.getOutput());
log.trace("outputJSON for {} is {}", toolCall.getName(), outputJSON);
log.info("tool: {}", toolCall);
log.info("tool.getTaskReferenceName: {}", toolCall.getTaskReferenceName());
responses.add(
new ToolResponseMessage.ToolResponse(
toolCall.getTaskReferenceName(),
name == null ? toolCall.getName() : name,
outputJSON));
}
log.trace("responses: {}", responses);
yield ToolResponseMessage.builder().responses(responses).build();
} catch (Exception e) {
log.error("error: {}", e.getMessage(), e);
yield new SystemMessage(chatMessage.getMessage());
}
}
};
}
private String toJSON(Object input) {
try {
if (input == null) {
return "{}";
}
return objectMapper.writeValueAsString(input);
} catch (JsonProcessingException jpe) {
return String.valueOf(input);
}
}
/**
* Extracts the "method" value from a map structure where the outer key is dynamic. The map
* structure is expected to be: { "dynamicKey": { "method": "methodName", ... } }
*
* @param inputParameters The map containing the nested structure
* @return The method value as a String, or null if not found
*/
@SuppressWarnings("unchecked")
@VisibleForTesting
String extractMethodFromInputParameters(Map<String, Object> inputParameters) {
if (inputParameters == null || inputParameters.isEmpty()) {
return null;
}
// Iterate through all entries to find the nested map with "method" key
for (Map.Entry<String, Object> entry : inputParameters.entrySet()) {
Object value = entry.getValue();
if (value instanceof Map) {
Map<String, Object> nestedMap = (Map<String, Object>) value;
if (nestedMap.containsKey("method")) {
Object methodValue = nestedMap.get("method");
return methodValue != null ? methodValue.toString() : null;
}
}
}
return null;
}
private Message getMessage(ChatMessage msg) {
List<String> rawMedia = msg.getMedia();
List<Media> media =
(rawMedia == null ? List.<String>of() : rawMedia).stream()
.map(m -> getMedia(msg.getMimeType(), m))
.filter(Objects::nonNull)
.toList();
return UserMessage.builder().text(msg.getMessage()).media(media).build();
}
private Media getMedia(String mimeType, String content) {
// content can be a URL or base64 encoded data
URI uri = AIModel.getURI(content);
if (uri == null) {
return Media.builder().data(content).mimeType(MimeType.valueOf(mimeType)).build();
}
Optional<byte[]> data =
documentLoaders.stream()
.filter(documentLoader -> documentLoader.supports(content))
.findFirst()
.map(loader -> loader.download(content));
final String mimeTypeResolved =
Optional.ofNullable(mimeType)
.orElse(MimeExtensionResolver.getMimeTypeFromUrl(content, ""));
return data.map(
bytes ->
Media.builder()
.data(bytes)
.mimeType(MimeType.valueOf(mimeTypeResolved))
.build())
.orElse(null);
}
private void storeMedia(
String location, List<org.conductoross.conductor.ai.model.Media> media) {
DocumentLoader documentLoader =
documentLoaders.stream()
.filter(loader -> loader.supports(location))
.findFirst()
.orElse(null);
if (documentLoader == null) {
log.debug("no document loaders found, media will not be stored");
return;
}
media.stream()
.filter(m1 -> m1.getData() != null)
.forEach(
m -> {
// Each media item gets a unique path with file extension
// to prevent overwriting when multiple items exist
// (e.g., video + thumbnail)
String ext = getExtension(m.getMimeType());
String uniqueLocation =
location + "_" + java.util.UUID.randomUUID() + ext;
String uploadLocation =
documentLoader.upload(
Map.of(), m.getMimeType(), m.getData(), uniqueLocation);
m.setLocation(uploadLocation);
m.setData(null);
});
}
/**
* Stores media from an InputStream, streaming directly to the DocumentLoader without buffering
* the full content in memory. Intended for large media files such as video.
*
* @param location Base storage location (e.g., file:///path/to/storage)
* @param mimeType MIME type of the media (e.g., "video/mp4")
* @param stream InputStream containing the media data
* @return The storage location where the media was written, or null if no loader was found
*/
public String storeMediaStream(String location, String mimeType, java.io.InputStream stream) {
Optional<DocumentLoader> docLoader =
documentLoaders.stream()
.filter(documentLoader -> documentLoader.supports(location))
.findFirst();
if (docLoader.isPresent()) {
String ext = getExtension(mimeType);
String uniqueLocation = location + "_" + java.util.UUID.randomUUID() + ext;
docLoader.get().upload(Map.of(), mimeType, stream, uniqueLocation);
return uniqueLocation;
}
return null;
}
private Map<String, String> getIntegrationNames(String toolCallName, List<ToolSpec> toolSpecs) {
Optional<ToolSpec> matched =
toolSpecs.stream()
.filter(toolSpec -> toolSpec.getName().equals(toolCallName))
.findFirst();
return matched.map(ToolSpec::getIntegrationNames).orElse(Collections.emptyMap());
}
/**
* Recursively parses JSON strings within a Map structure. This handles cases where the LLM
* generates nested JSON strings like "{\"value\":\"page\"}".
*
* @param input The input Map that may contain JSON strings
* @return A new Map with JSON strings parsed into their object representations
*/
@SuppressWarnings("unchecked")
@VisibleForTesting
Map<String, Object> parseNestedJsonStrings(Map<String, Object> input) {
if (input == null) {
return null;
}
Map<String, Object> result = new HashMap<>();
for (Map.Entry<String, Object> entry : input.entrySet()) {
String key = entry.getKey();
Object value = entry.getValue();
if (value instanceof String) {
String stringValue = (String) value;
if (isJsonString(stringValue)) {
try {
// Parse the JSON string into an object
Object parsedValue = objectMapper.readValue(stringValue, Object.class);
// Recursively parse any nested JSON strings in the parsed object
if (parsedValue instanceof Map) {
parsedValue = parseNestedJsonStrings((Map<String, Object>) parsedValue);
} else if (parsedValue instanceof List) {
parsedValue = parseNestedJsonStringsInList((List<Object>) parsedValue);
}
result.put(key, parsedValue);
} catch (Exception e) {
// If parsing fails, keep the original string value
log.debug(
"Failed to parse JSON string for key {}: {}", key, e.getMessage());
result.put(key, value);
}
} else {
result.put(key, value);
}
} else if (value instanceof Map) {
// Recursively parse nested Maps
result.put(key, parseNestedJsonStrings((Map<String, Object>) value));
} else if (value instanceof List) {
// Recursively parse nested Lists
result.put(key, parseNestedJsonStringsInList((List<Object>) value));
} else {
result.put(key, value);
}
}
return result;
}
/** Recursively parses JSON strings within a List structure. */
@SuppressWarnings("unchecked")
private List<Object> parseNestedJsonStringsInList(List<Object> input) {
if (input == null) {
return null;
}
List<Object> result = new ArrayList<>();
for (Object item : input) {
if (item instanceof String) {
String stringValue = (String) item;
if (isJsonString(stringValue)) {
try {
Object parsedValue = objectMapper.readValue(stringValue, Object.class);
if (parsedValue instanceof Map) {
parsedValue = parseNestedJsonStrings((Map<String, Object>) parsedValue);
} else if (parsedValue instanceof List) {
parsedValue = parseNestedJsonStringsInList((List<Object>) parsedValue);
}
result.add(parsedValue);
} catch (Exception e) {
log.debug("Failed to parse JSON string in list: {}", e.getMessage());
result.add(item);
}
} else {
result.add(item);
}
} else if (item instanceof Map) {
result.add(parseNestedJsonStrings((Map<String, Object>) item));
} else if (item instanceof List) {
result.add(parseNestedJsonStringsInList((List<Object>) item));
} else {
result.add(item);
}
}
return result;
}
/**
* Ensures the conversation ends with a user message. Some providers (e.g. Anthropic/Claude)
* reject requests where the last message has an assistant or tool_call role ("assistant message
* prefill"). This typically happens when the prior iteration ended with finishReason=MAX_TOKENS
* and the DO_WHILE loop continues with the partial assistant response as the last message in
* the history. Appending a user continuation prompt is safe for all providers — OpenAI and
* others simply treat it as the next user turn.
*
* @param messages The mutable list of messages to check and potentially modify
*/
@VisibleForTesting
void ensureLastMessageIsFromUser(List<Message> messages) {
if (messages.isEmpty()) return;
Message last = messages.getLast();
if (last instanceof UserMessage) return;
if (last instanceof AssistantMessage assistantMsg) {
// Replace trailing assistant message with a user message that includes
// the partial text as context + continuation instruction.
// This avoids the "assistant message prefill" error from Claude.
String partialText = assistantMsg.getText();
messages.removeLast();
String continuation =
partialText != null && !partialText.isBlank()
? "You were saying:\n\n"
+ partialText
+ "\n\nPlease continue where you left off."
: "Please continue where you left off.";
messages.add(new UserMessage(continuation));
} else {
// For any other non-user message type (tool_call, system, etc.)
messages.add(new UserMessage("Please continue where you left off."));
}
}
/** Checks if a string looks like JSON */
@VisibleForTesting
boolean isJsonString(String value) {
if (value == null || value.trim().isEmpty()) {
return false;
}
String trimmed = value.trim();
return (trimmed.startsWith("{") && trimmed.endsWith("}"))
|| (trimmed.startsWith("[") && trimmed.endsWith("]"));
}
}
@@ -0,0 +1,130 @@
/*
* Copyright 2025 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.conductoross.conductor.ai;
import java.util.List;
import java.util.Map;
import java.util.UUID;
import java.util.function.Consumer;
import org.conductoross.conductor.ai.document.DocumentLoader;
import org.conductoross.conductor.ai.model.AudioGenRequest;
import org.conductoross.conductor.ai.model.ChatCompletion;
import org.conductoross.conductor.ai.model.EmbeddingGenRequest;
import org.conductoross.conductor.ai.model.ImageGenRequest;
import org.conductoross.conductor.ai.model.LLMResponse;
import org.conductoross.conductor.ai.model.VideoGenRequest;
import org.conductoross.conductor.common.JsonSchemaValidator;
import org.conductoross.conductor.common.utils.StringTemplate;
import org.conductoross.conductor.config.AIIntegrationEnabledCondition;
import org.springframework.context.annotation.Conditional;
import org.springframework.stereotype.Component;
import com.netflix.conductor.common.metadata.tasks.Task;
import lombok.extern.slf4j.Slf4j;
import okhttp3.OkHttpClient;
@Component
@Conditional(AIIntegrationEnabledCondition.class)
@Slf4j
public class LLMs {
protected AIModelProvider modelProvider;
protected LLMHelper helper;
protected String payloadStoreLocation;
protected Consumer<TokenUsageLog> tokenUsageLogger =
(tokenUsageLog) -> {
log.info("{}", tokenUsageLog);
};
public LLMs(
List<DocumentLoader> documentLoaders,
JsonSchemaValidator jsonSchemaValidator,
AIModelProvider modelProvider,
OkHttpClient conductorAiHttpClient) {
this.modelProvider = modelProvider;
this.helper = new LLMHelper(jsonSchemaValidator, documentLoaders, conductorAiHttpClient);
this.payloadStoreLocation = modelProvider.getPayloadStoreLocation();
}
public LLMResponse chatComplete(Task task, ChatCompletion chatCompletion) {
AIModel llm = this.modelProvider.getModel(chatCompletion);
String prompt =
replacePromptVariables(
task,
chatCompletion.getInstructions(),
chatCompletion.getPromptVariables());
chatCompletion.setInstructions(prompt);
return helper.chatComplete(
task, llm, chatCompletion, getPayloadStoreLocation(task), tokenUsageLogger);
}
public LLMResponse generateImage(Task task, ImageGenRequest imageGenRequest) {
AIModel llm = this.modelProvider.getModel(imageGenRequest);
String prompt =
replacePromptVariables(
task, imageGenRequest.getPrompt(), imageGenRequest.getPromptVariables());
imageGenRequest.setPrompt(prompt);
return helper.generateImage(
task, llm, imageGenRequest, getPayloadStoreLocation(task), tokenUsageLogger);
}
public LLMResponse generateAudio(Task task, AudioGenRequest audioGenRequest) {
AIModel llm = this.modelProvider.getModel(audioGenRequest);
String prompt =
replacePromptVariables(
task, audioGenRequest.getPrompt(), audioGenRequest.getPromptVariables());
audioGenRequest.setPrompt(prompt);
return helper.generateAudio(
task, llm, audioGenRequest, getPayloadStoreLocation(task), tokenUsageLogger);
}
public List<Float> generateEmbeddings(Task task, EmbeddingGenRequest embeddingGenRequest) {
AIModel llm = this.modelProvider.getModel(embeddingGenRequest);
return helper.generateEmbeddings(task, llm, embeddingGenRequest, tokenUsageLogger);
}
public LLMResponse generateVideo(Task task, VideoGenRequest videoGenRequest) {
AIModel llm = this.modelProvider.getModel(videoGenRequest);
String prompt =
replacePromptVariables(
task, videoGenRequest.getPrompt(), videoGenRequest.getPromptVariables());
videoGenRequest.setPrompt(prompt);
return helper.generateVideo(
task, llm, videoGenRequest, getPayloadStoreLocation(task), tokenUsageLogger);
}
public LLMResponse checkVideoStatus(Task task, VideoGenRequest videoGenRequest) {
AIModel llm = this.modelProvider.getModel(videoGenRequest);
return helper.checkVideoStatus(task, llm, videoGenRequest, getPayloadStoreLocation(task));
}
public String replacePromptVariables(
Task task, String prompt, Map<String, Object> paramReplacement) {
if (paramReplacement != null) {
prompt = StringTemplate.fString(prompt, paramReplacement);
}
return prompt;
}
public String getPayloadStoreLocation(Task task) {
return payloadStoreLocation
+ "/"
+ task.getWorkflowInstanceId()
+ "/"
+ task.getTaskId()
+ "/"
+ UUID.randomUUID();
}
}
@@ -0,0 +1,179 @@
/*
* Copyright 2025 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.conductoross.conductor.ai;
import java.util.HashMap;
import java.util.Map;
public class MimeExtensionResolver {
private static final Map<String, String> mimeToExt = new HashMap<>();
private static final Map<String, String> extToMime = new HashMap<>();
static {
// ----- IMAGE -----
mimeToExt.put("image/jpeg", ".jpg");
mimeToExt.put("image/jpg", ".jpg");
mimeToExt.put("image/png", ".png");
mimeToExt.put("image/gif", ".gif");
mimeToExt.put("image/bmp", ".bmp");
mimeToExt.put("image/webp", ".webp");
mimeToExt.put("image/tiff", ".tiff");
mimeToExt.put("image/svg+xml", ".svg");
mimeToExt.put("image/x-icon", ".ico");
mimeToExt.put("image/heif", ".heif");
mimeToExt.put("image/heic", ".heic");
// ----- AUDIO -----
mimeToExt.put("audio/mpeg", ".mp3");
mimeToExt.put("audio/wav", ".wav");
mimeToExt.put("audio/x-wav", ".wav");
mimeToExt.put("audio/ogg", ".ogg");
mimeToExt.put("audio/flac", ".flac");
mimeToExt.put("audio/aac", ".aac");
mimeToExt.put("audio/mp4", ".m4a");
mimeToExt.put("audio/opus", ".opus");
mimeToExt.put("audio/webm", ".weba");
mimeToExt.put("audio/amr", ".amr");
// ----- VIDEO -----
mimeToExt.put("video/mp4", ".mp4");
mimeToExt.put("video/mpeg", ".mpeg");
mimeToExt.put("video/x-msvideo", ".avi");
mimeToExt.put("video/x-ms-wmv", ".wmv");
mimeToExt.put("video/quicktime", ".mov");
mimeToExt.put("video/webm", ".webm");
mimeToExt.put("video/3gpp", ".3gp");
mimeToExt.put("video/3gpp2", ".3g2");
mimeToExt.put("video/x-flv", ".flv");
mimeToExt.put("video/x-matroska", ".mkv");
// ----- DOCUMENTS -----
mimeToExt.put("application/pdf", ".pdf");
mimeToExt.put("application/msword", ".doc");
mimeToExt.put(
"application/vnd.openxmlformats-officedocument.wordprocessingml.document", ".docx");
mimeToExt.put("application/vnd.ms-excel", ".xls");
mimeToExt.put("application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", ".xlsx");
mimeToExt.put("application/vnd.ms-powerpoint", ".ppt");
mimeToExt.put(
"application/vnd.openxmlformats-officedocument.presentationml.presentation",
".pptx");
mimeToExt.put("application/rtf", ".rtf");
mimeToExt.put("application/zip", ".zip");
mimeToExt.put("application/x-7z-compressed", ".7z");
mimeToExt.put("application/x-rar-compressed", ".rar");
mimeToExt.put("application/json", ".json");
mimeToExt.put("application/xml", ".xml");
// ----- TEXT / CODE -----
mimeToExt.put("text/plain", ".txt");
mimeToExt.put("text/html", ".html");
mimeToExt.put("text/css", ".css");
mimeToExt.put("text/csv", ".csv");
mimeToExt.put("text/javascript", ".js");
// ----- Reverse mapping (ext → mime) -----
for (Map.Entry<String, String> e : mimeToExt.entrySet()) {
String ext = e.getValue().replaceFirst("^\\.", "");
extToMime.put(ext, e.getKey());
}
// aliases
extToMime.put("jpg", "image/jpeg");
extToMime.put("jpeg", "image/jpeg");
extToMime.put("htm", "text/html");
}
public static String getExtension(String input) {
if (input == null || input.isEmpty()) return "";
input = input.trim().toLowerCase();
// Case 1: Input looks like extension
if (!input.contains("/") && !input.contains("*")) {
if (input.startsWith(".")) input = input.substring(1);
String mime = extToMime.get(input);
if (mime != null) return mimeToExt.get(mime);
return "." + input; // fallback
}
// Case 2: Wildcard MIME
if (input.endsWith("/*")) {
String type = input.substring(0, input.indexOf('/'));
switch (type) {
case "image":
return ".jpg";
case "audio":
return ".mp3";
case "video":
return ".mp4";
case "text":
return ".txt";
case "application":
return ".bin";
default:
return "";
}
}
// Case 3: Exact MIME
return mimeToExt.getOrDefault(input, "");
}
public static String getMimeType(String ext) {
if (ext == null || ext.isEmpty()) return "";
if (ext.startsWith(".")) ext = ext.substring(1);
return extToMime.getOrDefault(ext.toLowerCase(), "application/octet-stream");
}
/**
* Attempts to detect the MIME type from a URL by examining its path for known file extensions.
* Useful when downloading media from external URLs where the Content-Type header may not be
* reliable.
*
* @param url The URL to examine
* @param defaultMimeType The default MIME type to return if no extension is detected
* @return The detected MIME type, or the default if detection fails
*/
public static String getMimeTypeFromUrl(String url, String defaultMimeType) {
if (url == null || url.isEmpty()) {
return defaultMimeType;
}
// Remove query string and fragment
String path = url;
int queryIdx = path.indexOf('?');
if (queryIdx > 0) {
path = path.substring(0, queryIdx);
}
int fragIdx = path.indexOf('#');
if (fragIdx > 0) {
path = path.substring(0, fragIdx);
}
// Find the last dot in the path
int lastDot = path.lastIndexOf('.');
if (lastDot > 0 && lastDot < path.length() - 1) {
// Extract extension (up to 5 chars to avoid false positives)
String ext = path.substring(lastDot + 1).toLowerCase();
if (ext.length() <= 5) {
String mimeType = extToMime.get(ext);
if (mimeType != null) {
return mimeType;
}
}
}
return defaultMimeType;
}
}
@@ -0,0 +1,27 @@
/*
* Copyright 2025 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.conductoross.conductor.ai;
import okhttp3.OkHttpClient;
/**
* Configuration for an {@link AIModel}. Implementations build the model via {@link #get()} and
* receive the shared {@link OkHttpClient} (typically Spring's {@code conductorAiHttpClient}) via
* {@link #setHttpClient(OkHttpClient)}. Providers that do not use OkHttp (e.g. Bedrock) may
* implement {@link #setHttpClient(OkHttpClient)} as a no-op.
*/
public interface ModelConfiguration<T extends AIModel> {
T get();
void setHttpClient(OkHttpClient httpClient);
}
@@ -0,0 +1,209 @@
/*
* Copyright 2026 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.conductoross.conductor.ai.a2a;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.util.Map;
import org.apache.commons.lang3.StringUtils;
import org.conductoross.conductor.ai.a2a.model.A2ATask;
import org.conductoross.conductor.ai.a2a.model.TaskState;
import org.conductoross.conductor.ai.model.A2ACallRequest;
import org.conductoross.conductor.config.AIIntegrationEnabledCondition;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.context.annotation.Conditional;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestHeader;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import com.netflix.conductor.common.config.ObjectMapperProvider;
import com.netflix.conductor.common.metadata.tasks.Task;
import com.netflix.conductor.common.metadata.tasks.TaskResult;
import com.netflix.conductor.service.TaskService;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
/**
* Receives A2A push notifications from remote agents and completes the corresponding {@code AGENT}
* task that is waiting in push mode.
*
* <p>Token is read from the {@code Authorization: Bearer <token>} header (preferred), then the
* {@code X-Conductor-A2A-Token} custom header. Comparison is constant-time. Tokens embed an expiry
* timestamp ({@code {uuid}:{epochMillis}}) and are rejected once expired.
*/
@RestController
@RequestMapping("/api/a2a")
@Conditional(AIIntegrationEnabledCondition.class)
public class A2ACallbackResource {
private static final Logger log = LoggerFactory.getLogger(A2ACallbackResource.class);
private final ObjectMapper objectMapper = new ObjectMapperProvider().getObjectMapper();
private final TaskService taskService;
private final A2AService a2aService;
public A2ACallbackResource(TaskService taskService, A2AService a2aService) {
this.taskService = taskService;
this.a2aService = a2aService;
}
@PostMapping("/callback/{taskId}")
public ResponseEntity<Void> onPushNotification(
@PathVariable("taskId") String taskId,
@RequestHeader(value = "Authorization", required = false) String authHeader,
@RequestHeader(value = "X-Conductor-A2A-Token", required = false) String customHeader,
@RequestBody(required = false) JsonNode payload) {
try (A2ALogging.Scope scope = A2ALogging.of(A2ALogging.TASK_ID, taskId)) {
String token = resolveToken(authHeader, customHeader);
Task task = loadTask(taskId);
if (task == null || !AgentTask.TASK_TYPE.equals(task.getTaskType())) {
return ResponseEntity.notFound().build();
}
scope.add(A2ALogging.WORKFLOW_ID, task.getWorkflowInstanceId())
.add(A2ALogging.REF, task.getReferenceTaskName());
Object storedToken =
task.getOutputData() != null
? task.getOutputData().get(A2AResults.KEY_PUSH_TOKEN)
: null;
if (!isValidToken(storedToken, token)) {
log.warn("A2A push for task {} rejected: token mismatch or expired", taskId);
return ResponseEntity.status(HttpStatus.FORBIDDEN).build();
}
if (task.getStatus() != Task.Status.IN_PROGRESS) {
// Already completed (or not waiting) — nothing to do; ack so the agent stops
// retrying.
return ResponseEntity.ok().build();
}
A2ACallRequest request =
objectMapper.convertValue(task.getInputData(), A2ACallRequest.class);
String agentTaskId = asString(task.getOutputData().get(A2AResults.KEY_TASK_ID));
if (StringUtils.isBlank(request.getAgentUrl()) || StringUtils.isBlank(agentTaskId)) {
return ResponseEntity.ok().build();
}
A2ATask agentTask;
try {
agentTask =
a2aService.getTask(
request.getAgentUrl(),
agentTaskId,
request.getHistoryLength(),
request.getHeaders());
} catch (Exception e) {
log.warn(
"A2A push for task {}: failed to fetch agent task {} from {}: {}",
taskId,
agentTaskId,
request.getAgentUrl(),
e.getMessage());
return ResponseEntity.status(HttpStatus.BAD_GATEWAY).build();
}
String state = agentTask.getStatus() != null ? agentTask.getStatus().getState() : null;
if (!TaskState.isTerminal(state) && !TaskState.isInterrupted(state)) {
// Not done yet — ack and wait for the next push.
return ResponseEntity.ok().build();
}
Map<String, Object> output = A2AResults.taskOutput(agentTask, objectMapper);
taskService.updateTask(
task.getWorkflowInstanceId(),
task.getReferenceTaskName(),
toResultStatus(state),
"a2a-callback",
output);
log.debug("A2A push completed task {} (agent state {})", taskId, state);
return ResponseEntity.ok().build();
}
}
/** Resolves the token: {@code Authorization: Bearer} header wins, then the custom header. */
private String resolveToken(String authHeader, String customHeader) {
if (authHeader != null && authHeader.startsWith("Bearer ")) {
return authHeader.substring(7).trim();
}
if (customHeader != null && !customHeader.isBlank()) {
return customHeader.trim();
}
return null;
}
/**
* Validates the inbound token against the stored one using constant-time comparison and checks
* the embedded expiry timestamp. Token format: {@code {uuid}:{expiryEpochMillis}}.
*/
private boolean isValidToken(Object stored, String inbound) {
if (stored == null || inbound == null) {
return false;
}
String storedStr = stored.toString();
// Constant-time comparison — prevents timing oracle on the UUID portion.
boolean match =
MessageDigest.isEqual(
storedStr.getBytes(StandardCharsets.UTF_8),
inbound.getBytes(StandardCharsets.UTF_8));
if (!match) {
return false;
}
// Extract and validate the expiry timestamp embedded in the token.
int sep = storedStr.lastIndexOf(':');
if (sep > 0) {
try {
long expiryMs = Long.parseLong(storedStr.substring(sep + 1));
if (System.currentTimeMillis() > expiryMs) {
log.warn("A2A push token is expired (expiry was {})", expiryMs);
return false;
}
} catch (NumberFormatException ignored) {
// Token pre-dates the timestamped format — accept it as valid (no expiry check).
}
}
return true;
}
private Task loadTask(String taskId) {
try {
return taskService.getTask(taskId);
} catch (Exception e) {
return null;
}
}
private TaskResult.Status toResultStatus(String state) {
switch (TaskState.normalize(state)) {
case TaskState.FAILED:
case TaskState.REJECTED:
return TaskResult.Status.FAILED;
default:
// completed / canceled / input-required / auth-required
return TaskResult.Status.COMPLETED;
}
}
private static String asString(Object value) {
return value == null ? null : value.toString();
}
}
@@ -0,0 +1,30 @@
/*
* Copyright 2026 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.conductoross.conductor.ai.a2a;
/**
* A retryable failure talking to a remote A2A agent (transport error, 5xx, or a transient JSON-RPC
* error). Terminal/non-retryable failures (4xx, method-not-found, task-not-found, …) are surfaced
* as {@link com.netflix.conductor.sdk.workflow.executor.task.NonRetryableException} instead, so
* both the annotated-worker path and {@code AgentTask} map them to a terminal task status.
*/
public class A2AException extends RuntimeException {
public A2AException(String message) {
super(message);
}
public A2AException(String message, Throwable cause) {
super(message, cause);
}
}
@@ -0,0 +1,75 @@
/*
* Copyright 2026 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.conductoross.conductor.ai.a2a;
import java.util.ArrayList;
import java.util.List;
import org.slf4j.MDC;
/**
* MDC correlation keys for A2A code paths. A {@link Scope} sets a batch of keys and removes exactly
* those it set on close, so log lines emitted while calling (or being called by) a remote agent can
* be grepped by workflow, task, remote task, or context id — and no key leaks onto the next task to
* run on a pooled worker thread.
*
* <p>Usage:
*
* <pre>{@code
* try (A2ALogging.Scope scope = A2ALogging.of(A2ALogging.WORKFLOW_ID, wfId, A2ALogging.TASK_ID, id)) {
* scope.add(A2ALogging.REMOTE_TASK_ID, agentTaskId); // once it's known
* ...
* }
* }</pre>
*/
public final class A2ALogging {
public static final String WORKFLOW_ID = "a2aWorkflowId";
public static final String TASK_ID = "a2aTaskId";
public static final String REF = "a2aRef";
public static final String REMOTE_TASK_ID = "a2aRemoteTaskId";
public static final String CONTEXT_ID = "a2aContextId";
public static final String MESSAGE_ID = "a2aMessageId";
public static final String AGENT = "a2aAgent";
public static final String METHOD = "a2aMethod";
private A2ALogging() {}
/** Open a scope pre-loaded with alternating key/value pairs (null keys/values are skipped). */
public static Scope of(String... kv) {
Scope scope = new Scope();
for (int i = 0; i + 1 < kv.length; i += 2) {
scope.add(kv[i], kv[i + 1]);
}
return scope;
}
/** An auto-closeable set of MDC keys; {@link #close()} removes only the keys this scope set. */
public static final class Scope implements AutoCloseable {
private final List<String> keys = new ArrayList<>();
/** Set an MDC key (no-op if key or value is null); removed on {@link #close()}. */
public Scope add(String key, String value) {
if (key != null && value != null) {
MDC.put(key, value);
keys.add(key);
}
return this;
}
@Override
public void close() {
keys.forEach(MDC::remove);
}
}
}
@@ -0,0 +1,81 @@
/*
* Copyright 2026 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.conductoross.conductor.ai.a2a;
import com.netflix.conductor.metrics.Monitors;
/**
* A2A metric emission, centralized so names and tags stay consistent and low-cardinality.
*
* <p>Counters are published through the shared {@link Monitors} registry (the same one the rest of
* the engine uses), so they show up alongside core Conductor metrics in whatever backend is wired
* in (Prometheus, Datadog, …). Tag values are always drawn from bounded sets — never agent URLs,
* workflow ids, or message ids — to avoid cardinality blow-ups.
*
* <table>
* <caption>Metrics</caption>
* <tr><th>Name</th><th>Tags</th><th>Meaning</th></tr>
* <tr><td>{@code a2a_client_calls}</td><td>{@code result}</td>
* <td>An AGENT task reached a terminal status (result = the Conductor status).</td></tr>
* <tr><td>{@code a2a_client_poll_failures}</td><td>—</td>
* <td>A single transient {@code tasks/get} poll failed (before exhausting retries).</td></tr>
* <tr><td>{@code a2a_rpc_errors}</td><td>{@code method}, {@code terminal}</td>
* <td>A JSON-RPC/HTTP error from a remote agent, by method and whether it was fatal.</td></tr>
* <tr><td>{@code a2a_ssrf_blocked}</td><td>—</td>
* <td>An outbound agent URL was rejected by the SSRF guard.</td></tr>
* <tr><td>{@code a2a_server_requests}</td><td>{@code method}</td>
* <td>An inbound A2A JSON-RPC request to a workflow-backed agent, by method.</td></tr>
* <tr><td>{@code a2a_server_resumes}</td><td>—</td>
* <td>A follow-up message/send resumed a paused workflow (multi-turn).</td></tr>
* </table>
*/
public final class A2AMetrics {
private A2AMetrics() {}
// ---- client (Conductor calling a remote agent) ------------------------------------------
/** An AGENT task settled into a terminal Conductor status (e.g. completed/failed). */
public static void clientCall(String result) {
Monitors.getCounter("a2a_client_calls", "result", result).increment();
}
/** One transient poll failure during a {@code tasks/get} loop (retry not yet exhausted). */
public static void clientPollFailure() {
Monitors.getCounter("a2a_client_poll_failures").increment();
}
/** A remote agent returned a JSON-RPC or HTTP error for the given method. */
public static void rpcError(String method, boolean terminal) {
Monitors.getCounter(
"a2a_rpc_errors", "method", method, "terminal", String.valueOf(terminal))
.increment();
}
/** An outbound agent URL was rejected by the SSRF guard. */
public static void ssrfBlocked() {
Monitors.getCounter("a2a_ssrf_blocked").increment();
}
// ---- server (Conductor exposed as an agent) ---------------------------------------------
/** An inbound A2A JSON-RPC request was dispatched for the given method. */
public static void serverRequest(String method) {
Monitors.getCounter("a2a_server_requests", "method", method).increment();
}
/** A follow-up message/send resumed a paused (input-required) workflow. */
public static void serverResume() {
Monitors.getCounter("a2a_server_resumes").increment();
}
}
@@ -0,0 +1,124 @@
/*
* Copyright 2026 Conductor Authors.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.conductoross.conductor.ai.a2a;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.conductoross.conductor.ai.a2a.model.A2AMessage;
import org.conductoross.conductor.ai.a2a.model.A2ATask;
import org.conductoross.conductor.ai.a2a.model.Artifact;
import org.conductoross.conductor.ai.a2a.model.Part;
import org.conductoross.conductor.ai.a2a.model.TaskState;
import com.fasterxml.jackson.databind.ObjectMapper;
/**
* Builds the task output that {@code AGENT} (polling/streaming) and the push-notification callback
* both surface, so the shape is identical regardless of how the result arrived.
*/
public final class A2AResults {
public static final String KEY_STATE = "state";
public static final String KEY_TASK_ID = "taskId";
public static final String KEY_CONTEXT_ID = "contextId";
public static final String KEY_ARTIFACTS = "artifacts";
public static final String KEY_TEXT = "text";
public static final String KEY_AGENT_MESSAGE = "agentMessage";
public static final String KEY_TASK = "task";
public static final String KEY_PUSH_TOKEN = "pushToken";
/** Bookkeeping (also surfaced for operator visibility): when the A2A call started. */
public static final String KEY_STARTED_AT = "a2aStartedAt";
/** Bookkeeping: consecutive transient poll failures so far. */
public static final String KEY_POLL_FAILURES = "a2aPollFailures";
private A2AResults() {}
/** Output for a remote {@link A2ATask}: normalized state, ids, artifacts, text, full task. */
public static Map<String, Object> taskOutput(A2ATask task, ObjectMapper objectMapper) {
Map<String, Object> output = new HashMap<>();
output.put(KEY_STATE, TaskState.normalize(stateOf(task)));
output.put(KEY_TASK_ID, task.getId());
output.put(KEY_CONTEXT_ID, task.getContextId());
if (task.getArtifacts() != null) {
output.put(KEY_ARTIFACTS, objectMapper.convertValue(task.getArtifacts(), List.class));
}
output.put(KEY_TASK, objectMapper.convertValue(task, Map.class));
String text = extractText(task);
if (text != null) {
output.put(KEY_TEXT, text);
}
A2AMessage statusMessage = task.getStatus() != null ? task.getStatus().getMessage() : null;
if (statusMessage != null) {
output.put(KEY_AGENT_MESSAGE, objectMapper.convertValue(statusMessage, Map.class));
}
return output;
}
/** Output for a direct {@link A2AMessage} reply (no task was created). */
public static Map<String, Object> messageOutput(A2AMessage message, ObjectMapper objectMapper) {
Map<String, Object> output = new HashMap<>();
output.put(KEY_STATE, "message");
if (message != null) {
output.put(KEY_CONTEXT_ID, message.getContextId());
output.put(KEY_TASK_ID, message.getTaskId());
output.put(KEY_AGENT_MESSAGE, objectMapper.convertValue(message, Map.class));
String text = partsText(message.getParts());
if (text != null) {
output.put(KEY_TEXT, text);
}
}
return output;
}
/** Concatenates the text of a task's artifact parts, falling back to its status message. */
public static String extractText(A2ATask task) {
StringBuilder sb = new StringBuilder();
if (task.getArtifacts() != null) {
for (Artifact artifact : task.getArtifacts()) {
appendParts(sb, artifact.getParts());
}
}
if (sb.length() == 0 && task.getStatus() != null && task.getStatus().getMessage() != null) {
appendParts(sb, task.getStatus().getMessage().getParts());
}
return sb.length() == 0 ? null : sb.toString();
}
static String partsText(List<Part> parts) {
StringBuilder sb = new StringBuilder();
appendParts(sb, parts);
return sb.length() == 0 ? null : sb.toString();
}
private static void appendParts(StringBuilder sb, List<Part> parts) {
if (parts == null) {
return;
}
for (Part part : parts) {
if (part != null && part.getText() != null) {
if (sb.length() > 0) {
sb.append("\n");
}
sb.append(part.getText());
}
}
}
static String stateOf(A2ATask task) {
return task.getStatus() != null ? task.getStatus().getState() : null;
}
}

Some files were not shown because too many files have changed in this diff Show More