chore: import upstream snapshot with attribution
CI / unit-test (push) Has been cancelled
CI / detect-changes (push) Has been cancelled
CI / build (push) Has been cancelled
Publish docs via GitHub Pages / Deploy docs (push) Has been cancelled
CI / test-harness (push) Has been cancelled
CI / generate-e2e-matrix (push) Has been cancelled
CI / e2e (push) Has been cancelled
CI / build-ui (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
UI v2 Integration CI / E2E (Integration) (push) Has been cancelled
UI v2 CI / Lint, Format & Test (push) Has been cancelled
UI v2 CI / E2E (Mocked) (push) Has been cancelled
CI / unit-test (push) Has been cancelled
CI / detect-changes (push) Has been cancelled
CI / build (push) Has been cancelled
Publish docs via GitHub Pages / Deploy docs (push) Has been cancelled
CI / test-harness (push) Has been cancelled
CI / generate-e2e-matrix (push) Has been cancelled
CI / e2e (push) Has been cancelled
CI / build-ui (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
UI v2 Integration CI / E2E (Integration) (push) Has been cancelled
UI v2 CI / Lint, Format & Test (push) Has been cancelled
UI v2 CI / E2E (Mocked) (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,68 @@
|
||||
---
|
||||
name: docs-writer
|
||||
description: Technical documentation specialist for Conductor workflow orchestration features. Creates clear, comprehensive documentation for APIs, workflows, tasks, and system architecture.
|
||||
tools: Read, Grep, Glob, Write, Edit, Bash
|
||||
model: inherit
|
||||
---
|
||||
|
||||
You are a technical documentation specialist for Conductor, an open-source workflow orchestration engine built at Netflix.
|
||||
|
||||
## Your Role
|
||||
|
||||
Create clear, comprehensive, and accurate documentation for Conductor features, including:
|
||||
- Workflow definitions and task types
|
||||
- REST API endpoints and payloads
|
||||
- System architecture and components
|
||||
- Configuration options and database integrations
|
||||
- SDK usage examples (Java, Python, JavaScript, Go, C#)
|
||||
- Developer guides and tutorials
|
||||
|
||||
## Documentation Process
|
||||
|
||||
1. **Understand the Feature**
|
||||
- Read relevant source code to understand implementation
|
||||
- Identify key classes, methods, and APIs
|
||||
- Test functionality if possible
|
||||
- Review existing related documentation
|
||||
|
||||
2. **Structure Documentation**
|
||||
- Start with a clear overview/summary
|
||||
- Include purpose and use cases
|
||||
- Provide syntax and parameters
|
||||
- Add practical examples
|
||||
- Document edge cases and limitations
|
||||
- Link to related documentation
|
||||
|
||||
3. **Follow Conductor Style**
|
||||
- Use clear, concise language
|
||||
- Include code examples in relevant languages
|
||||
- Use Markdown formatting consistently
|
||||
- Add diagrams or JSON examples for workflows
|
||||
- Follow existing documentation patterns in `/docs`
|
||||
|
||||
4. **Quality Standards**
|
||||
- Ensure technical accuracy
|
||||
- Test all code examples
|
||||
- Use proper terminology (workflows, tasks, workers, etc.)
|
||||
- Include error handling examples
|
||||
- Add troubleshooting sections when relevant
|
||||
|
||||
## Key Conductor Concepts to Reference
|
||||
|
||||
- **Workflows**: JSON-based orchestration definitions
|
||||
- **Tasks**: Units of work (HTTP, Lambda, Sub-workflow, etc.)
|
||||
- **Workers**: Services that execute tasks
|
||||
- **Task Definitions**: Reusable task configurations
|
||||
- **System Tasks**: Built-in task types
|
||||
- **Event Handlers**: Trigger workflows from events
|
||||
|
||||
## Output Format
|
||||
|
||||
Provide documentation in Markdown format suitable for the `/docs` directory, with:
|
||||
- Clear headings and sections
|
||||
- Code blocks with proper syntax highlighting
|
||||
- Tables for parameters and options
|
||||
- Links to related documentation
|
||||
- Version information when relevant
|
||||
|
||||
Always prioritize clarity and practical usefulness for developers using Conductor.
|
||||
@@ -0,0 +1,47 @@
|
||||
# Git/VCS
|
||||
.git
|
||||
.gitignore
|
||||
.gitattributes
|
||||
.github
|
||||
|
||||
# IDE/editor
|
||||
.idea
|
||||
.vscode
|
||||
.classpath
|
||||
.project
|
||||
.settings
|
||||
*.iml
|
||||
|
||||
# OS/filesystem noise
|
||||
.DS_Store
|
||||
|
||||
# Caches & temp
|
||||
**/.gradle
|
||||
**/.cache
|
||||
**/tmp
|
||||
**/logs
|
||||
**/*.log
|
||||
|
||||
# Build outputs (keep source in docker build)
|
||||
**/build
|
||||
!ui/build
|
||||
**/out
|
||||
**/target
|
||||
**/dist
|
||||
!ui-next/dist
|
||||
**/coverage
|
||||
|
||||
# Python
|
||||
venv
|
||||
**/__pycache__/
|
||||
**/.pytest_cache/
|
||||
|
||||
# JS tooling
|
||||
**/node_modules
|
||||
**/.npm
|
||||
**/.yarn
|
||||
**/.pnpm-store
|
||||
**/.eslintcache
|
||||
**/.parcel-cache
|
||||
**/.next
|
||||
|
||||
@@ -0,0 +1,11 @@
|
||||
gradlew eol=lf
|
||||
*.gradle eol=lf
|
||||
*.java eol=lf
|
||||
*.groovy eol=lf
|
||||
spring.factories eol=lf
|
||||
*.sh eol=lf
|
||||
|
||||
docs/* linguist-documentation
|
||||
server/src/main/resources/swagger-ui/* linguist-vendored
|
||||
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
---
|
||||
name: Bug report
|
||||
about: Create a report to help us improve
|
||||
title: ""
|
||||
labels: 'type: bug'
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
**Describe the bug**
|
||||
A clear and concise description of what the bug is.
|
||||
|
||||
**Details**
|
||||
Conductor version:
|
||||
Persistence implementation: Cassandra, Postgres, MySQL, Dynomite etc
|
||||
Queue implementation: Postgres, MySQL, Dynoqueues etc
|
||||
Lock: Redis or Zookeeper?
|
||||
Workflow definition:
|
||||
Task definition:
|
||||
Event handler definition:
|
||||
|
||||
|
||||
**To Reproduce**
|
||||
Steps to reproduce the behavior:
|
||||
1. Go to '...'
|
||||
2. Click on '....'
|
||||
3. Scroll down to '....'
|
||||
4. See error
|
||||
|
||||
**Code to reproduce (paste relevant snippet):**
|
||||
```
|
||||
[paste the minimal code that triggers the bug]
|
||||
```
|
||||
|
||||
**Expected behavior**
|
||||
A clear and concise description of what you expected to happen.
|
||||
|
||||
**Screenshots**
|
||||
If applicable, add screenshots to help explain your problem.
|
||||
|
||||
**Additional context**
|
||||
Add any other context about the problem here.
|
||||
@@ -0,0 +1,43 @@
|
||||
name: Bug Report
|
||||
description: Create a report to help us reproduce and fix the bug
|
||||
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: >
|
||||
#### Before submitting a bug, please make sure the issue hasn't been already addressed by searching through [the existing and past issues](https://github.com/conductor-oss/conductor/issues?q=is%3Aissue%20label%3Abug).
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: Describe the bug
|
||||
description: |
|
||||
Please provide a clear and concise description of what the bug is.
|
||||
|
||||
If relevant, add a minimal example so that we can reproduce the error by running the code. It is very important for the snippet to be as succinct (minimal) as possible, so please take time to trim down any irrelevant code to help us debug efficiently.
|
||||
Your example should be fully self-contained and not rely on any artifact that should be downloaded.
|
||||
For example:
|
||||
|
||||
```
|
||||
# A succinct reproducing example trimmed down to the essential parts
|
||||
# (any language is fine — Java, Python, Go, JavaScript, etc.)
|
||||
```
|
||||
|
||||
If the code is too long (hopefully, it isn't), feel free to put it in a public gist and link it in the issue: https://gist.github.com.
|
||||
|
||||
Please also paste or describe the results you observe instead of the expected results. If you observe an error, please paste the error message including the **full** traceback of the exception. It may be relevant to wrap error messages in ```` ```triple quotes blocks``` ````.
|
||||
|
||||
placeholder: |
|
||||
A clear and concise description of what the bug is.
|
||||
|
||||
```
|
||||
# Sample code to reproduce the problem (any language)
|
||||
```
|
||||
|
||||
```
|
||||
The error message you got, with the full traceback.
|
||||
```
|
||||
validations:
|
||||
required: true
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: >
|
||||
Thanks for contributing!
|
||||
@@ -0,0 +1,5 @@
|
||||
blank_issues_enabled: true
|
||||
contact_links:
|
||||
- name: Questions
|
||||
url: https://orkes-conductor.slack.com/join/shared_invite/zt-2vdbx239s-Eacdyqya9giNLHfrCavfaA#/shared-invite/email
|
||||
about: Ask questions and discuss with other Conductor community members
|
||||
@@ -0,0 +1,12 @@
|
||||
---
|
||||
name: Documentation
|
||||
about: Something in the documentation that needs improvement
|
||||
title: "[DOC]: "
|
||||
labels: 'type: docs'
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
## What are you missing in the docs
|
||||
|
||||
## Proposed text
|
||||
@@ -0,0 +1,24 @@
|
||||
name: Documentation
|
||||
description: Report an issue related to https://docs.conductor-oss.org/index.html
|
||||
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: >
|
||||
#### Note: Please report your documentation issue in English to ensure it can be understood and addressed by the development team.
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: The doc issue
|
||||
description: >
|
||||
A clear and concise description of what content on https://docs.conductor-oss.org/index.html is an issue.
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: Suggest a potential alternative/fix
|
||||
description: >
|
||||
Tell us how we could improve the documentation in this regard.
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: >
|
||||
Thanks for contributing!
|
||||
@@ -0,0 +1,20 @@
|
||||
---
|
||||
name: Feature request
|
||||
about: Propose a new feature
|
||||
title: "[FEATURE]: "
|
||||
labels: 'type: feature'
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
Please read our [contributor guide](https://github.com/conductor-oss/conductor/blob/main/CONTRIBUTING.md) before creating an issue.
|
||||
Also consider discussing your idea on the [discussion forum](https://github.com/conductor-oss/conductor/discussions) first.
|
||||
|
||||
## Describe the Feature Request
|
||||
_A clear and concise description of what the feature request is._
|
||||
|
||||
## Describe Preferred Solution
|
||||
_A clear and concise description of what you want to happen._
|
||||
|
||||
## Describe Alternatives
|
||||
_A clear and concise description of any alternative solutions or features you've considered._
|
||||
@@ -0,0 +1,29 @@
|
||||
name: Feature request
|
||||
description: Submit a proposal/request for a new Conductor feature
|
||||
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: >
|
||||
#### Note: Please write your feature request in English to ensure it can be understood and addressed by the development team.
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: The feature, motivation and pitch
|
||||
description: >
|
||||
A clear and concise description of the feature proposal. Please outline the motivation for the proposal. Is your feature request related to a specific problem? e.g., *"I'm working on X and would like Y to be possible"*. If this is related to another GitHub issue, please link here too.
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: Alternatives
|
||||
description: >
|
||||
A description of any alternative solutions or features you've considered, if any.
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: Additional context
|
||||
description: >
|
||||
Add any other context or screenshots about the feature request.
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: >
|
||||
Thanks for contributing!
|
||||
@@ -0,0 +1,21 @@
|
||||
Pull Request type
|
||||
----
|
||||
- [ ] Bugfix
|
||||
- [ ] Feature
|
||||
- [ ] Refactoring (no functional changes, no api changes)
|
||||
- [ ] Build related changes
|
||||
- [ ] WHOSUSING.md
|
||||
- [ ] Other (please describe):
|
||||
|
||||
**NOTE**: Please remember to run `./gradlew spotlessApply` to fix any format violations.
|
||||
|
||||
Changes in this PR
|
||||
----
|
||||
|
||||
_Describe the new behavior from this PR, and why it's needed_
|
||||
Issue #
|
||||
|
||||
Alternatives considered
|
||||
----
|
||||
|
||||
_Describe alternative implementation you have considered_
|
||||
@@ -0,0 +1,39 @@
|
||||
template: |
|
||||
## What’s Changed
|
||||
|
||||
$CHANGES
|
||||
|
||||
name-template: 'v$RESOLVED_VERSION'
|
||||
tag-template: 'v$RESOLVED_VERSION'
|
||||
|
||||
categories:
|
||||
- title: 'IMPORTANT'
|
||||
label: 'type: important'
|
||||
- title: 'New'
|
||||
label: 'type: feature'
|
||||
- title: 'Bug Fixes'
|
||||
label: 'type: bug'
|
||||
- title: 'Refactor'
|
||||
label: 'type: maintenance'
|
||||
- title: 'Documentation'
|
||||
label: 'type: docs'
|
||||
- title: 'Dependency Updates'
|
||||
label: 'type: dependencies'
|
||||
|
||||
version-resolver:
|
||||
minor:
|
||||
labels:
|
||||
- 'type: important'
|
||||
|
||||
patch:
|
||||
labels:
|
||||
- 'type: bug'
|
||||
- 'type: maintenance'
|
||||
- 'type: docs'
|
||||
- 'type: dependencies'
|
||||
- 'type: feature'
|
||||
|
||||
exclude-labels:
|
||||
- 'skip-changelog'
|
||||
- 'gradle-wrapper'
|
||||
- 'github_actions'
|
||||
@@ -0,0 +1,419 @@
|
||||
name: CI
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
paths-ignore:
|
||||
- "conductor-clients/**"
|
||||
pull_request:
|
||||
paths-ignore:
|
||||
- "conductor-clients/**"
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
redis_es8:
|
||||
description: "Redis + Elasticsearch 8"
|
||||
type: boolean
|
||||
default: true
|
||||
postgres:
|
||||
description: "PostgreSQL"
|
||||
type: boolean
|
||||
default: false
|
||||
mysql:
|
||||
description: "MySQL"
|
||||
type: boolean
|
||||
default: false
|
||||
redis_os3:
|
||||
description: "Redis + OpenSearch 3"
|
||||
type: boolean
|
||||
default: false
|
||||
redis_es7:
|
||||
description: "Redis + Elasticsearch 7"
|
||||
type: boolean
|
||||
default: false
|
||||
cassandra_es7:
|
||||
description: "Cassandra + Elasticsearch 7"
|
||||
type: boolean
|
||||
default: false
|
||||
|
||||
jobs:
|
||||
detect-changes:
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
heavy-persistence: ${{ steps.filter.outputs.heavy-persistence }}
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
with:
|
||||
ref: ${{ github.event.pull_request.head.sha }}
|
||||
fetch-depth: 0
|
||||
- uses: dorny/paths-filter@v3
|
||||
id: filter
|
||||
with:
|
||||
filters: |
|
||||
heavy-persistence:
|
||||
- 'cassandra-persistence/**'
|
||||
- 'es6-persistence/**'
|
||||
- 'es7-persistence/**'
|
||||
- 'es8-persistence/**'
|
||||
- 'mysql-persistence/**'
|
||||
- 'os-persistence/**'
|
||||
- 'os-persistence-v2/**'
|
||||
- 'os-persistence-v3/**'
|
||||
- 'scheduler/cassandra-persistence/**'
|
||||
- 'scheduler/mysql-persistence/**'
|
||||
|
||||
build:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
with:
|
||||
ref: ${{ github.event.pull_request.head.sha }}
|
||||
fetch-depth: 0
|
||||
- name: Free disk space
|
||||
run: |
|
||||
sudo rm -rf /usr/share/dotnet
|
||||
sudo rm -rf /usr/local/lib/android
|
||||
sudo rm -rf /opt/ghc
|
||||
- name: Gradle wrapper validation
|
||||
uses: gradle/wrapper-validation-action@v3
|
||||
- name: Set up Zulu JDK 21
|
||||
uses: actions/setup-java@v5
|
||||
with:
|
||||
distribution: "zulu"
|
||||
java-version: "21"
|
||||
- name: Cache SonarCloud packages
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: ~/.sonar/cache
|
||||
key: ${{ runner.os }}-sonar
|
||||
restore-keys: ${{ runner.os }}-sonar
|
||||
- name: Cache Gradle packages
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: |
|
||||
~/.gradle/caches
|
||||
~/.gradle/wrapper
|
||||
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
|
||||
restore-keys: ${{ runner.os }}-gradle-
|
||||
- name: Build with Gradle
|
||||
if: github.ref != 'refs/heads/main'
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
|
||||
run: |
|
||||
./gradlew build -x :conductor-test-harness:test -x test
|
||||
- name: Build and Publish snapshot
|
||||
if: github.event_name != 'pull_request' && github.ref == 'refs/heads/main'
|
||||
run: |
|
||||
echo "Running build for commit ${{ github.sha }}"
|
||||
./gradlew build -x :conductor-test-harness:test -x test
|
||||
- name: Generate aggregated coverage report
|
||||
if: always()
|
||||
run: ./gradlew jacocoAggregatedReport -x test || true
|
||||
- name: Publish Test Report
|
||||
uses: mikepenz/action-junit-report@v6
|
||||
if: always()
|
||||
with:
|
||||
report_paths: "**/build/test-results/test/TEST-*.xml"
|
||||
- name: Upload build artifacts
|
||||
uses: actions/upload-artifact@v7
|
||||
with:
|
||||
name: build-artifacts
|
||||
path: "**/build/reports"
|
||||
- name: Upload coverage report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: coverage-report
|
||||
path: build/reports/jacoco/aggregated
|
||||
unit-test:
|
||||
needs: detect-changes
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
with:
|
||||
ref: ${{ github.event.pull_request.head.sha }}
|
||||
fetch-depth: 0
|
||||
- name: Free disk space
|
||||
run: |
|
||||
sudo rm -rf /usr/share/dotnet
|
||||
sudo rm -rf /usr/local/lib/android
|
||||
sudo rm -rf /opt/ghc
|
||||
- name: Set up Zulu JDK 21
|
||||
uses: actions/setup-java@v5
|
||||
with:
|
||||
distribution: "zulu"
|
||||
java-version: "21"
|
||||
- name: Cache Gradle packages
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: |
|
||||
~/.gradle/caches
|
||||
~/.gradle/wrapper
|
||||
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
|
||||
restore-keys: ${{ runner.os }}-gradle-
|
||||
- name: Force Docker API Version
|
||||
run: echo 'api.version=1.44' > ~/.docker-java.properties
|
||||
- name: Run unit tests
|
||||
run: |
|
||||
# On main (post-merge): run everything
|
||||
if [ "${{ github.event_name }}" != "pull_request" ]; then
|
||||
./gradlew test -x :conductor-test-harness:test
|
||||
exit 0
|
||||
fi
|
||||
# On PRs: skip heavy container tests (cassandra, es, mysql, opensearch)
|
||||
# unless their code changed. Redis, postgres, sqlite always run.
|
||||
if [ "${{ needs.detect-changes.outputs.heavy-persistence }}" == "true" ]; then
|
||||
./gradlew test -x :conductor-test-harness:test
|
||||
else
|
||||
./gradlew test \
|
||||
-x :conductor-test-harness:test \
|
||||
-x :conductor-cassandra-persistence:test \
|
||||
-x :conductor-scheduler-cassandra-persistence:test \
|
||||
-x :conductor-es6-persistence:test \
|
||||
-x :conductor-es7-persistence:test \
|
||||
-x :conductor-es8-persistence:test \
|
||||
-x :conductor-mysql-persistence:test \
|
||||
-x :conductor-scheduler-mysql-persistence:test \
|
||||
-x :conductor-os-persistence:test \
|
||||
-x :conductor-os-persistence-v2:test \
|
||||
-x :conductor-os-persistence-v3:test
|
||||
fi
|
||||
- name: Publish Test Report
|
||||
uses: mikepenz/action-junit-report@v6
|
||||
if: always()
|
||||
with:
|
||||
report_paths: "**/build/test-results/test/TEST-*.xml"
|
||||
check_name: Unit Test Report
|
||||
- name: Upload unit-test reports
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: unit-test-reports
|
||||
path: "**/build/reports/tests"
|
||||
test-harness:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
with:
|
||||
ref: ${{ github.event.pull_request.head.sha }}
|
||||
fetch-depth: 0
|
||||
- name: Free disk space
|
||||
run: |
|
||||
sudo rm -rf /usr/share/dotnet
|
||||
sudo rm -rf /usr/local/lib/android
|
||||
sudo rm -rf /opt/ghc
|
||||
- name: Set up Zulu JDK 21
|
||||
uses: actions/setup-java@v5
|
||||
with:
|
||||
distribution: "zulu"
|
||||
java-version: "21"
|
||||
- name: Cache Gradle packages
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: |
|
||||
~/.gradle/caches
|
||||
~/.gradle/wrapper
|
||||
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
|
||||
restore-keys: ${{ runner.os }}-gradle-
|
||||
- name: Force Docker API Version
|
||||
run: echo 'api.version=1.44' > ~/.docker-java.properties
|
||||
- name: Cache Docker images
|
||||
uses: actions/cache@v5
|
||||
id: docker-cache
|
||||
with:
|
||||
path: /tmp/docker-images-test-harness.tar
|
||||
key: docker-test-harness-v2
|
||||
- name: Load cached Docker images
|
||||
if: steps.docker-cache.outputs.cache-hit == 'true'
|
||||
run: docker load -i /tmp/docker-images-test-harness.tar || true
|
||||
- name: Run test-harness tests
|
||||
run: |
|
||||
./gradlew :conductor-test-harness:test
|
||||
- name: Save Docker images for cache
|
||||
if: steps.docker-cache.outputs.cache-hit != 'true'
|
||||
run: |
|
||||
set -euo pipefail
|
||||
docker image prune -f
|
||||
mapfile -t images < <((docker images --format '{{.Repository}}:{{.Tag}}' \
|
||||
| grep -v '<none>' \
|
||||
| grep -E '(^|/)(elasticsearch|redis|postgres|mysql|mongo|cassandra)(:|/)|mockserver/mockserver|opensearchproject/opensearch|testcontainers/|orkesio/') || true)
|
||||
if [ "${#images[@]}" -eq 0 ]; then
|
||||
echo "No Testcontainers-related images to cache; writing empty tar for cache action."
|
||||
tar -cf /tmp/docker-images-test-harness.tar --files-from /dev/null
|
||||
exit 0
|
||||
fi
|
||||
printf '%s\n' "${images[@]}"
|
||||
docker save -o /tmp/docker-images-test-harness.tar "${images[@]}"
|
||||
- name: Publish Test Report
|
||||
uses: mikepenz/action-junit-report@v6
|
||||
if: always()
|
||||
with:
|
||||
report_paths: "test-harness/build/test-results/test/TEST-*.xml"
|
||||
- name: Upload test-harness reports
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: test-harness-reports
|
||||
path: "test-harness/build/reports"
|
||||
- name: Upload test-harness coverage report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: test-harness-coverage-report
|
||||
path: "test-harness/build/reports/jacoco"
|
||||
generate-e2e-matrix:
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
matrix: ${{ steps.set-matrix.outputs.matrix }}
|
||||
steps:
|
||||
- id: set-matrix
|
||||
shell: bash
|
||||
run: |
|
||||
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
|
||||
items=""
|
||||
[ "${{ inputs.redis_es8 }}" = "true" ] && items="${items}{\"name\":\"redis-es8\",\"script\":\"./e2e/run_tests-es8.sh\"},"
|
||||
[ "${{ inputs.postgres }}" = "true" ] && items="${items}{\"name\":\"postgres\",\"script\":\"./e2e/run_tests-postgres.sh\"},"
|
||||
[ "${{ inputs.mysql }}" = "true" ] && items="${items}{\"name\":\"mysql\",\"script\":\"./e2e/run_tests-mysql.sh\"},"
|
||||
[ "${{ inputs.redis_os3 }}" = "true" ] && items="${items}{\"name\":\"redis-os3\",\"script\":\"./e2e/run_tests-redis-os3.sh\"},"
|
||||
[ "${{ inputs.redis_es7 }}" = "true" ] && items="${items}{\"name\":\"redis-es7\",\"script\":\"./e2e/run_tests-redis-es7.sh\"},"
|
||||
[ "${{ inputs.cassandra_es7 }}" = "true" ] && items="${items}{\"name\":\"cassandra-es7\",\"script\":\"./e2e/run_tests-cassandra-es7.sh\"},"
|
||||
items="${items%,}"
|
||||
echo "matrix={\"include\":[${items}]}" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo 'matrix={"include":[{"name":"redis-es8","script":"./e2e/run_tests-es8.sh"}]}' >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
e2e:
|
||||
needs: generate-e2e-matrix
|
||||
if: ${{ needs.generate-e2e-matrix.outputs.matrix != '{"include":[]}' }}
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix: ${{ fromJson(needs.generate-e2e-matrix.outputs.matrix) }}
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
with:
|
||||
ref: ${{ github.event.pull_request.head.sha }}
|
||||
fetch-depth: 0
|
||||
- name: Free disk space
|
||||
run: |
|
||||
sudo rm -rf /usr/share/dotnet
|
||||
sudo rm -rf /usr/local/lib/android
|
||||
sudo rm -rf /opt/ghc
|
||||
- name: Set up Zulu JDK 21
|
||||
uses: actions/setup-java@v5
|
||||
with:
|
||||
distribution: "zulu"
|
||||
java-version: "21"
|
||||
- name: Cache Gradle packages
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: |
|
||||
~/.gradle/caches
|
||||
~/.gradle/wrapper
|
||||
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
|
||||
restore-keys: ${{ runner.os }}-gradle-
|
||||
- name: Run E2E tests (${{ matrix.name }})
|
||||
run: ${{ matrix.script }}
|
||||
- name: Publish Test Report
|
||||
uses: mikepenz/action-junit-report@v6
|
||||
if: always()
|
||||
with:
|
||||
report_paths: "e2e/build/test-results/test/TEST-*.xml"
|
||||
- name: Generate test summary table
|
||||
if: always()
|
||||
shell: python3 {0}
|
||||
run: |
|
||||
import os, glob, xml.etree.ElementTree as ET
|
||||
|
||||
xml_files = glob.glob("e2e/build/test-results/test/TEST-*.xml")
|
||||
rows = []
|
||||
totals = {"passed": 0, "failed": 0, "skipped": 0}
|
||||
|
||||
for f in sorted(xml_files):
|
||||
try:
|
||||
root = ET.parse(f).getroot()
|
||||
except ET.ParseError:
|
||||
continue
|
||||
for tc in root.iter("testcase"):
|
||||
name = tc.get("name", "?")
|
||||
classname = tc.get("classname", "").split(".")[-1]
|
||||
duration = float(tc.get("time", 0))
|
||||
if tc.find("skipped") is not None:
|
||||
status, totals["skipped"] = "⏭ skip", totals["skipped"] + 1
|
||||
elif tc.find("failure") is not None or tc.find("error") is not None:
|
||||
node = tc.find("failure") if tc.find("failure") is not None else tc.find("error")
|
||||
msg = (node.get("message") or "")[:120]
|
||||
status, totals["failed"] = f"❌ `{msg}`", totals["failed"] + 1
|
||||
else:
|
||||
status, totals["passed"] = "✅", totals["passed"] + 1
|
||||
rows.append((classname, name, f"{duration:.1f}s", status))
|
||||
|
||||
backend = "${{ matrix.name }}"
|
||||
lines = [
|
||||
f"## E2E results — {backend}",
|
||||
f"**✅ {totals['passed']} passed · ❌ {totals['failed']} failed · ⏭ {totals['skipped']} skipped**",
|
||||
"",
|
||||
"| Class | Test | Duration | Result |",
|
||||
"|-------|------|----------|--------|",
|
||||
]
|
||||
for cls, name, dur, status in rows:
|
||||
lines.append(f"| {cls} | {name} | {dur} | {status} |")
|
||||
|
||||
summary = os.environ.get("GITHUB_STEP_SUMMARY", "/dev/null")
|
||||
with open(summary, "a") as fh:
|
||||
fh.write("\n".join(lines) + "\n")
|
||||
- name: Upload E2E reports
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: e2e-reports-${{ matrix.name }}
|
||||
path: "e2e/build/reports"
|
||||
|
||||
build-ui:
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ui
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Set up Node
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: "22"
|
||||
cache: "yarn"
|
||||
cache-dependency-path: ui/yarn.lock
|
||||
|
||||
- name: Install Dependencies
|
||||
run: yarn install
|
||||
|
||||
- name: Build UI
|
||||
run: yarn run build
|
||||
|
||||
- name: Cache Playwright browsers
|
||||
uses: actions/cache@v5
|
||||
id: playwright-cache
|
||||
with:
|
||||
path: ~/.cache/ms-playwright
|
||||
key: playwright-chromium-${{ hashFiles('ui/yarn.lock') }}
|
||||
|
||||
- name: Install Playwright browsers
|
||||
if: steps.playwright-cache.outputs.cache-hit != 'true'
|
||||
run: npx playwright install --with-deps chromium
|
||||
|
||||
- name: Install Playwright browser deps (cached)
|
||||
if: steps.playwright-cache.outputs.cache-hit == 'true'
|
||||
run: npx playwright install-deps chromium
|
||||
|
||||
- name: Run Playwright E2E Tests
|
||||
run: yarn test:e2e
|
||||
|
||||
- name: Upload Playwright report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: failure()
|
||||
with:
|
||||
name: playwright-report
|
||||
path: ui/playwright-report
|
||||
retention-days: 7
|
||||
@@ -0,0 +1,34 @@
|
||||
name: Debug Docker Credentials
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
check-docker-user:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Check Docker Hub user for API key
|
||||
env:
|
||||
DOCKER_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
DOCKER_PASSWORD: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
run: |
|
||||
echo "Fetching JWT from Docker Hub..."
|
||||
RESPONSE=$(curl -s -X POST \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{\"username\": \"${DOCKER_USERNAME}\", \"password\": \"${DOCKER_PASSWORD}\"}" \
|
||||
https://hub.docker.com/v2/users/login)
|
||||
|
||||
JWT=$(echo "$RESPONSE" | jq -r '.token // empty')
|
||||
|
||||
if [ -z "$JWT" ]; then
|
||||
echo "Login failed. Response:"
|
||||
echo "$RESPONSE" | jq .
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Login successful. Fetching user info..."
|
||||
USER_INFO=$(curl -s -H "Authorization: Bearer $JWT" https://hub.docker.com/v2/user/)
|
||||
echo "Raw response:"
|
||||
echo "$USER_INFO" | jq .
|
||||
echo "Docker Hub account info:"
|
||||
echo "$USER_INFO" | jq '{username: .username, full_name: .full_name, email: .email, company: .company, date_joined: .date_joined}'
|
||||
@@ -0,0 +1,44 @@
|
||||
name: Deprecate conductor-standalone on Docker Hub
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
deprecate:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Login to Docker Hub
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
- name: Set up QEMU
|
||||
uses: docker/setup-qemu-action@v4
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v4
|
||||
|
||||
- name: Build and push :deprecated tag
|
||||
uses: docker/build-push-action@v7
|
||||
with:
|
||||
context: docker/conductor-standalone-deprecation
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
tags: conductoross/conductor-standalone:deprecated
|
||||
|
||||
- name: Update Docker Hub description
|
||||
run: |
|
||||
TOKEN=$(curl -s -X POST "https://hub.docker.com/v2/users/login" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{\"username\":\"${{ secrets.DOCKERHUB_USERNAME }}\",\"password\":\"${{ secrets.DOCKERHUB_TOKEN }}\"}" \
|
||||
| jq -r '.token')
|
||||
curl -s -X PATCH "https://hub.docker.com/v2/repositories/conductoross/conductor-standalone/" \
|
||||
-H "Authorization: Bearer ${TOKEN}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{\"full_description\":$(jq -Rs . < docker/conductor-standalone-deprecation/dockerhub-description.md)}"
|
||||
@@ -0,0 +1,25 @@
|
||||
name: Publish docs via GitHub Pages
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
build:
|
||||
name: Deploy docs
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout main
|
||||
uses: actions/checkout@v7
|
||||
|
||||
- name: Deploy docs
|
||||
uses: mhausenblas/mkdocs-deploy-gh-pages@master
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
CONFIG_FILE: mkdocs.yml
|
||||
REQUIREMENTS: requirements.txt
|
||||
@@ -0,0 +1,113 @@
|
||||
name: Publish Next (ui-next)
|
||||
# Publishes the server with ui-next as a parallel track alongside :latest.
|
||||
# Does NOT touch :latest, any versioned release tags, or the existing S3 JARs.
|
||||
#
|
||||
# Resulting artifacts:
|
||||
# Docker: conductoross/conductor:next
|
||||
# S3 JAR: conductor-server-next.jar
|
||||
#
|
||||
# Test the Docker image:
|
||||
# docker run -p 8080:8080 -p 5000:5000 conductoross/conductor:next
|
||||
# → API / Swagger: http://localhost:8080
|
||||
# → ui-next: http://localhost:5000
|
||||
#
|
||||
# Test via the CLI:
|
||||
# conductor server start --version next
|
||||
#
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
publish-next:
|
||||
runs-on: ubuntu-latest
|
||||
name: Build and publish ui-next artifacts
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
# ── Java ─────────────────────────────────────────────────────────────────
|
||||
- name: Set up Zulu JDK 21
|
||||
uses: actions/setup-java@v5
|
||||
with:
|
||||
distribution: 'zulu'
|
||||
java-version: '21'
|
||||
|
||||
- name: Cache Gradle packages
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: |
|
||||
~/.gradle/caches
|
||||
~/.gradle/wrapper
|
||||
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-gradle-
|
||||
|
||||
- name: Build Server JAR
|
||||
run: |
|
||||
./gradlew :conductor-server:build -x test -x spotlessCheck -x shadowJar \
|
||||
-Dorg.gradle.jvmargs=-Xmx2g --no-daemon
|
||||
|
||||
# ── UI (ui-next) ─────────────────────────────────────────────────────────
|
||||
- name: Set up Node.js
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: '22'
|
||||
|
||||
- name: Set up pnpm
|
||||
uses: pnpm/action-setup@v6
|
||||
with:
|
||||
version: "10.32.0"
|
||||
|
||||
- name: Build ui-next
|
||||
run: |
|
||||
cd ui-next
|
||||
pnpm install && pnpm build
|
||||
|
||||
# ── Stage pre-built artifacts for PREBUILT=true Docker build ─────────────
|
||||
- name: Stage artifacts
|
||||
run: |
|
||||
mkdir -p docker/server/libs
|
||||
cp server/build/libs/*boot*.jar docker/server/libs/conductor-server.jar
|
||||
|
||||
# ── Docker ───────────────────────────────────────────────────────────────
|
||||
- name: Login to Docker Hub
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
- name: Set up QEMU
|
||||
uses: docker/setup-qemu-action@v4
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v4
|
||||
|
||||
- name: Build and push :next image
|
||||
uses: docker/build-push-action@v7
|
||||
with:
|
||||
context: .
|
||||
file: docker/server/Dockerfile.next
|
||||
platforms: linux/arm64,linux/amd64
|
||||
push: true
|
||||
build-args: |
|
||||
PREBUILT=true
|
||||
tags: |
|
||||
conductoross/conductor:next
|
||||
|
||||
# ── S3 JAR ───────────────────────────────────────────────────────────────
|
||||
- name: Configure AWS credentials
|
||||
uses: aws-actions/configure-aws-credentials@v6
|
||||
with:
|
||||
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
aws-region: ${{ secrets.AWS_REGION }}
|
||||
|
||||
- name: Upload :next JAR to S3
|
||||
run: |
|
||||
aws s3 cp server/build/libs/*boot*.jar \
|
||||
s3://${{ secrets.AWS_S3_BUCKET }}/conductor-server-next.jar
|
||||
echo "Published: conductor-server-next.jar"
|
||||
echo "Test with: conductor server start --version next"
|
||||
@@ -0,0 +1,114 @@
|
||||
name: Publish Conductor OSS toMaven Central
|
||||
on:
|
||||
release:
|
||||
types:
|
||||
- released
|
||||
- prereleased
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
version:
|
||||
description: 'Version to publish (e.g., v1.0.0)'
|
||||
required: true
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
publish:
|
||||
runs-on: ubuntu-latest
|
||||
name: Gradle Build and Publish
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- name: Set up Zulu JDK 21
|
||||
uses: actions/setup-java@v5
|
||||
with:
|
||||
distribution: 'zulu'
|
||||
java-version: '21'
|
||||
- name: Cache Gradle packages
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: |
|
||||
~/.gradle/caches
|
||||
~/.gradle/wrapper
|
||||
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-gradle-
|
||||
- name: Publish release
|
||||
run: |
|
||||
export VERSION="${{github.ref_name}}"
|
||||
export PUBLISH_VERSION=`echo ${VERSION:1}`
|
||||
echo Publishing version $PUBLISH_VERSION
|
||||
./gradlew publish -PmavenCentral -Pversion=$PUBLISH_VERSION -Pusername=${{ secrets.SONATYPE_USERNAME }} -Ppassword=${{ secrets.SONATYPE_PASSWORD }}
|
||||
env:
|
||||
ORG_GRADLE_PROJECT_signingKeyId: ${{ secrets.SIGNING_KEY_ID }}
|
||||
ORG_GRADLE_PROJECT_signingKey: ${{ secrets.SIGNING_KEY }}
|
||||
ORG_GRADLE_PROJECT_signingPassword: ${{ secrets.SIGNING_PASSWORD }}
|
||||
|
||||
publish-docker:
|
||||
runs-on: ubuntu-latest
|
||||
name: Gradle Build and Publish to Container Registry
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- name: Set up Zulu JDK 21
|
||||
uses: actions/setup-java@v5
|
||||
with:
|
||||
distribution: 'zulu'
|
||||
java-version: '21'
|
||||
|
||||
- name: Login to Docker Hub Container Registry
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
- name: Set up Node.js
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: 'lts/*'
|
||||
|
||||
- name: Build ui-next and embed into server static resources
|
||||
run: |
|
||||
corepack enable
|
||||
cd ui-next
|
||||
pnpm install
|
||||
NODE_OPTIONS=--max-old-space-size=4096 pnpm build
|
||||
cd ..
|
||||
mkdir -p server/src/main/resources/static
|
||||
rm -rf server/src/main/resources/static/*
|
||||
cp -r ui-next/dist/. server/src/main/resources/static/
|
||||
|
||||
- name: Build Server JAR
|
||||
run: |
|
||||
export VERSION="${{github.ref_name}}"
|
||||
export PUBLISH_VERSION=`echo ${VERSION:1}`
|
||||
echo "RELEASE_VERSION=$PUBLISH_VERSION" >> $GITHUB_ENV
|
||||
echo Publishing version $PUBLISH_VERSION
|
||||
./gradlew :conductor-server:build -x test -x spotlessCheck -x shadowJar -x :conductor-os-persistence-v3:build \
|
||||
-Pversion=$PUBLISH_VERSION
|
||||
|
||||
- name: Stage artifacts for Docker build
|
||||
run: |
|
||||
mkdir -p docker/server/libs
|
||||
cp server/build/libs/*boot*.jar docker/server/libs/conductor-server.jar
|
||||
|
||||
- name: Set up QEMU
|
||||
uses: docker/setup-qemu-action@v4
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v4
|
||||
|
||||
- name: Build and push Server
|
||||
uses: docker/build-push-action@v7
|
||||
with:
|
||||
context: .
|
||||
file: docker/server/Dockerfile
|
||||
platforms: linux/arm64,linux/amd64
|
||||
push: true
|
||||
build-args: |
|
||||
PREBUILT=true
|
||||
tags: |
|
||||
conductoross/conductor:${{ env.RELEASE_VERSION }}
|
||||
orkesio/orkes-conductor-community-standalone:${{ env.RELEASE_VERSION }}
|
||||
orkesio/orkes-conductor-community:${{ env.RELEASE_VERSION }}
|
||||
${{ (github.event_name == 'release' && !github.event.release.prerelease && !contains(github.ref_name, 'rc') && !contains(github.ref_name, 'beta') && !contains(github.ref_name, 'alpha')) && 'conductoross/conductor:latest' || '' }}
|
||||
${{ (github.event_name == 'release' && !github.event.release.prerelease && !contains(github.ref_name, 'rc') && !contains(github.ref_name, 'beta') && !contains(github.ref_name, 'alpha')) && 'orkesio/orkes-conductor-community-standalone:latest' || '' }}
|
||||
${{ (github.event_name == 'release' && !github.event.release.prerelease && !contains(github.ref_name, 'rc') && !contains(github.ref_name, 'beta') && !contains(github.ref_name, 'alpha')) && 'orkesio/orkes-conductor-community:latest' || '' }}
|
||||
@@ -0,0 +1,100 @@
|
||||
name: Publish Conductor OSS Server
|
||||
on:
|
||||
release:
|
||||
types:
|
||||
- released
|
||||
- prereleased
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
version:
|
||||
description: 'Version to publish (e.g., v1.0.0)'
|
||||
required: true
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
publish:
|
||||
runs-on: ubuntu-latest
|
||||
name: Build and Publish the server
|
||||
permissions:
|
||||
contents: write
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- name: Set up Zulu JDK 21
|
||||
uses: actions/setup-java@v5
|
||||
with:
|
||||
distribution: 'zulu'
|
||||
java-version: '21'
|
||||
- name: Cache Gradle packages
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: |
|
||||
~/.gradle/caches
|
||||
~/.gradle/wrapper
|
||||
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-gradle-
|
||||
- name: Set version
|
||||
id: version
|
||||
run: |
|
||||
# Use inputs.version for workflow_dispatch, or ref_name for release events
|
||||
VERSION="${{ github.event.inputs.version || github.ref_name }}"
|
||||
# Strip the 'v' prefix if present
|
||||
PUBLISH_VERSION="${VERSION#v}"
|
||||
echo "version=$PUBLISH_VERSION" >> $GITHUB_OUTPUT
|
||||
echo "RELEASE_VERSION=$PUBLISH_VERSION" >> $GITHUB_ENV
|
||||
echo "Publishing version: $PUBLISH_VERSION"
|
||||
- name: Build Server JAR
|
||||
run: |
|
||||
./gradlew :conductor-server:build -x test -x spotlessCheck -x shadowJar -x :conductor-os-persistence-v3:build \
|
||||
-Pversion=${{ steps.version.outputs.version }}
|
||||
|
||||
- name: Set up Node.js
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: 'lts/*'
|
||||
|
||||
- name: Build ui-next and embed into server static resources
|
||||
run: |
|
||||
corepack enable
|
||||
cd ui-next
|
||||
pnpm install
|
||||
NODE_OPTIONS=--max-old-space-size=4096 pnpm build
|
||||
cd ..
|
||||
mkdir -p server/src/main/resources/static
|
||||
rm -rf server/src/main/resources/static/*
|
||||
cp -r ui-next/dist/. server/src/main/resources/static/
|
||||
|
||||
- name: Stage artifacts for Docker build
|
||||
run: |
|
||||
mkdir -p docker/server/libs
|
||||
cp server/build/libs/*boot*.jar docker/server/libs/conductor-server.jar
|
||||
|
||||
- name: Login to Docker Hub Container Registry
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
- name: Set up QEMU
|
||||
uses: docker/setup-qemu-action@v4
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v4
|
||||
|
||||
- name: Build and push Server
|
||||
uses: docker/build-push-action@v7
|
||||
with:
|
||||
context: .
|
||||
file: docker/server/Dockerfile
|
||||
platforms: linux/arm64,linux/amd64
|
||||
push: true
|
||||
build-args: |
|
||||
PREBUILT=true
|
||||
tags: |
|
||||
conductoross/conductor:${{ env.RELEASE_VERSION }}
|
||||
orkesio/orkes-conductor-community-standalone:${{ env.RELEASE_VERSION }}
|
||||
orkesio/orkes-conductor-community:${{ env.RELEASE_VERSION }}
|
||||
${{ (github.event_name == 'release' && !github.event.release.prerelease) && 'conductoross/conductor:latest' || '' }}
|
||||
${{ (github.event_name == 'release' && !github.event.release.prerelease) && 'orkesio/orkes-conductor-community-standalone:latest' || '' }}
|
||||
${{ (github.event_name == 'release' && !github.event.release.prerelease) && 'orkesio/orkes-conductor-community:latest' || '' }}
|
||||
@@ -0,0 +1,84 @@
|
||||
name: Publish Conductor Server to S3
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- docker_build # TEMPORARY - remove before merging to main
|
||||
release:
|
||||
types:
|
||||
- released
|
||||
- prereleased
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
version:
|
||||
description: 'Version to publish (e.g., v1.0.0)'
|
||||
required: true
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
publish-s3:
|
||||
runs-on: ubuntu-latest
|
||||
name: Build and Publish Server JAR to S3
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Set up Zulu JDK 21
|
||||
uses: actions/setup-java@v5
|
||||
with:
|
||||
distribution: 'zulu'
|
||||
java-version: '21'
|
||||
|
||||
- name: Cache Gradle packages
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: |
|
||||
~/.gradle/caches
|
||||
~/.gradle/wrapper
|
||||
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-gradle-
|
||||
|
||||
- name: Set version
|
||||
id: version
|
||||
run: |
|
||||
# Use inputs.version for workflow_dispatch, or ref_name for release events
|
||||
VERSION="${{ github.event.inputs.version || github.ref_name }}"
|
||||
# Strip the 'v' prefix if present
|
||||
PUBLISH_VERSION="${VERSION#v}"
|
||||
echo "version=$PUBLISH_VERSION" >> $GITHUB_OUTPUT
|
||||
echo "Publishing version: $PUBLISH_VERSION"
|
||||
|
||||
- name: Set up Node.js
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: 'lts/*'
|
||||
|
||||
- name: Build Server module
|
||||
run: |
|
||||
corepack enable
|
||||
./build_ui_next.sh
|
||||
./gradlew :conductor-server:bootJar -x test -Pversion=${{ steps.version.outputs.version }}
|
||||
|
||||
- name: Configure AWS credentials
|
||||
uses: aws-actions/configure-aws-credentials@v6
|
||||
with:
|
||||
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
aws-region: ${{ secrets.AWS_REGION }}
|
||||
|
||||
- name: Upload Server JAR to S3 (Versioned)
|
||||
run: |
|
||||
aws s3 cp server/build/libs/conductor-server-${{ steps.version.outputs.version }}-boot.jar \
|
||||
s3://${{ secrets.AWS_S3_BUCKET }}/conductor-server-${{ steps.version.outputs.version }}.jar
|
||||
|
||||
- name: Upload Server JAR to S3 (Latest)
|
||||
if: github.event_name == 'release' && !github.event.release.prerelease
|
||||
run: |
|
||||
aws s3 cp server/build/libs/conductor-server-${{ steps.version.outputs.version }}-boot.jar \
|
||||
s3://${{ secrets.AWS_S3_BUCKET }}/conductor-server-latest.jar
|
||||
|
||||
- name: Verify uploads
|
||||
run: |
|
||||
echo "Uploaded conductor-server-${{ steps.version.outputs.version }}.jar"
|
||||
echo "S3 bucket: ${{ secrets.AWS_S3_BUCKET }}"
|
||||
@@ -0,0 +1,22 @@
|
||||
name: Release Drafter
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
paths-ignore:
|
||||
- 'conductor-clients/**'
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
update_release_draft:
|
||||
permissions:
|
||||
contents: write # for release-drafter/release-drafter to create a github release
|
||||
pull-requests: write # for release-drafter/release-drafter to add label to PR
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: release-drafter/release-drafter@v7
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
@@ -0,0 +1,81 @@
|
||||
name: UI v2 CI
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches:
|
||||
- main
|
||||
paths:
|
||||
- "ui-next/**"
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
paths:
|
||||
- "ui-next/**"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
lint-format-test:
|
||||
name: Lint, Format & Test
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ui-next
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Setup pnpm
|
||||
uses: pnpm/action-setup@v6
|
||||
with:
|
||||
version: 10.32.0
|
||||
|
||||
# setup-node must come after pnpm/action-setup so it can locate the store.
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: 22
|
||||
cache: pnpm
|
||||
cache-dependency-path: ui-next/pnpm-lock.yaml
|
||||
|
||||
- name: Install dependencies
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
- name: Prettier check
|
||||
run: pnpm prettier:check
|
||||
|
||||
- name: Lint
|
||||
run: pnpm lint
|
||||
|
||||
- name: Type check
|
||||
run: pnpm typecheck
|
||||
|
||||
- name: Test
|
||||
run: pnpm test
|
||||
|
||||
- name: Build
|
||||
run: pnpm build
|
||||
|
||||
e2e-mocked:
|
||||
name: E2E (Mocked)
|
||||
runs-on: ubuntu-latest
|
||||
needs: lint-format-test
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v4
|
||||
|
||||
- name: Run E2E tests in Docker
|
||||
run: docker compose -f docker-compose.snapshots.yml run --rm playwright
|
||||
working-directory: ui-next
|
||||
|
||||
- name: Upload Playwright report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: failure()
|
||||
with:
|
||||
name: playwright-report
|
||||
path: ui-next/playwright-snapshots-report/
|
||||
retention-days: 7
|
||||
@@ -0,0 +1,103 @@
|
||||
name: UI v2 Integration CI
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches:
|
||||
- main
|
||||
paths:
|
||||
- "ui-next/**"
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
paths:
|
||||
- "ui-next/**"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
e2e-integration:
|
||||
name: E2E (Integration)
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Free disk space
|
||||
run: |
|
||||
sudo rm -rf /usr/share/dotnet
|
||||
sudo rm -rf /usr/local/lib/android
|
||||
sudo rm -rf /opt/ghc
|
||||
sudo rm -rf /usr/local/share/boost
|
||||
sudo apt-get clean
|
||||
|
||||
- name: Setup pnpm
|
||||
uses: pnpm/action-setup@v6
|
||||
with:
|
||||
version: 10.32.0
|
||||
|
||||
# setup-node must come after pnpm/action-setup so it can locate the store.
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: 22
|
||||
cache: pnpm
|
||||
cache-dependency-path: ui-next/pnpm-lock.yaml
|
||||
|
||||
- name: Install dependencies
|
||||
run: pnpm install --frozen-lockfile
|
||||
working-directory: ui-next
|
||||
|
||||
- name: Cache Playwright browsers
|
||||
id: playwright-cache
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: ~/.cache/ms-playwright
|
||||
key: ${{ runner.os }}-playwright-chromium-${{ hashFiles('ui-next/pnpm-lock.yaml') }}
|
||||
|
||||
- name: Install Playwright browsers
|
||||
if: steps.playwright-cache.outputs.cache-hit != 'true'
|
||||
run: pnpm exec playwright install chromium
|
||||
working-directory: ui-next
|
||||
|
||||
# OS-level dependencies (apt packages) cannot be cached — always install.
|
||||
- name: Install Playwright OS dependencies
|
||||
run: pnpm exec playwright install-deps chromium
|
||||
working-directory: ui-next
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v4
|
||||
|
||||
- name: Cache Docker layers for conductor:server
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: /tmp/.buildx-cache
|
||||
key: ${{ runner.os }}-conductor-server-${{ hashFiles('docker/server/Dockerfile', '**/build.gradle', 'settings.gradle') }}
|
||||
restore-keys: ${{ runner.os }}-conductor-server-
|
||||
|
||||
- name: Build conductor:server Docker image
|
||||
uses: docker/build-push-action@v7
|
||||
with:
|
||||
context: .
|
||||
file: docker/server/Dockerfile
|
||||
tags: conductor:server
|
||||
load: true
|
||||
cache-from: type=local,src=/tmp/.buildx-cache
|
||||
cache-to: type=local,dest=/tmp/.buildx-cache-new,mode=min
|
||||
|
||||
- name: Move Docker layer cache
|
||||
run: |
|
||||
rm -rf /tmp/.buildx-cache
|
||||
mv /tmp/.buildx-cache-new /tmp/.buildx-cache
|
||||
|
||||
- name: Run integration E2E tests
|
||||
run: pnpm test:e2e:integration
|
||||
working-directory: ui-next
|
||||
|
||||
- name: Upload Playwright integration report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: failure()
|
||||
with:
|
||||
name: playwright-integration-report
|
||||
path: ui-next/playwright-integration-report/
|
||||
retention-days: 7
|
||||
+58
@@ -0,0 +1,58 @@
|
||||
# Worktrees
|
||||
.worktrees/
|
||||
|
||||
# Java Build
|
||||
.gradle
|
||||
.classpath
|
||||
dump.rdb
|
||||
out
|
||||
bin
|
||||
target
|
||||
buildscan.log
|
||||
/docs/site
|
||||
|
||||
# Python
|
||||
/polyglot-clients/python/conductor.egg-info
|
||||
*.pyc
|
||||
|
||||
# OS & IDE
|
||||
.DS_Store
|
||||
.settings
|
||||
.vscode
|
||||
.idea
|
||||
.project
|
||||
*.iml
|
||||
|
||||
# JS & UI Related
|
||||
node_modules
|
||||
/ui/build
|
||||
/ui/public/monaco-editor
|
||||
|
||||
# publishing secrets
|
||||
secrets/signing-key
|
||||
|
||||
# local builds
|
||||
lib/
|
||||
build/
|
||||
*/build/
|
||||
|
||||
# asdf version file
|
||||
.tool-versions
|
||||
|
||||
# jenv version file
|
||||
.java-version
|
||||
|
||||
|
||||
.qodo
|
||||
conductorosstest.db
|
||||
conductorosstest.db
|
||||
yarn.lock
|
||||
.java-version
|
||||
/server/src/main/resources/static
|
||||
*.factorypath
|
||||
server/*.db*
|
||||
/site
|
||||
*.db
|
||||
*.db-shm
|
||||
*.db-wal
|
||||
docs/superpowers
|
||||
@@ -0,0 +1,178 @@
|
||||
# AGENTS.md
|
||||
|
||||
Instructions for AI coding agents working on the Conductor codebase.
|
||||
|
||||
## Project Overview
|
||||
|
||||
Conductor is an open-source, distributed workflow orchestration engine designed for microservices.
|
||||
It uses a pluggable architecture with interface-based abstractions for persistence, queuing, and indexing.
|
||||
The project is built with Java 21 and uses Gradle as the build system.
|
||||
|
||||
## Setup Commands
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `./gradlew build` | Build the entire project |
|
||||
| `./gradlew test` | Run all tests |
|
||||
| `./gradlew :module-name:test` | Run tests for a specific module |
|
||||
| `./gradlew spotlessApply` | Apply code formatting |
|
||||
| `./gradlew clean build` | Clean and rebuild |
|
||||
|
||||
> **Important**: Always run `./gradlew spotlessApply` after making code changes to ensure consistent formatting.
|
||||
|
||||
## Java Version References
|
||||
|
||||
**Never link to a specific Java distribution** (e.g., Adoptium, Temurin, OpenJDK.org, Amazon Corretto) in docs, READMEs, or comments. Just say "Java 21+" and let users install it however they prefer.
|
||||
|
||||
## Code Style
|
||||
|
||||
- Use the Spotless plugin for uniform code formatting—always run before committing
|
||||
- Conductor is pluggable: when introducing new concepts, always use an **interface-based approach**
|
||||
- DAO interfaces **MUST** be defined in the `core` module
|
||||
- Implementation classes go in their respective persistence modules (e.g., `postgres-persistence`, `redis-persistence`)
|
||||
- Follow existing patterns in the codebase for consistency
|
||||
- Do not use emojis such as ✅ in the code, logs, or comments. Keep comments professionals
|
||||
- When adding new logic, comment the algorithm, design etc.
|
||||
|
||||
## Architecture Guidelines
|
||||
|
||||
### Module Structure
|
||||
|
||||
- **core**: Contains interfaces, domain models, and core business logic
|
||||
- **persistence modules**: Implementations of DAO interfaces (postgres, redis, mysql, etc.)
|
||||
- **server**: Spring Boot application that brings everything together
|
||||
- **client**: SDK for interacting with Conductor
|
||||
- **ui**: React-based user interface
|
||||
|
||||
### Key Patterns
|
||||
|
||||
- DAOs are defined as interfaces in `core` and implemented in persistence modules
|
||||
- System tasks extend `WorkflowSystemTask` and are registered via Spring
|
||||
- Worker tasks use the `@WorkerTask` annotation for automatic discovery
|
||||
- Configuration is primarily done through Spring properties
|
||||
|
||||
## Testing
|
||||
|
||||
- **Avoid mocks**: Use real implementations whenever possible
|
||||
- **Test actual behavior**: Tests must verify real implementation logic, not duplicate it
|
||||
- **Use Testcontainers**: For database, cache, and other external dependencies
|
||||
- **Cover concurrency**: Ensure multi-threading scenarios are tested
|
||||
- **Run tests before submitting**: `./gradlew test` must pass
|
||||
|
||||
### Test Locations
|
||||
|
||||
- Unit tests: `src/test/java` in each module
|
||||
- Integration tests: `test-harness` module and `*-integration-test` modules
|
||||
- E2E tests: `e2e` module
|
||||
|
||||
## PR Guidelines
|
||||
|
||||
- Submit PRs against the `main` branch
|
||||
- Use clear, descriptive commit messages
|
||||
- Run `./gradlew spotlessApply` and `./gradlew test` before pushing
|
||||
- Add or update tests for any code changes
|
||||
- Keep PRs focused—one logical change per PR
|
||||
|
||||
## Dependency Pinning
|
||||
|
||||
Some dependencies have hard version constraints that **must not be auto-bumped**. These are marked with:
|
||||
|
||||
```groovy
|
||||
// PINNED (#964): <reason>
|
||||
```
|
||||
|
||||
The issue number links back to https://github.com/conductor-oss/conductor/issues/964, which documents the full audit and upgrade path for each constraint.
|
||||
|
||||
### What PINNED means
|
||||
|
||||
`// PINNED (#964):` means the version is intentionally locked and upgrading it without understanding the constraint will break the build or cause a runtime failure. Do not bump a PINNED dependency as part of routine dependency updates or refactoring.
|
||||
|
||||
### Current hard pins
|
||||
|
||||
| Dependency | Pinned at | Why |
|
||||
|---|---|---|
|
||||
| `com.google.protobuf:protobuf-java` | `3.x` | 4.x + GraalVM polyglot 25.x causes Gradle to require `polyglot4`, which does not exist on Maven Central |
|
||||
| `com.google.protobuf:protoc` | `3.25.5` | Must match `grpc-protobuf:1.73.0`, which depends on protobuf-java 3.x |
|
||||
| `org.graalvm.*` (all 5 artifacts) | same version | All must share one version — mixing causes a `"polyglot version X not compatible with Truffle Y"` runtime error |
|
||||
| `redis.clients:jedis` in `redis-concurrency-limit` | `3.6.0` | `revJedis` (6.0.0) does not work with Spring Data Redis in that module |
|
||||
| `org.codehaus.jettison:jettison` | `strictly 1.5.4` | Gradle `strictly` constraint — no higher version has been validated |
|
||||
| `org.conductoross:conductor-client` in `test-harness` | `5.0.1` | Fat JAR classpath conflict with conductor-common; resolved via a stripped JAR task |
|
||||
| `org.awaitility:awaitility` in functional tests | `4.x` | e2e tests call `pollInterval(Duration)` added in Awaitility 4.0 |
|
||||
|
||||
### Before bumping a PINNED dependency
|
||||
|
||||
1. Read the comment carefully — it will name the incompatibility and often link to an upstream issue.
|
||||
2. Check whether the upstream blocker has been resolved (e.g., new grpc-java release, new GraalVM release).
|
||||
3. Test locally: `./gradlew clean build` plus `./gradlew test` in the affected modules.
|
||||
4. If bumping GraalVM, bump **all five** `org.graalvm.*` artifacts together using `revGraalVM` in `dependencies.gradle`.
|
||||
5. Update or remove the `// PINNED` comment once the constraint is lifted.
|
||||
|
||||
### PINNED vs. version floors
|
||||
|
||||
Hard caps use `// PINNED (#964):`. Version floors — where a minimum is enforced but higher versions are always welcome — use one of two lowercase prefixes instead:
|
||||
|
||||
```groovy
|
||||
// Security: CVE-2025-12183 — lz4-java minimum patched version
|
||||
// Compat: commons-lang3 3.18.0+ required by Testcontainers/commons-compress
|
||||
```
|
||||
|
||||
- `// Security:` — minimum set to address a CVE or known vulnerability
|
||||
- `// Compat:` — minimum set for compatibility with another library or framework
|
||||
|
||||
These are grep-able (`grep "// Security:" **/*.gradle`, `grep "// Compat:" **/*.gradle`) but read as normal developer comments. Dependabot may raise these freely; no special review needed beyond the usual.
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- Never commit secrets, API keys, or credentials
|
||||
- Be cautious with external dependencies—prefer well-maintained libraries
|
||||
- Follow secure coding practices for input validation and error handling
|
||||
- Review [SECURITY.md](SECURITY.md) for vulnerability reporting procedures
|
||||
|
||||
## Writing Documentation
|
||||
|
||||
Documentation in this project is **derived from source**, not composed from memory. Open the source first, read what's there, then write the doc from what you find. The source is the spec; the doc is a rendering of it.
|
||||
|
||||
This matters because plausible-looking docs can be silently wrong. Concretely: a curl equivalent for `conductor workflow start --sync` was once written as `POST /api/workflow/{name}/run` — an endpoint that does not exist. Reading the controller first would have given the correct path immediately.
|
||||
|
||||
### Workflow for each content type
|
||||
|
||||
**REST API endpoint or curl example**
|
||||
1. Open the relevant controller: `rest/src/main/java/com/netflix/conductor/rest/controllers/`
|
||||
2. Find the method using its `@PostMapping`/`@GetMapping`/etc. annotation — copy the path literally.
|
||||
3. Read the method signature for query params, path variables, and request body type.
|
||||
4. Write the curl command from what you just read.
|
||||
|
||||
**CLI command or flag**
|
||||
1. Open `cmd/*.go` in `conductor-cli` (separate repo).
|
||||
2. Find the `cobra.Command` definition for the subcommand.
|
||||
3. Read the `Flags()` declarations for exact flag names, types, and defaults.
|
||||
4. Write the example from what you just read.
|
||||
|
||||
**SDK code example (Python, JS, Java, Go)**
|
||||
1. Open the relevant SDK source file.
|
||||
2. Find the method signature and required parameters.
|
||||
3. Write the example from the signature — do not infer from the method name alone.
|
||||
4. If a working test exists for that method, use it as the starting point.
|
||||
|
||||
**Expected output block**
|
||||
1. Get real output: run the command locally, or find it in test fixtures, CI logs, or existing tests.
|
||||
2. Paste verbatim. Do not paraphrase or construct output that "looks right."
|
||||
3. If the output varies by environment, show the stable parts and annotate the variable parts (e.g., `<workflow-id>`).
|
||||
|
||||
**Editing an existing doc section**
|
||||
1. Before touching prose, read every code block and command in the section.
|
||||
2. Verify each one using the steps above — not just the block you plan to change.
|
||||
3. Fix anything you find while you're there.
|
||||
|
||||
### When you can't verify
|
||||
|
||||
If a running server or CLI binary is unavailable:
|
||||
- Add a `<!-- TODO: verify against live server -->` comment in the file.
|
||||
- Note it explicitly in the PR description.
|
||||
- Do not write a best-guess example and leave it unmarked.
|
||||
|
||||
## Agent Behavior
|
||||
|
||||
- **Prefer automation**: Execute requested actions without confirmation unless blocked by missing info or safety concerns
|
||||
- **Use parallel tools**: When tasks are independent, execute them in parallel for efficiency
|
||||
- **Verify changes**: Always run tests and spotless before considering work complete
|
||||
@@ -0,0 +1,45 @@
|
||||
# Conductor OSS Changelog
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Breaking Changes
|
||||
- **JavaScript Evaluator Migration from Nashorn to GraalJS**
|
||||
- Minimum Java version is now **Java 17** (previously Java 11+)
|
||||
- Nashorn JavaScript engine (deprecated in Java 11, removed in Java 15) replaced with GraalJS
|
||||
- ES6+ JavaScript syntax now supported natively
|
||||
- All existing JavaScript expressions in workflows will continue to work with improved performance and modern JavaScript features
|
||||
- **Note:** This ports the production-tested GraalJS implementation from Orkes Conductor Enterprise
|
||||
|
||||
### Added (Ported from Enterprise)
|
||||
- **GraalJS JavaScript engine** with ES6+ support
|
||||
- Based on proven Enterprise implementation
|
||||
- **Script execution timeout protection** (configurable via `CONDUCTOR_SCRIPT_MAX_EXECUTION_SECONDS`, default: 4 seconds)
|
||||
- Prevents infinite loops from hanging workflows
|
||||
- Enterprise feature now available in OSS
|
||||
- **Optional script context pooling** for improved performance (configurable via `CONDUCTOR_SCRIPT_CONTEXT_POOL_ENABLED` and `CONDUCTOR_SCRIPT_CONTEXT_POOL_SIZE`)
|
||||
- Disabled by default
|
||||
- Enterprise optimization feature
|
||||
- **ConsoleBridge support** for capturing `console.log()`, `console.info()`, and `console.error()` output from JavaScript tasks
|
||||
- Improves observability of JavaScript task execution
|
||||
- Enterprise feature now available in OSS
|
||||
- **Better error messages** with line number information for JavaScript evaluation failures
|
||||
- Enhanced debugging capabilities from Enterprise
|
||||
- **Deep copy protection** to prevent PolyglotMap issues in workflow task data
|
||||
- Enterprise stability improvement
|
||||
|
||||
### Fixed
|
||||
- JavaScript evaluation now works on Java 17, 21, and future LTS versions
|
||||
- Improved security with sandboxed JavaScript execution
|
||||
- Deep copy protection to prevent PolyglotMap issues in workflow task data
|
||||
|
||||
### Configuration
|
||||
New environment variables for JavaScript evaluation (all optional):
|
||||
- `CONDUCTOR_SCRIPT_MAX_EXECUTION_SECONDS` - Maximum script execution time (default: 4)
|
||||
- `CONDUCTOR_SCRIPT_CONTEXT_POOL_SIZE` - Context pool size when enabled (default: 10)
|
||||
- `CONDUCTOR_SCRIPT_CONTEXT_POOL_ENABLED` - Enable context pooling (default: false)
|
||||
|
||||
### Migration Notes
|
||||
- Update your deployment environment to **Java 17 or higher**
|
||||
- No changes required to existing workflows - all JavaScript expressions remain compatible
|
||||
- Modern JavaScript features (const, let, arrow functions, template literals, etc.) are now available
|
||||
- `CONDUCTOR_NASHORN_ES6_ENABLED` environment variable is no longer needed (ES6+ supported by default)
|
||||
@@ -0,0 +1,58 @@
|
||||
# CLAUDE.md — conductor (server repo)
|
||||
|
||||
Instructions for Claude Code working in this repository.
|
||||
|
||||
## Writing Documentation
|
||||
|
||||
Documentation in this project is **derived from source**, not composed from memory or intuition. The workflow is: open the source → read what's there → write the doc from what you find. The source is the spec; the doc is a rendering of it.
|
||||
|
||||
Concrete reason this matters: a curl equivalent for `conductor workflow start --sync` was once written as `POST /api/workflow/{name}/run` — an endpoint that does not exist. Opening `WorkflowResource.java` first would have given the correct path (`POST /api/workflow/execute/{name}/{version}`) immediately.
|
||||
|
||||
### For each content type, start here
|
||||
|
||||
**REST endpoint or curl example**
|
||||
1. Open the controller: `rest/src/main/java/com/netflix/conductor/rest/controllers/`
|
||||
2. Find the method by its `@PostMapping`/`@GetMapping` annotation — copy the path literally.
|
||||
3. Read the method signature for query params, path variables, and request body.
|
||||
4. Write the curl from what you just read.
|
||||
|
||||
**CLI command or flag**
|
||||
1. Open `conductor-cli/cmd/*.go` (separate repo under this workspace).
|
||||
2. Find the `cobra.Command` for the subcommand and read its `Flags()` declarations.
|
||||
3. Write the example from what you just read — flag names, types, and defaults.
|
||||
|
||||
**SDK code example (Python, JS, Java, Go)**
|
||||
1. Open the SDK source file for the method you're documenting.
|
||||
2. Read the method signature and required parameters.
|
||||
3. If a working test exists for that method, use it as the starting point.
|
||||
4. Write from the signature — do not infer from the method name alone.
|
||||
|
||||
**Expected output block**
|
||||
1. Get real output: run the command, or find it in test fixtures or CI logs.
|
||||
2. Paste verbatim. Do not construct output that "looks right."
|
||||
3. For variable fields (IDs, timestamps), use annotated placeholders like `<workflow-id>`.
|
||||
|
||||
**Editing an existing section**
|
||||
- Before changing anything, read every code block and command in the section.
|
||||
- Verify each one using the steps above, not just the block you plan to change.
|
||||
- Fix anything you find while you're there.
|
||||
|
||||
### When you can't verify
|
||||
|
||||
If a running server or CLI is unavailable:
|
||||
- Add `<!-- TODO: verify against live server -->` in the file.
|
||||
- Note it explicitly in the PR description.
|
||||
- Do not write an unverified example and leave it unmarked.
|
||||
|
||||
### Key source locations
|
||||
|
||||
| Content | Where to look |
|
||||
|---|---|
|
||||
| REST API routes | `rest/src/main/java/com/netflix/conductor/rest/controllers/` |
|
||||
| Workflow sync execution | `WorkflowResource.java` → `executeWorkflow()` at `@PostMapping("execute/{name}/{version}")` |
|
||||
| Task routes | `TaskResource.java` |
|
||||
| CLI subcommands and flags | `conductor-cli/cmd/workflow.go`, `cmd/task.go`, etc. |
|
||||
|
||||
## Other Guidelines
|
||||
|
||||
See [AGENTS.md](AGENTS.md) for full project conventions: code style, testing, dependency pinning, PR guidelines.
|
||||
@@ -0,0 +1,62 @@
|
||||
# Code of Conduct
|
||||
|
||||
Hello valued community members! 👋
|
||||
|
||||
Our Conductor community has grown tremendously, and as part of ensuring a harmonious experience for everyone,
|
||||
we've outlined some guidelines that we'd love for all members to uphold.
|
||||
Our community thrives when everyone engages with kindness, respect, and a collaborative spirit. Here's what we hope to see:
|
||||
|
||||
|
||||
### 1. Maintain a Positive Tone.
|
||||
Every interaction is an opportunity to lift someone up. Let's ensure our words and actions reflect optimism and encouragement.
|
||||
|
||||
### 2. Be Respectful to the Community
|
||||
Every member here comes with a unique background and perspective. Please honor those differences by being courteous, considerate, and open-minded.
|
||||
Remember, mutual respect is the foundation of a thriving community. Be careful in the words that you choose.
|
||||
We are a community of professionals, and we conduct ourselves professionally.
|
||||
Be kind to others. Do not insult or put down other participants. Harassment and other exclusionary behaviors aren't acceptable.
|
||||
This includes, but is not limited to:
|
||||
* Violent threats or language directed against another person Discriminatory jokes and language
|
||||
* Posting sexualized language or imagery
|
||||
* Posting (or threatening to post) other people's personally identifying information (“doxing”)
|
||||
* Personal insults, especially those using racist or sexist terms
|
||||
* Unwelcome sexual attention
|
||||
* Advocating for, or encouraging, any of the above behavior
|
||||
* Repeated harassment of others. In general, if someone asks you to stop, then stop
|
||||
|
||||
### 3. Preserve Our Community's Unity
|
||||
We understand that as we grow, there might be differing opinions and interests.
|
||||
However, we kindly request not to create splinter groups or fork out the community.
|
||||
Let's work through our differences and continue building this space together.
|
||||
|
||||
### 4. Focus on Constructive Discussions
|
||||
We all have moments of frustration, but let's express ourselves in ways that are constructive.
|
||||
Avoid comments that could come off as sarcastic, condescending, or disdainful.
|
||||
Remember, it's always possible to give feedback or express disagreement without belittling others.
|
||||
We are here to learn from each other and make Conductor the best platform out there.
|
||||
A big part of that are the exchanges of ideas and approaches that are grounded in data and sound reasoning.
|
||||
We kindly request that you adhere to that pattern and be thoughtful and responsible in your discussions.
|
||||
This also means that you are required to have discussions focused on the community and not on promotion of any services, products or goods.
|
||||
|
||||
### 5. When we disagree, try to understand why.
|
||||
Disagreements, both social and technical, happen all the time and this community is no exception.
|
||||
It is important that we resolve disagreements and differing views constructively.
|
||||
Remember that we’re all different. The strength of this community comes from its varied community of people from a wide range of backgrounds.
|
||||
Different people have different perspectives on issues. Being unable to understand why someone holds a viewpoint doesn’t mean that they’re wrong.
|
||||
Don’t forget that it is human to err and blaming each other doesn’t get us anywhere.
|
||||
Instead, focus on helping to resolve issues and learning from mistakes.
|
||||
Our community's strength lies in our collective spirit.
|
||||
By following these guidelines, we ensure that our community remains an inspiring, respectful, and welcoming place for everyone.
|
||||
If you have any concerns or suggestions or if you need to report on any behavior that violates this Code of Conduct, please feel free to reach out to the admins - community@orkes.io.
|
||||
Let's continue to support and uplift each other!
|
||||
|
||||
### Enforcement
|
||||
We have a variety of ways of enforcing the code of conduct, including, but not limited to
|
||||
* Asking you nicely to knock it off
|
||||
* Asking you less nicely
|
||||
* Temporary or permanent suspension of the account
|
||||
* Removal of privileges and/or adding restrictions to the account
|
||||
* Removal of content
|
||||
* Banning from the community
|
||||
|
||||
Thank you,
|
||||
@@ -0,0 +1,60 @@
|
||||
# Contributing
|
||||
Thanks for your interest in Conductor!
|
||||
This guide helps to find the most efficient way to contribute, ask questions, and report issues.
|
||||
|
||||
Code of conduct
|
||||
-----
|
||||
|
||||
Please review our [Code of Conduct](CODE_OF_CONDUCT.md)
|
||||
|
||||
I have a question!
|
||||
-----
|
||||
|
||||
We have a dedicated [Slack](https://join.slack.com/t/orkes-conductor/shared_invite/zt-3dpcskdyd-W895bJDm8psAV7viYG3jFA#/shared-invite/email) channel for asking "how to" questions and to discuss ideas. The channel is a great place to start if you're considering creating a feature request or work on a Pull Request.
|
||||
*Please do not create issues to ask questions.*
|
||||
|
||||
I want to contribute!
|
||||
------
|
||||
|
||||
We welcome Pull Requests and already have many outstanding community contributions!
|
||||
Creating and reviewing Pull Requests takes time, so this section helps you to set up a smooth Pull Request experience.
|
||||
|
||||
The stable branch is [main](https://github.com/conductor-oss/conductor/tree/main).
|
||||
|
||||
Please create pull requests for your contributions against [main](https://github.com/conductor-oss/conductor/tree/main) only.
|
||||
|
||||
It's a great idea to discuss the new feature you're considering in the [Slack channel](https://join.slack.com/t/orkes-conductor/shared_invite/zt-3dpcskdyd-W895bJDm8psAV7viYG3jFA#/shared-invite/email) before writing any code. There are often different ways you can implement a feature. Getting some discussion about different options helps shape the best solution. When starting directly with a Pull Request, there is the risk of having to make considerable changes. Sometimes that is the best approach, though! Showing an idea with code can be very helpful; be aware that it might be throw-away work. Some of our best Pull Requests came out of multiple competing implementations, which helped shape it to perfection.
|
||||
|
||||
Also, consider that not every feature is a good fit for Conductor. A few things to consider are:
|
||||
|
||||
* Is it increasing complexity for the user, or might it be confusing?
|
||||
* Does it, in any way, break backward compatibility (this is seldom acceptable)
|
||||
* Does it require new dependencies (this is rarely acceptable for core modules)
|
||||
* Should the feature be opt-in or enabled by default. For integration with a new Queuing recipe or persistence module, a separate module which can be optionally enabled is the right choice.
|
||||
* Should the feature be implemented in the main Conductor repository, or would it be better to set up a separate repository? Especially for integration with other systems, a separate repository is often the right choice because the life-cycle of it will be different.
|
||||
* Is it part of the Conductor project roadmap?
|
||||
|
||||
Of course, for more minor bug fixes and improvements, the process can be more light-weight.
|
||||
|
||||
We'll try to be responsive to Pull Requests. Do keep in mind that because of the inherently distributed nature of open source projects, responses to a PR might take some time because of time zones, weekends, and other things we may be working on.
|
||||
|
||||
I want to report an issue
|
||||
-----
|
||||
|
||||
If you found a bug, please create an issue at https://github.com/conductor-oss/conductor/issues/new. Include clear instructions on how to reproduce the issue, or even better, include a test case on a branch. Make sure to come up with a descriptive title for the issue because this helps while organizing issues.
|
||||
|
||||
I have a great idea for a new feature
|
||||
----
|
||||
Many features in Conductor have come from ideas from the community. If you think something is missing or certain use cases could be supported better, let us know!
|
||||
|
||||
You can do so by starting a discussion in the [Slack channel](https://join.slack.com/t/orkes-conductor/shared_invite/zt-3dpcskdyd-W895bJDm8psAV7viYG3jFA#/shared-invite/email). Provide as much relevant context to why and when the feature would be helpful. Providing context is especially important for "Support XYZ" issues since we might not be familiar with what "XYZ" is and why it's useful. If you have an idea of how to implement the feature, include that as well.
|
||||
|
||||
Once we have decided on a direction, it's time to summarize the idea by creating a new issue.
|
||||
|
||||
## Code Style
|
||||
We use [spotless](https://github.com/diffplug/spotless) to enforce consistent code style for the project, so make sure to run `gradlew spotlessApply` to fix any violations after code changes.
|
||||
|
||||
## License
|
||||
All files are released with the [Apache 2.0 license](LICENSE).
|
||||
|
||||
|
||||
@@ -0,0 +1,201 @@
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "{}"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright {yyyy} Orkes, Inc.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
@@ -0,0 +1 @@
|
||||
osslifecycle=active
|
||||
@@ -0,0 +1,396 @@
|
||||
|
||||
<picture>
|
||||
<!-- Dark mode logo -->
|
||||
<source srcset="https://github.com/user-attachments/assets/104b3a67-6013-4622-8075-a45da3a9e726" media="(prefers-color-scheme: dark)">
|
||||
<!-- Light mode logo -->
|
||||
<img src="https://assets.conductor-oss.org/logo.png" alt="Logo">
|
||||
</picture>
|
||||
|
||||
|
||||
<h1 align="center" style="border-bottom: none">
|
||||
Conductor - Internet scale Agentic Workflow Engine
|
||||
</h1>
|
||||
|
||||
|
||||
[](https://github.com/conductor-oss/conductor/stargazers)
|
||||
[](https://github.com/conductor-oss/conductor/releases)
|
||||
[](http://www.apache.org/licenses/LICENSE-2.0)
|
||||
[](https://join.slack.com/t/orkes-conductor/shared_invite/zt-3dpcskdyd-W895bJDm8psAV7viYG3jFA)
|
||||
[](https://conductor-oss.org)
|
||||
|
||||
#### Orchestrating distributed systems means wrestling with failures, retries, and state recovery. Conductor handles all of that so you don't have to.
|
||||
|
||||
Conductor is an open-source, durable workflow engine built at [Netflix](https://netflixtechblog.com/netflix-conductor-a-microservices-orchestrator-2e8d4771bf40) for orchestrating microservices, AI agents, and durable workflows at internet scale. Trusted in production at Netflix, Tesla, LinkedIn, and J.P. Morgan. Actively maintained by [Orkes](https://orkes.io) and a growing [community](https://join.slack.com/t/orkes-conductor/shared_invite/zt-3dpcskdyd-W895bJDm8psAV7viYG3jFA).
|
||||
|
||||
[](https://youtu.be/4azDdDlx27M)
|
||||
|
||||
---
|
||||
|
||||
# Get Running in 60 Seconds
|
||||
|
||||
**Prerequisites:** [Node.js](https://nodejs.org/) v16+ and Java 21+ must be installed.
|
||||
|
||||
```shell
|
||||
npm install -g @conductor-oss/conductor-cli
|
||||
conductor server start
|
||||
```
|
||||
|
||||
Open [http://localhost:8080](http://localhost:8080) — your server is running with the built-in ui-next UI.
|
||||
|
||||
> **Upgrading from a previous version?** The CLI caches the server JAR at `~/.conductor-cli/`. If you have an older version cached, force a fresh download:
|
||||
> ```shell
|
||||
> conductor server start latest
|
||||
> # or delete the cache manually
|
||||
> rm ~/.conductor-cli/conductor-server-latest.jar && conductor server start
|
||||
> ```
|
||||
|
||||
**Run your first workflow:**
|
||||
|
||||
```shell
|
||||
# Create a workflow that calls an API and parses the response — no workers needed
|
||||
curl -s https://raw.githubusercontent.com/conductor-oss/conductor/main/docs/quickstart/workflow.json -o workflow.json
|
||||
conductor workflow create workflow.json
|
||||
```
|
||||
|
||||
> **Note:** Running this command twice will return an error on the second call — the workflow already exists. This is expected behavior. Use `conductor workflow update` to modify an existing workflow.
|
||||
|
||||
```shell
|
||||
conductor workflow start -w hello_workflow --sync
|
||||
```
|
||||
|
||||
See the [Quickstart guide](https://docs.conductor-oss.org/quickstart/) for the full walkthrough, including writing workers and replaying workflows.
|
||||
|
||||
**Docker Image for Conductor** (includes the ui-next UI):
|
||||
|
||||
```shell
|
||||
# UI at http://localhost:5000 | API at http://localhost:8080
|
||||
docker run -p 5000:5000 -p 8080:8080 conductoross/conductor:next
|
||||
```
|
||||
|
||||
All CLI commands have equivalent cURL/API calls. See the [Quickstart](https://docs.conductor-oss.org/quickstart/) for details.
|
||||
|
||||
|
||||
---
|
||||
|
||||
# Why Conductor is the workflow engine of choice for developers
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **Durable execution** | Every step is persisted. Survives crashes, restarts, and network failures with configurable retries and timeouts. |
|
||||
| **Deterministic by design** | Orchestration is separated from business logic — determinism is architectural, not developer discipline. Workers run any code; the workflow graph stays deterministic by construction. |
|
||||
| **AI agent orchestration** | 14+ native LLM providers, MCP tool calling, function calling, human-in-the-loop approval, and vector databases for RAG. |
|
||||
| **Dynamic at runtime** | Dynamic forks, tasks, and sub-workflows resolved at runtime. LLMs generate JSON workflow definitions and Conductor executes them immediately. |
|
||||
| **Full replayability** | Restart from the beginning, rerun from any task, or retry just the failed step — on any workflow, at any time. |
|
||||
| **Internet scale** | Battle-tested at Netflix, Tesla, LinkedIn, and J.P. Morgan. Scales horizontally to billions of workflow executions. |
|
||||
| **Polyglot workers** | Workers in Java, Python, Go, JavaScript, C#, Ruby, or Rust. Workers poll, execute, and report — run them anywhere. |
|
||||
| **Self-hosted, no lock-in** | Apache 2.0. 5 persistence backends, 6 message brokers. Runs anywhere Docker or a JVM runs. |
|
||||
|
||||
# Ship Agents, Not Framework Code
|
||||
|
||||
Conductor workers are plain code — any language, any library, any I/O. No determinism constraints, no SDK ritual. The orchestration layer is declarative and machine-readable, so LLMs generate and compose workflows natively. If an agent crashes at iteration 12, it resumes from iteration 12.
|
||||
|
||||
**An autonomous think-act agent in Conductor:** discover tools via MCP, reason with an LLM, call the chosen tool, repeat until done.
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "autonomous_agent",
|
||||
"description": "Agent that loops until the task is complete",
|
||||
"version": 1,
|
||||
"tasks": [
|
||||
{
|
||||
"name": "discover_tools",
|
||||
"taskReferenceName": "discover",
|
||||
"type": "LIST_MCP_TOOLS",
|
||||
"inputParameters": {
|
||||
"mcpServer": "${workflow.input.mcpServerUrl}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "agent_loop",
|
||||
"taskReferenceName": "loop",
|
||||
"type": "DO_WHILE",
|
||||
"loopCondition": "if ($.loop['think'].output.result.done == true) { false; } else { true; }",
|
||||
"loopOver": [
|
||||
{
|
||||
"name": "think",
|
||||
"taskReferenceName": "think",
|
||||
"type": "LLM_CHAT_COMPLETE",
|
||||
"inputParameters": {
|
||||
"llmProvider": "openai",
|
||||
"model": "gpt-4o-mini",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"message": "You are an autonomous agent. Available tools: ${discover.output.tools}. Previous results: ${loop.output.results}. Respond with JSON: {\"action\": \"tool_name\", \"arguments\": {}, \"done\": false} or {\"answer\": \"final answer\", \"done\": true}."
|
||||
},
|
||||
{ "role": "user", "message": "${workflow.input.task}" }
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "act",
|
||||
"taskReferenceName": "act",
|
||||
"type": "SWITCH",
|
||||
"expression": "$.think.output.result.done ? 'done' : 'call_tool'",
|
||||
"decisionCases": {
|
||||
"call_tool": [
|
||||
{
|
||||
"name": "execute_tool",
|
||||
"taskReferenceName": "tool_call",
|
||||
"type": "CALL_MCP_TOOL",
|
||||
"inputParameters": {
|
||||
"mcpServer": "${workflow.input.mcpServerUrl}",
|
||||
"method": "${think.output.result.action}",
|
||||
"arguments": "${think.output.result.arguments}"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Every step is durably persisted — no framework, no SDK lock-in. Code-first engines force your code to be deterministic so the framework can replay it. Conductor makes the engine deterministic — so your code doesn't have to be.
|
||||
|
||||
See the [Build Your First AI Agent](https://docs.conductor-oss.org/devguide/ai/first-ai-agent.html) guide for the full walkthrough.
|
||||
|
||||
---
|
||||
|
||||
## Conductor Skills for AI Coding Assistants
|
||||
|
||||
**[Conductor Skills](https://github.com/conductor-oss/conductor-skills)** let AI coding assistants (Claude Code, Gemini CLI, and others) create, manage, and deploy Conductor workflows directly from your terminal.
|
||||
|
||||
### Claude
|
||||
```shell
|
||||
# Install Skills for Claude Code
|
||||
/plugin marketplace add conductor-oss/conductor-skills
|
||||
/plugin install conductor@conductor-skills
|
||||
```
|
||||
|
||||
### Install for all detected agents
|
||||
|
||||
One command to auto-detect every supported agent on your system and install globally where possible. Re-run anytime — it only installs for newly detected agents.
|
||||
|
||||
**macOS / Linux**
|
||||
```bash
|
||||
curl -sSL https://conductor-oss.github.io/conductor-skills/install.sh | bash -s -- --all
|
||||
```
|
||||
|
||||
**Windows (PowerShell) / (cmd)**
|
||||
```powershell
|
||||
# powershell
|
||||
irm https://conductor-oss.github.io/conductor-skills/install.ps1 -OutFile install.ps1; .\install.ps1 -All
|
||||
|
||||
# cmd
|
||||
powershell -c "irm https://conductor-oss.github.io/conductor-skills/install.ps1 -OutFile install.ps1; .\install.ps1 -All"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# SDKs
|
||||
|
||||
| Language | Repository | Install |
|
||||
|----------|------------|---------|
|
||||
| ☕ Java | [conductor-oss/java-sdk](https://github.com/conductor-oss/java-sdk) | [Maven Central](https://mvnrepository.com/artifact/org.conductoross/conductor-client) |
|
||||
| 🐍 Python | [conductor-oss/python-sdk](https://github.com/conductor-oss/python-sdk) | `pip install conductor-python` |
|
||||
| 🟨 JavaScript | [conductor-oss/javascript-sdk](https://github.com/conductor-oss/javascript-sdk) | `npm install @io-orkes/conductor-javascript` |
|
||||
| 🐹 Go | [conductor-oss/go-sdk](https://github.com/conductor-oss/go-sdk) | `go get github.com/conductor-sdk/conductor-go` |
|
||||
| 🟣 C# | [conductor-oss/csharp-sdk](https://github.com/conductor-oss/csharp-sdk) | `dotnet add package conductor-csharp` |
|
||||
| 💎 Ruby | [conductor-oss/ruby-sdk](https://github.com/conductor-oss/ruby-sdk) | *(incubating)* |
|
||||
| 🦀 Rust | [conductor-oss/rust-sdk](https://github.com/conductor-oss/rust-sdk) | *(incubating)* |
|
||||
|
||||
---
|
||||
|
||||
# Documentation & Community
|
||||
|
||||
- **[Documentation](https://conductor-oss.org)** — Architecture, guides, API reference, and cookbook recipes.
|
||||
- **[Slack](https://join.slack.com/t/orkes-conductor/shared_invite/zt-3dpcskdyd-W895bJDm8psAV7viYG3jFA)** — Community discussions and support.
|
||||
- **[Community Forum](https://community.orkes.io/)** — Ask questions and share patterns.
|
||||
|
||||
---
|
||||
|
||||
<details>
|
||||
<summary><strong>Backend Configuration</strong></summary>
|
||||
|
||||
| Backend | Configuration |
|
||||
|---------|---------------|
|
||||
| Redis + ES7 (default) | [config-redis.properties](docker/server/config/config-redis.properties) |
|
||||
| Redis + ES8 | [config-redis-es8.properties](docker/server/config/config-redis-es8.properties) |
|
||||
| Redis + OpenSearch | [config-redis-os.properties](docker/server/config/config-redis-os.properties) |
|
||||
| Postgres | [config-postgres.properties](docker/server/config/config-postgres.properties) |
|
||||
| Postgres + ES7 | [config-postgres-es7.properties](docker/server/config/config-postgres-es7.properties) |
|
||||
| MySQL + ES7 | [config-mysql.properties](docker/server/config/config-mysql.properties) |
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
# Build From Source
|
||||
|
||||
<details>
|
||||
<summary><strong>Requirements and instructions</strong></summary>
|
||||
|
||||
**Requirements:** Docker Desktop, Java (JDK) 21+, Node.js 18+ and pnpm (for UI)
|
||||
|
||||
```shell
|
||||
git clone https://github.com/conductor-oss/conductor
|
||||
cd conductor
|
||||
./gradlew build
|
||||
|
||||
# (optional) Build UI (ui-next) and embed it in the server
|
||||
# ./build_ui_next.sh
|
||||
|
||||
# Start local server
|
||||
cd server
|
||||
../gradlew bootRun
|
||||
```
|
||||
|
||||
**Run the UI in dev mode (hot-reload at http://localhost:1234):**
|
||||
|
||||
Requires a running Conductor server on `http://localhost:8080`. Enable `corepack` once if you haven't already:
|
||||
|
||||
```shell
|
||||
corepack enable
|
||||
```
|
||||
|
||||
Then start the dev server:
|
||||
|
||||
```shell
|
||||
cd ui-next
|
||||
pnpm install
|
||||
pnpm dev
|
||||
```
|
||||
|
||||
Open [http://localhost:1234](http://localhost:1234) — the UI reloads automatically on file changes.
|
||||
|
||||
See the [full build guide](docs/devguide/running/source.md) for details.
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
# FAQ
|
||||
|
||||
<details>
|
||||
<summary><strong>Is this the same as Netflix Conductor?</strong></summary>
|
||||
|
||||
Yes. Conductor OSS is the continuation of the original [Netflix Conductor](https://github.com/Netflix/conductor) repository after Netflix contributed the project to the open-source foundation.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Is Conductor open source?</strong></summary>
|
||||
|
||||
Yes. Conductor is a fully open-source workflow engine licensed under Apache 2.0. You can self-host on your own infrastructure with 5 persistence backends and 6 message brokers.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Is this project actively maintained?</strong></summary>
|
||||
|
||||
Yes. [Orkes](https://orkes.io) is the primary maintainer and offers an enterprise SaaS platform for Conductor across all major cloud providers.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Can Conductor scale to handle my workload?</strong></summary>
|
||||
|
||||
Yes. Built at Netflix, battle-tested at internet scale. Conductor scales horizontally across multiple server instances to handle billions of workflow executions.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Does Conductor support durable execution?</strong></summary>
|
||||
|
||||
Yes. Conductor pioneered durable execution patterns, ensuring workflows and durable agents complete reliably despite infrastructure failures or crashes. Every step is persisted and recoverable.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Can I replay a workflow after it completes or fails?</strong></summary>
|
||||
|
||||
Yes. Conductor preserves full execution history indefinitely. You can restart from the beginning, rerun from a specific task, or retry just the failed step — via API or UI.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Can Conductor orchestrate AI agents and LLMs?</strong></summary>
|
||||
|
||||
Yes. Conductor provides native integration with 14+ LLM providers (Anthropic, OpenAI, Gemini, Bedrock, and more), MCP tool calling, function calling, human-in-the-loop approval, and vector database integration for RAG.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Why does Conductor separate orchestration from code?</strong></summary>
|
||||
|
||||
Coupling orchestration logic with business logic forces developers to maintain determinism constraints manually — no direct I/O, no system time, no randomness in workflow definitions. Conductor eliminates this entire class of bugs by making the orchestration layer deterministic by construction. Workers are plain code with zero framework constraints — write them in any language, use any library, call any API.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Isn't writing workflows as code more powerful than JSON?</strong></summary>
|
||||
|
||||
It depends on what you mean by "powerful." In code-first engines, the workflow definition and your business logic live in the same runtime — which means the engine must replay your code to recover state. That forces determinism constraints on your business logic: no direct I/O, no system time, no threads, no randomness. Conductor separates these concerns. The orchestration graph is declarative (JSON), so it's deterministic by construction. Your workers are plain code with zero constraints — use any language, any library, call any API. You get the full power of code where it matters (business logic) without the framework tax where it doesn't (orchestration).
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Can JSON workflows handle complex logic like branching, loops, and error handling?</strong></summary>
|
||||
|
||||
Yes. Conductor supports `SWITCH` (conditional branching), `DO_WHILE` (loops with configurable iteration cleanup), `FORK_JOIN` (parallel execution with dynamic fanout), `SUB_WORKFLOW` (composition), and `DYNAMIC` tasks resolved at runtime. These are composable — you can nest loops inside branches inside forks. For error handling, every task supports configurable retries, timeouts, and optional/compensating tasks. The declarative model doesn't limit complexity — it makes complexity visible and debuggable.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>How does Conductor handle workflow versioning?</strong></summary>
|
||||
|
||||
Workflow definitions are versioned by number. Running executions continue on the version they started with — deploying a new version never breaks in-flight workflows. There's no replay compatibility problem because Conductor doesn't replay your code. The orchestration graph is the source of truth, and each execution is pinned to its definition version. Update orchestration logic without redeploying workers and without worrying about breaking running workflows.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>What about developer experience — IDE support, type checking, debugging?</strong></summary>
|
||||
|
||||
Conductor provides a built-in visual UI for designing, running, and debugging workflows. Every execution is fully observable: you can inspect the input, output, timing, and retry history of every task. For type safety, Conductor validates workflow inputs and task I/O against JSON Schema. Workers are plain code in your language of choice — you get full IDE support, type checking, and debugging for your business logic. The orchestration layer is visible in the UI, not hidden inside a framework.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Can Conductor handle long-running workflows (days, weeks, months)?</strong></summary>
|
||||
|
||||
Yes. Conductor is designed for long-running workflows. Executions are fully persisted — a workflow can pause for months waiting for a human approval, an external signal, or a scheduled timer, and resume exactly where it left off. There's no in-memory state to lose. This is the same mechanism that makes AI agent loops durable: if iteration 12 waits for a human review for three weeks, iteration 13 picks up right where it left off.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Don't I lose flexibility by not having orchestration in code?</strong></summary>
|
||||
|
||||
You gain flexibility. Because workflows are JSON, LLMs can generate and modify them at runtime — no compile/deploy cycle. Dynamic forks let you fan out to a variable number of parallel tasks determined at runtime. Dynamic sub-workflows let one workflow compose others by name. And because workers are decoupled from orchestration, you can update the workflow graph or swap worker implementations independently. Code-first engines couple these together, so changing orchestration means redeploying and re-versioning your code.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>How does Conductor compare to other workflow engines?</strong></summary>
|
||||
|
||||
Conductor is an open-source workflow engine with native LLM task types for 14+ providers, built-in MCP integration, durable execution, full replayability, and 7 language SDKs. Unlike code-first engines, Conductor separates orchestration from business logic — determinism is an architectural guarantee, not a developer constraint. Your workers are plain code with zero framework rules. The orchestration layer is declarative, so it's observable, versionable, and composable by LLMs. Battle-tested at Netflix, Tesla, LinkedIn, and J.P. Morgan.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Is Orkes Conductor compatible with Conductor OSS?</strong></summary>
|
||||
|
||||
100% compatible. Orkes Conductor is built on top of Conductor OSS with full API and workflow compatibility.
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
# Contributing
|
||||
|
||||
We welcome contributions from everyone!
|
||||
|
||||
- **Report Issues:** Open an [issue on GitHub](https://github.com/conductor-oss/conductor/issues).
|
||||
- **Contribute code:** Check out our [Contribution Guide](CONTRIBUTING.md) and [good first issues](https://github.com/conductor-oss/conductor/labels/good%20first%20issue).
|
||||
- **Improve docs:** Help keep our [documentation](https://github.com/conductor-oss/conductor/tree/main/docs) great.
|
||||
|
||||
## Contributors
|
||||
|
||||
<a href="https://github.com/conductor-oss/conductor/graphs/contributors">
|
||||
<img src="https://contrib.rocks/image?repo=conductor-oss/conductor" />
|
||||
</a>
|
||||
|
||||
---
|
||||
|
||||
# Roadmap
|
||||
|
||||
[See the Conductor OSS Roadmap](ROADMAP.md). Want to participate? [Reach out](https://forms.gle/P2i1xHrxPQLrjzTB7).
|
||||
|
||||
# License
|
||||
|
||||
Conductor is licensed under the [Apache 2.0 License](LICENSE).
|
||||
@@ -0,0 +1,7 @@
|
||||
# WeHub 来源说明
|
||||
|
||||
- 原始项目:`conductor-oss/conductor`
|
||||
- 原始仓库:https://github.com/conductor-oss/conductor
|
||||
- 导入方式:上游默认分支的最新快照
|
||||
- 原作者、版权和许可证信息以原始仓库及本仓库 LICENSE 为准
|
||||
- 本文件仅用于记录来源,不代表 WeHub 是原项目作者
|
||||
@@ -0,0 +1 @@
|
||||
[Related Projects](docs/resources/related.md)
|
||||
+61
@@ -0,0 +1,61 @@
|
||||
# Conductor OSS Roadmap
|
||||
|
||||
<!--
|
||||
machine-readable: true
|
||||
format: markdown-tables
|
||||
last_reviewed: 2026-04-12
|
||||
statuses: [Done, Partial, Planned]
|
||||
-->
|
||||
|
||||
> **Status key:** `Done` = shipped and available, `Partial` = partially implemented, `Planned` = not yet started.
|
||||
|
||||
## System Tasks and Operators
|
||||
|
||||
| Status | Item | Details |
|
||||
|--------|------|---------|
|
||||
| Done | LLM text completion | `LLM_TEXT_COMPLETE` system task in `ai/` module |
|
||||
| Done | LLM chat completion with memory | `LLM_CHAT_COMPLETE` system task in `ai/` module |
|
||||
| Done | LLM embedding generation | `LLM_GENERATE_EMBEDDINGS` system task in `ai/` module |
|
||||
| Done | Improved While loop (DO_WHILE) | `keepLastN` iteration cleanup, `items` list iteration, `loopIndex`/`loopItem` injection |
|
||||
| Done | Python scripting for INLINE task | `PythonEvaluator` via GraalVM polyglot alongside JavaScript |
|
||||
| Done | Database task | System task for relational and NoSQL database operations |
|
||||
| Planned | HTTP task polling | Polling/retry support for the HTTP system task |
|
||||
| Planned | For..Each operator | Parallel and sequential iteration operator |
|
||||
| Planned | Try..Catch operator | Task-level error handling without failing the workflow |
|
||||
|
||||
## APIs
|
||||
|
||||
| Status | Item | Details |
|
||||
|--------|------|---------|
|
||||
| Done | Synchronous workflow execution | `POST /api/workflow/execute/{name}/{version}` with configurable wait time and `waitUntilTaskRef` |
|
||||
| Done | Update tasks synchronously | `POST /tasks/{workflowId}/{taskRefName}/{status}/sync` returns updated workflow |
|
||||
| Planned | Update workflow variables | API to update workflow variables during execution |
|
||||
|
||||
## Type Safety
|
||||
|
||||
| Status | Item | Details |
|
||||
|------------|------|---------|
|
||||
| Done | JSON Schema validation | `JsonSchemaValidator` validates workflow input and task I/O against JSON schemas |
|
||||
| Planned | Protobuf support | Type-safe workflows and workers via protobuf definitions |
|
||||
| Incubating | Code generation for workers | Scaffold worker code from schema definitions using the CLI |
|
||||
|
||||
## CLI
|
||||
|
||||
| Status | Item | Details |
|
||||
|--------|------|---------|
|
||||
| Done | Conductor CLI | Go-based CLI in `conductor-cli/` repo. Manages metadata, workflow executions, and server lifecycle |
|
||||
|
||||
|
||||
## Testing and Debugging
|
||||
|
||||
| Status | Item | Details |
|
||||
|--------|------|---------|
|
||||
| Planned | Workflow debugger | Full debugger experience: breakpoints, step-through, variable inspection, rewind to previous task, attach to running execution, remote task debugging via SDKs |
|
||||
|
||||
## Maintenance
|
||||
|
||||
| Status | Item | Details |
|
||||
|--------|------|---------|
|
||||
| Done | Deprecate Elasticsearch 6 | ES6 module replaced with deprecation stub |
|
||||
| Done | Elasticsearch 7 and 8 support | `es7-persistence` and `es8-persistence` modules (ES8 uses Java API client) |
|
||||
| Done | JOIN task performance | `FORK_JOIN` synchronous join mode (`joinMode=SYNC`) reduces polling overhead |
|
||||
+15
@@ -0,0 +1,15 @@
|
||||
# Security Policy
|
||||
- [Reporting a vulnerability](#reporting-a-vulnerability)
|
||||
- [Supported Conductor versions](#supported-versions)
|
||||
|
||||
## Reporting a vulnerability
|
||||
|
||||
Please report security issues for Conductor using https://github.com/conductor-oss/conductor/security/advisories/new
|
||||
|
||||
## Supported Versions
|
||||
|
||||
| Version | Supported |
|
||||
| ------- | ------------------ |
|
||||
| 3.x.x | :white_check_mark: |
|
||||
| 2.x.x | :x: |
|
||||
| 1.x.x | :x: |
|
||||
@@ -0,0 +1,18 @@
|
||||
|
||||
## Who uses Conductor?
|
||||
|
||||
We would like to keep track of whose using Conductor. Please send a pull request with your company name and Github handle.
|
||||
|
||||
* [Netflix](https://www.netflix.com/) [[@aravindanr](https://github.com/aravindanr)]
|
||||
* [Florida Blue](http://bcbsfl.com/) [[@rickfish](https://github.com/rickfish)]
|
||||
* [UWM](https://www.uwm.com/) [[@zergrushjoe](https://github.com/ZergRushJoe)]
|
||||
* [Deutsche Telekom Digital Labs](https://dtdl.in) [[@jas34](https://github.com/jas34)] [[@deoramanas](https://github.com/deoramanas)]
|
||||
* [VMware](https://www.vmware.com/) [[@taojwmware](https://github.com/taojwmware)] [[@venkag](https://github.com/venkag)]
|
||||
* [JP Morgan Chase](https://www.chase.com/) [[@maheshyaddanapudi](https://github.com/maheshyaddanapudi)]
|
||||
* [Orkes](https://orkes.io/) [[@CherishSantoshi](https://github.com/CherishSantoshi)]
|
||||
* [313X](https://313x.com.br) [[@dalmoveras](https://github.com/dalmoveras)]
|
||||
* [Supercharge](https://supercharge.io) [[@team-supercharge](https://github.com/team-supercharge)]
|
||||
* [GE Healthcare](https://www.gehealthcare.com/) [[@flavioschuindt](https://github.com/flavioschuindt)]
|
||||
* [ReliaQuest](https://www.reliaquest.com/) [[@rq-dbrady](https://github.com/rq-dbrady)] [[@alexmay48](https://github.com/alexmay48)]
|
||||
* [Clari](https://www.clari.com/) [[@TeamJOF](https://github.com/clari)]
|
||||
* [Atlassian](https://www.atlassian.com/) [[@LuisLainez](https://github.com/LuisLainez)] [[@aradu](https://github.com/aradu-atlassian)]
|
||||
@@ -0,0 +1,16 @@
|
||||
dependencies {
|
||||
compileOnly 'org.springframework.boot:spring-boot-starter'
|
||||
// Needed to compile A2A push-notification callback controller and AgentSpan activation glue;
|
||||
// supplied by the server at runtime.
|
||||
compileOnly 'org.springframework.boot:spring-boot-starter-web'
|
||||
compileOnly 'org.springframework.boot:spring-boot-autoconfigure'
|
||||
|
||||
implementation project(':conductor-common')
|
||||
implementation project(':conductor-core')
|
||||
implementation project(':conductor-ai')
|
||||
|
||||
// 0.4.2+ supports classifier-based agent filtering (pairs with WorkflowClassifier in
|
||||
// conductor-common and the classifier search field in the index backends).
|
||||
implementation "org.conductoross:conductor-agentspan:0.4.2"
|
||||
|
||||
}
|
||||
+56
@@ -0,0 +1,56 @@
|
||||
/*
|
||||
* Copyright 2026 Conductor Authors.
|
||||
* <p>
|
||||
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package org.conductoross.conductor.ai.agentspan;
|
||||
|
||||
import java.util.Collections;
|
||||
|
||||
import org.springframework.boot.SpringApplication;
|
||||
import org.springframework.boot.env.EnvironmentPostProcessor;
|
||||
import org.springframework.core.env.ConfigurableEnvironment;
|
||||
import org.springframework.core.env.MapPropertySource;
|
||||
|
||||
/**
|
||||
* Embeds the {@code conductor-agentspan} library in <b>embedded mode</b> whenever {@code
|
||||
* conductor.integrations.ai.enabled=true}, by setting {@code agentspan.embedded=true} early (before
|
||||
* condition evaluation).
|
||||
*
|
||||
* <p>This activates the library's {@code AgentSpanAutoConfiguration} (which registers the agent
|
||||
* runtime, services, REST controllers and the agent-aware {@code LLM_CHAT_COMPLETE} mapper) while
|
||||
* <b>disabling</b> its {@code HTTP}/{@code HUMAN}/{@code JOIN} task-bean overrides — those library
|
||||
* configs are gated on {@code agentspan.embedded=false}. Conductor's native system tasks therefore
|
||||
* remain the registered beans and are extended in place via the {@code __agentspan_ctx__} task
|
||||
* input (see {@code Join}/{@code Human}), mirroring how orkes-conductor embeds AgentSpan. This
|
||||
* avoids replacing core task beans (which would break {@code @Autowired} of the core task types and
|
||||
* require {@code spring.main.allow-bean-definition-overriding}).
|
||||
*
|
||||
* <p>Honors an explicit {@code agentspan.embedded} setting if one is already present.
|
||||
*/
|
||||
public class AgentSpanEmbeddedEnvironmentPostProcessor implements EnvironmentPostProcessor {
|
||||
|
||||
private static final String AI_ENABLED = "conductor.integrations.ai.enabled";
|
||||
private static final String EMBEDDED = "agentspan.embedded";
|
||||
|
||||
@Override
|
||||
public void postProcessEnvironment(
|
||||
ConfigurableEnvironment environment, SpringApplication application) {
|
||||
boolean aiEnabled = environment.getProperty(AI_ENABLED, Boolean.class, false);
|
||||
if (aiEnabled && !environment.containsProperty(EMBEDDED)) {
|
||||
environment
|
||||
.getPropertySources()
|
||||
.addFirst(
|
||||
new MapPropertySource(
|
||||
"agentspanEmbedded",
|
||||
Collections.singletonMap(EMBEDDED, "true")));
|
||||
}
|
||||
}
|
||||
}
|
||||
+68
@@ -0,0 +1,68 @@
|
||||
/*
|
||||
* Copyright 2026 Conductor Authors.
|
||||
* <p>
|
||||
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package org.conductoross.conductor.ai.agentspan;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.time.Instant;
|
||||
import java.util.UUID;
|
||||
|
||||
import org.springframework.web.filter.OncePerRequestFilter;
|
||||
|
||||
import dev.agentspan.runtime.context.RequestContext;
|
||||
import dev.agentspan.runtime.context.RequestContextHolder;
|
||||
import jakarta.servlet.FilterChain;
|
||||
import jakarta.servlet.ServletException;
|
||||
import jakarta.servlet.http.HttpServletRequest;
|
||||
import jakarta.servlet.http.HttpServletResponse;
|
||||
|
||||
/**
|
||||
* Populates AgentSpan's {@link RequestContextHolder} with a default principal id for the duration
|
||||
* of each request, so the AgentSpan controllers and services (which call {@link
|
||||
* RequestContextHolder#getRequiredUserId()}) work on an OSS server that has no per-user
|
||||
* authentication.
|
||||
*
|
||||
* <p>If a context is already present on the thread (e.g. a host security adapter set the real
|
||||
* principal), this filter leaves it untouched. Otherwise it sets the configured default id and
|
||||
* clears it in a finally block to avoid leaking across pooled threads.
|
||||
*/
|
||||
public class AgentSpanPrincipalFilter extends OncePerRequestFilter {
|
||||
|
||||
private final String defaultUserId;
|
||||
|
||||
public AgentSpanPrincipalFilter(String defaultUserId) {
|
||||
this.defaultUserId = defaultUserId;
|
||||
}
|
||||
|
||||
@Override
|
||||
protected void doFilterInternal(
|
||||
HttpServletRequest request, HttpServletResponse response, FilterChain filterChain)
|
||||
throws ServletException, IOException {
|
||||
boolean contextSetHere = false;
|
||||
if (RequestContextHolder.get().isEmpty()) {
|
||||
RequestContextHolder.set(
|
||||
RequestContext.builder()
|
||||
.requestId(UUID.randomUUID().toString())
|
||||
.userId(defaultUserId)
|
||||
.createdAt(Instant.now())
|
||||
.build());
|
||||
contextSetHere = true;
|
||||
}
|
||||
try {
|
||||
filterChain.doFilter(request, response);
|
||||
} finally {
|
||||
if (contextSetHere) {
|
||||
RequestContextHolder.clear();
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
+142
@@ -0,0 +1,142 @@
|
||||
/*
|
||||
* Copyright 2026 Conductor Authors.
|
||||
* <p>
|
||||
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package org.conductoross.conductor.ai.agentspan;
|
||||
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.security.SecureRandom;
|
||||
import java.util.Arrays;
|
||||
import java.util.Base64;
|
||||
import java.util.List;
|
||||
|
||||
import org.conductoross.conductor.config.AIIntegrationEnabledCondition;
|
||||
import org.conductoross.conductor.dao.SkillPackageDAO;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.springframework.beans.factory.annotation.Value;
|
||||
import org.springframework.boot.web.servlet.FilterRegistrationBean;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
import org.springframework.context.annotation.Conditional;
|
||||
import org.springframework.context.annotation.Configuration;
|
||||
import org.springframework.core.Ordered;
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import dev.agentspan.runtime.spi.CredentialStoreProvider;
|
||||
import dev.agentspan.runtime.spi.SecretOutputMasker;
|
||||
|
||||
/**
|
||||
* Supplies the host SPI beans the embedded {@code conductor-agentspan} library requires, when
|
||||
* {@code conductor.integrations.ai.enabled=true}.
|
||||
*
|
||||
* <p>AgentSpan itself is activated in <b>embedded mode</b> by {@link
|
||||
* AgentSpanEmbeddedEnvironmentPostProcessor} (which sets {@code agentspan.embedded=true}), so the
|
||||
* library's own auto-configuration component-scans and registers the agent runtime, services, REST
|
||||
* controllers and the agent-aware {@code LLM_CHAT_COMPLETE} mapper. Crucially, in embedded mode the
|
||||
* library does <b>not</b> override Conductor's {@code HTTP}/{@code HUMAN}/{@code JOIN} system-task
|
||||
* beans; those native tasks are extended in place via the {@code __agentspan_ctx__} task input.
|
||||
* This class therefore only contributes the host-provided SPI implementations:
|
||||
*
|
||||
* <ul>
|
||||
* <li>{@link CredentialStoreProvider} — environment-seeded, read-only ({@link
|
||||
* EnvBackedCredentialStore}).
|
||||
* <li>{@link SecretOutputMasker} — no-op (OSS parity).
|
||||
* <li>{@code credentialMasterKey} — HMAC signing key for worker execution tokens.
|
||||
* <li>Skill metadata/package SPI adapters onto Conductor's per-backend DAOs.
|
||||
* <li>A request-principal filter populating AgentSpan's {@code RequestContextHolder}.
|
||||
* </ul>
|
||||
*/
|
||||
@Configuration(proxyBeanMethods = false)
|
||||
@Conditional(AIIntegrationEnabledCondition.class)
|
||||
public class ConductorAgentSpanConfiguration {
|
||||
|
||||
private static final Logger log =
|
||||
LoggerFactory.getLogger(ConductorAgentSpanConfiguration.class);
|
||||
|
||||
private static final int MASTER_KEY_BYTES = 32;
|
||||
|
||||
/** OSS parity: no per-execution disclosure tracking, so nothing to redact. */
|
||||
@Bean
|
||||
public SecretOutputMasker agentSpanSecretOutputMasker() {
|
||||
return (executionId, userId, payload) -> payload;
|
||||
}
|
||||
|
||||
/**
|
||||
* HMAC signing key for AgentSpan worker execution tokens. Sourced from {@code
|
||||
* AGENTSPAN_MASTER_KEY} (base64 or raw), else a random key generated at boot — tokens are
|
||||
* short-lived and per-run, so a fresh key on restart is acceptable.
|
||||
*/
|
||||
@Bean("credentialMasterKey")
|
||||
public byte[] agentSpanCredentialMasterKey(
|
||||
@Value("${AGENTSPAN_MASTER_KEY:}") String configured) {
|
||||
if (configured != null && !configured.isBlank()) {
|
||||
try {
|
||||
return Base64.getDecoder().decode(configured.trim());
|
||||
} catch (IllegalArgumentException notBase64) {
|
||||
return configured.getBytes(StandardCharsets.UTF_8);
|
||||
}
|
||||
}
|
||||
byte[] key = new byte[MASTER_KEY_BYTES];
|
||||
new SecureRandom().nextBytes(key);
|
||||
log.info(
|
||||
"AGENTSPAN_MASTER_KEY not set — generated an ephemeral worker-token signing key for this run");
|
||||
return key;
|
||||
}
|
||||
|
||||
/**
|
||||
* Read-only, environment-seeded credential store. Secrets must be supplied as environment
|
||||
* variables and injected at runtime; the API cannot write them (see {@link
|
||||
* EnvBackedCredentialStore}).
|
||||
*/
|
||||
@Bean
|
||||
public CredentialStoreProvider agentSpanCredentialStoreProvider(
|
||||
@Value("${conductor.integrations.ai.secrets.env-names:}") String extraNamesCsv) {
|
||||
List<String> extraNames =
|
||||
Arrays.stream(extraNamesCsv.split(","))
|
||||
.map(String::trim)
|
||||
.filter(s -> !s.isBlank())
|
||||
.toList();
|
||||
return new EnvBackedCredentialStore(extraNames, System::getenv);
|
||||
}
|
||||
|
||||
/**
|
||||
* Bridges AgentSpan's skill-metadata SPI to Conductor's per-backend {@code SkillMetadataDAO}.
|
||||
*/
|
||||
@Bean
|
||||
public dev.agentspan.runtime.spi.SkillMetadataDAO agentSpanSkillMetadataDAO(
|
||||
org.conductoross.conductor.dao.SkillMetadataDAO skillMetadataDAO,
|
||||
ObjectMapper objectMapper) {
|
||||
return new SkillMetadataDaoAdapter(skillMetadataDAO, objectMapper);
|
||||
}
|
||||
|
||||
/** Bridges AgentSpan's skill-package SPI to Conductor's per-backend {@code SkillPackageDAO}. */
|
||||
@Bean
|
||||
public dev.agentspan.runtime.spi.SkillPackageStore agentSpanSkillPackageStore(
|
||||
SkillPackageDAO skillPackageDAO) {
|
||||
return new SkillPackageStoreAdapter(skillPackageDAO);
|
||||
}
|
||||
|
||||
/**
|
||||
* Populates AgentSpan's request principal for {@code /api/*}. Runs late so a host security
|
||||
* adapter (if any) can set the real principal first.
|
||||
*/
|
||||
@Bean
|
||||
public FilterRegistrationBean<AgentSpanPrincipalFilter> agentSpanPrincipalFilter(
|
||||
@Value("${conductor.integrations.ai.agentspan.default-user-id:agentspan-system}")
|
||||
String defaultUserId) {
|
||||
FilterRegistrationBean<AgentSpanPrincipalFilter> registration =
|
||||
new FilterRegistrationBean<>(new AgentSpanPrincipalFilter(defaultUserId));
|
||||
registration.addUrlPatterns("/api/*");
|
||||
registration.setOrder(Ordered.LOWEST_PRECEDENCE);
|
||||
registration.setName("agentSpanPrincipalFilter");
|
||||
return registration;
|
||||
}
|
||||
}
|
||||
+114
@@ -0,0 +1,114 @@
|
||||
/*
|
||||
* Copyright 2026 Conductor Authors.
|
||||
* <p>
|
||||
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package org.conductoross.conductor.ai.agentspan;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashSet;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.function.UnaryOperator;
|
||||
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import dev.agentspan.runtime.credentials.KnownProviderEnvVars;
|
||||
import dev.agentspan.runtime.model.credentials.CredentialMeta;
|
||||
import dev.agentspan.runtime.spi.CredentialStoreProvider;
|
||||
|
||||
/**
|
||||
* Read-only, environment-seeded {@link CredentialStoreProvider} for OSS Conductor.
|
||||
*
|
||||
* <p>On construction it scans the process environment for the well-known provider variables ({@link
|
||||
* KnownProviderEnvVars#NAMES}) plus any additional names configured via {@code
|
||||
* conductor.integrations.ai.secrets.env-names}, and populates an in-memory map. Agents resolve
|
||||
* credentials from this map at runtime.
|
||||
*
|
||||
* <p>Secrets are intentionally <b>not</b> writable through the API: {@link #set} throws, with a
|
||||
* message directing operators to inject secrets as environment variables (e.g. via Kubernetes
|
||||
* secrets) and restart. The AgentSpan {@code AgentExceptionHandler} renders that {@link
|
||||
* IllegalStateException} as an HTTP 400. {@link #get}, {@link #list} and {@link #delete} operate on
|
||||
* the in-memory map; deletions are not persisted and are re-seeded from the environment on the next
|
||||
* restart.
|
||||
*
|
||||
* <p>This store is global (not per-user): the {@code userId} argument is accepted for SPI
|
||||
* compatibility but ignored, because environment-injected secrets are shared by the process.
|
||||
*/
|
||||
public class EnvBackedCredentialStore implements CredentialStoreProvider {
|
||||
|
||||
private static final Logger log = LoggerFactory.getLogger(EnvBackedCredentialStore.class);
|
||||
|
||||
static final String READ_ONLY_MESSAGE =
|
||||
"Secrets are read-only via the Conductor API. Provide them as environment variables "
|
||||
+ "(for example OPENAI_API_KEY) so they are injected at runtime, then restart the server.";
|
||||
|
||||
private final Map<String, String> values = new ConcurrentHashMap<>();
|
||||
|
||||
public EnvBackedCredentialStore(List<String> extraNames, UnaryOperator<String> envLookup) {
|
||||
Set<String> names = new LinkedHashSet<>(KnownProviderEnvVars.NAMES);
|
||||
if (extraNames != null) {
|
||||
names.addAll(extraNames);
|
||||
}
|
||||
int seeded = 0;
|
||||
for (String name : names) {
|
||||
String value = envLookup.apply(name);
|
||||
if (value != null && !value.isBlank()) {
|
||||
values.put(name, value);
|
||||
seeded++;
|
||||
}
|
||||
}
|
||||
log.info(
|
||||
"AgentSpan EnvBackedCredentialStore seeded {} secret(s) from {} known environment variable name(s)",
|
||||
seeded,
|
||||
names.size());
|
||||
}
|
||||
|
||||
@Override
|
||||
public String get(String userId, String name) {
|
||||
return values.get(name);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void set(String userId, String name, String value) {
|
||||
// IllegalArgumentException maps to HTTP 400 via both Conductor's ApplicationExceptionMapper
|
||||
// and AgentSpan's AgentExceptionHandler (a read-only store is a client error, not 500).
|
||||
throw new IllegalArgumentException(READ_ONLY_MESSAGE);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void delete(String userId, String name) {
|
||||
values.remove(name);
|
||||
}
|
||||
|
||||
@Override
|
||||
public List<CredentialMeta> list(String userId) {
|
||||
List<CredentialMeta> out = new ArrayList<>();
|
||||
for (Map.Entry<String, String> entry : values.entrySet()) {
|
||||
out.add(
|
||||
CredentialMeta.builder()
|
||||
.name(entry.getKey())
|
||||
.partial(toPartial(entry.getValue()))
|
||||
.build());
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/** First 4 + "..." + last 4 characters, matching common API-key display conventions. */
|
||||
static String toPartial(String value) {
|
||||
if (value == null || value.length() < 8) {
|
||||
return "****...****";
|
||||
}
|
||||
return value.substring(0, 4) + "..." + value.substring(value.length() - 4);
|
||||
}
|
||||
}
|
||||
+91
@@ -0,0 +1,91 @@
|
||||
/*
|
||||
* Copyright 2026 Conductor Authors.
|
||||
* <p>
|
||||
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package org.conductoross.conductor.ai.agentspan;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Optional;
|
||||
|
||||
import com.fasterxml.jackson.core.JsonProcessingException;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import dev.agentspan.runtime.model.skill.SkillDetail;
|
||||
|
||||
/**
|
||||
* Bridges AgentSpan's {@link dev.agentspan.runtime.spi.SkillMetadataDAO} SPI onto Conductor's
|
||||
* backend-agnostic {@link org.conductoross.conductor.dao.SkillMetadataDAO}. The {@link SkillDetail}
|
||||
* manifest is serialized to JSON for storage so the persistence layer carries no dependency on
|
||||
* AgentSpan model types; it is rehydrated on read.
|
||||
*/
|
||||
public class SkillMetadataDaoAdapter implements dev.agentspan.runtime.spi.SkillMetadataDAO {
|
||||
|
||||
private final org.conductoross.conductor.dao.SkillMetadataDAO delegate;
|
||||
private final ObjectMapper objectMapper;
|
||||
|
||||
public SkillMetadataDaoAdapter(
|
||||
org.conductoross.conductor.dao.SkillMetadataDAO delegate, ObjectMapper objectMapper) {
|
||||
this.delegate = delegate;
|
||||
this.objectMapper = objectMapper;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void save(SkillDetail detail, boolean makeLatest) {
|
||||
delegate.save(
|
||||
detail.getOwnerId(),
|
||||
detail.getName(),
|
||||
detail.getVersion(),
|
||||
makeLatest,
|
||||
toJson(detail),
|
||||
detail.getCreatedAt(),
|
||||
detail.getUpdatedAt());
|
||||
}
|
||||
|
||||
@Override
|
||||
public Optional<SkillDetail> find(String ownerId, String name, String version) {
|
||||
return delegate.find(ownerId, name, version).map(this::fromJson);
|
||||
}
|
||||
|
||||
@Override
|
||||
public Optional<String> latestVersion(String ownerId, String name) {
|
||||
return delegate.latestVersion(ownerId, name);
|
||||
}
|
||||
|
||||
@Override
|
||||
public List<SkillDetail> listVersions(String ownerId, String name) {
|
||||
return delegate.listVersions(ownerId, name).stream().map(this::fromJson).toList();
|
||||
}
|
||||
|
||||
@Override
|
||||
public List<SkillDetail> list(String ownerId, boolean allVersions) {
|
||||
return delegate.list(ownerId, allVersions).stream().map(this::fromJson).toList();
|
||||
}
|
||||
|
||||
@Override
|
||||
public void delete(String ownerId, String name, String version) {
|
||||
delegate.delete(ownerId, name, version);
|
||||
}
|
||||
|
||||
private String toJson(SkillDetail detail) {
|
||||
try {
|
||||
return objectMapper.writeValueAsString(detail);
|
||||
} catch (JsonProcessingException e) {
|
||||
throw new IllegalStateException("Failed to serialize skill metadata", e);
|
||||
}
|
||||
}
|
||||
|
||||
private SkillDetail fromJson(String json) {
|
||||
try {
|
||||
return objectMapper.readValue(json, SkillDetail.class);
|
||||
} catch (JsonProcessingException e) {
|
||||
throw new IllegalStateException("Failed to deserialize skill metadata", e);
|
||||
}
|
||||
}
|
||||
}
|
||||
+69
@@ -0,0 +1,69 @@
|
||||
/*
|
||||
* Copyright 2026 Conductor Authors.
|
||||
* <p>
|
||||
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package org.conductoross.conductor.ai.agentspan;
|
||||
|
||||
import org.conductoross.conductor.dao.SkillPackageDAO;
|
||||
|
||||
import dev.agentspan.runtime.spi.SkillPackageStore;
|
||||
import dev.agentspan.runtime.spi.StoredSkillPackage;
|
||||
|
||||
/**
|
||||
* Bridges AgentSpan's {@link SkillPackageStore} SPI onto Conductor's backend-agnostic {@link
|
||||
* SkillPackageDAO}, so skill package bytes persist through whichever Conductor backend is
|
||||
* configured (Postgres, MySQL, SQLite, Redis).
|
||||
*
|
||||
* <p>The opaque handle is content-addressed ({@code conductor-db://<checksum>}); identical package
|
||||
* bytes therefore deduplicate to one stored row.
|
||||
*/
|
||||
public class SkillPackageStoreAdapter implements SkillPackageStore {
|
||||
|
||||
static final String STORAGE_TYPE = "conductor-db";
|
||||
private static final String HANDLE_PREFIX = "conductor-db://";
|
||||
|
||||
private final SkillPackageDAO delegate;
|
||||
|
||||
public SkillPackageStoreAdapter(SkillPackageDAO delegate) {
|
||||
this.delegate = delegate;
|
||||
}
|
||||
|
||||
@Override
|
||||
public String storageType() {
|
||||
return STORAGE_TYPE;
|
||||
}
|
||||
|
||||
@Override
|
||||
public StoredSkillPackage store(String name, String version, String checksum, byte[] bytes) {
|
||||
String handle = HANDLE_PREFIX + checksum;
|
||||
delegate.put(handle, bytes);
|
||||
return new StoredSkillPackage(handle, STORAGE_TYPE, bytes.length);
|
||||
}
|
||||
|
||||
@Override
|
||||
public byte[] read(String handle) {
|
||||
byte[] bytes = delegate.get(handle);
|
||||
if (bytes == null) {
|
||||
throw new IllegalArgumentException("Skill package not found: " + handle);
|
||||
}
|
||||
return bytes;
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean exists(String handle) {
|
||||
return delegate.exists(handle);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void delete(String handle) {
|
||||
delegate.delete(handle);
|
||||
}
|
||||
}
|
||||
+166
@@ -0,0 +1,166 @@
|
||||
/*
|
||||
* Copyright 2026 Conductor Authors.
|
||||
* <p>
|
||||
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package org.conductoross.conductor.ai.agentspan.listener;
|
||||
|
||||
import java.util.function.Consumer;
|
||||
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.springframework.beans.factory.ObjectProvider;
|
||||
|
||||
import com.netflix.conductor.core.listener.TaskStatusListener;
|
||||
import com.netflix.conductor.model.TaskModel;
|
||||
|
||||
/**
|
||||
* A {@link TaskStatusListener} that fans every callback out to all other {@code TaskStatusListener}
|
||||
* beans in the context.
|
||||
*
|
||||
* <p>The motivation mirrors {@link AgentSpanCompositeWorkflowStatusListener}: Conductor injects a
|
||||
* <b>single</b> {@code TaskStatusListener} bean, so the embedded {@code conductor-agentspan}
|
||||
* library's {@code @Primary AgentEventListener} would otherwise shadow an operator-configured
|
||||
* {@code task_publisher}. Registering this composite as the sole {@code @Primary} listener and
|
||||
* re-broadcasting to every registered listener lets agentspan's SSE listener and the configured
|
||||
* publisher coexist.
|
||||
*
|
||||
* <p>Each callback delegates to the <b>same</b> method on every other listener so each delegate
|
||||
* keeps its own gating semantics. The composite excludes itself to avoid recursion and isolates
|
||||
* delegates so one listener's failure cannot suppress the others. Delegates are resolved lazily via
|
||||
* {@link ObjectProvider} to avoid a self-referential construction cycle for this {@code @Primary}
|
||||
* bean.
|
||||
*/
|
||||
public class AgentSpanCompositeTaskStatusListener implements TaskStatusListener {
|
||||
|
||||
private static final Logger LOGGER =
|
||||
LoggerFactory.getLogger(AgentSpanCompositeTaskStatusListener.class);
|
||||
|
||||
private final ObjectProvider<TaskStatusListener> delegates;
|
||||
|
||||
public AgentSpanCompositeTaskStatusListener(ObjectProvider<TaskStatusListener> delegates) {
|
||||
this.delegates = delegates;
|
||||
}
|
||||
|
||||
/**
|
||||
* Invoke {@code action} on every registered listener except this composite, isolating each
|
||||
* delegate so one listener's failure does not stop the rest.
|
||||
*/
|
||||
private void fanOut(Consumer<TaskStatusListener> action) {
|
||||
delegates.stream()
|
||||
.filter(listener -> listener != this)
|
||||
.forEach(
|
||||
listener -> {
|
||||
try {
|
||||
action.accept(listener);
|
||||
} catch (Exception e) {
|
||||
LOGGER.warn(
|
||||
"TaskStatusListener {} threw during fan-out: {}",
|
||||
listener.getClass().getName(),
|
||||
e.getMessage(),
|
||||
e);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// ── *IfEnabled variants — the primary callback path used by WorkflowExecutorOps ──
|
||||
|
||||
@Override
|
||||
public void onTaskScheduledIfEnabled(TaskModel task) {
|
||||
fanOut(listener -> listener.onTaskScheduledIfEnabled(task));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onTaskInProgressIfEnabled(TaskModel task) {
|
||||
fanOut(listener -> listener.onTaskInProgressIfEnabled(task));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onTaskCanceledIfEnabled(TaskModel task) {
|
||||
fanOut(listener -> listener.onTaskCanceledIfEnabled(task));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onTaskFailedIfEnabled(TaskModel task) {
|
||||
fanOut(listener -> listener.onTaskFailedIfEnabled(task));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onTaskFailedWithTerminalErrorIfEnabled(TaskModel task) {
|
||||
fanOut(listener -> listener.onTaskFailedWithTerminalErrorIfEnabled(task));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onTaskCompletedIfEnabled(TaskModel task) {
|
||||
fanOut(listener -> listener.onTaskCompletedIfEnabled(task));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onTaskCompletedWithErrorsIfEnabled(TaskModel task) {
|
||||
fanOut(listener -> listener.onTaskCompletedWithErrorsIfEnabled(task));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onTaskTimedOutIfEnabled(TaskModel task) {
|
||||
fanOut(listener -> listener.onTaskTimedOutIfEnabled(task));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onTaskSkippedIfEnabled(TaskModel task) {
|
||||
fanOut(listener -> listener.onTaskSkippedIfEnabled(task));
|
||||
}
|
||||
|
||||
// ── Direct variants — fan out for any caller that bypasses the *IfEnabled path ──
|
||||
|
||||
@Override
|
||||
public void onTaskScheduled(TaskModel task) {
|
||||
fanOut(listener -> listener.onTaskScheduled(task));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onTaskInProgress(TaskModel task) {
|
||||
fanOut(listener -> listener.onTaskInProgress(task));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onTaskCanceled(TaskModel task) {
|
||||
fanOut(listener -> listener.onTaskCanceled(task));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onTaskFailed(TaskModel task) {
|
||||
fanOut(listener -> listener.onTaskFailed(task));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onTaskFailedWithTerminalError(TaskModel task) {
|
||||
fanOut(listener -> listener.onTaskFailedWithTerminalError(task));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onTaskCompleted(TaskModel task) {
|
||||
fanOut(listener -> listener.onTaskCompleted(task));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onTaskCompletedWithErrors(TaskModel task) {
|
||||
fanOut(listener -> listener.onTaskCompletedWithErrors(task));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onTaskTimedOut(TaskModel task) {
|
||||
fanOut(listener -> listener.onTaskTimedOut(task));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onTaskSkipped(TaskModel task) {
|
||||
fanOut(listener -> listener.onTaskSkipped(task));
|
||||
}
|
||||
}
|
||||
+173
@@ -0,0 +1,173 @@
|
||||
/*
|
||||
* Copyright 2026 Conductor Authors.
|
||||
* <p>
|
||||
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package org.conductoross.conductor.ai.agentspan.listener;
|
||||
|
||||
import java.util.function.Consumer;
|
||||
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.springframework.beans.factory.ObjectProvider;
|
||||
|
||||
import com.netflix.conductor.core.listener.WorkflowStatusListener;
|
||||
import com.netflix.conductor.model.WorkflowModel;
|
||||
|
||||
/**
|
||||
* A {@link WorkflowStatusListener} that fans every callback out to all other {@code
|
||||
* WorkflowStatusListener} beans in the context.
|
||||
*
|
||||
* <p>Conductor injects a <b>single</b> {@code WorkflowStatusListener} bean (see {@code
|
||||
* WorkflowExecutorOps}/{@code ExecutionService}), so when the embedded {@code conductor-agentspan}
|
||||
* library contributes its {@code @Primary AgentEventListener}, that one bean would otherwise shadow
|
||||
* any operator-configured publisher ({@code queue_publisher}, {@code kafka}, {@code archive},
|
||||
* {@code workflow_publisher}). To let agentspan's SSE listener and the configured publisher
|
||||
* coexist, this composite is registered as the sole {@code @Primary} listener and re-broadcasts
|
||||
* each event to every registered listener.
|
||||
*
|
||||
* <p>Each callback delegates to the <b>same</b> method on every other listener so that each
|
||||
* delegate applies its own gating semantics: the {@code *IfEnabled} variants honour each listener's
|
||||
* choice to gate on {@code workflowStatusListenerEnabled}, while agentspan's listener (which
|
||||
* overrides the {@code *IfEnabled} methods directly) fires unconditionally as it intends. The
|
||||
* composite excludes itself from the fan-out to avoid infinite recursion, and isolates delegates so
|
||||
* a failure in one listener cannot suppress the others.
|
||||
*
|
||||
* <p>Delegates are resolved lazily through an {@link ObjectProvider} so that wiring this
|
||||
* {@code @Primary} bean (which is itself a {@code WorkflowStatusListener}) does not create a
|
||||
* self-referential construction cycle.
|
||||
*/
|
||||
public class AgentSpanCompositeWorkflowStatusListener implements WorkflowStatusListener {
|
||||
|
||||
private static final Logger LOGGER =
|
||||
LoggerFactory.getLogger(AgentSpanCompositeWorkflowStatusListener.class);
|
||||
|
||||
private final ObjectProvider<WorkflowStatusListener> delegates;
|
||||
|
||||
public AgentSpanCompositeWorkflowStatusListener(
|
||||
ObjectProvider<WorkflowStatusListener> delegates) {
|
||||
this.delegates = delegates;
|
||||
}
|
||||
|
||||
/**
|
||||
* Invoke {@code action} on every registered listener except this composite, isolating each
|
||||
* delegate so one listener's failure does not stop the rest.
|
||||
*/
|
||||
private void fanOut(Consumer<WorkflowStatusListener> action) {
|
||||
delegates.stream()
|
||||
.filter(listener -> listener != this)
|
||||
.forEach(
|
||||
listener -> {
|
||||
try {
|
||||
action.accept(listener);
|
||||
} catch (Exception e) {
|
||||
LOGGER.warn(
|
||||
"WorkflowStatusListener {} threw during fan-out: {}",
|
||||
listener.getClass().getName(),
|
||||
e.getMessage(),
|
||||
e);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// ── *IfEnabled variants — the primary callback path used by WorkflowExecutorOps ──
|
||||
|
||||
@Override
|
||||
public void onWorkflowCompletedIfEnabled(WorkflowModel workflow) {
|
||||
fanOut(listener -> listener.onWorkflowCompletedIfEnabled(workflow));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onWorkflowTerminatedIfEnabled(WorkflowModel workflow) {
|
||||
fanOut(listener -> listener.onWorkflowTerminatedIfEnabled(workflow));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onWorkflowFinalizedIfEnabled(WorkflowModel workflow) {
|
||||
fanOut(listener -> listener.onWorkflowFinalizedIfEnabled(workflow));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onWorkflowStartedIfEnabled(WorkflowModel workflow) {
|
||||
fanOut(listener -> listener.onWorkflowStartedIfEnabled(workflow));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onWorkflowRestartedIfEnabled(WorkflowModel workflow) {
|
||||
fanOut(listener -> listener.onWorkflowRestartedIfEnabled(workflow));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onWorkflowRerunIfEnabled(WorkflowModel workflow) {
|
||||
fanOut(listener -> listener.onWorkflowRerunIfEnabled(workflow));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onWorkflowRetriedIfEnabled(WorkflowModel workflow) {
|
||||
fanOut(listener -> listener.onWorkflowRetriedIfEnabled(workflow));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onWorkflowPausedIfEnabled(WorkflowModel workflow) {
|
||||
fanOut(listener -> listener.onWorkflowPausedIfEnabled(workflow));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onWorkflowResumedIfEnabled(WorkflowModel workflow) {
|
||||
fanOut(listener -> listener.onWorkflowResumedIfEnabled(workflow));
|
||||
}
|
||||
|
||||
// ── Direct variants — fan out for any caller that bypasses the *IfEnabled path ──
|
||||
|
||||
@Override
|
||||
public void onWorkflowCompleted(WorkflowModel workflow) {
|
||||
fanOut(listener -> listener.onWorkflowCompleted(workflow));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onWorkflowTerminated(WorkflowModel workflow) {
|
||||
fanOut(listener -> listener.onWorkflowTerminated(workflow));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onWorkflowFinalized(WorkflowModel workflow) {
|
||||
fanOut(listener -> listener.onWorkflowFinalized(workflow));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onWorkflowStarted(WorkflowModel workflow) {
|
||||
fanOut(listener -> listener.onWorkflowStarted(workflow));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onWorkflowRestarted(WorkflowModel workflow) {
|
||||
fanOut(listener -> listener.onWorkflowRestarted(workflow));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onWorkflowRerun(WorkflowModel workflow) {
|
||||
fanOut(listener -> listener.onWorkflowRerun(workflow));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onWorkflowPaused(WorkflowModel workflow) {
|
||||
fanOut(listener -> listener.onWorkflowPaused(workflow));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onWorkflowResumed(WorkflowModel workflow) {
|
||||
fanOut(listener -> listener.onWorkflowResumed(workflow));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onWorkflowRetried(WorkflowModel workflow) {
|
||||
fanOut(listener -> listener.onWorkflowRetried(workflow));
|
||||
}
|
||||
}
|
||||
+111
@@ -0,0 +1,111 @@
|
||||
/*
|
||||
* Copyright 2026 Conductor Authors.
|
||||
* <p>
|
||||
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package org.conductoross.conductor.ai.agentspan.listener;
|
||||
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.springframework.beans.factory.ObjectProvider;
|
||||
import org.springframework.beans.factory.config.BeanDefinition;
|
||||
import org.springframework.beans.factory.config.BeanFactoryPostProcessor;
|
||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
import org.springframework.context.annotation.Configuration;
|
||||
import org.springframework.context.annotation.Primary;
|
||||
|
||||
import com.netflix.conductor.core.listener.TaskStatusListener;
|
||||
import com.netflix.conductor.core.listener.WorkflowStatusListener;
|
||||
|
||||
/**
|
||||
* Makes the embedded {@code conductor-agentspan} status listener coexist with any
|
||||
* operator-configured status publisher.
|
||||
*
|
||||
* <p>The agentspan library registers {@code AgentEventListener} as a {@code @Primary} bean for both
|
||||
* {@link WorkflowStatusListener} and {@link TaskStatusListener} (it streams agent events over SSE).
|
||||
* Conductor, however, injects a <b>single</b> listener bean of each type, so {@code @Primary} alone
|
||||
* would make agentspan's listener silently shadow a configured {@code queue_publisher} / {@code
|
||||
* kafka} / {@code archive} / {@code workflow_publisher} (or {@code task_publisher}) — and {@code
|
||||
* conductor.integrations.ai.enabled} defaults to {@code true}, so this would affect default
|
||||
* servers.
|
||||
*
|
||||
* <p>Active only in embedded mode ({@code agentspan.embedded=true}, set by {@link
|
||||
* org.conductoross.conductor.ai.agentspan.AgentSpanEmbeddedEnvironmentPostProcessor}), this
|
||||
* configuration:
|
||||
*
|
||||
* <ol>
|
||||
* <li>demotes agentspan's {@code AgentEventListener} from {@code @Primary} (via a {@link
|
||||
* BeanFactoryPostProcessor} that edits the bean definition before instantiation), and
|
||||
* <li>registers composite {@code @Primary} listeners ({@link
|
||||
* AgentSpanCompositeWorkflowStatusListener}, {@link AgentSpanCompositeTaskStatusListener})
|
||||
* that fan every callback out to <i>all</i> registered listeners — agentspan's listener and
|
||||
* the configured publisher alike.
|
||||
* </ol>
|
||||
*
|
||||
* <p>The composite becomes the sole {@code @Primary} candidate, so the single-bean injection points
|
||||
* in {@code WorkflowExecutorOps}/{@code ExecutionService} resolve to it, and every listener
|
||||
* receives every event.
|
||||
*/
|
||||
@Configuration(proxyBeanMethods = false)
|
||||
@ConditionalOnProperty(name = "agentspan.embedded", havingValue = "true")
|
||||
public class AgentSpanListenerCoexistenceConfiguration {
|
||||
|
||||
private static final Logger LOGGER =
|
||||
LoggerFactory.getLogger(AgentSpanListenerCoexistenceConfiguration.class);
|
||||
|
||||
/**
|
||||
* Fully-qualified name of the agentspan listener. Matched by name rather than imported type to
|
||||
* avoid coupling to a library-internal class and to keep the post-processor free of side
|
||||
* effects (no premature class loading).
|
||||
*/
|
||||
private static final String AGENT_EVENT_LISTENER_CLASS =
|
||||
"dev.agentspan.runtime.service.AgentEventListener";
|
||||
|
||||
/**
|
||||
* Demotes agentspan's {@code @Primary AgentEventListener} to an ordinary candidate so the
|
||||
* composite beans below can become the sole {@code @Primary} listeners. Declared {@code static}
|
||||
* so the post-processor is instantiated early (before regular bean instantiation) without
|
||||
* forcing the enclosing configuration to initialise prematurely. Runs after {@code
|
||||
* ConfigurationClassPostProcessor} (which registers agentspan's component-scanned definition),
|
||||
* so the target definition is present; it is a no-op if agentspan is absent or already
|
||||
* non-primary.
|
||||
*/
|
||||
@Bean
|
||||
public static BeanFactoryPostProcessor agentSpanListenerPrimaryDemoter() {
|
||||
return beanFactory -> {
|
||||
for (String name : beanFactory.getBeanDefinitionNames()) {
|
||||
BeanDefinition definition = beanFactory.getBeanDefinition(name);
|
||||
if (AGENT_EVENT_LISTENER_CLASS.equals(definition.getBeanClassName())
|
||||
&& definition.isPrimary()) {
|
||||
definition.setPrimary(false);
|
||||
LOGGER.info(
|
||||
"Demoted agentspan listener bean '{}' from @Primary so workflow/task "
|
||||
+ "status listeners can coexist via the composite",
|
||||
name);
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
@Bean
|
||||
@Primary
|
||||
public WorkflowStatusListener agentSpanCompositeWorkflowStatusListener(
|
||||
ObjectProvider<WorkflowStatusListener> workflowStatusListeners) {
|
||||
return new AgentSpanCompositeWorkflowStatusListener(workflowStatusListeners);
|
||||
}
|
||||
|
||||
@Bean
|
||||
@Primary
|
||||
public TaskStatusListener agentSpanCompositeTaskStatusListener(
|
||||
ObjectProvider<TaskStatusListener> taskStatusListeners) {
|
||||
return new AgentSpanCompositeTaskStatusListener(taskStatusListeners);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,2 @@
|
||||
org.springframework.boot.env.EnvironmentPostProcessor=\
|
||||
org.conductoross.conductor.ai.agentspan.AgentSpanEmbeddedEnvironmentPostProcessor
|
||||
+92
@@ -0,0 +1,92 @@
|
||||
/*
|
||||
* Copyright 2026 Conductor Authors.
|
||||
* <p>
|
||||
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package org.conductoross.conductor.ai.agentspan;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import dev.agentspan.runtime.model.credentials.CredentialMeta;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
import static org.junit.jupiter.api.Assertions.assertNull;
|
||||
import static org.junit.jupiter.api.Assertions.assertThrows;
|
||||
import static org.junit.jupiter.api.Assertions.assertTrue;
|
||||
|
||||
class EnvBackedCredentialStoreTest {
|
||||
|
||||
private static final String USER = "agentspan-system";
|
||||
|
||||
private EnvBackedCredentialStore store(Map<String, String> env, List<String> extra) {
|
||||
return new EnvBackedCredentialStore(extra, env::get);
|
||||
}
|
||||
|
||||
@Test
|
||||
void seedsKnownProviderVarsFromEnvironment() {
|
||||
EnvBackedCredentialStore store =
|
||||
store(
|
||||
Map.of("OPENAI_API_KEY", "sk-abcdefgh", "UNRELATED_VAR", "ignored"),
|
||||
List.of());
|
||||
|
||||
assertEquals("sk-abcdefgh", store.get(USER, "OPENAI_API_KEY"));
|
||||
// Not a known provider var and not in the extra list -> not seeded.
|
||||
assertNull(store.get(USER, "UNRELATED_VAR"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void seedsConfiguredExtraNames() {
|
||||
EnvBackedCredentialStore store =
|
||||
store(Map.of("MY_CUSTOM_SECRET", "value-1234"), List.of("MY_CUSTOM_SECRET"));
|
||||
|
||||
assertEquals("value-1234", store.get(USER, "MY_CUSTOM_SECRET"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void listReturnsMaskedMetadataNeverPlaintext() {
|
||||
EnvBackedCredentialStore store =
|
||||
store(Map.of("OPENAI_API_KEY", "sk-test-1234567890abcdef"), List.of());
|
||||
|
||||
List<CredentialMeta> list = store.list(USER);
|
||||
assertEquals(1, list.size());
|
||||
CredentialMeta meta = list.get(0);
|
||||
assertEquals("OPENAI_API_KEY", meta.getName());
|
||||
assertEquals("sk-t...cdef", meta.getPartial());
|
||||
assertTrue(!meta.getPartial().contains("567890"), "partial must not leak the full value");
|
||||
}
|
||||
|
||||
@Test
|
||||
void setIsRejectedWithGuidance() {
|
||||
EnvBackedCredentialStore store = store(Map.of(), List.of());
|
||||
|
||||
IllegalArgumentException ex =
|
||||
assertThrows(
|
||||
IllegalArgumentException.class, () -> store.set(USER, "ANYTHING", "v"));
|
||||
assertTrue(ex.getMessage().contains("environment variables"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void deleteRemovesFromInMemoryView() {
|
||||
EnvBackedCredentialStore store = store(Map.of("OPENAI_API_KEY", "sk-abcdefgh"), List.of());
|
||||
|
||||
store.delete(USER, "OPENAI_API_KEY");
|
||||
assertNull(store.get(USER, "OPENAI_API_KEY"));
|
||||
assertTrue(store.list(USER).isEmpty());
|
||||
}
|
||||
|
||||
@Test
|
||||
void shortValuesAreFullyMasked() {
|
||||
assertEquals("****...****", EnvBackedCredentialStore.toPartial("short"));
|
||||
assertEquals("****...****", EnvBackedCredentialStore.toPartial(null));
|
||||
}
|
||||
}
|
||||
+169
@@ -0,0 +1,169 @@
|
||||
/*
|
||||
* Copyright 2026 Conductor Authors.
|
||||
* <p>
|
||||
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package org.conductoross.conductor.ai.agentspan.listener;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.stream.Stream;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.springframework.beans.factory.ObjectProvider;
|
||||
|
||||
import com.netflix.conductor.common.metadata.workflow.WorkflowDef;
|
||||
import com.netflix.conductor.core.listener.WorkflowStatusListener;
|
||||
import com.netflix.conductor.model.WorkflowModel;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
import static org.junit.jupiter.api.Assertions.assertFalse;
|
||||
import static org.junit.jupiter.api.Assertions.assertTrue;
|
||||
|
||||
/**
|
||||
* Verifies the fan-out semantics that let agentspan's listener coexist with a configured publisher:
|
||||
* the composite broadcasts to every other listener, excludes itself (no recursion), isolates
|
||||
* failures, and preserves each delegate's own {@code *IfEnabled} gating.
|
||||
*/
|
||||
class AgentSpanCompositeWorkflowStatusListenerTest {
|
||||
|
||||
/** Real listener that records which callbacks it received. */
|
||||
private static final class RecordingListener implements WorkflowStatusListener {
|
||||
final List<String> events = new ArrayList<>();
|
||||
|
||||
@Override
|
||||
public void onWorkflowCompleted(WorkflowModel workflow) {
|
||||
events.add("completed:" + workflow.getWorkflowId());
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onWorkflowTerminated(WorkflowModel workflow) {
|
||||
events.add("terminated:" + workflow.getWorkflowId());
|
||||
}
|
||||
}
|
||||
|
||||
/** Listener that always throws, to prove failures are isolated from siblings. */
|
||||
private static final class ThrowingListener implements WorkflowStatusListener {
|
||||
boolean invoked = false;
|
||||
|
||||
@Override
|
||||
public void onWorkflowCompleted(WorkflowModel workflow) {
|
||||
invoked = true;
|
||||
throw new RuntimeException("boom");
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onWorkflowTerminated(WorkflowModel workflow) {
|
||||
throw new RuntimeException("boom");
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Minimal {@link ObjectProvider} backed by a mutable list. The composite only ever calls {@link
|
||||
* ObjectProvider#stream()}, so the remaining methods are intentionally unsupported.
|
||||
*/
|
||||
private static ObjectProvider<WorkflowStatusListener> providerOf(
|
||||
List<WorkflowStatusListener> registry) {
|
||||
return new ObjectProvider<>() {
|
||||
@Override
|
||||
public Stream<WorkflowStatusListener> stream() {
|
||||
return registry.stream();
|
||||
}
|
||||
|
||||
@Override
|
||||
public WorkflowStatusListener getObject(Object... args) {
|
||||
throw new UnsupportedOperationException();
|
||||
}
|
||||
|
||||
@Override
|
||||
public WorkflowStatusListener getObject() {
|
||||
throw new UnsupportedOperationException();
|
||||
}
|
||||
|
||||
@Override
|
||||
public WorkflowStatusListener getIfAvailable() {
|
||||
throw new UnsupportedOperationException();
|
||||
}
|
||||
|
||||
@Override
|
||||
public WorkflowStatusListener getIfUnique() {
|
||||
throw new UnsupportedOperationException();
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
private static WorkflowModel workflow(String id, boolean listenerEnabled) {
|
||||
WorkflowModel workflow = new WorkflowModel();
|
||||
workflow.setWorkflowId(id);
|
||||
WorkflowDef def = new WorkflowDef();
|
||||
def.setName("test_wf");
|
||||
def.setWorkflowStatusListenerEnabled(listenerEnabled);
|
||||
workflow.setWorkflowDefinition(def);
|
||||
return workflow;
|
||||
}
|
||||
|
||||
@Test
|
||||
void fansOutToAllDelegatesAndExcludesItself() {
|
||||
List<WorkflowStatusListener> registry = new ArrayList<>();
|
||||
AgentSpanCompositeWorkflowStatusListener composite =
|
||||
new AgentSpanCompositeWorkflowStatusListener(providerOf(registry));
|
||||
RecordingListener first = new RecordingListener();
|
||||
RecordingListener second = new RecordingListener();
|
||||
// Include the composite itself to prove it is filtered out (otherwise this would recurse
|
||||
// infinitely and overflow the stack).
|
||||
registry.add(composite);
|
||||
registry.add(first);
|
||||
registry.add(second);
|
||||
|
||||
composite.onWorkflowCompleted(workflow("wf-1", true));
|
||||
|
||||
assertEquals(List.of("completed:wf-1"), first.events);
|
||||
assertEquals(List.of("completed:wf-1"), second.events);
|
||||
}
|
||||
|
||||
@Test
|
||||
void isolatesFailingDelegateFromTheRest() {
|
||||
List<WorkflowStatusListener> registry = new ArrayList<>();
|
||||
AgentSpanCompositeWorkflowStatusListener composite =
|
||||
new AgentSpanCompositeWorkflowStatusListener(providerOf(registry));
|
||||
RecordingListener before = new RecordingListener();
|
||||
ThrowingListener throwing = new ThrowingListener();
|
||||
RecordingListener after = new RecordingListener();
|
||||
registry.add(composite);
|
||||
registry.add(before);
|
||||
registry.add(throwing);
|
||||
registry.add(after);
|
||||
|
||||
// Must not propagate the delegate's exception.
|
||||
composite.onWorkflowCompleted(workflow("wf-2", true));
|
||||
|
||||
assertTrue(throwing.invoked);
|
||||
assertEquals(List.of("completed:wf-2"), before.events);
|
||||
assertEquals(List.of("completed:wf-2"), after.events);
|
||||
}
|
||||
|
||||
@Test
|
||||
void ifEnabledPathHonoursEachDelegateGating() {
|
||||
List<WorkflowStatusListener> registry = new ArrayList<>();
|
||||
AgentSpanCompositeWorkflowStatusListener composite =
|
||||
new AgentSpanCompositeWorkflowStatusListener(providerOf(registry));
|
||||
RecordingListener listener = new RecordingListener();
|
||||
registry.add(composite);
|
||||
registry.add(listener);
|
||||
|
||||
// Delegate uses the interface default *IfEnabled gating: enabled -> delivered.
|
||||
composite.onWorkflowTerminatedIfEnabled(workflow("wf-on", true));
|
||||
// Disabled -> the delegate's own gating suppresses delivery.
|
||||
composite.onWorkflowTerminatedIfEnabled(workflow("wf-off", false));
|
||||
|
||||
assertEquals(List.of("terminated:wf-on"), listener.events);
|
||||
assertFalse(listener.events.contains("terminated:wf-off"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,627 @@
|
||||
# Contributing to Conductor AI Module
|
||||
|
||||
Thank you for your interest in contributing to the Conductor AI module! This guide will help you add new LLM providers, vector database integrations, workers, and other enhancements.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Architecture Overview](#architecture-overview)
|
||||
- [Adding a New LLM Provider](#adding-a-new-llm-provider)
|
||||
- [Adding a Vector Database Integration](#adding-a-vector-database-integration)
|
||||
- [Adding New Workers/Tasks](#adding-new-workerstasks)
|
||||
- [Adding MCP Tools](#adding-mcp-tools)
|
||||
- [Testing Guidelines](#testing-guidelines)
|
||||
- [Code Style and Best Practices](#code-style-and-best-practices)
|
||||
|
||||
---
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
The AI module is organized into several key packages:
|
||||
|
||||
```
|
||||
org.conductoross.conductor.ai/
|
||||
├── providers/ # LLM provider implementations (OpenAI, Anthropic, etc.)
|
||||
├── vectordb/ # Vector database integrations (Pinecone, MongoDB, etc.)
|
||||
├── video/ # Video generation abstractions (VideoModel, AsyncVideoModel, etc.)
|
||||
├── tasks/ # Worker task definitions
|
||||
│ ├── mapper/ # Input/output parameter mappers
|
||||
│ └── worker/ # Worker implementations
|
||||
├── mcp/ # Model Context Protocol implementation
|
||||
├── models/ # Request/response models
|
||||
└── document/ # Document readers and parsers
|
||||
```
|
||||
|
||||
Key interfaces:
|
||||
- **`AIModel`**: Base interface for LLM providers
|
||||
- **`VideoModel`**: Functional interface for synchronous video generation (mirrors Spring AI's `ImageModel`)
|
||||
- **`AsyncVideoModel`**: Extends `VideoModel` with async polling via `checkStatus(String jobId)`
|
||||
- **`VectorDBProvider`**: Base interface for vector databases
|
||||
- **`@WorkerTask`**: Annotation for defining worker tasks
|
||||
|
||||
---
|
||||
|
||||
## Adding a New LLM Provider
|
||||
|
||||
### Step 1: Create Provider Package
|
||||
|
||||
Create a new package under `providers/`:
|
||||
|
||||
```
|
||||
org.conductoross.conductor.ai.providers.yourprovider/
|
||||
├── YourProvider.java # Main provider implementation
|
||||
└── YourProviderConfiguration.java # Spring configuration
|
||||
```
|
||||
|
||||
### Step 2: Implement AIModel Interface
|
||||
|
||||
Create your provider class implementing `AIModel`:
|
||||
|
||||
```java
|
||||
package org.conductoross.conductor.ai.providers.yourprovider;
|
||||
|
||||
import org.conductoross.conductor.ai.AIModel;
|
||||
import org.springframework.ai.chat.model.ChatModel;
|
||||
import org.springframework.ai.embedding.EmbeddingModel;
|
||||
|
||||
public class YourProvider implements AIModel {
|
||||
|
||||
private final ChatModel chatModel;
|
||||
private final EmbeddingModel embeddingModel;
|
||||
|
||||
public YourProvider(ChatModel chatModel, EmbeddingModel embeddingModel) {
|
||||
this.chatModel = chatModel;
|
||||
this.embeddingModel = embeddingModel;
|
||||
}
|
||||
|
||||
@Override
|
||||
public String getModelProvider() {
|
||||
return "your_provider_name"; // Used in workflow definitions
|
||||
}
|
||||
|
||||
@Override
|
||||
public ChatModel getChatModel() {
|
||||
return chatModel;
|
||||
}
|
||||
|
||||
@Override
|
||||
public EmbeddingModel getEmbeddingModel() {
|
||||
return embeddingModel;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Step 3: Create Configuration Class
|
||||
|
||||
Use `@ConditionalOnProperty` to ensure the provider only loads when configured:
|
||||
|
||||
```java
|
||||
package org.conductoross.conductor.ai.providers.yourprovider;
|
||||
|
||||
import org.conductoross.conductor.ai.ModelConfiguration;
|
||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
|
||||
import org.springframework.boot.context.properties.EnableConfigurationProperties;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
import org.springframework.context.annotation.Configuration;
|
||||
|
||||
@Configuration
|
||||
@EnableConfigurationProperties(YourProviderProperties.class)
|
||||
@ConditionalOnProperty(prefix = "conductor.ai.your-provider", name = "api-key")
|
||||
public class YourProviderConfiguration {
|
||||
|
||||
@Bean
|
||||
public ModelConfiguration<YourProvider> yourProviderConfiguration(
|
||||
YourProviderProperties properties) {
|
||||
return () -> {
|
||||
// Initialize chat and embedding models
|
||||
ChatModel chatModel = // ... create from properties
|
||||
EmbeddingModel embeddingModel = // ... create from properties
|
||||
|
||||
return new YourProvider(chatModel, embeddingModel);
|
||||
};
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Step 4: Create Properties Class
|
||||
|
||||
```java
|
||||
package org.conductoross.conductor.ai.providers.yourprovider;
|
||||
|
||||
import org.springframework.boot.context.properties.ConfigurationProperties;
|
||||
import lombok.Data;
|
||||
|
||||
@Data
|
||||
@ConfigurationProperties(prefix = "conductor.ai.your-provider")
|
||||
public class YourProviderProperties {
|
||||
private String apiKey;
|
||||
private String baseUrl = "https://api.yourprovider.com";
|
||||
private String model = "default-model";
|
||||
// Add other configuration properties
|
||||
}
|
||||
```
|
||||
|
||||
### Step 5: Add Tests
|
||||
|
||||
Create `YourProviderConfigurationTest.java`:
|
||||
|
||||
```java
|
||||
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
|
||||
class YourProviderConfigurationTest {
|
||||
|
||||
@Test
|
||||
void testProviderLoadsWhenConfigured() {
|
||||
ApplicationContextRunner contextRunner =
|
||||
new ApplicationContextRunner()
|
||||
.withConfiguration(
|
||||
AutoConfigurations.of(YourProviderConfiguration.class))
|
||||
.withPropertyValues(
|
||||
"conductor.ai.your-provider.api-key=test-key");
|
||||
|
||||
contextRunner.run(
|
||||
context -> {
|
||||
assertThat(context).hasSingleBean(ModelConfiguration.class);
|
||||
});
|
||||
}
|
||||
|
||||
@Test
|
||||
void testProviderDoesNotLoadWithoutApiKey() {
|
||||
ApplicationContextRunner contextRunner =
|
||||
new ApplicationContextRunner()
|
||||
.withConfiguration(
|
||||
AutoConfigurations.of(YourProviderConfiguration.class));
|
||||
|
||||
contextRunner.run(
|
||||
context -> {
|
||||
assertThat(context).doesNotHaveBean(ModelConfiguration.class);
|
||||
});
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Step 6: Add Video Generation Support (Optional)
|
||||
|
||||
If your provider supports video generation, implement video model support using the `video/` package abstractions. Video generation is async by nature (submit a job, poll for results), so most providers will implement `AsyncVideoModel`.
|
||||
|
||||
#### 6a. Create a Video Model Class
|
||||
|
||||
```java
|
||||
package org.conductoross.conductor.ai.providers.yourprovider;
|
||||
|
||||
import org.conductoross.conductor.ai.video.*;
|
||||
|
||||
public class YourVideoModel implements AsyncVideoModel {
|
||||
|
||||
private final String apiKey;
|
||||
|
||||
public YourVideoModel(String apiKey) {
|
||||
this.apiKey = apiKey;
|
||||
}
|
||||
|
||||
@Override
|
||||
public VideoResponse call(VideoPrompt prompt) {
|
||||
// Submit video generation job to provider API
|
||||
// Return a VideoResponse with jobId in metadata
|
||||
VideoResponseMetadata metadata = new VideoResponseMetadata();
|
||||
metadata.put("jobId", submittedJobId);
|
||||
metadata.put("status", "PENDING");
|
||||
return new VideoResponse(List.of(), metadata);
|
||||
}
|
||||
|
||||
@Override
|
||||
public VideoResponse checkStatus(String jobId) {
|
||||
// Poll provider API for job status
|
||||
// When complete, download video bytes and return Video objects
|
||||
// Set mimeType on each Video (e.g., "video/mp4", "image/webp" for thumbnails)
|
||||
Video video = new Video(videoUrl, null, "video/mp4");
|
||||
VideoGeneration generation = new VideoGeneration(video);
|
||||
|
||||
VideoResponseMetadata metadata = new VideoResponseMetadata();
|
||||
metadata.put("jobId", jobId);
|
||||
metadata.put("status", "COMPLETED");
|
||||
return new VideoResponse(List.of(generation), metadata);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### 6b. Wire Video Model into Your Provider
|
||||
|
||||
Override the video-related methods in your `AIModel` implementation:
|
||||
|
||||
```java
|
||||
@Override
|
||||
public VideoModel getVideoModel() {
|
||||
if (videoModel == null) {
|
||||
videoModel = new YourVideoModel(apiKey);
|
||||
}
|
||||
return videoModel;
|
||||
}
|
||||
|
||||
@Override
|
||||
public LLMResponse generateVideo(VideoGenRequest request) {
|
||||
VideoOptions options = getVideoOptions(request);
|
||||
VideoPrompt prompt = new VideoPrompt(
|
||||
List.of(new VideoMessage(request.getPrompt())), options);
|
||||
VideoResponse response = getVideoModel().call(prompt);
|
||||
// Convert to LLMResponse with jobId
|
||||
}
|
||||
|
||||
@Override
|
||||
public LLMResponse checkVideoStatus(VideoGenRequest request) {
|
||||
AsyncVideoModel asyncModel = (AsyncVideoModel) getVideoModel();
|
||||
VideoResponse response = asyncModel.checkStatus(request.getJobId());
|
||||
// Convert to LLMResponse with media list
|
||||
}
|
||||
```
|
||||
|
||||
The `video/` package mirrors Spring AI's `Image*` abstraction pattern:
|
||||
- `VideoPrompt` -> `ImagePrompt` (request wrapper)
|
||||
- `VideoResponse` -> `ImageResponse` (response wrapper)
|
||||
- `VideoGeneration` -> `ImageGeneration` (individual result)
|
||||
- `Video` -> `Image` (the actual media, with url, b64Json, and mimeType fields)
|
||||
- `VideoOptions` -> `ImageOptions` (generation parameters)
|
||||
|
||||
### Step 7: Update Documentation
|
||||
|
||||
Add your provider to `README.md` under the supported providers section with configuration examples.
|
||||
|
||||
---
|
||||
|
||||
## Adding a Vector Database Integration
|
||||
|
||||
### Step 1: Create Config Class
|
||||
|
||||
Create a new configuration class in the database package (e.g., `org.conductoross.conductor.ai.vectordb.yourdb`):
|
||||
|
||||
```java
|
||||
@Data
|
||||
@NoArgsConstructor
|
||||
@AllArgsConstructor
|
||||
public class YourDBConfig implements VectorDBConfig<YourVectorDB> {
|
||||
|
||||
private String connectionString;
|
||||
// other properties
|
||||
|
||||
@Override
|
||||
public YourVectorDB get() {
|
||||
throw new UnsupportedOperationException("Use get(String name) instead");
|
||||
}
|
||||
|
||||
public YourVectorDB get(String name) {
|
||||
return new YourVectorDB(name, this);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Implement VectorDB Class
|
||||
|
||||
Extend the `VectorDB` abstract class:
|
||||
|
||||
```java
|
||||
public class YourVectorDB extends VectorDB {
|
||||
|
||||
public static final String TYPE = "yourdb";
|
||||
private final YourDBConfig config;
|
||||
|
||||
public YourVectorDB(String name, YourDBConfig config) {
|
||||
super(name, TYPE);
|
||||
this.config = config;
|
||||
}
|
||||
|
||||
@Override
|
||||
public int updateEmbeddings(String indexName, String namespace, String doc, String parentDocId, String id, List<Float> embeddings, Map<String, Object> metadata) {
|
||||
// Implement logic to store embeddings
|
||||
}
|
||||
|
||||
@Override
|
||||
public List<IndexedDoc> search(String indexName, String namespace, List<Float> embeddings, int maxResults) {
|
||||
// Implement logic to search embeddings
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Step 3: Register in VectorDBInstanceConfig
|
||||
|
||||
Add your database type to the `createVectorDB` method and the `VectorDBInstance` inner class in `org.conductoross.conductor.ai.vectordb.VectorDBInstanceConfig`.
|
||||
|
||||
### Step 4: Add Integration Tests
|
||||
|
||||
Use Testcontainers for integration testing:
|
||||
|
||||
```java
|
||||
@Testcontainers
|
||||
class YourVectorDBTest {
|
||||
|
||||
@Container
|
||||
static GenericContainer<?> yourdb =
|
||||
new GenericContainer<>("yourdb:latest")
|
||||
.withExposedPorts(1234);
|
||||
|
||||
@Test
|
||||
void testStoreAndSearch() {
|
||||
// Test vector storage and similarity search
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Adding New Workers/Tasks
|
||||
|
||||
### Step 1: Create Request Model
|
||||
|
||||
```java
|
||||
package org.conductoross.conductor.ai.model;
|
||||
|
||||
import lombok.Data;
|
||||
import lombok.EqualsAndHashCode;
|
||||
|
||||
@Data
|
||||
@EqualsAndHashCode(callSuper = false)
|
||||
public class YourTaskRequest extends LLMWorkerInput {
|
||||
private String parameter1;
|
||||
private String parameter2;
|
||||
// Add task-specific parameters
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Create Worker Class
|
||||
|
||||
```java
|
||||
package org.conductoross.conductor.ai.tasks.worker;
|
||||
|
||||
import com.netflix.conductor.sdk.workflow.annotations.WorkerTask;
|
||||
import org.conductoross.conductor.ai.model.YourTaskRequest;
|
||||
|
||||
@Component
|
||||
public class YourWorker {
|
||||
|
||||
private final YourService yourService;
|
||||
|
||||
public YourWorker(YourService yourService) {
|
||||
this.yourService = yourService;
|
||||
}
|
||||
|
||||
@WorkerTask("YOUR_TASK_NAME")
|
||||
public @OutputParam("result") YourTaskResult executeTask(YourTaskRequest request) {
|
||||
// Implement task logic
|
||||
return yourService.processRequest(request);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Step 3: Add Task Tests
|
||||
|
||||
```java
|
||||
class YourWorkerTest {
|
||||
|
||||
@Test
|
||||
void testTaskExecution() {
|
||||
YourWorker worker = new YourWorker(mockService);
|
||||
YourTaskRequest request = new YourTaskRequest();
|
||||
request.setParameter1("test");
|
||||
|
||||
YourTaskResult result = worker.executeTask(request);
|
||||
|
||||
assertNotNull(result);
|
||||
// Add assertions
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Adding MCP Tools
|
||||
|
||||
Model Context Protocol (MCP) allows external tools to be called from workflows.
|
||||
|
||||
### Adding MCP Server Support
|
||||
|
||||
The `MCPService` already supports:
|
||||
- HTTP/SSE transports
|
||||
- stdio (local process) transports
|
||||
- Direct JSON-RPC fallback
|
||||
|
||||
To add a new MCP server:
|
||||
|
||||
1. **Deploy your MCP server** (HTTP or local script)
|
||||
2. **Use existing `CALL_MCP_TOOL` task** in workflows:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "call_your_tool",
|
||||
"taskReferenceName": "your_tool",
|
||||
"type": "CALL_MCP_TOOL",
|
||||
"inputParameters": {
|
||||
"mcpServer": "http://localhost:3000",
|
||||
"methodName": "your_tool_name",
|
||||
"param1": "value1",
|
||||
"param2": "value2"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Extending MCP Capabilities
|
||||
|
||||
To add new MCP-related features, modify:
|
||||
- `MCPService.java` - Core MCP communication logic
|
||||
- `MCPWorkers.java` - Worker task definitions
|
||||
- `models/MCP*.java` - Request/response models
|
||||
|
||||
---
|
||||
|
||||
## Testing Guidelines
|
||||
|
||||
### Unit Tests
|
||||
|
||||
- Place in `src/test/java` mirroring the source structure
|
||||
- Use MockBean for Spring dependencies
|
||||
- Test individual methods and edge cases
|
||||
- Aim for 80%+ code coverage
|
||||
|
||||
### Integration Tests
|
||||
|
||||
- Use `@SpringBootTest` for full context testing
|
||||
- Use Testcontainers for external dependencies (databases, servers)
|
||||
- Test real interactions between components
|
||||
|
||||
### Test Naming Convention
|
||||
|
||||
```java
|
||||
// Unit test method format
|
||||
void test<MethodName>_<Scenario>_<ExpectedResult>()
|
||||
|
||||
// Examples:
|
||||
void testGetModel_WithValidProvider_ReturnsModel()
|
||||
void testGetModel_WithInvalidProvider_ThrowsException()
|
||||
```
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
./gradlew :conductor-ai:test
|
||||
|
||||
# Run specific test class
|
||||
./gradlew :conductor-ai:test --tests YourProviderTest
|
||||
|
||||
# Run with coverage
|
||||
./gradlew :conductor-ai:test jacocoTestReport
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Code Style and Best Practices
|
||||
|
||||
### Lombok Usage
|
||||
|
||||
Use Lombok annotations consistently:
|
||||
- `@Data` for simple POJOs
|
||||
- `@Builder` for complex object construction
|
||||
- `@Slf4j` for logging
|
||||
- `@AllArgsConstructor` / `@NoArgsConstructor` for constructors
|
||||
|
||||
### Logging
|
||||
|
||||
- Use SLF4J via `@Slf4j`
|
||||
- Log levels:
|
||||
- `log.debug()` - Detailed diagnostic information
|
||||
- `log.info()` - Important business events
|
||||
- `log.warn()` - Recoverable issues
|
||||
- `log.error()` - Errors requiring attention
|
||||
|
||||
### Error Handling
|
||||
|
||||
- Throw descriptive exceptions
|
||||
- Include context in error messages
|
||||
- Use try-catch for recoverable errors
|
||||
- Let unchecked exceptions propagate for programming errors
|
||||
|
||||
### Configuration Properties
|
||||
|
||||
- Use `@ConfigurationProperties` for type-safe configuration
|
||||
- Provide sensible defaults
|
||||
- Document all properties in javadoc
|
||||
- Use `@ConditionalOnProperty` to make features optional
|
||||
|
||||
### Spring Beans
|
||||
|
||||
- Prefer constructor injection over field injection
|
||||
- Use `@Component` for auto-detected beans
|
||||
- Use `@Configuration` for explicit bean definitions
|
||||
- Apply `@ConditionalOnProperty` for optional features
|
||||
|
||||
### Documentation
|
||||
|
||||
- Add Javadoc to all public classes and methods
|
||||
- Include usage examples in class-level Javadoc
|
||||
- Update `README.md` with new features
|
||||
- Provide workflow examples for new tasks
|
||||
|
||||
---
|
||||
|
||||
## Development Workflow
|
||||
|
||||
### 1. Create a Feature Branch
|
||||
|
||||
```bash
|
||||
git checkout -b feature/add-your-provider
|
||||
```
|
||||
|
||||
### 2. Implement Your Changes
|
||||
|
||||
Follow the patterns above for your contribution type.
|
||||
|
||||
### 3. Write Tests
|
||||
|
||||
Ensure your code has comprehensive test coverage.
|
||||
|
||||
### 4. Run Tests and Checks
|
||||
|
||||
```bash
|
||||
./gradlew :conductor-ai:test
|
||||
./gradlew :conductor-ai:compileJava
|
||||
```
|
||||
|
||||
### 5. Update Documentation
|
||||
|
||||
- Update `README.md` with examples
|
||||
- Add Javadoc to new classes
|
||||
- Update this CONTRIBUTING.md if adding new patterns
|
||||
|
||||
### 6. Submit Pull Request
|
||||
|
||||
- Provide clear description of changes
|
||||
- Reference any related issues
|
||||
- Include test results
|
||||
- Update changelog if applicable
|
||||
|
||||
---
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### Conditional Bean Creation
|
||||
|
||||
Always use `@ConditionalOnProperty` for optional integrations:
|
||||
|
||||
```java
|
||||
@ConditionalOnProperty(
|
||||
prefix = "conductor.ai.your-feature",
|
||||
name = "enabled",
|
||||
havingValue = "true"
|
||||
)
|
||||
```
|
||||
|
||||
### Parameter Mapping
|
||||
|
||||
For workers with dynamic parameters, use `@JsonAnySetter`:
|
||||
|
||||
```java
|
||||
@JsonAnySetter
|
||||
public void setAdditionalProperty(String key, Object value) {
|
||||
additionalProperties.put(key, value);
|
||||
}
|
||||
```
|
||||
|
||||
### Resource Cleanup
|
||||
|
||||
Implement `DisposableBean` for cleanup:
|
||||
|
||||
```java
|
||||
@Override
|
||||
public void destroy() throws Exception {
|
||||
// Clean up resources
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Getting Help
|
||||
|
||||
- Check existing implementations in `providers/` for examples
|
||||
- Review `README.md` for usage patterns
|
||||
- Look at test files for testing patterns
|
||||
- Open a GitHub issue for questions
|
||||
|
||||
## License
|
||||
|
||||
By contributing, you agree that your contributions will be licensed under the Apache License 2.0.
|
||||
@@ -0,0 +1,257 @@
|
||||
# JDBC Configuration
|
||||
|
||||
This document describes the configuration format for JDBC database connections in Conductor.
|
||||
|
||||
## Overview
|
||||
|
||||
Conductor supports configuring **multiple named JDBC instances** for use by the `JDBC` worker task. This allows you to:
|
||||
|
||||
- Connect to multiple databases (MySQL, PostgreSQL, Oracle, etc.)
|
||||
- Separate environments (prod, dev, staging)
|
||||
- Use different connection pool settings per use case (read-heavy vs write-heavy)
|
||||
|
||||
## Configuration Format
|
||||
|
||||
JDBC instances are configured using a list-based approach under `conductor.jdbc.instances`:
|
||||
|
||||
```yaml
|
||||
conductor:
|
||||
jdbc:
|
||||
instances:
|
||||
- name: "instance-name" # Unique identifier for this instance
|
||||
connection: # Connection configuration
|
||||
datasourceURL: "jdbc:..." # JDBC connection URL
|
||||
jdbcDriver: "..." # JDBC driver class (optional, auto-detected from URL)
|
||||
user: "..." # Database username
|
||||
password: "..." # Database password
|
||||
# ... pool settings
|
||||
```
|
||||
|
||||
## Configuration Examples
|
||||
|
||||
### Single MySQL Instance
|
||||
|
||||
```yaml
|
||||
conductor:
|
||||
jdbc:
|
||||
instances:
|
||||
- name: "mysql-prod"
|
||||
connection:
|
||||
datasourceURL: "jdbc:mysql://prod-db:3306/myapp"
|
||||
jdbcDriver: "com.mysql.cj.jdbc.Driver"
|
||||
user: "conductor"
|
||||
password: "secret"
|
||||
maximumPoolSize: 20
|
||||
minimumIdle: 5
|
||||
```
|
||||
|
||||
### Multiple Instances
|
||||
|
||||
```yaml
|
||||
conductor:
|
||||
jdbc:
|
||||
instances:
|
||||
- name: "mysql-prod"
|
||||
connection:
|
||||
datasourceURL: "jdbc:mysql://prod-db:3306/myapp"
|
||||
jdbcDriver: "com.mysql.cj.jdbc.Driver"
|
||||
user: "conductor"
|
||||
password: "prod-secret"
|
||||
maximumPoolSize: 20
|
||||
|
||||
- name: "postgres-analytics"
|
||||
connection:
|
||||
datasourceURL: "jdbc:postgresql://analytics-db:5432/warehouse"
|
||||
user: "analyst"
|
||||
password: "analytics-secret"
|
||||
maximumPoolSize: 10
|
||||
|
||||
- name: "mysql-staging"
|
||||
connection:
|
||||
datasourceURL: "jdbc:mysql://staging-db:3306/myapp"
|
||||
jdbcDriver: "com.mysql.cj.jdbc.Driver"
|
||||
user: "conductor"
|
||||
password: "staging-secret"
|
||||
maximumPoolSize: 5
|
||||
minimumIdle: 1
|
||||
```
|
||||
|
||||
## Usage in Workflows
|
||||
|
||||
When using the JDBC task in your workflows, reference the instance by its configured name using `connectionId`:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "query_users",
|
||||
"taskReferenceName": "query_users_ref",
|
||||
"type": "JDBC",
|
||||
"inputParameters": {
|
||||
"connectionId": "mysql-prod",
|
||||
"type": "SELECT",
|
||||
"statement": "SELECT id, name, email FROM users WHERE status = ?",
|
||||
"parameters": ["active"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### SELECT Example
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "find_orders",
|
||||
"taskReferenceName": "find_orders_ref",
|
||||
"type": "JDBC",
|
||||
"inputParameters": {
|
||||
"connectionId": "postgres-analytics",
|
||||
"type": "SELECT",
|
||||
"statement": "SELECT order_id, total FROM orders WHERE customer_id = ?",
|
||||
"parameters": ["${workflow.input.customerId}"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Output:
|
||||
```json
|
||||
{
|
||||
"result": [
|
||||
{"order_id": 101, "total": 49.99},
|
||||
{"order_id": 205, "total": 129.50}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### UPDATE Example
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "update_status",
|
||||
"taskReferenceName": "update_status_ref",
|
||||
"type": "JDBC",
|
||||
"inputParameters": {
|
||||
"connectionId": "mysql-prod",
|
||||
"type": "UPDATE",
|
||||
"statement": "UPDATE orders SET status = ? WHERE order_id = ?",
|
||||
"parameters": ["shipped", "${workflow.input.orderId}"],
|
||||
"expectedUpdateCount": 1
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Output:
|
||||
```json
|
||||
{
|
||||
"update_count": 1
|
||||
}
|
||||
```
|
||||
|
||||
If the actual update count does not match `expectedUpdateCount`, the transaction is rolled back and the task fails.
|
||||
|
||||
## Connection Configuration Options
|
||||
|
||||
| Property | Type | Default | Description |
|
||||
|----------|------|---------|-------------|
|
||||
| `datasourceURL` | String | Required | JDBC connection URL |
|
||||
| `jdbcDriver` | String | Auto-detected | JDBC driver class name |
|
||||
| `user` | String | Optional | Database username |
|
||||
| `password` | String | Optional | Database password |
|
||||
| `maximumPoolSize` | Integer | 32 | Maximum connections in the pool |
|
||||
| `minimumIdle` | Integer | 2 | Minimum idle connections |
|
||||
| `idleTimeoutMs` | Long | 30000 | Idle connection timeout (ms) |
|
||||
| `connectionTimeout` | Long | 30000 | Connection acquisition timeout (ms) |
|
||||
| `leakDetectionThreshold` | Long | 60000 | Leak detection threshold (ms) |
|
||||
| `maxLifetime` | Long | 1800000 | Maximum connection lifetime (ms) |
|
||||
|
||||
## Migration from Old Configuration
|
||||
|
||||
### Old Format
|
||||
|
||||
```properties
|
||||
conductor.worker.jdbc.connectionIds=mysql,postgres
|
||||
conductor.worker.jdbc.mysql.connectionURL=jdbc:mysql://localhost:3306/db
|
||||
conductor.worker.jdbc.mysql.driverClassName=com.mysql.cj.jdbc.Driver
|
||||
conductor.worker.jdbc.mysql.username=root
|
||||
conductor.worker.jdbc.mysql.password=secret
|
||||
conductor.worker.jdbc.mysql.maximum-pool-size=10
|
||||
|
||||
conductor.worker.jdbc.postgres.connectionURL=jdbc:postgresql://localhost:5432/db
|
||||
conductor.worker.jdbc.postgres.driverClassName=org.postgresql.Driver
|
||||
conductor.worker.jdbc.postgres.username=pguser
|
||||
conductor.worker.jdbc.postgres.password=pgpass
|
||||
```
|
||||
|
||||
### New Format
|
||||
|
||||
```yaml
|
||||
conductor:
|
||||
jdbc:
|
||||
instances:
|
||||
- name: "mysql"
|
||||
connection:
|
||||
datasourceURL: "jdbc:mysql://localhost:3306/db"
|
||||
jdbcDriver: "com.mysql.cj.jdbc.Driver"
|
||||
user: "root"
|
||||
password: "secret"
|
||||
maximumPoolSize: 10
|
||||
|
||||
- name: "postgres"
|
||||
connection:
|
||||
datasourceURL: "jdbc:postgresql://localhost:5432/db"
|
||||
jdbcDriver: "org.postgresql.Driver"
|
||||
user: "pguser"
|
||||
password: "pgpass"
|
||||
```
|
||||
|
||||
**Note:** The old `conductor.worker.jdbc.*` format is still supported for backwards compatibility. If no `conductor.jdbc.instances` are configured, the system automatically falls back to reading the legacy format. The old and new formats are mutually exclusive -- if new-format instances are found, the legacy format is ignored.
|
||||
|
||||
### Property Name Mapping
|
||||
|
||||
| Old Property | New Property |
|
||||
|---|---|
|
||||
| `connectionURL` | `datasourceURL` |
|
||||
| `driverClassName` | `jdbcDriver` |
|
||||
| `username` | `user` |
|
||||
| `password` | `password` |
|
||||
| `maximum-pool-size` | `maximumPoolSize` |
|
||||
| `idle-timeout-ms` | `idleTimeoutMs` |
|
||||
| `minimum-idle` | `minimumIdle` |
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use descriptive names**: Choose instance names that clearly indicate their purpose (e.g., `mysql-prod`, `postgres-analytics`, `oracle-reporting`)
|
||||
|
||||
2. **Separate read/write pools**: For high-throughput systems, configure separate instances for read and write operations with appropriate pool sizes
|
||||
|
||||
3. **Right-size connection pools**: Set `maximumPoolSize` based on your database capacity and workload. A common formula is `connections = (core_count * 2) + effective_spindle_count`
|
||||
|
||||
4. **Enable leak detection**: The default `leakDetectionThreshold` of 60 seconds logs warnings for connections held longer than expected
|
||||
|
||||
5. **Use parameterized queries**: Always use `?` placeholders with the `parameters` list instead of string concatenation to prevent SQL injection
|
||||
|
||||
6. **Set expectedUpdateCount**: For critical UPDATE/INSERT/DELETE operations, set `expectedUpdateCount` to automatically rollback if the affected row count doesn't match
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Instance Not Found
|
||||
|
||||
If you see "JDBC instance not found: xyz", check:
|
||||
|
||||
1. The `connectionId` in your workflow matches the configured `name` exactly
|
||||
2. The instance is properly configured in your application.yml/properties
|
||||
3. The application has been restarted after configuration changes
|
||||
|
||||
### Connection Timeout
|
||||
|
||||
If connections are timing out:
|
||||
|
||||
1. Verify network connectivity to the database
|
||||
2. Check `connectionTimeout` value (default 30 seconds)
|
||||
3. Ensure the connection pool is not exhausted (increase `maximumPoolSize` if needed)
|
||||
4. Check database max connections limit
|
||||
|
||||
### Connection Leaks
|
||||
|
||||
If you see leak detection warnings:
|
||||
|
||||
1. Ensure all connections are properly closed (the JDBC worker handles this automatically)
|
||||
2. If using custom integrations, wrap connection usage in try-with-resources
|
||||
3. Review `leakDetectionThreshold` setting
|
||||
+1699
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,331 @@
|
||||
# Vector Database Configuration
|
||||
|
||||
This document describes the configuration format for vector databases in Conductor.
|
||||
|
||||
## Overview
|
||||
|
||||
Conductor supports multiple vector database providers with the ability to configure **multiple named instances** of each type. This allows you to:
|
||||
|
||||
- Use multiple databases of the same type (e.g., multiple PostgreSQL instances)
|
||||
- Connect to different environments (prod, dev, staging)
|
||||
- Separate concerns by use case (embeddings, search, recommendations)
|
||||
|
||||
## Supported Vector Databases
|
||||
|
||||
- **PostgreSQL** (with pgvector extension)
|
||||
- **MongoDB** (with Atlas Vector Search)
|
||||
- **Pinecone**
|
||||
- **SQLite** (with the [sqlite-vec](https://github.com/asg017/sqlite-vec) extension) — embedded, zero-infrastructure backend for local development, demos and small deployments
|
||||
|
||||
## Configuration Format
|
||||
|
||||
Vector databases are configured using a list-based approach under `conductor.vectordb.instances`:
|
||||
|
||||
```yaml
|
||||
conductor:
|
||||
vectordb:
|
||||
instances:
|
||||
- name: "instance-name" # Unique identifier for this instance
|
||||
type: "database-type" # Type: postgres, mongodb, pinecone, or sqlite
|
||||
<type-specific-config>: # Configuration block for the database type
|
||||
# ... type-specific properties
|
||||
```
|
||||
|
||||
## Configuration Examples
|
||||
|
||||
### Single PostgreSQL Instance
|
||||
|
||||
```yaml
|
||||
conductor:
|
||||
vectordb:
|
||||
instances:
|
||||
- name: "postgres-main"
|
||||
type: "postgres"
|
||||
postgres:
|
||||
datasourceURL: "jdbc:postgresql://localhost:5432/vectors"
|
||||
user: "conductor"
|
||||
password: "secret"
|
||||
dimensions: 1536
|
||||
connectionPoolSize: 10
|
||||
indexingMethod: "hnsw" # Options: hnsw, ivfflat
|
||||
distanceMetric: "cosine" # Options: l2, cosine, inner_product
|
||||
tablePrefix: "conductor"
|
||||
```
|
||||
|
||||
### Multiple PostgreSQL Instances
|
||||
|
||||
```yaml
|
||||
conductor:
|
||||
vectordb:
|
||||
instances:
|
||||
- name: "postgres-prod"
|
||||
type: "postgres"
|
||||
postgres:
|
||||
datasourceURL: "jdbc:postgresql://prod-db:5432/vectors"
|
||||
user: "conductor"
|
||||
password: "prod-secret"
|
||||
dimensions: 1536
|
||||
|
||||
- name: "postgres-dev"
|
||||
type: "postgres"
|
||||
postgres:
|
||||
datasourceURL: "jdbc:postgresql://dev-db:5432/vectors"
|
||||
user: "conductor"
|
||||
password: "dev-secret"
|
||||
dimensions: 768
|
||||
```
|
||||
|
||||
### MongoDB Atlas Vector Search
|
||||
|
||||
```yaml
|
||||
conductor:
|
||||
vectordb:
|
||||
instances:
|
||||
- name: "mongodb-embeddings"
|
||||
type: "mongodb"
|
||||
mongodb:
|
||||
connectionString: "mongodb+srv://user:pass@cluster.mongodb.net/"
|
||||
database: "conductor"
|
||||
collection: "embeddings"
|
||||
numCandidates: 100
|
||||
```
|
||||
|
||||
### Pinecone
|
||||
|
||||
```yaml
|
||||
conductor:
|
||||
vectordb:
|
||||
instances:
|
||||
- name: "pinecone-search"
|
||||
type: "pinecone"
|
||||
pinecone:
|
||||
apiKey: "your-pinecone-api-key"
|
||||
```
|
||||
|
||||
### SQLite (sqlite-vec)
|
||||
|
||||
```yaml
|
||||
conductor:
|
||||
vectordb:
|
||||
instances:
|
||||
- name: "sqlite-local"
|
||||
type: "sqlite"
|
||||
sqlite:
|
||||
dbPath: "/var/lib/conductor/vectordb.db" # use ":memory:" for an ephemeral DB
|
||||
dimensions: 1536
|
||||
distanceMetric: "cosine" # Options: l2, cosine, l1
|
||||
connectionPoolSize: 5
|
||||
tablePrefix: "conductor"
|
||||
# extensionPath is optional — the native vec0 binary is bundled in the jar.
|
||||
# Set it only to override with a custom build:
|
||||
# extensionPath: "/opt/sqlite-vec/vec0"
|
||||
```
|
||||
|
||||
> **The native `vec0` extension is bundled.** Conductor ships the official, checksum-pinned sqlite-vec
|
||||
> loadable binaries for linux (x86_64/aarch64), macOS (x86_64/aarch64) and windows (x86_64) inside the
|
||||
> AI jar, and extracts the right one at runtime — so `extensionPath` is normally unnecessary. Provide
|
||||
> `extensionPath` (the file name without its platform suffix, e.g. `/opt/sqlite-vec/vec0`) only to
|
||||
> override the bundled binary or to support a platform that is not bundled. sqlite-vec is pre-v1 and
|
||||
> performs exact (brute-force) KNN with no ANN index, making it suitable for thousands to low-millions
|
||||
> of vectors.
|
||||
|
||||
### Zero-config default (SQLite persistence + AI)
|
||||
|
||||
When the server runs with **both** `conductor.db.type=sqlite` and `conductor.integrations.ai.enabled=true`,
|
||||
Conductor automatically registers a vector DB instance named **`default`** backed by the bundled
|
||||
sqlite-vec extension — no `conductor.vectordb.instances` entry required. Workflows can target it with
|
||||
`"vectorDB": "default"`. The defaults below can be overridden:
|
||||
|
||||
```yaml
|
||||
conductor:
|
||||
vectordb:
|
||||
sqlite-default:
|
||||
name: "default" # instance name workflows reference
|
||||
db-path: "" # default: a *_vectordb.db file next to the persistence DB
|
||||
dimensions: 256 # must match the embedding model's output dimensions
|
||||
distance-metric: "cosine" # l2, cosine or l1
|
||||
extension-path: "" # default: the bundled vec0 binary
|
||||
```
|
||||
|
||||
An explicitly configured instance named `default` takes precedence over the auto-registered one.
|
||||
|
||||
### Mixed Configuration (Multiple Types)
|
||||
|
||||
```yaml
|
||||
conductor:
|
||||
vectordb:
|
||||
instances:
|
||||
- name: "postgres-prod"
|
||||
type: "postgres"
|
||||
postgres:
|
||||
datasourceURL: "jdbc:postgresql://prod:5432/vectors"
|
||||
user: "conductor"
|
||||
password: "secret"
|
||||
dimensions: 1536
|
||||
|
||||
- name: "pinecone-embeddings"
|
||||
type: "pinecone"
|
||||
pinecone:
|
||||
apiKey: "pk-xxx"
|
||||
|
||||
- name: "mongodb-cache"
|
||||
type: "mongodb"
|
||||
mongodb:
|
||||
connectionString: "mongodb://localhost:27017"
|
||||
database: "conductor"
|
||||
```
|
||||
|
||||
## Usage in Workflows
|
||||
|
||||
When using vector database tasks in your workflows, reference the instance by its configured name:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "store_embeddings",
|
||||
"taskReferenceName": "store_embeddings_ref",
|
||||
"type": "LLM_STORE_EMBEDDINGS",
|
||||
"inputParameters": {
|
||||
"vectorDB": "postgres-prod",
|
||||
"index": "documents",
|
||||
"namespace": "my_namespace",
|
||||
"embeddings": "${embedding_task.output.embeddings}",
|
||||
"metadata": {
|
||||
"documentId": "${workflow.input.docId}"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## PostgreSQL Configuration Options
|
||||
|
||||
| Property | Type | Default | Description |
|
||||
|----------|------|---------|-------------|
|
||||
| `datasourceURL` | String | Required | JDBC connection URL |
|
||||
| `user` | String | Required | Database username |
|
||||
| `password` | String | Required | Database password |
|
||||
| `dimensions` | Integer | 256 | Vector dimensions |
|
||||
| `connectionPoolSize` | Integer | 5 | Connection pool size |
|
||||
| `indexingMethod` | String | "hnsw" | Index method (hnsw or ivfflat) |
|
||||
| `distanceMetric` | String | "l2" | Distance metric (l2, cosine, inner_product) |
|
||||
| `invertedListCount` | Integer | 100 | IVFFlat index parameter |
|
||||
| `tablePrefix` | String | null | Prefix for table names |
|
||||
|
||||
## MongoDB Configuration Options
|
||||
|
||||
| Property | Type | Default | Description |
|
||||
|----------|------|---------|-------------|
|
||||
| `connectionString` | String | Required | MongoDB connection string |
|
||||
| `database` | String | Required | Database name |
|
||||
| `collection` | String | Optional | Collection name |
|
||||
| `numCandidates` | Integer | Optional | Vector search parameter |
|
||||
|
||||
## Pinecone Configuration Options
|
||||
|
||||
| Property | Type | Default | Description |
|
||||
|----------|------|---------|-------------|
|
||||
| `apiKey` | String | Required | Pinecone API key |
|
||||
|
||||
## SQLite Configuration Options
|
||||
|
||||
| Property | Type | Default | Description |
|
||||
|----------|------|---------|-------------|
|
||||
| `dbPath` | String | Required | Path to the SQLite database file, or `:memory:` for an in-memory DB |
|
||||
| `extensionPath` | String | bundled | Override path to the `vec0` extension (without platform suffix); defaults to the binary bundled in the jar |
|
||||
| `dimensions` | Integer | 256 | Vector dimensions |
|
||||
| `distanceMetric` | String | "l2" | Distance metric (l2, cosine, l1) |
|
||||
| `connectionPoolSize` | Integer | 5 | Connection pool size (forced to 1 for `:memory:`) |
|
||||
| `tablePrefix` | String | null | Prefix for table names |
|
||||
|
||||
## Migration from Old Configuration
|
||||
|
||||
### Old Format (Single Instance Per Type)
|
||||
|
||||
```yaml
|
||||
conductor:
|
||||
vectordb:
|
||||
postgres:
|
||||
datasourceURL: "jdbc:postgresql://localhost:5432/vectors"
|
||||
user: "conductor"
|
||||
password: "secret"
|
||||
```
|
||||
|
||||
### New Format (Named Instances)
|
||||
|
||||
```yaml
|
||||
conductor:
|
||||
vectordb:
|
||||
instances:
|
||||
- name: "pgvectordb" # Use old type name for backward compatibility
|
||||
type: "postgres"
|
||||
postgres:
|
||||
datasourceURL: "jdbc:postgresql://localhost:5432/vectors"
|
||||
user: "conductor"
|
||||
password: "secret"
|
||||
```
|
||||
|
||||
**Note:** The type identifiers have been simplified:
|
||||
- `pgvectordb` → `postgres`
|
||||
- `mongovectordb` → `mongodb`
|
||||
- `pineconedb` → `pinecone`
|
||||
|
||||
However, for backward compatibility, you can still reference instances using the old type names if you name your instance accordingly.
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use descriptive names**: Choose instance names that clearly indicate their purpose (e.g., `postgres-prod`, `pinecone-embeddings-search`)
|
||||
|
||||
2. **Separate environments**: Use different instances for different environments to avoid accidental data mixing
|
||||
|
||||
3. **Optimize dimensions**: Configure `dimensions` to match your embedding model to avoid runtime errors
|
||||
|
||||
4. **Connection pooling**: Adjust `connectionPoolSize` based on your workload and database capacity
|
||||
|
||||
5. **Index selection**:
|
||||
- Use `hnsw` for better query performance (default)
|
||||
- Use `ivfflat` for faster indexing with slightly lower query performance
|
||||
|
||||
6. **Distance metrics**:
|
||||
- Use `cosine` for normalized embeddings
|
||||
- Use `l2` (Euclidean) for absolute distances
|
||||
- Use `inner_product` for dot product similarity
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Instance Not Found
|
||||
|
||||
If you see an error like "Vector DB instance not found: xyz", check:
|
||||
|
||||
1. The instance name in your workflow matches the configured name exactly
|
||||
2. The instance is properly configured in your application.yml/properties
|
||||
3. The application has been restarted after configuration changes
|
||||
|
||||
### PostgreSQL Connection Issues
|
||||
|
||||
- Ensure pgvector extension is installed: `CREATE EXTENSION vector;`
|
||||
- Verify JDBC URL format and network connectivity
|
||||
- Check database user permissions
|
||||
|
||||
### MongoDB Vector Search
|
||||
|
||||
- Vector search requires MongoDB Atlas or MongoDB 6.0+ with Atlas Search
|
||||
- Ensure vector search index is created on your collection
|
||||
- Local MongoDB containers don't support vector search
|
||||
|
||||
### Pinecone
|
||||
|
||||
- Verify API key is valid and has necessary permissions
|
||||
- Ensure index exists in your Pinecone account before using it
|
||||
|
||||
### SQLite (sqlite-vec)
|
||||
|
||||
- "no such function: load_extension" — extension loading is disabled; the backend enables it at the
|
||||
driver level, so this usually means the driver in use does not permit it
|
||||
- "no such module: vec0" — the `vec0` extension could not be loaded. The binary is bundled for common
|
||||
platforms; if your platform is not bundled (the log shows "No bundled sqlite-vec extension for ..."),
|
||||
install vec0 and set `extensionPath` to the compiled extension (Linux `.so`, macOS `.dylib`, Windows
|
||||
`.dll`), ensuring it is readable by the server process
|
||||
- "Embeddings must be of dimensions : N" — the instance's `dimensions` must equal the embedding model's
|
||||
output size; for the auto-registered `default` instance set `conductor.vectordb.sqlite-default.dimensions`
|
||||
or request that many dimensions from the embedding model
|
||||
- Local-only: each instance maps to a single SQLite file on the server host and is not shared across
|
||||
a cluster — use pgvector/Pinecone/MongoDB for distributed deployments
|
||||
+119
@@ -0,0 +1,119 @@
|
||||
import java.security.MessageDigest
|
||||
|
||||
plugins {
|
||||
id 'java'
|
||||
}
|
||||
|
||||
repositories {
|
||||
maven { url 'https://repo.spring.io/snapshot' }
|
||||
maven { url 'https://repo.spring.io/milestone' }
|
||||
}
|
||||
|
||||
// --- sqlite-vec loadable extensions -------------------------------------------------------------
|
||||
// sqlite-vec has no Maven artifact and no JVM binding, so we download the official, checksum-pinned
|
||||
// loadable binaries at build time and package them as jar resources under /sqlite-vec/<platform>/.
|
||||
// SqliteVecExtensions extracts the right one at runtime. Apache-2.0/MIT licensed, redistributable.
|
||||
ext.sqliteVecVersion = '0.1.9'
|
||||
ext.sqliteVecArtifacts = [
|
||||
'linux-x86_64' : 'b959baa1d8dc88861b1edb337b8587178cdcb12d60b4998f9d10b6a82052d5d7',
|
||||
'linux-aarch64' : 'ea03d39541e478fab5974253c461e1cb5d77742f69e40cf96e3fad5bc309a37c',
|
||||
'macos-aarch64' : '8282126333399ddfe98bbbcc7a1936e7252625aac49df056a98be602e46bfd29',
|
||||
'macos-x86_64' : '53ad76e400786515e2edcaed2f01271dda846316390b761fadbd2dcf56aa4713',
|
||||
'windows-x86_64': '51581189d52066b4dfc6631f6d7a3eab7dedc2260656ab09ca97ab3fb8165983',
|
||||
]
|
||||
def sqliteVecResourceDir = layout.buildDirectory.dir('generated/resources/sqlite-vec')
|
||||
|
||||
def downloadSqliteVec = tasks.register('downloadSqliteVec') {
|
||||
description = 'Downloads, verifies and stages the sqlite-vec loadable extensions as jar resources.'
|
||||
group = 'build'
|
||||
inputs.property('version', sqliteVecVersion)
|
||||
inputs.property('artifacts', sqliteVecArtifacts)
|
||||
outputs.dir(sqliteVecResourceDir)
|
||||
doLast {
|
||||
def base = sqliteVecResourceDir.get().asFile
|
||||
def cache = new File(temporaryDir, 'archives')
|
||||
cache.mkdirs()
|
||||
sqliteVecArtifacts.each { platform, expectedSha ->
|
||||
def archive = new File(cache, "sqlite-vec-${platform}.tar.gz")
|
||||
if (!archive.exists()) {
|
||||
def url = "https://github.com/asg017/sqlite-vec/releases/download/v${sqliteVecVersion}/sqlite-vec-${sqliteVecVersion}-loadable-${platform}.tar.gz"
|
||||
logger.lifecycle("Downloading sqlite-vec ${platform} from ${url}")
|
||||
ant.get(src: url, dest: archive, verbose: false)
|
||||
}
|
||||
def digest = MessageDigest.getInstance('SHA-256')
|
||||
archive.eachByte(64 * 1024) { buf, len -> digest.update(buf, 0, len) }
|
||||
def actualSha = digest.digest().encodeHex().toString()
|
||||
if (actualSha != expectedSha) {
|
||||
throw new GradleException("Checksum mismatch for sqlite-vec ${platform}: expected ${expectedSha}, got ${actualSha}")
|
||||
}
|
||||
def platformDir = new File(base, "sqlite-vec/${platform}")
|
||||
platformDir.mkdirs()
|
||||
copy {
|
||||
from tarTree(resources.gzip(archive))
|
||||
into platformDir
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Register the task provider (not the bare directory) as a resource source dir so that every
|
||||
// consumer of the resources — processResources, sourcesJar, etc. — automatically depends on
|
||||
// downloadSqliteVec instead of racing it.
|
||||
sourceSets.main.resources.srcDir(downloadSqliteVec)
|
||||
|
||||
dependencies {
|
||||
compileOnly 'org.springframework.boot:spring-boot-starter'
|
||||
// Needed to compile A2A push-notification callback controller and AgentSpan activation glue;
|
||||
// supplied by the server at runtime.
|
||||
compileOnly 'org.springframework.boot:spring-boot-starter-web'
|
||||
compileOnly 'org.springframework.boot:spring-boot-autoconfigure'
|
||||
|
||||
implementation project(':conductor-common')
|
||||
implementation project(':conductor-core')
|
||||
|
||||
api "org.springframework.ai:spring-ai-model:${revSpringAI}"
|
||||
api "org.springframework.ai:spring-ai-client-chat:${revSpringAI}"
|
||||
api "org.springframework.ai:spring-ai-mistral-ai:${revSpringAI}"
|
||||
api "org.springframework.ai:spring-ai-bedrock-converse:${revSpringAI}"
|
||||
api "org.springframework.ai:spring-ai-ollama:${revSpringAI}"
|
||||
|
||||
api "com.google.auth:google-auth-library-oauth2-http:1.33.0"
|
||||
api "com.networknt:json-schema-validator:${revJSonSchemaValidator}"
|
||||
|
||||
api("io.modelcontextprotocol.sdk:mcp-core:${revMCP}")
|
||||
api "com.squareup.okhttp3:okhttp:4.12.0"
|
||||
api "org.apache.commons:commons-lang3"
|
||||
|
||||
// Markdown parsing and PDF generation
|
||||
// Exclude openhtmltopdf-pdfbox (pulled in by flexmark-pdf-converter inside flexmark-all):
|
||||
// it was compiled against fontbox 2.x and calls TTFParser.parse(InputStream) which was
|
||||
// removed in fontbox 3.x, causing a NoSuchMethodError at runtime in GENERATE_PDF tasks.
|
||||
// The custom MarkdownToPdfConverter uses PDFBox 3.x directly and does not need openhtmltopdf.
|
||||
implementation("com.vladsch.flexmark:flexmark-all:${revFlexmark}") {
|
||||
exclude group: "com.openhtmltopdf", module: "openhtmltopdf-pdfbox"
|
||||
exclude group: "de.rototor.pdfbox", module: "graphics2d"
|
||||
}
|
||||
implementation "org.apache.pdfbox:pdfbox:3.0.5"
|
||||
|
||||
//Vector Databases
|
||||
|
||||
api "org.mongodb:mongodb-driver-sync:${mongodb}"
|
||||
api "org.mongodb:mongodb-driver-core:${mongodb}"
|
||||
api "org.mongodb:bson:${mongodb}"
|
||||
api "io.pinecone:pinecone-client:${pinecone}"
|
||||
api "com.pgvector:pgvector:${pgVector}"
|
||||
// SQLite + sqlite-vec backend. The native vec0 extension is supplied by the operator at
|
||||
// runtime (no JVM binding / Maven artifact exists); see SqliteConfig.
|
||||
api "org.xerial:sqlite-jdbc:${sqliteJdbc}"
|
||||
api "org.springframework.boot:spring-boot-starter-jdbc"
|
||||
|
||||
|
||||
testImplementation 'org.springframework.boot:spring-boot-starter-web'
|
||||
testImplementation "com.squareup.okhttp3:mockwebserver:4.12.0"
|
||||
testImplementation "org.testcontainers:mongodb:${revTestContainer}"
|
||||
testImplementation "org.testcontainers:postgresql:${revTestContainer}"
|
||||
testImplementation "org.postgresql:postgresql:${revPostgres}"
|
||||
testImplementation "org.apache.commons:commons-compress:${revCommonsCompress}"
|
||||
testImplementation "com.h2database:h2:2.4.240"
|
||||
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
{
|
||||
"name": "chat_workflow",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"tasks": [
|
||||
{
|
||||
"name": "chat_task",
|
||||
"taskReferenceName": "chat",
|
||||
"type": "LLM_CHAT_COMPLETE",
|
||||
"inputParameters": {
|
||||
"llmProvider": "openai",
|
||||
"model": "gpt-4o-mini",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"message": "You are a helpful assistant."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"message": "What is the capital of France?"
|
||||
}
|
||||
],
|
||||
"temperature": 0.7,
|
||||
"maxTokens": 500
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"name": "embedding_workflow",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"tasks": [
|
||||
{
|
||||
"name": "generate_embeddings",
|
||||
"taskReferenceName": "embeddings",
|
||||
"type": "LLM_GENERATE_EMBEDDINGS",
|
||||
"inputParameters": {
|
||||
"llmProvider": "openai",
|
||||
"model": "text-embedding-3-small",
|
||||
"text": "Conductor is an orchestration platform"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
{
|
||||
"name": "image_gen_workflow",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"tasks": [
|
||||
{
|
||||
"name": "generate_image",
|
||||
"taskReferenceName": "image",
|
||||
"type": "GENERATE_IMAGE",
|
||||
"inputParameters": {
|
||||
"llmProvider": "openai",
|
||||
"model": "dall-e-3",
|
||||
"prompt": "A futuristic cityscape at sunset",
|
||||
"width": 1024,
|
||||
"height": 1024,
|
||||
"n": 1,
|
||||
"style": "vivid"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
{
|
||||
"name": "tts_workflow",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"tasks": [
|
||||
{
|
||||
"name": "generate_audio",
|
||||
"taskReferenceName": "audio",
|
||||
"type": "GENERATE_AUDIO",
|
||||
"inputParameters": {
|
||||
"llmProvider": "openai",
|
||||
"model": "tts-1",
|
||||
"text": "Hello, this is a test of text to speech.",
|
||||
"voice": "alloy"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
{
|
||||
"name": "semantic_search_workflow",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"tasks": [
|
||||
{
|
||||
"name": "index_documents",
|
||||
"taskReferenceName": "index",
|
||||
"type": "LLM_INDEX_TEXT",
|
||||
"inputParameters": {
|
||||
"vectorDB": "postgres-prod",
|
||||
"namespace": "documentation",
|
||||
"index": "tech_docs",
|
||||
"embeddingModelProvider": "openai",
|
||||
"embeddingModel": "text-embedding-3-small",
|
||||
"text": "Conductor is a workflow orchestration platform",
|
||||
"docId": "doc_001"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "search_documents",
|
||||
"taskReferenceName": "search",
|
||||
"type": "LLM_SEARCH_INDEX",
|
||||
"inputParameters": {
|
||||
"vectorDB": "postgres-prod",
|
||||
"namespace": "documentation",
|
||||
"index": "tech_docs",
|
||||
"embeddingModelProvider": "openai",
|
||||
"embeddingModel": "text-embedding-3-small",
|
||||
"query": "workflow orchestration",
|
||||
"llmMaxResults": 5
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
{
|
||||
"name": "rag_workflow",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"tasks": [
|
||||
{
|
||||
"name": "search_knowledge_base",
|
||||
"taskReferenceName": "search",
|
||||
"type": "LLM_SEARCH_INDEX",
|
||||
"inputParameters": {
|
||||
"vectorDB": "postgres-prod",
|
||||
"namespace": "kb",
|
||||
"index": "articles",
|
||||
"embeddingModelProvider": "openai",
|
||||
"embeddingModel": "text-embedding-3-small",
|
||||
"query": "${workflow.input.question}",
|
||||
"llmMaxResults": 3
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "generate_answer",
|
||||
"taskReferenceName": "answer",
|
||||
"type": "LLM_CHAT_COMPLETE",
|
||||
"inputParameters": {
|
||||
"llmProvider": "anthropic",
|
||||
"model": "claude-3-5-sonnet-20241022",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"message": "Answer based on the following context: ${search.output.result}"
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"message": "${workflow.input.question}"
|
||||
}
|
||||
],
|
||||
"temperature": 0.3
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
{
|
||||
"name": "complete_rag_demo",
|
||||
"description": "Index documents, search, and generate RAG answer",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"tasks": [
|
||||
{
|
||||
"name": "index_doc_1",
|
||||
"taskReferenceName": "index_doc_1_ref",
|
||||
"type": "LLM_INDEX_TEXT",
|
||||
"inputParameters": {
|
||||
"vectorDB": "postgres-prod",
|
||||
"index": "demo_index",
|
||||
"namespace": "demo_docs",
|
||||
"docId": "intro-001",
|
||||
"text": "Conductor is a distributed workflow orchestration engine that runs in the cloud. It allows developers to build complex stateful applications by orchestrating microservices.",
|
||||
"embeddingModelProvider": "openai",
|
||||
"embeddingModel": "text-embedding-3-small",
|
||||
"dimensions": 1536,
|
||||
"metadata": { "category": "introduction" }
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "index_doc_2",
|
||||
"taskReferenceName": "index_doc_2_ref",
|
||||
"type": "LLM_INDEX_TEXT",
|
||||
"inputParameters": {
|
||||
"vectorDB": "postgres-prod",
|
||||
"index": "demo_index",
|
||||
"namespace": "demo_docs",
|
||||
"docId": "features-002",
|
||||
"text": "Conductor supports multiple vector databases including PostgreSQL (pgvector), MongoDB Atlas, and Pinecone. It also integrates with LLM providers like OpenAI, Anthropic, and Azure OpenAI.",
|
||||
"embeddingModelProvider": "openai",
|
||||
"embeddingModel": "text-embedding-3-small",
|
||||
"dimensions": 1536,
|
||||
"metadata": { "category": "features" }
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "index_doc_3",
|
||||
"taskReferenceName": "index_doc_3_ref",
|
||||
"type": "LLM_INDEX_TEXT",
|
||||
"inputParameters": {
|
||||
"vectorDB": "postgres-prod",
|
||||
"index": "demo_index",
|
||||
"namespace": "demo_docs",
|
||||
"docId": "config-003",
|
||||
"text": "You can configure multiple named instances of the same vector database type for different environments like production, development, and staging.",
|
||||
"embeddingModelProvider": "openai",
|
||||
"embeddingModel": "text-embedding-3-small",
|
||||
"dimensions": 1536,
|
||||
"metadata": { "category": "configuration" }
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "search_index",
|
||||
"taskReferenceName": "search_ref",
|
||||
"type": "LLM_SEARCH_INDEX",
|
||||
"inputParameters": {
|
||||
"vectorDB": "postgres-prod",
|
||||
"index": "demo_index",
|
||||
"namespace": "demo_docs",
|
||||
"query": "What vector databases does Conductor support?",
|
||||
"embeddingModelProvider": "openai",
|
||||
"embeddingModel": "text-embedding-3-small",
|
||||
"dimensions": 1536,
|
||||
"maxResults": 3
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "generate_rag_answer",
|
||||
"taskReferenceName": "answer_ref",
|
||||
"type": "LLM_CHAT_COMPLETE",
|
||||
"inputParameters": {
|
||||
"llmProvider": "openai",
|
||||
"model": "gpt-4o-mini",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"message": "You are a technical expert. Answer the question using only the provided context."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"message": "Context:\n${search_ref.output.result}\n\nQuestion: What vector databases does Conductor support?"
|
||||
}
|
||||
],
|
||||
"temperature": 0.2
|
||||
}
|
||||
}
|
||||
],
|
||||
"outputParameters": {
|
||||
"indexed_docs": ["${index_doc_1_ref.output}", "${index_doc_2_ref.output}", "${index_doc_3_ref.output}"],
|
||||
"search_results": "${search_ref.output.result}",
|
||||
"answer": "${answer_ref.output.result}"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"name": "mcp_list_tools_workflow",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"tasks": [
|
||||
{
|
||||
"name": "list_mcp_tools",
|
||||
"taskReferenceName": "list_tools",
|
||||
"type": "LIST_MCP_TOOLS",
|
||||
"inputParameters": {
|
||||
"mcpServer": "http://localhost:3001/mcp"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
{
|
||||
"name": "mcp_weather_workflow",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"tasks": [
|
||||
{
|
||||
"name": "get_weather",
|
||||
"taskReferenceName": "weather",
|
||||
"type": "CALL_MCP_TOOL",
|
||||
"inputParameters": {
|
||||
"mcpServer": "http://localhost:3001/mcp",
|
||||
"method": "get_weather",
|
||||
"location": "New York",
|
||||
"units": "fahrenheit"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
{
|
||||
"name": "a2a_agent",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"tasks": [
|
||||
{
|
||||
"name": "call_currency_agent",
|
||||
"taskReferenceName": "agent",
|
||||
"type": "AGENT",
|
||||
"inputParameters": {
|
||||
"agentType": "a2a",
|
||||
"agentUrl": "http://localhost:9999",
|
||||
"text": "convert 100 USD to EUR",
|
||||
"pollIntervalSeconds": 5,
|
||||
"headers": {
|
||||
"Authorization": "Bearer ${workflow.input.agentToken}"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
{
|
||||
"name": "mcp_ai_agent_workflow",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"tasks": [
|
||||
{
|
||||
"name": "list_available_tools",
|
||||
"taskReferenceName": "discover_tools",
|
||||
"type": "LIST_MCP_TOOLS",
|
||||
"inputParameters": {
|
||||
"mcpServer": "http://localhost:3001/mcp"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "decide_which_tools_to_use",
|
||||
"taskReferenceName": "plan",
|
||||
"type": "LLM_CHAT_COMPLETE",
|
||||
"inputParameters": {
|
||||
"llmProvider": "anthropic",
|
||||
"model": "claude-3-5-sonnet-20241022",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"message": "You are an AI agent. Available tools: ${discover_tools.output.tools}. User wants to: ${workflow.input.task}"
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"message": "Which tool should I use and what parameters? Respond with JSON: {method: string, arguments: object}"
|
||||
}
|
||||
],
|
||||
"temperature": 0.1,
|
||||
"maxTokens": 500
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "execute_tool",
|
||||
"taskReferenceName": "execute",
|
||||
"type": "CALL_MCP_TOOL",
|
||||
"inputParameters": {
|
||||
"mcpServer": "http://localhost:3001/mcp",
|
||||
"method": "${plan.output.result.method}",
|
||||
"arguments": "${plan.output.result.arguments}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "summarize_result",
|
||||
"taskReferenceName": "summarize",
|
||||
"type": "LLM_CHAT_COMPLETE",
|
||||
"inputParameters": {
|
||||
"llmProvider": "openai",
|
||||
"model": "gpt-4o-mini",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"message": "Summarize this result for the user: ${execute.output.content}"
|
||||
}
|
||||
],
|
||||
"maxTokens": 200
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"name": "a2a_get_agent_card_workflow",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"tasks": [
|
||||
{
|
||||
"name": "discover_agent",
|
||||
"taskReferenceName": "discover",
|
||||
"type": "GET_AGENT_CARD",
|
||||
"inputParameters": {
|
||||
"agentUrl": "http://localhost:9999"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"name": "video_gen_openai_sora",
|
||||
"description": "Generates a video using OpenAI Sora. The task is async -- it submits a job and polls until the video is ready.",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"tasks": [
|
||||
{
|
||||
"name": "generate_video",
|
||||
"taskReferenceName": "sora_video",
|
||||
"type": "GENERATE_VIDEO",
|
||||
"inputParameters": {
|
||||
"llmProvider": "openai",
|
||||
"model": "sora-2",
|
||||
"prompt": "A slow cinematic aerial shot of a coastal city at golden hour, waves crashing against cliffs",
|
||||
"duration": 8,
|
||||
"size": "1280x720",
|
||||
"n": 1,
|
||||
"style": "cinematic"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"name": "order_pizza",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"description": "Takes a pizza order — exposed as an A2A agent via metadata.a2a.enabled",
|
||||
"ownerEmail": "a2a@example.com",
|
||||
"metadata": {
|
||||
"a2a.enabled": true,
|
||||
"a2a.tags": ["ordering", "demo"]
|
||||
},
|
||||
"tasks": [
|
||||
{
|
||||
"name": "confirm_order",
|
||||
"taskReferenceName": "confirm",
|
||||
"type": "INLINE",
|
||||
"inputParameters": {
|
||||
"evaluatorType": "graaljs",
|
||||
"expression": "({ orderId: 'ORD-' + ($.text || 'unknown') })",
|
||||
"text": "${workflow.input._a2a_text}"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
{
|
||||
"name": "video_gen_gemini_veo",
|
||||
"description": "Generates a video using Google Gemini Veo. Supports negative prompts, person generation controls, and optional audio generation.",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"tasks": [
|
||||
{
|
||||
"name": "generate_video",
|
||||
"taskReferenceName": "veo_video",
|
||||
"type": "GENERATE_VIDEO",
|
||||
"inputParameters": {
|
||||
"llmProvider": "vertex_ai",
|
||||
"model": "veo-3",
|
||||
"prompt": "A time-lapse of a blooming flower in a sunlit garden, soft bokeh background",
|
||||
"duration": 8,
|
||||
"aspectRatio": "16:9",
|
||||
"resolution": "720p",
|
||||
"personGeneration": "dont_allow",
|
||||
"generateAudio": true,
|
||||
"negativePrompt": "blurry, low quality, text overlay",
|
||||
"n": 1
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
{
|
||||
"name": "image_to_video_pipeline",
|
||||
"description": "A two-step creative pipeline: generates a still image with DALL-E, then creates a video continuation using OpenAI Sora with a prompt inspired by the scene.",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"tasks": [
|
||||
{
|
||||
"name": "generate_image",
|
||||
"taskReferenceName": "source_image",
|
||||
"type": "GENERATE_IMAGE",
|
||||
"inputParameters": {
|
||||
"llmProvider": "openai",
|
||||
"model": "dall-e-3",
|
||||
"prompt": "A serene mountain lake at dawn with mist rising from the water, photorealistic, wide landscape",
|
||||
"width": 1792,
|
||||
"height": 1024,
|
||||
"n": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "generate_video",
|
||||
"taskReferenceName": "animated_video",
|
||||
"type": "GENERATE_VIDEO",
|
||||
"inputParameters": {
|
||||
"llmProvider": "openai",
|
||||
"model": "sora-2",
|
||||
"prompt": "A serene mountain lake at dawn, gentle ripples spread across the water as mist slowly drifts, birds flying in the distance, cinematic slow motion",
|
||||
"duration": 8,
|
||||
"size": "1280x720",
|
||||
"style": "cinematic"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"name": "image_gen_stabilityai",
|
||||
"description": "Generates an image using Stability AI's SD3.5 Large model via the v2beta API.",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"tasks": [
|
||||
{
|
||||
"name": "generate_image",
|
||||
"taskReferenceName": "stable_diffusion_image",
|
||||
"type": "GENERATE_IMAGE",
|
||||
"inputParameters": {
|
||||
"llmProvider": "stabilityai",
|
||||
"model": "sd3.5-large",
|
||||
"prompt": "A fantasy castle perched on a floating island surrounded by clouds, digital art, highly detailed",
|
||||
"width": 1024,
|
||||
"height": 1024,
|
||||
"n": 1,
|
||||
"style": "cinematic"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"name": "pdf_generation_workflow",
|
||||
"description": "Generate a PDF document from markdown content with custom layout options",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"tasks": [
|
||||
{
|
||||
"name": "generate_pdf",
|
||||
"taskReferenceName": "pdf",
|
||||
"type": "GENERATE_PDF",
|
||||
"inputParameters": {
|
||||
"markdown": "# Monthly Sales Report\n\n## Executive Summary\n\nThis report covers sales performance for **Q4 2025**.\n\n| Region | Revenue | Growth |\n|--------|---------|--------|\n| North America | $2.4M | +12% |\n| Europe | $1.8M | +8% |\n| Asia Pacific | $1.2M | +15% |\n\n## Key Highlights\n\n- Total revenue reached **$5.4M**, exceeding target by 10%\n- Customer acquisition increased by *23%* across all regions\n- Product satisfaction score: **4.7/5.0**\n\n## Action Items\n\n1. Expand APAC sales team by Q1 2026\n2. Launch enterprise tier in European market\n3. Increase marketing budget for North America\n\n> *\"Our best quarter yet -- the team delivered exceptional results across every metric.\"* -- VP of Sales\n\n---\n\nGenerated by Conductor Workflow Engine",
|
||||
"pageSize": "LETTER",
|
||||
"theme": "default",
|
||||
"baseFontSize": 11,
|
||||
"pdfMetadata": {
|
||||
"title": "Monthly Sales Report - Q4 2025",
|
||||
"author": "Conductor Workflow",
|
||||
"subject": "Quarterly Sales Performance"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
{
|
||||
"name": "llm_to_pdf_pipeline",
|
||||
"description": "End-to-end pipeline: LLM generates a markdown report from user input, then converts it to a PDF document",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"inputParameters": ["topic", "audience"],
|
||||
"tasks": [
|
||||
{
|
||||
"name": "generate_report_markdown",
|
||||
"taskReferenceName": "llm_report",
|
||||
"type": "LLM_CHAT_COMPLETE",
|
||||
"inputParameters": {
|
||||
"llmProvider": "openai",
|
||||
"model": "gpt-4o-mini",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"message": "You are a professional report writer. Generate well-structured markdown reports with headings, tables, bullet points, and bold/italic emphasis. Always include an executive summary, key findings, and recommendations sections."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"message": "Write a detailed report about: ${workflow.input.topic}\n\nTarget audience: ${workflow.input.audience}\n\nUse markdown formatting with:\n- A clear title (# heading)\n- Executive summary section\n- Key findings with a data table\n- Bullet-point recommendations\n- A blockquote conclusion"
|
||||
}
|
||||
],
|
||||
"temperature": 0.7,
|
||||
"maxTokens": 2000
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "convert_to_pdf",
|
||||
"taskReferenceName": "pdf_output",
|
||||
"type": "GENERATE_PDF",
|
||||
"inputParameters": {
|
||||
"markdown": "${llm_report.output.result}",
|
||||
"pageSize": "A4",
|
||||
"theme": "default",
|
||||
"baseFontSize": 11,
|
||||
"pdfMetadata": {
|
||||
"title": "${workflow.input.topic}",
|
||||
"author": "Conductor AI Pipeline",
|
||||
"subject": "Auto-generated report"
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"outputParameters": {
|
||||
"reportMarkdown": "${llm_report.output.result}",
|
||||
"pdfLocation": "${pdf_output.output.result.location}",
|
||||
"pdfSizeBytes": "${pdf_output.output.result.sizeBytes}"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
{
|
||||
"name": "web_search_workflow",
|
||||
"description": "Chat completion with built-in web search — the LLM can search the web for real-time information",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"inputParameters": ["question"],
|
||||
"tasks": [
|
||||
{
|
||||
"name": "web_search_chat",
|
||||
"taskReferenceName": "chat",
|
||||
"type": "LLM_CHAT_COMPLETE",
|
||||
"inputParameters": {
|
||||
"llmProvider": "openai",
|
||||
"model": "gpt-4o-mini",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"message": "You are a helpful research assistant. Use web search to find current information."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"message": "${workflow.input.question}"
|
||||
}
|
||||
],
|
||||
"webSearch": true,
|
||||
"temperature": 0.3,
|
||||
"maxTokens": 1000
|
||||
}
|
||||
}
|
||||
],
|
||||
"outputParameters": {
|
||||
"answer": "${chat.output.result}"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
{
|
||||
"name": "code_execution_workflow",
|
||||
"description": "Chat completion with built-in code execution — the LLM can write and run code in a sandbox",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"inputParameters": ["task"],
|
||||
"tasks": [
|
||||
{
|
||||
"name": "code_execution_chat",
|
||||
"taskReferenceName": "chat",
|
||||
"type": "LLM_CHAT_COMPLETE",
|
||||
"inputParameters": {
|
||||
"llmProvider": "google_gemini",
|
||||
"model": "gemini-2.5-flash",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"message": "You are a data analyst. Use code execution to compute results, generate charts, and analyze data."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"message": "${workflow.input.task}"
|
||||
}
|
||||
],
|
||||
"codeInterpreter": true,
|
||||
"temperature": 0.2,
|
||||
"maxTokens": 2000
|
||||
}
|
||||
}
|
||||
],
|
||||
"outputParameters": {
|
||||
"result": "${chat.output.result}"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,78 @@
|
||||
{
|
||||
"name": "coding_agent",
|
||||
"description": "A coding agent that uses OpenAI code_interpreter to write, run, and iterate on code",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"inputParameters": ["task"],
|
||||
"tasks": [
|
||||
{
|
||||
"name": "plan_implementation",
|
||||
"taskReferenceName": "plan",
|
||||
"type": "LLM_CHAT_COMPLETE",
|
||||
"inputParameters": {
|
||||
"llmProvider": "openai",
|
||||
"model": "gpt-4o",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"message": "You are a senior software engineer. Break down the coding task into clear steps. Output a numbered plan."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"message": "${workflow.input.task}"
|
||||
}
|
||||
],
|
||||
"temperature": 0.2,
|
||||
"maxTokens": 1000
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "write_and_run_code",
|
||||
"taskReferenceName": "code",
|
||||
"type": "LLM_CHAT_COMPLETE",
|
||||
"inputParameters": {
|
||||
"llmProvider": "openai",
|
||||
"model": "gpt-4o",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"message": "You are a coding assistant with access to a code execution sandbox. Write the code, run it, verify the output, and fix any errors. Return the final working code and its output."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"message": "Implementation plan:\n${plan.output.result}\n\nOriginal task: ${workflow.input.task}\n\nWrite the code, execute it, and return the working result."
|
||||
}
|
||||
],
|
||||
"codeInterpreter": true,
|
||||
"temperature": 0.1,
|
||||
"maxTokens": 4000
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "review_code",
|
||||
"taskReferenceName": "review",
|
||||
"type": "LLM_CHAT_COMPLETE",
|
||||
"inputParameters": {
|
||||
"llmProvider": "openai",
|
||||
"model": "gpt-4o-mini",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"message": "You are a code reviewer. Review the implementation for correctness, edge cases, and code quality. Provide a brief summary of what was built and any suggestions."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"message": "Task: ${workflow.input.task}\n\nImplementation:\n${code.output.result}"
|
||||
}
|
||||
],
|
||||
"temperature": 0.3,
|
||||
"maxTokens": 1000
|
||||
}
|
||||
}
|
||||
],
|
||||
"outputParameters": {
|
||||
"plan": "${plan.output.result}",
|
||||
"code": "${code.output.result}",
|
||||
"review": "${review.output.result}"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
{
|
||||
"name": "extended_thinking_workflow",
|
||||
"description": "Chat completion with extended thinking — gives the LLM a token budget for step-by-step reasoning",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"inputParameters": ["problem"],
|
||||
"tasks": [
|
||||
{
|
||||
"name": "think_deeply",
|
||||
"taskReferenceName": "think",
|
||||
"type": "LLM_CHAT_COMPLETE",
|
||||
"inputParameters": {
|
||||
"llmProvider": "anthropic",
|
||||
"model": "claude-sonnet-4-20250514",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"message": "${workflow.input.problem}"
|
||||
}
|
||||
],
|
||||
"thinkingTokenLimit": 10000,
|
||||
"maxTokens": 16000
|
||||
}
|
||||
}
|
||||
],
|
||||
"outputParameters": {
|
||||
"answer": "${think.output.result}"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
{
|
||||
"name": "web_research_agent",
|
||||
"description": "An autonomous research agent that uses web search to gather information and synthesize a report",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"inputParameters": ["topic"],
|
||||
"tasks": [
|
||||
{
|
||||
"name": "gather_information",
|
||||
"taskReferenceName": "research",
|
||||
"type": "LLM_CHAT_COMPLETE",
|
||||
"inputParameters": {
|
||||
"llmProvider": "openai",
|
||||
"model": "gpt-4o",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"message": "You are a research analyst. Use web search to find comprehensive, current information about the topic. Search for multiple perspectives and recent developments."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"message": "Research this topic thoroughly: ${workflow.input.topic}"
|
||||
}
|
||||
],
|
||||
"webSearch": true,
|
||||
"temperature": 0.3,
|
||||
"maxTokens": 3000
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "synthesize_report",
|
||||
"taskReferenceName": "report",
|
||||
"type": "LLM_CHAT_COMPLETE",
|
||||
"inputParameters": {
|
||||
"llmProvider": "anthropic",
|
||||
"model": "claude-sonnet-4-20250514",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"message": "You are a technical writer. Synthesize the research into a well-structured markdown report with sections, key findings, and citations."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"message": "Topic: ${workflow.input.topic}\n\nResearch findings:\n${research.output.result}\n\nWrite a comprehensive report in markdown format."
|
||||
}
|
||||
],
|
||||
"thinkingTokenLimit": 5000,
|
||||
"maxTokens": 8000
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "convert_to_pdf",
|
||||
"taskReferenceName": "pdf",
|
||||
"type": "GENERATE_PDF",
|
||||
"inputParameters": {
|
||||
"markdown": "${report.output.result}",
|
||||
"pageSize": "A4",
|
||||
"theme": "default",
|
||||
"pdfMetadata": {
|
||||
"title": "${workflow.input.topic}",
|
||||
"author": "Conductor Research Agent"
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"outputParameters": {
|
||||
"report": "${report.output.result}",
|
||||
"pdf": "${pdf.output.result.location}"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
{
|
||||
"name": "multi_turn_chain",
|
||||
"description": "Two-step conversation using previousResponseId to avoid resending history",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"inputParameters": ["topic"],
|
||||
"tasks": [
|
||||
{
|
||||
"name": "first_turn",
|
||||
"taskReferenceName": "turn1",
|
||||
"type": "LLM_CHAT_COMPLETE",
|
||||
"inputParameters": {
|
||||
"llmProvider": "openai",
|
||||
"model": "gpt-4o",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"message": "You are a technical architect. Be concise."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"message": "Design a high-level architecture for: ${workflow.input.topic}"
|
||||
}
|
||||
],
|
||||
"temperature": 0.3,
|
||||
"maxTokens": 2000
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "follow_up",
|
||||
"taskReferenceName": "turn2",
|
||||
"type": "LLM_CHAT_COMPLETE",
|
||||
"inputParameters": {
|
||||
"llmProvider": "openai",
|
||||
"model": "gpt-4o",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"message": "Now list the key risks and mitigations for this architecture."
|
||||
}
|
||||
],
|
||||
"previousResponseId": "${turn1.output.responseId}",
|
||||
"temperature": 0.3,
|
||||
"maxTokens": 2000
|
||||
}
|
||||
}
|
||||
],
|
||||
"outputParameters": {
|
||||
"architecture": "${turn1.output.result}",
|
||||
"risks": "${turn2.output.result}"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"name": "a2a_agent_streaming",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"description": "Calls a remote agent in streaming mode — consumes the agent's message/stream (SSE) and aggregates events to completion. Requires the agent card to advertise capabilities.streaming=true.",
|
||||
"ownerEmail": "a2a@example.com",
|
||||
"tasks": [
|
||||
{
|
||||
"name": "stream_from_agent",
|
||||
"taskReferenceName": "agent",
|
||||
"type": "AGENT",
|
||||
"inputParameters": {
|
||||
"agentUrl": "http://localhost:9999",
|
||||
"text": "summarize the attached document",
|
||||
"streaming": true
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
{
|
||||
"name": "a2a_agent_push",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"description": "Calls a remote agent in push mode — the agent calls Conductor's webhook when the task reaches a terminal state, so no worker thread polls. Requires conductor.a2a.callback.url to be set (otherwise it falls back to polling). A slow backstop poll still runs so a lost webhook can't hang the task.",
|
||||
"ownerEmail": "a2a@example.com",
|
||||
"tasks": [
|
||||
{
|
||||
"name": "call_research_agent",
|
||||
"taskReferenceName": "agent",
|
||||
"type": "AGENT",
|
||||
"inputParameters": {
|
||||
"agentUrl": "http://localhost:9999",
|
||||
"text": "research the latest on durable agent protocols",
|
||||
"pushNotification": true,
|
||||
"pushBackstopPollSeconds": 300,
|
||||
"maxDurationSeconds": 3600
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
{
|
||||
"name": "book_appointment",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"description": "Multi-turn A2A agent: asks a clarifying question (HUMAN task → A2A 'input-required'), then confirms. Exposed via metadata.a2a.enabled. A client resumes it by sending another message/send carrying the returned task id; that follow-up completes the HUMAN task and the workflow continues.",
|
||||
"ownerEmail": "a2a@example.com",
|
||||
"metadata": {
|
||||
"a2a.enabled": true,
|
||||
"a2a.tags": ["scheduling", "multi-turn"]
|
||||
},
|
||||
"tasks": [
|
||||
{
|
||||
"name": "ask_preferred_time",
|
||||
"taskReferenceName": "ask",
|
||||
"type": "HUMAN"
|
||||
},
|
||||
{
|
||||
"name": "confirm_appointment",
|
||||
"taskReferenceName": "confirm",
|
||||
"type": "INLINE",
|
||||
"inputParameters": {
|
||||
"evaluatorType": "graaljs",
|
||||
"expression": "({ status: 'confirmed', when: $.when })",
|
||||
"when": "${ask.output._a2a_text}"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
{
|
||||
"name": "a2a_cancel_agent",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"description": "Starts work on a remote agent (AGENT) then cancels it (CANCEL_AGENT → A2A tasks/cancel), passing the remote task id from the first task's output. Illustrative — cancel is typically wired into fork/timeout or compensation flows.",
|
||||
"ownerEmail": "a2a@example.com",
|
||||
"tasks": [
|
||||
{
|
||||
"name": "start_agent_task",
|
||||
"taskReferenceName": "start",
|
||||
"type": "AGENT",
|
||||
"inputParameters": {
|
||||
"agentUrl": "http://localhost:9999",
|
||||
"text": "begin a long-running job",
|
||||
"maxDurationSeconds": 60
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "cancel_agent_task",
|
||||
"taskReferenceName": "cancel",
|
||||
"type": "CANCEL_AGENT",
|
||||
"inputParameters": {
|
||||
"agentUrl": "http://localhost:9999",
|
||||
"taskId": "${start.output.taskId}"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
{
|
||||
"name": "a2a_multi_agent_orchestration",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"description": "Durable multi-agent orchestration: call several remote A2A agents in parallel with FORK_JOIN, then JOIN their results. Each branch is an independent, crash-safe AGENT — if Conductor restarts mid-flight, every in-flight agent call resumes.",
|
||||
"ownerEmail": "a2a@example.com",
|
||||
"tasks": [
|
||||
{
|
||||
"name": "fork_agents",
|
||||
"taskReferenceName": "fork",
|
||||
"type": "FORK_JOIN",
|
||||
"forkTasks": [
|
||||
[
|
||||
{
|
||||
"name": "call_flights_agent",
|
||||
"taskReferenceName": "flights",
|
||||
"type": "AGENT",
|
||||
"inputParameters": {
|
||||
"agentUrl": "${workflow.input.flightsAgentUrl}",
|
||||
"text": "Find flights for: ${workflow.input.request}"
|
||||
}
|
||||
}
|
||||
],
|
||||
[
|
||||
{
|
||||
"name": "call_hotels_agent",
|
||||
"taskReferenceName": "hotels",
|
||||
"type": "AGENT",
|
||||
"inputParameters": {
|
||||
"agentUrl": "${workflow.input.hotelsAgentUrl}",
|
||||
"text": "Find hotels for: ${workflow.input.request}"
|
||||
}
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "join_agents",
|
||||
"taskReferenceName": "join",
|
||||
"type": "JOIN",
|
||||
"joinOn": ["flights", "hotels"]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
{
|
||||
"name": "a2a_llm_pick_skill",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"description": "Agentic A2A: discover a remote agent's Agent Card (GET_AGENT_CARD), let an LLM choose the best prompt from the advertised skills (LLM_CHAT_COMPLETE), then call the agent (AGENT). Discovery + reasoning + invocation, all durable.",
|
||||
"ownerEmail": "a2a@example.com",
|
||||
"tasks": [
|
||||
{
|
||||
"name": "discover_agent",
|
||||
"taskReferenceName": "discover",
|
||||
"type": "GET_AGENT_CARD",
|
||||
"inputParameters": { "agentUrl": "${workflow.input.agentUrl}" }
|
||||
},
|
||||
{
|
||||
"name": "choose_request",
|
||||
"taskReferenceName": "choose",
|
||||
"type": "LLM_CHAT_COMPLETE",
|
||||
"inputParameters": {
|
||||
"llmProvider": "${workflow.input.llmProvider}",
|
||||
"model": "${workflow.input.model}",
|
||||
"instructions": "You are given an A2A agent's Agent Card (its skills) and a user goal. Reply with ONLY the single best prompt to send that agent.",
|
||||
"userInput": "Agent card: ${discover.output.agentCard}\nUser goal: ${workflow.input.goal}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "call_agent",
|
||||
"taskReferenceName": "call",
|
||||
"type": "AGENT",
|
||||
"inputParameters": {
|
||||
"agentUrl": "${workflow.input.agentUrl}",
|
||||
"text": "${choose.output.result}"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
{
|
||||
"name": "a2a_client_multi_turn",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"description": "Client-side multi-turn conversation: call a remote agent; if it returns input-required, branch (SWITCH on the agent's state) and call again with the SAME contextId/taskId carrying the answer — resuming the same remote task.",
|
||||
"ownerEmail": "a2a@example.com",
|
||||
"tasks": [
|
||||
{
|
||||
"name": "ask_agent",
|
||||
"taskReferenceName": "ask",
|
||||
"type": "AGENT",
|
||||
"inputParameters": {
|
||||
"agentUrl": "${workflow.input.agentUrl}",
|
||||
"text": "${workflow.input.prompt}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "branch_on_state",
|
||||
"taskReferenceName": "branch",
|
||||
"type": "SWITCH",
|
||||
"evaluatorType": "value-param",
|
||||
"expression": "state",
|
||||
"inputParameters": { "state": "${ask.output.state}" },
|
||||
"decisionCases": {
|
||||
"input-required": [
|
||||
{
|
||||
"name": "answer_agent",
|
||||
"taskReferenceName": "answer",
|
||||
"type": "AGENT",
|
||||
"inputParameters": {
|
||||
"agentUrl": "${workflow.input.agentUrl}",
|
||||
"text": "${workflow.input.answer}",
|
||||
"contextId": "${ask.output.contextId}",
|
||||
"taskId": "${ask.output.taskId}"
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"defaultCase": []
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
{
|
||||
"name": "rag_sqlite_vec_demo",
|
||||
"description": "Zero-infrastructure RAG using the bundled SQLite + sqlite-vec vector store. Requires conductor.db.type=sqlite and conductor.integrations.ai.enabled=true, which auto-registers the 'default' vector DB instance. Embeddings are requested at 256 dimensions to match the default instance.",
|
||||
"version": 1,
|
||||
"schemaVersion": 2,
|
||||
"tasks": [
|
||||
{
|
||||
"name": "index_doc_1",
|
||||
"taskReferenceName": "index_doc_1_ref",
|
||||
"type": "LLM_INDEX_TEXT",
|
||||
"inputParameters": {
|
||||
"vectorDB": "default",
|
||||
"index": "demo_index",
|
||||
"namespace": "demo_docs",
|
||||
"docId": "intro-001",
|
||||
"text": "Conductor is a distributed workflow orchestration engine. It lets developers build complex stateful applications by orchestrating microservices and AI agents.",
|
||||
"embeddingModelProvider": "openai",
|
||||
"embeddingModel": "text-embedding-3-small",
|
||||
"dimensions": 256,
|
||||
"metadata": { "category": "introduction" }
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "index_doc_2",
|
||||
"taskReferenceName": "index_doc_2_ref",
|
||||
"type": "LLM_INDEX_TEXT",
|
||||
"inputParameters": {
|
||||
"vectorDB": "default",
|
||||
"index": "demo_index",
|
||||
"namespace": "demo_docs",
|
||||
"docId": "vectordb-002",
|
||||
"text": "Conductor supports several vector databases: PostgreSQL (pgvector), MongoDB Atlas, Pinecone, and an embedded SQLite backend powered by the sqlite-vec extension that needs no external server.",
|
||||
"embeddingModelProvider": "openai",
|
||||
"embeddingModel": "text-embedding-3-small",
|
||||
"dimensions": 256,
|
||||
"metadata": { "category": "features" }
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "index_doc_3",
|
||||
"taskReferenceName": "index_doc_3_ref",
|
||||
"type": "LLM_INDEX_TEXT",
|
||||
"inputParameters": {
|
||||
"vectorDB": "default",
|
||||
"index": "demo_index",
|
||||
"namespace": "demo_docs",
|
||||
"docId": "sqlite-003",
|
||||
"text": "When SQLite persistence and the AI integration are both enabled, Conductor bundles the sqlite-vec native extension and registers a default vector store automatically, so semantic search works out of the box.",
|
||||
"embeddingModelProvider": "openai",
|
||||
"embeddingModel": "text-embedding-3-small",
|
||||
"dimensions": 256,
|
||||
"metadata": { "category": "configuration" }
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "search_index",
|
||||
"taskReferenceName": "search_ref",
|
||||
"type": "LLM_SEARCH_INDEX",
|
||||
"inputParameters": {
|
||||
"vectorDB": "default",
|
||||
"index": "demo_index",
|
||||
"namespace": "demo_docs",
|
||||
"query": "${workflow.input.question}",
|
||||
"embeddingModelProvider": "openai",
|
||||
"embeddingModel": "text-embedding-3-small",
|
||||
"dimensions": 256,
|
||||
"maxResults": 3
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "generate_rag_answer",
|
||||
"taskReferenceName": "answer_ref",
|
||||
"type": "LLM_CHAT_COMPLETE",
|
||||
"inputParameters": {
|
||||
"llmProvider": "openai",
|
||||
"model": "gpt-4o-mini",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"message": "You are a technical expert. Answer the question using only the provided context."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"message": "Context:\n${search_ref.output.result}\n\nQuestion: ${workflow.input.question}"
|
||||
}
|
||||
],
|
||||
"temperature": 0.2
|
||||
}
|
||||
}
|
||||
],
|
||||
"inputParameters": ["question"],
|
||||
"outputParameters": {
|
||||
"search_results": "${search_ref.output.result}",
|
||||
"answer": "${answer_ref.output.result}"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,545 @@
|
||||
# Conductor AI Workflow Examples
|
||||
|
||||
This folder contains ready-to-use workflow examples demonstrating the AI capabilities of Conductor.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### 1. Start Conductor Server
|
||||
|
||||
Ensure Conductor is running with AI integrations enabled:
|
||||
|
||||
```bash
|
||||
# From the conductor root directory
|
||||
./gradlew bootRun
|
||||
```
|
||||
|
||||
### 2. Configure AI Providers
|
||||
|
||||
Set environment variables before starting the server:
|
||||
|
||||
```bash
|
||||
# OpenAI (required for most examples)
|
||||
export OPENAI_API_KEY=sk-your-openai-api-key
|
||||
|
||||
# Anthropic (optional, for RAG examples)
|
||||
export ANTHROPIC_API_KEY=sk-ant-your-anthropic-key
|
||||
|
||||
# Google Gemini (optional, for Gemini/Veo examples)
|
||||
# Option 1: API key (simplest)
|
||||
export GEMINI_API_KEY=your-gemini-api-key
|
||||
# Option 2: Vertex AI — set project and location in application.properties
|
||||
```
|
||||
|
||||
For vector database examples, add to `application.properties`:
|
||||
|
||||
```properties
|
||||
# PostgreSQL Vector DB (for RAG/embedding examples)
|
||||
conductor.vectordb.instances[0].name=postgres-prod
|
||||
conductor.vectordb.instances[0].type=postgres
|
||||
conductor.vectordb.instances[0].postgres.datasourceURL=jdbc:postgresql://localhost:5432/vectors
|
||||
conductor.vectordb.instances[0].postgres.user=conductor
|
||||
conductor.vectordb.instances[0].postgres.password=secret
|
||||
conductor.vectordb.instances[0].postgres.dimensions=1536
|
||||
```
|
||||
|
||||
### 3. MCP Test Server (for MCP examples)
|
||||
|
||||
Install and start the MCP test server:
|
||||
|
||||
```bash
|
||||
# Install mcp-testkit — a test MCP server with 65 deterministic tools
|
||||
pip install mcp-testkit
|
||||
|
||||
# Start the server in HTTP mode
|
||||
mcp-testkit --transport http
|
||||
```
|
||||
|
||||
The server will be available at `http://localhost:3001/mcp`.
|
||||
|
||||
---
|
||||
|
||||
## Available Examples
|
||||
|
||||
| File | Description | Requirements |
|
||||
|------|-------------|--------------|
|
||||
| `01-chat-completion.json` | Basic chat with GPT-4o-mini | OpenAI |
|
||||
| `02-generate-embeddings.json` | Generate text embeddings | OpenAI |
|
||||
| `03-image-generation.json` | Generate images with DALL-E 3 | OpenAI |
|
||||
| `04-audio-generation.json` | Text-to-speech with OpenAI TTS | OpenAI |
|
||||
| `05-semantic-search.json` | Index and search documents | OpenAI, PostgreSQL |
|
||||
| `06-rag-basic.json` | Basic RAG with search + answer | OpenAI/Anthropic, PostgreSQL |
|
||||
| `07-rag-complete.json` | Full RAG demo (index + search + answer) | OpenAI, PostgreSQL |
|
||||
| `08-mcp-list-tools.json` | List tools from MCP server | MCP Server |
|
||||
| `09-mcp-call-tool.json` | Call MCP tool (weather) | MCP Server |
|
||||
| `10-mcp-ai-agent.json` | AI agent with MCP tools | OpenAI/Anthropic, MCP Server |
|
||||
| `11-video-openai-sora.json` | Generate video with OpenAI Sora-2 (async) | OpenAI |
|
||||
| `12-video-gemini-veo.json` | Generate video with Google Veo-3 (async) | Google Vertex AI |
|
||||
| `13-image-to-video-pipeline.json` | Image + video generation pipeline | OpenAI |
|
||||
| `14-stabilityai-image.json` | Image generation with Stability AI (SD3.5) | Stability AI |
|
||||
| `15-pdf-generation.json` | Generate PDF from markdown content | None (built-in) |
|
||||
| `16-llm-to-pdf-pipeline.json` | LLM generates report → convert to PDF | OpenAI |
|
||||
| `17-web-search.json` | Chat with built-in web search for real-time info | OpenAI |
|
||||
| `18-code-execution.json` | Chat with built-in code execution sandbox | Google Gemini |
|
||||
| `19-coding-agent.json` | Coding agent: plan → write & run code → review | OpenAI |
|
||||
| `20-extended-thinking.json` | Extended thinking with token budget for reasoning | Anthropic |
|
||||
| `21-web-search-research-agent.json` | Research agent: web search → synthesize → PDF | OpenAI, Anthropic |
|
||||
| `22-multi-turn-chain.json` | Multi-turn conversation chaining with previousResponseId | OpenAI |
|
||||
| `30-rag-sqlite-vec.json` | Zero-infra RAG on the bundled SQLite + sqlite-vec store | OpenAI, SQLite (built-in) |
|
||||
|
||||
### A2A (Agent2Agent) examples
|
||||
|
||||
Conductor as an A2A **client** (calling remote agents) and **server** (exposing a workflow as an
|
||||
agent). The client tasks (`AGENT`, `GET_AGENT_CARD`, `CANCEL_AGENT`) need a reachable A2A
|
||||
agent — see `ai/src/test/resources/a2a/` for a runnable test agent. The server examples are exposed
|
||||
by registering them with `metadata.a2a.enabled=true` and `conductor.a2a.server.enabled=true`.
|
||||
|
||||
| File | Description | Requirements |
|
||||
|------|-------------|--------------|
|
||||
| `10-a2a-call-agent.json` | Call a remote agent (poll mode) | A2A agent |
|
||||
| `11-a2a-get-agent-card.json` | Discover an agent's skills/capabilities | A2A agent |
|
||||
| `12-a2a-server-workflow.json` | Expose a workflow as an A2A agent (server) | `conductor.a2a.server.enabled=true` |
|
||||
| `23-a2a-streaming.json` | Call an agent in streaming (SSE) mode | A2A agent (`capabilities.streaming=true`) |
|
||||
| `24-a2a-push.json` | Call an agent in push-notification mode | A2A agent, `conductor.a2a.callback.url` |
|
||||
| `25-a2a-server-multi-turn.json` | Multi-turn server agent (HUMAN task → input-required → resume) | `conductor.a2a.server.enabled=true` |
|
||||
| `26-a2a-cancel.json` | Start then cancel a remote agent task | A2A agent |
|
||||
| `27-a2a-multi-agent.json` | Call multiple agents in parallel (FORK_JOIN → JOIN) | A2A agents |
|
||||
| `28-a2a-llm-pick-skill.json` | Discover an agent, let an LLM pick the prompt, then call it | A2A agent, OpenAI/Anthropic |
|
||||
| `29-a2a-client-multi-turn.json` | Client multi-turn: branch on input-required, re-call with the same context | A2A agent |
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Step 1: Register a Workflow
|
||||
|
||||
```bash
|
||||
# Register the chat completion workflow
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @01-chat-completion.json
|
||||
```
|
||||
|
||||
### Step 2: Execute the Workflow
|
||||
|
||||
```bash
|
||||
# Run the workflow (no input needed for hardcoded examples)
|
||||
curl -X POST 'http://localhost:8080/api/workflow/chat_workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{}'
|
||||
```
|
||||
|
||||
### Step 3: Check the Result
|
||||
|
||||
```bash
|
||||
# Get workflow execution status (replace {workflowId} with the returned ID)
|
||||
curl -X GET 'http://localhost:8080/api/workflow/{workflowId}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Example Commands
|
||||
|
||||
### 1. Chat Completion
|
||||
|
||||
```bash
|
||||
# Register
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @01-chat-completion.json
|
||||
|
||||
# Execute
|
||||
curl -X POST 'http://localhost:8080/api/workflow/chat_workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{}'
|
||||
```
|
||||
|
||||
### 2. Generate Embeddings
|
||||
|
||||
```bash
|
||||
# Register
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @02-generate-embeddings.json
|
||||
|
||||
# Execute
|
||||
curl -X POST 'http://localhost:8080/api/workflow/embedding_workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{}'
|
||||
```
|
||||
|
||||
### 3. Image Generation
|
||||
|
||||
```bash
|
||||
# Register
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @03-image-generation.json
|
||||
|
||||
# Execute
|
||||
curl -X POST 'http://localhost:8080/api/workflow/image_gen_workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{}'
|
||||
```
|
||||
|
||||
### 4. Audio Generation (TTS)
|
||||
|
||||
```bash
|
||||
# Register
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @04-audio-generation.json
|
||||
|
||||
# Execute
|
||||
curl -X POST 'http://localhost:8080/api/workflow/tts_workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{}'
|
||||
```
|
||||
|
||||
### 5. Semantic Search
|
||||
|
||||
```bash
|
||||
# Register
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @05-semantic-search.json
|
||||
|
||||
# Execute
|
||||
curl -X POST 'http://localhost:8080/api/workflow/semantic_search_workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{}'
|
||||
```
|
||||
|
||||
### 6. RAG (Basic)
|
||||
|
||||
```bash
|
||||
# Register
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @06-rag-basic.json
|
||||
|
||||
# Execute with a question
|
||||
curl -X POST 'http://localhost:8080/api/workflow/rag_workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"question": "What is Conductor?"}'
|
||||
```
|
||||
|
||||
### 7. RAG (Complete Demo)
|
||||
|
||||
```bash
|
||||
# Register
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @07-rag-complete.json
|
||||
|
||||
# Execute (no input needed - fully self-contained)
|
||||
curl -X POST 'http://localhost:8080/api/workflow/complete_rag_demo' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{}'
|
||||
```
|
||||
|
||||
### 30. RAG on SQLite (sqlite-vec, zero infrastructure)
|
||||
|
||||
Runs the full index → search → answer RAG loop against the **embedded** SQLite + sqlite-vec vector
|
||||
store — no PostgreSQL, MongoDB or Pinecone required. When the server runs with `conductor.db.type=sqlite`
|
||||
and `conductor.integrations.ai.enabled=true`, Conductor bundles the native `vec0` extension and
|
||||
auto-registers a vector DB instance named `default`, which this workflow targets. Embeddings are
|
||||
requested at 256 dimensions to match that default instance.
|
||||
|
||||
```bash
|
||||
# Register
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @30-rag-sqlite-vec.json
|
||||
|
||||
# Execute with a question
|
||||
curl -X POST 'http://localhost:8080/api/workflow/rag_sqlite_vec_demo' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"question": "What vector databases does Conductor support?"}'
|
||||
```
|
||||
|
||||
### 8. MCP List Tools
|
||||
|
||||
```bash
|
||||
# Start MCP server first (see Prerequisites)
|
||||
|
||||
# Register
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @08-mcp-list-tools.json
|
||||
|
||||
# Execute
|
||||
curl -X POST 'http://localhost:8080/api/workflow/mcp_list_tools_workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{}'
|
||||
```
|
||||
|
||||
### 9. MCP Call Tool (Weather)
|
||||
|
||||
```bash
|
||||
# Start MCP server first (see Prerequisites)
|
||||
|
||||
# Register
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @09-mcp-call-tool.json
|
||||
|
||||
# Execute
|
||||
curl -X POST 'http://localhost:8080/api/workflow/mcp_weather_workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{}'
|
||||
```
|
||||
|
||||
### 10. MCP AI Agent
|
||||
|
||||
```bash
|
||||
# Start MCP server first (see Prerequisites)
|
||||
|
||||
# Register
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @10-mcp-ai-agent.json
|
||||
|
||||
# Execute with a task
|
||||
curl -X POST 'http://localhost:8080/api/workflow/mcp_ai_agent_workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"task": "Get the current weather in San Francisco"}'
|
||||
```
|
||||
|
||||
### 11. Video Generation (OpenAI Sora)
|
||||
|
||||
```bash
|
||||
# Register
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @11-video-openai-sora.json
|
||||
|
||||
# Execute (async -- returns workflowId immediately, polls internally until video is ready)
|
||||
curl -X POST 'http://localhost:8080/api/workflow/video_gen_openai_sora' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{}'
|
||||
```
|
||||
|
||||
### 12. Video Generation (Google Gemini Veo)
|
||||
|
||||
```bash
|
||||
# Requires Google Vertex AI credentials (see Prerequisites)
|
||||
|
||||
# Register
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @12-video-gemini-veo.json
|
||||
|
||||
# Execute
|
||||
curl -X POST 'http://localhost:8080/api/workflow/video_gen_gemini_veo' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{}'
|
||||
```
|
||||
|
||||
### 13. Image-to-Video Pipeline
|
||||
|
||||
```bash
|
||||
# Register
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @13-image-to-video-pipeline.json
|
||||
|
||||
# Execute (generates a DALL-E image first, then a Sora video)
|
||||
curl -X POST 'http://localhost:8080/api/workflow/image_to_video_pipeline' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{}'
|
||||
```
|
||||
|
||||
### 14. Image Generation (Stability AI)
|
||||
|
||||
```bash
|
||||
# Requires STABILITY_API_KEY environment variable
|
||||
|
||||
# Register
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @14-stabilityai-image.json
|
||||
|
||||
# Execute
|
||||
curl -X POST 'http://localhost:8080/api/workflow/image_gen_stabilityai' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{}'
|
||||
```
|
||||
|
||||
### 15. PDF Generation (Markdown to PDF)
|
||||
|
||||
```bash
|
||||
# No external API keys required -- uses built-in PDFBox renderer
|
||||
|
||||
# Register
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @15-pdf-generation.json
|
||||
|
||||
# Execute
|
||||
curl -X POST 'http://localhost:8080/api/workflow/pdf_generation_workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{}'
|
||||
```
|
||||
|
||||
### 16. LLM-to-PDF Pipeline (Report Generation)
|
||||
|
||||
```bash
|
||||
# Register
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @16-llm-to-pdf-pipeline.json
|
||||
|
||||
# Execute with a topic and audience
|
||||
curl -X POST 'http://localhost:8080/api/workflow/llm_to_pdf_pipeline' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"topic": "Cloud Migration Best Practices", "audience": "CTO and engineering leadership"}'
|
||||
```
|
||||
|
||||
### 17. Web Search
|
||||
|
||||
```bash
|
||||
# Register
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @17-web-search.json
|
||||
|
||||
# Execute with a question about current events
|
||||
curl -X POST 'http://localhost:8080/api/workflow/web_search_workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"question": "What are the latest developments in AI regulation?"}'
|
||||
```
|
||||
|
||||
### 18. Code Execution
|
||||
|
||||
```bash
|
||||
# Register
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @18-code-execution.json
|
||||
|
||||
# Execute with a data analysis task
|
||||
curl -X POST 'http://localhost:8080/api/workflow/code_execution_workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"task": "Generate the first 50 Fibonacci numbers and calculate the golden ratio convergence"}'
|
||||
```
|
||||
|
||||
### 19. Coding Agent
|
||||
|
||||
```bash
|
||||
# Register
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @19-coding-agent.json
|
||||
|
||||
# Execute — the agent plans, writes code, executes, and reviews
|
||||
curl -X POST 'http://localhost:8080/api/workflow/coding_agent' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"task": "Write a Python function that converts Roman numerals to integers, with unit tests"}'
|
||||
```
|
||||
|
||||
### 20. Extended Thinking
|
||||
|
||||
```bash
|
||||
# Register
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @20-extended-thinking.json
|
||||
|
||||
# Execute with a complex reasoning problem
|
||||
curl -X POST 'http://localhost:8080/api/workflow/extended_thinking_workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"problem": "Design a distributed consensus algorithm for a system with up to 3 Byzantine nodes out of 10 total. Explain the correctness proof."}'
|
||||
```
|
||||
|
||||
### 21. Web Research Agent
|
||||
|
||||
```bash
|
||||
# Register
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @21-web-search-research-agent.json
|
||||
|
||||
# Execute — researches the topic, writes a report, converts to PDF
|
||||
curl -X POST 'http://localhost:8080/api/workflow/web_research_agent' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"topic": "The state of WebAssembly in 2026"}'
|
||||
```
|
||||
|
||||
### 22. Multi-Turn Conversation Chain
|
||||
|
||||
```bash
|
||||
# Register
|
||||
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @22-multi-turn-chain.json
|
||||
|
||||
# Execute — second turn uses previousResponseId to continue the conversation without resending history
|
||||
curl -X POST 'http://localhost:8080/api/workflow/multi_turn_chain' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"topic": "Real-time collaborative document editor"}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Register All Workflows at Once
|
||||
|
||||
```bash
|
||||
# Register all example workflows
|
||||
for f in *.json; do
|
||||
echo "Registering $f..."
|
||||
curl -s -X POST 'http://localhost:8080/api/metadata/workflow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d @"$f"
|
||||
echo ""
|
||||
done
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "VectorDB not found: postgres-prod"
|
||||
|
||||
Ensure you have configured the PostgreSQL vector database in your `application.properties`:
|
||||
|
||||
```properties
|
||||
conductor.vectordb.instances[0].name=postgres-prod
|
||||
conductor.vectordb.instances[0].type=postgres
|
||||
conductor.vectordb.instances[0].postgres.datasourceURL=jdbc:postgresql://localhost:5432/vectors
|
||||
conductor.vectordb.instances[0].postgres.user=conductor
|
||||
conductor.vectordb.instances[0].postgres.password=secret
|
||||
conductor.vectordb.instances[0].postgres.dimensions=1536
|
||||
```
|
||||
|
||||
### "No configuration found for: openai"
|
||||
|
||||
Ensure you have set the OpenAI API key environment variable:
|
||||
|
||||
```bash
|
||||
export OPENAI_API_KEY=sk-your-openai-api-key
|
||||
```
|
||||
|
||||
### MCP Server Connection Refused
|
||||
|
||||
1. Verify the MCP server is running:
|
||||
```bash
|
||||
curl http://localhost:3001/mcp
|
||||
```
|
||||
|
||||
2. Check the server logs for errors
|
||||
|
||||
3. Ensure you're using the correct port in the workflow (default: 3001)
|
||||
|
||||
### PostgreSQL Vector Extension Not Found
|
||||
|
||||
Ensure the `pgvector` extension is installed in your PostgreSQL database:
|
||||
|
||||
```sql
|
||||
CREATE EXTENSION IF NOT EXISTS vector;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
Copyright 2026 Conductor Authors. Licensed under the Apache License 2.0.
|
||||
@@ -0,0 +1,235 @@
|
||||
/*
|
||||
* Copyright 2025 Conductor Authors.
|
||||
* <p>
|
||||
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package org.conductoross.conductor.ai;
|
||||
|
||||
import java.net.URI;
|
||||
import java.net.URISyntaxException;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.function.Function;
|
||||
|
||||
import org.conductoross.conductor.ai.model.AudioGenRequest;
|
||||
import org.conductoross.conductor.ai.model.ChatCompletion;
|
||||
import org.conductoross.conductor.ai.model.EmbeddingGenRequest;
|
||||
import org.conductoross.conductor.ai.model.ImageGenRequest;
|
||||
import org.conductoross.conductor.ai.model.LLMResponse;
|
||||
import org.conductoross.conductor.ai.model.ToolSpec;
|
||||
import org.conductoross.conductor.ai.model.VideoGenRequest;
|
||||
import org.conductoross.conductor.ai.video.VideoModel;
|
||||
import org.conductoross.conductor.ai.video.VideoOptions;
|
||||
import org.conductoross.conductor.ai.video.VideoOptionsBuilder;
|
||||
import org.springframework.ai.chat.model.ChatModel;
|
||||
import org.springframework.ai.chat.prompt.ChatOptions;
|
||||
import org.springframework.ai.image.ImageModel;
|
||||
import org.springframework.ai.image.ImageOptions;
|
||||
import org.springframework.ai.image.ImageOptionsBuilder;
|
||||
import org.springframework.ai.model.tool.ToolCallingChatOptions;
|
||||
import org.springframework.ai.tool.ToolCallback;
|
||||
import org.springframework.ai.tool.function.FunctionToolCallback;
|
||||
|
||||
import com.netflix.conductor.common.config.ObjectMapperProvider;
|
||||
|
||||
import com.fasterxml.jackson.core.JsonProcessingException;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
|
||||
/** Interface for LLM implementations. */
|
||||
public interface AIModel {
|
||||
|
||||
enum ConductorTask {
|
||||
CHAT_COMPLETE,
|
||||
GENERATE_IMAGE,
|
||||
GENERATE_VIDEO,
|
||||
TEXT_TO_SPEECH
|
||||
}
|
||||
|
||||
ObjectMapper objectMapper = new ObjectMapperProvider().getObjectMapper();
|
||||
|
||||
/**
|
||||
* @return name of the foundation model provider. e.g. openai, anthropic etc.
|
||||
*/
|
||||
String getModelProvider();
|
||||
|
||||
/**
|
||||
* @return alternative provider names that resolve to this same provider
|
||||
*/
|
||||
default List<String> getProviderAliases() {
|
||||
return List.of();
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether this provider accepts a chat-completion request whose last message has the {@code
|
||||
* assistant} role — i.e. assistant-message prefill, used to nudge the model's response to start
|
||||
* a certain way (and, in this codebase, the format in which {@link
|
||||
* org.conductoross.conductor.ai.tasks.mapper.ChatCompleteTaskMapper}'s loop-history injection
|
||||
* carries prior DO_WHILE iteration outputs back into the next iteration).
|
||||
*
|
||||
* <p>When this returns {@code false}, the mapper suppresses the same-refName loop-iteration
|
||||
* assistant injection: those auto-attached assistant messages would either be rejected outright
|
||||
* by the provider's API (e.g. Anthropic Claude Sonnet 4.6+ returns {@code 400 "This model does
|
||||
* not support assistant message prefill. The conversation must end with a user message."}) or
|
||||
* be silently mis-handled. Participants, tool calls, and sub-workflow context are unaffected by
|
||||
* this flag.
|
||||
*
|
||||
* <p>Default is {@code true} to preserve historical behavior for providers (OpenAI Responses
|
||||
* API with {@code previousResponseId}, Gemini, etc.) that accept trailing assistant messages.
|
||||
*/
|
||||
default boolean supportsAssistantPrefill() {
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Embedding generation
|
||||
*
|
||||
* @param embeddingGenRequest request
|
||||
* @return embeddings
|
||||
*/
|
||||
List<Float> generateEmbeddings(EmbeddingGenRequest embeddingGenRequest);
|
||||
|
||||
/**
|
||||
* @return Chat Completion model
|
||||
*/
|
||||
ChatModel getChatModel();
|
||||
|
||||
/**
|
||||
* @param input request to do chat completion
|
||||
* @return Options
|
||||
*/
|
||||
default ChatOptions getChatOptions(ChatCompletion input) {
|
||||
return ToolCallingChatOptions.builder()
|
||||
.model(input.getModel())
|
||||
.maxTokens(input.getMaxTokens())
|
||||
.topP(input.getTopP())
|
||||
.temperature(input.getTemperature())
|
||||
.toolCallbacks(getToolCallback(input))
|
||||
.stopSequences(input.getStopWords())
|
||||
.frequencyPenalty(input.getFrequencyPenalty())
|
||||
.topK(input.getTopK())
|
||||
.internalToolExecutionEnabled(false)
|
||||
.presencePenalty(input.getPresencePenalty())
|
||||
.build();
|
||||
}
|
||||
|
||||
/**
|
||||
* @param input Image gen request
|
||||
* @return Options
|
||||
*/
|
||||
default ImageOptions getImageOptions(ImageGenRequest input) {
|
||||
return ImageOptionsBuilder.builder()
|
||||
.model(input.getModel())
|
||||
.height(input.getHeight())
|
||||
.width(input.getWidth())
|
||||
.N(input.getN())
|
||||
.responseFormat("b64_json")
|
||||
.style(input.getStyle())
|
||||
.build();
|
||||
}
|
||||
|
||||
/**
|
||||
* @return Model to generate images
|
||||
*/
|
||||
ImageModel getImageModel();
|
||||
|
||||
/**
|
||||
* @param input Video gen request
|
||||
* @return Options
|
||||
*/
|
||||
default VideoOptions getVideoOptions(VideoGenRequest input) {
|
||||
return VideoOptionsBuilder.builder()
|
||||
.model(input.getModel())
|
||||
.duration(input.getDuration())
|
||||
.width(input.getWidth())
|
||||
.height(input.getHeight())
|
||||
.fps(input.getFps())
|
||||
.outputFormat(input.getOutputFormat())
|
||||
.n(input.getN())
|
||||
.style(input.getStyle())
|
||||
.motion(input.getMotion())
|
||||
.seed(input.getSeed())
|
||||
.guidanceScale(input.getGuidanceScale())
|
||||
.aspectRatio(input.getAspectRatio())
|
||||
.generateThumbnail(input.getGenerateThumbnail())
|
||||
.thumbnailTimestamp(input.getThumbnailTimestamp())
|
||||
.inputImage(input.getInputImage())
|
||||
.negativePrompt(input.getNegativePrompt())
|
||||
.personGeneration(input.getPersonGeneration())
|
||||
.resolution(input.getResolution())
|
||||
.generateAudio(input.getGenerateAudio())
|
||||
.size(input.getSize())
|
||||
.build();
|
||||
}
|
||||
|
||||
/**
|
||||
* @return Model to generate videos
|
||||
*/
|
||||
default VideoModel getVideoModel() {
|
||||
return null; // Default: video generation not supported
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate video (async job submission)
|
||||
*
|
||||
* @param request Video generation request
|
||||
* @return Response with job ID
|
||||
*/
|
||||
default LLMResponse generateVideo(VideoGenRequest request) {
|
||||
throw new UnsupportedOperationException("Video generation not supported by this provider");
|
||||
}
|
||||
|
||||
/**
|
||||
* Check video generation job status (polling)
|
||||
*
|
||||
* @param request Video generation request with jobId
|
||||
* @return Response with current status or completed video
|
||||
*/
|
||||
default LLMResponse checkVideoStatus(VideoGenRequest request) {
|
||||
throw new UnsupportedOperationException("Video generation not supported by this provider");
|
||||
}
|
||||
|
||||
default LLMResponse generateAudio(AudioGenRequest request) {
|
||||
throw new UnsupportedOperationException();
|
||||
}
|
||||
|
||||
default List<ToolCallback> getToolCallback(ChatCompletion input) {
|
||||
if (input.getTools() == null || input.getTools().isEmpty()) {
|
||||
return List.of();
|
||||
}
|
||||
List<ToolCallback> functions = new ArrayList<>();
|
||||
try {
|
||||
for (ToolSpec tool : input.getTools()) {
|
||||
FunctionToolCallback<Object, Object> function =
|
||||
FunctionToolCallback.builder(tool.getName(), Function.identity())
|
||||
.description(tool.getDescription())
|
||||
.inputSchema(objectMapper.writeValueAsString(tool.getInputSchema()))
|
||||
.inputType(Map.class) // does not matter, we are not doing
|
||||
// internal tool calling!
|
||||
.build();
|
||||
functions.add(function);
|
||||
}
|
||||
} catch (JsonProcessingException jpe) {
|
||||
throw new RuntimeException(jpe);
|
||||
}
|
||||
return functions;
|
||||
}
|
||||
|
||||
static URI getURI(String input) {
|
||||
if (input == null || input.isBlank()) {
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
return new URI(input);
|
||||
} catch (URISyntaxException e) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,77 @@
|
||||
/*
|
||||
* Copyright 2025 Conductor Authors.
|
||||
* <p>
|
||||
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package org.conductoross.conductor.ai;
|
||||
|
||||
import java.io.File;
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.function.Consumer;
|
||||
|
||||
import org.conductoross.conductor.ai.model.LLMWorkerInput;
|
||||
import org.springframework.core.env.Environment;
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
import lombok.Getter;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
@Getter
|
||||
@Component
|
||||
@Slf4j
|
||||
public class AIModelProvider {
|
||||
|
||||
private final Map<String, AIModel> providerToLLM = new HashMap<>();
|
||||
|
||||
private String payloadStoreLocation;
|
||||
|
||||
public AIModelProvider(
|
||||
List<ModelConfiguration<? extends AIModel>> modelConfigurations, Environment env) {
|
||||
String defaultPayloadStoreLocation = System.getProperty("user.home") + "/worker-payload/";
|
||||
|
||||
for (ModelConfiguration<? extends AIModel> modelConfiguration : modelConfigurations) {
|
||||
try {
|
||||
AIModel llm = modelConfiguration.get();
|
||||
payloadStoreLocation =
|
||||
env.getProperty(
|
||||
"conductor.file-storage.parentDir", defaultPayloadStoreLocation);
|
||||
boolean result = new File(payloadStoreLocation).mkdirs();
|
||||
log.info(
|
||||
"Created directory {} ? {} for storing worker payload data",
|
||||
payloadStoreLocation,
|
||||
result);
|
||||
providerToLLM.put(llm.getModelProvider(), llm);
|
||||
for (String alias : llm.getProviderAliases()) {
|
||||
providerToLLM.put(alias, llm);
|
||||
}
|
||||
} catch (Throwable t) {
|
||||
log.error("cannot init {} model, reason: {}", modelConfiguration, t.getMessage());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public AIModel getModel(LLMWorkerInput input) {
|
||||
String name = input.getLlmProvider();
|
||||
if (name == null) {
|
||||
throw new RuntimeException("llmProvider not specified: " + name);
|
||||
}
|
||||
AIModel model = providerToLLM.get(name);
|
||||
if (model == null) {
|
||||
throw new RuntimeException("no configuration found for: " + name);
|
||||
}
|
||||
return model;
|
||||
}
|
||||
|
||||
public Consumer<TokenUsageLog> getTokenUsageLogger() {
|
||||
return usageLog -> log.info("{}", usageLog);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,909 @@
|
||||
/*
|
||||
* Copyright 2025 Conductor Authors.
|
||||
* <p>
|
||||
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package org.conductoross.conductor.ai;
|
||||
|
||||
import java.net.URI;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Base64;
|
||||
import java.util.Collections;
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
import java.util.Optional;
|
||||
import java.util.Set;
|
||||
import java.util.UUID;
|
||||
import java.util.function.Consumer;
|
||||
import java.util.stream.Collectors;
|
||||
|
||||
import org.apache.commons.lang3.StringUtils;
|
||||
import org.conductoross.conductor.ai.document.DocumentLoader;
|
||||
import org.conductoross.conductor.ai.http.AIHttpClients;
|
||||
import org.conductoross.conductor.ai.model.AudioGenRequest;
|
||||
import org.conductoross.conductor.ai.model.ChatCompletion;
|
||||
import org.conductoross.conductor.ai.model.ChatMessage;
|
||||
import org.conductoross.conductor.ai.model.EmbeddingGenRequest;
|
||||
import org.conductoross.conductor.ai.model.ImageGenRequest;
|
||||
import org.conductoross.conductor.ai.model.LLMResponse;
|
||||
import org.conductoross.conductor.ai.model.ToolCall;
|
||||
import org.conductoross.conductor.ai.model.ToolSpec;
|
||||
import org.conductoross.conductor.ai.model.VideoGenRequest;
|
||||
import org.conductoross.conductor.common.JsonSchemaValidator;
|
||||
import org.conductoross.conductor.common.utils.StringTemplate;
|
||||
import org.springframework.ai.chat.client.ChatClient;
|
||||
import org.springframework.ai.chat.messages.AssistantMessage;
|
||||
import org.springframework.ai.chat.messages.Message;
|
||||
import org.springframework.ai.chat.messages.SystemMessage;
|
||||
import org.springframework.ai.chat.messages.ToolResponseMessage;
|
||||
import org.springframework.ai.chat.messages.UserMessage;
|
||||
import org.springframework.ai.chat.model.ChatModel;
|
||||
import org.springframework.ai.chat.model.ChatResponse;
|
||||
import org.springframework.ai.chat.model.Generation;
|
||||
import org.springframework.ai.chat.prompt.ChatOptions;
|
||||
import org.springframework.ai.chat.prompt.Prompt;
|
||||
import org.springframework.ai.content.Media;
|
||||
import org.springframework.ai.image.ImageGeneration;
|
||||
import org.springframework.ai.image.ImageMessage;
|
||||
import org.springframework.ai.image.ImageModel;
|
||||
import org.springframework.ai.image.ImageOptions;
|
||||
import org.springframework.ai.image.ImagePrompt;
|
||||
import org.springframework.ai.image.ImageResponse;
|
||||
import org.springframework.util.MimeType;
|
||||
|
||||
import com.netflix.conductor.common.config.ObjectMapperProvider;
|
||||
import com.netflix.conductor.common.metadata.SchemaDef;
|
||||
import com.netflix.conductor.common.metadata.tasks.Task;
|
||||
|
||||
import com.fasterxml.jackson.core.JsonProcessingException;
|
||||
import com.fasterxml.jackson.core.type.TypeReference;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.google.common.annotations.VisibleForTesting;
|
||||
import com.networknt.schema.JsonSchemaException;
|
||||
import com.networknt.schema.ValidationMessage;
|
||||
import lombok.SneakyThrows;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import okhttp3.OkHttpClient;
|
||||
import okhttp3.Request;
|
||||
import okhttp3.Response;
|
||||
import okhttp3.ResponseBody;
|
||||
|
||||
import static com.netflix.conductor.common.metadata.tasks.TaskType.TASK_TYPE_SIMPLE;
|
||||
|
||||
import static org.conductoross.conductor.ai.MimeExtensionResolver.getExtension;
|
||||
import static org.conductoross.conductor.ai.MimeExtensionResolver.getMimeTypeFromUrl;
|
||||
|
||||
@Slf4j
|
||||
public class LLMHelper {
|
||||
private static final TypeReference<Map<String, Object>> MAP_OF_STRING_TO_OBJ =
|
||||
new TypeReference<>() {};
|
||||
private static final Map<String, String> finishReasonMap =
|
||||
Map.of("end_turn", "STOP", "tool_use", "TOOL_CALLS", "refusal", "CONTENT_FILTER");
|
||||
private final ObjectMapper objectMapper = new ObjectMapperProvider().getObjectMapper();
|
||||
|
||||
private final JsonSchemaValidator jsonSchemaValidator;
|
||||
private final List<DocumentLoader> documentLoaders;
|
||||
private final OkHttpClient httpClient;
|
||||
|
||||
public LLMHelper(
|
||||
JsonSchemaValidator jsonSchemaValidator, List<DocumentLoader> documentLoaders) {
|
||||
this(jsonSchemaValidator, documentLoaders, AIHttpClients.defaultClient());
|
||||
}
|
||||
|
||||
public LLMHelper(
|
||||
JsonSchemaValidator jsonSchemaValidator,
|
||||
List<DocumentLoader> documentLoaders,
|
||||
OkHttpClient httpClient) {
|
||||
this.jsonSchemaValidator = jsonSchemaValidator;
|
||||
this.documentLoaders = documentLoaders;
|
||||
this.httpClient = httpClient;
|
||||
}
|
||||
|
||||
public LLMResponse chatComplete(
|
||||
Task task,
|
||||
AIModel llm,
|
||||
ChatCompletion chatCompletion,
|
||||
String payloadStoreLocation,
|
||||
Consumer<TokenUsageLog> tokenUsageLogger) {
|
||||
|
||||
ChatModel chatModel = llm.getChatModel();
|
||||
ChatOptions chatOptions = llm.getChatOptions(chatCompletion);
|
||||
LLMResponse response = chatComplete(chatModel, chatOptions, chatCompletion);
|
||||
|
||||
String prompt =
|
||||
replacePromptVariables(
|
||||
chatCompletion.getInstructions(), chatCompletion.getPromptVariables());
|
||||
chatCompletion.setPrompt(prompt);
|
||||
extractResponse(response, chatCompletion);
|
||||
storeMedia(payloadStoreLocation, response.getMedia());
|
||||
|
||||
TokenUsageLog usage =
|
||||
TokenUsageLog.builder()
|
||||
.taskId(task.getTaskId())
|
||||
.api(chatCompletion.getModel())
|
||||
.integrationName(chatCompletion.getLlmProvider())
|
||||
.completionTokens(response.getCompletionTokens())
|
||||
.promptTokens(response.getPromptTokens())
|
||||
.totalTokens(response.getTokenUsed())
|
||||
.build();
|
||||
|
||||
tokenUsageLogger.accept(usage);
|
||||
return response;
|
||||
}
|
||||
|
||||
public LLMResponse generateImage(
|
||||
Task task,
|
||||
AIModel llm,
|
||||
ImageGenRequest imageGenRequest,
|
||||
String payloadStoreLocation,
|
||||
Consumer<TokenUsageLog> tokenUsageLogger) {
|
||||
|
||||
String prompt =
|
||||
replacePromptVariables(
|
||||
imageGenRequest.getPrompt(), imageGenRequest.getPromptVariables());
|
||||
imageGenRequest.setPrompt(prompt);
|
||||
ImageOptions options = llm.getImageOptions(imageGenRequest);
|
||||
ImageModel model = llm.getImageModel();
|
||||
LLMResponse response = generateImage(model, options, imageGenRequest);
|
||||
storeMedia(payloadStoreLocation, response.getMedia());
|
||||
|
||||
TokenUsageLog usage =
|
||||
TokenUsageLog.builder()
|
||||
.taskId(task.getTaskId())
|
||||
.api(imageGenRequest.getModel())
|
||||
.integrationName(imageGenRequest.getLlmProvider())
|
||||
.completionTokens(response.getCompletionTokens())
|
||||
.promptTokens(response.getPromptTokens())
|
||||
.totalTokens(response.getTokenUsed())
|
||||
.build();
|
||||
tokenUsageLogger.accept(usage);
|
||||
return response;
|
||||
}
|
||||
|
||||
public List<Float> generateEmbeddings(
|
||||
Task task,
|
||||
AIModel llm,
|
||||
EmbeddingGenRequest embeddingGenRequest,
|
||||
Consumer<TokenUsageLog> tokenUsageLogger) {
|
||||
return llm.generateEmbeddings(embeddingGenRequest);
|
||||
}
|
||||
|
||||
public LLMResponse generateAudio(
|
||||
Task task,
|
||||
AIModel llm,
|
||||
AudioGenRequest request,
|
||||
String payloadStoreLocation,
|
||||
Consumer<TokenUsageLog> tokenUsageLogger) {
|
||||
LLMResponse response = llm.generateAudio(request);
|
||||
storeMedia(payloadStoreLocation, response.getMedia());
|
||||
TokenUsageLog usage =
|
||||
TokenUsageLog.builder()
|
||||
.taskId(task.getTaskId())
|
||||
.api(request.getModel())
|
||||
.integrationName(request.getLlmProvider())
|
||||
.completionTokens(response.getCompletionTokens())
|
||||
.promptTokens(response.getPromptTokens())
|
||||
.totalTokens(response.getTokenUsed())
|
||||
.build();
|
||||
tokenUsageLogger.accept(usage);
|
||||
return response;
|
||||
}
|
||||
|
||||
public LLMResponse generateVideo(
|
||||
Task task,
|
||||
AIModel llm,
|
||||
VideoGenRequest videoGenRequest,
|
||||
String payloadStoreLocation,
|
||||
Consumer<TokenUsageLog> tokenUsageLogger) {
|
||||
|
||||
return llm.generateVideo(videoGenRequest);
|
||||
}
|
||||
|
||||
public LLMResponse checkVideoStatus(
|
||||
Task task, AIModel llm, VideoGenRequest videoGenRequest, String payloadStoreLocation) {
|
||||
|
||||
LLMResponse response = llm.checkVideoStatus(videoGenRequest);
|
||||
|
||||
// If completed, download and store media
|
||||
if ("COMPLETED".equals(response.getFinishReason())) {
|
||||
storeMedia(payloadStoreLocation, response.getMedia());
|
||||
}
|
||||
|
||||
return response;
|
||||
}
|
||||
|
||||
// Helper methods
|
||||
|
||||
private String replacePromptVariables(String prompt, Map<String, Object> paramReplacement) {
|
||||
if (StringUtils.isBlank(prompt)) {
|
||||
return prompt;
|
||||
}
|
||||
if (paramReplacement != null) {
|
||||
prompt = StringTemplate.fString(prompt, paramReplacement);
|
||||
}
|
||||
return prompt;
|
||||
}
|
||||
|
||||
@SneakyThrows
|
||||
@SuppressWarnings({"raw", "unchecked"})
|
||||
private void extractResponse(LLMResponse llmResponse, ChatCompletion input) {
|
||||
if (llmResponse.getResult() == null || llmResponse.getResult().toString().isEmpty()) {
|
||||
// empty response
|
||||
log.debug("empty response: finishReason: {}", llmResponse.getFinishReason());
|
||||
return;
|
||||
}
|
||||
Object result = llmResponse.getResult();
|
||||
switch (result) {
|
||||
case null -> llmResponse.setResult(Map.of());
|
||||
case String ignored ->
|
||||
llmResponse.setResult(
|
||||
tryToConvertToJSON(llmResponse.getResult().toString(), input));
|
||||
case List<?> resultList -> {
|
||||
List<String> errors = new ArrayList<>();
|
||||
List<Object> output = new ArrayList<>();
|
||||
boolean hasJsonOutput = false;
|
||||
for (Object o : resultList) {
|
||||
String responseText = o.toString();
|
||||
var responseObj = tryToConvertToJSON(responseText, input);
|
||||
hasJsonOutput = true;
|
||||
if (input.getOutputSchema() != null) {
|
||||
String error = null;
|
||||
if (!(responseObj instanceof Map)) {
|
||||
error = "not a JSON response: %s".formatted(responseObj);
|
||||
} else {
|
||||
error =
|
||||
validateJsonSchema(
|
||||
input.getInputSchema(),
|
||||
(Map<String, Object>) responseObj);
|
||||
}
|
||||
if (error != null) {
|
||||
errors.add(
|
||||
String.format(
|
||||
"Output does not confirm to the schema. errors: %s",
|
||||
error));
|
||||
}
|
||||
}
|
||||
output.add(responseObj);
|
||||
}
|
||||
llmResponse.setResult(output);
|
||||
if (input.isJsonOutput() && !hasJsonOutput && !llmResponse.hasToolCalls()) {
|
||||
Map<String, Object> outputErrors = Map.of("error", errors, "response", output);
|
||||
throw new RuntimeException(objectMapper.writeValueAsString(outputErrors));
|
||||
}
|
||||
}
|
||||
default -> llmResponse.setResult(result.toString());
|
||||
}
|
||||
}
|
||||
|
||||
@SneakyThrows
|
||||
private Object tryToConvertToJSON(String responseText, ChatCompletion chatCompletion) {
|
||||
try {
|
||||
responseText = responseText.trim();
|
||||
if (responseText.startsWith("```json")) {
|
||||
responseText = responseText.substring("```json".length());
|
||||
responseText =
|
||||
responseText.substring(0, responseText.length() - "```".length() - 1);
|
||||
}
|
||||
Map<String, Object> map = objectMapper.readValue(responseText, MAP_OF_STRING_TO_OBJ);
|
||||
if (chatCompletion.getOutputSchema() != null) {
|
||||
String error = validateJsonSchema(chatCompletion.getInputSchema(), map);
|
||||
if (error != null) {
|
||||
throw new RuntimeException(
|
||||
String.format(
|
||||
"Output does not confirm to the schema. errors: %s", error));
|
||||
}
|
||||
}
|
||||
// llmResponse.setResult(map);
|
||||
return map;
|
||||
|
||||
} catch (JsonProcessingException e) {
|
||||
if (chatCompletion.isJsonOutput()) {
|
||||
log.error(
|
||||
"error converting to json, response: {}, error: {}",
|
||||
responseText,
|
||||
e.getMessage(),
|
||||
e);
|
||||
Map<String, Object> outputErrors =
|
||||
Map.of("error", e.getMessage(), "response", responseText);
|
||||
throw new RuntimeException(objectMapper.writeValueAsString(outputErrors));
|
||||
}
|
||||
return responseText;
|
||||
}
|
||||
}
|
||||
|
||||
private String validateJsonSchema(final SchemaDef schema, Map<String, Object> data) {
|
||||
try {
|
||||
// Order in which we use the schema
|
||||
// 1. If there is data -- inline schema def, we use that
|
||||
// 2. Else use name + version to lookup
|
||||
// 3. externalRef if present, in future we will use it -- currently not supported
|
||||
String schemaContent = objectMapper.writeValueAsString(schema.getData());
|
||||
if (schemaContent == null) {
|
||||
return null;
|
||||
}
|
||||
|
||||
Set<ValidationMessage> validationMessages =
|
||||
jsonSchemaValidator.validate(schemaContent, data);
|
||||
|
||||
if (validationMessages != null && !validationMessages.isEmpty()) {
|
||||
return String.format(
|
||||
"Schema validation failed %s",
|
||||
validationMessages.stream()
|
||||
.map(ValidationMessage::getMessage)
|
||||
.collect(Collectors.joining(", ")));
|
||||
}
|
||||
return null;
|
||||
} catch (JsonSchemaException jpe) {
|
||||
throw new RuntimeException(
|
||||
"Bad/Unsupported schema? : " + jpe.getValidationMessages().toString());
|
||||
} catch (JsonProcessingException jpe) {
|
||||
throw new RuntimeException("Error parsing the json schema : " + jpe.getMessage(), jpe);
|
||||
}
|
||||
}
|
||||
|
||||
@SneakyThrows
|
||||
private LLMResponse chatComplete(
|
||||
ChatModel chatModel, ChatOptions chatOptions, ChatCompletion input) {
|
||||
ChatClient chatClient = ChatClient.create(chatModel);
|
||||
if (StringUtils.isNotBlank(input.getInstructions())) {
|
||||
input.getMessages()
|
||||
.addFirst(new ChatMessage(ChatMessage.Role.system, input.getInstructions()));
|
||||
}
|
||||
|
||||
List<Message> messages =
|
||||
new ArrayList<>(input.getMessages().stream().map(this::constructMessage).toList());
|
||||
|
||||
ensureLastMessageIsFromUser(messages);
|
||||
|
||||
Prompt prompt = new Prompt(messages, chatOptions);
|
||||
ChatResponse chatResponse = chatClient.prompt(prompt).call().chatResponse();
|
||||
if (chatResponse == null) {
|
||||
throw new RuntimeException("No response generated");
|
||||
}
|
||||
if (chatResponse.getResults().isEmpty()) {
|
||||
String result = objectMapper.writeValueAsString(chatResponse);
|
||||
return LLMResponse.builder()
|
||||
.result(result)
|
||||
.completionTokens(chatResponse.getMetadata().getUsage().getCompletionTokens())
|
||||
.promptTokens(chatResponse.getMetadata().getUsage().getPromptTokens())
|
||||
.tokenUsed(chatResponse.getMetadata().getUsage().getTotalTokens())
|
||||
.build();
|
||||
}
|
||||
|
||||
List<ToolCall> tools = null;
|
||||
String finishReason = null;
|
||||
List<String> responses = new ArrayList<>();
|
||||
List<org.conductoross.conductor.ai.model.Media> media = new ArrayList<>();
|
||||
for (Generation result : chatResponse.getResults()) {
|
||||
if (result.getOutput().hasToolCalls()) {
|
||||
List<AssistantMessage.ToolCall> toolCalls = result.getOutput().getToolCalls();
|
||||
tools = new ArrayList<>();
|
||||
for (AssistantMessage.ToolCall toolCall : toolCalls) {
|
||||
String name = toolCall.name();
|
||||
String id = toolCall.id();
|
||||
if (id == null || id.isBlank()) {
|
||||
id = UUID.randomUUID().toString();
|
||||
}
|
||||
String argsAsString = toolCall.arguments();
|
||||
Map<String, Object> args = new HashMap<>();
|
||||
try {
|
||||
@SuppressWarnings("unchecked")
|
||||
Map<String, Object> parsedArgs =
|
||||
objectMapper.readValue(argsAsString, Map.class);
|
||||
// Recursively parse any nested JSON strings
|
||||
args = parseNestedJsonStrings(parsedArgs);
|
||||
} catch (JsonProcessingException ignored) {
|
||||
log.warn(ignored.getMessage(), ignored);
|
||||
}
|
||||
args.put("method", name);
|
||||
|
||||
Optional<ToolSpec> matched =
|
||||
input.getTools().stream()
|
||||
.filter(toolSpec -> toolSpec.getName().equals(name))
|
||||
.findFirst();
|
||||
|
||||
String type = matched.map(ToolSpec::getType).orElse(TASK_TYPE_SIMPLE);
|
||||
tools.add(
|
||||
ToolCall.builder()
|
||||
.taskReferenceName(id)
|
||||
.name(name)
|
||||
.inputParameters(args)
|
||||
.integrationNames(getIntegrationNames(name, input.getTools()))
|
||||
.type(type)
|
||||
.build());
|
||||
}
|
||||
finishReason = result.getMetadata().getFinishReason();
|
||||
} else {
|
||||
responses.add(result.getOutput().getText());
|
||||
result.getOutput()
|
||||
.getMedia()
|
||||
.forEach(
|
||||
m ->
|
||||
media.add(
|
||||
org.conductoross.conductor.ai.model.Media.builder()
|
||||
.data(m.getDataAsByteArray())
|
||||
.mimeType(m.getMimeType().toString())
|
||||
.build()));
|
||||
// storeMedia(outputLocation, result.getOutput().getMedia());
|
||||
if (finishReason == null) {
|
||||
finishReason = result.getMetadata().getFinishReason();
|
||||
}
|
||||
}
|
||||
}
|
||||
Object result = responses;
|
||||
if (responses.size() == 1) {
|
||||
result = responses.getFirst();
|
||||
}
|
||||
finishReason = finishReasonMap.getOrDefault(finishReason, finishReason).toUpperCase();
|
||||
|
||||
// Extract response_id if present (set by OpenAI Responses API for chaining)
|
||||
String responseId = null;
|
||||
Object respIdObj = chatResponse.getMetadata().get("response_id");
|
||||
if (respIdObj instanceof String rid) {
|
||||
responseId = rid;
|
||||
}
|
||||
|
||||
// Reasoning summary + reasoning token count. Surfaced by the OpenAI
|
||||
// Responses API, Anthropic extended thinking, and Gemini thought
|
||||
// summaries — the chat-model adapters normalize all three onto these
|
||||
// two metadata keys before we read them here.
|
||||
String reasoning = null;
|
||||
Object reasoningObj = chatResponse.getMetadata().get("reasoning");
|
||||
if (reasoningObj instanceof String r && !r.isBlank()) {
|
||||
reasoning = r;
|
||||
}
|
||||
Integer reasoningTokens = null;
|
||||
Object rtObj = chatResponse.getMetadata().get("reasoning_tokens");
|
||||
// ``Number`` rather than ``Integer`` so a Long survives the Jackson
|
||||
// round-trip ChatResponseMetadata may go through. Token counts won't
|
||||
// overflow int — convert and move on.
|
||||
if (rtObj instanceof Number rt) {
|
||||
reasoningTokens = rt.intValue();
|
||||
}
|
||||
|
||||
return LLMResponse.builder()
|
||||
.result(result)
|
||||
.media(media)
|
||||
.toolCalls(tools)
|
||||
.finishReason(finishReason)
|
||||
.responseId(responseId)
|
||||
.reasoning(reasoning)
|
||||
.reasoningTokens(reasoningTokens)
|
||||
.completionTokens(chatResponse.getMetadata().getUsage().getCompletionTokens())
|
||||
.promptTokens(chatResponse.getMetadata().getUsage().getPromptTokens())
|
||||
.tokenUsed(chatResponse.getMetadata().getUsage().getTotalTokens())
|
||||
.build();
|
||||
}
|
||||
|
||||
private LLMResponse generateImage(
|
||||
ImageModel imageModel, ImageOptions options, ImageGenRequest request) {
|
||||
ImageMessage imageMessage = new ImageMessage(request.getPrompt(), request.getWeight());
|
||||
ImagePrompt prompt = new ImagePrompt(List.of(imageMessage), options);
|
||||
ImageResponse response = imageModel.call(prompt);
|
||||
LLMResponse mediaGenResponse = new LLMResponse();
|
||||
List<org.conductoross.conductor.ai.model.Media> mediaList = new ArrayList<>();
|
||||
for (ImageGeneration result : response.getResults()) {
|
||||
var image = result.getOutput();
|
||||
String url = image.getUrl();
|
||||
String base64 = image.getB64Json();
|
||||
|
||||
// Determine the mime type from the output format
|
||||
String mimeType =
|
||||
"image/"
|
||||
+ (request.getOutputFormat() != null
|
||||
? request.getOutputFormat()
|
||||
: "png");
|
||||
|
||||
if (base64 != null) {
|
||||
// Base64 data provided - decode and store
|
||||
mediaList.add(
|
||||
org.conductoross.conductor.ai.model.Media.builder()
|
||||
.data(Base64.getDecoder().decode(base64))
|
||||
.mimeType(mimeType)
|
||||
.build());
|
||||
} else if (url != null) {
|
||||
// URL provided - download the image bytes so we can store locally
|
||||
// This ensures the image is persisted even after the provider's URL expires
|
||||
byte[] imageBytes = downloadImageFromUrl(url);
|
||||
if (imageBytes != null) {
|
||||
// Detect mime type from URL extension if possible
|
||||
String detectedMimeType = getMimeTypeFromUrl(url, mimeType);
|
||||
mediaList.add(
|
||||
org.conductoross.conductor.ai.model.Media.builder()
|
||||
.data(imageBytes)
|
||||
.mimeType(detectedMimeType)
|
||||
.build());
|
||||
} else {
|
||||
// Fallback: if download fails, keep the URL (will expire but better than
|
||||
// nothing)
|
||||
log.warn("Failed to download image from URL, keeping external URL: {}", url);
|
||||
mediaList.add(
|
||||
org.conductoross.conductor.ai.model.Media.builder()
|
||||
.location(url)
|
||||
.mimeType(mimeType)
|
||||
.build());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
mediaGenResponse.setMedia(mediaList);
|
||||
|
||||
return mediaGenResponse;
|
||||
}
|
||||
|
||||
/**
|
||||
* Downloads image bytes from a URL. Used to persist images locally instead of relying on
|
||||
* provider-hosted URLs that may expire.
|
||||
*
|
||||
* @param url The image URL to download from
|
||||
* @return The image bytes, or null if download failed
|
||||
*/
|
||||
private byte[] downloadImageFromUrl(String url) {
|
||||
try {
|
||||
Request request = new Request.Builder().url(url).get().build();
|
||||
|
||||
try (Response response = httpClient.newCall(request).execute()) {
|
||||
if (!response.isSuccessful()) {
|
||||
log.error(
|
||||
"Failed to download image from URL {}: HTTP {}", url, response.code());
|
||||
return null;
|
||||
}
|
||||
ResponseBody body = response.body();
|
||||
if (body == null) {
|
||||
log.error("Empty response body when downloading image from URL: {}", url);
|
||||
return null;
|
||||
}
|
||||
byte[] bytes = body.bytes();
|
||||
log.debug("Downloaded {} bytes from image URL: {}", bytes.length, url);
|
||||
return bytes;
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.error("Exception downloading image from URL {}: {}", url, e.getMessage());
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
@SneakyThrows
|
||||
private Message constructMessage(ChatMessage chatMessage) {
|
||||
return switch (chatMessage.getRole()) {
|
||||
case user -> getMessage(chatMessage);
|
||||
case assistant -> new AssistantMessage(chatMessage.getMessage());
|
||||
case system -> new SystemMessage(chatMessage.getMessage());
|
||||
case tool_call ->
|
||||
AssistantMessage.builder()
|
||||
.content("{}")
|
||||
.toolCalls(
|
||||
chatMessage.getToolCalls().stream()
|
||||
.map(
|
||||
tc -> {
|
||||
var name =
|
||||
extractMethodFromInputParameters(
|
||||
tc.getInputParameters());
|
||||
return new AssistantMessage.ToolCall(
|
||||
tc.getTaskReferenceName(),
|
||||
"function",
|
||||
name == null ? tc.getName() : name,
|
||||
toJSON(tc.getInputParameters()));
|
||||
})
|
||||
.toList())
|
||||
.build();
|
||||
case tool -> {
|
||||
List<ToolCall> toolCalls = chatMessage.getToolCalls();
|
||||
if (toolCalls == null) {
|
||||
log.warn("chat message role: {}, but toolCalls is null", chatMessage.getRole());
|
||||
toolCalls = new ArrayList<>();
|
||||
}
|
||||
try {
|
||||
List<ToolResponseMessage.ToolResponse> responses = new ArrayList<>();
|
||||
if (toolCalls.isEmpty()) {
|
||||
log.info("toolCalls is empty for {}", chatMessage);
|
||||
}
|
||||
for (ToolCall toolCall : toolCalls) {
|
||||
Map<String, Object> inputJson = toolCall.getInputParameters();
|
||||
var name = extractMethodFromInputParameters(inputJson);
|
||||
String outputJSON = objectMapper.writeValueAsString(toolCall.getOutput());
|
||||
|
||||
log.trace("outputJSON for {} is {}", toolCall.getName(), outputJSON);
|
||||
log.info("tool: {}", toolCall);
|
||||
log.info("tool.getTaskReferenceName: {}", toolCall.getTaskReferenceName());
|
||||
responses.add(
|
||||
new ToolResponseMessage.ToolResponse(
|
||||
toolCall.getTaskReferenceName(),
|
||||
name == null ? toolCall.getName() : name,
|
||||
outputJSON));
|
||||
}
|
||||
log.trace("responses: {}", responses);
|
||||
yield ToolResponseMessage.builder().responses(responses).build();
|
||||
} catch (Exception e) {
|
||||
log.error("error: {}", e.getMessage(), e);
|
||||
yield new SystemMessage(chatMessage.getMessage());
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
private String toJSON(Object input) {
|
||||
try {
|
||||
if (input == null) {
|
||||
return "{}";
|
||||
}
|
||||
return objectMapper.writeValueAsString(input);
|
||||
} catch (JsonProcessingException jpe) {
|
||||
return String.valueOf(input);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Extracts the "method" value from a map structure where the outer key is dynamic. The map
|
||||
* structure is expected to be: { "dynamicKey": { "method": "methodName", ... } }
|
||||
*
|
||||
* @param inputParameters The map containing the nested structure
|
||||
* @return The method value as a String, or null if not found
|
||||
*/
|
||||
@SuppressWarnings("unchecked")
|
||||
@VisibleForTesting
|
||||
String extractMethodFromInputParameters(Map<String, Object> inputParameters) {
|
||||
if (inputParameters == null || inputParameters.isEmpty()) {
|
||||
return null;
|
||||
}
|
||||
|
||||
// Iterate through all entries to find the nested map with "method" key
|
||||
for (Map.Entry<String, Object> entry : inputParameters.entrySet()) {
|
||||
Object value = entry.getValue();
|
||||
if (value instanceof Map) {
|
||||
Map<String, Object> nestedMap = (Map<String, Object>) value;
|
||||
if (nestedMap.containsKey("method")) {
|
||||
Object methodValue = nestedMap.get("method");
|
||||
return methodValue != null ? methodValue.toString() : null;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
private Message getMessage(ChatMessage msg) {
|
||||
List<String> rawMedia = msg.getMedia();
|
||||
List<Media> media =
|
||||
(rawMedia == null ? List.<String>of() : rawMedia).stream()
|
||||
.map(m -> getMedia(msg.getMimeType(), m))
|
||||
.filter(Objects::nonNull)
|
||||
.toList();
|
||||
return UserMessage.builder().text(msg.getMessage()).media(media).build();
|
||||
}
|
||||
|
||||
private Media getMedia(String mimeType, String content) {
|
||||
// content can be a URL or base64 encoded data
|
||||
URI uri = AIModel.getURI(content);
|
||||
if (uri == null) {
|
||||
return Media.builder().data(content).mimeType(MimeType.valueOf(mimeType)).build();
|
||||
}
|
||||
Optional<byte[]> data =
|
||||
documentLoaders.stream()
|
||||
.filter(documentLoader -> documentLoader.supports(content))
|
||||
.findFirst()
|
||||
.map(loader -> loader.download(content));
|
||||
final String mimeTypeResolved =
|
||||
Optional.ofNullable(mimeType)
|
||||
.orElse(MimeExtensionResolver.getMimeTypeFromUrl(content, ""));
|
||||
return data.map(
|
||||
bytes ->
|
||||
Media.builder()
|
||||
.data(bytes)
|
||||
.mimeType(MimeType.valueOf(mimeTypeResolved))
|
||||
.build())
|
||||
.orElse(null);
|
||||
}
|
||||
|
||||
private void storeMedia(
|
||||
String location, List<org.conductoross.conductor.ai.model.Media> media) {
|
||||
|
||||
DocumentLoader documentLoader =
|
||||
documentLoaders.stream()
|
||||
.filter(loader -> loader.supports(location))
|
||||
.findFirst()
|
||||
.orElse(null);
|
||||
if (documentLoader == null) {
|
||||
log.debug("no document loaders found, media will not be stored");
|
||||
return;
|
||||
}
|
||||
media.stream()
|
||||
.filter(m1 -> m1.getData() != null)
|
||||
.forEach(
|
||||
m -> {
|
||||
// Each media item gets a unique path with file extension
|
||||
// to prevent overwriting when multiple items exist
|
||||
// (e.g., video + thumbnail)
|
||||
String ext = getExtension(m.getMimeType());
|
||||
String uniqueLocation =
|
||||
location + "_" + java.util.UUID.randomUUID() + ext;
|
||||
String uploadLocation =
|
||||
documentLoader.upload(
|
||||
Map.of(), m.getMimeType(), m.getData(), uniqueLocation);
|
||||
m.setLocation(uploadLocation);
|
||||
m.setData(null);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Stores media from an InputStream, streaming directly to the DocumentLoader without buffering
|
||||
* the full content in memory. Intended for large media files such as video.
|
||||
*
|
||||
* @param location Base storage location (e.g., file:///path/to/storage)
|
||||
* @param mimeType MIME type of the media (e.g., "video/mp4")
|
||||
* @param stream InputStream containing the media data
|
||||
* @return The storage location where the media was written, or null if no loader was found
|
||||
*/
|
||||
public String storeMediaStream(String location, String mimeType, java.io.InputStream stream) {
|
||||
Optional<DocumentLoader> docLoader =
|
||||
documentLoaders.stream()
|
||||
.filter(documentLoader -> documentLoader.supports(location))
|
||||
.findFirst();
|
||||
if (docLoader.isPresent()) {
|
||||
String ext = getExtension(mimeType);
|
||||
String uniqueLocation = location + "_" + java.util.UUID.randomUUID() + ext;
|
||||
docLoader.get().upload(Map.of(), mimeType, stream, uniqueLocation);
|
||||
return uniqueLocation;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
private Map<String, String> getIntegrationNames(String toolCallName, List<ToolSpec> toolSpecs) {
|
||||
Optional<ToolSpec> matched =
|
||||
toolSpecs.stream()
|
||||
.filter(toolSpec -> toolSpec.getName().equals(toolCallName))
|
||||
.findFirst();
|
||||
return matched.map(ToolSpec::getIntegrationNames).orElse(Collections.emptyMap());
|
||||
}
|
||||
|
||||
/**
|
||||
* Recursively parses JSON strings within a Map structure. This handles cases where the LLM
|
||||
* generates nested JSON strings like "{\"value\":\"page\"}".
|
||||
*
|
||||
* @param input The input Map that may contain JSON strings
|
||||
* @return A new Map with JSON strings parsed into their object representations
|
||||
*/
|
||||
@SuppressWarnings("unchecked")
|
||||
@VisibleForTesting
|
||||
Map<String, Object> parseNestedJsonStrings(Map<String, Object> input) {
|
||||
if (input == null) {
|
||||
return null;
|
||||
}
|
||||
|
||||
Map<String, Object> result = new HashMap<>();
|
||||
|
||||
for (Map.Entry<String, Object> entry : input.entrySet()) {
|
||||
String key = entry.getKey();
|
||||
Object value = entry.getValue();
|
||||
|
||||
if (value instanceof String) {
|
||||
String stringValue = (String) value;
|
||||
if (isJsonString(stringValue)) {
|
||||
try {
|
||||
// Parse the JSON string into an object
|
||||
Object parsedValue = objectMapper.readValue(stringValue, Object.class);
|
||||
// Recursively parse any nested JSON strings in the parsed object
|
||||
if (parsedValue instanceof Map) {
|
||||
parsedValue = parseNestedJsonStrings((Map<String, Object>) parsedValue);
|
||||
} else if (parsedValue instanceof List) {
|
||||
parsedValue = parseNestedJsonStringsInList((List<Object>) parsedValue);
|
||||
}
|
||||
result.put(key, parsedValue);
|
||||
} catch (Exception e) {
|
||||
// If parsing fails, keep the original string value
|
||||
log.debug(
|
||||
"Failed to parse JSON string for key {}: {}", key, e.getMessage());
|
||||
result.put(key, value);
|
||||
}
|
||||
} else {
|
||||
result.put(key, value);
|
||||
}
|
||||
} else if (value instanceof Map) {
|
||||
// Recursively parse nested Maps
|
||||
result.put(key, parseNestedJsonStrings((Map<String, Object>) value));
|
||||
} else if (value instanceof List) {
|
||||
// Recursively parse nested Lists
|
||||
result.put(key, parseNestedJsonStringsInList((List<Object>) value));
|
||||
} else {
|
||||
result.put(key, value);
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/** Recursively parses JSON strings within a List structure. */
|
||||
@SuppressWarnings("unchecked")
|
||||
private List<Object> parseNestedJsonStringsInList(List<Object> input) {
|
||||
if (input == null) {
|
||||
return null;
|
||||
}
|
||||
|
||||
List<Object> result = new ArrayList<>();
|
||||
|
||||
for (Object item : input) {
|
||||
if (item instanceof String) {
|
||||
String stringValue = (String) item;
|
||||
if (isJsonString(stringValue)) {
|
||||
try {
|
||||
Object parsedValue = objectMapper.readValue(stringValue, Object.class);
|
||||
if (parsedValue instanceof Map) {
|
||||
parsedValue = parseNestedJsonStrings((Map<String, Object>) parsedValue);
|
||||
} else if (parsedValue instanceof List) {
|
||||
parsedValue = parseNestedJsonStringsInList((List<Object>) parsedValue);
|
||||
}
|
||||
result.add(parsedValue);
|
||||
} catch (Exception e) {
|
||||
log.debug("Failed to parse JSON string in list: {}", e.getMessage());
|
||||
result.add(item);
|
||||
}
|
||||
} else {
|
||||
result.add(item);
|
||||
}
|
||||
} else if (item instanceof Map) {
|
||||
result.add(parseNestedJsonStrings((Map<String, Object>) item));
|
||||
} else if (item instanceof List) {
|
||||
result.add(parseNestedJsonStringsInList((List<Object>) item));
|
||||
} else {
|
||||
result.add(item);
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensures the conversation ends with a user message. Some providers (e.g. Anthropic/Claude)
|
||||
* reject requests where the last message has an assistant or tool_call role ("assistant message
|
||||
* prefill"). This typically happens when the prior iteration ended with finishReason=MAX_TOKENS
|
||||
* and the DO_WHILE loop continues with the partial assistant response as the last message in
|
||||
* the history. Appending a user continuation prompt is safe for all providers — OpenAI and
|
||||
* others simply treat it as the next user turn.
|
||||
*
|
||||
* @param messages The mutable list of messages to check and potentially modify
|
||||
*/
|
||||
@VisibleForTesting
|
||||
void ensureLastMessageIsFromUser(List<Message> messages) {
|
||||
if (messages.isEmpty()) return;
|
||||
Message last = messages.getLast();
|
||||
if (last instanceof UserMessage) return;
|
||||
|
||||
if (last instanceof AssistantMessage assistantMsg) {
|
||||
// Replace trailing assistant message with a user message that includes
|
||||
// the partial text as context + continuation instruction.
|
||||
// This avoids the "assistant message prefill" error from Claude.
|
||||
String partialText = assistantMsg.getText();
|
||||
messages.removeLast();
|
||||
String continuation =
|
||||
partialText != null && !partialText.isBlank()
|
||||
? "You were saying:\n\n"
|
||||
+ partialText
|
||||
+ "\n\nPlease continue where you left off."
|
||||
: "Please continue where you left off.";
|
||||
messages.add(new UserMessage(continuation));
|
||||
} else {
|
||||
// For any other non-user message type (tool_call, system, etc.)
|
||||
messages.add(new UserMessage("Please continue where you left off."));
|
||||
}
|
||||
}
|
||||
|
||||
/** Checks if a string looks like JSON */
|
||||
@VisibleForTesting
|
||||
boolean isJsonString(String value) {
|
||||
if (value == null || value.trim().isEmpty()) {
|
||||
return false;
|
||||
}
|
||||
String trimmed = value.trim();
|
||||
return (trimmed.startsWith("{") && trimmed.endsWith("}"))
|
||||
|| (trimmed.startsWith("[") && trimmed.endsWith("]"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,130 @@
|
||||
/*
|
||||
* Copyright 2025 Conductor Authors.
|
||||
* <p>
|
||||
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package org.conductoross.conductor.ai;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.UUID;
|
||||
import java.util.function.Consumer;
|
||||
|
||||
import org.conductoross.conductor.ai.document.DocumentLoader;
|
||||
import org.conductoross.conductor.ai.model.AudioGenRequest;
|
||||
import org.conductoross.conductor.ai.model.ChatCompletion;
|
||||
import org.conductoross.conductor.ai.model.EmbeddingGenRequest;
|
||||
import org.conductoross.conductor.ai.model.ImageGenRequest;
|
||||
import org.conductoross.conductor.ai.model.LLMResponse;
|
||||
import org.conductoross.conductor.ai.model.VideoGenRequest;
|
||||
import org.conductoross.conductor.common.JsonSchemaValidator;
|
||||
import org.conductoross.conductor.common.utils.StringTemplate;
|
||||
import org.conductoross.conductor.config.AIIntegrationEnabledCondition;
|
||||
import org.springframework.context.annotation.Conditional;
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
import com.netflix.conductor.common.metadata.tasks.Task;
|
||||
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import okhttp3.OkHttpClient;
|
||||
|
||||
@Component
|
||||
@Conditional(AIIntegrationEnabledCondition.class)
|
||||
@Slf4j
|
||||
public class LLMs {
|
||||
|
||||
protected AIModelProvider modelProvider;
|
||||
protected LLMHelper helper;
|
||||
protected String payloadStoreLocation;
|
||||
protected Consumer<TokenUsageLog> tokenUsageLogger =
|
||||
(tokenUsageLog) -> {
|
||||
log.info("{}", tokenUsageLog);
|
||||
};
|
||||
|
||||
public LLMs(
|
||||
List<DocumentLoader> documentLoaders,
|
||||
JsonSchemaValidator jsonSchemaValidator,
|
||||
AIModelProvider modelProvider,
|
||||
OkHttpClient conductorAiHttpClient) {
|
||||
this.modelProvider = modelProvider;
|
||||
this.helper = new LLMHelper(jsonSchemaValidator, documentLoaders, conductorAiHttpClient);
|
||||
this.payloadStoreLocation = modelProvider.getPayloadStoreLocation();
|
||||
}
|
||||
|
||||
public LLMResponse chatComplete(Task task, ChatCompletion chatCompletion) {
|
||||
AIModel llm = this.modelProvider.getModel(chatCompletion);
|
||||
String prompt =
|
||||
replacePromptVariables(
|
||||
task,
|
||||
chatCompletion.getInstructions(),
|
||||
chatCompletion.getPromptVariables());
|
||||
chatCompletion.setInstructions(prompt);
|
||||
return helper.chatComplete(
|
||||
task, llm, chatCompletion, getPayloadStoreLocation(task), tokenUsageLogger);
|
||||
}
|
||||
|
||||
public LLMResponse generateImage(Task task, ImageGenRequest imageGenRequest) {
|
||||
AIModel llm = this.modelProvider.getModel(imageGenRequest);
|
||||
String prompt =
|
||||
replacePromptVariables(
|
||||
task, imageGenRequest.getPrompt(), imageGenRequest.getPromptVariables());
|
||||
imageGenRequest.setPrompt(prompt);
|
||||
return helper.generateImage(
|
||||
task, llm, imageGenRequest, getPayloadStoreLocation(task), tokenUsageLogger);
|
||||
}
|
||||
|
||||
public LLMResponse generateAudio(Task task, AudioGenRequest audioGenRequest) {
|
||||
AIModel llm = this.modelProvider.getModel(audioGenRequest);
|
||||
String prompt =
|
||||
replacePromptVariables(
|
||||
task, audioGenRequest.getPrompt(), audioGenRequest.getPromptVariables());
|
||||
audioGenRequest.setPrompt(prompt);
|
||||
return helper.generateAudio(
|
||||
task, llm, audioGenRequest, getPayloadStoreLocation(task), tokenUsageLogger);
|
||||
}
|
||||
|
||||
public List<Float> generateEmbeddings(Task task, EmbeddingGenRequest embeddingGenRequest) {
|
||||
AIModel llm = this.modelProvider.getModel(embeddingGenRequest);
|
||||
return helper.generateEmbeddings(task, llm, embeddingGenRequest, tokenUsageLogger);
|
||||
}
|
||||
|
||||
public LLMResponse generateVideo(Task task, VideoGenRequest videoGenRequest) {
|
||||
AIModel llm = this.modelProvider.getModel(videoGenRequest);
|
||||
String prompt =
|
||||
replacePromptVariables(
|
||||
task, videoGenRequest.getPrompt(), videoGenRequest.getPromptVariables());
|
||||
videoGenRequest.setPrompt(prompt);
|
||||
return helper.generateVideo(
|
||||
task, llm, videoGenRequest, getPayloadStoreLocation(task), tokenUsageLogger);
|
||||
}
|
||||
|
||||
public LLMResponse checkVideoStatus(Task task, VideoGenRequest videoGenRequest) {
|
||||
AIModel llm = this.modelProvider.getModel(videoGenRequest);
|
||||
return helper.checkVideoStatus(task, llm, videoGenRequest, getPayloadStoreLocation(task));
|
||||
}
|
||||
|
||||
public String replacePromptVariables(
|
||||
Task task, String prompt, Map<String, Object> paramReplacement) {
|
||||
if (paramReplacement != null) {
|
||||
prompt = StringTemplate.fString(prompt, paramReplacement);
|
||||
}
|
||||
return prompt;
|
||||
}
|
||||
|
||||
public String getPayloadStoreLocation(Task task) {
|
||||
return payloadStoreLocation
|
||||
+ "/"
|
||||
+ task.getWorkflowInstanceId()
|
||||
+ "/"
|
||||
+ task.getTaskId()
|
||||
+ "/"
|
||||
+ UUID.randomUUID();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,179 @@
|
||||
/*
|
||||
* Copyright 2025 Conductor Authors.
|
||||
* <p>
|
||||
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package org.conductoross.conductor.ai;
|
||||
|
||||
import java.util.HashMap;
|
||||
import java.util.Map;
|
||||
|
||||
public class MimeExtensionResolver {
|
||||
|
||||
private static final Map<String, String> mimeToExt = new HashMap<>();
|
||||
private static final Map<String, String> extToMime = new HashMap<>();
|
||||
|
||||
static {
|
||||
// ----- IMAGE -----
|
||||
mimeToExt.put("image/jpeg", ".jpg");
|
||||
mimeToExt.put("image/jpg", ".jpg");
|
||||
mimeToExt.put("image/png", ".png");
|
||||
mimeToExt.put("image/gif", ".gif");
|
||||
mimeToExt.put("image/bmp", ".bmp");
|
||||
mimeToExt.put("image/webp", ".webp");
|
||||
mimeToExt.put("image/tiff", ".tiff");
|
||||
mimeToExt.put("image/svg+xml", ".svg");
|
||||
mimeToExt.put("image/x-icon", ".ico");
|
||||
mimeToExt.put("image/heif", ".heif");
|
||||
mimeToExt.put("image/heic", ".heic");
|
||||
|
||||
// ----- AUDIO -----
|
||||
mimeToExt.put("audio/mpeg", ".mp3");
|
||||
mimeToExt.put("audio/wav", ".wav");
|
||||
mimeToExt.put("audio/x-wav", ".wav");
|
||||
mimeToExt.put("audio/ogg", ".ogg");
|
||||
mimeToExt.put("audio/flac", ".flac");
|
||||
mimeToExt.put("audio/aac", ".aac");
|
||||
mimeToExt.put("audio/mp4", ".m4a");
|
||||
mimeToExt.put("audio/opus", ".opus");
|
||||
mimeToExt.put("audio/webm", ".weba");
|
||||
mimeToExt.put("audio/amr", ".amr");
|
||||
|
||||
// ----- VIDEO -----
|
||||
mimeToExt.put("video/mp4", ".mp4");
|
||||
mimeToExt.put("video/mpeg", ".mpeg");
|
||||
mimeToExt.put("video/x-msvideo", ".avi");
|
||||
mimeToExt.put("video/x-ms-wmv", ".wmv");
|
||||
mimeToExt.put("video/quicktime", ".mov");
|
||||
mimeToExt.put("video/webm", ".webm");
|
||||
mimeToExt.put("video/3gpp", ".3gp");
|
||||
mimeToExt.put("video/3gpp2", ".3g2");
|
||||
mimeToExt.put("video/x-flv", ".flv");
|
||||
mimeToExt.put("video/x-matroska", ".mkv");
|
||||
|
||||
// ----- DOCUMENTS -----
|
||||
mimeToExt.put("application/pdf", ".pdf");
|
||||
mimeToExt.put("application/msword", ".doc");
|
||||
mimeToExt.put(
|
||||
"application/vnd.openxmlformats-officedocument.wordprocessingml.document", ".docx");
|
||||
mimeToExt.put("application/vnd.ms-excel", ".xls");
|
||||
mimeToExt.put("application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", ".xlsx");
|
||||
mimeToExt.put("application/vnd.ms-powerpoint", ".ppt");
|
||||
mimeToExt.put(
|
||||
"application/vnd.openxmlformats-officedocument.presentationml.presentation",
|
||||
".pptx");
|
||||
mimeToExt.put("application/rtf", ".rtf");
|
||||
mimeToExt.put("application/zip", ".zip");
|
||||
mimeToExt.put("application/x-7z-compressed", ".7z");
|
||||
mimeToExt.put("application/x-rar-compressed", ".rar");
|
||||
mimeToExt.put("application/json", ".json");
|
||||
mimeToExt.put("application/xml", ".xml");
|
||||
|
||||
// ----- TEXT / CODE -----
|
||||
mimeToExt.put("text/plain", ".txt");
|
||||
mimeToExt.put("text/html", ".html");
|
||||
mimeToExt.put("text/css", ".css");
|
||||
mimeToExt.put("text/csv", ".csv");
|
||||
mimeToExt.put("text/javascript", ".js");
|
||||
|
||||
// ----- Reverse mapping (ext → mime) -----
|
||||
for (Map.Entry<String, String> e : mimeToExt.entrySet()) {
|
||||
String ext = e.getValue().replaceFirst("^\\.", "");
|
||||
extToMime.put(ext, e.getKey());
|
||||
}
|
||||
// aliases
|
||||
extToMime.put("jpg", "image/jpeg");
|
||||
extToMime.put("jpeg", "image/jpeg");
|
||||
extToMime.put("htm", "text/html");
|
||||
}
|
||||
|
||||
public static String getExtension(String input) {
|
||||
if (input == null || input.isEmpty()) return "";
|
||||
|
||||
input = input.trim().toLowerCase();
|
||||
|
||||
// Case 1: Input looks like extension
|
||||
if (!input.contains("/") && !input.contains("*")) {
|
||||
if (input.startsWith(".")) input = input.substring(1);
|
||||
String mime = extToMime.get(input);
|
||||
if (mime != null) return mimeToExt.get(mime);
|
||||
return "." + input; // fallback
|
||||
}
|
||||
|
||||
// Case 2: Wildcard MIME
|
||||
if (input.endsWith("/*")) {
|
||||
String type = input.substring(0, input.indexOf('/'));
|
||||
switch (type) {
|
||||
case "image":
|
||||
return ".jpg";
|
||||
case "audio":
|
||||
return ".mp3";
|
||||
case "video":
|
||||
return ".mp4";
|
||||
case "text":
|
||||
return ".txt";
|
||||
case "application":
|
||||
return ".bin";
|
||||
default:
|
||||
return "";
|
||||
}
|
||||
}
|
||||
|
||||
// Case 3: Exact MIME
|
||||
return mimeToExt.getOrDefault(input, "");
|
||||
}
|
||||
|
||||
public static String getMimeType(String ext) {
|
||||
if (ext == null || ext.isEmpty()) return "";
|
||||
if (ext.startsWith(".")) ext = ext.substring(1);
|
||||
return extToMime.getOrDefault(ext.toLowerCase(), "application/octet-stream");
|
||||
}
|
||||
|
||||
/**
|
||||
* Attempts to detect the MIME type from a URL by examining its path for known file extensions.
|
||||
* Useful when downloading media from external URLs where the Content-Type header may not be
|
||||
* reliable.
|
||||
*
|
||||
* @param url The URL to examine
|
||||
* @param defaultMimeType The default MIME type to return if no extension is detected
|
||||
* @return The detected MIME type, or the default if detection fails
|
||||
*/
|
||||
public static String getMimeTypeFromUrl(String url, String defaultMimeType) {
|
||||
if (url == null || url.isEmpty()) {
|
||||
return defaultMimeType;
|
||||
}
|
||||
|
||||
// Remove query string and fragment
|
||||
String path = url;
|
||||
int queryIdx = path.indexOf('?');
|
||||
if (queryIdx > 0) {
|
||||
path = path.substring(0, queryIdx);
|
||||
}
|
||||
int fragIdx = path.indexOf('#');
|
||||
if (fragIdx > 0) {
|
||||
path = path.substring(0, fragIdx);
|
||||
}
|
||||
|
||||
// Find the last dot in the path
|
||||
int lastDot = path.lastIndexOf('.');
|
||||
if (lastDot > 0 && lastDot < path.length() - 1) {
|
||||
// Extract extension (up to 5 chars to avoid false positives)
|
||||
String ext = path.substring(lastDot + 1).toLowerCase();
|
||||
if (ext.length() <= 5) {
|
||||
String mimeType = extToMime.get(ext);
|
||||
if (mimeType != null) {
|
||||
return mimeType;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return defaultMimeType;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
/*
|
||||
* Copyright 2025 Conductor Authors.
|
||||
* <p>
|
||||
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package org.conductoross.conductor.ai;
|
||||
|
||||
import okhttp3.OkHttpClient;
|
||||
|
||||
/**
|
||||
* Configuration for an {@link AIModel}. Implementations build the model via {@link #get()} and
|
||||
* receive the shared {@link OkHttpClient} (typically Spring's {@code conductorAiHttpClient}) via
|
||||
* {@link #setHttpClient(OkHttpClient)}. Providers that do not use OkHttp (e.g. Bedrock) may
|
||||
* implement {@link #setHttpClient(OkHttpClient)} as a no-op.
|
||||
*/
|
||||
public interface ModelConfiguration<T extends AIModel> {
|
||||
T get();
|
||||
|
||||
void setHttpClient(OkHttpClient httpClient);
|
||||
}
|
||||
@@ -0,0 +1,209 @@
|
||||
/*
|
||||
* Copyright 2026 Conductor Authors.
|
||||
* <p>
|
||||
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package org.conductoross.conductor.ai.a2a;
|
||||
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.security.MessageDigest;
|
||||
import java.util.Map;
|
||||
|
||||
import org.apache.commons.lang3.StringUtils;
|
||||
import org.conductoross.conductor.ai.a2a.model.A2ATask;
|
||||
import org.conductoross.conductor.ai.a2a.model.TaskState;
|
||||
import org.conductoross.conductor.ai.model.A2ACallRequest;
|
||||
import org.conductoross.conductor.config.AIIntegrationEnabledCondition;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.springframework.context.annotation.Conditional;
|
||||
import org.springframework.http.HttpStatus;
|
||||
import org.springframework.http.ResponseEntity;
|
||||
import org.springframework.web.bind.annotation.PathVariable;
|
||||
import org.springframework.web.bind.annotation.PostMapping;
|
||||
import org.springframework.web.bind.annotation.RequestBody;
|
||||
import org.springframework.web.bind.annotation.RequestHeader;
|
||||
import org.springframework.web.bind.annotation.RequestMapping;
|
||||
import org.springframework.web.bind.annotation.RestController;
|
||||
|
||||
import com.netflix.conductor.common.config.ObjectMapperProvider;
|
||||
import com.netflix.conductor.common.metadata.tasks.Task;
|
||||
import com.netflix.conductor.common.metadata.tasks.TaskResult;
|
||||
import com.netflix.conductor.service.TaskService;
|
||||
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
|
||||
/**
|
||||
* Receives A2A push notifications from remote agents and completes the corresponding {@code AGENT}
|
||||
* task that is waiting in push mode.
|
||||
*
|
||||
* <p>Token is read from the {@code Authorization: Bearer <token>} header (preferred), then the
|
||||
* {@code X-Conductor-A2A-Token} custom header. Comparison is constant-time. Tokens embed an expiry
|
||||
* timestamp ({@code {uuid}:{epochMillis}}) and are rejected once expired.
|
||||
*/
|
||||
@RestController
|
||||
@RequestMapping("/api/a2a")
|
||||
@Conditional(AIIntegrationEnabledCondition.class)
|
||||
public class A2ACallbackResource {
|
||||
|
||||
private static final Logger log = LoggerFactory.getLogger(A2ACallbackResource.class);
|
||||
|
||||
private final ObjectMapper objectMapper = new ObjectMapperProvider().getObjectMapper();
|
||||
private final TaskService taskService;
|
||||
private final A2AService a2aService;
|
||||
|
||||
public A2ACallbackResource(TaskService taskService, A2AService a2aService) {
|
||||
this.taskService = taskService;
|
||||
this.a2aService = a2aService;
|
||||
}
|
||||
|
||||
@PostMapping("/callback/{taskId}")
|
||||
public ResponseEntity<Void> onPushNotification(
|
||||
@PathVariable("taskId") String taskId,
|
||||
@RequestHeader(value = "Authorization", required = false) String authHeader,
|
||||
@RequestHeader(value = "X-Conductor-A2A-Token", required = false) String customHeader,
|
||||
@RequestBody(required = false) JsonNode payload) {
|
||||
|
||||
try (A2ALogging.Scope scope = A2ALogging.of(A2ALogging.TASK_ID, taskId)) {
|
||||
String token = resolveToken(authHeader, customHeader);
|
||||
|
||||
Task task = loadTask(taskId);
|
||||
if (task == null || !AgentTask.TASK_TYPE.equals(task.getTaskType())) {
|
||||
return ResponseEntity.notFound().build();
|
||||
}
|
||||
scope.add(A2ALogging.WORKFLOW_ID, task.getWorkflowInstanceId())
|
||||
.add(A2ALogging.REF, task.getReferenceTaskName());
|
||||
|
||||
Object storedToken =
|
||||
task.getOutputData() != null
|
||||
? task.getOutputData().get(A2AResults.KEY_PUSH_TOKEN)
|
||||
: null;
|
||||
if (!isValidToken(storedToken, token)) {
|
||||
log.warn("A2A push for task {} rejected: token mismatch or expired", taskId);
|
||||
return ResponseEntity.status(HttpStatus.FORBIDDEN).build();
|
||||
}
|
||||
|
||||
if (task.getStatus() != Task.Status.IN_PROGRESS) {
|
||||
// Already completed (or not waiting) — nothing to do; ack so the agent stops
|
||||
// retrying.
|
||||
return ResponseEntity.ok().build();
|
||||
}
|
||||
|
||||
A2ACallRequest request =
|
||||
objectMapper.convertValue(task.getInputData(), A2ACallRequest.class);
|
||||
String agentTaskId = asString(task.getOutputData().get(A2AResults.KEY_TASK_ID));
|
||||
if (StringUtils.isBlank(request.getAgentUrl()) || StringUtils.isBlank(agentTaskId)) {
|
||||
return ResponseEntity.ok().build();
|
||||
}
|
||||
|
||||
A2ATask agentTask;
|
||||
try {
|
||||
agentTask =
|
||||
a2aService.getTask(
|
||||
request.getAgentUrl(),
|
||||
agentTaskId,
|
||||
request.getHistoryLength(),
|
||||
request.getHeaders());
|
||||
} catch (Exception e) {
|
||||
log.warn(
|
||||
"A2A push for task {}: failed to fetch agent task {} from {}: {}",
|
||||
taskId,
|
||||
agentTaskId,
|
||||
request.getAgentUrl(),
|
||||
e.getMessage());
|
||||
return ResponseEntity.status(HttpStatus.BAD_GATEWAY).build();
|
||||
}
|
||||
|
||||
String state = agentTask.getStatus() != null ? agentTask.getStatus().getState() : null;
|
||||
if (!TaskState.isTerminal(state) && !TaskState.isInterrupted(state)) {
|
||||
// Not done yet — ack and wait for the next push.
|
||||
return ResponseEntity.ok().build();
|
||||
}
|
||||
|
||||
Map<String, Object> output = A2AResults.taskOutput(agentTask, objectMapper);
|
||||
taskService.updateTask(
|
||||
task.getWorkflowInstanceId(),
|
||||
task.getReferenceTaskName(),
|
||||
toResultStatus(state),
|
||||
"a2a-callback",
|
||||
output);
|
||||
log.debug("A2A push completed task {} (agent state {})", taskId, state);
|
||||
return ResponseEntity.ok().build();
|
||||
}
|
||||
}
|
||||
|
||||
/** Resolves the token: {@code Authorization: Bearer} header wins, then the custom header. */
|
||||
private String resolveToken(String authHeader, String customHeader) {
|
||||
if (authHeader != null && authHeader.startsWith("Bearer ")) {
|
||||
return authHeader.substring(7).trim();
|
||||
}
|
||||
if (customHeader != null && !customHeader.isBlank()) {
|
||||
return customHeader.trim();
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Validates the inbound token against the stored one using constant-time comparison and checks
|
||||
* the embedded expiry timestamp. Token format: {@code {uuid}:{expiryEpochMillis}}.
|
||||
*/
|
||||
private boolean isValidToken(Object stored, String inbound) {
|
||||
if (stored == null || inbound == null) {
|
||||
return false;
|
||||
}
|
||||
String storedStr = stored.toString();
|
||||
// Constant-time comparison — prevents timing oracle on the UUID portion.
|
||||
boolean match =
|
||||
MessageDigest.isEqual(
|
||||
storedStr.getBytes(StandardCharsets.UTF_8),
|
||||
inbound.getBytes(StandardCharsets.UTF_8));
|
||||
if (!match) {
|
||||
return false;
|
||||
}
|
||||
// Extract and validate the expiry timestamp embedded in the token.
|
||||
int sep = storedStr.lastIndexOf(':');
|
||||
if (sep > 0) {
|
||||
try {
|
||||
long expiryMs = Long.parseLong(storedStr.substring(sep + 1));
|
||||
if (System.currentTimeMillis() > expiryMs) {
|
||||
log.warn("A2A push token is expired (expiry was {})", expiryMs);
|
||||
return false;
|
||||
}
|
||||
} catch (NumberFormatException ignored) {
|
||||
// Token pre-dates the timestamped format — accept it as valid (no expiry check).
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
private Task loadTask(String taskId) {
|
||||
try {
|
||||
return taskService.getTask(taskId);
|
||||
} catch (Exception e) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private TaskResult.Status toResultStatus(String state) {
|
||||
switch (TaskState.normalize(state)) {
|
||||
case TaskState.FAILED:
|
||||
case TaskState.REJECTED:
|
||||
return TaskResult.Status.FAILED;
|
||||
default:
|
||||
// completed / canceled / input-required / auth-required
|
||||
return TaskResult.Status.COMPLETED;
|
||||
}
|
||||
}
|
||||
|
||||
private static String asString(Object value) {
|
||||
return value == null ? null : value.toString();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
/*
|
||||
* Copyright 2026 Conductor Authors.
|
||||
* <p>
|
||||
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package org.conductoross.conductor.ai.a2a;
|
||||
|
||||
/**
|
||||
* A retryable failure talking to a remote A2A agent (transport error, 5xx, or a transient JSON-RPC
|
||||
* error). Terminal/non-retryable failures (4xx, method-not-found, task-not-found, …) are surfaced
|
||||
* as {@link com.netflix.conductor.sdk.workflow.executor.task.NonRetryableException} instead, so
|
||||
* both the annotated-worker path and {@code AgentTask} map them to a terminal task status.
|
||||
*/
|
||||
public class A2AException extends RuntimeException {
|
||||
|
||||
public A2AException(String message) {
|
||||
super(message);
|
||||
}
|
||||
|
||||
public A2AException(String message, Throwable cause) {
|
||||
super(message, cause);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,75 @@
|
||||
/*
|
||||
* Copyright 2026 Conductor Authors.
|
||||
* <p>
|
||||
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package org.conductoross.conductor.ai.a2a;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
|
||||
import org.slf4j.MDC;
|
||||
|
||||
/**
|
||||
* MDC correlation keys for A2A code paths. A {@link Scope} sets a batch of keys and removes exactly
|
||||
* those it set on close, so log lines emitted while calling (or being called by) a remote agent can
|
||||
* be grepped by workflow, task, remote task, or context id — and no key leaks onto the next task to
|
||||
* run on a pooled worker thread.
|
||||
*
|
||||
* <p>Usage:
|
||||
*
|
||||
* <pre>{@code
|
||||
* try (A2ALogging.Scope scope = A2ALogging.of(A2ALogging.WORKFLOW_ID, wfId, A2ALogging.TASK_ID, id)) {
|
||||
* scope.add(A2ALogging.REMOTE_TASK_ID, agentTaskId); // once it's known
|
||||
* ...
|
||||
* }
|
||||
* }</pre>
|
||||
*/
|
||||
public final class A2ALogging {
|
||||
|
||||
public static final String WORKFLOW_ID = "a2aWorkflowId";
|
||||
public static final String TASK_ID = "a2aTaskId";
|
||||
public static final String REF = "a2aRef";
|
||||
public static final String REMOTE_TASK_ID = "a2aRemoteTaskId";
|
||||
public static final String CONTEXT_ID = "a2aContextId";
|
||||
public static final String MESSAGE_ID = "a2aMessageId";
|
||||
public static final String AGENT = "a2aAgent";
|
||||
public static final String METHOD = "a2aMethod";
|
||||
|
||||
private A2ALogging() {}
|
||||
|
||||
/** Open a scope pre-loaded with alternating key/value pairs (null keys/values are skipped). */
|
||||
public static Scope of(String... kv) {
|
||||
Scope scope = new Scope();
|
||||
for (int i = 0; i + 1 < kv.length; i += 2) {
|
||||
scope.add(kv[i], kv[i + 1]);
|
||||
}
|
||||
return scope;
|
||||
}
|
||||
|
||||
/** An auto-closeable set of MDC keys; {@link #close()} removes only the keys this scope set. */
|
||||
public static final class Scope implements AutoCloseable {
|
||||
private final List<String> keys = new ArrayList<>();
|
||||
|
||||
/** Set an MDC key (no-op if key or value is null); removed on {@link #close()}. */
|
||||
public Scope add(String key, String value) {
|
||||
if (key != null && value != null) {
|
||||
MDC.put(key, value);
|
||||
keys.add(key);
|
||||
}
|
||||
return this;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() {
|
||||
keys.forEach(MDC::remove);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,81 @@
|
||||
/*
|
||||
* Copyright 2026 Conductor Authors.
|
||||
* <p>
|
||||
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package org.conductoross.conductor.ai.a2a;
|
||||
|
||||
import com.netflix.conductor.metrics.Monitors;
|
||||
|
||||
/**
|
||||
* A2A metric emission, centralized so names and tags stay consistent and low-cardinality.
|
||||
*
|
||||
* <p>Counters are published through the shared {@link Monitors} registry (the same one the rest of
|
||||
* the engine uses), so they show up alongside core Conductor metrics in whatever backend is wired
|
||||
* in (Prometheus, Datadog, …). Tag values are always drawn from bounded sets — never agent URLs,
|
||||
* workflow ids, or message ids — to avoid cardinality blow-ups.
|
||||
*
|
||||
* <table>
|
||||
* <caption>Metrics</caption>
|
||||
* <tr><th>Name</th><th>Tags</th><th>Meaning</th></tr>
|
||||
* <tr><td>{@code a2a_client_calls}</td><td>{@code result}</td>
|
||||
* <td>An AGENT task reached a terminal status (result = the Conductor status).</td></tr>
|
||||
* <tr><td>{@code a2a_client_poll_failures}</td><td>—</td>
|
||||
* <td>A single transient {@code tasks/get} poll failed (before exhausting retries).</td></tr>
|
||||
* <tr><td>{@code a2a_rpc_errors}</td><td>{@code method}, {@code terminal}</td>
|
||||
* <td>A JSON-RPC/HTTP error from a remote agent, by method and whether it was fatal.</td></tr>
|
||||
* <tr><td>{@code a2a_ssrf_blocked}</td><td>—</td>
|
||||
* <td>An outbound agent URL was rejected by the SSRF guard.</td></tr>
|
||||
* <tr><td>{@code a2a_server_requests}</td><td>{@code method}</td>
|
||||
* <td>An inbound A2A JSON-RPC request to a workflow-backed agent, by method.</td></tr>
|
||||
* <tr><td>{@code a2a_server_resumes}</td><td>—</td>
|
||||
* <td>A follow-up message/send resumed a paused workflow (multi-turn).</td></tr>
|
||||
* </table>
|
||||
*/
|
||||
public final class A2AMetrics {
|
||||
|
||||
private A2AMetrics() {}
|
||||
|
||||
// ---- client (Conductor calling a remote agent) ------------------------------------------
|
||||
|
||||
/** An AGENT task settled into a terminal Conductor status (e.g. completed/failed). */
|
||||
public static void clientCall(String result) {
|
||||
Monitors.getCounter("a2a_client_calls", "result", result).increment();
|
||||
}
|
||||
|
||||
/** One transient poll failure during a {@code tasks/get} loop (retry not yet exhausted). */
|
||||
public static void clientPollFailure() {
|
||||
Monitors.getCounter("a2a_client_poll_failures").increment();
|
||||
}
|
||||
|
||||
/** A remote agent returned a JSON-RPC or HTTP error for the given method. */
|
||||
public static void rpcError(String method, boolean terminal) {
|
||||
Monitors.getCounter(
|
||||
"a2a_rpc_errors", "method", method, "terminal", String.valueOf(terminal))
|
||||
.increment();
|
||||
}
|
||||
|
||||
/** An outbound agent URL was rejected by the SSRF guard. */
|
||||
public static void ssrfBlocked() {
|
||||
Monitors.getCounter("a2a_ssrf_blocked").increment();
|
||||
}
|
||||
|
||||
// ---- server (Conductor exposed as an agent) ---------------------------------------------
|
||||
|
||||
/** An inbound A2A JSON-RPC request was dispatched for the given method. */
|
||||
public static void serverRequest(String method) {
|
||||
Monitors.getCounter("a2a_server_requests", "method", method).increment();
|
||||
}
|
||||
|
||||
/** A follow-up message/send resumed a paused (input-required) workflow. */
|
||||
public static void serverResume() {
|
||||
Monitors.getCounter("a2a_server_resumes").increment();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,124 @@
|
||||
/*
|
||||
* Copyright 2026 Conductor Authors.
|
||||
* <p>
|
||||
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||||
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations under the License.
|
||||
*/
|
||||
package org.conductoross.conductor.ai.a2a;
|
||||
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
import org.conductoross.conductor.ai.a2a.model.A2AMessage;
|
||||
import org.conductoross.conductor.ai.a2a.model.A2ATask;
|
||||
import org.conductoross.conductor.ai.a2a.model.Artifact;
|
||||
import org.conductoross.conductor.ai.a2a.model.Part;
|
||||
import org.conductoross.conductor.ai.a2a.model.TaskState;
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
|
||||
/**
|
||||
* Builds the task output that {@code AGENT} (polling/streaming) and the push-notification callback
|
||||
* both surface, so the shape is identical regardless of how the result arrived.
|
||||
*/
|
||||
public final class A2AResults {
|
||||
|
||||
public static final String KEY_STATE = "state";
|
||||
public static final String KEY_TASK_ID = "taskId";
|
||||
public static final String KEY_CONTEXT_ID = "contextId";
|
||||
public static final String KEY_ARTIFACTS = "artifacts";
|
||||
public static final String KEY_TEXT = "text";
|
||||
public static final String KEY_AGENT_MESSAGE = "agentMessage";
|
||||
public static final String KEY_TASK = "task";
|
||||
public static final String KEY_PUSH_TOKEN = "pushToken";
|
||||
|
||||
/** Bookkeeping (also surfaced for operator visibility): when the A2A call started. */
|
||||
public static final String KEY_STARTED_AT = "a2aStartedAt";
|
||||
|
||||
/** Bookkeeping: consecutive transient poll failures so far. */
|
||||
public static final String KEY_POLL_FAILURES = "a2aPollFailures";
|
||||
|
||||
private A2AResults() {}
|
||||
|
||||
/** Output for a remote {@link A2ATask}: normalized state, ids, artifacts, text, full task. */
|
||||
public static Map<String, Object> taskOutput(A2ATask task, ObjectMapper objectMapper) {
|
||||
Map<String, Object> output = new HashMap<>();
|
||||
output.put(KEY_STATE, TaskState.normalize(stateOf(task)));
|
||||
output.put(KEY_TASK_ID, task.getId());
|
||||
output.put(KEY_CONTEXT_ID, task.getContextId());
|
||||
if (task.getArtifacts() != null) {
|
||||
output.put(KEY_ARTIFACTS, objectMapper.convertValue(task.getArtifacts(), List.class));
|
||||
}
|
||||
output.put(KEY_TASK, objectMapper.convertValue(task, Map.class));
|
||||
String text = extractText(task);
|
||||
if (text != null) {
|
||||
output.put(KEY_TEXT, text);
|
||||
}
|
||||
A2AMessage statusMessage = task.getStatus() != null ? task.getStatus().getMessage() : null;
|
||||
if (statusMessage != null) {
|
||||
output.put(KEY_AGENT_MESSAGE, objectMapper.convertValue(statusMessage, Map.class));
|
||||
}
|
||||
return output;
|
||||
}
|
||||
|
||||
/** Output for a direct {@link A2AMessage} reply (no task was created). */
|
||||
public static Map<String, Object> messageOutput(A2AMessage message, ObjectMapper objectMapper) {
|
||||
Map<String, Object> output = new HashMap<>();
|
||||
output.put(KEY_STATE, "message");
|
||||
if (message != null) {
|
||||
output.put(KEY_CONTEXT_ID, message.getContextId());
|
||||
output.put(KEY_TASK_ID, message.getTaskId());
|
||||
output.put(KEY_AGENT_MESSAGE, objectMapper.convertValue(message, Map.class));
|
||||
String text = partsText(message.getParts());
|
||||
if (text != null) {
|
||||
output.put(KEY_TEXT, text);
|
||||
}
|
||||
}
|
||||
return output;
|
||||
}
|
||||
|
||||
/** Concatenates the text of a task's artifact parts, falling back to its status message. */
|
||||
public static String extractText(A2ATask task) {
|
||||
StringBuilder sb = new StringBuilder();
|
||||
if (task.getArtifacts() != null) {
|
||||
for (Artifact artifact : task.getArtifacts()) {
|
||||
appendParts(sb, artifact.getParts());
|
||||
}
|
||||
}
|
||||
if (sb.length() == 0 && task.getStatus() != null && task.getStatus().getMessage() != null) {
|
||||
appendParts(sb, task.getStatus().getMessage().getParts());
|
||||
}
|
||||
return sb.length() == 0 ? null : sb.toString();
|
||||
}
|
||||
|
||||
static String partsText(List<Part> parts) {
|
||||
StringBuilder sb = new StringBuilder();
|
||||
appendParts(sb, parts);
|
||||
return sb.length() == 0 ? null : sb.toString();
|
||||
}
|
||||
|
||||
private static void appendParts(StringBuilder sb, List<Part> parts) {
|
||||
if (parts == null) {
|
||||
return;
|
||||
}
|
||||
for (Part part : parts) {
|
||||
if (part != null && part.getText() != null) {
|
||||
if (sb.length() > 0) {
|
||||
sb.append("\n");
|
||||
}
|
||||
sb.append(part.getText());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static String stateOf(A2ATask task) {
|
||||
return task.getStatus() != null ? task.getStatus().getState() : null;
|
||||
}
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user