wehub-resource-sync
a9cd7750f4
CI / unit-test (push) Has been cancelled
CI / detect-changes (push) Has been cancelled
CI / build (push) Has been cancelled
Publish docs via GitHub Pages / Deploy docs (push) Has been cancelled
CI / test-harness (push) Has been cancelled
CI / generate-e2e-matrix (push) Has been cancelled
CI / e2e (push) Has been cancelled
CI / build-ui (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
UI v2 Integration CI / E2E (Integration) (push) Has been cancelled
UI v2 CI / Lint, Format & Test (push) Has been cancelled
UI v2 CI / E2E (Mocked) (push) Has been cancelled
89 lines
3.9 KiB
Markdown
89 lines
3.9 KiB
Markdown
|
|
# Conductor Docker Builds
|
|
|
|
## Pre-built docker images
|
|
|
|
Conductor server with support for the following backend:
|
|
1. Redis
|
|
2. Postgres
|
|
3. Mysql
|
|
4. Cassandra
|
|
|
|
### Docker File for Server and UI
|
|
|
|
[Docker Image Source for Server with UI](server/Dockerfile)
|
|
|
|
### Configuration Guide for Conductor Server
|
|
Conductor uses a persistent store for managing state.
|
|
The choice of backend is quite flexible and can be configured at runtime using `conductor.db.type` property.
|
|
|
|
Refer to the table below for various supported backend and required configurations to enable each of them.
|
|
|
|
> [!IMPORTANT]
|
|
>
|
|
> See [config.properties](docker/server/config/config.properties) for the required properties for each of the backends.
|
|
>
|
|
> | Backend | Property |
|
|
> |------------|------------------------------------|
|
|
> | postgres | conductor.db.type=postgres |
|
|
> | redis | conductor.db.type=redis_standalone |
|
|
> | mysql | conductor.db.type=mysql |
|
|
> | cassandra | conductor.db.type=cassandra |
|
|
>
|
|
|
|
Conductor is using Elasticsearch or OpenSearch for indexing the workflow data.
|
|
Currently, Elasticsearch 7 and OpenSearch 2.x/3.x are supported.
|
|
|
|
We welcome community contributions for other indexing backends.
|
|
|
|
**Note:** Docker images use Elasticsearch 7 by default. Elasticsearch 6 and OpenSearch 1.x are deprecated.
|
|
|
|
## Helm Charts
|
|
TODO: Link to the helm charts
|
|
|
|
## Run Docker Compose Locally
|
|
### Use the docker-compose to bring up the local conductor server.
|
|
|
|
| Docker Compose | Description |
|
|
|--------------------------------------------------------------|----------------------------|
|
|
| [docker-compose.yaml](docker-compose.yaml) | Redis + Elasticsearch 7 |
|
|
| [docker-compose-postgres.yaml](docker-compose-postgres.yaml) | Postgres + Elasticsearch 7 |
|
|
| [docker-compose-mysql.yaml](docker-compose-mysql.yaml) | Mysql + Elasticsearch 7 |
|
|
| [docker-compose-redis-os.yaml](docker-compose-redis-os.yaml) | Redis + OpenSearch 2.x (legacy - use os2) |
|
|
| [docker-compose-redis-os2.yaml](docker-compose-redis-os2.yaml) | Redis + OpenSearch 2.x |
|
|
| [docker-compose-redis-os3.yaml](docker-compose-redis-os3.yaml) | Redis + OpenSearch 3.x |
|
|
|
|
### Network errors during UI build with yarn
|
|
|
|
It has been observed, that the UI build may fail with an error message like
|
|
|
|
```
|
|
> [linux/arm64 ui-builder 5/7] RUN yarn install && cp -r node_modules/monaco-editor public/ && yarn build:
|
|
269.9 at Object.onceWrapper (node:events:633:28)
|
|
269.9 at TLSSocket.emit (node:events:531:35)
|
|
269.9 at Socket._onTimeout (node:net:590:8)
|
|
269.9 at listOnTimeout (node:internal/timers:573:17)
|
|
269.9 at process.processTimers (node:internal/timers:514:7)
|
|
269.9 info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
|
|
281.2 info There appears to be trouble with your network connection. Retrying...
|
|
313.5 info There appears to be trouble with your network connection. Retrying...
|
|
920.3 info There appears to be trouble with your network connection. Retrying...
|
|
953.6 info There appears to be trouble with your network connection. Retrying...
|
|
```
|
|
|
|
This does not necessarily mean, that the network is unavailable, but can be caused by too high latency, as well. `yarn` accepts the option `--network-timeout <#ms>` to set a custom timeout in milliseconds.
|
|
|
|
For passing arguments to `yarn`, in [this Dockerfile](server/Dockerfile) the _optional_ build arg `YARN_OPTS` has been added. This argument will be added to each `yarn` call.
|
|
|
|
When using one of the `docker-compose-*` files, you can set this via the environment variable `YARN_OPTS`, e.g.:
|
|
|
|
```
|
|
YARN_OPTS='--network-timeout 10000000' docker compose -f docker-compose.yaml up
|
|
```
|
|
|
|
When building a Docker image using `docker`, you must call it like e.g.
|
|
|
|
```
|
|
docker build --build-arg='YARN_OPTS=--network-timeout 10000000' .. -f server/Dockerfile -t oss-conductor:v3.21.9
|
|
```
|