Conductor: Durable Execution for Workflows and Agents

What Conductor is

Conductor is a workflow orchestration server written in Java. The README headline is "Durable Execution for Workflows and Agents", and the project describes itself as "an open-source durable execution platform for microservices, AI agents, and adaptive workflow graphs." Its job is to take the decisions a distributed process makes at runtime (loops, branching, fan-out, tool calls, approvals, retries and cancellation) and turn them into execution that is persisted and can be inspected after the fact.

On history, the README says the project originated at Netflix and is actively maintained by Orkes and the community. Its FAQ answers "Is this the same as Netflix Conductor?" with a yes: Conductor OSS is the continuation of the original Netflix Conductor repository after Netflix contributed the project to the open-source foundation. The documentation FAQ adds that the original Netflix repository has transitioned to Conductor OSS, which is now the project's home, and that active development and maintenance continue there.

The problem it targets is familiar to anyone who has stitched services together. The "Why Conductor" page puts it plainly: every distributed process has to survive failure, and without coordination each service carries its own retry, timeout and recovery logic, "duplicated everywhere and owned by no one." The docs contrast two approaches. In choreography, services react to each other's events and the overall flow exists only as an implied chain of event contracts. In orchestration, Conductor's approach, the business process is defined in one place while the work stays distributed. Conductor owns the flow, the state and the recovery, and workers stay stateless.

The audience is developers. The FAQ is explicit that Conductor is not a low-code or no-code platform: workflows can be defined in JSON, but the value comes from workers and tasks written in your own language. Typical users are teams orchestrating microservices, running long-lived business processes that span days or weeks, building event-driven automation, and, increasingly, running LLM agents that need durable state, tool calls and human approval steps.

How it works

Server, queues and workers

The architecture overview describes a worker-task queue model. Each task type (HTTP, Event, Wait, or a custom task) gets its own dedicated queue. A state machine evaluator schedules tasks onto those queues, hands them to workers when they poll, and tracks whether each task completes, retries or fails. Task workers poll the server over HTTP or gRPC, execute the task, and report the status back. Data stores hold workflow and task metadata, the queues and execution history, and REST APIs give programmatic access. By default Conductor uses Redis as its data store with Elasticsearch for indexing, and the docs describe these storage layers as pluggable.

Communication is RPC-style: workers run on separate machines and talk to the server over HTTP endpoints. By default, workers poll every 100ms, and the interval can be tuned per worker type. When an application starts a workflow it receives a workflow ID, which it can use at any point to check progress, pause or terminate the execution. Conductor does not run your workers; it only runs its own system tasks inside the server JVM.

The production deployment guide breaks the server into named components: an API server exposing REST and gRPC, the decider (the core state machine that schedules the next set of tasks), a sweeper background process that re-evaluates running workflows and is required for long-running workflows to make progress, system task workers, an event processor that listens to event buses, and four backing services: database, queue, index and a distributed lock.

Durable execution semantics

The durable execution page spells out what is persisted for every execution: a snapshot of the workflow definition (immutable after start), the workflow state (status, input, output, correlation ID and variables), every task execution (status, input, output, timestamps, retry count and worker ID) and the queue state. All of it is written to the configured store before the next step proceeds, so a restarted server resumes from the last persisted state.

Task delivery is at-least-once. If a worker polls a task and never responds, a response timeout triggers redelivery. The docs include a failure matrix covering worker crashes before and after side effects, explicit failures, terminal errors, server restarts (the sweeper picks up in-progress workflows), long waits across deploys, network partitions, and definition changes while executions are running. Because each execution is pinned to the definition snapshot it started with, updating or even deleting a workflow version does not affect in-flight runs.

Tasks move through a small state machine: SCHEDULED, IN_PROGRESS, then COMPLETED, FAILED (retried until exhausted), FAILED_WITH_TERMINAL_ERROR, TIMED_OUT or CANCELED. Each transition is persisted before any subsequent action. In multi-node deployments, distributed locking (Zookeeper or Redis) ensures only one decide evaluation runs per workflow at a time, with fencing tokens to reject stale updates.

Orchestration as data

Workflows are JSON definitions, versioned by number. The README frames this as a deliberate design choice: "Code-first engines force your code to be deterministic so the framework can replay it. Conductor makes the engine deterministic — so your code doesn't have to be." Because the graph is data, the docs note that LLMs and services can generate or modify definitions at runtime, which Conductor recommends validating before they are started.

Key features

• Persisted, recoverable executions. Every step is stored, and failed tasks are retried according to retryCount, retryDelaySeconds and retryLogic (FIXED, EXPONENTIAL_BACKOFF or LINEAR_BACKOFF), with separate timeouts for overall duration, worker response and polling.
• Restart, rerun and retry. Restart re-executes a workflow from the beginning (optionally on the latest definition), rerun resumes from a specific task while reusing earlier outputs, and retry re-attempts the last failed task. All three work on workflows in any terminal state.
• Built-in system tasks. HTTP, INLINE (JavaScript or GraalVM Python), EVENT, WAIT, HUMAN, KAFKA_PUBLISH, JSON_JQ_TRANSFORM, NOOP, JDBC and PULL_WORKFLOW_MESSAGES run on the server with no external worker.
• Flow control operators. FORK_JOIN, FORK_JOIN_DYNAMIC, JOIN, EXCLUSIVE_JOIN, SWITCH, DO_WHILE, SUB_WORKFLOW, START_WORKFLOW, SET_VARIABLE, TERMINATE and DYNAMIC express branching, loops, parallelism and composition, and they can be nested.
• Native AI tasks. The ai module adds LLM_CHAT_COMPLETE and LLM_TEXT_COMPLETE, six embedding and vector-search task types, media and PDF generation, and A2A agent tasks. The LLM orchestration guide lists providers including Anthropic, OpenAI, Azure OpenAI, Google Gemini, AWS Bedrock, Mistral, Cohere, HuggingFace and Ollama.
• MCP tool calling. LIST_MCP_TOOLS discovers the tools an MCP server exposes and CALL_MCP_TOOL invokes one, each as an ordinary task with retries, timeouts and a persisted record of the method, arguments and response.
• Vector workflows for RAG. Pinecone, pgvector and MongoDB Atlas Vector Search are listed for storing embeddings, indexing text and semantic search, so indexing, retrieval and generation can live in one workflow.
• Human-in-the-loop and long waits. HUMAN and WAIT tasks stay IN_PROGRESS in storage until a signal, API call or timer resolves them, which the docs say survives restarts and deploys.
• Event-driven integration. Workflows can be triggered by, and publish to, Kafka, NATS, NATS Streaming, AMQP (RabbitMQ), SQS or Conductor's internal queue.
• Polyglot workers. SDKs are listed for Java, Python, JavaScript, Go and C#, with Ruby and Rust marked as incubating. Any language that can poll and update tasks over HTTP can host a worker.
• Operational controls. Task domains, rateLimitPerFrequency, concurrentExecLimit, JSON Schema validation of inputs and task I/O, a built-in scheduler using Spring cron expressions, and Prometheus-compatible metrics.
• Built-in UI. The server ships with the ui-next UI for designing, running and debugging workflows and inspecting the input, output, timing and retry history of each task.

Getting started

Run a local server with the CLI

The README's quickest path uses the Conductor CLI. It lists Node.js v16+ and Java 25+ as prerequisites:

npm install -g @conductor-oss/conductor-cli
conductor server start

The server then listens on http://localhost:8080 with the built-in UI. To register and run the sample workflow, which calls an API and parses the response without any workers:

curl -s https://raw.githubusercontent.com/conductor-oss/conductor/main/docs/quickstart/workflow.json -o workflow.json
conductor workflow create workflow.json
conductor workflow start -w hello_workflow --sync

Running create twice returns an error because the workflow already exists; the README points to conductor workflow update for changes. To upgrade the server the CLI manages:

conductor server stop
conductor server update
conductor server start

The "Connect to Conductor" page shows how to point the CLI and SDKs at the local server, and how to pass an LLM provider key by exporting it before the server starts:

export CONDUCTOR_SERVER_URL=http://localhost:8080/api
conductor workflow list
export OPENAI_API_KEY=<your-openai-api-key>

Docker

The README gives a Docker command that includes the ui-next UI, with the UI on port 5000 and the API on 8080:

docker run -p 5000:5000 -p 8080:8080 conductoross/conductor:next

The production deployment guide describes a standalone image that bundles the server, the UI and SQLite-backed persistence with no external dependencies, serving the UI, Swagger docs and API on port 8080. It recommends pinning a release tag rather than latest in production:

docker run -p 8080:8080 conductoross/conductor:latest

For production-style backends, the repository ships Compose files. The default pairs the server with Redis for database and queue and Elasticsearch 8 for indexing:

git clone https://github.com/conductor-oss/conductor
cd conductor
docker compose -f docker/docker-compose.yaml up

Other files cover PostgreSQL-only, MySQL, Cassandra and OpenSearch combinations:

# Example: PostgreSQL for everything
docker compose -f docker/docker-compose-postgres.yaml up

A first MCP task

Tasks are JSON objects inside a workflow definition. This one, from the MCP guide, asks an MCP server for its tool list; the output can be passed straight to an LLM task:

{
  "name": "discover_tools",
  "taskReferenceName": "discover",
  "type": "LIST_MCP_TOOLS",
  "inputParameters": {
    "mcpServer": "${workflow.input.mcpServerUrl}"
  }
}

AI task types are only registered when conductor.integrations.ai.enabled=true is set and the relevant provider, vector database or MCP server is configured.

Production configuration

Configuration is done through Spring Boot properties or environment variables. The deployment guide's "simplest" production stack uses PostgreSQL for database, queue and indexing, with Redis only for locking. A fragment of it:

conductor.db.type=postgres
conductor.queue.type=postgres
conductor.indexing.enabled=true
conductor.indexing.type=postgres
conductor.workflow-execution-lock.type=redis
conductor.app.workflowExecutionLockEnabled=true

Build from source

Building requires Docker Desktop, JDK 25+, and Node.js 18+ with pnpm for the UI:

git clone https://github.com/conductor-oss/conductor
cd conductor
./gradlew build

The from-source guide notes that the default configuration of a source build uses in-memory persistence with no indexing, so all data is lost when the server stops.

Use cases

• Microservice orchestration. The docs' own example is order processing: payment, inventory, shipping and notification as separate services coordinated by one workflow, with retries and timeouts defined centrally rather than in each service.
• Long-running business processes. Insurance claims, loan approvals and onboarding flows that span days or weeks, using WAIT and HUMAN tasks that persist across deploys.
• Saga-style distributed transactions. A failureWorkflow runs compensation logic with the failure reason and workflow ID when the main workflow fails.
• Batch fan-out. FORK_JOIN_DYNAMIC creates a variable number of parallel branches at runtime, which the docs describe for spreading work across many workers.
• Event-driven automation. React to Kafka or other broker events, start workflows and publish results back.
• Durable AI agents. The README's example agent discovers tools via MCP, reasons with an LLM inside a bounded DO_WHILE loop, routes with SWITCH and calls the chosen tool, with every iteration persisted. The FAQ notes that an iteration can wait weeks for human review and then continue.
• RAG pipelines. The LLM orchestration guide shows a three-task workflow (LLM_INDEX_TEXT, LLM_SEARCH_INDEX, LLM_CHAT_COMPLETE) where the vector database, embedding model and LLM provider are just parameters.
• Exposing workflows to other agents. The MCP guide describes publishing workflows as MCP tools through an MCP Gateway, so other agents can discover and invoke them, with workflow inputs and outputs becoming the tool schema.

How it compares

The documentation does not publish feature comparisons against named products, so any comparison here is about approach. The clearest distinction the README draws is with code-first durable execution engines, which replay workflow code and therefore require that code to be deterministic. Conductor keeps orchestration as a JSON graph that the engine evaluates, and workers are ordinary code with no determinism constraint. The trade-off is that the flow lives in a definition rather than in your programming language; the FAQ argues this makes complexity visible and lets LLMs generate or change graphs without a compile and deploy cycle, while acknowledging the question "Isn't writing workflows as code more powerful than JSON?" comes up often enough to answer directly.

Against event choreography, the "Why Conductor" page argues orchestration gives a single, visible definition of the process and avoids correlating logs across services to debug a failure. Against agent frameworks, Conductor positions itself as a durable execution layer rather than a replacement: the quickstart offers a path for running agents built with OpenAI Agents, LangGraph, LangChain or Google ADK "unchanged" while Conductor owns their durable execution.

The other comparison readers will want is with Orkes Conductor, the commercial product from the project's primary maintainer. The docs say Orkes Conductor is built on top of Conductor OSS with full API and workflow compatibility, and that Orkes adds further features in its hosted offering. The repository's CHANGELOG also records several features "ported from Enterprise" into the open-source server, such as the GraalJS script engine and script timeout protection.

Things to know before adopting

• Orkes and the open-source project are closely linked. Orkes is described as the primary maintainer, the community Slack is run by Orkes, and the "Connect to Conductor" page labels the free Orkes Developer Edition as the recommended way to connect, with a local server as the alternative. The hosted-solutions page says the Developer Edition comes with Orkes' enterprise features, including a visual workflow editor and AI orchestration suite. Check which features you rely on are part of the Apache-licensed server before planning a self-hosted deployment.
• A real deployment has several moving parts. Production needs a database, a queue, an indexing backend and a distributed lock. Supported databases are PostgreSQL (recommended for production), MySQL (which needs Redis as the queue), Redis and Cassandra. SQLite is the default and is marked local development only. Indexing options are PostgreSQL, Elasticsearch 8 (the default), Elasticsearch 7 (legacy), OpenSearch 2 and 3; disabling indexing breaks search in the UI.
• Locking is mandatory with more than one server. The deployment guide marks distributed locking as required in production and warns that the local_only lock is not safe for multiple instances. The FAQ traces duplicate tasks both showing "attempt 0" to exactly this misconfiguration.
• Workers must be idempotent. At-least-once delivery means a task can run more than once, for example when a worker crashes after a side effect but before reporting completion. The docs are explicit that side-effecting workers and tools must handle redelivery.
• Script execution defaults deserve attention. The app configuration reference says conductor.app.graaljs.restricted-host-access currently defaults to false, leaving unrestricted Java host access enabled for compatibility. It warns this mode can allow scripts to execute arbitrary host code and should only be used when all script and definition authors are trusted, and that Conductor 3.35 will disable it by default. Setting it to true opts in now.
• Authentication is not covered in the docs reviewed. The open-source deployment guide and configuration reference describe backends, locking and metrics but not API authentication or access control; the access-key flow on the connect page targets Orkes Developer Edition. Plan network-level protection for a self-hosted server and verify the current options before exposing it.
• Legacy Elasticsearch carries a known CVE. The docs keep Elasticsearch 7 only as an opt-in build, noting it is end-of-life and carries the unpatchable CVE-2025-37731, and that the ES8 client cannot talk to a 7.17 cluster.
• Payload limits apply. Workflow and task inputs and outputs above configurable thresholds move to external payload storage (S3, PostgreSQL or Azure Blob), and hard limits fail the task or workflow. Workflow variables are capped at 256 KB by default.
• Java requirements are moving. The README and from-source guide list Java 25+, and the unreleased changelog section records a breaking switch from the Nashorn JavaScript engine to GraalJS along with a separate breaking API change to the scheduler endpoint. Pin image tags and read the changelog before upgrading.
• AI features are opt-in and still evolving. AI tasks require conductor.integrations.ai.enabled=true plus provider configuration, and the roadmap lists items such as a For..Each operator, Try..Catch operator and HTTP task polling as planned. Ruby and Rust SDKs are marked incubating.
• Supported versions. The security policy lists 3.x as supported and 2.x and 1.x as unsupported. Vulnerabilities are reported through GitHub security advisories.

Project activity

As of November 2026 the repository has about 32,500 stars and around 1,060 forks on GitHub. The conductor-oss/conductor repository was created in December 2023 as the continuation of Netflix Conductor, is written primarily in Java, and develops on the main branch. The roadmap file was last reviewed in April 2026 and tracks LLM tasks, Python scripting for INLINE, a database task, synchronous workflow execution and JSON Schema validation as done.

The code is licensed under Apache-2.0. Documentation lives at docs.conductor-oss.org, and the source, issues and releases are at github.com/conductor-oss/conductor. SDKs are maintained in separate repositories under the conductor-oss organisation, and the README points to a community Slack and forum for support.