wehub-resource-sync
e071084ebe
govulncheck / govulncheck (push) Has been cancelled
Lint / golangci-lint (push) Has been cancelled
Run Tests / Unit Tests (push) Has been cancelled
Run Tests / Etcd Integration Tests (push) Has been cancelled
Harness (E2E) / Harnesses (mock LLM) (push) Has been cancelled
Harness (E2E) / Provider harnesses (live LLM conformance) (push) Has been cancelled
206 lines
10 KiB
Markdown
206 lines
10 KiB
Markdown
---
|
|
layout: blog
|
|
title: "From Chat to Flows: What If Your Services Could Orchestrate Themselves?"
|
|
permalink: /blog/9
|
|
description: "Exploring the concept of LLM-powered service orchestration — what happens when micro chat becomes a persistent, event-driven flow engine."
|
|
---
|
|
|
|
# From Chat to Flows: What If Your Services Could Orchestrate Themselves?
|
|
|
|
*May 29, 2026 • By the Go Micro Team*
|
|
|
|
<img src="/images/generated/blog-agents-demo.jpg" alt="Service orchestration concept" style="width: 100%; border-radius: 8px; margin: 1rem 0 1.5rem;" />
|
|
|
|
We shipped `micro chat` recently — an interactive terminal where you talk to your services through an LLM. You say "list all users over 30 and send each a welcome email," and the model figures out which services to call, in what order, with what arguments.
|
|
|
|
It works. But it's interactive and ephemeral. You type a prompt, get a result, move on. What if you could save that prompt as a **flow** and trigger it from an event?
|
|
|
|
This post explores that idea. We're not shipping anything yet — we're thinking out loud about where the intersection of microservices, MCP, and LLMs could go.
|
|
|
|
## What We Have Today
|
|
|
|
Go Micro already has the building blocks:
|
|
|
|
- **Services as tools**: every endpoint is discoverable via MCP with typed schemas
|
|
- **`ai.Tools`**: programmatic discovery and execution — `ai.DiscoverTools(reg)` gives you the tool list, `ai.NewTools(reg).Handler()` executes RPCs
|
|
- **`ai.History`**: multi-turn conversation state so the LLM has context across steps
|
|
- **`micro chat`**: the interactive agent loop that ties it together
|
|
- **Broker/events**: pub/sub for async communication between services
|
|
|
|
The gap is between "I can interactively ask an LLM to orchestrate my services" and "I can define a persistent, event-driven workflow that does it automatically."
|
|
|
|
## The Flow Concept
|
|
|
|
Imagine you could define a flow like this:
|
|
|
|
```yaml
|
|
name: onboard-user
|
|
trigger: events.user.created
|
|
prompt: |
|
|
A new user was just created: {{.event.data}}.
|
|
1. Send them a welcome email via the email service
|
|
2. Create a default workspace via the workspace service
|
|
3. If they're on an enterprise plan, assign them to the enterprise onboarding queue
|
|
4. Log everything via the audit service
|
|
```
|
|
|
|
When a `user.created` event fires, the flow engine:
|
|
1. Discovers available services from the registry
|
|
2. Builds the tool list from their endpoints
|
|
3. Feeds the event data + prompt to an LLM
|
|
4. The LLM decides which tools to call, in what order, with what arguments
|
|
5. The engine executes the RPCs, feeds results back, and lets the LLM continue until done
|
|
6. The flow result is logged/stored for observability
|
|
|
|
This is essentially `micro chat` but triggered by an event instead of a human, with the prompt pre-defined instead of typed interactively.
|
|
|
|
## How It Compares to Traditional Orchestration
|
|
|
|
| | Step Functions / Temporal | LLM Flow |
|
|
|---|---|---|
|
|
| **Definition** | State machine in JSON/YAML | Natural language prompt |
|
|
| **Branching** | Explicit if/else states | LLM decides based on context |
|
|
| **Error handling** | Retry policies, catch blocks | "If this fails, try X instead" |
|
|
| **New service** | Update the state machine | LLM discovers it automatically |
|
|
| **Determinism** | 100% reproducible | Probabilistic (same prompt, slightly different execution) |
|
|
| **Cost** | Compute only | Compute + LLM tokens per flow |
|
|
| **Debugging** | Step-by-step state trace | Conversation log |
|
|
|
|
The tradeoff is clear: **traditional orchestration is deterministic but rigid. LLM flows are flexible but probabilistic.**
|
|
|
|
For a payment processing pipeline, you want Step Functions. For "onboard this user and do whatever makes sense based on their plan," an LLM flow could be genuinely better — it adapts to new services without code changes.
|
|
|
|
## What This Would Look Like in Go Micro
|
|
|
|
The simplest version is just `micro chat` with a saved prompt and a trigger:
|
|
|
|
```go
|
|
flow := ai.NewFlow("onboard-user",
|
|
ai.WithTrigger("events.user.created"),
|
|
ai.WithPrompt(`A new user was created: {{.Data}}.
|
|
Send welcome email, create workspace,
|
|
assign to enterprise queue if enterprise plan.`),
|
|
ai.WithProvider("atlascloud"),
|
|
ai.WithAPIKey(key),
|
|
)
|
|
|
|
// Register with the service
|
|
service := micro.NewService("flow-runner")
|
|
flow.Register(service)
|
|
service.Run()
|
|
```
|
|
|
|
Under the hood, `Flow` would:
|
|
1. Subscribe to the broker topic
|
|
2. On each event, create an `ai.History` with the prompt + event data
|
|
3. Call `m.Generate()` with history messages until the LLM stops requesting tool calls
|
|
4. Log the full conversation for audit
|
|
|
|
The building blocks already exist. `ai.History` manages the conversation. `ai.Tools` discovers and executes services. The broker delivers events. A `Flow` just connects them.
|
|
|
|
## Why We Haven't Built It Yet
|
|
|
|
Three honest reasons:
|
|
|
|
**1. Non-determinism is dangerous for workflows.** If you run the same flow twice with the same input, the LLM might call tools in a different order or skip a step. For many workflows, that's a bug, not a feature. We'd need guardrails: required steps, validation, output schemas.
|
|
|
|
**2. Cost unpredictability.** Each flow execution costs LLM tokens. A complex flow with 5 tool calls might cost $0.01. At 10,000 events per day, that's $100/day just for orchestration. Traditional orchestrators cost effectively nothing per execution.
|
|
|
|
**3. Scope creep.** Go Micro is a microservices framework, not a workflow engine. Adding persistent flow state, retry logic, dead letter queues, and flow versioning is a big commitment. Temporal exists. Step Functions exist. We should be honest about where the framework ends and the platform begins.
|
|
|
|
## Where It Makes Sense
|
|
|
|
Despite those caveats, there are use cases where this is genuinely better than traditional orchestration:
|
|
|
|
- **Ops automation**: "When a service health check fails, investigate by checking logs, recent deployments, and related services, then post a summary to Slack." This is inherently fuzzy — you don't know which tools you'll need until you see what's wrong.
|
|
|
|
- **Customer support flows**: "A customer filed a ticket about a billing issue. Look up their account, check recent invoices, and draft a response." The flow adapts to what it finds.
|
|
|
|
- **Data pipeline glue**: "New CSV uploaded. Parse it, validate the schema, create records in the appropriate service, and report any errors." The LLM handles schema variations that would break a rigid pipeline.
|
|
|
|
- **Development workflows**: "Run the test suite, analyze failures, check if they're flaky, and create issues for real failures." This is `micro chat` for CI.
|
|
|
|
## What You Can Do Today
|
|
|
|
You don't need a flow engine to get most of this value. The `ai.Tools` package already gives you programmatic access:
|
|
|
|
```go
|
|
tools := ai.NewTools(service.Registry())
|
|
discovered, _ := tools.Discover()
|
|
|
|
m := ai.New("atlascloud",
|
|
ai.WithAPIKey(key),
|
|
ai.WithTools(tools),
|
|
)
|
|
|
|
hist := ai.NewHistory(50)
|
|
|
|
// React to an event
|
|
broker.Subscribe("user.created", func(e broker.Event) error {
|
|
prompt := fmt.Sprintf("New user created: %s. Send welcome email and create workspace.", string(e.Message().Body))
|
|
resp, _ := m.Generate(ctx, &ai.Request{Prompt: prompt, SystemPrompt: "You are a service orchestrator.", Tools: discovered, Messages: hist.Messages()})
|
|
log.Infof("Flow result: %s", resp.Answer)
|
|
hist.Reset() // fresh history for next event
|
|
return nil
|
|
})
|
|
```
|
|
|
|
That's ~15 lines. No flow engine, no YAML, no new abstractions. Just a broker subscription that feeds events into the LLM with your services as tools.
|
|
|
|
## Update: We Built It
|
|
|
|
After publishing this post, we went ahead and built the `ai/flow` package. It wraps the pattern above into a reusable primitive:
|
|
|
|
```go
|
|
import "go-micro.dev/v5/ai/flow"
|
|
|
|
f := flow.New("onboard-user",
|
|
flow.Trigger("events.user.created"),
|
|
flow.Prompt("New user: {{.Data}}. Send welcome email and create workspace."),
|
|
flow.Provider("anthropic"),
|
|
flow.APIKey(key),
|
|
)
|
|
|
|
f.Register(service.Registry(), service.Options().Broker, service.Client())
|
|
service.Run() // flow listens and reacts to events
|
|
```
|
|
|
|
Under the hood, each event triggers a fresh `ai.History` + `tools.Discover` + `model.Generate` cycle. The flow records every execution with timing, tool calls, and errors.
|
|
|
|
There's also a CLI:
|
|
|
|
```bash
|
|
# Subscribe to events and react
|
|
micro flow run --trigger events.user.created \
|
|
--prompt "New user: {{.Data}}. Send welcome email." \
|
|
--provider anthropic
|
|
|
|
# One-shot execution
|
|
micro flow exec --prompt "List all users and count them" \
|
|
--provider anthropic
|
|
```
|
|
|
|
The output is JSON — flow name, prompt, tool calls made, reply, answer, duration. Pipe it to `jq` or log it.
|
|
|
|
We kept it deliberately minimal: no persistent state, no retry policies, no YAML config. Each event gets a fresh history, the LLM decides what to do, and the result is recorded. If you need Temporal-grade durability, use Temporal. But for "when X happens, let the LLM figure out what services to call" — this is enough.
|
|
|
|
The questions from the original post still stand. We'd love feedback on what guardrails people need and whether this should grow or stay small.
|
|
|
|
## The Bigger Picture
|
|
|
|
The thesis behind Go Micro's AI-native direction is that **services should be composable by agents, not just by code.** MCP made services discoverable. `ai.Tools` made them callable. `micro chat` made them interactive. Flows would make them orchestratable.
|
|
|
|
Each layer builds on the previous one. And at each layer, the question is the same: does this belong in the framework, or is it better left to the user? So far, we've been conservative — `ai.Tools` is 150 lines, `History` is 80, `micro chat` is 170. Small, composable building blocks rather than a big orchestration framework.
|
|
|
|
We think that's the right approach. But we're watching to see if the community says otherwise.
|
|
|
|
---
|
|
|
|
*Go Micro is an open source framework for distributed systems development. [Star us on GitHub](https://github.com/micro/go-micro).*
|
|
|
|
<div class="post-nav">
|
|
<div><a href="/blog/8">← Atlas Cloud Sponsors Go Micro</a></div>
|
|
<div><a href="/blog/">All Posts</a></div>
|
|
<div><a href="/blog/10">micro chat →</a></div>
|
|
</div>
|