# Stepwise — Documentation for AI Agents

> Workflow orchestration engine for AI agents and humans.
> Define multi-step pipelines in YAML. Run them with a real-time web UI.

Install: curl -fsSL https://raw.githubusercontent.com/zackham/stepwise/master/install.sh | sh
Demo: stepwise run @stepwise:demo --watch
Homepage: https://stepwise.run
GitHub: https://github.com/zackham/stepwise

The essential docs are included in full below. Reference docs are
summarized at the end — fetch them for complete details when needed.

---

<!-- source: https://stepwise.run/docs/quickstart -->
# Quickstart

Zero to running workflow in 5 minutes.

## Install

```bash
curl -fsSL https://raw.githubusercontent.com/zackham/stepwise/master/install.sh | sh
```

Or install directly: `uv tool install stepwise-run@git+https://github.com/zackham/stepwise.git`

## Try it instantly

Run the interactive demo — no setup, no API keys, no files to create:

```bash
stepwise run @stepwise:demo --watch
```

This opens a browser with the live DAG viewer. You'll see steps execute in real time, provide input at an external gate, and watch the flow branch based on your decision.

Have API keys configured? Try a real code review:

```bash
stepwise run @stepwise:code-review --watch
```

An agent reviews your code, pauses for your decision, and continues based on your input. Three executor types in one flow.

Want to browse a flow without running it?

```bash
stepwise open @stepwise:demo
```

## Your first flow

No `.stepwise/` directory needed. No configuration. Just a YAML file.

Create `hello.flow.yaml`:

```yaml
name: hello
steps:
  greet:
    run: echo '{"message": "Hello from Stepwise!"}'
    outputs: [message]

  shout:
    run: echo "{\"loud\": \"$(echo $message | tr '[:lower:]' '[:upper:]')\"}"
    inputs:
      message: greet.message
    outputs: [loud]
```

Run it:

```bash
stepwise run hello.flow.yaml
```

Two steps, one dependency. `greet` runs first and outputs JSON to stdout. `shout` runs with `greet`'s output wired in as `$message`. The engine figures out the order from the `inputs:` declaration.

That's the core model: steps produce typed outputs, downstream steps consume them, the engine resolves the DAG.

## A real workflow

Create `code-review.flow.yaml`:

```yaml
name: code-review
description: AI-powered code review with human approval

steps:
  gather-context:
    run: |
      git diff main --stat && git log main..HEAD --oneline
    outputs: [diff_summary, commits]

  review:
    executor: agent
    prompt: |
      Review this code change. Identify bugs, style issues, and suggest improvements.
      Diff: $diff_summary
      Commits: $commits
    inputs:
      diff_summary: gather-context.diff_summary
      commits: gather-context.commits
    outputs: [verdict, issues, suggestions]

  decide:
    executor: external
    prompt: |
      Review found these issues: $issues

      Apply fixes or skip?
    outputs: [decision]
    inputs:
      issues: review.issues

  apply-fixes:
    executor: agent
    prompt: "Apply these fixes: $suggestions"
    inputs:
      suggestions: review.suggestions
    outputs: [result]
    after: [decide]
```

Four steps, three executor types:

- **gather-context** — a shell script. Runs commands, outputs JSON to stdout.
- **review** — an agent step. A full agentic session with tools, streaming output.
- **decide** — an external step. The job pauses, waits for your input via the web UI or terminal.
- **apply-fixes** — another agent step. Runs after `decide` completes.

Dependencies are implicit from `inputs:`. Steps with no dependencies run in parallel automatically.

## Running modes

### With the live DAG viewer

```bash
stepwise run code-review --watch
```

Opens the browser. Steps light up as they run, agents stream output live, external steps show an inline input form. This is the fastest way to understand what Stepwise does.

### Headless

```bash
stepwise run code-review
```

Step-by-step progress in the terminal. External steps prompt at the command line. Exits when done.

### As a tool for agents

```bash
stepwise run code-review --wait --input repo_path="/path/to/repo"
```

Pure JSON on stdout. Zero logging, zero progress noise. Your agent parses the output and acts on it. See [Agent Integration](agent-integration.md).

### Generate a report

```bash
stepwise run code-review --report
```

Runs the flow and produces a self-contained HTML report with DAG visualization, step timeline, and expandable details for every step.

## Adding a loop

Make the review iterative — if the human requests changes, loop back:

```yaml
  decide:
    executor: external
    prompt: |
      Review verdict: $verdict
      Issues: $issues

      Accept, or request changes?
    outputs: [decision]
    inputs:
      verdict: review.verdict
      issues: review.issues
    exits:
      - name: accept
        when: "outputs.decision == 'accept'"
        action: advance

      - name: request-changes
        when: "outputs.decision == 'request-changes'"
        action: loop
        target: review

      - name: give-up
        when: "attempt >= 3"
        action: advance
```

`exits:` evaluates rules in order after the step completes. `action: loop` with `target: review` re-runs the review step. The `attempt` variable tracks iterations, so you can cap it.

## Adding an external gate

Any step can be replaced with an `external` executor to add an approval point:

```yaml
  approve-deploy:
    executor: external
    prompt: |
      All fixes applied. Deploy to production?

      Changes: $result
    outputs: [approved, note]
    inputs:
      result: apply-fixes.result
```

The flow suspends. With `--watch`, you see the prompt in the web UI with typed input fields. Provide your decision and the flow continues. In headless mode, it prompts at the terminal.

## Running flows from kits

Kits group related flows. Run a specific flow from a kit using `kit/flow` syntax:

```bash
# Run a flow from a local kit
stepwise run swdev/plan --input spec="new auth feature" --watch

# Install and run a kit from the registry
stepwise get @zack:swdev
stepwise run @zack:swdev/implement --input spec="build the API" --watch
```

## Staging and batching jobs

For multi-job workflows, stage jobs before running them:

```bash
# Create staged jobs in a group
stepwise job create my-flow --input task="Build API" --group sprint-1
stepwise job create my-flow --input task="Write tests" --group sprint-1

# Wire data: second job uses first job's output
stepwise job create my-flow \
  --input spec=job-<first-id>.result \
  --group sprint-1

# Review, then release the batch
stepwise job show --group sprint-1
stepwise job run --group sprint-1
```

Jobs auto-start when their dependencies complete. See [Concepts: Job Staging](concepts.md#job-staging).

## What's next

- [Concepts](concepts.md) — the full mental model: jobs, steps, executors, trust, agents
- [Why Stepwise](why-stepwise.md) — the philosophy: the harness, not the intelligence
- [Writing Flows](writing-flows.md) — all step types, wiring, and control flow
- [Use Cases](use-cases.md) — real patterns: podcast pipelines, research synthesis, deploy gates
- [Flow Reference](flow-reference.md) — complete `.flow.yaml` schema

---

<!-- source: https://stepwise.run/docs/concepts -->
# Concepts

Stepwise doesn't do the pedaling. Your agents, scripts, and humans do the work. Stepwise shows you the watts: what's running, what's waiting, what failed, and what needs your attention.

The system has three runtime primitives (jobs, steps, executors), a dependency system (inputs, ordering), and control flow (exit rules, branching, loops). Everything else — for-each, caching, sub-jobs, human gates — is built on top of these.

### Quick reference

| Concept | What it is | Key detail |
|---------|-----------|------------|
| **Job** | A unit of work with inputs and a workflow | Persists to SQLite, survives crashes, can spawn sub-jobs |
| **Step** | A typed node in the workflow graph | Declares outputs, executor, inputs, exit rules |
| **Executor** | What does the work inside a step | script, llm, agent, external, poll |
| **Input binding** | Pulls data from upstream outputs | `findings: research.findings` |
| **Exit rule** | Decides what happens after step completion | advance, loop, escalate, abandon |
| **For-each** | Iterates over a list with embedded sub-flows | Items execute in parallel |
| **Branching** | Conditional activation via step-level `when` | Pull-based: each step decides when it runs |
| **Kit** | A collection of related flows with a `KIT.yaml` manifest | Shared as a unit, referenced as `kit/flow` |

## Jobs

A **job** is a unit of work. It has an objective, initial inputs, and a workflow to execute.

```bash
stepwise run code-review --input repo="/path/to/repo" --input branch="feature-x"
```

Jobs track their own lifecycle: created -> running -> completed/failed. They persist to SQLite — if the process restarts, the job resumes where it left off. No re-running completed steps, no lost progress.

Jobs can spawn **sub-jobs**. A planning step might decompose a large objective into smaller pieces, each running its own workflow. The parent step waits for the sub-job to complete, then collects its output. This recurses to any depth.

### Server vs CLI ownership

Jobs are owned by whoever created them:

- **CLI-owned** — `stepwise run` creates and manages the job in its process. Ctrl+C orphans the job; the server detects this via heartbeat expiry and can adopt it.
- **Server-owned** — created through the web UI or API. The server monitors them, adopts orphans, and broadcasts status via WebSocket.

`stepwise jobs` lists all jobs regardless of owner. `stepwise server status` shows what the server is managing.

## Steps

A **step** is a typed node in the workflow graph. Each step declares:

- **Outputs** — the fields it produces (e.g., `[findings, sources]`)
- **Executor** — what does the work (script, LLM, agent, external, or poll)
- **Inputs** — data pulled from other steps' outputs
- **Exit rules** — what happens after completion (advance, loop, escalate)

```yaml
review:
  executor: external
  prompt: "Review this draft. Approve or request revisions."
  outputs: [decision, feedback]
  inputs:
    content: draft.content
  exits:
    - name: approve
      when: "outputs.decision == 'approve'"
      action: advance
    - name: revise
      when: "outputs.decision == 'revise' and attempt < 5"
      action: loop
      target: draft
```

Steps are **pure functions** — inputs in, outputs out, no shared state. This is what makes retry, parallelism, and observability work cleanly.

Each execution of a step is a **step run** with its own attempt number, status, timing, and result. A step that loops 3 times has 3 step runs, each with its own recorded inputs and outputs.

## Executors

An **executor** is what does the actual work inside a step. Five types, covering the spectrum from deterministic scripts to full agentic sessions:

| Type | What it does | Use when |
|------|-------------|----------|
| **Script** | Runs a shell command | Data processing, API calls, builds, anything deterministic |
| **LLM** | Single LLM call via OpenRouter | Scoring, classification, text generation, structured extraction |
| **Agent** | Full agentic session (LLM + tools, iterating) | Complex tasks: code generation, research, multi-step reasoning |
| **External** | Suspends for input via web UI or API | Approvals, creative judgment, any decision needing a person |
| **Poll** | Runs a check command on an interval | Waiting for CI, deployments, PR reviews, external conditions |

```yaml
# Script — runs a command, parses JSON output
fetch:
  run: python3 scripts/fetch_data.py
  outputs: [data, count]

# LLM — single API call with structured output
score:
  executor: llm
  model: anthropic/claude-sonnet-4
  prompt: "Score this content 0-10: $content"
  outputs: [score, reasoning]

# Agent — full agentic session with tool access
research:
  executor: agent
  prompt: "Research $topic thoroughly"
  outputs: [findings, sources]

# External — waits for human input
approve:
  executor: external
  prompt: "Approve this deployment?"
  outputs: [approved, reason]

# Poll — waits for an external condition
wait-for-ci:
  executor: poll
  check_command: 'gh pr checks $pr --json conclusion --jq "select(.conclusion != \"\") | {done: true}"'
  interval_seconds: 30
  outputs: [done]
```

Executors are serializable references — a type name plus configuration. No live Python objects in the durable model. Jobs can be persisted, resumed, and inspected without executing code.

See the [Executors guide](executors.md) for configuration details and the [Writing Flows guide](writing-flows.md) for authorship patterns.

## Dependencies

Steps connect through two mechanisms.

### Input bindings — data flow

An input binding pulls a specific field from an upstream step's output:

```yaml
summarize:
  outputs: [summary]
  inputs:
    findings: research.findings      # "findings" comes from research step
    scores: evaluate.scores          # "scores" comes from evaluate step
    topic: $job.topic                # "topic" comes from job-level input
```

The local name (`findings`) is what the executor sees. This decouples the executor from the graph topology — rewire inputs without changing the executor's code.

Input bindings create **data dependencies**. The engine won't run a step until all its input sources have completed.

### After — pure ordering

When you need a step to wait for another without taking data:

```yaml
notify:
  run: scripts/send_notification.py
  outputs: [sent]
  after: [deploy]         # wait for deploy, don't use its output
```

### Parallel execution

Steps with no dependencies run in parallel automatically. The engine resolves the DAG and launches everything it can:

```yaml
steps:
  # These three run in parallel — no dependencies between them
  research_a:
    outputs: [findings]
    inputs: { topic: $job.topic_a }
  research_b:
    outputs: [findings]
    inputs: { topic: $job.topic_b }
  research_c:
    outputs: [findings]
    inputs: { topic: $job.topic_c }

  # This waits for all three
  synthesize:
    outputs: [report]
    inputs:
      a: research_a.findings
      b: research_b.findings
      c: research_c.findings
```

## Exit rules and loops

Exit rules fire after a step completes. They evaluate conditions against the step's output and decide what happens next.

```yaml
exits:
  - name: passed
    when: "outputs.score >= 0.8"
    action: advance           # continue to downstream steps

  - name: needs_work
    when: "outputs.score < 0.8 and attempt < 3"
    action: loop              # re-run a step (creates a new attempt)
    target: draft             # which step to re-run

  - name: give_up
    when: "attempt >= 3"
    action: escalate          # pause the job for human inspection
```

| Action | What it does |
|--------|-------------|
| `advance` | Normal progression to downstream steps |
| `loop` | Re-run the `target` step (new attempt). Downstream steps wait for fresh output. |
| `escalate` | Pause the job. A human inspects and decides what to do. |
| `abandon` | Fail the job. |

If no exit rules match (or none are defined), the step advances by default. When explicit `advance` rules exist but none match, the step **fails** — preventing silent advancement past unhandled cases.

Loops are **control flow, not graph cycles**. The workflow definition is always a DAG. When a loop fires, the engine creates a new step run (attempt N+1) for the target. The key mechanism is **supersession** — the new run invalidates the previous one, and that invalidation cascades downstream. Steps only run when all their dependencies are fresh.

## For-each

For-each steps iterate over a list, running an embedded sub-flow for each item:

```yaml
process_sections:
  for_each: plan.sections        # iterate over this list
  as: section                    # name for current item
  on_error: continue             # or "fail_fast" (default)
  outputs: [results]

  flow:
    steps:
      generate:
        executor: llm
        prompt: "Generate content for: $section"
        outputs: [html]

      review:
        executor: llm
        prompt: "Review this HTML for quality"
        outputs: [pass, feedback]
        inputs:
          html: generate.html
        exits:
          - name: good
            when: "outputs.pass == True"
            action: advance
          - name: retry
            when: "outputs.pass == False and attempt < 3"
            action: loop
            target: generate
```

Each iteration runs as an independent sub-job. Results are collected in source list order. Items execute in parallel. `on_error: continue` means one failed item doesn't block the rest.

## Conditional branching

Branching is **pure-pull**: each step declares its own activation condition via `when`, evaluated against its resolved inputs. Merge points use `any_of` inputs to take from whichever branch completed.

```yaml
steps:
  classify:
    run: scripts/classify.sh
    outputs: [category]

  quick-path:
    run: scripts/quick.sh
    inputs: { category: classify.category }
    outputs: [result]
    when: "category == 'simple'"

  deep-path:
    executor: agent
    prompt: "Deep analysis..."
    inputs: { category: classify.category }
    outputs: [result]
    when: "category == 'complex'"

  final:
    run: scripts/report.sh
    inputs:
      result:
        any_of:
          - quick-path.result
          - deep-path.result
    outputs: [report]
```

When `classify` outputs `category == 'simple'`, only `quick-path` activates. `deep-path` stays not-ready. When no steps are in motion and nothing new can activate, the engine **settles** — skipping never-started steps and completing the job.

Key distinctions:
- `after: [step-x]` = ordering only
- `inputs: { field: step-x.field }` = data dependency (implies ordering)
- `when: "expr"` = conditional gate on resolved inputs

## Coordination rules

The validator enforces a set of coordination rules to prevent races between steps that share state (named sessions, forked sessions, shared workspaces). These rules are enforced both at parse time (`yaml_loader`) and by the coordination validator (`stepwise validate`). Run `stepwise validate <flow.yaml>` before every run to catch session-writer races, unguarded loop-back bindings, and cycle errors early.

**Pair safety for session writers (§7.3).** Any two steps writing to the same named session must be provably non-concurrent. The validator proves this by showing either (a) one step's `after:`-transitive closure includes the other (linear chain), or (b) their `when:` predicates are pairwise mutex (conditional branches). If neither can be proven, the validator emits `pair_unsafe` with a fix hint pointing at adding `after:` or a mutex `when:` gate.

**`after.any_of` and the universal-prefix rule (§7.2, §10).** When a step declares `after: [{any_of: [a, b, c]}]`, the "first success wins" eligibility model allows the step to launch as soon as any of the branches completes. Losing branches keep running (no cancellation in v1.0). For ordering proofs, only the **intersection** of mhb-ancestors across all `any_of` branches carries — i.e., a step that appears before EVERY branch of an `any_of` group is an mhb-ancestor of the joined step.

**Fork from step name (§8.2).** `fork_from: <step_name>` (or `fork_from: $job.<input>` with `type: session`) anchors a new session at a specific step's completion tail. The target step must declare its own `session:`. The forking step can optionally declare its own `session:` for chain continuation; without `session:`, the fork is an ephemeral one-shot (§9.7.1). Three ergonomic inferences apply (§9.7.5): `agent: claude` is auto-inferred from `fork_from:` on agent steps; `working_dir` is inherited from the fork source step; and `flow.inputs:` on embedded `for_each` sub_flows is inferred from parent bindings.

**Conditional fork rejoin (§8.3).** Multiple chain roots on the same forked session are permitted when their `when:` clauses are pairwise mutex (e.g., two alternative chain roots gated on a routing step's output tag). The parse-time validator allows this structurally; the coordination validator verifies the mutex proof.

**Retries and cache are prohibited on session-writing and fork-source steps (§7.4).** The validator rejects flows that combine `session:` or `fork_from:` with `max_attempts > 1` or `cache:`. For crash recovery, re-executing a session-writing step carries documented duplicate-turn risk (§9.3); this is the acceptable v1.0 limitation. Authors who need retry semantics should use ephemeral one-shot agent steps (no `session:`), which can be retried freely because they don't accumulate session state.

**Eager snapshot via filesystem copy (§9).** When a step is a fork source (some downstream step declares `fork_from: <this_step>`), the engine serializes its post-exit lifecycle inside an exclusive `fcntl.flock`, copies its session JSONL file to a new UUID via `temp → os.replace → fsync(parent_dir)`, persists the snapshot UUID atomically with the completion record, then releases the lock. Downstream forks resume from the snapshot UUID (via `claude --resume <snap_uuid> --fork-session`), not from the live session tail. This eliminates the race where the parent session keeps mutating past the intended fork point.

**Running the coordination validator.** `stepwise validate <flow.yaml>` runs both the structural parse-time checks and the coordination validator. Parse-time errors are fatal; coordination validator findings are surfaced as warnings (a future `--strict` flag will make them fatal).

**Loop-back bindings and the LoopFrame stack (§11).** A loop-back binding is an `optional: true` or `any_of` input whose source is closed by an enclosing loop exit rule (`action: loop` or `action: escalate` with a `target:`). The parser marks such bindings with `is_back_edge=True` and assigns them a `closing_loop_id` (the loop target step name, which serves as the frame id). The static cycle detector excludes marked back-edges when proving the forward DAG acyclic; unguarded cycles (plain bindings forming a cycle without any escape hatch) are rejected at parse time.

At runtime, the engine maintains a `LoopFrame` stack on each `Job`. A frame is lazily allocated when a loop first fires; its `iteration_index` starts at 0 and bumps on every `action: loop` (or escalate-routed loop) fire. The three-state input resolver returns `(inputs, dep_run_ids, presence)` — the `presence` side-table is keyed by the binding's local name and marks whether each binding resolved to a value. Loop-back bindings are treated as *absent* whenever their closing-loop frame has not yet fired (`iteration_index == 0`), so on iteration 1 the binding resolves to `None` (for plain optionals) or falls through to a non-back-edge `any_of` source.

Presence is what makes `is_present: true/false` and `is_null: true/false` work on `when:` predicates attached to loop-back bindings. Nested loops get independent frames; bumping a parent frame invalidates all child frames (via `_invalidate_child_frames`), so an inner loop's presence always starts absent on every outer iteration. See §11 of `data/reports/2026-04-07-stepwise-coordination-and-validation-model.md` for the full presence truth table and `flows/test-loop-back-nested/FLOW.yaml` for a worked nested canary.

## Job staging

Create jobs in a **STAGED** state, build up a batch with dependencies, review, then release:

```
STAGED -> (add deps, wire data, review) -> job run -> PENDING -> RUNNING -> COMPLETED/FAILED
```

### The create → dep → run pattern

The core workflow: create all jobs upfront, wire dependencies between them, then release the group. The engine handles execution ordering automatically.

```bash
# 1. Create staged jobs — capture IDs from JSON output
RESEARCH=$(stepwise job create research-v2 \
  --input topic="Widget architecture" \
  --group widget --name "research: widget arch" \
  --output json | jq -r .id)

PLAN=$(stepwise job create plan \
  --input spec="Design widget system" \
  --input project="my-app" \
  --group widget --name "plan: widget system" \
  --output json | jq -r .id)

IMPL=$(stepwise job create implement \
  --input spec="Build widget system" \
  --input project="my-app" \
  --group widget --name "impl: widget system" \
  --output json | jq -r .id)

# 2. Wire dependencies — plan waits for research, impl waits for plan
stepwise job dep $PLAN --after $RESEARCH
stepwise job dep $IMPL --after $PLAN

# 3. Review the DAG
stepwise job show --group widget

# 4. Release and wait — engine cascades execution in dependency order
stepwise job run --group widget --wait
```

### Data wiring between jobs

Use `--input key=job-id.field` to pass outputs from one job as inputs to another. This **auto-creates a dependency edge** — no separate `job dep` call needed:

```bash
# Plan job ran and produced outputs including "plan" and "plan_file"
IMPL=$(stepwise job create implement \
  --input spec="Build auth middleware" \
  --input plan_file=$PLAN.plan_file \
  --group auth --name "impl: auth middleware" \
  --output json | jq -r .id)
```

The `$PLAN.plan_file` syntax means "the `plan_file` output from the job whose ID is in `$PLAN`." The engine resolves this at runtime when the upstream job completes.

### Ordering vs data dependencies

- **Data wiring** (`--input key=job-id.field`) — passes data AND creates ordering. Use when the downstream job needs the upstream job's output.
- **Ordering only** (`job dep A --after B`) — no data flow, just "B must finish before A starts." Use when jobs must run sequentially but don't share data (e.g., both write to the same directory).

### Parallel workstreams

Jobs in the same group with no dependencies between them run in parallel:

```bash
# These three run concurrently — no deps between them
stepwise job create plan --input spec="Memory game" --group gumball --name "plan: memory game"
stepwise job create plan --input spec="Reading module" --group gumball --name "plan: reading module"
stepwise job create research-v2 --input topic="Scene composers" --group gumball --name "research: scene composer"

# Release all and wait — engine runs all three in parallel
stepwise job run --group gumball --wait
```

### Concurrency control

```bash
# Limit to 2 concurrent jobs in the group
stepwise job run --group gumball --max-concurrent 2
```

## The trust model

Stepwise is built around **packaged trust** — the idea that the real barrier to AI delegation isn't capability, it's confidence. You need to know what happened, why, and whether to let it continue.

### Observable runs

Every step run is recorded with:
- **Inputs** — the exact values passed to the executor
- **Outputs** — the artifact produced, validated against declared output fields
- **Timing** — start time, duration, queue wait
- **Executor metadata** — model used, token counts, cost
- **Attempt count** — which iteration this is (for looped steps)

This isn't optional logging. It's the execution model. The engine can't run a step without recording these facts, because downstream steps depend on them.

### Scoped delegation

Each step has bounded authority: declared inputs, declared outputs, a specific executor type. An agent step can't silently access data from an unrelated step. A script can't produce outputs it didn't declare. The engine validates artifact keys against the step's `outputs` list.

This scoping makes mixed workflows safe. You can have an untrusted script fetch data, a trusted agent analyze it, and a human approve the result — each step's scope is explicit in the YAML.

### Human gates

External steps are the trust primitive. They pause the job and present full context of what happened before the gate. The `escalate` exit rule is the safety valve: when an agent has tried 3 times and is still failing, the job pauses for human triage instead of burning more tokens.

External steps + escalation rules = workflows that delegate aggressively but fail safely. The human is always in the loop — not as a bottleneck, but as a circuit breaker.

### Hardware boundaries — containment

Scoped delegation bounds what an agent *should* access. Containment bounds what it *can* access. Agent steps can run inside hardware-isolated microVMs (via Cloud-Hypervisor), so that even a compromised agent can only reach the filesystem paths, credentials, and network endpoints explicitly declared in its configuration. Steps with different security profiles run in separate VMs — a research agent with web-search tools can never access a deploy agent's AWS credentials, because they execute in different hardware boundaries. Containment is opt-in and transparent to the ACP protocol layer. See the [Containment guide](containment.md) for architecture and setup.

### Audit trail

Every state transition, input resolution, and cost event is persisted as a structured step event. This powers the web UI's event timeline, HTML reports via `--report`, and direct queries against the SQLite store.

When something goes wrong at step 4 of a 7-step pipeline, you see exactly what inputs it received, what it produced, and why the exit rule fired the way it did.

## Kits

A **kit** is a directory containing a `KIT.yaml` manifest and one or more flow subdirectories. Kits group related flows into a single, shareable package — for example, a software development kit might bundle `plan`, `implement`, and `research` flows.

```
swdev/
  KIT.yaml                 # manifest: name, description, includes, defaults
  plan/FLOW.yaml
  implement/FLOW.yaml
  research/FLOW.yaml
```

Locally, kit flows are referenced as `kit/flow` (e.g., `stepwise run swdev/plan`). On the registry, installed kits use `@author:kit/flow` (e.g., `stepwise run @zack:swdev/plan`).

Kits can declare **includes** — references to other registry flows that are auto-fetched during `stepwise get`. This lets a kit depend on shared utility flows without bundling them.

See [Flow and Kit Sharing](flow-sharing.md) for publishing and installation.

## How agents fit in

Agents interact with Stepwise in three roles.

### As callers

An agent calls a flow like a CLI tool:

```bash
stepwise run deploy --wait --input repo="/path" --input branch="main"
```

`--wait` prints pure JSON to stdout. Exit codes are explicit (0=success, 1=failed, 2=input error, 3=timeout, 4=cancelled). No MCP servers, no protocol layers. Just bash commands.

### As workers

Agent steps (`executor: agent`) use an LLM with tools inside a step. The agent is scoped to its step's inputs and outputs — it iterates autonomously but only within the boundaries the step defines.

With `session: <name>`, steps sharing the same session name maintain context across iterations and across steps, saving tokens by continuing the conversation rather than re-injecting context each time.

### As architects

With `emit_flow: true`, an agent step can dynamically create sub-workflows. The agent analyzes a task, writes a flow definition, and the engine executes it as a sub-job. Results propagate back to the parent step.

This is recursive delegation: an agent decides *how* to break down work, not just *what* to do.

## Handoff envelopes

When a step completes, it produces a **handoff envelope**:

- **Artifact** — the output data (a dict matching the step's declared outputs)
- **Sidecar** — optional metadata: decisions made, assumptions, confidence levels
- **Executor metadata** — model used, token counts, cost, latency

The envelope is the contract between steps. Downstream steps receive the artifact fields they bind to. The sidecar and executor metadata are available for observability and reporting.

## Observability

Every state transition, input/output handoff, and cost event is persisted as a structured step event. This powers:

- **Web UI** — real-time DAG visualization, step detail panels, event timeline. See the [Web UI guide](web-ui.md).
- **HTML reports** — `stepwise run flow.yaml --report` generates a self-contained trace document
- **Programmatic access** — query the SQLite store directly for custom analysis

## Hooks and notifications

**Shell hooks** (`.stepwise/hooks/on-suspend`, `on-complete`, `on-fail`) run in the engine's process context for local automation.

**Server notifications** (`--notify URL`) fire HTTP webhooks on job events for remote integrations.

See [Extensions](extensions.md) for details.

## What's next

- [Writing Flows](writing-flows.md) — author workflows using all step types, wiring, and control flow
- [Executors](executors.md) — deep dive into executor configuration and decorators
- [Flow Reference](flow-reference.md) — complete field-by-field YAML schema
- [Agent Integration](agent-integration.md) — making flows callable by AI agents
- [Web UI](web-ui.md) — the dashboard, DAG viewer, and step detail

---

<!-- source: https://stepwise.run/docs/why-stepwise -->
# Why Stepwise

## The felt problem

You ask an agent to do something that takes 20 minutes. While it runs, you do something else. When you come back: did it work? Did it go off the rails at step 3? Did it burn $14 in API credits retrying a hallucinated command?

You don't know. So you read the logs. You re-run parts of it. You check the output manually. The 20 minutes of agent time saved you 20 minutes of verification time. Net gain: zero.

Now multiply that by a flow that takes 3 hours. Or one that runs overnight. Or one where a human needs to approve step 7 before step 8 can proceed. The agent is capable. But you can't *trust* the run, because the run isn't observable, isn't recoverable, and isn't auditable.

This is the gap Stepwise fills. Not the intelligence — the harness.

## The harness, not the intelligence

The intelligence commoditizes. Claude, GPT, Gemini, Codex — they get better every quarter. The models are not the bottleneck.

The bottleneck is packaging AI work into something you can delegate, observe, gate, and audit. A brisket doesn't need a better cow. It needs a smoker with a reliable thermometer, predictable airflow, and a way to check the bark without opening the door every 20 minutes.

Stepwise is the smoker. Your agents are the brisket.

Concretely, this means:

- **Observable runs.** Every step records inputs, outputs, timing, cost, and attempt count. Not as optional logging — as the execution model. Downstream steps depend on this data.
- **Human gates.** External steps pause the flow and wait for judgment. In the web UI, CLI, or via API. The human sees full context of what happened before the gate.
- **Crash recovery.** Everything persists to SQLite. Kill the process, restart, jobs resume from the last completed step. Orphaned jobs get adopted automatically.
- **Audit trail.** Every state transition is an event. The `--report` flag renders a self-contained HTML trace. You can see exactly what went wrong at step 4, with the exact inputs it received.
- **Cost controls.** Per-step limits on dollars, wall-clock time, and iterations. When a limit fires, the step fails cleanly and exit rules route to a fallback or human escalation.

## Step over role

Most agent frameworks model *who* does the work. CrewAI gives you "Senior Researcher" and "Technical Writer." AutoGen gives you named agents in conversations.

But an LLM doesn't *become* a Senior Researcher by reading a backstory. It gets a system prompt, a set of tools, and instructions. The persona is semantic sugar — and it comes with real costs: identity conflicts when you parallelize, tool rigidity, and opaque prompt construction.

Stepwise models *what* the work is. Each step declares its executor type, inputs, and outputs. A "research" step needs search tools and a topic. A "review" step needs the draft and approval criteria. Same LLM, different configuration. The step IS the role.

```yaml
steps:
  research:
    executor: agent
    prompt: "Research $topic and produce structured findings"
    outputs: [findings, sources]
    inputs:
      topic: $job.topic

  score:
    executor: llm
    model: anthropic/claude-sonnet-4
    prompt: "Score these findings 0-10 on depth and relevance: $findings"
    outputs: [score, reasoning]
    inputs:
      findings: research.findings

  review:
    executor: external
    prompt: "Score: $score. Approve or request deeper research?"
    outputs: [decision]
    inputs:
      score: score.score
```

Three steps, three executor types, zero role definitions. The agent researches because its step says to research, not because it was cast as a researcher.

## Deterministic orchestration, nondeterministic execution

The engine's control plane is explicit and reproducible. Dependencies are a DAG. Exit rules are evaluated expressions. Parallel execution follows from graph topology. You can reason about flow structure without reasoning about LLM behavior.

The nondeterminism lives inside executors — inside agent sessions, LLM calls, and human decisions. The engine doesn't care what an executor does internally. It cares about the contract: declared inputs in, declared outputs out, within stated limits.

This separation is what makes the system trustworthy. When you look at a flow definition, you see the structure. When you look at a step run, you see the content. They don't bleed into each other.

## What if AI work lived in a system instead of a vibe?

Right now, most AI delegation looks like: copy context into a chat window, prompt carefully, hope for the best, manually verify. The "workflow" lives in your head and your clipboard.

Stepwise makes the workflow explicit:

- **Declared, not coded.** Flows are YAML files. Version-controlled, diffable, shareable, runnable on any machine. Non-programmers can read and review them.
- **Mixed executors that compose.** A shell script fetches data. An LLM scores it. An agent implements the fix. A poll waits for CI. A human approves the deploy. One DAG, zero glue code.
- **Loops with safety caps.** Exit rules fire after each step. If quality is too low, loop back. If attempts hit the ceiling, escalate to a human. Declared in YAML, enforced by the engine.
- **External fulfillment as a primitive.** The `external` executor pauses a step and waits for input from *anyone* — a human, a webhook, another agent. It's the same execution model as every other step, not a bolt-on.

## Design principles

1. **Steps are pure functions.** Inputs in, outputs out. No shared mutable state. This is what makes retry, parallelism, and observability clean.

2. **Deterministic orchestration, nondeterministic execution.** The engine is explicit and reproducible. The AI lives inside executors.

3. **Human gates over human management.** People approve, redirect, and judge at key points. They don't micromanage every step.

4. **Halt on failure, inspect, rerun.** When something breaks, the job stops. You look at the data, fix the issue, rerun the failed step. No hidden retry logic masking bugs.

5. **Observable by default.** Every transition, every handoff, every cost event is logged — because downstream steps depend on it.

6. **Single machine, zero infrastructure.** SQLite. One process. `curl | sh` to install. No servers, no workers, no cloud accounts.

## Who it's for

- **The founder with one ugly process held together by duct tape.** You don't think "I need workflow orchestration." You think "I need a better spreadsheet." Stepwise is the upgrade.
- **Developers building AI pipelines** who want structure without framework lock-in.
- **Solo builders and small teams** who need orchestration without infrastructure.
- **Anyone mixing AI with human judgment** — content pipelines, code review, research, anything where quality gates matter.

## What it's not

- Not a hosted platform — it runs on your machine
- Not an agent framework — it orchestrates agents, it doesn't build them
- Not enterprise infrastructure — if you need distributed workers and compliance controls, look at Temporal
- Not a drag-and-drop builder — it's YAML-first, designed for people who read and write config files

---

<!-- source: https://stepwise.run/docs/writing-flows -->
# Writing Flows

A flow is a YAML file that defines a workflow — a directed acyclic graph of steps. Each step declares what it does, what it needs, and what it produces. The engine handles ordering, parallelism, retries, and persistence.

This guide covers everything you need to author flows from scratch. For the complete field-by-field schema, see the [YAML Format](yaml-format.md).

## Flow file structure

A flow file is any file ending in `.flow.yaml`. It can live anywhere — project root, a `flows/` directory, or nested in subdirectories. No special directory structure required.

Minimal flow:

```yaml
name: my-flow
steps:
  greet:
    run: echo '{"message": "hello"}'
    outputs: [message]
```

Every flow needs `name` and `steps`. Each step needs at least an executor (explicit or implied by `run:`) and `outputs`.

**Flow directories:** For flows with supporting files (scripts, prompts, data), use a directory with a `FLOW.yaml` inside:

```
flows/
  deploy/
    FLOW.yaml
    scripts/
      build.sh
      health-check.sh
```

Create a new flow directory with `stepwise new my-flow`.

**Validation:** Always validate before running:

```bash
stepwise validate my-flow.flow.yaml
```

This catches structural errors, missing references, unbounded loops, and more — without executing anything.

**Archiving:** Set `archived: true` as a top-level key to hide a flow from default listings (`stepwise flows`, `stepwise agent-help`, the web Flows page) without removing it. Archived flows remain runnable — `stepwise run <archived-flow>` works unchanged. Toggle via `stepwise flow archive` / `stepwise flow unarchive`, or hard-remove with `stepwise flow delete`. See [Flow Lifecycle Commands](cli.md#flow-lifecycle-commands).

## Script steps

The simplest step type. Runs a shell command and parses JSON from stdout.

```yaml
name: fetch-example
steps:
  fetch:
    run: |
      curl -s "https://api.example.com/data?q=$query" | jq '{count: .total, items: .results}'
    inputs:
      query: $job.search_term
    outputs: [count, items]
```

**How it works:**
- The `run:` field is shorthand for `executor: script`
- Input values are available as `$variable_name` in the command and as `STEPWISE_INPUT_<NAME>` environment variables
- The command's stdout must be a JSON object whose keys match the declared `outputs`
- Non-zero exit code = step failure

**Multi-line scripts** work naturally with YAML `|` blocks. For complex logic, put it in a script file:

```yaml
  process:
    run: python3 scripts/process.py
    inputs:
      data: fetch.items
    outputs: [result, summary]
```

The script receives inputs as environment variables (`STEPWISE_INPUT_DATA`) and prints JSON to stdout.

## Agent steps

An agent step runs a full agentic session — an LLM with tools, iterating until it completes the task.

```yaml
name: research-example
steps:
  research:
    executor: agent
    prompt: |
      Research $topic thoroughly. Find primary sources, verify claims,
      and produce a structured summary.
    inputs:
      topic: $job.topic
    outputs: [summary, sources, confidence]
```

**Key options:**

| Field | Description |
|-------|-------------|
| `prompt` | The task description sent to the agent |
| `working_dir` | Directory where the agent runs (loads CLAUDE.md from there) |
| `outputs` | Declared outputs — agent receives instructions to write these as JSON |
| `emit_flow` | If `true`, agent can create sub-workflows dynamically |
| `session` | Named session — steps with the same name share a conversation |
| `loop_prompt` | Alternate prompt used on attempt > 1 |
| `max_continuous_attempts` | Circuit breaker for continued sessions |
| `output_mode` | `"effect"` (default), `"stream_result"`, or `"file"` |
| `output_path` | File path for `output_mode: file` |

When `outputs` is declared, the agent automatically receives a `STEPWISE_OUTPUT_FILE` environment variable and prompt instructions explaining the expected JSON structure. The agent writes the file; the engine reads and validates it.

**Agent output modes:**

| Mode | Artifact | Use When |
|---|---|---|
| `"effect"` (default) | `{"status": "completed"}` | Agent modifies files; workspace IS the output |
| `"stream_result"` | `{"result": "<full agent text>"}` | You need the agent's textual response downstream |
| `"file"` | Parsed JSON from `output_path` | Agent writes structured JSON to a specific file |

```yaml
analyze:
  executor: agent
  output_mode: file
  output_path: .stepwise/analysis.json
  prompt: |
    Analyze the codebase. Write your findings as JSON to .stepwise/analysis.json
    with keys: overview, modules, risks.
  outputs: [overview, modules, risks]
```

**`output_mode: file` requires explicit prompt instructions.** The engine reads `output_path` after the agent finishes and parses it as JSON. Your prompt must tell the agent to write JSON to that location with keys matching the declared `outputs`.

**Dynamic sub-flows** with `emit_flow: true`:

```yaml
name: emit-example
steps:
  implement:
    executor: agent
    prompt: "Break this into steps and implement: $spec"
    emit_flow: true
    inputs:
      spec: $job.spec
    outputs: [result]
```

The agent can write a `.stepwise/emit.flow.yaml` file to its working directory. The engine launches the emitted flow as a sub-job and propagates results back.

## LLM steps

A single LLM API call — no tools, no iteration. Faster and cheaper than agent steps when you just need text generation or structured extraction.

```yaml
name: score-example
steps:
  score:
    executor: llm
    model: anthropic/claude-sonnet-4
    prompt: |
      Score this content on a 0-10 scale. Return JSON with "score" and "reasoning".
      Content: $content
    inputs:
      content: $job.content
    outputs: [score, reasoning]
```

**Configuration fields** (all set at step level, not nested in `config:`):

| Field | Required | Description |
|-------|----------|-------------|
| `model` | Yes | Full model ID (e.g., `anthropic/claude-sonnet-4`) or tier alias (e.g., `balanced`) |
| `prompt` | Yes | The user message. Supports `$variable` substitution from inputs. |
| `system` | No | System prompt |
| `temperature` | No | Sampling temperature (default: 0.0) |
| `max_tokens` | No | Maximum output tokens (default: 4096) |

The response must be parseable as JSON matching the declared outputs. The LLM executor uses structured output tooling to enforce this.

## External steps

External steps pause the job and wait for human input. This is the primary mechanism for human-in-the-loop workflows — approvals, creative judgment, decisions that need a person.

```yaml
name: approval-example
steps:
  review:
    executor: external
    prompt: |
      The agent produced this analysis. Review and decide:

      Analysis: $analysis
      Confidence: $confidence

      Approve for publication, or request revisions?
    inputs:
      analysis: analyze.analysis
      confidence: analyze.confidence
    outputs: [decision, feedback]
```

When the job reaches this step, it suspends. The prompt appears in the web UI with input fields for each declared output. You can also fulfill from the CLI or API:

```bash
stepwise fulfill <run-id> '{"decision": "approve", "feedback": "Looks good"}'
```

**Typed fields** with `output_fields` for richer input forms:

```yaml
    output_fields:
      decision:
        type: choice
        options: [approve, revise, reject]
        description: "Your decision"
      feedback:
        type: text
        description: "Optional notes"
```

Valid field types: `str`, `text`, `number`, `bool`, `choice`.

## Poll steps

Poll steps wait for an external condition by running a check command on an interval.

```yaml
name: poll-example
steps:
  wait-for-ci:
    executor: poll
    check_command: |
      gh pr view $pr_number --json statusCheckRollup \
        --jq 'select(.statusCheckRollup[0].conclusion != "") | {status: .statusCheckRollup[0].conclusion}'
    interval_seconds: 30
    prompt: "Waiting for CI checks on PR #$pr_number"
    inputs:
      pr_number: create-pr.pr_number
    outputs: [status]
```

**How it works:**
- `check_command` runs every `interval_seconds`
- Empty stdout or non-zero exit = not ready yet, keep polling
- JSON dict on stdout = fulfilled (the dict becomes the step's artifact)
- `$variable` placeholders in `check_command` and `prompt` are interpolated from inputs

Use poll steps for: CI status, deployment health, PR reviews, external API readiness — anything where you're waiting for a condition that changes on its own.

## Wiring inputs and outputs

Steps connect through input bindings. Three sources:

```yaml
inputs:
  # From another step's output
  data: fetch-data.raw_data         # step-name.field-name

  # From job-level inputs (--input flags)
  query: $job.search_term            # $job.field-name

  # Optional binding (resolves to None if source unavailable)
  previous_score:
    from: review.score
    optional: true
```

**Optional inputs** are key for iterative patterns. On the first iteration, an optional input resolves to `None`. On subsequent iterations (after a loop), the source has a value. This lets cycles work without deadlocks:

```yaml
name: iterate-example
steps:
  generate:
    executor: agent
    prompt: "Write content about $topic. Previous score: $score"
    inputs:
      topic: $job.topic
      score:
        from: review.score
        optional: true
    outputs: [content]

  review:
    executor: llm
    model: anthropic/claude-sonnet-4
    prompt: "Score this content 0-10: $content"
    inputs:
      content: generate.content
    outputs: [score]
    exits:
      - when: "float(outputs.score) >= 8"
        action: advance
      - when: "attempt < 3"
        action: loop
        target: generate
```

**`any_of` inputs** take from whichever branch completed (used with conditional branching):

```yaml
inputs:
  result:
    any_of:
      - quick-path.result
      - deep-path.result
```

**Nested paths** work for deeply structured outputs: `step-name.field.nested.path`.

## For-each

Iterate over a list, running an embedded sub-flow for each item:

```yaml
name: foreach-example
steps:
  plan:
    executor: agent
    prompt: "Break this into sections: $spec"
    inputs:
      spec: $job.spec
    outputs: [sections]

  process-sections:
    for_each: plan.sections
    as: section
    on_error: continue
    outputs: [results]
    flow:
      steps:
        write:
          executor: agent
          prompt: "Write content for: $section"
          outputs: [content]

        review:
          executor: llm
          model: anthropic/claude-sonnet-4
          prompt: "Review quality: $content"
          inputs:
            content: write.content
          outputs: [pass, feedback]
          exits:
            - when: "outputs.pass == True"
              action: advance
            - when: "attempt < 3"
              action: loop
              target: write
```

Each iteration runs as an independent sub-job. Items execute in parallel. Results are collected in source list order.

- `on_error: continue` — other items keep running if one fails (default: `fail_fast`)
- The `as` variable is available as an input to steps within the sub-flow via `$job.<as_variable>`
- Empty source lists complete immediately with `{"results": []}`
- If all items fail under `on_error: continue`, the for-each step itself fails

For-each steps support `when` conditions for conditional activation, just like regular steps.

## Sub-flow composition

Steps can delegate to other flows via the `flow:` field:

```yaml
steps:
  evaluate:
    flow: evaluate-quality              # bare name — resolved from project
    inputs:
      content: generate.report          # becomes $job.content in sub-flow
      rubric: "Score on depth, accuracy"
    outputs: [scores, average, critique]
```

Sub-flow sources can be:
- **Bare flow names** — `flow: evaluate-quality` (resolved from `flows/`, project root, `.stepwise/flows/`)
- **File paths** — `flow: ./sub-flows/eval.flow.yaml`
- **Registry refs** — `flow: @alice:evaluate-quality`
- **Inline dicts** — embed a `flow: { steps: { ... } }` directly

Sub-flow steps support `when` conditions for conditional activation.

## Exit rules

Exit rules fire after a step completes, evaluating conditions to decide what happens next.

```yaml
exits:
  - name: success
    when: "outputs.status == 'done'"
    action: advance

  - name: stuck
    when: "attempt >= 3"
    action: escalate

  - name: retry
    when: "True"
    action: loop
    target: implement
    max_iterations: 5
```

**Four actions:**

| Action | Effect |
|--------|--------|
| `advance` | Continue to downstream steps |
| `loop` | Re-run `target` step (new attempt, fresh run) |
| `escalate` | Pause the job for human inspection |
| `abandon` | Fail the job |

Rules evaluate in order — first match wins. No match with explicit `advance` rules = step fails (prevents silent advancement past unhandled cases). No match with only loop/escalate/abandon rules = implicit advance. No exit rules at all = implicit advance.

**The escalate pattern:** Use `escalate` as a safety bound between success and retry:

```yaml
exits:
  - name: success
    when: "outputs.passed == true"
    action: advance
  - name: stuck
    when: "attempt >= 3"
    action: escalate          # pauses for human triage
  - name: retry
    when: "True"
    action: loop
    target: implement
```

Priority: success first, then escalate as ceiling, then loop as fallback. Escalated jobs appear in `stepwise list --suspended`.

**Boomerang steps:** Steps with no `advance` exit rules (only loop + escalate/abandon) are excluded from terminal step detection. They exist purely as loop machinery, not as workflow outputs.

## Conditional branching

Steps declare their own activation condition via `when`, evaluated against resolved inputs:

```yaml
name: branch-example
steps:
  classify:
    run: scripts/classify.sh
    outputs: [category]

  quick-path:
    run: scripts/quick.sh
    inputs:
      category: classify.category
    when: "category == 'simple'"
    outputs: [result]

  deep-path:
    executor: agent
    prompt: "Deep analysis of $category data"
    inputs:
      category: classify.category
    when: "category == 'complex'"
    outputs: [result]

  report:
    run: scripts/report.sh
    inputs:
      result:
        any_of:
          - quick-path.result
          - deep-path.result
    outputs: [report]
```

Branching is **pull-based** — each step decides when it activates. When `classify` outputs `category == 'simple'`, `quick-path` activates and `deep-path` stays not-ready. At settlement, never-started steps get SKIPPED.

**Key distinction:**
- `after: [step-x]` — ordering only (wait, but no data)
- `inputs: { f: step-x.f }` — data dependency (implies ordering)
- `when: "expr"` — conditional gate on resolved inputs

### Loop-back bindings and presence predicates

Loops can carry data across iterations via **loop-back bindings**: an `optional: true` or `any_of` input whose source is closed by an enclosing `loop` (or `escalate` with a `target:`) exit rule. On iteration 1 the binding is *absent*; on iter-N > 1 it carries the previous iteration's output.

```yaml
steps:
  analyze:
    run: scripts/analyze.sh
    inputs:
      seed: $job.seed
      prev_note:
        from: critique.note
        optional: true          # loop-back: absent on iter-1, carries critique.note on iter-N
    outputs: [text]

  critique:
    run: scripts/critique.sh
    inputs:
      text: analyze.text
    outputs: [note, verdict]
    exits:
      - when: "outputs.verdict == 'done'"
        action: advance
      - when: "True"
        action: loop
        target: analyze          # loop closes the analyze ← critique back-edge
        max_iterations: 5
```

When you want different logic on iter-1 vs. iter-N, use the `is_present:` / `is_null:` predicates (see yaml-format.md for the full truth table):

```yaml
analyze-init:
  when:
    input: prev_note
    is_present: false       # only runs on iter-1 (before any loop fires)
  inputs:
    prev_note:
      from: critique.note
      optional: true
  outputs: [text]

analyze-refine:
  when:
    input: prev_note
    is_present: true        # only runs on iter-N > 1 (after loop bump)
  inputs:
    prev_note:
      from: critique.note
      optional: true
  outputs: [text]
```

The validator accepts loops closed by `any_of` (iter-1 fallback to a non-loop producer) or `optional: true` (iter-1 resolves to `None`). Plain bindings forming a cycle without one of these escape hatches are rejected at parse time — you must declare the fallback explicitly.

Nested loops get independent iteration frames: when the outer loop bumps, the inner frame resets to iteration 0, so the inner step's loop-back binding is absent on the first inner iteration of every outer iteration. See `flows/test-loop-back-nested/FLOW.yaml` in the vita repo for a worked nested canary.

## Derived outputs

Compute fields deterministically from a step's executor output. Evaluated after the executor returns but before exit rules.

```yaml
score:
  executor: llm
  prompt: |
    Score this plan on 8 dimensions (1-5 each).
    Respond with ONLY: {"scores": {"completeness": 4, "grounding": 3, ...}}
  outputs: [scores]
  derived_outputs:
    average: "sum(scores.values()) / len(scores)"
    passed: "sum(scores.values()) / len(scores) >= 4.0"
    lowest_three: "sorted(scores, key=scores.get)[:3]"
```

The LLM returns only `scores`. The engine computes `average`, `passed`, and `lowest_three` deterministically. All three become real step outputs that downstream steps and exit rules can reference.

**Expression environment:** Artifact fields as local variables, plus Python builtins (`sum`, `len`, `sorted`, `min`, `max`, `float`, `int`, `str`, `list`, `dict`, `set`, `tuple`, `round`, `abs`, `any`, `all`, `enumerate`, `zip`, `map`, `filter`, `range`, `True`, `False`, `None`) and `regex_extract(pattern, text, default)`.

## Named sessions

Agent and LLM steps can share conversations using **named sessions**. Steps with the same `session: <name>` reuse the same session across loop iterations and across steps, continuing the conversation instead of starting fresh.

```yaml
implement:
  executor: agent
  session: impl
  prompt: "Implement: $spec"
  loop_prompt: "Tests failed:\n$failures\nFix the issues."
  max_continuous_attempts: 5
  inputs:
    spec: $job.spec
    failures:
      from: run-tests.failures
      optional: true
  outputs: [result]
```

| Field | Type | Default | Description |
|---|---|---|---|
| `session` | string | --- | Named session. Steps with the same name share a conversation |
| `loop_prompt` | string | --- | Alternate prompt template on attempt > 1 (falls back to `prompt`) |
| `max_continuous_attempts` | int | --- | After N iterations, force a fresh session |
| `fork_from` | string | --- | Fork an independent session from the completion tail of a named **step** (see below) |

**Cross-step session sharing:** Steps with the same `session` name share a conversation — no special input bindings needed:

```yaml
steps:
  plan:
    executor: agent
    agent: claude
    session: main
    prompt: "Plan: $spec"
    inputs: { spec: $job.spec }
    outputs: [plan]

  implement:
    executor: agent
    agent: claude
    session: main
    after: [plan]
    prompt: "Implement the plan."
    inputs:
      plan: plan.plan
    outputs: [result]
```

**Forking sessions:** Use `fork_from: <step_name>` to create an independent session from a specific step's completion tail:

```yaml
  review:
    executor: agent
    agent: claude
    session: review_session    # forks MUST declare their own fresh session
    fork_from: plan            # STEP name (not a session name) — the snapshot anchor
    after: [plan]              # fork target must be in after: chain
    prompt: "Review the plan critically."
    outputs: [feedback]
```

**`fork_from` rules** (enforced by the parse-time validator):
- `fork_from` references a **step name** or `$job.<input>` (where the input has `type: session`). The target step must declare its own `session:`.
- `fork_from` without `session:` is legal — this creates an **ephemeral one-shot fork** (transient session, no downstream can continue it). Declare `session:` only when another step needs to continue the forked session.
- The fork target must appear in the forking step's `after:` chain.
- `max_attempts > 1` and `cache:` are prohibited on session-writers and fork sources. One-shot agent steps (no `session:`, no `fork_from:`) may retry freely.

**Ergonomic inferences (§9.7.5):** Three things are auto-inferred on fork steps to reduce boilerplate:
1. **`agent: claude`** is inferred from `fork_from:` on agent steps — the fork mechanism is inherently claude. Parent session writers still need explicit `agent: claude`.
2. **`working_dir`** is inherited from the fork source step when not set — the fork must run in the same project context as the session snapshot.
3. **`flow.inputs:`** on embedded `for_each` sub_flows is inferred from the parent step's `inputs:` bindings + `item_var` when not declared. `_session` sources infer `type: session`; all others infer `type: str`.

The engine snapshots the fork target's session state atomically at its completion under an exclusive file lock, guaranteeing forks see the parent's completion-tail state even if downstream writers keep mutating the live session. **Multiple chain roots** on the same forked session are permitted when their `when:` clauses are pairwise mutex (conditional rejoin pattern).

### The `_session` virtual output

Any step with `session:` declared automatically exposes a `_session` virtual output. It resolves to the session's snapshot UUID — a handle you can pass to other steps or sub_flows via input bindings. Consuming `_session` triggers the engine to snapshot the source step's session at completion.

Use `_session` to pass session context to `for_each` sub_flows without hardcoding step names:

```yaml
build_context:
  executor: agent
  agent: claude
  session: research
  working_dir: ./myproject
  outputs: [angles]
  prompt: "Analyze this codebase. Propose 3 angles."

explore:
  for_each:
    items: build_context.angles
    item_var: angle
  inputs:
    context: build_context._session    # captures the session snapshot
  flow:
    steps:
      deep_dive:
        executor: agent
        fork_from: $job.context        # forks from the passed-in session
        outputs: [result]
        prompt: "Deep-dive into: $angle"

synthesize:
  executor: agent
  agent: claude
  session: research
  after: [explore]
  outputs: [result]
  inputs:
    findings: explore.result
  prompt: "Synthesize: $findings"
```

The sub_flow declares `fork_from: $job.context` — the `$job.` prefix reads the session UUID from the job's inputs. For standalone flow files (not inline), declare the input with `type: session`:

```yaml
# standalone-explore.flow.yaml
inputs:
  angle:
    type: str
  context:
    type: session
steps:
  deep_dive:
    executor: agent
    fork_from: $job.context
    outputs: [result]
    prompt: "Deep-dive into: $angle"
```

For embedded sub_flows (inline `flow:` blocks), the input types are inferred automatically from the parent's bindings — no explicit `inputs:` block needed.

## Caching

Opt-in, content-addressable caching for step results:

```yaml
steps:
  fetch:
    run: 'curl -s "$url" | jq .'
    inputs:
      url: $job.url
    outputs: [data]
    cache: true                    # enable with default TTL

  analyze:
    executor: llm
    model: anthropic/claude-sonnet-4
    prompt: "Analyze: $data"
    inputs:
      data: fetch.data
    outputs: [analysis]
    cache:
      ttl: 30m                    # custom TTL (default: 1h for script, 24h for llm/agent)
      key_extra: v2                # bump to invalidate existing cache
```

Cache key = SHA-256 of resolved inputs + executor config. Same inputs + config = cache hit, skipping execution entirely.

**Default TTLs:** script = 1 hour, llm/agent = 24 hours. External, poll, and emit_flow steps are never cached.

**Bypass cache** for a specific step in a single run:

```bash
stepwise run my-flow --rerun fetch
```

**Manage cache:**

```bash
stepwise cache stats                     # entries, hits, size
stepwise cache clear                     # clear all
stepwise cache clear --step fetch        # clear one step
stepwise cache debug my-flow fetch --input url=https://...  # inspect cache key
```

## Input variables and config variables

Flows support two kinds of declared variables, both mapping to `$job.*` input bindings at runtime:

- **`inputs:`** — per-run parameters that change every job (e.g., a topic to research, a URL to fetch). Shown in the run dialog, passed via `--input` on the CLI.
- **`config:`** — set-and-forget settings configured once and reused across runs (e.g., API keys, model names, persona prompts). Saved to `config.local.yaml`, shown in the settings panel.

Both use the same field schema (`description`, `type`, `default`, `required`, `example`, `options`, `sensitive`).

```yaml
inputs:
  topic:
    description: "Subject to research"
    type: str
    required: true

config:
  persona:
    description: "Your AI persona"
    type: str
    required: true
    example: "You are a researcher..."
  api_key:
    description: "Service API key"
    sensitive: true                # masks in output, resolves from STEPWISE_VAR_API_KEY
  max_rounds:
    type: number
    default: 5
  voice_style:
    type: choice
    options: [conversational, formal, casual]
    default: conversational
```

**Resolution priority** (highest wins): `--input` > inputs (run dialog) > `config.local.yaml` > `STEPWISE_VAR_{NAME}` env vars > config/input defaults.

**`config.local.yaml`** only stores `config:` values. Input values are transient — passed at run time. Use `stepwise config init` to scaffold a `config.local.yaml` from the flow's `config:` block (it does not include `inputs:` variables).

## Requirements

Declare external tool dependencies in a top-level `requires:` block.

```yaml
requires:
  - name: ffmpeg
    description: "Audio processing"
    check: "ffmpeg -version"
    install: "apt install ffmpeg"
    url: "https://ffmpeg.org"
  - camofox                        # shorthand: just a name
```

Requirements are checked by `stepwise validate`, `stepwise info`, and `stepwise preflight`. They are advisory — they don't block `stepwise run`.

## Validation and preflight

Always validate before running. `stepwise validate` catches errors without executing anything:

```bash
stepwise validate my-flow.flow.yaml
```

**What it catches:**
- YAML syntax errors
- Missing step references in inputs and exit rules
- Invalid input bindings (referencing undeclared outputs)
- Unbounded loops (no `attempt` safety cap or `max_iterations`)
- Uncovered output combinations in external steps
- Type coercion warnings (`float()` on potentially None values)

**Preflight check** goes further — verifies runtime requirements:

```bash
stepwise preflight my-flow.flow.yaml
```

Checks that required API keys are configured, models are accessible, and script files exist.

**Treat warnings as defects.** A warning-free validate is the quality bar for production flows.

## Organizing flows into kits

When you have several related flows — for example, `plan`, `implement`, and `research` for a software development workflow — group them into a **kit**. A kit is a directory with a `KIT.yaml` manifest and flow subdirectories.

```
flows/swdev/
  KIT.yaml
  plan/FLOW.yaml
  implement/FLOW.yaml
  research/FLOW.yaml
```

The `KIT.yaml` declares kit metadata:

```yaml
name: swdev
description: Software development flows — plan, implement, research
```

Kit flows are referenced as `kit/flow`:

```bash
stepwise run swdev/plan --input spec="new feature"
stepwise run swdev/implement --input spec="build the API"
```

Use kits when flows share a common purpose and are typically installed together. Kits can be shared to the registry as a single package with `stepwise share swdev`. See [Flow and Kit Sharing](flow-sharing.md) for details.

## What's next

- [YAML Format](yaml-format.md) — complete field-by-field schema for every YAML option
- [Executors](executors.md) — deep dive into executor configuration and decorators
- [Patterns](patterns.md) — advanced idioms: session continuity, iterative delegation, fan-out/fan-in
- [Troubleshooting](troubleshooting.md) — error messages and fixes

---

## Reference Documentation

Each doc below is summarized. Fetch the full Markdown via the link when you need complete details.

### [Agent Integration](https://stepwise.run/api/docs/agent-integration/raw)

Guide for AI agents calling Stepwise flows as CLI tools. Covers discovery via `stepwise schema`, blocking and async execution, handling response shapes, external step fulfillment, and exit code semantics.

### [Agent Session Continuity](https://stepwise.run/api/docs/agent-session-continuity-proposal/raw)

Design doc for optional inputs, session continuity (`continue_session: true`), and cross-step session sharing via `_session_id`. Explains how these reduce token waste in multi-turn agent workflows.

### [API Reference](https://stepwise.run/api/docs/api/raw)

Full REST API and WebSocket reference for the Stepwise server. Documents all endpoints for jobs, runs, flows, engine control, configuration, templates, and the real-time WebSocket protocol.

### [CLI Reference](https://stepwise.run/api/docs/cli/raw)

Complete CLI reference covering all commands: run, open, new, validate, job management, staging, server lifecycle, registry, configuration, and utilities. Also documents global flags, exit codes, and project hooks.

### [Stepwise vs Other Tools](https://stepwise.run/api/docs/comparison/raw)

Tradeoff guide comparing Stepwise against Temporal/Airflow, LangGraph, CrewAI, GitHub Actions, LangChain, and Dify. Focuses on Stepwise's zero-infrastructure model, declarative YAML, and external fulfillment primitives.

### [Agent Containment](https://stepwise.run/api/docs/containment/raw)

### [Executors](https://stepwise.run/api/docs/executors/raw)

Reference for the six executor types: script, LLM, agent (ACP), external (human input), poll, and mock_llm. Also covers decorators (timeout, retry, fallback) and includes a decision flowchart for choosing the right executor.

### [Extensions](https://stepwise.run/api/docs/extensions/raw)

Guide for integrating external systems via shell hooks, webhooks, and WebSocket event streams with filtering and replay. Covers job metadata, fulfilling external steps, and design principles.

### [Stepwise Flow Reference](https://stepwise.run/api/docs/flow-reference/raw)

Canonical YAML format spec. Covers flow directory structure, all executor types, input bindings, exit rules, loops, for-each, flow composition, session continuity, decorators, and a 17-item gotchas list.

### [Flow and Kit Sharing](https://stepwise.run/api/docs/flow-sharing/raw)

Documentation for the Stepwise registry: publishing with `stepwise share`, downloading with `stepwise get`, searching, forking. Covers authentication, directory bundles, naming rules, and versioning.

### [Flows vs Skills](https://stepwise.run/api/docs/flows-vs-skills/raw)

Decision guide for when to use a flow (multi-step orchestration) versus a skill (one-shot context injection via SKILL.md). Includes a decision matrix and explains how they compose together.

### [Using Stepwise with Non-Claude Agents](https://stepwise.run/api/docs/how-to-generic-agents/raw)

Guide for running agent steps with non-Claude backends (Codex, Gemini) via ACP. Shows multi-agent patterns like adversarial review and model-appropriate task routing.

### [How to Build a Stepwise Extension](https://stepwise.run/api/docs/how-to-plugins/raw)

Step-by-step guide for building CLI extensions using git-style PATH discovery. Covers manifest JSON, Bash/Python examples, and interaction patterns (WebSocket, fulfillment, REST API).

### [How to Create a Stepwise Agent Skill](https://stepwise.run/api/docs/how-to-skills/raw)

Guide for authoring SKILL.md files that inject domain knowledge into agent sessions. Covers frontmatter format, file placement, and tips for effective skills.

### [Named Sessions + Fork Support — Implementation Plan](https://stepwise.run/api/docs/named-sessions-and-fork-plan/raw)

### [Stepwise Patterns](https://stepwise.run/api/docs/patterns/raw)

Nine advanced workflow patterns: file-based context passing, dynamic fan-out with `emit_flow`, escalation boundaries, progressive refinement loops, flow composition, error recovery, multi-job DAGs, and conditional gating.

### [Troubleshooting](https://stepwise.run/api/docs/troubleshooting/raw)

Centralized error reference: validator errors, engine runtime errors, and CLI errors. Includes a quick diagnostic workflow (validate -> preflight -> logs -> status).

### [Use Cases](https://stepwise.run/api/docs/use-cases/raw)

Five complete workflow examples with full YAML: podcast production, research synthesis with fan-out, deploy pipeline with approval gate, code review with feedback loops, and data pipeline with retries.

### [Web UI](https://stepwise.run/api/docs/web-ui/raw)

Documentation for the web dashboard: job list, DAG view, four view modes, step detail panel, external input fulfillment, canvas view, flow editor with AI chat, and settings.

### [Stepwise YAML Workflow Format](https://stepwise.run/api/docs/yaml-format/raw)

Complete field-by-field YAML specification. Covers all step types, input bindings, exit rules, loops, for-each, flow composition, conditional branching, session continuity, decorators, and 17 common gotchas.