Codex Cloud workers
Intendant treats a Codex Cloud container as an ephemeral worker lease, not as a permanent federated peer. The provider task and a live network attachment are separate pieces of state:
- Provider state comes from the authenticated local
codex cloudCLI: queued, running, finished, failed, cancelled, or unknown. - Attachment state describes optional live access to the task container: not requested, awaiting, connected, disconnected, or expired.
This distinction matters because a task can become ready or error while a
container is still reachable for a short time. Conversely, the provider can
reclaim a container without producing the orderly disconnect expected from a
durable Intendant daemon. A terminal provider task therefore never proves that
a live attachment still exists.
provider lane (codex cloud CLI) attachment lane (broker/operator)
─────────────────────────────── ─────────────────────────────────
queued → running → finished not requested → awaiting → connected
↘ failed │ │ │
↘ cancelled │ terminal task │ TTL lapses or
│ or broker loss │ terminal + stale
▼ ▼ ▼
(unchanged) disconnected → expired
What a worker really is (runtime model)
Empirical testing (2026-07-24) sharpened the model. Three kinds of state must never be conflated:
| Layer | What it is | What we observed |
|---|---|---|
| Environment/setup cache | Prepared container state from the setup script | Materialized into separate workers (different hostname/boot id) with identical dependency content; this is what the documented “up to 12 hours” covers |
| Task workspace state | The repo diff and filesystem artifacts of one task and its follow-ups | A warm same-task follow-up kept a 336 MB ignored cargo target/ and ran an identical build 68x faster (189s → 2.8s) |
| Live worker state | CPU, RAM, PIDs, sockets, tunnels | Warm continuity measured across ~8 minutes (same hostname, boot id, PID 1, inodes); allocation beyond a turn is not guaranteed |
The first controlled cold-resume checkpoint (one run, 2026-07-24; not yet
a measured boundary) sharpened the downside: after ~34 quiet minutes a
same-task follow-up landed on a replacement worker — different hostname,
boot id, and PID 1 — and every tested ignored/task-created path was gone
(target/, /root, /tmp probes, heartbeat, the 336 MB cargo tree); only
the selected repository revision survived (selected_state_only). One
observation does not establish an eviction timeout, but it proves the loss
mode is real: an external build cache is a requirement for reuse that must
survive replacement, not an optimization.
Compute capacity is a lease property
Do not treat the environment name as a machine size. OpenAI’s published Cloud environment contract describes isolated containers and setup caching, but does not specify a fixed CPU/RAM allocation or expose a machine-size selector. Measured capacity has varied between tasks. Intendant therefore records the worker’s current effective parallelism in every attachment fingerprint; that number is evidence about this lease, not a promise about the next one.
For example, the 2026-08-01 acceptance worker exposed CPUs 0-2, so Linux
nproc reported 3, while its cgroup cpu.max was 200000 100000: a hard
quota of two CPU-core equivalents. Rust’s quota-aware
available_parallelism() and the Intendant fingerprint correctly reported 2.
Use the fingerprint value for scheduling and performance expectations. Remote
execution can still be worthwhile when its purpose is to remove load from
home, but it must not be advertised as faster than a particular local machine
without a measurement from the attached lease.
Consequences Intendant encodes:
- An environment is a template, not a machine. New tasks — sequential or
concurrent — get isolated workers; nothing crosses
/root,/tmp, or process state between tasks. Cross-task build reuse needs a remote cache, not wishful thinking about a shared box. - Same-task follow-ups are the warm lever.
intendant codex-cloud followup(below) reuses the same warm worker and its ignored build artifacts while the worker survives. Keep repeated work in one task — and treat anything that must outlive a possible replacement as needingpullor an external cache. - Warmth is tracked honestly. Every lease derives
warm(actively running, or last activity within ~10 minutes — just past the measured window),unknown(through the 12-hour setup-cache horizon), orcold. Refreshes detect web-driven follow-ups as terminal → running → terminal flaps and count completed turns. The label is a heuristic from measured behavior, never a guarantee — the observed ~34-minute eviction sits well inside theunknownband, which is exactly why that band does not claim warmth. (probe --taskbelow is the cheap way to measure instead of guessing; more checkpoints may later tighten these windows.) - The 12-hour figure is the setup cache, not the worker. Put expensive,
stable toolchain work in
setup.shwhere the cache amortizes it across workers; keep task-specific mutable state out of it.
Worker fingerprints
intendant codex-cloud probe --env <ENV_ID> submits a canned diagnostic
task whose only output is one file: a single-line JSON fingerprint
(hostname, boot id, PID 1 start, toolchain, sizes), measured fresh on every
probe turn. The fingerprint travels in the task diff — the one channel the
CLI reliably exposes — and refresh collects it automatically whenever a
probe task finishes a turn. pull also parses a fingerprint
opportunistically from any diff that carries one.
intendant codex-cloud probe --task <TASK_ID> is the cross-turn instrument:
it re-probes an existing task with a follow-up turn that rewrites the
fingerprint file. Matching hostname + boot_id + pid1_start confirm the
same booted worker; a mismatch against the recorded fingerprint is a
detected cold replacement — the displaced fingerprint moves into the
lease’s worker_history and cold_replacements_observed increments (shown
by status and the dashboard card). This turns the runtime findings’
cold-resume methodology into a one-command check.
Controller commands
The commands below use the user’s existing Codex CLI authentication. Intendant
does not copy Codex credentials into the cloud container, and every provider
subprocess runs in a disposable working directory (the upstream CLI writes an
account-bearing error.log into its cwd).
# Verify the CLI and Cloud authentication.
intendant codex-cloud doctor
# Submit a task. Use -- to keep the task prompt separate from wrapper flags.
intendant codex-cloud exec \
--env <environment-id> \
--branch feature/example \
-- "Run the requested checks"
# Refresh the provider-owned lease store.
intendant codex-cloud list
intendant codex-cloud list --json
# Inspect a tracked task or its diff.
intendant codex-cloud status task_e_...
intendant codex-cloud diff task_e_...
# Bring a finished task home (see below).
intendant codex-cloud pull task_e_...
# Fingerprint a worker; re-probe a task to detect cold replacement.
intendant codex-cloud probe --env <environment-id>
intendant codex-cloud probe --task task_e_...
# Continue a finished task with a new turn (see "Follow-ups" below).
intendant codex-cloud followup task_e_... -m "Now also fix the tests"
# Attach the live worker to this daemon (direct mTLS by default; see below).
intendant codex-cloud attach task_e_... --home-url wss://your-host:8765 --send
# Drop terminal leases with no live attachment (default: older than 7 days).
intendant codex-cloud prune
intendant codex-cloud prune --all
list shows the provider’s current window plus any tracked lease with a
live attachment (awaiting/connected) that has fallen out of that window —
liveness outlives the provider’s list. Its REMOTE column says whether this
daemon has a usable live command channel; --json exposes that independent
fact as remote_compute_usable. The separate CACHE column carries the
derived warmth label (warm/unknown/cold; JSON field warmth). A lease can
therefore be cache-warm but remote-compute-offline. When the provider returns a
pagination cursor, list prints the ready-made --cursor invocation for the
next page, and --json carries it as cursor.
The lease store defaults to
$XDG_DATA_HOME/intendant/codex-cloud/leases.json (or the platform data
directory). INTENDANT_CODEX_CLOUD_STATE overrides the exact path, and
INTENDANT_CODEX_COMMAND overrides the Codex executable. Every
read-modify-write of the store takes a sidecar advisory file lock
(leases.json.lock), so concurrent CLI invocations, the daemon’s MCP tools,
and the dashboard route cannot clobber each other’s updates — and each
terminal transition is observed by exactly one refresher.
Pulling results home
pull closes the loop: it fetches the task’s unified diff through the Codex
CLI (in the disposable directory, never inside your repository) and applies it
with git apply --3way onto a fresh branch in a new git worktree under
.intendant/worktrees/:
intendant codex-cloud pull task_e_... # branch codex-cloud/task_e_...
intendant codex-cloud pull task_e_... --attempt 2 # best-of-N: pick an attempt
intendant codex-cloud pull task_e_... --branch fix/cloud-result --dir ../review
Nothing is committed: the worktree is left for review, and a conflicted
three-way apply leaves standard conflict markers with the conflicting paths
listed. A diff that applies nowhere removes the branch and worktree again. The
upstream codex cloud apply command is deliberately not wrapped — it would
either run in the disposable cwd (a no-op) or inside your repository (the
error.log hazard); piping diff into our own git sidesteps both.
Follow-ups: continuing a task’s warm worker
A follow-up appends a new user turn to an existing task, and it is the warm
lever: an active turn holds the task’s worker, and a worker that kept its
ignored build artifacts rebuilt an identical tree 68× faster in the 2026-07-24
measurements. The product supports follow-ups, but the public Codex CLI has no
verb for them (its Cloud surface is exec/status/list/apply/diff;
upstream issue #24777 is an
unassigned proposal). followup therefore rides the same private backend the
web UI uses — empirically validated end to end — under a deliberately
conservative contract:
intendant codex-cloud followup task_e_... -m "Now also fix the tests"
intendant codex-cloud followup task_e_... --json < prompt.txt # stdin keeps prompts out of shell history
- Auth is the Codex CLI’s own ChatGPT login (
auth.jsonunder$CODEX_HOME, default~/.codex) — no browser, no cookies, no separate credential. The bearer token and account id are read into process memory for the two requests and are never printed, logged, or serialized into receipts. API-key-only Codex auth cannot drive Cloud follow-ups. - Idle tasks only, serialized per task. A fresh provider refresh (or, for tasks outside the list window, the upstream status verb) must show the task terminal before anything is sent; a per-task sidecar lock serializes concurrent invocations machine-wide.
- The parent turn is resolved immediately before sending from the task
detail’s
current_turn_id, and must be an assistant turn — the validated behavior is HTTP 404 for anything else. - Fail closed on drift. HTTP 404/409/422, a missing
current_turn_id, or a 200 response that no longer references the task are reported as compatibility breaks of the private schema — never retried around. When upstream ships an official follow-up command, prefer it and retire this lane. - The lease learns immediately: an accepted follow-up records a running edge (warmth stays warm, and the next refresh’s terminal edge counts the turn), and the receipt carries the new turn ids the response referenced.
INTENDANT_CODEX_CLOUD_BACKEND overrides the backend base URL (tests point
it at a local stub; the default is the production web backend).
Attaching a worker to home
A worker has no inbound reachability and no durable identity, so attaching it inverts the usual peer pairing. Home runs the whole ceremony:
# Mint a single-use token bound to the task and deliver the attach prompt
# as a follow-up turn into the warm worker:
intendant codex-cloud attach task_e_... --home-url wss://your-host:8765 --send
# Or print the prompt to deliver by hand (task page, initial submit):
intendant codex-cloud attach task_e_... --home-url wss://your-host:8765
The composed prompt tells the worker to run intendant codex-cloud agent
through run-worker.sh (task-local state), with the token on stdin — never
argv. The agent then:
- generates a keypair in its task-local state root (the private key never leaves the worker; the daemon signs a public key, never mints one);
- redeems
{token, public key}at home’s publicPOST /api/codex-cloud/enrollroute and receives a client certificate whose identity record carries the system-issuedcloud-workerprofile and a hard expiry (--identity-ttl-s, default 3600); - dials home’s dedicated
/api/codex-cloud/attachWebSocket over mTLS, pinned to the fingerprint baked into the prompt, and holds the socket in the foreground. The accepted socket is the attachment: the lease flipsconnectedwhile it lives anddisconnectedwhen it dies.
Security shape: the enrollment token is stored hashed, minutes-TTL
(--token-ttl-s, default 900), bound to one task, and burned atomically on
first redemption — an unknown, used, or expired token refuses identically,
and the public doorbell is rate-limited. The cloud-worker profile is
recognized by every enforcement path but grants no operation at all
(strictly less than presence-only) and is never operator-assignable; the
gateway routes an attaching cloud-worker socket to the attachment lane
before any dashboard grant exists, so the certificate authenticates exactly
one thing: this heartbeat. Authority over the worker flows home→worker in
later slices; the worker’s inbound authority on home stays nothing.
The worker’s egress must be able to reach --home-url: in a
network-restricted Codex Cloud environment, add the home host to the
environment’s allowlist or the dial is refused before TLS.
HTTPS proxies and TLS-terminating reverse proxies
The worker honors HTTPS_PROXY / https_proxy (then ALL_PROXY /
all_proxy) for its WebSocket by opening an HTTP CONNECT tunnel itself;
NO_PROXY / no_proxy bypasses matching hosts. This matters in Codex Cloud:
an ordinary tungstenite WebSocket dial does not consult those variables and
would try a blocked direct socket even though HTTP requests work.
Prefer raw TCP passthrough when the egress proxy and public ingress both carry it: direct mTLS then stays end to end. If the only reachable ingress is an HTTPS reverse proxy that terminates TLS before Intendant, opt in explicitly:
intendant codex-cloud attach task_e_... \
--home-url wss://your-host.example/api/codex-cloud/attach \
--tls-terminated-proxy --send
For automatic remote_command acquisition, configure both:
INTENDANT_CODEX_CLOUD_HOME_URL=wss://your-host.example/api/codex-cloud/attach
INTENDANT_CODEX_CLOUD_TLS_TERMINATED_PROXY=1
In that mode the worker validates the public endpoint with normal WebPKI instead of pinning Intendant’s inner TLS certificate. It still generates the same task-local P-256 key and receives the same zero-authority client certificate. The security-critical enrollment request carries the same bounded JSON in its normal POST body and in a base64url header because some managed egress proxies have been observed to preserve request headers while dropping POST bodies. Informational worker fingerprint data follows on the authenticated attachment hello so neither duplicate exceeds managed custom-header ceilings. Home accepts the header only as a body fallback and requires the two byte copies to agree when both arrive; the single-use token never enters the URL. Every WebSocket attempt signs a transcript containing the exact attachment path, certificate fingerprint, task id, random nonce, and current timestamp. Home verifies it against the public key stored at enrollment and atomically consumes the nonce; a captured request cannot be replayed. The gateway recognizes this proof only on the dedicated Cloud attachment path, routes it directly to the closed worker bridge, and never creates a dashboard principal or IAM grant. Direct mTLS remains accepted and takes precedence.
This flag is deliberately not automatic. The selected WebPKI endpoint and
the component terminating TLS can see and relay attachment traffic, so use it
only for infrastructure the owner trusts. With Tailscale HTTPS Funnel, TLS is
terminated by tailscaled on the home machine and then reverse-proxied to the
local Intendant HTTPS listener; that local Tailscale process is the additional
trusted component. A representative setup is:
tailscale funnel --bg --https=443 https+insecure://127.0.0.1:8765
The public enrollment token remains single-use and short-lived, and the worker proof still grants no authority over home. The explicit trust choice is in the other direction: the WebPKI endpoint is accepted as the home allowed to send commands into that ephemeral worker.
Terminal on a live worker
A connected lease renders a Terminal button on its Cloud-card row (and
appears in the Terminal tab’s host picker as cloud:<task_id>). The shell
runs inside the worker: the dashboard’s ordinary terminal frames, after
clearing the same per-frame IAM gate a local shell demands (opening also
requires shell.spawn), are bridged by this daemon over the task’s
attachment socket; the worker serves them from its own PTY registry and
streams output frames back. The browser only ever talks to home — a worker
behind a cloud egress allowlist can never terminate a direct browser
connection, so the peer-terminal WebRTC path deliberately does not apply.
Fail-closed rules: the bridge forwards only terminal request kinds to the
worker, accepts only terminal reply kinds back (anything else off the
socket is dropped — the worker’s inbound authority on home stays nothing),
and routes replies solely to dashboard subscribers of that cloud: host.
Worker shells are unscoped (ShellSpawnPolicy.scope = None): the container
is the sandbox and home’s principal owns the worker, so the Landlock/
Seatbelt scoped-shell machinery never engages. Sessions survive a socket
reconnect (the worker keeps its registry across redials) and die with the
task turn or the identity expiry. Sharing is refused on cloud hosts, and
the bridge rides the dashboard tunnel only — the legacy /ws fallback
serves local terminals exclusively. Because the tunnel is the only lane,
opening a cloud terminal starts the dashboard-control transport on
demand when it is not already up (the default browser posture keeps the
legacy /ws as the event lane and no tunnel): the transport comes up as a
data lane, the event-lane preference is untouched, and the queued open
replays automatically once it connects. On the worker, each terminal key
keeps exactly one output forwarder — a re-open (browser reload, host
round-trip) replaces the listener on the surviving PTY rather than
stacking a second one, which would double every output chunk.
Live view of a worker (display)
A connected lease also renders a View button: the worker starts (or
attaches) a virtual display — Xvfb on a Linux container (installed by
setup.sh; the synthetic test card under the mock rig pair) — captures it
with the standard X11 backend, and streams it through the real tile
pipeline into a single-subscriber socket stream over the attachment
(DisplaySession::spawn_tile_socket_stream: tile mode only, no video
fallback, whole-snapshot delivery, 10 s re-anchor). Home bridges the
frames onto the dashboard tunnel (display_open/display_close under
display.view; tile frames arrive as display_tiles), and the Cloud
card’s viewer paints them with the same transport-agnostic tile
compositor the peer display path ships. Pointer and keyboard input ride
the existing display_input frame (display.input) with a
cloud:<task_id> host, delivered to the worker’s ordered input queue and
injected via XTEST.
WebRTC is deliberately absent from this path: a cloud worker can neither
accept nor dial a peer connection (no inbound reachability; passive-only
ICE-TCP; UDP-only TURN), so the attachment socket is the one lane —
which also means the worker never runs video encoders at all
(disable_video_bank): tiles are pure-Rust encodes, and capture idles to
keepalive cadence when no viewer is subscribed. The same fail-closed
rules as the terminal apply: only display reply kinds cross back from
the worker, routed solely to the viewing connection.
Computer use on a live worker
execute_cu_actions accepts display_target: "cloud:<task_id>": the whole
batch inverts over the task’s attachment as one cu_execute frame, the
worker runs the standard CU executor against its own display session
(screenshots and input ride the session — the same virtual display the
View button streams — with an unscoped actor, since the container is the
sandbox), and the correlated cu_result carries the reduced outcome
home: per-action status lines in the usual ok/injected/failed
vocabulary, the observation description, and the trailing screenshot.
Normalized coordinate_space batches are denormalized on the worker
against its own display size. The MCP caller is gated on home exactly
like a local CU call; the worker applies no further gates (home’s
authority over the worker is total), and cu_result joins the closed
reply-kind allowlist like every other worker frame. A worker without a
display capability answers with the named error from the display
resolver rather than guessing.
Provider-neutral remote commands
An acquired or explicitly attached worker can run a bounded, non-interactive
command for any session on the daemon’s machine. The implementation is one
daemon-side tool named remote_command, not codex_cloud_shell — Codex
Cloud is only the first host adapter — and its delivery is the
intendant ctl remote verb family plus the intendant-remote-compute
skill: native Intendant sessions keep the built-in tool, while external
supervised backends and unsupervised harness shells reach the same lane
through ctl. The schema deliberately does not ride supervised MCP session
toolsets (context rent; ratified 2026-08-07) — profile shaping only hides
the listing, so the daemon surface keeps answering tools/call by name and
intendant ctl tools schema remote_command still serves the contract.
The job vocabulary has four operations: start, status, wait, and
cancel. start returns immediately with a daemon-local job id; status
polls it, one wait operation waits for at most 60 seconds, and cancel
terminates the worker process tree. Omitted host (auto) reuses a live
worker whose environment and exact Git revision match, or submits and
enrolls one. An operator can still select an already-connected lease with
--host cloud:<task-id>:
git push origin feature/example
intendant ctl remote start --branch feature/example --revision 0123456789abcdef \
-- cargo test -p intendant-core
intendant ctl remote wait remote-<id> --for 1800
ctl remote wait chunks bounded server-side waits until the job is
terminal or its --for budget is spent, prints the bounded remote
stdout/stderr, and exits 0 only for a succeeded job; daemon refusals
surface verbatim. The generic raw form remains for scripting the tool
directly:
intendant ctl tools call remote_command --args '{
"op": "start",
"argv": ["cargo", "test", "-p", "intendant-core"],
"branch": "feature/example",
"expected_revision": "0123456789abcdef",
"require_clean": true,
"timeout_s": 900
}'
An automatically acquired job remains in acquiring while a cold provider
task runs setup and attaches. Keep polling the returned job id rather than
submitting another job. Its job.acquisition object reports stage
(checking_for_worker, submitting_task, waiting_for_worker, attached,
provider_ended, or timed_out), whether the request coalesced, the selected
branch, task id and URL, provider status, attachment state, latest bounded
provider-refresh error, timeout, and absolute deadline. Matching concurrent
environment/revision/branch requests share one acquisition and followers see
the leader’s task and deadline.
Uncommitted or not-yet-pushed source uses an explicit working-tree snapshot:
intendant ctl remote start --source working_tree --cache durable_sccache \
-- cargo check --workspace
The contract is intentionally stricter than an interactive terminal:
- Commands are transported as an argv array and are not implicitly parsed by
a shell.
cwd, when present, is repository-relative and cannot escape the selected checkout. Explicit environment entries are additions to the worker’s agent-phase environment. The command does not inherit the attachment agent’s privateINTENDANT_HOME; a caller may deliberately supply a different value in the explicit environment. source: "git_revision"is the default and requiresexpected_revision. The worker refuses a different checkout; abbreviated object ids are accepted from 7 hexadecimal characters. An optionalbranchnames the pushed provider branch containing that revision; it takes precedence overINTENDANT_REMOTE_COMPUTE_BRANCHand a branch derived from the supervised project, but never weakens the revision check.source: "working_tree"captures a binary Git patch plus non-ignored untracked regular files relative toexpected_revision, orINTENDANT_REMOTE_COMPUTE_BASE_REF(defaultorigin/main) when omitted. Capture runs twice and proceeds only when the content-addressed bytes match. Ignored files — notably.envandtarget/— do not cross the attachment. The compressed transfer is capped at 64 MiB (128 MiB expanded, 16 MiB per untracked file, 4096 untracked files), chunked over the authenticated attachment, and verified by digest before the worker materializes an isolated Git worktree.require_cleandefaults to true. For a pushed revision it means a clean checkout; for a snapshot it means unchanged from the exact selected snapshot. Commands using one snapshot are serialized. A command that mutates selected source invalidates that prepared workspace; ignored build outputs such astarget/remain warm and reusable.cache: "durable_sccache"is explicit. By default, the worker starts a loopback-only WebDAV sidecar and carries cache objects over the existing authenticated attachment. Home stores them in an owner-private, per-repository namespace under$INTENDANT_HOME/remote-cache/sccache-v1/; the default ceiling is 20 GiB with oldest-file eviction. Each command receives an opaque relay capability valid only for that task and job. Home accepts no general filesystem operation from it: keys must have sccache’s content-addressed shape, objects are capped at 128 MiB, job writes at 8 GiB, chunks are ordered and digest-checked, and incomplete staging files are discarded. Cache frames use a private bounded route rather than the worker’s general reply broadcast. Thus a worker gets durable cache reuse without home credentials or general authority on home.INTENDANT_REMOTE_CACHE_HOME_DIRmay select another absolute home-side directory, andINTENDANT_REMOTE_CACHE_MAX_BYTESmay set a 256 MiB–1 TiB byte ceiling. Intendant configuresRUSTC_WRAPPER=sccache,CARGO_INCREMENTAL=0, and a stableSCCACHE_BASEDIRS, then reports hit/miss/write/error deltas. The loopback sidecar is automatically added to bothNO_PROXYspellings so a Cloud egress proxy cannot intercept its worker-local HTTP traffic. The default remainscache: "none". The Cloud environment must provide sccache 0.14 or newer; an explicit durable-cache job fails before the requested command when sccache or its relay cannot start. During a running build, sccache can still compile uncached after a backend read/write error; the result reports those error counters rather than pretending every compilation was cached.- Deployments that deliberately want the worker to contact an external
sccache backend itself can set
INTENDANT_REMOTE_CACHE_TRANSPORT=direct. Only in that mode does home map dedicatedINTENDANT_REMOTE_CACHE_variables for sccache and its documented credential families (SCCACHE_*,AWS_*,ACTIONS_*,ALIBABA_CLOUD_*, andTENCENTCLOUD_*) into the authenticated command. Direct mode requires an external backend, refuses task-localSCCACHE_DIR, and carries the same configuration into the build so an automatic sccache-server restart cannot silently fall back to local disk. Use it only when the Cloud egress path is known to preserve that backend’s requests byte-for-byte. - Stdout and stderr are each bounded to 128 KiB. On overflow the result keeps the first 32 KiB and the latest tail and marks that stream truncated. Timeout, cancellation, and attachment loss terminate the owned process tree. The result reports the exact terminal state, exit code when one exists, worker revision, duration, and whether the checkout became dirty.
- Jobs are owned by the supervised session that started them. Another session receives the same not-found response as an unknown id; an unrestricted local owner surface may inspect them. Jobs are in daemon memory and do not survive a home-daemon restart.
- The worker accepts command start/cancel frames only from its authenticated
home attachment. Home accepts only the correlated result frame back; the
cloud-worker identity still has zero authority over home. MCP call-time IAM
additionally requires
shell.spawn.
Automatic acquisition requires INTENDANT_CODEX_CLOUD_ENVIRONMENT and the
reachable INTENDANT_CODEX_CLOUD_HOME_URL; the environment bootstrap must
install the matching Intendant binary and allow egress to home. Set
INTENDANT_CODEX_CLOUD_TLS_TERMINATED_PROXY=1 only for the explicitly trusted
reverse-proxy mode described above. Optional
INTENDANT_REMOTE_COMPUTE_BRANCH supplies a fallback provider checkout and
INTENDANT_REMOTE_COMPUTE_IDLE_TIMEOUT_S controls retirement.
INTENDANT_REMOTE_COMPUTE_ACQUIRE_TIMEOUT_S bounds the separate cold-worker
wait: default 3600 seconds, clamped to 10–7200. The one-time enrollment remains
valid for at least that wait plus five minutes; after redemption, the worker’s
zero-authority identity lasts one hour. Intendant refreshes provider state
after 15 seconds and then once per minute (each probe is capped at 20 seconds),
so a terminal provider task fails acquisition early with its real status and
task URL. A timeout reports the last known provider/attachment state and does
not cancel the provider task. Only workers created by this daemon process
are auto-retired; manually attached workers are never retired behind their
operator’s back. Acquisition state is process-local, so a daemon restart may
leave an acquired task until its expiring enrollment/identity and provider turn
end.
Attachment lifecycle
The enrollment broker above records the attachment state for its workers; an external broker or operator can also record it manually:
intendant codex-cloud attachment task_e_... awaiting
intendant codex-cloud attachment task_e_... connected
intendant codex-cloud attachment task_e_... disconnected
Refreshes age attachments by three rules:
awaitingordisconnectedon a terminal task becomesexpired— the broker is gone or will never arrive.connectedcarriesattached_at_unix_msand expires after a staleness TTL (INTENDANT_CODEX_CLOUD_ATTACH_TTL_S, default 3600) unless re-asserted: recordingconnectedagain restarts the clock — a crashed broker cannot leave a leaseconnectedforever.connectedwithin the TTL survives even a terminal provider task, because reachability must be checked independently of provider state.
Terminal transitions land on the Agenda
Whoever refreshes the store and observes a task leave the live states —
queued/running → finished/failed/cancelled — parks a note on the
daemon’s Agenda: the task title, its URL, and the
ready-made pull command. The store lock guarantees each edge is observed
exactly once, so one finished task produces one note, whichever lane (CLI,
MCP tool, dashboard) happened to see it first. The bare CLI parks through the
local daemon’s lane when a daemon is up; without one, the printed notice is
the whole delivery. A task first seen already-terminal is history, not an
edge, and is never parked.
Dashboard card
The dashboard’s Sessions → Cloud subtab renders the lease store: provider
chip and attachment chip per lease (independent, like everything else here),
the provider’s task link, and the ready-made pull and followup commands
for terminal tasks. The default paint is a cached read; Sync with provider hits
GET /api/codex-cloud/workers?refresh=1, which re-syncs through the daemon
host’s Codex CLI and parks agenda notes for any transitions it observes. A
failed sync degrades to the cached view with the error shown — the card never
goes blank because the provider CLI is missing.
The subtab also carries the submit form — prompt, environment id,
optional branch/attempts/title, exactly the exec verb’s parameters.
Submitting posts POST /api/codex-cloud/submit (tunnel twin
api_codex_cloud_submit, Task-classed like every start-agent-work
surface), which rides the same submit_task lane as the CLI verb and the
MCP tool: the daemon host’s authenticated Codex CLI creates the task and
the worker lease is recorded before the response returns, so the form’s
immediate follow-up sync lists the new task even while the provider window
lags. Environment suggestions derive from the tracked leases (there is no
provider env-list verb); the last environment submitted from that browser
is remembered locally.
Remote Compute plugin
The dashboard’s Plugins & Skills destination carries Codex Cloud Remote Compute — a bundled, default-off plugin wrapping this chapter’s lane in a product surface. It is a declarative bundle, not an extension system: enabling it does two bounded things.
- Readiness. A cheap, mutation-free probe of the three prerequisites
automatic acquisition needs —
INTENDANT_CODEX_CLOUD_ENVIRONMENT, a validINTENDANT_CODEX_CLOUD_HOME_URL, and the daemon gateway TLS identity (or explicit TLS-terminating-proxy mode). Blocked layers name their fix on the card; the probe never mints enrollments, spawns the provider CLI, or submits tasks. Cached-lease and live-attachment counts render as information, not gates — acquiring workers on demand is the lane’s own job. - Skill materialization. While enabled AND ready, the shared
intendant-remote-computeagent skill — the workflow teaching that used to live in system prompts: route heavy platform-neutral work throughintendant ctl remote, never silently fall back to heavy local work,working_treesnapshots for iteration vs pushedgit_revisionfor authoritative validation, honest cache expectations — is installed into~/.agents/skills/and~/.claude/skills/with plugin provenance markers. Its description triggers on the WORK (a heavy platform-neutral build/test/lint sweep about to run locally), not on tool possession, so supervised backends, native sessions, and plain coding-harness sessions all route through it. Disabling (or losing readiness) sweeps exactly the Intendant-managed copies; a user’s same-named skill directory is never touched. Enable state lives under the daemon state root, survives restarts, and reconciles at every boot.
GET /api/plugins (tunnel api_plugins_list) serves the catalog — enabled
flag, derived lifecycle state (available / needs_setup / enabled /
setup_failed), readiness layers, per-skill install facts — and
POST /api/plugins/{plugin_id} (tunnel api_plugin_set_enabled,
Settings-classed) toggles a plugin, reporting the installer’s per-root
outcome in the same response. The remote_command tool itself remains
available and provider-neutral regardless of the plugin: the plugin governs
the teaching, not the capability.
MCP tools
The daemon’s full MCP tool profile exposes the provider operations
list_codex_cloud_workers, submit_codex_cloud_task, and
follow_up_codex_cloud_task. This lets an Intendant agent refresh worker
state, delegate a Codex task, or continue one — the full warm-builder loop
(submit → probe → follow up → pull) is drivable end to end by an agent using
the daemon host’s authenticated Codex CLI. These provider operations are
intentionally omitted from the compact/core tool profile; agents can discover
and invoke them through intendant ctl tools list and intendant ctl tools call. The list tool reports the same shape as list --json (window,
tracked-active, cursor, transitions) plus how many agenda notes it parked; the
follow-up tool returns the acceptance receipt (parent turn, new turn ids)
under the same fail-closed contract as the CLI verb.
remote_command is different: it does not create provider work or ask Codex
to reason. It spends shell authority on an attached compute host and is
IAM-classed as shell.spawn. Since 2026-08-07 its schema no longer rides
the compact/core profile either — supervised backends reach the lane through
"$INTENDANT" ctl remote (their injected session token binds the call to
their session, so job ownership and cache namespacing are unchanged), and
the unprofiled daemon listing keeps serving the schema for ctl discovery.
Claude Code, Kimi Code, Pi, and Codex therefore use the same attached Linux
worker without changing which model is doing the reasoning.
Environment bootstrap
Generate the bundle from the same Intendant revision used by the controller:
intendant codex-cloud bootstrap --output ./intendant-codex-cloud
Paste setup.sh and maintenance.sh into the matching fields in the Codex
Cloud environment settings. They are intentionally split by lifecycle:
setup.shinstalls Intendant, sccache, and the task-time launcher. It either builds the checked-out repository with Cargo or downloads a binary when bothINTENDANT_CLOUD_BINARY_URLand its mandatoryINTENDANT_CLOUD_BINARY_SHA256are configured.maintenance.shrefreshes the installation after a cached container resumes, clears the per-user task-runtime directory, and creates a new boot nonce.run-worker.shruns only during the agent phase. It creates fresh XDG and Intendant state roots under$XDG_RUNTIME_DIR(or a per-user/tmpdirectory), thenexecs the supplied foreground command without shell re-parsing.
Pinned binary fast path (recommended)
Building the release binary from source is the slow half of a cold worker:
on an observed two-vCPU lease it consumed most of the first turn (14m41s to
provider-ready, 2026-08-02), while a checksum-pinned download attaches well
inside it. Every tagged release publishes standalone Linux binaries as
PGP-signed, transparency-logged assets — intendant-linux-x86_64 and
intendant-linux-aarch64 — and the release notes carry the ready-made pin
block (v0.1.0 predates these assets; pin a newer release):
INTENDANT_CLOUD_BINARY_URL=https://github.com/intendant-dev/Intendant/releases/download/<tag>/intendant-linux-x86_64
INTENDANT_CLOUD_BINARY_SHA256=<from the release notes or the .sha256 asset>
Set both in the Codex Cloud environment (setup-time configuration, not
secrets). Take the SHA-256 from a release you have verified — gpg --verify against the committed RELEASE-SIGNING-KEY.asc and intendant hosted-verify --releases <tag> check every asset against its detached
signature and the public transparency log (see
Verifying a release); the hash
pin is what the container itself enforces. setup.sh refuses a mismatched
download, and a binary that cannot start fails the bootstrap loudly (the
trailing codex-cloud --help probe) instead of silently rebuilding.
Two honest bounds. The assets are built on ubuntu-24.04 and need that
image’s library baseline at runtime — glibc 2.39+, libvpx, libpipewire,
libxcb; the Codex universal image satisfies it, and an older container
keeps the source-build path by leaving the pin unset. And the pin is a
compatibility choice, not only a speed one: a source-built worker runs
whatever revision the task checked out, so its attachment agent floats
with the branch under test, while a pinned asset keeps the attachment
agent deterministic per environment and tracking the home daemon’s
release. A worker whose attachment protocol has drifted fails enrollment
with the remedy in the error text (unsupported enrollment request version N: this daemon speaks M — repin INTENDANT_CLOUD_BINARY_URL/_SHA256 to a matching release, or rebuild the worker from source), and the
authenticated attachment hello records the worker binary’s version, commit,
and target, so status shows what actually attached (worker binary: line).
For sccache, setup reuses any installed version 0.14 or newer. Otherwise it
downloads the pinned 0.15.0 official Linux musl archive for x86_64 or aarch64,
verifies its hard-coded SHA-256, and installs the single binary; compiling the
same pinned release with Cargo is only a fallback for unsupported or failed
downloads. Set INTENDANT_CLOUD_SKIP_SCCACHE=1 when the environment
deliberately must not install it. See the upstream
sccache v0.15.0 release.
The scripts can also be printed for direct pasting:
intendant codex-cloud bootstrap --print setup
intendant codex-cloud bootstrap --print maintenance
intendant codex-cloud bootstrap --print worker
Codex Cloud runs setup and maintenance in shells which finish before the agent phase. Do not start the Intendant daemon, Chisel, or another attachment supervisor there. Start the worker launcher in a task-owned background terminal and keep the supervisor in the foreground of that terminal:
~/.local/libexec/intendant-cloud/run-worker.sh -- <supervisor> <args...>
The supervisor is deployment-specific. It may start an Intendant daemon and an outbound tunnel or connect an edge transport to a broker, but it must:
- use a one-time, short-lived enrollment credential;
- keep its identity and certificates inside the launcher’s task-local runtime state;
- keep the public relay/domain allowlist exact;
- remain in the foreground so task cancellation tears it down;
- report connection loss so the controller expires the attachment.
The bootstrap scripts deliberately do not embed relay credentials, private keys, static peer identity, AWS details, or a fixed reverse port. Codex Cloud environment caches may be reused for up to 12 hours — that figure covers prepared container state, which is materialized into fresh workers, not a promise that any particular worker stays allocated — and Business/Enterprise caches can be shared by users with access to the environment. Secrets are available to setup scripts but are removed before the agent phase, so cached setup state is the wrong place for a per-task identity or enrollment secret. See the official Codex Cloud environments documentation.
Cache strategy on ephemeral workers
The 189 s → 2.8 s warm result above came from a surviving task-local
target/; it is valuable but disposable. The cold-resume observation proved
that a replacement worker can lose the entire ignored tree. Treat this like
an ephemeral hosted CI runner:
- Keep repeated commands on the same attached task while it remains warm.
- Put stable toolchain/package preparation in the environment setup cache.
- Use
cache: "durable_sccache"when cold replacement performance matters. The default attachment relay persists objects on home without exposing object-store credentials to the worker. A task-local sccache directory is never used because it is lost with the same worker astarget/. - Do not promise a fully warm Rust build from sccache alone. Existing cross-worktree measurements show it can repopulate identical dependency outputs, but local incremental workspace crates, build scripts, non-cacheable crate types, test binaries, and final links still need a fresh target tree. Correctness never depends on cache hits, and the job result exposes the measured cache deltas instead of claiming warmth.
For optional direct transport, the daemon-side prefixes are a custody
boundary, not new sccache option names: for example,
INTENDANT_REMOTE_CACHE_SCCACHE_BUCKET becomes SCCACHE_BUCKET, and
INTENDANT_REMOTE_CACHE_AWS_ACCESS_KEY_ID becomes AWS_ACCESS_KEY_ID only
inside the authenticated remote command. The worker container is a
shell-authority boundary, not a credential enclave: another same-UID process
could inspect those direct-mode values. Use only a cache-only principal
restricted to one cache namespace, never a general AWS/account credential or
any daemon/provider credential. Consult sccache’s upstream
configuration reference
for backend-specific variables and its Rust caveats
for what can and cannot be cached.
The practical split is therefore: Linux workers run platform-neutral
cargo check, tests, clippy, code generation, and other heavy computation;
the platform CI matrix remains authoritative, including the macOS runner.
Agents should still run small, targeted macOS checks when a change directly
touches Apple APIs, entitlements, the app bundle, or the repository’s
macOS-only deterministic WASM artifact path. Remote Linux success reduces
feedback time; it does not replace CI.
Current boundary
This integration covers automatic reuse/acquisition, explicit working-tree
snapshots, durable compiler-cache configuration, the job/control plane, the
safe setup/maintenance contract, and the enrollment ceremony that attaches a
live worker to home over direct mTLS or an explicitly trusted, proof-bound
HTTPS reverse proxy. The attachment carries terminal, tile display,
computer-use, bounded source-transfer, and provider-neutral remote-command
frames in addition to liveness; each direction has a closed allowlist, and
worker replies are never dispatched as authority on home. Both endpoints send
periodic WebSocket pings so a long command with no output survives idle egress
proxies and a dead connection surfaces promptly. Workers are
ephemeral enrollments with zero-authority expiring identities, not static
[[peer]] registry entries, and home must be reachable from the worker’s
egress allowlist (there is no relay tier).
Two boundaries remain deliberate. Linux results do not replace the cross-platform CI matrix or a small macOS-specific check when Apple APIs, entitlements, bundles, or deterministic macOS-generated WASM are touched. And remote compilation does not reduce the resident RAM used by Claude, Codex, Kimi, or Pi themselves on the supervisor; it moves their heavy child build/test processes, not their reasoning process.