In Depth: Foreground Execution & Runner Slots

Business view

A CI job can run work in one of two styles:

• Detached (fire-and-forget): the job kicks off a background task and then

exits immediately. The task keeps running on its own, outside the job's view. The job goes green fast — but "green" only means "I launched it," not "it finished."

• Foreground (attached): the job runs the work inside itself and stays

open until the work is done. It's slower to go green, but "green" means "the work actually finished," and you can watch it happen the whole time.

The old update-docs used detached. That's why you'd get a thumbs-up while the docs didn't exist yet — the "job finished, but where are my docs?" problem. This PR switches it to foreground, so the job now tells the truth about completion and streams live progress.

There's a catch, and it's the reason this couldn't just be flipped: a foreground job holds a build machine ("runner slot") for its entire run — potentially an hour. With only one build slot, that one long job would block the quick per-PR deploys everyone depends on. So a second slot was brought online, and the job's time limit was raised to fit the longer run.

Technical view

The exact diff

# .github/workflows/sandbox-ops.yml
   jobs:
     ops:
       runs-on: [self-hosted, homelab, dentolize]
-      timeout-minutes: 30
+      # update-docs runs the docs agent in the FOREGROUND (live progress in this
+      # job log; job success = site published) — can take up to ~1h when it also
+      # has to deploy a missing sandbox first. Other actions finish well within.
+      timeout-minutes: 90
       steps:
         ...
-            # Detached — the docs agent outlives this job (single runner slot);
-            # the refreshed site replaces the old one when the container ends.
-            update-docs) /opt/homelab/sandbox/bin/sandbox-docs dentolize "${{ github.ref_name }}" refresh --detach ;;
+            # Foreground — the job streams the agent's live activity and
+            # completes when the site is published. A second runner slot keeps
+            # PR deploys unblocked meanwhile. (PR-open auto-generation stays
+            # detached in sandbox.yml — nobody is watching that job.)
+            update-docs) /opt/homelab/sandbox/bin/sandbox-docs dentolize "${{ github.ref_name }}" refresh ;;

Two functional lines (timeout-minutes and the removal of --detach); the rest is comment. That's the whole PR.

What "runner slot" means here

All three sandbox workflows target the same self-hosted runner label set:

runs-on: [self-hosted, homelab, dentolize]

(.github/workflows/sandbox-ops.yml:44, sandbox.yml:40/:167, sandbox-qa.yml:57.)

A self-hosted runner executes one job per slot at a time. GitHub queues jobs and dispatches them as slots free up. So concurrency between these workflows is governed less by their concurrency: groups (which are all different keys) and more by how many runner slots physically exist.

• One slot (old world): a foreground job that runs for an hour would starve

everything else — including PR deploys from sandbox.yml. That is precisely why update-docs was detached: to release the single slot in seconds. The cost was the misleading completion signal.

• Two slots (new world): one slot can hold a long, watched docs job while

the other stays free for PR deploys. This is what makes foregrounding safe — and it's asserted in the comment at .github/workflows/sandbox-ops.yml:62.

The concurrency groups (for completeness)

Each workflow has its own concurrency key, so they do not serialize against each other via concurrency — only via slots:

Consequence for update-docs: a second dispatch on the same branch queues behind the first (won't cancel it), because cancel-in-progress: false.

Why the timeout went 30 → 90

A detached job exits in seconds, so 30 minutes was never binding for it. Foreground, the job's lifetime is the agent's lifetime: ~15–40 min for docs, plus — per the comment — a possible full sandbox deploy first when none exists, pushing toward ~1 hour. 90 minutes gives headroom without being unbounded.

What was intentionally left alone

• sandbox.yml auto-generation stays detached (generate --detach,

sandbox.yml:66): no human watches a PR-open job, so holding a slot buys nothing. Explicitly called out in the new comment.

• sandbox-qa.yml already runs foreground with timeout-minutes: 300 and

is the other long job the second slot is meant to accommodate.

Verification & honesty flags

• The second runner slot is the linchpin of the safety argument but is

not in this repository — it's homelab runner configuration. If it isn't actually online, foregrounding reintroduces the deploy-blocking it claims to prevent. Verify two runners are registered before merge.

• sandbox-qa.yml:18–19 still describes a "single-slot self-hosted runner,"

which contradicts the two-slot claim. Stale comment; not fixed here.

• The sandbox-docs CLI's refresh behavior (including deploy-if-missing) is

on the runner, not in the repo — the timing/behavior claims can be confirmed only by running it. See For Quality for the test matrix.