Editable handoffs

The single idea this PR is built around: the AI's output is a draft, and the next stage reads your corrections.

Business view

Why an editable comment, and not a file

The team could have made the agent write its review to a file in the repo and asked humans to commit corrections. They chose a PR comment instead, for a practical reason: correcting a comment costs a click and thirty seconds, and anyone with access to the PR can do it. Committing a file costs a branch, a push, and a CI cycle. If the correction step is expensive, nobody does it, and the checkpoint is theatre.

So: the agent posts, you edit in place, you dispatch the next action, and the next action reads what is currently in that comment — not what the agent originally wrote.

What that looks like on the PR

Each stage owns exactly one comment, which is created the first time and overwritten on every subsequent run of the same stage. You will see at most three AI comments on a PR:

• 🔍 QA code review — from review
• 🧪 QA test plan — from plan
• 🧪 QA review (the results summary) — from test, fix, full-auto

The review and plan comments carry an explicit invitation in their headers:

### 🔍 QA code review (AI — edit freely; plan builds on your edits)

### 🧪 QA test plan (AI — edit freely; test runs exactly this)

Both are verbatim from the code (.github/workflows/sandbox-qa.yml:131-133).

The rules of editing, in plain terms

• Edit anything. Rewrite paragraphs, delete findings you disagree with, add

scenarios the AI missed. The whole comment body is handed over.

• Do not delete the comment if you want the next stage to see it. A deleted

comment means the stage runs with nothing from that source — silently, with no warning.

• Do not delete the hidden marker line at the top. It looks like

<!-- sandbox-qa-plan --> and is invisible in rendered markdown, but GitHub's editor shows it. It is how the workflow finds the comment. Remove it and the next plan run posts a second comment instead of updating yours.

• Re-running a stage discards your edits to that stage's own comment.

If you edit the plan and then dispatch plan again, the agent's fresh output overwrites your version. Edit, then move forward.

Technical view

The marker protocol

Every managed comment begins with an HTML comment that renders invisibly:

Reading edits back in

const { data: prs } = await github.rest.pulls.list({
  owner: context.repo.owner, repo: context.repo.repo, state: "open",
  head: `${context.repo.owner}:${context.ref.replace("refs/heads/", "")}`,
});
if (!prs.length) { core.info("no open PR — using stored state only"); return; }
const { data: comments } = await github.rest.issues.listComments({
  owner: context.repo.owner, repo: context.repo.repo, issue_number: prs[0].number, per_page: 100,
});
for (const [marker, file] of [
  ["<!-- sandbox-qa-review -->", "/qa-review.md"],
  ["<!-- sandbox-qa-plan -->", "/qa-plan.md"],
]) {
  const c = comments.find(x => x.body && x.body.includes(marker));
  if (c) fs.writeFileSync(process.env.RUNNER_TEMP + file, c.body.replace(marker, ""));
}

— .github/workflows/sandbox-qa.yml:64-82

The files land at $RUNNER_TEMP/qa-review.md and $RUNNER_TEMP/qa-plan.md, and the agent step points at them via QA_REVIEW_FILE and QA_PLAN_FILE (.github/workflows/sandbox-qa.yml:89-90).

Four properties follow directly from this code:

Missing comment ⇒ missing file, not an error. The if (c) guard means a deleted or never-created comment simply produces no file. Contrast with the old execute mode, which hard-failed:

if (!plan) { core.setFailed("no posted QA plan found — run plan-only first"); return; }

That setFailed is gone. The new behavior is deliberate — test is documented to auto-plan when no plan exists (.github/workflows/sandbox-qa.yml:10) — but it means a dispatch that silently did less than you expected now looks identical to a successful one. If you deleted the plan comment by accident, test will quietly generate a new plan and test that instead of telling you.

No open PR ⇒ proceed anyway. Line 72 downgraded a setFailed to a core.info. The workflow is dispatched on a ref, so a branch with no PR is a legitimate case; state comes from QA_STATE_DIR on the host instead.

First match wins. comments.find(...) returns the earliest comment containing the marker. Since the publish side uses the same find (.github/workflows/sandbox-qa.yml:119), reads and writes agree. But if a human quotes an AI comment in a reply — quoting preserves the marker — and that reply somehow precedes the original, the workflow would latch onto the reply. In practice the AI comment is always older, so this is a theoretical edge.

Only the marker is stripped. c.body.replace(marker, "") removes the first occurrence of the marker string. The human-facing header, the timestamp line, and the run link stay in the text handed to the agent (.github/workflows/sandbox-qa.yml:134-138 builds them, nothing strips them). The agent therefore receives a few lines of chrome above the real content.

Writing the editable comment

const upsertComment = async (marker, body) => {
  if (!pr) { core.info("no open PR — kept as artifact only"); return; }
  const { data: comments } = await github.rest.issues.listComments({ ... });
  const existing = comments.find(c => c.body && c.body.includes(marker));
  if (existing) await github.rest.issues.updateComment({ ..., comment_id: existing.id, body });
  else await github.rest.issues.createComment({ ..., issue_number: pr.number, body });
};

— .github/workflows/sandbox-qa.yml:114-122

Upsert, not append. One comment per stage, forever. The consequence is that previous versions are not preserved on the PR — if you want to compare this run's review to last run's, you need the workflow artifacts, not the comment thread.

The review/plan publish branch

if (isReview || isPlan) {
  const file = isReview ? "${{ steps.qa.outputs.QA_REVIEW }}" : "${{ steps.qa.outputs.QA_PLAN }}";
  const marker = isReview ? "<!-- sandbox-qa-review -->" : "<!-- sandbox-qa-plan -->";
  if (!file || !fs.existsSync(file)) { core.setFailed(`agent produced no ${action} output — see agent.log in the artifacts`); return; }
  let content = fs.readFileSync(file, "utf8");
  if (content.length > 60000) content = content.slice(0, 60000) + "\n\n…(truncated — full version in the workflow artifacts)";
  ...
  await upsertComment(marker, [ marker, header, "", `_${now} · [run](${runUrl}) · branch \`${branch}\`_`, "", content ].join("\n"));
  return;
}

— .github/workflows/sandbox-qa.yml:124-140

This block is the generalization at the heart of the PR. The old code handled one artifact (the plan); the new code parameterizes file, marker, and header so both stages share a path.

• Empty output is a hard failure (:128) — unlike the input side, the

output side does fail loudly, pointing at agent.log in the artifacts.

• 60 000-character cap (:130). GitHub's comment limit is 65 536; the

header and timestamp are added after the slice, so the total stays under. A truncated review is still valid input for plan — but it is the truncated text that gets fed forward, since the fetch step reads the comment, not the file. Long reviews silently lose their tail from the pipeline.

• return at :139 — review and plan never reach the issue-filing code.

Where the loop closes

review ──posts──► 🔍 comment ──you edit──► fetched at :76-82 ──► plan
plan   ──posts──► 🧪 comment ──you edit──► fetched at :76-82 ──► test
test   ──posts──► issues + 📋 summary ────────────────────────► fix

fix is the one stage whose input is not a PR comment — it reads the stored test results from the host-side state directory. So the two editable checkpoints are exactly the two reasoning steps, and the two execution steps are not editable. That is a coherent design: you correct judgement, not evidence.