Dentolize · Composable Sandbox QA Walkthrough
On this pageBusiness viewSession planThe rules to hand outCommon questions from the room

For Training

A ready-to-run session for engineers and QA staff. Roughly 45 minutes with a live pull request.

Business view

Learning objectives

By the end, a participant can:

  1. Explain why the QA agent was split into four actions.
  2. Dispatch each action from the Actions tab.
  3. Edit an AI-generated review or plan so the next stage uses their version.
  4. Read the results — issues, labels, summary tally — correctly, including the

two places it misleads.

  1. Say which actions can write to their branch, and choose accordingly.

The one sentence to open with

The AI still does all the work; you now get to correct it twice before it commits to anything.

The mental model

Draw this on the board and leave it up:

  review  ──►  🔍 comment  ──►  [ YOU EDIT ]  ──┐
                                                │
  plan    ◄───────────────────────────────────  ┘
    │
    └──►  🧪 comment  ──►  [ YOU EDIT ]  ──┐
                                           │
  test   ◄─────────────────────────────────┘
    │
    └──►  issues + summary  ──►  fix  ──►  updates the same issues

Two edit gates. They sit on the two reasoning steps (what does this change do? what should we try?) and not on the two execution steps (run it, repair it). That is the design principle: humans correct judgement, machines produce evidence.

Session plan

Part 1 — Why (5 min)

Ask the room: "Last time the QA agent misunderstood a change, what did you do about it?" The honest answer used to be "re-ran it and hoped", because the first artifact you could touch was the test plan — already two reasoning steps downstream of the code. The AI's understanding was never visible.

That is the gap this closes. review surfaces the understanding, and it is editable.

Part 2 — Dispatch review (10 min)

Have everyone open a real PR on a branch with a live sandbox.

  1. Actions tab → Sandbox QA → Run workflow.
  2. Select the branch — this is the branch that gets reviewed. Getting this

wrong is the single most common mistake; the workflow runs on the ref you pick, not on a PR you have open in another tab.

  1. Action: review. Scope: leave all — it only affects test, and

GitHub shows the dropdown regardless (.github/workflows/sandbox-qa.yml:37-44).

  1. Run it. Watch the job log — agent progress streams live.

When the comment appears, read it together. Ask: is any of this wrong?

Part 3 — Edit and hand off (10 min)

This is the core exercise. Do it slowly.

  1. Click Edit on the 🔍 comment.
  2. Point out the invisible first line: <!-- sandbox-qa-review -->. **Rule:

never delete this.** It is how the workflow finds the comment (.github/workflows/sandbox-qa.yml:77). Delete it and the next run posts a duplicate instead of updating yours.

  1. Have each participant insert a distinctive sentinel sentence — something like

"IMPORTANT: also consider the VAT-exempt invoice path."

  1. Save. Dispatch plan.
  2. When the plan comment appears, find the story that came from the sentinel.

That moment — seeing your own sentence turn into a test story — is the lesson. Mechanically, the workflow read the current comment body off the PR and handed it to the agent (.github/workflows/sandbox-qa.yml:76-82).

Part 4 — Edit the plan, then test (10 min)

  1. Edit the 🧪 plan comment. Delete all but two or three stories — this keeps the

exercise short and proves the point.

  1. Dispatch test.
  2. When it finishes, show the three outputs together:
  • Issues — one per story, titled [QA-n] <emoji> <title>
  • Labels — ai-automated and pr-<number>
  • Summary comment — the tally

Count the issues. It should match the number of stories they left in the plan.

Part 5 — Reading results honestly (10 min)

Two traps. Teach them explicitly; they are not obvious.

Trap 1: closed ≠ passed. Issues close on PASS, FIXED, and SKIPPED (.github/workflows/sandbox-qa.yml:176). A story the agent never attempted looks exactly like one that succeeded.

Trust the tally, not the issue list. 12 stories: 9 ✅ · 2 ❌ · 0 🔧 · 1 ⏭️ — the ⏭️ is the only place skips show.

Trap 2: green ≠ tested. If the agent's stories.json is unparsable, the workflow logs a warning and finishes green with zero stories (.github/workflows/sandbox-qa.yml:156). A green run that tested nothing.

Then show fix: it updates the same issues in place with a **Re-test:** comment (:179-182) rather than filing new ones — and it is one of the two actions that commits to the branch.

The rules to hand out

Print these. They cover ~90% of the mistakes people make.

  1. Pick the right branch in the dispatch form. The action runs on the ref

you select.

  1. Never delete the <!-- sandbox-qa-… --> marker when editing.
  2. Edit forward, not sideways. Re-running a stage overwrites that stage's

own comment with new AI output. Edit review → dispatch plan. Do not edit review → dispatch review.

  1. Use test, not full-auto, after editing. full-auto regenerates

everything and ignores your edits (.github/workflows/sandbox-qa.yml:61).

  1. review, plan, test never commit. fix and full-auto do.
  2. Close old issues before re-testing. test does not deduplicate — two

runs means two full sets of issues.

  1. Read the tally, not the checkmark.
  2. Long runs block everyone. One runner slot, up to a 5-hour timeout. Cancel

wedged runs.

Common questions from the room

"Can I skip review and go straight to plan?" Yes. plan will build on whatever review comment exists, or generate its own understanding if there is none — the fetch step just writes no file when the comment is absent (.github/workflows/sandbox-qa.yml:80). You lose the first correction gate.

"What if my branch has no PR yet?" It runs. The workflow logs "no open PR — using stored state only" and keeps results in the run's artifacts (.github/workflows/sandbox-qa.yml:72, :115). Nothing gets posted, because there is nowhere to post it.

"Where do old versions of the comments go?" Nowhere on the PR — each stage owns one comment that is overwritten (.github/workflows/sandbox-qa.yml:114-122). Previous versions are in the workflow run artifacts, kept 14 days.

"My review is enormous — is all of it used?" Comments are capped at 60 000 characters (.github/workflows/sandbox-qa.yml:130). Beyond that it is truncated with a footer, and it is the truncated text that plan receives.

"How do I re-run just the failures?" Dispatch test with scope: failed-only.