Dentolize · Jira/Confluence QA Pipeline Walkthrough
On this pageWhat actually changed (scope of test)Test checklist — the workflow itselfEdge cases & failure modesThe big coverage boundary — say it out loudGaps between description and code (verify, don't assume)Suggested Stories to hand back to the epicSee also

For Quality

What to test in this PR, and where the edges are. Meta-note: this is a change to the QA tooling itself, so "quality" here means verifying the plumbing, not clinic features.

What actually changed (scope of test)

One file: .github/workflows/sandbox-qa.yml, +62 / −189. Everything to verify is in that file plus the contract it has with the off-repo sandbox-qa binary. You cannot test the binary from this repo — flag that as a coverage boundary.

Test checklist — the workflow itself

Dispatch & inputs

  • [ ] Actions → Sandbox QA offers exactly prepare, test, retest, fix; default is prepare (.github/workflows/sandbox-qa.yml:37-42).
  • [ ] No scope input remains (removed).
  • [ ] Dispatch is branch-scoped ($GITHUB_REF_NAME), not PR-event-driven (:64).

Agent step

  • [ ] Only GITHUB_TOKEN is passed as env (:60-61) — QA_SCOPE/QA_REVIEW_FILE/QA_PLAN_FILE are gone.
  • [ ] set -e present; the phase name is the third CLI arg (:63-64).
  • [ ] Non-zero agent exit fails the job — but post-steps still run (see below).

Summary comment (the meat of the diff)

  • [ ] Runs on always() && steps.qa.outputs.QA_STATE_DIR (:67) — verify it posts even when the agent step "fails," as long as a state dir exists.
  • [ ] No open PR → no comment, no error (:82-86). Test: dispatch on a branch with no PR; expect the QA work to run and the comment step to log "no open PR" and return cleanly.
  • [ ] Single upsert by marker <!-- sandbox-qa-report --> — running two phases produces one comment, rewritten, not two (:104-114).
  • [ ] Jira/Confluence links appear only when JIRA_EPIC_URL/CONFLUENCE_URL are non-empty (:95-96). Test the link-less path (agent emits neither) — comment should still be valid.
  • [ ] prepare gets the "edit stories, then dispatch test" CTA; every other phase gets the "results are on the tickets" CTA (:98-100).
  • [ ] QA_SUMMARY missing/empty → comment still renders (summary is "", :78-80).

Artifact

  • [ ] Artifact name is sandbox-qa-<QA_SLUG>-<run_id> and uploads the QA_STATE_DIR (:116-121).
  • [ ] retention-days: 14, if-no-files-found: warn (:122-123).
  • [ ] Branch with a / in its name still yields a legal artifact name (that's why QA_SLUG exists).

Concurrency & safety

  • [ ] Two dispatches on the same branch don't run concurrently, and the first is not cancelled (:44-46).
  • [ ] fix commits carry [skip ci] so sandbox.yml doesn't redeploy (verify no deploy-loop). Header contract :19-20, :22-26.

Edge cases & failure modes

EdgeExpectedRisk if wrong
Agent dies before emitting QA_STATE_DIRBoth post-steps skip; nothing posted/uploadedUser sees a bare failed job, no clue — check homelab logs
Branch has no open PRQA runs, comment step returns quietlyLooks like a no-op to the user
Agent emits QA_STATE_DIR but no links/summaryMinimal comment with heading + timestamp onlyAcceptable degrade
Two markers from old PRs (review/plan) lingerIgnored — nothing reads them nowCosmetic only
fix on an unconfirmed failureShould not happen — gate is human; but nothing in the workflow enforces "confirmed"Trust boundary — relies on the agent + human

The big coverage boundary — say it out loud

  • The Jira/Confluence behavior is untestable from this repo. Story creation, PASS→Done transitions, screenshot attachments, Bug tickets, the Confluence results table — all live in qa-agent/atlassian-qa.py / sandbox-qa, which are not in /work/repo. To QA those, you need access to the homelab and a real Atlassian instance. Treat the PR body's claims about them as unverified from here.
  • The "reads human comments as instructions" logic (header :17-18) is likewise off-repo. You can only verify it end-to-end on a live run, not by reading this diff.

Gaps between description and code (verify, don't assume)

  1. PR body: "screenshots attached per ticket," "Bug tickets," "cross-linked with PR/sandbox/docs." The diff contains none of this — only JIRA_EPIC_URL/CONFLUENCE_URL links. Verify against a live run.
  2. PR body: "GitHub story-issues are retired." Verifiable and true — the entire issue-creation block is deleted.
  3. PR body: "Foreground with live agent streaming." Consistent with the single non-detached run: step (contrast sandbox.yml:66, which detaches the docs agent).

Suggested Stories to hand back to the epic

  • As a QA engineer, dispatching prepare on a branch with an open PR posts one 🧪 Quality Review comment with a Jira link.
  • Dispatching any phase on a branch with no open PR completes green and posts nothing.
  • Re-dispatching a phase rewrites the same comment rather than adding a second.
  • A fix run pushes a [skip ci] commit and does not trigger a sandbox redeploy via sandbox.yml.
  • An agent step that exits non-zero after emitting QA_STATE_DIR still uploads the artifact.

See also