For Quality

What to test, and where the edges are, for .github/workflows/sandbox.yml:98-188.

Precedence correctness

• PR-body value overrides a branch-scoped variable, which overrides a

global (SANDBOX_ALL_*) variable — for the same toggle/env key.

• Within one scope (ALL or a given branch), a secret overrides a

variable of the same name — e.g. SANDBOX_ALL_CRON set as both a variable (false) and a secret (true) should resolve to true (sandbox.yml:153, ordering matters: variable listed before secret in seq).

• A branch-scoped variable overrides a global secret of the same

key — branch scope beats global scope even across the variable/secret split (sandbox.yml:153-155: both ALL entries come before both branch entries in seq, regardless of var/secret).

• A key set only at one layer, with no conflict, survives untouched all the

way to out.

Branch-name sanitization edge cases

ref_key = re.sub(r"[^A-Z0-9]+", "_", HEAD_REF.upper()).strip("_") (sandbox.yml:146).

• Branch names with /, -, . (e.g. feature/foo-bar.v2) — confirm the

expected collapsed key (FEATURE_FOO_BAR_V2).

• Two distinct branch names that sanitize to the same key (e.g.

ci/foo and ci-foo both → CI_FOO) — confirm this is at minimum a known, documented collision rather than a silent one; there's no disambiguation in the current logic.

• An empty or missing head_ref (ref_key empty) — confirm the

branch-scoped layers are skipped entirely (if ref_key: at sandbox.yml:154) rather than matching every SANDBOX__* key.

Toggle parsing (truthy(), sandbox.yml:149)

• Values that should be truthy: 1, true, True, YES, on, with

incidental whitespace (" true ").

• Values that should be falsy: 0, false, no, off, empty string, and

anything not in the truthy set (e.g. 2, yeah).

• A toggle key present in a lower layer but absent from the PR block must

keep the lower layer's resolved boolean, not reset to a default.

SEED / seed handling

• Passed through as a raw string (sandbox.yml:161, 175) — confirm no

truthy-style coercion is accidentally applied to it, and that an unexpected value (not basic/rich) is at least handled predictably downstream by the sandbox CLI, even if this step itself doesn't validate it.

Secret resolution (sandbox.yml:163-173)

• secret:<NAME> where <NAME> exists → resolves to the real value in the

generated settings file (not observable in logs — verify by checking the settings file content directly in a test run, or the deployed container's environment).

• secret:<NAME> where <NAME> does not exist → step fails, error

message names the missing secret, no settings file is produced with a half-resolved value.

• secret:github_token and secret:GITHUB_TOKEN (case variants) → both

rejected.

• A plain (non-secret:) env value that happens to start with the

literal text secret: but isn't meant as a reference — confirm this is accepted as expected (current logic treats any string starting with secret: as a reference, with no escape hatch); flag if a real value should ever legitimately start with that prefix.

PR-body extraction (unchanged path, but feeds this logic)

• No ##-heading containing "sandbox" anywhere in the PR body → PR layer

is {}, GitHub-side result passes through unchanged (this is the exact case this PR's own sandbox deployed under — see Walkthrough).

• A heading containing "sandbox" but with no fenced block immediately

after it → still {} (block stays empty).

• yq not installed on the runner → falls back to {} rather than

failing the whole step (sandbox.yml:125-129).

• Malformed YAML inside the fenced block → yq failure is swallowed

(2>/dev/null || echo '{}'), so this degrades to "PR block ignored" rather than failing the deploy — confirm that's the intended behavior for a typo'd settings block, since it can silently hide an author's mistake.

End-to-end output shape

• Confirm the final JSON always has only the keys env, whatsapp,

two_regions, cron, queue, seed (only present when set) — no bleed-through of an unrelated SANDBOX_* variable/secret name that doesn't map to a reserved toggle and wasn't meant as an env override.

• Confirm the "same JSON shape as before" claim holds for a sandbox that

sets only a PR-body block and nothing on the GitHub side — the output should be indistinguishable from the pre-PR behavior.