commit-generated-on-merge skips commit-back step because head-SHA lookup misses the issue_comment workflow run

[alex-frankel]

Summary

After @ouldsid's #14754 fix landed in ca5c529, I ran #14764 as the upstream-branch canary to verify the full auto-gen path end-to-end (the verification couldn't complete on #14762 because that PR was on a fork branch). Everything worked through /validate:

• adx-deployment-validation succeeded on PR head 29608c273722b763fd7ed94a8d04214767353948.
• The generated-azuredeploy-14764-29608c27... artifact uploaded by selected-pipeline had the correct repo-relative path (quickstarts/microsoft.keyvault/key-vault-certificate-create/azuredeploy.json) — confirming the commit-generated-on-merge writes azuredeploy.json to repo root, not the sample folder (path-stripping in upload-artifact@v4) #14754 staging-directory fix works as designed.

But on merge, commit-generated-on-merge ran to "success" without actually committing anything. Steps 4–7 (locate artifact, checkout, download, commit) all skipped. Master is left without azuredeploy.json in the sample folder.

Root cause

In the Resolve artifact source run step (lines roughly 2440–2460 of ValidateSampleDeployments.yml), the lookup is:

RUN_ID=$(gh api \
  "/repos/${REPO}/actions/runs?head_sha=${HEAD_SHA}&event=issue_comment&status=success&per_page=20" \
  --jq "[.workflow_runs[] | select(.name == \"${WORKFLOW}\")] | sort_by(.run_started_at) | last | .id // \"\"")

with HEAD_SHA=${{ github.event.pull_request.head.sha }} — i.e. 29608c273722b763fd7ed94a8d04214767353948.

The problem: for issue_comment-triggered workflow runs, GitHub Actions registers the run against the default branch HEAD at the time the comment was posted, not against the PR's head SHA. Verified via gh run list:

2026-05-13T19:44:12Z 25822348426 issue_comment sha=f6946a30 success   ← THE successful /validate run
2026-05-13T19:43:39Z 25822320414 issue_comment sha=f6946a30 skipped
2026-05-13T19:41:20Z 25822203265 issue_comment sha=f6946a30 skipped

All issue_comment runs targeting #14764 are registered against f6946a30 (master HEAD at the time), not the PR head 29608c27. So the lookup by head_sha=29608c27 returns zero results, the step logs "No successful issue_comment run of 'Validate PR (Summarize + ADX)' found for 29608c27...", and the subsequent commit-back steps all skip.

The artifact itself is still findable by its name (generated-azuredeploy-${PR_NUM}-${PR_HEAD_SHA}), but the workflow uses run-SHA lookup as a first hop, not artifact-name search.

Suggested fix

Look up the artifact by name rather than by run-SHA. The artifact name encodes both the PR number and the PR head SHA. Either:

1. List all artifacts in the repo filtered by name pattern:

   ARTIFACT_ID=$(gh api "/repos/${REPO}/actions/artifacts?per_page=100" \
     --jq "[.artifacts[] | select(.name == \"generated-azuredeploy-${PR_NUM}-${HEAD_SHA}\") | select(.expired == false)] | sort_by(.created_at) | last | .id // \"\"")

   Then proceed straight to download with the artifact ID.

2. Or, in the run-lookup, drop the head_sha= filter and instead pass ?check_suite_id=... or filter by pull_requests[].number == ${PR_NUM} client-side. The workflow_runs API entries do carry a pull_requests array with the PR number for issue_comment-triggered runs.

Option 1 is more direct because it avoids re-deriving the run identity at all — the artifact name is the contract.

Severity

Silent regression: same shape as #14754 — the job runs successfully, but master is left without the regenerated file. Anyone running a clean upstream-branch canary that exercises the auto-gen path today will hit this.

Recovery

I restored azuredeploy.json to the sample folder via a direct push to master, using the (correct) artifact from run 25822348426. The PR's expected outcome (regenerated JSON on master) is now in place; the issue is purely that it took a manual step instead of the on-merge auto-commit.

Context

Original canary: #14764 (merged at e2742e81). Successful /validate run: https://github.com/Azure/azure-quickstart-templates/actions/runs/25822348426. Failing on-merge run: https://github.com/Azure/azure-quickstart-templates/actions/runs/25822588613.

Related: #14754, #14753.

cc @ouldsid