Skip to content

fix(gateway-deploy): async build + poll; log gateway v0.1.2 #32

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ghchinoy merged 1 commit into from
Sep 28, 2026
Merged

fix(gateway-deploy): async build + poll; log gateway v0.1.2 #32

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions benchmarks/runs/20260925-serving-speed/deployments.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@
| 09-28 23:10 | Cloud Run `dgemma`, `dgemma-gateway`; Vertex model registry | Pruned to n and n-1: 81 revisions deleted (kept gateway `00057`, `00055`; serving `00034`, `00032`); 3 older model resources deleted (kept v0.1.0 and `504638d`) | — | — |
| 09-28 22:50–23:01 | Monitoring | Health check `dgem-probe-hourly` / `-daily` (Cloud Run jobs + Scheduler, SA `dgem-probe-sa`), 4 log-based metrics, 7 `dgem:` alert policies (email) | First runs: Vertex ok 78 ms, gateway ok 192 ms, Cloud Run ok 82 ms | — |

| 09-28 23:39 | Cloud Run `dgemma-gateway` → revision `00061` | **v0.1.2** (gateway/monitoring): decision logs carry `dgem_backend_requested`, failed requests log backend `none` | All checks pass; new log fields confirmed; failover and p95 alerts now have data | Revision `00057` (v0.1.1); older revisions pruned |

Follow-ups:
- ~~Rotate the Hugging Face token~~ done (version 2).
- ~~Cloud Run health lacks `warmed`~~ not a bug: the earlier check ran before warmup finished; later checks show
Expand Down
16 changes: 13 additions & 3 deletions scripts/deploy_cloudrun_gateway.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,11 +63,21 @@ steps:
args: [build, --build-arg=DGEM_VERSION=${GW_VERSION}, --build-arg=DGEM_REVISION=$(git rev-parse --short HEAD), -t, "${IMAGE}", .]
images: ["${IMAGE}"]
YAML
gcloud builds submit "${TMP_CTX}" \
# Submit asynchronously and poll: a synchronous submit exits non-zero when the caller cannot stream from the
# default logs bucket, even though the build succeeds.
BUILD_ID="$(gcloud builds submit "${TMP_CTX}" \
--project="${PROJECT}" \
--config="${TMP_CTX}/cloudbuild.yaml" \
--suppress-logs \
--quiet
--async --format='value(id)' --quiet)"
echo " build ${BUILD_ID}"
while true; do
BUILD_STATUS="$(gcloud builds describe "${BUILD_ID}" --project="${PROJECT}" --format='value(status)')"
case "${BUILD_STATUS}" in
SUCCESS) break ;;
QUEUED|WORKING|PENDING) sleep 10 ;;
*) echo "ERROR: build ${BUILD_ID} ended with status ${BUILD_STATUS}" >&2; exit 1 ;;
esac
done

GPU_IDLE_TTL="${GPU_IDLE_TTL:-3h}"
VERTEX_ENDPOINT_ID="${DGEM_VERTEX_URL:-}"
Expand Down
Loading