"Error watching metadata: context canceled" starting from 2025 07 04

[Edvinas-Matulaitis]

Starting from 2025 07 04 on Google Cloud we get this error that links function to this repo at the bottom of this log entry:

{
  "insertId": "oguo51fnbt0fd",
  "jsonPayload": {
    "localTimestamp": "2025-07-07T05:19:00.8012Z",
    "omitempty": null,
    "message": "Error watching metadata: context canceled"
  },
  "resource": {
    "type": "gce_instance",
    "labels": {
      "instance_id": "3371365647560141590",
      "project_id": "xxxxxxxx",
      "zone": "europe-west3-a"
    }
  },
  "timestamp": "2025-07-07T05:19:00.808556387Z",
  "severity": "ERROR",
  "labels": {
    "instance_name": "aef-default-20250706t193350-58h2"
  },
  "logName": "projects/xxxxxxxx/logs/GCEGuestAgent",
  "sourceLocation": {
    "file": "metadata.go",
    "line": "68",
    "function": "github.com/GoogleCloudPlatform/guest-agent/google_guest_agent/events/metadata.(*Watcher).Run"
  },
  "receiveTimestamp": "2025-07-07T05:19:00.954340019Z"
}

Starting with those errors we started getting random 502 bad gateway requests from 2025 07 04 so not sure if that is related with guest agent or no.

[drewhli]

Hello.

The error message here merely indicates that the guest agent is exiting. It's not an actual error with the guest agent.
There may be something else that's causing the guest agent to restart, or some other issue that's causing the guest agent to restart, which would be dependent on the environment.

Can you confirm the following for me?

1. Were there any changes to your environment around the time this issue started happening? (like guest agent upgrade)
2. What guest agent version did you see this issue on?
3. What are these 502 bad gateway requests? Can you explain what the source of these are?
4. Can you provide serial port logs of the VM(s) facing this issue? You can get serial port logs for your instance by following these steps. Make sure to omit any personally identifiable information.
5. Is the image being used derived, or are they the images provided by us on Google Cloud?

[Edvinas-Matulaitis]

Hi,

I contacted Google support for this also as this error is not from my image and not from VM. I can't answer most of your questions because this error comes from Google App Engine using their PHP runtime not my VM.

We have two different environments with different services in each instance. No code changes were made for 3+ weeks. Starting from 2025 07 04 night all services in different environments started to get random 502 Bad Gateway errors with "failed_to_connect_to_backend" in error message and this "Error watching metadata: context canceled" error message.