Getting ACCESS_TOKEN_TYPE_UNSUPPORTED - map[method:google.logging.v2.LoggingServiceV2.ListLogEntries service:logging.googleapis.com

[thalesac]

Hi team,
when trying to setup Cloud logging DS in grafana I'm getting:

failed to run test query: list entries: rpc error: code = Unauthenticated desc = Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project. error details: name = ErrorInfo reason = ACCESS_TOKEN_TYPE_UNSUPPORTED domain = metadata = map[method:google.logging.v2.LoggingServiceV2.ListLogEntries service:logging.googleapis.com]

Permissions are set as per documentation:

JWT Json key file properly loaded in grafana:

is this possibly a bug?

[thalesac]

I've give a try with new gemini-cli trying to pin point this issue.
it says:

You've confirmed Grafana is loading the key, which means the problem lies in how the plugin's backend code processes that key after receiving it from Grafana.


  I've reviewed the code again, focusing on the jwtAuthentication path in pkg/plugin/plugin.go.


   1. Data Flow: When you save the datasource configuration, Grafana sends the settings to the plugin backend. The privateKey is received as a Go string, presumably with
      correct newline characters (\n).
   2. JSON Marshalling: The plugin then constructs a new JSON object to pass to the Google Cloud SDK. It uses json.Marshal to do this.
   3. The Core Problem: The json.Marshal function escapes special characters to create a valid JSON string. This means every newline (\n) in your private key string gets
      converted into a literal backslash followed by an 'n' (\\n). The resulting JSON looks something like this: {"private_key": "-----BEGIN PRIVATE 
      KEY-----\\nMIIC...\\n-----END PRIVATE KEY-----\\n"}.
   4. Google SDK Expectation: The Google Cloud SDK receives this JSON, but it expects the private key to have actual newlines (\n), not escaped ones (\\n). When it sees
      \\n, it fails to parse the PEM-encoded private key, leading to the "Invalid authentication credentials" error you're seeing.


  The line I previously identified, privateKey = strings.ReplaceAll(privateKey, "\n", "\n"), is indeed a no-op and a red herring. The real issue is the
  double-escaping of newlines caused by json.Marshal.


  To fix this, I need to modify the code to ensure the private key is correctly formatted before it's marshalled into JSON. The fix is to replace the escaped newlines
  (\\n) with actual newlines (\n) in the private key string.

According to Gemini seems like this is the issue.

This is the suggested diff

diff --git a/pkg/plugin/plugin.go b/pkg/plugin/plugin.go
index 3ed32c1..c8ba70c 100644
--- a/pkg/plugin/plugin.go
+++ b/pkg/plugin/plugin.go
@@ -102,11 +102,12 @@ func NewCloudLoggingDatasource(ctx context.Context, settings backend.DataSourceI
        switch conf.AuthType {
        case jwtAuthentication:
                privateKey, ok := settings.DecryptedSecureJSONData[privateKeyKey]
-               if !ok || privateKey == "" {
-                       return nil, errMissingCredentials
-               }
+        if !ok || privateKey == "" {
+            return nil, errMissingCredentials
+        }
+        privateKey = strings.ReplaceAll(privateKey, "\n", "\n")
 
-               serviceAccount, err := conf.toServiceAccountJSON(privateKey)
+        serviceAccount, err := conf.toServiceAccountJSON(privateKey)
                if err != nil {
                        return nil, fmt.Errorf("create credentials: %w", err)
                }

[xiangshen-dk]

If you just copy/paste or upload the service account JSON key file, it should be fine. However, if the permissions are miss configured, you will the see the error. Can you verify your service account has the correct permissions? You can do it from the console. You can also run the following commands:

 gcloud auth activate-service-account --key-file=your-key.json
 gcloud logging read "timestamp>\"2025-01-01\"" --limit=1

[dhruv-ic-ghcode]

@thalesac Have you resolved this issue? If yes can you please share the details?

Also @xiangshen-dk I have tried this, In CLI it works but while using same SA key in Grafana throws the same error.

If you just copy/paste or upload the service account JSON key file, it should be fine. However, if the permissions are miss configured, you will the see the error. Can you verify your service account has the correct permissions? You can do it from the console. You can also run the following commands:

 gcloud auth activate-service-account --key-file=your-key.json
 gcloud logging read "timestamp>\"2025-01-01\"" --limit=1

[zodd3131]

I think thalesac is working on S3NS. The plugin tries to acces googleapis.com with S3NS token instead of s3nsapis.fr

We need a parameter to configure GOOGLE_CLOUD_UNIVERSE_DOMAIN.