Skip to content

configuring-a-platform.md

Latest commit

 

History

History
364 lines (288 loc) · 15.6 KB

configuring-a-platform.md

File metadata and controls

364 lines (288 loc) · 15.6 KB

Configuring a Platform

In This Document

Overview

Function configuration carries information about the specific function (how it's triggered, the runtime type, etc.) whereas a platform configuration carries information about the platform on which functions are run. For example, where should the function log to? What sort of metric mechanism is in place? Which port should the function listen on for health checks?

While this could theoretically be passed in the function configuration, it would make configuration updates a complex task of regenerating the configuration for all provisioned functions. The platform configuration is therefore stored separately, shared amongst all functions that share a platform.

Note: A "platform" could be a cluster or any sub resource of that cluster like a namespace. If, for example, you have a namespace per tenant, you configure logging, metrics, etc. differently for each tenant

Creating a platform configuration in Kubernetes

In Kubernetes, a platform configuration is stored as a ConfigMap named platform-config in the namespace of the function. For example, to create a ConfigMap in the "nuclio" namespace from a local file called platform.yaml, run the following from a command line:

kubectl create configmap platform-config  --namespace nuclio --from-file platform.yaml

Configuration elements

Log sinks (logger)

Configuring where a function logs to is a two step process. First, you create a named logger sink and provide it with configuration. Then, you reference this logger sink at the desired scope with a given log level. Scopes include the following:

  • System logging - This is where logs from services like the controller, the dashboard, etc. are shipped to
  • Function logging - Unless overridden per function, this is where the function logs are shipped to
  • A specific function - An optional override per function, allowing specific functions to ship elsewhere than the platform function logger

Let's say you want to ship all function logs and only warning/error logs from the system to Azure App Insights. However, you want all system logs to also go to stdout. Your logger section in the platform.yaml would look like this:

logger:
  sinks:
    myStdoutLogger:
      kind: stdout
    myAppInsightsLogger:
      kind: appinsights
      attributes:
        instrumentationKey: something
        maxBatchSize: 512
        maxBatchInterval: 10s
  system:
  - level: debug
    sink: myStdoutLogger
  - level: warning
    sink: myAppInsightsLogger
  functions:
  - level: debug
    sink: myAppInsightsLogger

First, you declared the two sinks: myStdoutLogger and myAppInsightsLogger. Then, you bound system:debug (which catches all logs at the severity level and higher) to myStdoutLogger, and system:warning, functions:debug to myAppInsightsLogger.

Supported log sinks

All log sinks support the following fields:

  • kind - The kind of output
  • url - The URL at which the sink resides
  • attributes - Kind specific attributes

Standard output (stdout)

The standard output sink currently does not support any specific attributes.

Azure Application Insights (appinsights)
  • attributes.instrumentationKey - The instrumentation key from Azure
  • attributes.maxBatchSize - Max number of records to batch together before sending to Azure (defaults to 1024)
  • attributes.maxBatchInterval - Time to wait for maxBatchSize records (valid time units are ns, us (or µs), ms, s, m, h), after which whatever is gathered will be sent towards Azure (defaults to 3s)

Metric sinks (metrics)

Metric sinks behave similarly to logger sinks in that first you declare a sink and then bind a scope to it. To illustrate with an example, if you would (for some reason) want all of your system metrics to be pulled by Prometheus whereas all function metrics pushed to a Prometheus push proxy, your metrics section in the platform.yaml would look like this:

metrics:
  sinks:
    myPromPush:
      kind: prometheusPush
      url: http://prometheus-prometheus-pushgateway:9091
      attributes:
        jobName: myPushJob
        instanceName: myPushInstance
        interval: 10s
    myPromPull:
      kind: prometheusPull
      url: :8090
      attributes:
        jobName: myPullJob
        instanceName: myPullInstance
    myAppInsights:
      kind: appisights
      attributes:
        interval: 10s
        instrumentationKey: something
        maxBatchSize: 2048
        maxBatchInterval: 60s
  system:
  - myPromPull
  - myAppInsights
  functions:
  - myPromPush

Supported metric sinks

All metric sinks support the following fields:

  • kind - The kind of output
  • url - The URL at which the sink resides
  • attributes - Kind specific attributes

Prometheus push (prometheusPush)
  • url - The URL at which the push proxy resides
  • attributes.jobName - The Prometheus job name
  • attributes.instanceName - The Prometheus instance name
  • attributes.interval - A string holding the interval to which the push occurs such as 10s, 1h or 2h45m. Valid time units are ns, us (or µs), ms, s, m, h

Prometheus pull (prometheusPull)
  • url - The URL at which the HTTP listener serves pull requests
  • attributes.jobName - The Prometheus job name
  • attributes.instanceName - The Prometheus instance name

Azure Application Insights (appinsights)
  • attributes.interval - A string holding the interval to which the push occurs such as 10s, 1h or 2h45m. Valid time units are ns, us (or µs), ms, s, m, h
  • attributes.instrumentationKey - The instrumentation key from Azure
  • attributes.maxBatchSize - Max number of records to batch together before sending to Azure (defaults to 1024)
  • attributes.maxBatchInterval - Time to wait for maxBatchSize records (valid time units are ns, us (or µs), ms, s, m, h), after which whatever is gathered will be sent towards Azure (defaults to 3s)

Webadmin (webAdmin)

Functions can optionally serve requests to get and update their configuration via HTTP. By default this is enabled at address :8081 but can be overridden by the configuration:

  • enabled - Whether or not to listen to requests. true, by default
  • listenAddress - The address to listen on. :8081, by default

For example, the following configuration can be used listen on port :10000:

webAdmin:
  listenAddress: :10000

Health check (healthCheck)

An important part of the function life cycle is to verify its health via HTTP. By default this is enabled at address :8082 but can be overridden by the configuration:

  • enabled - Whether or not to listen to requests. true, by default
  • listenAddress - The address to listen on. :8082, by default

For example, the following configuration disables responses to health checks:

healthCheck:
  enabled: false

Cron-trigger creation mode (cronTriggerCreationMode)

The cronTriggerCreationMode configuration field determines how to run Cron triggers:

  • "processor" (default) - Run Cron triggers from the Nuclio processor.
  • "kube" - [Tech Preview] Run Cron triggers as Kubernetes CronJobs; applicable only on Kubernetes platforms.

For example, the following configuration implements Cron triggers as Kubernetes CronJobs on a Kubernetes platform:

cronTriggerCreationMode: "kube"

For more information, see the Cron-trigger reference.

Runtime (runtime)

The runtime sections allows you to configure various runtime related parameters. For example to define custom PyPI repository, add the following section:

  runtime:
    python:
      buildArgs:
        PIP_INDEX_URL: "https://test.pypi.org/simple"

envFrom (runtime.common.envFrom)

envFrom is a configuration of []v1.envFromSource type which enables the specification of secrets/configMaps for propagation to all functions. Platform values take lower precedence compared to function config values, which can be specified in the function config spec under envFrom (function configuration documentation).

  envFrom:
    - secretRef:
        name: test-secret

Sensitive fields

Nuclio resources (currently function configuration and api gateway configuration) may contain sensitive information such as passwords, tokens, etc. When the 'masking sensitive fields' feature is enabled, these fields get obfuscated, and their raw values are stored separately (in a Kubernetes secret). They are then populated internally when needed, during function deployment or api gateway creation.

In api gateway config only password field is masked if 'masking sensitive fields' is enabled. For function config there are some config fields that are masked by default. You can add custom sensitive fields to mask by specifying the regex to the path in the function configuration. The masked fields are replaced with references ($ref) in the function configuration. Example:

  sensitiveFields:
    maskSensitiveFields: true
    customSensitiveFields:
    - "^/spec/triggers/.+/url$"

Liveness probe

Liveness probe is a configuration of v1.Probe type.
In general, a Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic.
In particular, the liveness probe is used to check if the function is alive and should be restarted if it is not.
For more information, check out the Kubernetes docs.
Example:

  livenessProbe:
    initialDelaySeconds: 30
    timeoutSeconds: 5
    periodSeconds: 10
    failureThreshold: 3

Readiness probe

Readiness probe is a configuration of v1.Probe type.
In general, a Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic.
In particular, the readiness probe is used to check if the function is ready to receive traffic.
For more information, check out the Kubernetes docs.
Example:

  readinessProbe:
    initialDelaySeconds: 30
    timeoutSeconds: 5
    periodSeconds: 10
    failureThreshold: 3

Proxy logs from ElasticSearch or OpenSearch

To proxy logs from ElasticSearch or OpenSearch, you can use the elasticsearch configuration section. This allows you to specify the connection details and how logs should be retrieved.

Example:

kube:
  elasticSearchConfig:
    url: {url}
    sslVerificationMode: none # either "none" or "full"
    username: {username}
    customQueryParameter: {any_custom_query_parameter} # optional
    index: {elastic-search-index} # index regexp

To add a password, you need to create a secret with a password for the ElasticSearch or OpenSearch instance:

kubectl create secret generic nuclio-es-secret --from-literal=password='password'

and set these parameters in helm values:

  elasticSearchPassword:
    secretName: ""
    secretKey: ""

OR mount it to nuclio-dashboard deployment manually:

kubectl patch deployment nuclio-dashboard \
  -n <ns> \
  --type='json' \
  -p='[
    {
      "op": "add",
      "path": "/spec/template/spec/containers/0/env/-",
      "value": {
        "name": "NUCLIO_ELASTIC_SEARCH_PASSWORD",
        "valueFrom": {
          "secretKeyRef": {
            "name": "nuclio-es-secret",
            "key": "password"
          }
        }
      }
    }
  ]'

Project Secret-Based Service Account Enrichment and Validation

Nuclio now supports enriching and validating service accounts using project-specific Kubernetes secrets. This enables fine-grained control over which service accounts are allowed to run functions for a given project.

Platform Configuration Options

Add the following fields to your Nuclio platform configuration:

platform:
  kube:
    projectSecretTemplate: "{{ .ProjectName }}-nuclio-project-secret"
    projectSecretAllowedServiceAccountsKey: "allowedServiceAccounts"
    projectSecretDefaultServiceAccountKey: "defaultServiceAccount"
Field Description
projectSecretTemplate A Go template used to compute the name of the Kubernetes secret that holds project-specific settings.
projectSecretAllowedServiceAccountsKey The key in the secret that lists comma-separated allowed service accounts for the project.
projectSecretDefaultServiceAccountKey The key in the secret containing the default service account for the project, used when no service account is explicitly specified.

The projectSecretTemplate is rendered using the following context:

templateData := map[string]interface{}{
    "ProjectName": projectName,
}

This allows to dynamically construct the secret name based on the project and namespace.

Data in a secret example:

  defaultServiceAccount: "project-service-account"
  allowedServiceAccounts: "project-service-account,team-a-sa,team-b-sa"

Enrichment and Validation logic overview

The function's service account (SA) is enriched or validated based on the presence of a project-level secret and whether a SA is explicitly provided in the function spec. The logic works as follows:

✅ Scenario 1: Function SA is provided & project secret exists

  • Validate that the provided SA is included in the list defined under the allowedServiceAccounts key in the secret.
  • ❌ If the SA is not in the allowed list, the function deployment fails.

✅ Scenario 2: Function SA is provided & project secret does not exist

  • Use the function's provided SA as-is, with no validation or enrichment.

✅ Scenario 3: Function SA is not provided & project secret exists

  • Use the value from the secret's defaultServiceAccount key as the service account.

✅ Scenario 4: Function SA is not provided & project secret does not exist

  • Use the platform's default service account from the platform configuration (this is controlled by a separate feature).