SLF4J logging warnings and missing application logs on OKDP Spark jobs

[achraf-ibrah]

Hello,

SLF4J falls back to a no-operation logger implementation because no compatible provider is detected. In addition, legacy SLF4J bindings (1.7.x) are present on the classpath and ignored due to incompatibility with the SLF4J API used by Spark in the OKDP image.
As a result, application logs are not produced correctly:

Environment

Image: quay.io/okdp/spark:spark-3.4.2-scala-2.13-java-17-2.0.2
Spark version in manifest: 3.4.2 (also seen with 3.5.0)
Main: org.apache.spark.examples.SparkPi
Jar: local:///opt/spark/examples/jars/spark-examples_2.13-3.4.2.jar

How to reproduce

Submit any minimal SparkApplication (e.g., SparkPi) with the image above via Spark Operator.
Watch the spark-submit logs.
The SLF4J warnings appear consistently.

Likely cause

Classpath mismatch: SLF4J 2.x API with a 1.7 binding artifact.

References

https://www.slf4j.org/codes.html#noProviders
https://www.slf4j.org/codes.html#ignoredBindings

regards,

@SBatais @lioneloh

[achraf-ibrah]

Hi @idirze ,
Just re-launching this issue — could you please have a look when you get a chance? Thanks!

[achraf-ibrah]

More information

Image: quay.io/okdp/spark:spark-3.5.6-scala-2.13-java-17
Spark version 3.5
Main class: org.apache.spark.examples.SparkPi
Example JAR: local:///opt/spark/examples/jars/spark-examples_2.13-3.5.6.jar

sparkapp.yaml

apiVersion: sparkoperator.k8s.io/v1beta2
kind: SparkApplication
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"sparkoperator.k8s.io/v1beta2","kind":"SparkApplication","metadata":{"annotations":{},"name":"spark-okdp","namespace":"dataproc-test"},"spec":{"driver":{"coreLimit":"1.2","cores":1,"labels":{"version":"3.5"},"memory":"512m"},"executor":{"coreLimit":"1.2","cores":1,"instances":1,"labels":{"version":"3.5"},"memory":"512m"},"image":"quay.io/okdp/spark:spark-3.5.6-scala-2.13-java-17","imagePullPolicy":"Always","mainApplicationFile":"local:///opt/spark/examples/jars/spark-examples_2.13-3.5.6.jar","mainClass":"org.apache.spark.examples.SparkPi","mode":"cluster","restartPolicy":{"type":"Never"},"sparkConf":{"spark.jars.ivy":"/tmp/.ivy2","spark.ui.view.acls":"bngs7327"},"sparkVersion":"3.5","type":"Scala"}}
  creationTimestamp: "2026-02-12T14:38:03Z"
  generation: 1
  labels:
    app.kubernetes.io/name: spark-job
  name: spark-okdp
  namespace: dataproc-test
  resourceVersion: ""
  uid: 

spec:
  batchScheduler: yunikorn
  deps: {}

  driver:
    coreLimit: "1.2"
    cores: 1
    memory: 512m
    serviceAccount: spark
    labels:
      domain: appli
      productname: spark-application-driver
      setname: datablok-compute
      version: "3.5"
    affinity:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
            - matchExpressions:
                - key: dedicated
                  operator: In
                  values:
                    - proj-std
    tolerations:
      - key: dedicated
        operator: Equal
        value: proj-std
        effect: NoSchedule
    securityContext:
      privileged: false
      readOnlyRootFilesystem: false
      capabilities:
        drop:
          - ALL
    secrets:
      - name: 
        path: 
        secretType: Generic
    volumeMounts:
      - name: tmp
        mountPath: /tmp

  executor:
    coreLimit: "1.2"
    cores: 1
    instances: 1
    memory: 512m
    labels:
      domain: appli
      productname: spark-application-executor
      setname: datablok-compute
      version: "3.5"
    affinity:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
            - matchExpressions:
                - key: dedicated
                  operator: In
                  values:
                    - proj-spot
    tolerations:
      - key: dedicated
        operator: Equal
        value: proj-spot
        effect: NoSchedule
    securityContext:
      readOnlyRootFilesystem: false
      capabilities:
        drop:
          - ALL
    secrets:
      - name: 
        path: 
        secretType: Generic
    volumeMounts:
      - name: tmp
        mountPath: /tmp

  image: quay.io/okdp/spark:spark-3.5.6-scala-2.13-java-17
  imagePullPolicy: Always
  mainApplicationFile: local:///opt/spark/examples/jars/spark-examples_2.13-3.5.6.jar
  mainClass: org.apache.spark.examples.SparkPi
  mode: cluster
  sparkVersion: "3.5"
  type: Scala

  restartPolicy:
    type: Never

  monitoring:
    exposeDriverMetrics: true
    exposeExecutorMetrics: true
    prometheus:
      jmxExporterJar:
      port:

  hadoopConf:
    fs.defaultFS: gs://
    fs.gs.project.id: 
    fs.gs.working.dir: spark_history
    google.cloud.auth.service.account.enable: "true"
    google.cloud.auth.service.account.json.keyfile:

  sparkConf:
    +(spark.local.dir): ""
    spark.acls.enable: "true"
    spark.decommission.enabled: "true"
    spark.driver.extraClassPath: /etc/log4j2
    spark.eventLog.enabled: "true"
    spark.eventLog.dir: gs://
    spark.executor.decommission.killInterval: 1s
    spark.executor.extraClassPath: /etc/log4j2
    spark.hadoop.fs.s3a.downgrade.syncable.exceptions: "true"
    spark.hadoop.google.cloud.auth.service.account.json.keyfile: 
    spark.jars.ivy: /tmp/.ivy2
    spark.sql.adaptive.enabled: "true"
    spark.sql.autoBroadcastJoinThreshold: ""
    spark.sql.cbo.enabled: "true"
    spark.sql.cbo.joinReorder.enabled: "true"
    spark.storage.decommission.enabled: "true"
    spark.storage.decommission.rddBlocks.enabled: "true"
    spark.storage.decommission.shuffleBlocks.enabled: "true"
    spark.ui.view.acls: 
    spark.ui.view.acls.groups: 
    spark.ui.requestHeaderSize: 16k

  timeToLiveSeconds: 86400

  volumes:
    - name: tmp
      emptyDir: {}

status:
  applicationState:
    state: COMPLETED
  driverInfo:
    podName: spark-okdp-driver
    webUIAddress: 
    webUIPort: 
    webUIServiceName: spark-okdp-ui-svc
  executionAttempts: 1
  executorState:
    spark-pi-1150999b370c1b6c-exec-1: COMPLETED
  lastSubmissionAttemptTime: "2026-02-12T14:38:03Z"
  terminationTime: "2026-02-12T14:38:55Z"
  sparkApplicationId: 
  submissionAttempts: 1
  submissionID: