RuntimeError: Failed to execute code, which is likely a network issue, please check if the sandbox service is running. ( Error: Server disconnected without sending a response. )

[xSantiago3]

Describe the bug

When running workflows that execute code through the sandbox, the execution fails with:

RuntimeError: Failed to execute code, which is likely a network issue, please check if the sandbox service is running. (Error: Server disconnected without sending a response.)

The sandbox service is reachable and the health endpoint responds correctly, but the container restarts during code execution.

To Reproduce

Install Dify using the Helm chart in an OpenShift cluster.
Verify that the sandbox service is running.
Execute a workflow containing a code execution node.
The execution fails and the sandbox container restarts.

✔️ Expected Behavior

The sandbox should execute the Python code and return the result to the worker without crashing or restarting.

❌ Actual Behavior

The workflow fails with:

RuntimeError: Failed to execute code, which is likely a network issue, please check if the sandbox service is running. (Error: Server disconnected without sending a response.)

The sandbox pod logs show that the container restarts after code execution.
The API logs contain errors similar to frontend:
RuntimeError: Failed to execute code, which is likely a network issue, please check if the sandbox service is running.


dify@dify-helm-worker-65f6d7bfb4-rq87f:/app/api$ curl http://dify-helm-sandbox:8194/health
"ok"

oc exec -it dify-helm-worker-65f6d7bfb4-rq87f -n awx-ns1-h -- env | grep CODE
Defaulted container "worker" out of: worker, check-api (init)
CODE_EXECUTION_ENDPOINT=http://dify-helm-sandbox:8194/

Screenshots

API LOG:

Environment (please complete the following information):

Additional context

Add any other context (e.g. minimal values.yaml to reproduce the problem) here.
I'm using externalPostgres, externalPGVector and externalRedis.

api:
  enabled: true
  replicas: 1
  resources: {}
  nodeSelector: {}
  affinity: {}
  tolerations: []
  priorityClassName: ""
  autoscaling:
    enabled: false
    minReplicas: 1
    maxReplicas: 100
    targetCPUUtilizationPercentage: 80
  livenessProbe:
    enabled: true
    initialDelaySeconds: 30
    periodSeconds: 30
    timeoutSeconds: 5
    failureThreshold: 5
    successThreshold: 1
  readinessProbe:
    enabled: true
    initialDelaySeconds: 10
    periodSeconds: 10
    timeoutSeconds: 5
    failureThreshold: 5
    successThreshold: 1
  startupProbe:
    enabled: false
    initialDelaySeconds: 5
    periodSeconds: 10
    timeoutSeconds: 5
    failureThreshold: 5
    successThreshold: 1
  updateStrategy: {}
  customLivenessProbe: {}
  customReadinessProbe: {}
  customStartupProbe: {}
  podSecurityContext:
    enabled: true
    fsGroup: 1001
    fsGroupChangePolicy: OnRootMismatch
  containerSecurityContext:
    enabled: true
    runAsUser: 1001
    runAsGroup: 1001
    allowPrivilegeEscalation: false
  extraInitContainers:
    - name: init-permissions
      image: busybox:latest
      command:
        - sh
        - -c
        - |
          FLAG_FILE="/app/api/storage/.init_permissions"
          if [ -f "$FLAG_FILE" ]; then
            echo "Permissions already initialized. Exiting."
            exit 0
          fi
          echo "Initializing permissions for /app/api/storage"
          chown -R 1001:1001 /app/api/storage || true
          touch "$FLAG_FILE"
          echo "Permissions initialized. Exiting."
      volumeMounts:
        - name: storage
          mountPath: /app/api/storage
  extraEnv:
    - name: TRIGGER_URL
      value: https://xxxxx.com.br/
    - name: CHECK_UPDATE_URL
      value: ""
    - name: CODE_MAX_NUMBER
      value: "9223372036854775807"
    - name: CODE_MIN_NUMBER
      value: "-9223372036854775808"
    - name: CODE_MAX_STRING_LENGTH
      value: "80000"
    - name: TEMPLATE_TRANSFORM_MAX_LENGTH
      value: "80000"
    - name: CODE_MAX_STRING_ARRAY_LENGTH
      value: "30"
    - name: CODE_MAX_OBJECT_ARRAY_LENGTH
      value: "30"
    - name: CODE_MAX_NUMBER_ARRAY_LENGTH
      value: "1000"
  otel:
    enabled: false
    traceEndpoint: ""
    metricEndpoint: ""
    baseEndpoint: http://localhost:4318
    apiKey: ""
    exporterProtocol: ""
    exporterType: otlp
    samplingRate: 0.1
    batchExportScheduleDelay: 5000
    maxQueueSize: 2048
    maxExportBatchSize: 512
    metricExportInterval: 60000
    batchExportTimeout: 10000
    metricExportTimeout: 30000
  service:
    port: 5001
    annotations: {}
    labels: {}
    clusterIP: ""
  logLevel: INFO
  url:
    consoleApi: ""
    consoleWeb: ""
    serviceApi: ""
    appApi: ""
    appWeb: ""
    files: ""
    marketplaceApi: https://marketplace.dify.ai
    marketplace: https://marketplace.dify.ai
  mail:
    defaultSender: YOUR EMAIL FROM (e.g.: no-reply <no-reply@dify.ai>)
    type: resend
    resend:
      apiKey: xxxx
      apiUrl: https://api.resend.com
    smtp:
      server: smtp.server.com
      port: 465
      username: YOUR EMAIL
      password: YOUR EMAIL PASSWORD
      tls:
        enabled: true
        optimistic: false
  migration: true
  secretKey: ""
  persistence:
    mountPath: /app/api/storage
    annotations:
      helm.sh/resource-policy: keep
    persistentVolumeClaim:
      existingClaim: ""
      storageClass: ocs-storagecluster-cephfs
      accessModes: ReadWriteMany
      size: 5Gi
      subPath: ""
  extraVolumes: []
  extraVolumeMounts: []
  serviceAccount:
    create: false
    name: ""
    automountServiceAccountToken: false
    annotations: {}

worker:
  enabled: true
  replicas: 1
  resources: {}
  nodeSelector: {}
  affinity: {}
  tolerations: []
  priorityClassName: ""
  autoscaling:
    enabled: false
    minReplicas: 1
    maxReplicas: 100
    targetCPUUtilizationPercentage: 80
  customLivenessProbe: {}
  customReadinessProbe: {}
  customStartupProbe: {}
  updateStrategy: {}
  podSecurityContext:
    enabled: false
    fsGroup: 1001
  containerSecurityContext:
    enabled: false
    runAsUser: 1001
  extraEnv:
    - name: CODE_EXECUTION_ENDPOINT
      value: http://dify-helm-sandbox:8194
  logLevel: INFO
  extraVolumes: []
  extraVolumeMounts: []
  serviceAccount:
    create: false
    name: ""
    automountServiceAccountToken: false
    annotations: {}

beat:
  enabled: false
  resources: {}
  nodeSelector: {}
  affinity: {}
  tolerations: []
  priorityClassName: ""
  customLivenessProbe: {}
  customReadinessProbe: {}
  customStartupProbe: {}
  updateStrategy:
    type: Recreate
  podSecurityContext:
    enabled: false
    fsGroup: 1001
  containerSecurityContext:
    enabled: false
    runAsUser: 1001
  extraEnv: []
  logLevel: INFO
  extraVolumes: []
  extraVolumeMounts: []
  serviceAccount:
    create: false
    name: ""
    automountServiceAccountToken: false
    annotations: {}

proxy:
  enabled: true
  replicas: 1
  resources: {}
  nodeSelector: {}
  affinity: {}
  tolerations: []
  priorityClassName: ""
  customLivenessProbe: {}
  customReadinessProbe: {}
  customStartupProbe: {}
  clientMaxBodySize: ""
  workerProcesses: auto
  updateStrategy: {}
  podSecurityContext:
    fsGroup: 1003880000
    fsGroupChangePolicy: null
  containerSecurityContext:
    runAsUser: null
    runAsGroup: null
    allowPrivilegeEscalation: false
    readOnlyRootFilesystem: false
    runAsNonRoot: true
    capabilities:
      drop:
        - ALL
  extraEnv: []
  log:
    persistence:
      enabled: false
      mountPath: /var/log/nginx
      annotations:
        helm.sh/resource-policy: keep
      persistentVolumeClaim:
        existingClaim: ""
        storageClass: ocs-storagecluster-cephfs
        accessModes: ReadWriteMany
        size: 1Gi
        subPath: ""
  extraVolumes:
    - name: nginx-temp
      emptyDir: {}
    - name: nginx-run
      emptyDir: {}
  extraVolumeMounts:
    - name: nginx-temp
      mountPath: /var/cache/nginx/client_temp
    - name: nginx-run
      mountPath: /var/run
  serviceAccount:
    create: false
    name: ""
    automountServiceAccountToken: false
    annotations: {}

web:
  enabled: true
  replicas: 1
  resources: {}
  nodeSelector: {}
  affinity: {}
  tolerations: []
  priorityClassName: ""
  autoscaling:
    enabled: false
    minReplicas: 1
    maxReplicas: 100
    targetCPUUtilizationPercentage: 80
  livenessProbe:
    enabled: true
    initialDelaySeconds: 5
    periodSeconds: 30
    timeoutSeconds: 5
    failureThreshold: 5
    successThreshold: 1
  readinessProbe:
    enabled: true
    initialDelaySeconds: 5
    periodSeconds: 10
    timeoutSeconds: 5
    failureThreshold: 5
    successThreshold: 1
  startupProbe:
    enabled: false
    initialDelaySeconds: 5
    periodSeconds: 10
    timeoutSeconds: 5
    failureThreshold: 5
    successThreshold: 1
  updateStrategy: {}
  customLivenessProbe: {}
  customReadinessProbe: {}
  customStartupProbe: {}
  podSecurityContext: {}
  containerSecurityContext: {}
  extraEnv: []
  service:
    port: 3000
    annotations: {}
    labels: {}
    clusterIP: ""
  serviceAccount:
    create: false
    name: ""
    automountServiceAccountToken: false
    annotations: {}
  enableServiceLinks: false
  extraVolumes: []
  extraVolumeMounts: []

sandbox:
  enabled: true
  replicas: 1
  resources: {}
  nodeSelector: {}
  affinity: {}
  tolerations: []
  priorityClassName: ""
  autoscaling:
    enabled: false
    minReplicas: 1
    maxReplicas: 100
    targetCPUUtilizationPercentage: 80
  livenessProbe:
    enabled: true
    initialDelaySeconds: 1
    periodSeconds: 5
    timeoutSeconds: 5
    failureThreshold: 2
    successThreshold: 1
  readinessProbe:
    enabled: true
    initialDelaySeconds: 10
    periodSeconds: 10
    timeoutSeconds: 5
    failureThreshold: 2
    successThreshold: 1
  startupProbe:
    enabled: false
    initialDelaySeconds: 5
    periodSeconds: 10
    timeoutSeconds: 5
    failureThreshold: 2
    successThreshold: 1
  updateStrategy: {}
  customLivenessProbe: {}
  customReadinessProbe: {}
  customStartupProbe: {}
  podSecurityContext:
    runAsGroup: 10001
    fsGroup: 10001
    allowPrivilegeEscalation: false
    readOnlyRootFilesystem: false
  containerSecurityContext: {}
  extraEnv:
    - name: WORKER_TIMEOUT
      value: "15"
    - name: PYTHONPYCACHEPREFIX
      value: /tmp/pycache
    - name: LD_LIBRARY_PATH
      value: /var/sandbox/sandbox-python/usr/local/lib:/usr/local/lib:/usr/lib:/lib
  extraVolumes:
    - name: dify-sandbox-tmp
      emptyDir: {}
    - name: dify-sandbox-logs
      emptyDir: {}
  extraVolumeMounts:
    - name: dify-sandbox-tmp
      mountPath: /tmp
    - name: dify-sandbox-logs
      mountPath: /logs
  service:
    port: 8194
    annotations: {}
    labels: {}
    clusterIP: ""
  auth:
    apiKey: dify-sandbox
  serviceAccount:
    create: false
    name: ""
    automountServiceAccountToken: false
    annotations: {}

ssrfProxy:
  enabled: false
  replicas: 1
  resources: {}
  nodeSelector: {}
  affinity: {}
  tolerations: []
  priorityClassName: ""
  customLivenessProbe: {}
  customReadinessProbe: {}
  customStartupProbe: {}
  updateStrategy: {}
  podSecurityContext: {}
  containerSecurityContext: {}
  extraEnv: []
  service:
    port: 3128
    annotations: {}
    labels: {}
    clusterIP: ""
  extraVolumes: []
  extraVolumeMounts: []
  serviceAccount:
    create: false
    name: ""
    automountServiceAccountToken: false
    annotations: {}

pluginDaemon:
  enabled: true
  replicas: 1
  resources: {}
  nodeSelector: {}
  affinity: {}
  tolerations: []
  priorityClassName: ""
  customLivenessProbe: {}
  customReadinessProbe: {}
  customStartupProbe: {}
  updateStrategy: {}
  podSecurityContext:
    enabled: true
    fsGroup: 10001
  containerSecurityContext:
    enabled: true
    runAsUser: 10001
  extraEnv: []
  service:
    ports:
      daemon: 5002
      pluginInstall:
    annotations: {}
    labels: {}
    clusterIP: ""
  auth:
    serverKey: lYkiYYT6owG+71oLerGzA7GXCgOT++6ovaezWAjpCjf+Sjc3ZtU+qUEi
    difyApiKey: QaHbTe77CtuXmsfyhR7+vRjI/+XbV1AaFy691iy+kGDv2Jvy0/eAh8Y1
  persistence:
    mountPath: /app/storage
    annotations:
      helm.sh/resource-policy: keep
    persistentVolumeClaim:
      existingClaim: ""
      storageClass: ocs-storagecluster-cephfs
      accessModes: ReadWriteMany
      size: 5Gi
      subPath: ""
  extraVolumes:
    - name: cache
      emptyDir: {}
  extraVolumeMounts:
    - name: app-data
      mountPath: /app/cwd
      subPath: cwd
    - name: cache
      mountPath: /home/plugin/.cache
  marketplace:
    enabled: true
    apiProxyEnabled: false
  serviceAccount:
    create: false
    name: ""
    automountServiceAccountToken: false
    annotations: {}

[BorisPolonsky]

Please provide use the following info for us to determine the reason why sandbox restarts and if it's caused by the chart:

1. Logs from kubectl logs -n awx-ns1-h -p $SANDBOX_POD_NAME (executed after the sandbox restarts, make sure to include the -p option for logs prior to the restart)
2. Reason of pod restart from kubectl describe po -n awx-ns1-h $SANDBOX_POD_NAME
3. The configuration and the code applied in the code-execution node for sandbox to execute to trigger the restart.