High CPU

[roelbeun]

Language / Voertaal

🇳🇱 Nederlandstalig

Inquiry

Bug

Description

Hi,

I've moved the DSMR container to a new server (v6 docker, container mode server_remote_datalogger receiving telegrams from RasPi next to the meter via HTTP), and I notice a fairly high but stable CPU load i did not notice on the old server.

I see up to 8% CPU spread over the following proceses:

USERNAME 2949516 4.1 2.5 242592 207700 ? SNs 22:47 1:09 /opt/venv/bin/python -u /app/manage.py dsmr_backend
USERNAME 2949551 4.4 2.6 250480 212976 ? SN 22:47 1:15 /opt/venv/bin/python /opt/venv/bin/gunicorn dsmrreader.wsgi --bind unix:/run/gunicorn/dsmr_webinterface.socket --timeout 120 --max-requests 5000 --graceful-timeout 30 --error-logfile - --access-logfile /dev/null

I've always set the Datalogger proces sleep to 10 and still have.

Is this expected CPU load? Memory usage is around 460MB stable

! the memory of the DSMRdb container suddenly jumped from 100MB to 600MB.

DSMR-reader version

v6.0.2

DSMR-reader platform

Docker (e.g. Xirixiz's DSMR-reader Docker)

Optional: Container Compose YAML file

Optional: Debug info dump (of DSMR-reader)

Optional: Smart meter telegram

[dennissiemensma]

Welke versie van PostgreSQL draai je? En voor de zekerheid, je draait niet de datalogger los en hebt die binnen de container ook nog eens draaien toch? Dat was eerder een issue voor hoge load: #2123 (comment)

[roelbeun]

Dank Dennis. Ik gebruik postgres v16. aan de hand van #2123 de compose file iets aangepast (niet mbt datalogger), hierbij de debug logs van container start.

het lijkt alsof de datalogger wel wordt gestart.. en daarna disabled ik heb ook de compose file even hier geplakt.

dsmrdb has been recreated
dsmrdb  | 
dsmrdb  | PostgreSQL Database directory appears to contain a database; Skipping initialization
dsmrdb  | 
dsmrdb  | 2026-04-01 09:40:50.811 CEST [1] LOG:  starting PostgreSQL 16.13 on x86_64-pc-linux-musl, compiled by gcc (Alpine 15.2.0) 15.2.0, 64-bit
dsmrdb  | 2026-04-01 09:40:50.811 CEST [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
dsmrdb  | 2026-04-01 09:40:50.811 CEST [1] LOG:  listening on IPv6 address "::", port 5432
dsmrdb  | 2026-04-01 09:40:50.827 CEST [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
dsmrdb  | 2026-04-01 09:40:50.848 CEST [28] LOG:  database system was shut down at 2026-04-01 09:40:50 CEST
dsmrdb  | 2026-04-01 09:40:50.864 CEST [1] LOG:  database system is ready to accept connections
dsmr    | s6-rc: info: service s6rc-fdholder: starting
dsmr    | s6-rc: info: service s6rc-oneshot-runner: starting
dsmr    | s6-rc: info: service s6rc-oneshot-runner successfully started
dsmr    | s6-rc: info: service fix-attrs: starting
dsmr    | s6-rc: info: service init-start: starting
dsmr    | s6-rc: info: service s6rc-fdholder successfully started
dsmr    | s6-rc: info: service svc-nginx-log: starting
dsmr    | s6-rc: info: service svc-dsmr-webinterface-log: starting
dsmr    | s6-rc: info: service svc-dsmr-remote-datalogger-log: starting
dsmr    | s6-rc: info: service svc-dsmr-datalogger-log: starting
dsmr    | s6-rc: info: service svc-dsmr-backend-log: starting
dsmr    | s6-rc: info: service svc-dsmr-backend-log successfully started
dsmr    | s6-rc: info: service fix-attrs successfully started
dsmr    | s6-rc: info: service legacy-cont-init: starting
dsmr    | s6-rc: info: service svc-nginx-log successfully started
dsmr    | s6-rc: info: service svc-dsmr-webinterface-log successfully started
dsmr    | s6-rc: info: service svc-dsmr-datalogger-log successfully started
[ INFO ] init-start | Starting
dsmr    | s6-rc: info: service svc-dsmr-remote-datalogger-log successfully started
[ INFO ] init-start | Architecture: X64
dsmr    | s6-rc: info: service legacy-cont-init successfully started
dsmr    | s6-rc: info: service init-start successfully started
dsmr    | s6-rc: info: service init-useradd: starting
[ INFO ] init-useradd | Starting
dsmr    | usermod: no changes
dsmr    | 
dsmr    |     ───────────────────────────────────────────────────
dsmr    | 
dsmr    |      ___  ___ __  __ ___   ___             _
dsmr    |     |   \/ __|  \/  | _ \ | _ \___ __ _ __| |___ _ _
dsmr    |     | |) \__ \ |\/| |   / |   / -_) _` / _` / -_) '_|
dsmr    |     |___/|___/_|  |_|_|_\ |_|_\___\__,_\__,_\___|_|
dsmr    |                  __           _
dsmr    |                 |   \ ___  __| |_____ _ _
dsmr    |                 | |) / _ \/ _| / / -_) '_|
dsmr    |                 |___/\___/\__|_\_\___|_|
dsmr    | 
dsmr    |     DSMR-protocol reader, telegram data storage and
dsmr    |             energy consumption visualizer.
dsmr    | 
dsmr    | 
dsmr    |     ───────────────────────────────────────────────────
dsmr    |     User UID: 1000
dsmr    |     User GID: 1003
dsmr    |     ───────────────────────────────────────────────────
dsmr    |     DSMR Reader version: 6.0.2
dsmr    |     DSMR Reader Docker version: 20260227.408
dsmr    |     DSMR Reader Docker build-date: 20260227-1936
dsmr    |     ───────────────────────────────────────────────────
dsmr    | 
[ INFO ] init-useradd | Setting up directories and permissions
dsmr    | s6-rc: info: service init-useradd successfully started
dsmr    | s6-rc: info: service init-validate: starting
dsmr    | s6-rc: info: service init-devices: starting
dsmr    | s6-rc: info: service init-container-secrets: starting
dsmr    | s6-rc: info: service init-container-run-mode: starting
[ INFO ] init-validate | Starting
[ INFO ] init-devices | Starting
[ INFO ] init-devices | Checking tty devices (/dev/ttyUSB*, /dev/ttyAMA*, /dev/ttyACM*)
dsmr    | s6-rc: info: service init-devices successfully started
[ INFO ] init-validate | Operation mode: server_remote_datalogger
[ INFO ] init-container-run-mode | Starting
dsmr    | s6-rc: info: service init-container-secrets successfully started
[ OKAY ] init-container-run-mode | Running as SERVER REMOTE DATALOGGER (expects remote datalogger input)
dsmr    | s6-rc: info: service init-container-run-mode successfully started
[ WARN ] init-validate | Default DJANGO_DATABASE_PASSWORD detected, this is a security risk, please set a secure value
[ OKAY ] init-validate | Validation complete
dsmr    | s6-rc: info: service init-validate successfully started
dsmr    | s6-rc: info: service init-dsmr-and-db: starting
[ INFO ] init-dsmr-and-db | Starting
[ INFO ] init-dsmr-and-db | Connecting to database: dsmrdb:5432
[ OKAY ] init-dsmr-and-db | Database connection established after 1 attempt(s)
[ INFO ] init-dsmr-and-db | Running Django post-configuration
dsmr    | Using PRODUCTION configuration: dsmrreader.config.production
dsmr    | Operations to perform:
dsmr    |   Apply all migrations: admin, auth, contenttypes, dsmr_api, dsmr_backend, dsmr_backup, dsmr_consumption, dsmr_datalogger, dsmr_dropbox, dsmr_frontend, dsmr_influxdb, dsmr_mindergas, dsmr_mqtt, dsmr_notification, dsmr_pvoutput, dsmr_stats, dsmr_weather, sessions
dsmr    | Running migrations:
dsmr    |   No migrations to apply.
dsmr    | Using PRODUCTION configuration: dsmrreader.config.production
dsmr    | 
dsmr    | 0 static files copied to '/var/www/dsmrreader/static', 559 unmodified.
dsmr    | Using PRODUCTION configuration: dsmrreader.config.production
dsmr    | Updating password of superuser "admin"
dsmr    | Deactivating any other existing superusers
[ OKAY ] init-dsmr-and-db | Django configuration completed
dsmr    | s6-rc: info: service init-dsmr-and-db successfully started
dsmr    | s6-rc: info: service init-nginx: starting
[ INFO ] init-nginx | Starting
[ WARN ] init-nginx | DJANGO_ALLOWED_HOSTS not set, defaulting to '*'
[ WARN ] init-nginx | DJANGO_CSRF_TRUSTED_ORIGINS not set, leaving unset
[ INFO ] init-nginx | Setting up directories and permissions - init-nginx
[ INFO ] init-nginx | Configuring NGINX to listen on port 80
[ INFO ] init-nginx | Disabling NGINX access logs
[ INFO ] init-nginx | SSL disabled
[ INFO ] init-nginx | HTTP authentication disabled
[ INFO ] init-nginx | Client certificate authentication disabled
dsmr    | s6-rc: info: service init-nginx successfully started
dsmr    | s6-rc: info: service svc-dsmr-webinterface: starting
dsmr    | s6-rc: info: service svc-dsmr-remote-datalogger: starting
dsmr    | s6-rc: info: service svc-dsmr-datalogger: starting
dsmr    | s6-rc: info: service svc-dsmr-backend: starting
dsmr    | s6-rc: info: service svc-dsmr-webinterface successfully started
dsmr    | s6-rc: info: service svc-nginx: starting
dsmr    | s6-rc: info: service svc-dsmr-remote-datalogger successfully started
dsmr    | s6-rc: info: service svc-nginx successfully started
dsmr    | s6-rc: info: service svc-dsmr-datalogger successfully started
dsmr    | s6-rc: info: service svc-dsmr-backend successfully started
dsmr    | s6-rc: info: service legacy-services: starting
[ INFO ] svc-dsmr-datalogger | Disabled for mode server_remote_datalogger
[ INFO ] svc-dsmr-webinterface | Starting
[ INFO ] svc-dsmr-remote-datalogger | Disabled for mode server_remote_datalogger
[ INFO ] svc-dsmr-backend | Starting
[ INFO ] svc-nginx | Waiting for webinterface socket (1/20)
dsmr    | s6-rc: info: service legacy-services successfully started
dsmr    | [2026-04-01 09:41:19 +0200] [224] [INFO] Starting gunicorn 23.0.0
dsmr    | [2026-04-01 09:41:19 +0200] [224] [INFO] Listening at: unix:/run/gunicorn/dsmr_webinterface.socket (224)
dsmr    | [2026-04-01 09:41:19 +0200] [224] [INFO] Using worker: sync
dsmr    | [2026-04-01 09:41:19 +0200] [271] [INFO] Booting worker with pid: 271
[ OKAY ] svc-nginx | Webinterface socket ready

DSMR + DSMRdb (ontvangende server, geen usb connectie naar P1)

services:
  dsmrdb:
    image: postgres:16-alpine
    container_name: dsmrdb
    networks:
      - has-net
    restart: always
    volumes:
      - /home/<LINUX_GEBRUIKER>/docker-configs/dsmr/dsmrdb/data:/var/lib/postgresql/data
    environment:
      TZ: Europe/Amsterdam
      PG_TZ: Europe/Amsterdam
      POSTGRES_USER: <DATABASE_GEBRUIKER>
      POSTGRES_PASSWORD: <DATABASE_WACHTWOORD>
      POSTGRES_DB: <DATABASE_NAAM>
    healthcheck:
      test:
        - CMD-SHELL
        - pg_isready -U <DATABASE_GEBRUIKER> -d <DATABASE_NAAM>
      interval: 10s
      timeout: 5s
      retries: 10
  dsmr:
    image: ghcr.io/xirixiz/dsmr-reader-docker:6
    networks:
      - has-net
    depends_on:
      dsmrdb:
        condition: service_healthy
    container_name: dsmr
    restart: unless-stopped
    volumes:
      - /home/<LINUX_GEBRUIKER>/docker-configs/dsmr/backups:/app/backups
      - /home/<LINUX_GEBRUIKER>/docker-configs/dsmr/logs:/var/log/supervisor
    environment:
      TZ: Europe/Amsterdam
      DJANGO_TIME_ZONE: Europe/Amsterdam
      VIRTUAL_HOST: localhost
      DJANGO_DATABASE_HOST: dsmrdb
      DJANGO_DATABASE_NAME: <DATABASE_NAAM>
      DJANGO_DATABASE_PORT: "5432"
      DJANGO_DATABASE_USER: <DATABASE_GEBRUIKER>
      DJANGO_DATABASE_PASSWORD: <DATABASE_WACHTWOORD>
      DJANGO_SECRET_KEY: <DJANGO_SECRET>
      CONTAINER_RUN_MODE: server_remote_datalogger
      DSMRREADER_ADMIN_USER: admin
      DSMRREADER_ADMIN_PASSWORD: <DSMR_WACHTWOORD>
      DSMRREADER_REMOTE_DATALOGGER_DEBUG_LOGGING: DEBUG
      DSMRREADER_REMOTE_DATALOGGER_SLEEP: "10"
      DUID: 1000 # match user id for backup rights
      DGID: 1003 # match group id for backup rights
    ports:
      - 7777:80
networks:
  has-net:
    external: true

DSMR (p1 metingen)

services:
  dsmr:
    image: xirixiz/dsmr-reader-docker:6
    container_name: dsmr
    cap_add:
      - NET_ADMIN
    restart: always
    environment:
      - TZ=Europe/Amsterdam
      - DJANGO_TIME_ZONE=Europe/Amsterdam
      - VIRTUAL_HOST=localhost
      - CONTAINER_RUN_MODE=remote_datalogger
      - DSMRREADER_REMOTE_DATALOGGER_API_HOSTS=http://192.168.0.83:7777
      - DSMRREADER_REMOTE_DATALOGGER_API_KEYS=<API_KEY>
      - DSMRREADER_REMOTE_DATALOGGER_INPUT_METHOD=serial
      - DSMRREADER_REMOTE_DATALOGGER_SERIAL_BAUDRATE=115200
      - DSMRREADER_REMOTE_DATALOGGER_SERIAL_PORT=/dev/ttyUSB0
      - DSMRREADER_REMOTE_DATALOGGER_SERIAL_BYTESIZE=8
    ports:
      - 7777:80
      - 7779:443
    devices:
      - /dev/ttyUSB0:/dev/ttyUSB0
    healthcheck:
      disable: true
networks: {}

[dennissiemensma]

Je kunt sowieso upgraden naar PostgreSQL 17. Ik verwacht niet de wereld aan performance-winst, maar het is wel aan te raden gezien we alles met die versie doen (en dan weet je zeker dat daar geen verschil zit).

---

Voor de rest heb je zelf al je container run mode op de goede staan zonder datalogger.
Je kunt het beste even in de container(s) zelf kijken wat de load is, welke processen het meeste doen, etc.

Kijk ook naar wa op de derde regel bij CPU, die geeft aan of er gewacht wordt op iets (vaak I/O). Dat kan ook nog meespelen.

podman-compose exec dsmr top

Ter vergelijking binnen de dsmr containers:

Pi 4

Pi 5

[dennissiemensma]

@roelbeun ben je inmiddels wat wijzer over het hoge CPU gebruik?

[roelbeun]

he @dennissiemensma excuus voor de verlate reactie. Ik heb DMSRdb nu geupgrade naar postgresql 17. Ik monitor het even een dagje en kom er op terug!

Het valt me al wel op dat de twee hoofdprocessen een stuk meer CPU % verbruiken dan in jouw screenshots. Al is dat natuurlijk CPU/kracht afhankelijk. Draai nu in een VM op 2 cores (maar meerdere processen)

DSMr top in container (een snapshot)

>    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
>     288 app       23   3  248216 208452  17536 S   4.7   2.6   1:11.96 gunicorn
>     248 app       25   5  242624 206768  20352 S   3.7   2.5   1:01.98 python

De eerste metingen laten zien dat de CPU load zelfs een beetje is toegenomen (1% gemiddeld)

[dennissiemensma]

@roelbeun ik zie je bericht nu pas, sorry, github geeft geen notificaties voor comment edits.

Zie je nog fouten of iets als je podman-compose logs -f dsmr doet?

[TommiieeSV]

Ik ervaar hetzelfde probleem. Mijn DSMR-reader is ook als server_remote_datalogger geconfigureerd, al gebruik ik op de datalogger (Pi 4) zelf nog het python script uit de v5 instructie ipv een container. Ik gebruik Beszel voor de monitoring van mijn systemen en containers en DSMR-reader steekt met kop en schouders boven al mijn andere containers uit qua CPU verbruik.

Ik zie geen bijzonderheden in de logs. De top van mijn dsmr-container:

 PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
  238 app       25   5  242492 207872  21260 R  72.8   1.3   3:40.01 python
  276 app       23   3  244028 206976  19080 S   1.7   1.3   0:20.56 gunicorn

De CPU 'wa' schommelt tussen 2.8 en 17.4. Op de remote datalogger is deze 0.0.

[TommiieeSV]

Ik heb het kunnen oplossen.

Configuration > Backend configuration > Backend process sleep op 1 seconde gezet (deze instelling was leeg bij mij). CPU verbruik is nu gemiddeld 2,5%.

@roelbeun werkt dit voor jou ook?

[dennissiemensma]

@TommiieeSV dank voor de update! Dat verklaart een hoop, want dan draaide het achtergrondproces telkens maximaal, zelfs als er niets te doen was.

[dennissiemensma]

@roelbeun hoe is de load inmiddels? Wat staat bij wa in top?