Possible Memory Leak

[johannwesely]

Hello,
in Certification Process occured the following issue "We use the output of the standard command 'top' to take a rough look at the resource requirements of the process. During the endurance test, the value of the resident memory used increased. This can (but does not have to) indicate a potential leak. During the endurance test, we had 5 OPC UA clients connected to the server, which monitored values via subscriptions. The network connection to the last client was turned off/on every 10 minutes during the entire test duration."

I started then to profile memory usage with VisualVM and what i see is that milo netty event loops are gaining more and more memory when connecting and reconnecting with a Client (prosys seems to be worst) multiple times . The milo netty event loops are never cleansed even when performing garbage collection. In contrast the milo shared thread pool releases memory after GC.

Is this normal behaviour or do i need to handle milo netty event loops in any manner?

Greetings Johann

[kevinherron]

There's nothing to manage with the event loop.

If you don't configure one yourself, Milo ends up using one from Stack.sharedEventLoop(), which uses one from Netty configured to have possibly a lot of threads:

    static {
        DEFAULT_EVENT_LOOP_THREADS = Math.max(1, SystemPropertyUtil.getInt(
                "io.netty.eventLoopThreads", NettyRuntime.availableProcessors() * 2));

        if (logger.isDebugEnabled()) {
            logger.debug("-Dio.netty.eventLoopThreads: {}", DEFAULT_EVENT_LOOP_THREADS);
        }
    }

so you can limit it yourself with that system property, or configure your own on the OpcTcpServerTransportConfig.

Netty's is focused on high concurrency / scalability applications and its default is probably too aggressive for most OPC UA applications.

[johannwesely]

Yes this seems to be ok, the amount of threads is limited in my case to 40 milo netty event loops but they never get closed and the allocated memory of each thread rises constantly in a round robin manner when connecting disconnecting clients.

[kevinherron]

The threads will never go away, and any off-heap memory they allocate will remain allocated. Netty uses direct memory for its pooled buffers. Once all 40 threads have ended up being created I would expect that memory growth would start to plateau.

I haven't seen any evidence of a memory leak, but if you think there is one then you can get a heap dump of your application to analyze in JProfiler and find out for sure.

[johannwesely]

Hello FYI,
we did some research on that and couldn´t find any memory leaks.
For people who are interrested you could reduce netty event loops with VM command:
-Dio.netty.eventLoopThreads=8.

Greetings Johann