Alternative to Full Static Alocation when modbus tcp devices are unknown and can be dynamically added/deleted

[cosmicKev]

Hey, first of all great project! This is way better than esp-modbus which is really cumbersome to understand and even work due to so much abstraction.

Kudos to make all this statically allocated. I wonder if you are open to suggestions and give some thinking on my following suggestions.

Problem

Im building an "extension" around you project to manage Modbus devices. But i'm struggling because of the xTaskCreateStatic, the StaticTask needs to be stored in DRAM, but the buffer can be placed on PSRAM (whcih is the usecase i need)

The idea is having a ModbusHub, where the user can add devices in which will then create "nodes" that at the moment will represent logically a Client. The node is responsible to create the ModbusHal / Interface / Client for each node. (feedback is appreciated btw)
Peharps i dont need a task for each Node but a single task for the Hub to sweep through all nodes and make async requests.

Each node will then use its its own task to poll through the devices associated to that node client and make requests queued in the ModbusDevice itself.

Note: Im using PSRAM this is the only reason why this ModbusHub would even work because atm, since i only have around 50K left on IRAM. The application has a full-fledge webapp with devices management and such.

Solution

• Statically allocate up to a maximum of 10 clients (HAL/Interface/C)
  • Requires some Task Management system based on 10 task pools where initialization would get the data from.
• Allow some hooks to use Mallocs.
  • Would allow the usage of heap but also Memory Pools. In my case i divide the system into memory pools big enough so i can monitor each chunk of the pool without the heap corruption.
• Allow task handler to be passed to client and Hal and would then be created outside the EzModbus

I can contribute with any of this approaches, but would like some input from you since i don't want to waste my time in something that might not be accepted.

On another note:

• I have a very problematic Modbus TCP device that at some point just disconnects and requires a full restart of the websocket. Have you consider a restart procedure for when the modbus is disconnected? I do get that isReady() = false. Just wondering.

Kind regards
Kevin

[pierrejay]

Hi Kev,

Thanks for your message! For sure, I am totally open to contributions to make it more suitable to real-world use cases as long as the lib stays focused on its primary mission without adding too much complexity.

I completely understand your use case : you want to dynamically manage EZModbus stacks while maintaining fine-grained control over memory allocation. I believe the most straightforward solution would be to add an option (via constructor parameter or define flag) that prevents automatic task creation. Instead, you'd call a public method like processEvents() from your own task context. This would give you complete freedom to allocate EZModbus objects wherever you want (PSRAM, memory pools, etc.) & manage task creation with your own TCB/stack placement strategy.

The core processing logic would remain identical, just the execution context changes. What do you think of this approach? I believe it looks like the last option you proposed.

Regarding your "problematic" Modbus TCP device, this shouldn't be a problem especially since the fixes i deployed in v1.1.4. ModbusHAL::TCP automatically checks connection health on each request and reconnects transparently if needed. You might just need to adjust the client request timeout if your device typically takes >1 second to reconnect.

Cheers,

Pierre

[pierrejay]

Hi again,

I've been thinking more about your ModbusNode/ModbusHub design. Looking at your architecture diagram, I think it makes a lot of sense, but I'd like to share some thoughts on the implementation approach.

Node/Hub role mapping

Overall, I think the best approach depends on how many ModbusNodes you expect to have at maximum.

• For few nodes (e.g. <5) : keep one task per ModbusNode. Each node handles both its own request/response logic (with async API) AND the EZModbus internal task execution.

• For many nodes : split the responsibilities

  • ModbusNode level : request/response logic only
  • ModbusHub level : centralized "super-task" that executes all EZModbus internal handlers

This would offer the best compromise between simplicity, reactivity & memory/scheduler overhead.

Technical challenge : internal task blocking

The current EZModbus internal tasks use blocking calls to yield efficiently :

• rxTxTask in RTU/TCP interfaces : xQueueSelectFromSet(), with 100ms timeout
• tcpTask in TCP driver : select() from ESP Network API, with 1000ms timeout

Problem : you can't just call several of these handlers in a loop as-is : the first one would block all others.

Solution : switch to non-blocking mode when task management is externalized :

• Override timeouts to xTicksToWait = 0
• Round-robin through all task handlers
• Add your own vTaskDelay() for CPU management as you wish (after all handlers, or in-between handlers...)

Obviously there is a trade-off : you lose the "native" event-driven efficiency (no more sleeping until an actual UART/TCP event occurs) and go back to polling, which means your CPU will be constantly busy even if there's no Modbus request/response to process. But the impact is quite manageable (few % of CPU load when idle instead of zero).

Conclusion

Given your memory constraints and the need for dynamic node management, I think switching to polling approach with the above-proposed Node/Hub scenarios makes the most sense. The CPU overhead is negligible compared to the memory & scheduler savings from avoiding single-purpose tasks (context switches, task stack overhead duplication, etc.).

Also if you go the "super-task in Hub" route, remember to provision enough memory to handle the maximum expected number of ModbusNodes, since you cannot increase task stack size once launched (I guess the marginal cost is around 5KB per additional Modbus Node for interface + driver).

Let me know if this approach sounds good for your use case! I'll be happy to make it clearer if needed.

Best regards,

[cosmicKev]

Hi!!
Sorry for taking some time to answer. Just saw the notification now.
Let me disclose a bit about my application:

• LEMS (Local Energy Management System)
• I have a non-known number of inverters, meters and batteries
• Some have modbus connection via RTU or TCP.
• Some might have a shared TCP:Port but different modbus address.

Answers to your comments/questions

"Overall, I think the best approach depends on how many ModbusNodes you expect to have at maximum."
At the moment i want to support a maximum 10 nodes. (Pre-allocated FreeRTOS task stack on PSRAM via something i call ModbusTaskPool)
1x RTU via UART_NUM_2
9x TCP:Port combinations.
- Use case: imagine multiple inverters connections

"Technical challenge : internal task blocking"

• Non-blocking was something i actually considered. I might consider refactor it in the future for that. But currently, my time is very limited to add this.
• Might be overkill, but besides the EzModbus Tasks i want to have 1x Node task to handle devices polling
• For RTU we can have up to >200 devices on that node. If one device timeout then i accept the latency due to that on other devices, because then something must be wrong with the bus like interference, long-wires, badly configured device. This will warn the user via a logbook on webapp.
  • Data is being polled periodically at low rates, like 5 to 10 seconds and even if it was being polled at 30

Current state

The architecture already evolved a bit more. Now there is another Class below Modbus Device called ModbusData that contains, well, data about the device.
Each holding multiple registers or simply 1 register. This might add some overhead but atm i'm not concerned with that and i can refactor later. Most devices have always a periodic data required to be polled and very little single registers. Its mostly when its necessary to write something into the modbus device itself.

Responsibilities:

Hub:

• Allows user to add/delete modbus devices given a combination of IP:Port or if device is from an added RTU channel. The combo of IP:PORT or added RTU channel are going to be the node names in which you add devices. If it does not exist then a new node will be created till it cant.
• Internally creates nodes based on the requirements when the user adds the Modbus device.

Node

• Responsible for refreshing modbus devices data, periodically or on request (some devices are nice to be refreshed periodically, think of a Modbus Meter like ABB B23/B24. Or an inverter where you want to limit dynamically the power generated.
• Responsible to initialize EzModbus
  • Also responsible to create the tasks for EzModbus.

Device

• Responsible to manage all device data.
• Direct interface for the driver to access its modbus data.

ModbusData

• This one is new since last time we spoke
• Its essentially sort of pre-frame that is linked to some device data.
• Timestamps last time it was transferred
• Provides ways to convert data to the right endianess/byte order.
• Contains frame exceptions, but as well if the device timedout on that specific data request.
  • Why? Well i had 1 device that if it doesnt allow me to write data it just does not answer to my request and timesout instead returning some invalid modbus error.

Examples

Here is an example of a possible scenario. On the real scenario the devices are added via a Webapp by the user.
`
ModbusHub *hub = new ModbusHub();

// Adds RTU as available interface. necessary because hub is abstracted from the board where its running or how many RTU interfaces exist.

hub->add_phy_rtu_iface(MODBUS_RS485_IFACE_NAME, UART_NUM_2, MODBUS_RS485_PIN_TXD, MODBUS_RS485_PIN_RXD, MODBUS_RS485_PIN_RTS, MODBUS_RS485_DEFAULT_BAUDRATE, MODBUS_RS485_DEFAULT_DATA_BITS, MODBUS_RS485_DEFAULT_PARITY, MODBUS_RS485_DEFAULT_STOP_BITS);

// Adds Device to hub. Hub manages which node will take ownership of the device based on its IP:Port.
ModbusDevice *abb_device = hub->add_rtu_device("ABB", MODBUS_RS485_IFACE_NAME, 38400, UART_DATA_8_BITS, MODBUS_RS485_DEFAULT_PARITY, UART_STOP_BITS_1, 2 );

// Requests data periodically of all registers in memory_1 of ABB meter.
ModbusData *abb_memory_1 = abb_device->add_read_request("memory_1", 0x5B00, sizeof(ABB_B23_B24_memory_map_1_t) / 2, Modbus::READ_HOLDING_REGISTERS,
ModbusPeriodicRead::PERIODIC, 5000);
`

the memory from the meter can be like. And its faster to transfer all in one go than make multiple requests. Because some devices require to wait some time between requests.
`
struct
{
uint32_t voltage_l1_10V;
uint32_t voltage_l2_10V;
uint32_t voltage_l3_10V;
uint32_t voltage_l1_l2_10V;
uint32_t voltage_l3_l2_10V;
uint32_t voltage_l1_l3_10V;
uint32_t current_l1_100A;
uint32_t current_l2_100A;
uint32_t current_l3_100A;
uint32_t current_n_100A;
int32_t active_power_total_100W;
int32_t active_power_l1_100W;
int32_t active_power_l2_100W;
int32_t active_power_l3_100W;
int32_t not_important[8]; // reactive power , apparent power
uint16_t frequency_100HZ;
};

Node & EzModbus initialization.

Regarding the node task. Below is how the node creates the task. (psram_malloc i just a wrapper on heap_malloc of esp)

Below is a piece of code regarding task creation. I have a task pool that creates up to 20 tasks.
`
// Allocate & Create for PHY
tcp_phy = static_castModbusHAL::TCP*(psram_malloc(sizeof(ModbusHAL::TCP)));
tcp_interface = static_castModbusInterface::TCP*(psram_malloc(sizeof(ModbusInterface::TCP)));
client = static_castModbus::Client*(psram_malloc(sizeof(Modbus::Client)));

    if(!tcp_phy || !tcp_interface || !client)
    {
        ESP_LOGE(TAG, "Failed to allocate TCP PHY, interface or client in PSRAM");
        if(tcp_phy)
        {
            free(tcp_phy);
            tcp_phy = nullptr;
        }
        if(tcp_interface)
        {
            free(tcp_interface);
            tcp_interface = nullptr;
        }
        if(client)
        {
            free(client);
            client = nullptr;
        }
        return;
    }

    tcp_phy = new (tcp_phy) ModbusHAL::TCP(ip, port);
    tcp_interface = new (tcp_interface) ModbusInterface::TCP(*tcp_phy,Modbus::CLIENT);

    // Initialize the TCP connection
    bool success = tcp_phy->begin();
    if (!success)
    {
        ESP_LOGE(TAG, "%s:123 Failed to initialize communication on TCP", name_);
        return;
    }
    ez_phy_task_ = ModbusTaskPool::g_taskPool.createTask(tcp_phy->tcpTask, name_, 10*4096, tcp_phy, 5);

    client = new (client) Modbus::Client(*tcp_interface, 10000);
    Modbus::Client::Result result = client->begin();
    if (result != Modbus::Client::SUCCESS)
    {
        ESP_LOGE(TAG, "%s: Failed to initialize client on TCP: %s", name_, Modbus::Client::toString(result));
        return;
    }
    else
    {
        ESP_LOGI(TAG, "%s: Initialized communication on TCP", name_);
    }
    ez_interface_task_ = ModbusTaskPool::g_taskPool.createTask(tcp_interface->rxTxTask, name_, 10*4096, tcp_interface, 5);

`

I'm soon going on holidays, but after my holidays (8-09) i intend to make this code for modbus-hub open source.
At the moment is not in a state that i yet like and i still need to migrate from my main application to a component.
After i cover all my use cases: (Modbus Write, Read, Periodic Read, driver errors) and exceptions i will post here :)
Also i still need to cover lots of tests.

Thank you for this nice discussion.
Cheers.

[cosmicKev]

Ah and i forgot to show an example how i read data. It provides the correct order how to obtain data since the data when you read from the modbus is big-endian (for some manufacturers) others just have a mix even.

Feel free to integrate something like this to the EzModbus library. Esp-modbus gives something similar but very complicated. This is something i integrated on ModbusData.
voltage_l1 = (float)(ModbusData::get_uint32(memory_map1->voltage_l1_10V, ModbusBytesOrder::CDAB))*0.1f; voltage_l2 = (float)(ModbusData::get_uint32(memory_map1->voltage_l2_10V, ModbusBytesOrder::CDAB))*0.1f; voltage_l3 = (float)(ModbusData::get_uint32(memory_map1->voltage_l3_10V, ModbusBytesOrder::CDAB))*0.1f; current_l1 = (float)(ModbusData::get_uint32(memory_map1->current_l1_100A, ModbusBytesOrder::CDAB))*0.01f; current_l2 = (float)(ModbusData::get_uint32(memory_map1->current_l2_100A, ModbusBytesOrder::CDAB))*0.01f; current_l3 = (float)(ModbusData::get_uint32(memory_map1->current_l3_100A, ModbusBytesOrder::CDAB))*0.01f;

[pierrejay]

Thanks for your insights on your project! The custom PSRAM allocator + placement new for instance creation is smart and your Task Pool approach looks definitely adequate at this stage.

I'm aware a lot of vendors have they own interpretation of the Modbus spec and use exotic implementations which can be challenging to get it working smoothly.

Regarding data type conversion with custom byte order, it will be indeed useful to support that in the library, thanks for the suggestion. I think I'll add some utility methods to the Modbus::Frame class (i.e. refactoring setters/getters like setCoil, getRegister etc.) so it can be easier to load/unload values to/from a Frame however they are encoded on the device, without needing intermediate storage and explicit conversion.

Looking forward to see your component when it's ready. I'm sure it will be very valuable for people that struggled to implement esp-modbus for managing multiple devices.

Bye,

[cosmicKev]

Hey!
Im looking forward to complete this.
Btw,
Any idea how to properly kill the ezmodbus component? I want to destroy the whole phy, interface, client but it seems it always crashes.
Im speaking regarding TCP.

I have the following order for the deinitialization:

        ModbusTaskPool::g_taskPool.deleteTask(ez_phy_task_);
        ModbusTaskPool::g_taskPool.deleteTask(ez_interface_task_);
        delete tcp_interface;
        tcp_interface = nullptr;
        // Kill all rest.
        tcp_phy->stop();
        delete tcp_phy;
        tcp_phy = nullptr;
        delete client;
        client = nullptr;

[pierrejay]

Yes normally the destructor of each object should take care of resource cleanup. Do you know at which step you get a crash ? And maybe check the stack trace ?

In the TCP test suite I have a full destruction/recreation cycle at some point, and this doesn't trigger a crash:

// Stop the client HAL & destruct the client & interface
halForClient.stop();   // tcp_phy (== ModbusHAL::~TCP())
ezm.~TCP();            // tcp_interface
client.~Client();      // client
vTaskDelay(pdMS_TO_TICKS(500));

// Reconstruct the client & client interface
new (&ezm) ModbusInterface::TCP(halForClient, Modbus::CLIENT);
new (&client) Modbus::Client(ezm);

// Restart the client HAL, client should appear disconnected
halForClient.begin();

Anyway the order you are destroying the resource (tasks first, then objects) looks ok

[cosmicKev]

Hey,
Here is what I've promissed. Its a bit too raw but i will start soon adding documentation + examples.
https://github.com/cosmicKev/modbus_hub
Also i have a fork from your project but still need to fix some compilation issues due the addition of dynamic memory allocation. Hope i can fix that one day this week so i can create a PR for this
https://github.com/cosmicKev/EZModbus/tree/feature-external-tasks-queues

[pierrejay]

Thanks a lot for sharing! FYI, I have created a new release (v1.1.5) with the latest improvements & bugfixes we discussed. You might want to check and pull the changes, I believe this shouldn't be breaking your current code.

Cheers,