Skip to content

mavutil: fix mavtcp.recv() crashing when autoreconnect is disabled - #1246

Open
khancyr wants to merge 2 commits into
ArduPilot:masterfrom
khancyr:fix/mavtcp-recv-crash-no-autoreconnect
Open

mavutil: fix mavtcp.recv() crashing when autoreconnect is disabled#1246
khancyr wants to merge 2 commits into
ArduPilot:masterfrom
khancyr:fix/mavtcp-recv-crash-no-autoreconnect

Conversation

@khancyr

@khancyr khancyr commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

recv() called self.reconnect() when self.port was None, but reconnect() is a no-op unless autoreconnect=True was passed to the constructor. With the default autoreconnect=False, a lost connection left self.port as None and the very next self.port.recv(n) call crashed with AttributeError instead of a clear "no data" result.

write() already has this same guard a few lines down (checks self.port is None after attempting reconnect and returns early); recv() was just missing its equivalent.

Now returns b"" (matching the existing "no data available right now" convention on the EAGAIN/EWOULDBLOCK exception path a few lines below) when the socket is still unset after the reconnect attempt.

Done with Claude.
tested with

#!/usr/bin/env python3

"""
test that mavtcp.recv() doesn't crash when the socket is gone and
autoreconnect is disabled (the default)
"""

import socket
import unittest

from pymavlink import mavutil


class MavtcpRecvNoAutoreconnectTest(unittest.TestCase):
    '''recv() must not raise when self.port is None and reconnect() is a no-op'''

    def test_recv_returns_empty_when_port_is_none(self):
        listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        listener.bind(('127.0.0.1', 0))
        listener.listen(1)
        port = listener.getsockname()[1]

        conn = mavutil.mavtcp('127.0.0.1:%d' % port, autoreconnect=False)
        try:
            # simulate a lost connection: no autoreconnect, so reconnect()
            # inside recv() is a no-op and self.port stays None
            conn.port.close()
            conn.port = None

            data = conn.recv()  # must not raise AttributeError
            self.assertEqual(data, b"")
        finally:
            conn.close()
            listener.close()


if __name__ == '__main__':
    unittest.main()

@peterbarker peterbarker left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please commit the test

Comment thread mavutil.py
@khancyr
khancyr force-pushed the fix/mavtcp-recv-crash-no-autoreconnect branch 2 times, most recently from 463d734 to 2afad2c Compare August 6, 2026 14:52
@khancyr

khancyr commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

@peterbarker update with the fd closing and some tests

@peterbarker

Copy link
Copy Markdown
Contributor

Unfortunately this does appear to break some things:

  ⎿  mavutil.py
       ● 1303 [correctness] Returning b"" whenever the socket is gone turns a permanently dead link (autoreconnect=False) into a silent infinite hang instead of a reported error.
       ● 1284 [correctness] close() now sets self.port = None, which makes close() reversible for autoreconnect connections: any later recv()/write() hits the `if self.port is None:
                            self.reconnect()` path and silently opens a brand-new TCP connection to the peer.
       ● 1285 [correctness] close() clears self.fd but reconnect()/do_connect() never restore it, so a reconnected mavtcp is left with fd=None and mavfile.select() stops waiting on the
                            socket.
       ● 1299 [correctness] With autoreconnect=False the guarded state (self.port is None on a live object) is only reachable via this same PR's new close(), so the guard largely
                            protects against a condition the PR itself introduces rather than the one the title describes.
       ● 1299 [cleanup]     The new recv() guard is a copy of the block write() already has at 1320-1326, minus write()'s try/except, so recv() still crashes on the autoreconnect=True
                            path the PR claims to fix.
       ● 1334 [cleanup]     reconnect() re-implements the teardown that close() now owns (port.close(); port=None) but does not clear self.fd, leaving the two paths inconsistent after
                            this diff — it should just call self.close().
       ● 1285 [cleanup]     Clearing self.fd on close() is applied only to mavtcp, while mavserial/mavudp/mavmcast/mavtcpin close() all still leave a stale fd — a per-class special case
                            where a shared convention is needed.
     tests/test_mavtcp.py
       ●   51 [cleanup]     test_recv_returns_empty_when_port_is_none_without_close hand-mutates self.port to None and its explanatory comment is factually wrong, so it proves only that
                            the guard survives external mutation — not that the claimed disconnect crash is fixed.
       ●   22 [cleanup]     No test constructs mavtcp with autoreconnect=True, so nothing covers the reconnect path that the new fd=None line actually breaks.
       ●   80 [correctness] test_recv_receives_data retries recv() 50 times with no sleep between attempts, so the whole retry budget can be consumed in microseconds before loopback
                            data is readable.
       ●   14 [correctness] get_free_port() closes the probe socket before setUp rebinds the port, a TOCTOU race that can make every test in the class error.
       ●   76 [cleanup]     test_recv_receives_data calls accept() directly on the non-blocking listen socket with no retry, bypassing the mavtcpin API.

Perhaps we do need to rethink setting that fd to None in this branch and do it separately?

@khancyr
khancyr force-pushed the fix/mavtcp-recv-crash-no-autoreconnect branch 2 times, most recently from 08c90ea to 93a8a80 Compare August 7, 2026 11:46
khancyr and others added 2 commits August 7, 2026 17:59
close() called self.port.close() unconditionally, so closing a mavtcp whose
connect had already failed (do_connect() nulls self.port after exhausting its
retries) raised AttributeError, and closing twice raised on the second call.
Guard it and null the port, so close() is idempotent and leaves the object in a
state the rest of the class already tests for.

recv() then needs to say something sensible when it finds self.port None. It
calls reconnect() first, but reconnect() is a no-op unless autoreconnect=True,
so the socket can still be gone when it returns, and the next self.port.recv(n)
raised AttributeError.

Returning b"" would be the smaller change, but it puts a permanent condition in
a transient bucket. recv() already uses b"" to mean "no data right now, retry" -
that is the EAGAIN/EWOULDBLOCK path - and already raises for a link that is
actually broken, since ECONNRESET/EPIPE calls handle_disconnect() and then
re-raises. A socket that is gone and has nothing to reopen it belongs with the
second group, not the first. Answering b"" there makes recv_msg() return None
forever and recv_match(blocking=True) spin at the select() cadence with nothing
ever reported.

So raise OSError(errno.ENOTCONN, "TCP socket is closed"). ENOTCONN is not in
CPython's errno-to-subclass map, so this stays a plain OSError with errno set -
which matters because every decision in this class is made by inspecting
e.errno, and a bare ConnectionError carries None. It also remains catchable by
the existing `except OSError` / `except socket.error` handlers in callers.

Both entry points now go through _ensure_port(), so there is one place that
decides whether a socket is usable and one exception type for "it is not".
write() keeps its existing contract by catching that and returning: its callers
are sending a mavlink message through self.mav and are not in a position to
handle a dead link. That subsumes the try/except socket.error it used to wrap
reconnect() in, and makes the silence deliberate rather than incidental. A
failed reconnect still surfaces its own, more specific error - verified as
ConnectionRefusedError (errno 111) rather than being flattened to ENOTCONN.

Behaviour change worth noting for callers: recv() after close() now raises where
it previously would have dereferenced None. Anything closing a link from another
thread or a signal handler to break out of a blocking recv_match() will see
OSError instead of an exception about NoneType. With autoreconnect=False, close()
is the only way to reach this state - reconnect() is guarded by
`if self.autoreconnect:` and returns without touching the port, and do_connect()
only nulls it when it is about to raise - so this fires precisely when a closed
connection is used.

Deliberately not clearing self.fd here, as suggested during review. mavtcp only
ever assigns self.fd in mavfile.__init__ - neither do_connect() nor reconnect()
maintains it - so clearing it on close() leaves a reconnected connection with
fd=None, and mavfile.select() then takes its "no fd" branch and sleeps 0.5s
claiming readiness instead of waiting on the socket. Measured: select(1.0)
returned True after 0.500s. Making fd consistent needs do_connect() to set it,
and wants doing across the other mavfile subclasses too, so it is left for a
separate change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Covers what the previous commit changes: fd and port after construction, close()
being idempotent, recv()/write()/select() after close(), a data round-trip, and a
reconnect on the autoreconnect=True path.

recv() after close() is asserted to raise OSError with errno ENOTCONN, and to not
be a ConnectionError - a bare ConnectionError would carry errno None, which is
useless to the callers most likely to catch it. write() after close() is asserted
to stay silent, since it keeps its fire-and-forget contract.

Points raised in review and addressed here:

- the port is obtained by binding tcpin to port 0 and reading back what the OS
  assigned, rather than probing for a free port and closing the probe socket
  before rebinding it, which left a window for something else to take it.

- the data round-trip drives the listener through mavtcpin's own API - recv() is
  what performs the accept, and write() sends to the accepted socket - rather
  than reaching past it into listen.accept(). Since the listening socket is
  non-blocking, recv() is pumped against a deadline instead of called once.

- the receive loop runs against a deadline with a sleep between attempts, rather
  than spending a fixed 50 iterations that could all be consumed before loopback
  data became readable.

- test_recv_when_port_is_none previously carried a comment claiming reconnect()
  nulls self.port on disconnect. It does not - the body is guarded by
  `if self.autoreconnect:`. Relabelled as what it is: a unit test of the guard
  itself, with close() covered separately as the reachable route.

- added autoreconnect=True coverage, which was missing entirely. That test uses a
  plain socket listener instead of mavtcpin: it needs two successive server-side
  connections, and mavtcpin tracks only one at a time and re-accepts only after
  its own recv() hits an error. settimeout() lets accept() block up to the
  timeout, so it needs no polling.

No assertion is made about self.fd after close(), since mavtcp does not maintain
it across close()/do_connect(); that is left for the separate change that fixes
it rather than pinned here.

25 consecutive runs of this file are clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@khancyr
khancyr force-pushed the fix/mavtcp-recv-crash-no-autoreconnect branch from 93a8a80 to 47123ac Compare August 7, 2026 23:05

@peterbarker peterbarker left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I think we also need a test for the "deliberately closed" cases.

Comment thread mavutil.py
pass
if self.port is None:
try:
self._ensure_port()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Problem here is that we haven't checked whether we intentionally closed the port.

It only kind of worked by accident as it was, but now if you've marked the port as automatically-reconnect then when we go through this path we will reconnect even if the thing was closed on purpose.

We probably need some self._closed state to fix this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have https://github.com/khancyr/pymavlink/tree/fix/mavtcp-close-is-final as follow up. I just didn't play with it yet ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants