Skip to content

fix(lambda): share HTTP sessions so URLSession teardown stops killing the process - #48

Merged
bradwindy merged 1 commit into
mainfrom
claude/nervous-sammet-cada62
Aug 24, 2026
Merged

fix(lambda): share HTTP sessions so URLSession teardown stops killing the process#48
bradwindy merged 1 commit into
mainfrom
claude/nervous-sammet-cada62

Conversation

@bradwindy

Copy link
Copy Markdown
Owner

The bug

GET /image returned {"message":"Internal Server Error"} on roughly one request in four. CloudWatch showed the handler completing all of its work first, then the process dying:

[NZImageApiLambda] Got second response, result count Optional(304452)
Object 0xffffa0030460 of class _MultiHandle deallocated with non-zero retain count 2. ...
REPORT ... Status: error  Error Type: Runtime.ExitError

Root cause

NetworkRequestManager built a fresh Alamofire Session on every call, in five different methods, and URLProcessor calls into it up to a dozen times per request.

On Linux an Alamofire Session owns a URLSession (Session.deinit calls invalidateAndCancel), and releasing that URLSession runs swift-corelibs-foundation's URLSession._MultiHandle.deinit. That deinit calls curl_multi_remove_handle and curl_multi_cleanup, which synchronously re-enter the registered CURLMOPT_TIMERFUNCTION callback. For a zero timeout the callback reaches updateTimeoutTimer(to: .immediate):

case .immediate:
    timeoutSource = nil
    queue.async { nonisolatedSelf.timeoutTimerFired() }

which takes a strong reference to the object currently being deinitialized. The Swift runtime detects the resurrected reference and aborts the process (sometimes it segfaults on the dangling reference instead). Source: swift-corelibs-foundation, swift-6.3-RELEASE, Sources/FoundationNetworking/URLSession/libcurl/MultiHandle.swift.

The crash backtrace from the local repro confirms the path end to end:

 9 URLSession.deinit in libFoundationNetworking.so
10 URLSession.__deallocating_deinit in libFoundationNetworking.so
13 closure #1 in CompositeEventMonitor.urlSession(_:didBecomeInvalidWithError:)

It is a race, so it only fires under scheduling pressure, which a 512 MB Lambda has plenty of.

The fix

The three distinct session configurations become process-lifetime static lets on NetworkRequestManager (browserSession, shortTimeoutBrowserSession, rangeProbeSession), keeping exactly the headers and timeouts each caller had before. Nothing is ever deallocated, so the teardown never runs. DigitalNZ requests already used Alamofire's process-lifetime AF global, which is why the two DigitalNZ calls in the logs never crashed.

Alternative considered and rejected for now: migrating to AsyncHTTPClient. It would also avoid corelibs URLSession, but it rewrites every request path, validation, decoding, redirect and HEAD/Range handling across 50+ collection strategies, for no additional benefit over a session that is never released.

Verification

Linux container (swift:6.3-amazonlinux2023, arm64, --cpus 0.3 -m 512m, plus busy loops for scheduling pressure) running the Lambda's local server, hammering POST /invoke with random collections:

Build Result
Before ok=35 fail=5 of 40, all 5 with the _MultiHandle fatal error
After ok=60 fail=0 of 60, zero fatal errors

CPU pressure is required to reproduce: unstressed runs pass 60/60 even on the broken build.

swift test: 132 tests, 0 failures.

Regression test

Tests/NZImageApiLambdaTests/NetworkRequestManagerSessionTests.swift checks each shared session still carries its expected User-Agent, Range header and timeout, and scans the Lambda target's source for any session construction outside the single shared factory. That source scan was verified to actually fail: reintroducing a per-call Session(configuration:) makes it report the offending file and line.

Docs

CLAUDE.md gains a gotcha, and .claude/rules/architecture.md gains an "HTTP client lifecycle" section explaining the rule and how to re-verify it.

🤖 Generated with Claude Code

… the process

`/image` returned a 500 on roughly one request in four. The handler always
finished its work first; the process then died with:

    Object 0x... of class _MultiHandle deallocated with non-zero retain count 2.

and Lambda reported `Runtime.ExitError`.

`NetworkRequestManager` built a fresh Alamofire `Session` on every call, and
`URLProcessor` calls into it up to a dozen times per request. On Linux an
Alamofire `Session` owns a `URLSession`, and releasing it runs
swift-corelibs-foundation's `URLSession._MultiHandle.deinit`. That deinit calls
`curl_multi_remove_handle`/`curl_multi_cleanup`, which synchronously re-enter the
registered `CURLMOPT_TIMERFUNCTION`; for a zero timeout that reaches
`updateTimeoutTimer(to: .immediate)`, which does `queue.async { nonisolatedSelf... }`
and takes a strong reference to the object being deinitialized. The Swift runtime
turns that into a fatal error, or segfaults on the dangling reference.

The three per-call sessions become process-lifetime statics, keeping their exact
headers and timeouts. Nothing is deallocated, so the teardown never runs. The
DigitalNZ requests already went through Alamofire's process-lifetime `AF`.

Reproduced in a Linux container (swift:6.3-amazonlinux2023, arm64) running the
Lambda's local server under CPU pressure: 5 of 40 requests failed, all five with
the `_MultiHandle` fatal error. After the fix, 60 of 60 passed under the same
harness.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 24, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 0dcee75b-f996-402b-b6bc-81f557197e48


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@bradwindy
bradwindy merged commit 978c56b into main Aug 24, 2026
4 checks passed
@bradwindy
bradwindy deleted the claude/nervous-sammet-cada62 branch August 24, 2026 09:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant