Skip to content

fix: stop unbounded memory growth that hangs the OS - #66

Open
abolfazlakbarzadeh wants to merge 5 commits into
CluvexStudio:mainfrom
abolfazlakbarzadeh:fix/memory-usage
Open

fix: stop unbounded memory growth that hangs the OS#66
abolfazlakbarzadeh wants to merge 5 commits into
CluvexStudio:mainfrom
abolfazlakbarzadeh:fix/memory-usage

Conversation

@abolfazlakbarzadeh

@abolfazlakbarzadeh abolfazlakbarzadeh commented Jul 28, 2026

Copy link
Copy Markdown

Summary

Aether’s RSS climbed during normal use until the machine hung. This PR plugs the main leak and growth paths across netstack, WireGuard obfuscation, SOCKS teardown, MASQUE capsule parsing, and gool reconnect.

Root causes addressed

  1. DNS / SOCKS UDP socket leak (netstack)
    Hostname connects open a UDP socket for DNS, then drop the sender without UdpClose. UdpSender/TcpSender had no Drop cleanup, and service_udp did not remove entries when the app channel died. Each leak held large smoltcp buffers (~64–256 KiB), so browsing many domains grew memory steadily on every protocol.

  2. WireGuard obfuscation task storm
    jc_after_hs junk was spawned on every encapsulate WriteToNetwork (every outbound IP packet), not once after handshake. Keepalive junk also spawned a new task on each timer tick. Under traffic this flooded Tokio with short-lived tasks and allocations.

  3. Unbounded TCP send staging
    App data was drained from the shared data_in channel straight into uncapped per-connection pending vectors, with no backpressure when the TCP window stalled.

  4. Incomplete teardown on reconnect
    socks_task.abort() only stopped the accept loop; per-client tasks kept running. In gool, establish_wg and the UDP forwarder were fire-and-forget, so reconnect left orphan tunnels/forwarders behind.

  5. CapsuleParser growth
    Incomplete/malformed capsule streams could extend the reassembly buffer without a ceiling.

What changed

Scope Change
netstack Drop → close on TCP/UDP senders & conns; per-conn pending cap + deferral; free sockets on TimeWait; remove UDP when app is gone
wireguard Post-handshake junk once; keepalive junk inline; reuse encrypt/timer buffers
socks Track clients in JoinSet so server abort tears children down
masque Cap capsule buffer at 256 KiB and reset on overflow
main (gool) AbortOnDrop guards for outer/inner WG tasks and UDP forwarders

Commits

  • fix(netstack): stop socket and pending-buffer memory growth
  • fix(wireguard): prevent obfuscation task and alloc storms
  • fix(socks): abort client tasks when the proxy server stops
  • fix(masque): bound CapsuleParser reassembly buffer
  • fix(gool): abort nested tunnels and forwarders on teardown

Test plan

  • Build: cargo build --release from the aether/ tree (with sibling quiche/)
  • MASQUE: run for 20–30+ minutes of normal browsing (many hostnames); watch RSS stay flat instead of climbing
  • WireGuard with default/balanced aethernoize: sustained download/upload; confirm no task-explosion / RSS spike
  • Force tunnel drop / reconnect several times; confirm old SOCKS clients and tunnel tasks do not accumulate
  • gool (warp-in-warp): connect, use briefly, let it reconnect; confirm outer/inner tunnels and forwarders are aborted on session end
  • DNS-heavy workload (many domain SOCKS connects): confirm UDP conn count in netstack does not grow without bound
  • Smoke: SOCKS CONNECT + UDP ASSOCIATE still work; ironclad/scan paths still resolve DNS

Close TCP/UDP flows from Drop so DNS lookups and aborted SOCKS
clients release smoltcp sockets instead of leaking buffers forever.

Cap per-connection TCP pending with deferral/backpressure, free
established sockets on TimeWait, and remove UDP entries when the
app channel is closed.
Send post-handshake junk once after the first encapsulated packet
instead of spawning a task on every WriteToNetwork under traffic.

Run keepalive junk inline on the timer task, and reuse the encrypt
and timer output buffers across iterations to cut RSS churn.
Track accepted SOCKS clients in a JoinSet so aborting or dropping
the serve task tears down child handlers instead of leaving orphaned
relays that keep netstack connections alive across reconnects.
Reset the capsule byte buffer if it would exceed 256 KiB so a
malformed or stalled stream cannot grow memory without limit.
Return AbortOnDrop guards from establish_wg and spawn_udp_forwarder
so warp-in-warp reconnect drops outer/inner WireGuard tasks and the
UDP forwarder pair instead of leaking them across session loops.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant