[BUG] - swarm particles being lost across processors resulting in 'gaps'

[bknight1]

I am running a model with swarm particles but large gaps are appearing within the domain, which correlates to processor boundaries. I haven't got a basic script to replicate but can make one if required.

[bknight1]

I should add this is when the particles are being advected around the domain

[bknight1]

Reproduction script for np > 2:

import underworld3 as uw
import numpy as np
from mpi4py import MPI

mesh = uw.meshing.UnstructuredSimplexBox(
    minCoords=(0.0, 0.0),
    maxCoords=(1.0, 1.0),
    cellSize=0.1,
)

swarm = uw.swarm.Swarm(mesh)

# Initialize a variable to track particles
var = uw.swarm.SwarmVariable("test_var", swarm, 1)

swarm.populate(fill_param=1)
var.data[...] = uw.mpi.rank

# Initial count
comm = uw.mpi.comm
initial_count = comm.allreduce(swarm.dm.getLocalSize(), op=MPI.SUM)
if uw.mpi.rank == 0:
    print(f"Initial total particles: {initial_count}")

# Define a constant velocity field to move particles from left to right
# V = (1.0, 0.0)
import sympy
v_fn = sympy.Matrix([1.0, 0.0])

# Advect for a bit
dt = 0.6 # move 0.4 units, should cross x=0.5 if they start at x < 0.5
swarm.advection(v_fn, dt, order=1)

# Final count
final_count = comm.allreduce(swarm.dm.getLocalSize(), op=MPI.SUM)
if uw.mpi.rank == 0:
    print(f"Final total particles: {final_count}")

if uw.mpi.rank == 0:
    if final_count < initial_count:
        print(f"FAILED: Lost {initial_count - final_count} particles!")
    else:
        print("PASSED: No particles lost.")

Looks like the advection function is forced to use max_steps=1, causing issues. Seems a simple fix but will investigate further before finalising

[lmoresi]

Definitely a regression- seen this before !

…

--- 8< ------------------------------ Prof Louis Moresi Computational Geophysics Research School of Earth Sciences The Australian National University +61 2 6125 5590 ***@***.*** TEQSA Provider ID: PRV12002 (Australian University) | CRICOS Provider Code: 00120C The Australian National University acknowledges, celebrates and pays our respects to the Ngunnawal and Ngambri people of the Canberra region and to all First Nations Australians on whose traditional lands we meet and work, and whose cultures are among the oldest continuing cultures in human history.

________________________________ From: Ben Knight ***@***.***> Sent: Friday, May 8, 2026 12:30:43 PM To: underworldcode/underworld3 ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [underworldcode/underworld3] [BUG] - swarm particles being lost across processors resulting in 'gaps' (Issue #175) [https://avatars.githubusercontent.com/u/55677727?s=20&v=4]bknight1 left a comment (underworldcode/underworld3#175)<#175 (comment)> Reproduction script for np > 2: import underworld3 as uw import numpy as np from mpi4py import MPI mesh = uw.meshing.UnstructuredSimplexBox( minCoords=(0.0, 0.0), maxCoords=(1.0, 1.0), cellSize=0.1, ) swarm = uw.swarm.Swarm(mesh) # Initialize a variable to track particles var = uw.swarm.SwarmVariable("test_var", swarm, 1) swarm.populate(fill_param=1) var.data[...] = uw.mpi.rank # Initial count comm = uw.mpi.comm initial_count = comm.allreduce(swarm.dm.getLocalSize(), op=MPI.SUM) if uw.mpi.rank == 0: print(f"Initial total particles: {initial_count}") # Define a constant velocity field to move particles from left to right # V = (1.0, 0.0) import sympy v_fn = sympy.Matrix([1.0, 0.0]) # Advect for a bit dt = 0.6 # move 0.4 units, should cross x=0.5 if they start at x < 0.5 swarm.advection(v_fn, dt, order=1) # Final count final_count = comm.allreduce(swarm.dm.getLocalSize(), op=MPI.SUM) if uw.mpi.rank == 0: print(f"Final total particles: {final_count}") if uw.mpi.rank == 0: if final_count < initial_count: print(f"FAILED: Lost {initial_count - final_count} particles!") else: print("PASSED: No particles lost.") Looks like the advection function is forced to use max_steps=1<https://github.com/underworldcode/underworld3/blob/93d02801ef342adb0367628de71e7daf2aa08430/src/underworld3/swarm.py#L4543>, causing issues. Seems a simple fix but will investigate further before finalising — Reply to this email directly, view it on GitHub<#175 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADABPI2HAI2IO7X7NX27YWD4ZVBFHAVCNFSM6AAAAACYUGWWPGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DIMBSHAYTOOBVGQ>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[lmoresi]

@bknight1 — your diagnosis was on the right track. The regression came from a max_its=1 argument on the end-of-advection self.migrate(...) call (introduced in bd209c4, looks like leftover debugging). With max_its=1 only the closest domain centroid is tried for each unclaimed particle, so anything whose owner is the 2nd-or-3rd closest rank — typical near processor cuts — failed points_in_domain and got dropped.

PR #177 drops the override so the default max_its=10 retry is used. Your reproducer now reports zero loss on np=4. Could you give it a run at higher proc counts on the HPC when you get a chance?