Skip to content

Add segmented object affine transformation module with benchmarking#9

Open
rojas70 wants to merge 1 commit intomainfrom
codex/create-image-transformation-module
Open

Add segmented object affine transformation module with benchmarking#9
rojas70 wants to merge 1 commit intomainfrom
codex/create-image-transformation-module

Conversation

@rojas70
Copy link
Copy Markdown

@rojas70 rojas70 commented Mar 27, 2026

Motivation

  • Provide a segmentation-first pipeline to apply Euclidean affine transforms to target objects in images rather than warping entire frames.
  • Couple image-space manipulation with robot planar state updates so pixel-space transforms can inform external robot state.
  • Offer a simple performance study to decide between full-frame vs ROI-only warping strategies.

Description

  • Add franka_env.utils.image_object_affine implementing EuclideanAffine2D, build_affine_matrix, preprocess_mask, transform_segmented_object, update_planar_robot_state, and benchmark_affine_strategies using NumPy and scipy.ndimage for warping and morphology.
  • Implement inverse-mapped warping, nearest-neighbor/linear sampling via ndimage.map_coordinates, background hole-filling via distance_transform_edt, and mask denoising with morphological close/open.
  • Export the new utilities from franka_env.utils by updating __init__.py.
  • Add unit tests in serl_robot_infra/tests/test_image_object_affine.py covering object movement, robot-state coupling, and benchmark metrics.

Testing

  • Ran pytest -q serl_robot_infra/tests/test_image_object_affine.py, which reports 3 passed after replacing an initial OpenCV-based implementation with a SciPy-based one to avoid libGL.so.1 runtime issues.
  • Benchmarks API benchmark_affine_strategies is exercised by the tests and returns the expected keys (full_frame_ms, roi_only_ms, speedup_x).

Codex Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant