Woven

🖥️ System Requirements

This project has been tested and is supported on the following configurations:

• Operating System

  • Linux (tested on Ubuntu 18.04 and 20.04)

• GPU Support

  • NVIDIA GPU with CUDA capability
    (tested with NVIDIA Driver Version: 555.42.06, CUDA Version: 12.5)

• Container Runtime

  • Singularity / Apptainer
    (tested with apptainer version 1.3.6-1.el8)

---

⚙️ Setup and Running

1. Clone the repository

git clone https://github.com/CNCLgithub/Woven.git
cd Woven

2. Download the singularity container and required data

bash setup.sh all

3. Run the inference process (corresponding to Equation 1 in the paper)

./run.sh julia src/exp_basic.jl <folder_index>/<scene_name>_<mass_value>_<stiffness_value>

Parameters

• <folder_index> corresponds to the scenario:

  • 1 → wind
  • 2 → drape
  • 3 → ball
  • 4 → rotate

• <scene_name> is one of: wind, drape, ball, rotate

• <mass_value> and <stiffness_value> specify the physical parameters

Example Commands

./run.sh julia src/exp_basic.jl 1/wind_0.5_2.0
./run.sh julia src/exp_basic.jl 2/drape_0.5_0.0078125
./run.sh julia src/exp_basic.jl 3/ball_0.5_0.0078125
./run.sh julia src/exp_basic.jl 4/rotate_0.5_2.0

4. Run the marginalization process (corresponding to Equation 2 in the paper)

./run.sh julia src/exp_basic_marg.jl <folder_index>/<target_scene>_<target_mass>_<target_stiffness> \
  <experiment_tag> \
  <random_seed>_<test_scene>_<test_mass>_<test_stiffness>_<random_suffix> \
  <inferred_test_mass> \
  <inferred_test_stiff> \
  <particle_weights_test>

Parameters

• <folder_index> corresponds to the scenario:

  • 1 → wind
  • 2 → drape
  • 3 → ball
  • 4 → rotate

• <target_scene> specify the scene of the target item, should be one of: wind, drape, ball, rotate.

• <target_mass> and <target_stiffness> specify the physical parameters of the target item.

• <experiment_tag> should be stiffness or mass.

• <test_scene> specify the scene of the test item.

• <test_mass> and <test_stiffness> specify the physical parameters of the test item.

• <inferred_test_mass>: Underscore (_) separated list of inferred mass values of the test item from all particles in the final step of the particle filter. For example, if you use 20 particles, this will contain 20 mass values (one for each particle). Example: 0.25_0.5_0.5_0.75_0.6_... (20 values total).

• <inferred_test_stiff>: Underscore (_) separated list of inferred stiffness values of the test item from all particles in the final step of the particle filter. For example, if you use 20 particles, this will contain 20 stiffness values (one for each particle).

• <particle_weights_test>: Underscore (_) separated list of weights for the particles used in the inference of the test item. Each weight corresponds to one particle's posterior probability from the final step of the particle filter. The weights should sum to 1 (up to numerical precision). Example: 0.05_0.10_0.07_0.08_... (20 values total).

Example Commands

bash demo_marg.sh

---

🚀 Performing the Experiment

Take the stiffness matching task as an example. Suppose we have a trial consisting of:

• Test (match): wind_0.5_0.0078125
• Target: drape_2.0_0.0078125
• Test (distractor): rotate_0.25_2.0

We proceed as follows:

1. Run the inference process (see step 3 above) separately for each of the two test items:

   • This produces 20 particles per test item.
   • Each particle carries an inferred mass and stiffness, along with an associated weight from the particle filter.

2. For each test item, use its particle filter results as the prior to run the marginalization process on the target item (see step 4 above).

3. Repeat this process for every trial in the dataset.

---

Model Parameterization

The key tunable parameters of the model are listed below. These can be adjusted to optimize performance or explore model behavior under different settings.

In src/exp_basic.jl:

num_particles = 20      # Number of SMC particles
total_masks   = 3       # Defines length of flow mask

In src/model/generative_model.jl:

# Defines mass random walk
MASS_VAR_SMALL = 0.04
MASS_VAR_LARGE = 0.8
MASS_BERNOULLI = 0.8
# Defines stiffness random walk
STIFF_VAR_SMALL = 0.02
STIFF_VAR_LARGE = 0.4
STIFF_BERNOULLI = 0.8
# Defines observation noise
DEPTH_MAP_VAR = 8
# Defines wind random walk
WIND_VAR = 0.3