Skip to content
Ivan edited this page Nov 26, 2013 · 2 revisions

This page contains explanation how the code should be run to reproduce results in the paper.

Preprocessing

Extraction of the object trajectories

In preprocessing/Background Subtraction/sparse velocity run main.m. You might need to update folders containing sparse energy and frames from the videos.

Head trajectories extraction

Algorithmically

  1. detect head in each of the first 10 frames. Number of detections per frames can be bigger than 1.
  2. filter detections
  3. track each detection using TLD famework.
  4. assign to each detection it's own weight and a label, if tracked detection coincide with the current one merge their label and add 1 to their weight.
  5. best trajectory is chosen as the one having the highest weight.
  6. track the best trajectory until the beginning and until the end to get full trajectory.

Programmatically

All files are in the folder: cd /home/ibogun2010/Code/interaction-recognition/preprocessing/head detection/face-release1.0-basic

This detects heads (step 1) detectFaces.m. This should return a cell array headDetectedTrajectories.

Filtering and further post-processing are in the directory cd /home/ibogun2010/Code/OpenTLD/TrackingHeads. Run filterDetectionsWhichOvelapTooMuch.m to delete static detections and the ones which are below the torso point ( script uses output from the previous step, for convenience the data is saved in headDetectionsAllLowThreshold.mat file). Result is the cell array filteredAllDetections . This finishes step 2.

Run main.m to finish steps 3 and 4. This will give indices of the 'good' detections. Using good detections we can initialize the tracker and track it both directions. trackingProcessedDetections.m is doing just that, it takes detections using output of the main.m ( detections[good indices] ) and tracks it both direction.

Some trajectories might contain NaN, so we post-process them using postProcessTrajectories.m to get clean trajectories.

Trajectories normalization

In the folder preprocessing run prepareDifferentSizeDataCalculatedTrajectories.m. Head trajectories cell array should be used here for normalization.

Object localization

  • main.m to get location of a sparse energy larger than a threshold on the lowest position.
  • prepareLocalizedPairs.m will extract big patches around the object before and during interaction.
  • calculateImageDifferences.m will calculate differences between images with and without the object ( for better repositioning)
  • finalizeLocalizations.m will extract small bounding box around objects in the first 10 frames with better centering.

Learning

Object recognition

Once objects are localized cd /Interaction_appearance/ and run pLSA_CrossValidationRunsForBestNumberOfClusters.m. Unfortunately the code is written so that the objects will be reordered as (flashlight,cup,...,phone), so they will need to be reordered back to the ordering. Also, this scripts takes a lot of time to execute.

Interaction recognition

cd experiments/MKSVM and run . (code at home -> push it and finish this page)