Segmentation Layer

Overview

The segmentation layer plugin takes in 2D segmentation data from a camera and projects it onto a ROS costmap. By doing this, the layer can provide data to the navigation stack that a laser-based system normally wouldn't be able to provide. For example, a robot could detect grass as an obstacle fairly easily.

Usage

1. Set up a segmentation network.

Any method which produces a per-pixel labelling of an image will work. Note that the segmentation costmap plugin expects a "mono8" ROS image, where the value of each pixel corresponds to a different class. We recommend using the caffe_ros node, as it also provides nodes helpful for debugging. It is up to you which model to use; with the right calibration and parameter changes, the segmentation plugin will work with a model of any architecture. For example, the grass demo provided uses FCNAlexNet trained on the Cityscapes dataset (which can also be found in the caffe_ros repo). Of course, you will want to test a few images from your particular use case before adopting any particular model (which caffe_ros can help with, as seen below).

2. Calibrate your camera.

Any camera that is supported by ROS will work with the segmentation layer. However, you will need to calculate the homography matrix between the camera (in whatever position you mount it) and the costmap. To do so, we have provided tools in the calculate_homography folder. For more detailed instructions, see this page.

3. Activate the plugin.

Clone this repository, then copy or link the deep_nav_layers repo into your catkin workspace. Then rebuild your workspace. Once the plugin has been built, simply modify your costmap parameters file to include the following line in the "plugins" section (after any other plug-ins you may have):

plugins:	  
  - {name: segmentation_layer, type: "segmentation_layer::SegmentationLayer"}

The deep_nav_layers are standard ROS costmap plug-ins so you can also look at this tutorial if you need more details.