Semantic parsing for priming object detection in indoors RGB-D scenes

Abstract

The semantic mapping of the environment requires simultaneous segmentation and categorization of the acquired stream of sensory information. The existing methods typically consider the semantic mapping as the final goal and differ in the number and types of considered semantic categories. We envision semantic understanding of the environment as an on-going process and seek representations which can be refined and adapted depending on the task and robot’s interaction with the environment. In this work we propose a novel and efficient method for semantic parsing, which can be adapted to the task at hand and enables localization of objects of interest in indoor environments. For basic mobility tasks we demonstrate how to obtain initial semantic segmentation of the scene into ground, structure, furniture and props categories which constitute the first level of hierarchy. Then, we propose a simple and efficient method for predicting locations of objects that based on their size afford a manipulation task. In our experiments we use the publicly available NYU V2 dataset and obtain better or comparable results than the state of the art at a fraction of the computational cost. We show the generalization of our approach on two more publicly available datasets.