Active Bayesian Multi-class Mapping from
Range and Semantic Segmentation Observations

Arash Asgharivaskasi
Nikolay Atanasov
Department of Electrical and Computer Engineering
Contextual Robotics Institute
University of California, San Diego

Many robot applications call for autonomous exploration and mapping of unknown and unstructured environments. Information-based exploration techniques, such as Cauchy-Schwarz quadratic mutual information (CSQMI) and fast Shannon mutual information (FSMI), have successfully achieved active binary occupancy mapping with range measurements. However, as we envision robots performing complex tasks specified with semantically meaningful objects, it is necessary to capture semantic categories in the measurements, map representation, and exploration objective. This work develops a Bayesian multi-class mapping algorithm utilizing range-category measurements. We derive a closed-form efficiently computable lower bound for the Shannon mutual information between the multi-class map and the measurements. The bound allows rapid evaluation of many potential robot trajectories for autonomous exploration and mapping. We compare our method against frontier-based and FSMI exploration and apply it in a 3-D photo-realistic simulation environment.


In this work, we want to construct a semantic map of the environment using range-category observations by finding the most informative trajectory.


We first generalize log-odds mapping to semantic environments by defining a range-category observation model.

For information computation, we derive a closed-form lower bound for the Shannon mutual information between the semantic map and range-category observations; leading to a summation over all object classes and map cells along the ray.

Unlike traditional class-agnostic exploration methods, our semantic information measure incorporates the uncertainty of different classes, leading to more accurate exploration. For example, in the figure below, all three colored edges have the same occupancy probability, while their per-class probabilities vary. This introduces different semantic information values between the bottom and lateral edges, while the information measured by FSMI cannot detect the disparity.




We gratefully acknowledge support from ARL DCIST CRA W911NF-17-2-0181 and ONR N00014-18-1-2828. The Unity simulation used for evaluation is developed by ARL for the DCIST project.
This webpage template was borrowed from Thai Duong.