SSMI: Semantic OcTree Mapping and
Shannon Mutual Information Computation for Robot Exploration


Arash Asgharivaskasi
Nikolay Atanasov
Department of Electrical and Computer Engineering
Contextual Robotics Institute
University of California, San Diego
[Conference Version]
[Journal Version]
[Source Code]

Autonomous robot operation in unstructured and unknown environments requires efficient techniques for mapping and exploration based on streaming range and visual observations. Information-based exploration techniques, such as Cauchy-Schwarz quadratic mutual information (CSQMI) and fast Shannon mutual information (FSMI), have successfully achieved active binary occupancy mapping with range measurements. However, as we envision robots performing complex tasks specified with semantically meaningful objects, it is necessary to capture semantic categories in the measurements, map representation, and exploration objective. This work presents a Semantic octree mapping and Shannon Mutual Information (SSMI) for robot exploration. In particular, we propose a compressed Bayesian multi-class mapping algorithm based on the OcTree data structure, where each voxel maintains a categorical distribution over object classes. Furthermore, we derive a closed-form efficiently computable lower bound for the Shannon mutual information between the proposed multi-class OctoMap and a set of range-category measurements using Run-Length Encoding (RLE) of observation rays. The bound allows rapid evaluation of many potential robot trajectories for autonomous exploration and mapping. We compare our method against frontier-based and FSMI exploration and apply it in a variety of simulated and real-world experiments.

Problem

In this work, we want to construct a semantic map of the environment using range-category observations by finding the most informative trajectory.

Approach

We first generalize log-odds mapping to semantic environments by defining a range-category observation model.

A multi-class OcTree data structure is proposed in order to compress the environment representation. Each OcTree node stores a categorical probability distribution over semantic labels, as well as pointers to its children nodes.

For information computation, we derive a closed-form lower bound for the Shannon mutual information between the semantic map and range-category observations; leading to a summation over all object classes and map cells along the ray.

Exploiting the OcTree structure of the environment representation, we can compress each ray-cast of the observation using run-length encoding technique, where instead of storing the multi-class probabilities of each map cell along the ray, we simply store the groups of consecutive cells that share the same multi-class probabilities, plus the size of each group. This will lead to faster evaluation of path informativeness since it reduces the number of summation terms in our mutual information formula.

Unlike traditional class-agnostic exploration methods, our semantic information measure incorporates the uncertainty of different classes, leading to more accurate exploration. For example, in the figure below, all three colored edges have the same occupancy probability, while their per-class probabilities vary. This introduces different semantic information values between the bottom and lateral edges, while the information measured by FSMI cannot detect the disparity.

Videos

Presentation

Conference Version
Journal Version

Demo

Active Binary 2-D Mapping | Python Simulation
Active Multi-class 2-D Mapping | Python Simulation
Active Multi-class 3-D Mapping | Unity Simulation
Active Multi-class 3-D Mapping | Real-world Experiment

Citation


If you find our papers/code useful for your research, please cite our work as follows:

1. A. Asgharivaskasi, N. Atanasov. Active Bayesian Multi-class Mapping from Range and Semantic Segmentation Observations. IEEE Int. Conf. on Robotics and Automation (ICRA), 2021.

@InProceedings{Asgharivaskasi-ICRA21,
  title={Active Bayesian Multi-class Mapping from Range and Semantic Segmentation Observations},
  author={Asgharivaskasi, Arash and Atanasov, Nikolay},
  booktitle={IEEE International Conference on Robotics and Automation (ICRA)},
  year={2021},
  pages={1-7},
  url = {https://arashasgharivaskasi-bc.github.io/SSMI_webpage/},
}


2. A. Asgharivaskasi, N. Atanasov. Semantic OcTree Mapping and Shannon Mutual Information Computation for Robot Exploration. IEEE Transactions on Robotics (TRO), 2023.

@Article{Asgharivaskasi-TRO23,
  title={Semantic OcTree Mapping and Shannon Mutual Information Computation for Robot Exploration},
  author={Arash Asgharivaskasi and Nikolay Atanasov},
  journal = {IEEE Transactions on Robotics},
  year={2023},
  url = {https://arashasgharivaskasi-bc.github.io/SSMI_webpage/},
}

Acknowledgements

We gratefully acknowledge support from ARL DCIST CRA W911NF-17-2-0181, ONR N00014-18-1-2828, and NSF FRR CAREER 2045945. The Unity simulation used for evaluation is developed by ARL for the DCIST project.
This webpage template was borrowed from Thai Duong.