Author/Contact: Gaoang Wang
Abstract
Autonomous systems have to understand the environment and make the right decisions. Current autonomous systems often focus on hand-crafted rules or purely learned models from a limited amount of biased data. Learning the semantic information of the surroundings and obtaining knowledge from humans are essential for autonomous systems. To meet this goal, we propose the situation aware system, an autonomous system that can actively learn from the environment by incorporating human knowledge to make trustworthy decisions and discover novel events. We explore the situation learning strategies to bridge models, data, and knowledge for reliable and predictable situation awareness systems.
This study topic specifically explores the learning for situation awareness from three different angles (DUR: Diversity, Uncertainty, and Reliability):
- Multi-sensor and multi-modality representation learning (Diversity). Multiple sensors with different modalities can be installed in the autonomous systems. In some cases, the system can also access to the data captured by environment. How to mine the useful semantic information from noisy data and collaborate with different sources and modalities for representation learning is in high demand.
- Learning for rare and unseen situation discovery (Uncertainty). Humans have a good sense of danger discovery in the environment even if the situation has never happened before. The ability of discovering the rare and unseen situation is important for autonomous systems. Situation learning for novelty discovery, anomaly detection, and out-of-distribution generalization should be much enhanced in future autonomous systems.
- Learning for trustworthy and explainable systems (Reliability). Reasoning is highly required in future autonomous systems. The decisions by the system should be explainable to humans. To realize this goal, the system should take account of priors of the situation, the common knowledges of humans, and the latent causality from input to output. With the combination of data, knowledge and reasoning, autonomous systems can become trustworthy and explainable in the decision making.
Video links
Multi-modal learning
Reasoning for discovery
Reliability for autonomous vehicles
You might be interested to explore more on the topic:
- Hongmei H., Gray J., Cangelosi A., Meng Q., McGinnity T. M., Mehnen J. The Challenges and Opportunities of Artificial Intelligence for Trustworthy Robots and Autonomous Systems. In Proc. 3rd International Conference on Intelligent Robotic and Control Engineering (IRCE), pp. 68-74. IEEE, 2020.
- Bride H., Dong J.S., Hóu Z., MahonyB., Oxenham M. Towards Trustworthy AI for Autonomous Systems. In Proc. International Conference on Formal Engineering Methods, pp. 407-411. Springer, Cham, 2018.
- Cunningham S. Causal Inference – The Mixtape. Yale University Press, 2020.
- Pearl J., Glymour M., Jewell N.P. Causal inference in statistics: A primer. John Wiley & Sons, 2016.
- Janai J., Güney F., Behl A., Geiger A. Computer vision for autonomous vehicles: Problems, datasets and state of the art. Foundations and Trends in Computer Graphics and Vision 12, Now Publishers Inc. 2020.
- Tang Y., Zhao C., Wang J., Zhang C., Sun Q., Zheng W., Du W., Qian F., Kurths J. An overview of perception and decision-making in autonomous systems in the era of learning. arXiv:2001.02319, 2020.
- Guo, W., Wang J., Wang S. Deep multimodal representation learning: A survey. IEEE Access 7: 63373-63394, 2019.
- Pang G., Shen C., Cao L, Van Den Hengel A. Deep learning for anomaly detection: A review. ACM Computing Surveys, 54(2):1-38, 2021.
- Kiran B. R., Sobh I., Talpaert V., Mannion P., Al Sallab A. A., Yogamani S., Pérez, P. Deep reinforcement learning for autonomous driving: A survey. IEEE Transactions on Intelligent Transportation Systems, 2021.
ASI Forum board member in charge: Gaoang Wang
Gaoang Wang joined the international campus of Zhejiang University as an Assistant Professor in September 2020. He is also an Adjunct Assistant Professor at University of Illinois Urbana-Champaign. Gaoang Wang received a B.S. degree at Fudan University in 2013, a M.S. degree at the University of Wisconsin-Madison in 2015, and a Ph.D. degree from the Information Processing Laboratory of the Electrical and Computer Engineering department at the University of Washington in 2019. After that, he joined Megvii US research center in July 2019 as a research scientist working on multi-frame fusion. He then joined Wyze Labs in November 2019 working on deep neural network design for edge-cloud collaboration. His research interests are computer vision, machine learning, artificial intelligence, including multi-object tracking, representation learning, and active learning. Gaoang Wang published papers in many renowned journals and conferences, including IEEE T-IP, IEEE T-CSVT, IEEE T-VT, CVPR, ICCV, ACM MM, etc.
Personal Website: https://person.zju.edu.cn/en/gaoangwang