Human-Autonomous Systems Interaction

Author/Contact: Sabrina Iarlori

Abstract

Autonomous systems can greatly enhance human effectiveness in complex environments by handling routine or cognitively challenging operations.

The objective of this study group is to promote technology development for improved human-autonomy interaction by facilitating collaboration among researchers in human-computer interaction with a special focus on autonomous systems. It was especially interested in technology and case studies relevant to complex, applied environments in which people interact with autonomous systems regularly, particularly in the context of ambient assisted living. Such systems include autonomous control of mobile based robots, robots/systems that interact with people, and software for assisting complex human tasks, such as navigation and path planning, but also motion or action recognition through RGB-D camera or vision sensors. In particular, it is possible organize the study focusing on:

• Approaches that include task inputs from humans: how to model humans and their tasks and at what level of details 
• Studies on cycles of learning for autonomous system for human interaction: learning from human demonstration, human intervention, human evaluation, reinforcement learning techniques
• Human in-the loop approaches to facilitate the goal achievement, to reduce the anomalies to reduce unexpected responses from the system or inappropriate responses by the human in order to enhance human safety

Human-system interaction must provide people with an understanding of an autonomous system’s decisions and actions, the ability to interact at appropriate levels of abstraction, and the ability to override the system’s actions. New human-system engineering techniques are needed to ensure autonomous systems will be smoothly and readily adopted into society.

Autonomous systems that work together in the environment should integrate the connections and interactions between them, over networks, with the physical environment, and with humans—must be assured, resilient, productive, and fair in the autonomous future. Systems engineering is critical for ensuring the operational success for which the autonomous systems were intended. Autonomous systems should be analysed including concept, context, requirements, design, integration, operationalization, validation, testing and evaluation, and maintenance. The study could be organized focusing on different aspects:
• Algorithms that must sustain the intended processing workload and data storage needs under a variety of conditions
• Communications between systems and their interoperability to support highly dynamic interactions and enable cooperation across multiple systems with the human supervision
• Algorithms that allow the autonomous system to increase the levels of autonomy places managing the interactions between autonomous platforms.

In order to achieve these purposes, autonomous system interaction includes the request for information, and the support for help, or guide for action. The information for the autonomous system can be static information that is used for the analysis of objectives and evaluation or dynamic that is used for organizing future courses of behavior.

Keywords:

Robotic design and prototype; advanced sensor design; sensor and information processing; machine learning approaches; control system design and implementation; advanced materials design; human biomechanical and cognitive modelling; virtual reality; theories of cognition.

---

You might be interested to explore more on the topic:

• Waytowich, N.R., Goecks, V.G., Lawhern, V., Cycle-of-Learning for Autonomous Systems from Human Interaction. arXiv:1808.09572, 2018.

• Lundberg, J., Johansson, B.J.E. A framework for describing interaction between human operators and autonomous, automated, and manual control systems. Cognition, Technology & Work, 23, 381–401, 2021.

• Robb, David A. and Lopes, Jos\'{e} and Padilla, Stefano and Laskov, Atanas and Chiyah Garcia, Francisco J. and Liu, Xingkun and Scharff Willners, Jonatan and Valeyrie, Nicolas and Lohan, Katrin and Lane, David and Patron, Pedro and Petillot, Yvan and Chantler, Mike J. and Hastie, Helen, Exploring Interaction with Remote Autonomous Systems Using Conversational Agents, In Proc. Designing Interactive Systems Conference, pages 1543–1556, 2019.

• Ahle, E., Söffker, D., Interaction of intelligent and autonomous systems – part II: realization of cognitive technical systems, Mathematical and Computer Modelling of Dynamical Systems, 14(4):319-339, Taylor & Francis, 2008.

• Hassenzahl, M., Borchers, J., Boll, S., Rosenthal-von der Pütten, A., Wulf V. Otherware: how to best interact with autonomous systems. Interactions, 28(1):54–57. 2021.

• Shahrdar S., Menezes L., Nojoumian M. A Survey on Trust in Autonomous Systems. In Proc. Science and Information Conference: Intelligent Computing, pages 368–386, 2018.