More Like this

1. Registering historical context in a spoken dialogue system for spatial question answering in a physical blocks world

   Kane, Benjamin; Platonov, Georgiy; Lenhart K. Schubert, Georgiy (September 2020, 23rd Int. Conf. on Text, Speech and Dialogue (TSD 2020))

   null (Ed.)

   Task-oriented dialogue-based spatial reasoning systems need to maintain history of the world/discourse states in order to convey that the dialogue agent is mentally present and engaged with the task, as well as to be able to refer to earlier states, which may be crucial in collaborative planning (e.g., for diagnosing a past misstep). We approach the problem of spatial memory in a multi-modal spoken dialogue system capable of answering questions about interaction history in a physical blocks world setting. We employ a pipeline consisting of a vision system, speech I/O mediated by an animated avatar, a dialogue system that robustly interprets queries, and a constraint solver that derives answers based on 3D spatial modelling. The contributions of this work include a semantic parser competent in this domain and a symbolic dialogue con- text allowing for interpreting and answering free-form historical questions using world and discourse history. 

   more » « less
2. A spoken dialogue system for spatial question answering in a physical blocks world

   Platonov, Georgiy; Kane, Benjamin; Gindi, Aaron; Schubert, Lenhart K. (June 2020, Proceedings of SigDial 2020, ISBN 978-1-952148-02-6)

   A physical blocks world, despite its relative simplicity, requires (in fully interactive form) a rich set of functional capabilities, ranging from vision to natural language understanding. In this work we tackle spatial question answering in a holistic way, using a vision system, speech input and output mediated by an animated avatar, a dialogue system that robustly interprets spatial queries, and a constraint solver that derives answers based on 3-D spatial modeling. The contributions of this work include a semantic parser that maps spatial questions into logical forms consistent with a general approach to meaning representation, a dialogue manager based on a schema representation, and a constraint solver for spatial questions that provides answers in agreement with human perception. These and other components are integrated into a multi-modal human-computer interaction pipeline. 

   more » « less
3. A References Architecture for Human Cyber Physical Systems, Part II: Fundamental Design Principles for Human-CPS Interaction

   https://doi.org/10.1145/3622880  

   Bengler, Klaus; Damm, Werner; Luedtke, Andreas; Jochem, Reiger; Austel, Benedikt; Biebl, Bianca; Fränzle, Martin; Hagemann, Willem; Held, Moritz; Hess, David; et al (January 2024, ACM Transactions on Cyber-Physical Systems)

   Chenyang Lu (Ed.)

   As automation increases qualitatively and quantitatively in safety-critical human cyber-physical systems, it is becoming more and more challenging to increase the probability or ensure that human operators still perceive key artifacts and comprehend their roles in the system. In the companion paper, we proposed an abstract reference architecture capable of expressing all classes of system-level interactions in human cyber-physical systems. Here we demonstrate how this reference architecture supports the analysis of levels of communication between agents and helps to identify the potential for misunderstandings and misconceptions. We then develop a metamodel for safe human machine interaction. Therefore, we ask what type of information exchange must be supported on what level so that humans and systems can cooperate as a team, what is the criticality of exchanged information, what are timing requirements for such interactions, and how can we communicate highly critical information in a limited time frame in spite of the many sources of a distorted perception. We highlight shared stumbling blocks and illustrate shared design principles, which rest on established ontologies specific to particular application classes. In order to overcome the partial opacity of internal states of agents, we anticipate a key role of virtual twins of both human and technical cooperation partners for designing a suitable communication. 

   more » « less
4. Smart City Concept Based on Cyber-Physical Social Systems with Hierarchical Ethical Agents Approach

   https://doi.org/10.1007/978-3-030-78095-1_31  

   Mata, O. (July 2021, 23rd International Conference on Human-Computer Interaction (HCII 2021) - Universal Access in Human-Computer Interaction. Access to Media, Learning and Assistive Environments)

   A smart city is considered a sustainable city that manages needed resources and makes autonomous decisions to improve the quality of life of its citizens. On the other hand, Cyber-Physical Systems (CPS) have been implemented as isolated systems inside the city. For instance, the traffic lights, autonomous navigation for cars, and so on. Instead, consider a smart city with an integrated CPS for independent blocks that can be interconnected in a central unit. However, when a CPS makes decisions about the integration of ethical concepts based on human perception, social space must be added, and so a CPS must be transformed into a Cyber-Physical Social System (CPSS). Furthermore, a new type of social interaction between all the elements in a CPSS within a smart city presents human behavioral challenges such as virtual-morality. This paper first proposes an Artificial Moral Agent with machine learning algorithms to regulate the interaction within the CPSS, adding itself to all the subsystems’ communication. Additionally, a moral agent structure is proposed with a morality filter as its fundamental component. 

   more » « less
5. Curbing Feature Coding: Strictly Local Feature Assignment

   Graf, Thomas (January 2020, Proceedings of the Society for Computation in Linguistics)

   Graf (2017) warns that every syntactic formalism faces a severe overgeneration problem because of the hidden power of subcategorization. Any constraint definable in monadic second-order logic can be compiled into the category system so that it is indirectly enforced as part of subcategorization. Not only does this kind of feature coding deprive syntactic proposals of their empirical bite, it also undermines computational efforts to limit syntactic formalisms via subregular complexity. This paper presents a subregular solution to feature coding. Instead of features being a cheap resource that comes for free, features must be assigned by a transduction. In particular, category features must be assigned by an input strictly local (ISL) tree-tot-tree transduction, defined here for the first time. The restriction to ISL transductions correctly rules out various deviant category systems. 

   more » « less