Publications

The sample-inefficiency problem in Artificial Intelligence refers to the inability of current Deep Reinforcement Learning models to optimize action policies within a small number of episodes. Recent studies have tried to overcome this limitation by adding memory systems and architectural biases to improve learning speed, such as in Episodic Reinforcement Learning. However, despite achieving incremental improvements, their performance is still not comparable to how humans learn behavioral policies. In this paper, we capitalize on the design principles of the Distributed Adaptive Control (DAC) theory of mind and brain to build a novel cognitive architecture (DAC-ML) that, by incorporating a hippocampus-inspired sequential memory system, can rapidly converge to effective action policies that maximize reward acquisition in a challenging foraging task.

JTD Keywords: Cognitive architecture, Distributed adaptive control, Reinforcement learning, Sample-inefficiency problem, Sequence learning

Animals in their ecological context behave not only in response to external events, such as opportunities and threats but also according to their internal needs. As a result, the survival of the organism is achieved through regulatory behaviour. Although homeostatic and allostatic principles play an important role in such behaviour, how an animal's brain implements these principles is not fully understood yet. In this paper, we propose a new model of regulatory behaviour inspired by the functioning of the medial Reticular Formation (mRF). This structure is spread throughout the brainstem and has shown generalized Central Nervous System (CNS) arousal control and fundamental action-selection properties. We propose that a model based on the mRF allows the flexibility needed to be implemented in diverse domains, while it would allow integration of other components such as place cells to enrich the agent's performance. Such a model will be implemented in a mobile robot that will navigate replicating the behaviour of the sand-diving lizard, a benchmark for regulatory behaviour. © 2020 Elsevier B.V.. All rights reserved.

JTD Keywords: Action selection, Allostasi, Allostasis, Animal brain, Animals, Behavior-based, Brainstem, Central nervous systems, Cognitive architecture, Cognitive architectures, Elsevier, Homeostasis, Homoeostasis, Magnetorheological fluids, Regulatory behavior, Regulatory behaviour, Reticular formation, Robots

In the past decade, society has experienced notable growth in a variety of technological areas. However, the Fourth Industrial Revolution has not been embraced yet. Industry 4.0 imposes several challenges which include the necessity of new architectural models to tackle the uncertainty that open environments represent to cyber-physical systems (CPS). Waste Electrical and Electronic Equipment (WEEE) recycling plants stand for one of such open environments. Here, CPSs must work harmoniously in a changing environment, interacting with similar and not so similar CPSs, and adaptively collaborating with human workers. In this paper, we support the Distributed Adaptive Control (DAC) theory as a suitable Cognitive Architecture for managing a recycling plant. Specifically, a recursive implementation of DAC (between both single-agent and large-scale levels) is proposed to meet the expected demands of the European Project HR-Recycler. Additionally, with the aim of having a realistic benchmark for future implementations of the recursive DAC, a micro-recycling plant prototype is presented.

JTD Keywords: Cognitive architecture, Distributed Adaptive Control, Recycling plant, Navigation, Motor control, Human-Robot Interaction

We present a new computational framework for modeling moral decision-making in artificial agents based on the notion of ‘Machine Morality as Cooperation’. This framework integrates recent advances from cross-disciplinary moral decision-making literature into a single architecture. We build upon previous work outlining cognitive elements that an artificial agent would need for exhibiting latent morality, and we extend it by providing a computational realization of the cognitive architecture of such an agent. Our work has implications for cognitive and social robotics. Recent studies in human neuroimaging have pointed to three different decision-making processes, Pavlovian, model-free and model-based, that are defined by distinct neural substrates in the brain. Here, we describe how computational models of these three cognitive processes can be implemented in a single cognitive architecture by using the distributed and hierarchical organization proposed by the DAC theoretical framework. Moreover, we propose that a pro-social drive to cooperate exists at the Pavlovian level that can also bias the rest of the decision system, thus extending current state-of-the-art descriptive models based on harm-aversion.

JTD Keywords: Morality, Moral decision-making, Computational models, Cognitive architectures, Cognitive robotics, Human-robot interaction

In this paper, we argue that the future of Artificial Intelligence research resides in two keywords: integration and embodiment. We support this claim by analyzing the recent advances in the field. Regarding integration, we note that the most impactful recent contributions have been made possible through the integration of recent Machine Learning methods (based in particular on Deep Learning and Recurrent Neural Networks) with more traditional ones (e.g. Monte-Carlo tree search, goal babbling exploration or addressable memory systems). Regarding embodiment, we note that the traditional benchmark tasks (e.g. visual classification or board games) are becoming obsolete as state-of-the-art learning algorithms approach or even surpass human performance in most of them, having recently encouraged the development of first-person 3D game platforms embedding realistic physics. Building on this analysis, we first propose an embodied cognitive architecture integrating heterogeneous subfields of Artificial Intelligence into a unified framework. We demonstrate the utility of our approach by showing how major contributions of the field can be expressed within the proposed framework. We then claim that benchmarking environments need to reproduce ecologically-valid conditions for bootstrapping the acquisition of increasingly complex cognitive skills through the concept of a cognitive arms race between embodied agents.

JTD Keywords: Cognitive Architectures, Embodied Artificial Intelligence, Evolutionary Arms Race, Unified Theories of Cognition

This chapter introduces the “Capabilities” section of the Handbook of Living Machines. Where the previous section considered building blocks, we recognize that components or modules do not automatically make systems. Hence, in the remainder of this handbook, the emphasis is toward the capabilities of living systems and their emulation in artifacts. Capabilities often arise from the integration of multiple components and thus sensitize us to the need to develop a system-level perspective on living machines. Here we summarize and consider the 14 contributions in this section which cover perception, action, cognition, communication, and emotion, and the integration of these through cognitive architectures into systems that can emulate the full gamut of integrated behaviors seen in animals including, potentially, our own capacity for consciousness.

JTD Keywords: Action, Cognition, Cognitive architecture, Communication, Consciousness, Emotion, Perception

A major challenge in cognitive science and AI is to understand how intelligent agents might be able to predict mental states of other agents during complex social interactions. What are the computational principles of such a Theory of Mind (ToM)? In previous work, we have investigated hypotheses of how the human brain might realize a ToM of other agents in a multi-agent social scenario. In particular, we have proposed control-based cognitive architectures to predict the model of other agents in a game-theoretic task (Battle of the Exes). Our multi-layer architecture implements top-down predictions from adaptive to reactive layers of control and bottom-up error feedback from reactive to adaptive layers. We tested cooperative and competitive strategies among different multi-agent models, demonstrating that while pure RL leads to reasonable efficiency and fairness in social interactions, there are other architectures that can perform better in specific circumstances. However, we found that even the best predictive models fall short of human data in terms of stability of social convention formation. In order to explain this gap between humans and predictive AI agents, in this work we propose introducing the notion of trust in the form of mutual agreements between agents that might enhance stability in the formation of conventions such as turn-taking.

JTD Keywords: Cognitive Architectures, Game Theory, Multi-Agent Models, Reinforcement Learning, Theory of Mind