by Keyword: Adaptive control
Freire IT, Amil AF, Vouloutsi V, Verschure PFMJ, (2021). Towards sample-efficient policy learning with DAC-ML Procedia Computer Science 190, 256-262 
The sample-inefficiency problem in Artificial Intelligence refers to the inability of current Deep Reinforcement Learning models to optimize action policies within a small number of episodes. Recent studies have tried to overcome this limitation by adding memory systems and architectural biases to improve learning speed, such as in Episodic Reinforcement Learning. However, despite achieving incremental improvements, their performance is still not comparable to how humans learn behavioral policies. In this paper, we capitalize on the design principles of the Distributed Adaptive Control (DAC) theory of mind and brain to build a novel cognitive architecture (DAC-ML) that, by incorporating a hippocampus-inspired sequential memory system, can rapidly converge to effective action policies that maximize reward acquisition in a challenging foraging task.
JTD Keywords: Cognitive architecture, Distributed adaptive control, Reinforcement learning, Sample-inefficiency problem, Sequence learning
Guerrero, O., Verschure, P., (2020). Distributed adaptive control: An ideal cognitive architecture candidate for managing a robotic recycling plant Biomimetic and Biohybrid Systems 
9th International Conference, Living Machines 2020 (Lecture Notes in Computer Science) , Springer International Publishing (Freiburg, Germany) 12413, 153-164
In the past decade, society has experienced notable growth in a variety of technological areas. However, the Fourth Industrial Revolution has not been embraced yet. Industry 4.0 imposes several challenges which include the necessity of new architectural models to tackle the uncertainty that open environments represent to cyber-physical systems (CPS). Waste Electrical and Electronic Equipment (WEEE) recycling plants stand for one of such open environments. Here, CPSs must work harmoniously in a changing environment, interacting with similar and not so similar CPSs, and adaptively collaborating with human workers. In this paper, we support the Distributed Adaptive Control (DAC) theory as a suitable Cognitive Architecture for managing a recycling plant. Specifically, a recursive implementation of DAC (between both single-agent and large-scale levels) is proposed to meet the expected demands of the European Project HR-Recycler. Additionally, with the aim of having a realistic benchmark for future implementations of the recursive DAC, a micro-recycling plant prototype is presented.
JTD Keywords: Cognitive architecture, Distributed Adaptive Control, Recycling plant, Navigation, Motor control, Human-Robot Interaction
Moulin-Frier, C., Fischer, T., Petit, M., Pointeau, G., Puigbo, J., Pattacini, U., Low, S. C., Camilleri, D., Nguyen, P., Hoffmann, M., Chang, H. J., Zambelli, M., Mealier, A., Damianou, A., Metta, G., Prescott, T. J., Demiris, Y., Dominey, P. F., Verschure, P. F. M. J., (2018). DAC-h3: A proactive robot cognitive architecture to acquire and express knowledge about the world and the self IEEE Transactions on Cognitive and Developmental Systems 10, (4), 1005-1022
This paper introduces a cognitive architecture for a humanoid robot to engage in a proactive, mixed-initiative exploration and manipulation of its environment, where the initiative can originate from both the human and the robot. The framework, based on a biologically-grounded theory of the brain and mind, integrates a reactive interaction engine, a number of state-of-the art perceptual and motor learning algorithms, as well as planning abilities and an autobiographical memory. The architecture as a whole drives the robot behavior to solve the symbol grounding problem, acquire language capabilities, execute goal-oriented behavior, and express a verbal narrative of its own experience in the world. We validate our approach in human-robot interaction experiments with the iCub humanoid robot, showing that the proposed cognitive architecture can be applied in real time within a realistic scenario and that it can be used with naive users.
JTD Keywords: Autobiographical Memory., Biology, Cognition, Cognitive Robotics, Computer architecture, Distributed Adaptive Control, Grounding, Human-Robot Interaction, Humanoid robots, Robot sensing systems, Symbol Grounding
Verschure, P., (2018). A chronology of Distributed Adaptive Control Living machines: A handbook of research in biomimetics and biohybrid systems (ed. Prescott, T. J., Lepora, Nathan, Verschure, P.), Oxford Scholarship (Oxford, UK) , 346-360
This chapter presents the Distributed Adaptive Control (DAC) theory of the mind and brain of living machines. DAC provides an explanatory framework for biological brains and an integration framework for synthetic ones. DAC builds on several themes presented in the handbook: it integrates different perspectives on mind and brain, exemplifies the synthetic method in understanding living machines, answers well-defined constraints faced by living machines, and provides a route for the convergent validation of anatomy, physiology, and behavior in our explanation of biological living machines. DAC addresses the fundamental question of how a living machine can obtain, retain, and express valid knowledge of its world. We look at the core components of DAC, specific benchmarks derived from the engagement with the physical and the social world (the H4W and the H5W problems) in foraging and human–robot interaction tasks. Lastly we address how DAC targets the UTEM benchmark and the relation with contemporary developments in AI.
JTD Keywords: Distributed Adaptive Control, Problem of priors, Symbol grounding problem, Convergent validation, Foraging, brain, Architecture, system
Rajasekaran, Vijaykumar, Aranda, Joan, Casals, Alicia, Pons, Jose L., (2015). An adaptive control strategy for postural stability using a wearable robot Robotics and Autonomous Systems , 73, 16-23
Abstract Wearable robots are expected to expand the use of robotics in rehabilitation since they can widen the assistance application context. An important aspect of a rehabilitation therapy, in terms of lower extremity assistance, is balance control. In this article, we propose and evaluate an adaptive control strategy for robotic rehabilitation therapies to guarantee static stability using a wearable robot. Postural balance control can be implemented either acting on the hip, on the ankle joint or on both, depending on the kind of perturbation acting on the subject: internal or external. Internal perturbations can be produced by any voluntary movement of the body, such as bending the trunk. External perturbations, in the form of an impact force, are applied by the exoskeleton without any prior notice to observe the proactive response of the subject. We have used a 6 degree of freedom planar lower limb exoskeleton, H1, to perform this analysis. The developed control strategy has been designed to provide the necessary assistance, related to balance recovery and postural stability, under the “Assist-as-needed†paradigm. The interaction forces between orthosis and subject are monitored, as they play a relevant role in the definition of assistive and resistive movements to be applied to the joints. The proposed method has been tested with 5 healthy subjects in presence of internal and external disturbances. The results demonstrate that knowing the stability limit of each subject, in combination with a therapeutically selected scaling factor, the proposed adaptive control helps in providing an effective assistance in therapy. This method is efficient in handling the individual and combined effect of external perturbations acting on any joint movements.
JTD Keywords: Exoskeleton controls, Postural stability, Balance controls, Adaptive control
Rajasekaran, V., Aranda, J., Casals, A., (2015). Compliant gait assistance triggered by user intention Engineering in Medicine and Biology Society (EMBC) 37th Annual International Conference of the IEEE , IEEE (Milan, Italy) , 3885-3888
An automatic gait initialization strategy based on user intention sensing in the context of rehabilitation with a lower-limb wearable robot is proposed and evaluated. The proposed strategy involves monitoring the human-orthosis interaction torques and initial position deviation to determine the gait initiation instant and to modify orthosis operation for gait assistance, when needed. During gait, the compliant control algorithm relies on the adaptation of the joints' stiffness in function of their interaction torques and their deviation from the desired trajectories, while maintaining the dynamic stability. As a reference input, the average of a set of recorded gaits obtained from healthy subjects is used. The algorithm has been tested with five healthy subjects showing its efficient behavior in initiating the gait and maintaining the equilibrium while walking in presence of external forces. The work is performed as a preliminary study to assist patients suffering from incomplete Spinal cord injury and Stroke.
JTD Keywords: Biomedical monitoring, Exoskeletons, Joints, Knee, Legged locomotion, Trajectory, Exoskeleton, adaptive control, gait assistance, gait initiation, rehabilitation, wearable robot
Rajasekaran, V., Aranda, J., Casals, A., (2015). User intention driven adaptive gait assistance using a wearable exoskeleton Robot 2015: Second Iberian Robotics Conference (ed. Paulo Reis, L., Paulo Moreira, A., Lima, P. U., Montano, L., Muñoz-Martinez, V.), Springer International (Lausanne, Switzerland) 418, 289-301
A user intention based rehabilitation strategy for a lower-limb wearable robot is proposed and evaluated. The control strategy, which involves monitoring the human-orthosis interaction torques, determines the gait initiation instant and modifies orthosis operation for gait assistance, when needed. Orthosis operation is classified as assistive or resistive in function of its evolution with respect to a normal gait pattern. The control algorithm relies on the adaptation of the joints’ stiffness in function of their interaction torques and their deviation from the desired trajectories. An average of recorded gaits obtained from healthy subjects is used as reference input. The objective of this work is to develop a control strategy that can trigger the gait initiation from the user’s intention and maintain the dynamic stability, using an efficient real-time stiffness adaptation for multiple joints, simultaneously maintaining their synchronization. The algorithm has been tested with five healthy subjects showing its efficient behavior in initiating the gait and maintaining the equilibrium while walking in presence of external forces. The work is performed as a preliminary study to assist patients suffering from incomplete Spinal cord injury and Stroke.
JTD Keywords: Adaptive control, Exoskeleton, Gait assistance, Gait initiation, Wearable robot
