by Keyword: Learning

By year:[ 2021 | 2020 | 2019 | 2018 | 2017 | 2016 | 2015 | 2014 | 2013 | 2012 | 2011 | 2010 | 2009 | 2008 | 2007 | 2006 | 2005 ]

Pacheco Estefan, D., Zucca, Riccardo, Arsiwalla, X. D., Principe, A., Zhang, Hui, Rocamora, R., Axmacher, N., Verschure, P., (2021). Volitional learning promotes theta phase coding in the human hippocampus Proceedings of the National Academy of Sciences of the United States of America 118, (10), e2021238118

Electrophysiological studies in rodents show that active navigation enhances hippocampal theta oscillations (4–12 Hz), providing a temporal framework for stimulus-related neural codes. Here we show that active learning promotes a similar phase coding regime in humans, although in a lower frequency range (3–8 Hz). We analyzed intracranial electroencephalography (iEEG) from epilepsy patients who studied images under either volitional or passive learning conditions. Active learning increased memory performance and hippocampal theta oscillations and promoted a more accurate reactivation of stimulus-specific information during memory retrieval. Representational signals were clustered to opposite phases of the theta cycle during encoding and retrieval. Critically, during active but not passive learning, the temporal structure of intracycle reactivations in theta reflected the semantic similarity of stimuli, segregating conceptually similar items into more distant theta phases. Taken together, these results demonstrate a multilayered mechanism by which active learning improves memory via a phylogenetically old phase coding scheme.

Keywords: Active learning, Intracranial EEG, Theta oscillations, Neural phase coding, Hippocampus

Sánchez-Fibla, M., Forestier, S., Moulin-Frier, C., Puigbò, J. Y., Verschure, P., (2020). From motor to visually guided bimanual affordance learning Adaptive Behavior 28, (2), 63-78

The mechanisms of how the brain orchestrates multi-limb joint action have yet to be elucidated and few computational sensorimotor (SM) learning approaches have dealt with the problem of acquiring bimanual affordances. We propose a series of bidirectional (forward/inverse) SM maps and its associated learning processes that generalize from uni- to bimanual interaction (and affordances) naturally, reinforcing the motor equivalence property. The SM maps range from a SM nature to a solely sensory one: full body control, delta SM control (through small action changes), delta sensory co-variation (how body-related perceptual cues covariate with object-related ones). We make several contributions on how these SM maps are learned: (1) Context and Behavior-Based Babbling: generalizing goal babbling to the interleaving of absolute and local goals including guidance of reflexive behaviors; (2) Event-Based Learning: learning steps are driven by visual, haptic events; and (3) Affordance Gradients: the vectorial field gradients in which an object can be manipulated. Our modeling of bimanual affordances is in line with current robotic research in forward visuomotor mappings and visual servoing, enforces the motor equivalence property, and is also consistent with neurophysiological findings like the multiplicative encoding scheme.

Keywords: Affordances, Bimanual affordances, Goal babbling, Interlimb coordination, Motor equivalence, Sensorimotor learning

Vouloutsi, Vasiliki, Mura, Anna, Tauber, F., Speck, T., Prescott, T. J., Verschure, P., (2020). Biomimetic and Biohybrid Systems 9th International Conference, Living Machines 2020, Freiburg, Germany, July 28–30, 2020, Proceedings , Springer, Cham (Lausanne, Switzerland) 12413, 1-428

This book constitutes the proceedings of the )th International Conference on Biomimetic and Biohybrid Systems, Living Machines 2020, held in Freiburg, Germany, in July 2020. Due to COVID-19 pandemic the conference was held virtually. The 32 full and 7 short papers presented in this volume were carefully reviewed and selected from 45 submissions. They deal with research on novel life-like technologies inspired by the scientific investigation of biological systems, biomimetics, and research that seeks to interface biological and artificial systems to create biohybrid systems.

Keywords: Artificial intelligence, Soft robotics, Biomimetics, Insect navigation, Synthetic nervous system, Computer vision, Bio-inspired materials, Visual homing, Locomotion+, Image processing, Intelligent robots, Human-robot interaction, Machine learning, Snake robot, Mobile robots, Robotic systems, Drosophila, Robots, Sensors, Signal processing

Burgués, J., Marco, S., (2019). Wind-independent estimation of gas source distance from transient features of metal oxide sensor signals IEEE Access 7, 140460-140469

The intermittency of the instantaneous concentration of a turbulent chemical plume is a fundamental cue for estimating the chemical source distance using chemical sensors. Such estimate is useful in applications such as environmental monitoring or localization of fugitive gas emissions by mobile robots or sensor networks. However, the inherent low-pass filtering of metal oxide (MOX) gas sensors-typically used in odor-guided robots and dense sensor networks due to their low cost, weight and size-hinders the quantification of concentration intermittency. In this paper, we design a digital differentiator to invert the low-pass dynamics of the sensor response, thus obtaining a much faster signal from which the concentration intermittency can be effectively computed. Using a fast photo-ionization detector as a reference instrument, we demonstrate that the filtered signal is a good approximation of the instantaneous concentration in a real turbulent plume. We then extract transient features from the filtered signal-the so-called “bouts”-to predict the chemical source distance, focusing on the optimization of the filter parameters and the noise threshold to make the predictions robust against changing wind conditions. This represents an advantage over previous bout-based models which require wind measurements-typically taken with expensive and bulky anemometers-to produce accurate predictions. The proposed methodology is demonstrated in a wind tunnel scenario where a MOX sensor is placed at various distances downwind of an emitting chemical source and the wind speed varies in the range 10-34 cm/s. The results demonstrate that models optimized with our methodology can provide accurate source distance predictions at different wind speeds.

Keywords: Gas detectors, Chemical sensors, Signal processing, Machine learning, Time series analysis

Herreros, Ivan, Miquel, Laia, Blithikioti, Chrysanthi, Nuño, Laura, Rubio Ballester, Belen, Grechuta, Klaudia, Gual, Antoni, Balcells-Oliveró, Mercè, Verschure, P., (2019). Motor adaptation impairment in chronic cannabis users assessed by a visuomotor rotation task Journal of Clinical Medicine 8, (7), 1049

Background—The cerebellum has been recently suggested as an important player in the addiction brain circuit. Cannabis is one of the most used drugs worldwide, and its long-term effects on the central nervous system are not fully understood. No valid clinical evaluations of cannabis impact on the brain are available today. The cerebellum is expected to be one of the brain structures that are highly affected by prolonged exposure to cannabis, due to its high density in endocannabinoid receptors. We aim to use a motor adaptation paradigm to indirectly assess cerebellar function in chronic cannabis users (CCUs). Methods—We used a visuomotor rotation (VMR) task that probes a putatively-cerebellar implicit motor adaptation process together with the learning and execution of an explicit aiming rule. We conducted a case-control study, recruiting 18 CCUs and 18 age-matched healthy controls. Our main measure was the angular aiming error. Results—Our results show that CCUs have impaired implicit motor adaptation, as they showed a smaller rate of adaptation compared with healthy controls (drift rate: 19.3 +/− 6.8° vs. 27.4 +/− 11.6°; t(26) = −2.1, p = 0.048, Cohen’s d = −0.8, 95% CI = (−1.7, −0.15)). Conclusions—We suggest that a visuomotor rotation task might be the first step towards developing a useful tool for the detection of alterations in implicit learning among cannabis users.

Keywords: Cerebellum, Cannabis, Implicit motor learning, Motor adaptation, Visuomotor rotation

Maier, Martina, Ballester, Belén Rubio, Verschure, P., (2019). Principles of neurorehabilitation after stroke based on motor learning and brain plasticity mechanisms Frontiers in Systems Neuroscience 13, 74

What are the principles underlying effective neurorehabilitation? The aim of neurorehabilitation is to exploit interventions based on human and animal studies about learning and adaptation, as well as to show that the activation of experience-dependent neuronal plasticity augments functional recovery after stroke. Instead of teaching compensatory strategies that do not reduce impairment but allow the patient to return home as soon as possible, functional recovery might be more sustainable as it ensures a long-term reduction in impairment and an improvement in quality of life. At the same time, neurorehabilitation permits the scientific community to collect valuable data, which allows inferring about the principles of brain organization. Hence neuroscience sheds light on the mechanisms of learning new functions or relearning lost ones. However, current rehabilitation methods lack the exact operationalization of evidence gained from skill learning literature, leading to an urgent need to bridge motor learning theory and present clinical work in order to identify a set of ingredients and practical applications that could guide future interventions. This work aims to unify the neuroscientific literature relevant to the recovery process and rehabilitation practice in order to provide a synthesis of the principles that constitute an effective neurorehabilitation approach. Previous attempts to achieve this goal either focused on a subset of principles or did not link clinical application to the principles of motor learning and recovery. We identified 15 principles of motor learning based on existing literature: massed practice, spaced practice, dosage, task-specific practice, goal-oriented practice, variable practice, increasing difficulty, multisensory stimulation, rhythmic cueing, explicit feedback/knowledge of results, implicit feedback/knowledge of performance, modulate effector selection, action observation/embodied practice, motor imagery, and social interaction. We comment on trials that successfully implemented these principles and report evidence from experiments with healthy individuals as well as clinical work.

Keywords: Neurorehabilitation, Motor learning, Plasticity, Stroke, Principles

Martinez-Hernandez, Uriel, Vouloutsi, Vasiliki, Mura, Anna, Mangan, Michael, Asada, Minoru, Prescott, T. J., Verschure, P., (2019). Biomimetic and Biohybrid Systems 8th International Conference, Living Machines 2019, Nara, Japan, July 9–12, 2019, Proceedings , Springer, Cham (Lausanne, Switzerland) 11556, 1-384

This book constitutes the proceedings of the 8th International Conference on Biomimetic and Biohybrid Systems, Living Machines 2019, held in Nara, Japan, in July 2019. The 26 full and 16 short papers presented in this volume were carefully reviewed and selected from 45 submissions. They deal with research on novel life-like technologies inspired by the scientific investigation of biological systems, biomimetics, and research that seeks to interface biological and artificial systems to create biohybrid systems.

Keywords: Artificial intelligence, Biomimetics, Computer architecture, Human robot interaction, Human-Computer Interaction (HCI), Humanoid robot, Image processing, Learning algorithms, Mobile robots, Multipurpose robots, Neural networks, Quadruped robots, Reinforcement learning, Robot learning, Robotics, Robots, Sensor, Sensors, Swarm robotics, User interfaces

Fonollosa, Jordi, Solórzano, Ana, Marco, Santiago, (2018). Chemical sensor systems and associated algorithms for fire detection: A review Sensors 18, (2), 553

Indoor fire detection using gas chemical sensing has been a subject of investigation since the early nineties. This approach leverages the fact that, for certain types of fire, chemical volatiles appear before smoke particles do. Hence, systems based on chemical sensing can provide faster fire alarm responses than conventional smoke-based fire detectors. Moreover, since it is known that most casualties in fires are produced from toxic emissions rather than actual burns, gas-based fire detection could provide an additional level of safety to building occupants. In this line, since the 2000s, electrochemical cells for carbon monoxide sensing have been incorporated into fire detectors. Even systems relying exclusively on gas sensors have been explored as fire detectors. However, gas sensors respond to a large variety of volatiles beyond combustion products. As a result, chemical-based fire detectors require multivariate data processing techniques to ensure high sensitivity to fires and false alarm immunity. In this paper, we the survey toxic emissions produced in fires and defined standards for fire detection systems. We also review the state of the art of chemical sensor systems for fire detection and the associated signal and data processing algorithms. We also examine the experimental protocols used for the validation of the different approaches, as the complexity of the test measurements also impacts on reported sensitivity and specificity measures. All in all, further research and extensive test under different fire and nuisance scenarios are still required before gas-based fire detectors penetrate largely into the market. Nevertheless, the use of dynamic features and multivariate models that exploit sensor correlations seems imperative

Keywords: Fire detection, Gas sensor, Pattern recognition, Sensor fusion, Machine learning, Toxicants, Carbon monoxide, Hydrogen cyanide, Standard test fires, Transducers, Smoke

Puigbò, J. Y., Arsiwalla, X. D., Verschure, P., (2018). Challenges of machine learning for living machines Biomimetic and Biohybrid Systems 7th International Conference, Living Machines 2018 (Lecture Notes in Computer Science) , Springer International Publishing (Paris, France) 10928, 382-386

Machine Learning algorithms (and in particular Reinforcement Learning (RL)) have proved very successful in recent years. These have managed to achieve super-human performance in many different tasks, from video-games to board-games and complex cognitive tasks such as path-planning or Theory of Mind (ToM) on artificial agents. Nonetheless, this super-human performance is also super-artificial. Despite some metrics are better than what a human can achieve (i.e. cumulative reward), in less common metrics (i.e. time to learning asymptote) the performance is significantly worse. Moreover, the means by which those are achieved fail to extend our understanding of the human or mammal brain. Moreover, most approaches used are based on black-box optimization, making any comparison beyond performance (e.g. at the architectural level) difficult. In this position paper, we review the origins of reinforcement learning and propose its extension with models of learning derived from fear and avoidance behaviors. We argue that avoidance-based mechanisms are required when training on embodied, situated systems to ensure fast and safe convergence and potentially overcome some of the current limitations of the RL paradigm.

Keywords: Avoidance, Neural networks, Reinforcement learning

Herreros, I., (2018). Learning and control Living machines: A handbook of research in biomimetics and biohybrid systems (ed. Prescott, T. J., Lepora, Nathan, Verschure, P.), Oxford Scholarship (Oxford, UK) , 239-255

This chapter discusses basic concepts from control theory and machine learning to facilitate a formal understanding of animal learning and motor control. It first distinguishes between feedback and feed-forward control strategies, and later introduces the classification of machine learning applications into supervised, unsupervised, and reinforcement learning problems. Next, it links these concepts with their counterparts in the domain of the psychology of animal learning, highlighting the analogies between supervised learning and classical conditioning, reinforcement learning and operant conditioning, and between unsupervised and perceptual learning. Additionally, it interprets innate and acquired actions from the standpoint of feedback vs anticipatory and adaptive control. Finally, it argues how this framework of translating knowledge between formal and biological disciplines can serve us to not only structure and advance our understanding of brain function but also enrich engineering solutions at the level of robot learning and control with insights coming from biology.

Keywords: Feedback control, Feed-forward control, Supervised learning, Unsupervised learning, Reinforcement, Learning, Classical conditioning, Operant conditioning, Reflex, Anticipatory reflex

Freire, I. T., Arsiwalla, X. D., Puigbò, J. Y., Verschure, P., (2018). Limits of multi-agent predictive models in the formation of social conventions Frontiers in Artificial Intelligence and Applications (ed. Falomir, Z., Gibert, K., Plaza, E.), IOS Press (Amsterdam, The Netherlands) Volume 308: Artificial Intelligence Research and Development, 297-301

A major challenge in cognitive science and AI is to understand how intelligent agents might be able to predict mental states of other agents during complex social interactions. What are the computational principles of such a Theory of Mind (ToM)? In previous work, we have investigated hypotheses of how the human brain might realize a ToM of other agents in a multi-agent social scenario. In particular, we have proposed control-based cognitive architectures to predict the model of other agents in a game-theoretic task (Battle of the Exes). Our multi-layer architecture implements top-down predictions from adaptive to reactive layers of control and bottom-up error feedback from reactive to adaptive layers. We tested cooperative and competitive strategies among different multi-agent models, demonstrating that while pure RL leads to reasonable efficiency and fairness in social interactions, there are other architectures that can perform better in specific circumstances. However, we found that even the best predictive models fall short of human data in terms of stability of social convention formation. In order to explain this gap between humans and predictive AI agents, in this work we propose introducing the notion of trust in the form of mutual agreements between agents that might enhance stability in the formation of conventions such as turn-taking.

Keywords: Cognitive Architectures, Game Theory, Multi-Agent Models, Reinforcement Learning, Theory of Mind

Vouloutsi, Vasiliki, Halloy, José, Mura, Anna, Mangan, Michael, Lepora, Nathan, Prescott, T. J., Verschure, P., (2018). Biomimetic and Biohybrid Systems 7th International Conference, Living Machines 2018, Paris, France, July 17–20, 2018, Proceedings , Springer International Publishing (Lausanne, Switzerland) 10928, 1-551

This book constitutes the proceedings of the 7th International Conference on Biomimetic and Biohybrid Systems, Living Machines 2018, held in Paris, France, in July 2018. The 40 full and 18 short papers presented in this volume were carefully reviewed and selected from 60 submissions. The theme of the conference targeted at the intersection of research on novel life-like technologies inspired by the scientific investigation of biological systems, biomimetics, and research that seeks to interface biological and artificial systems to create biohybrid systems.

Keywords: Artificial neural network, Bio-actuators, Bio-robotics, Biohybrid systems, Biomimetics, Bipedal robots, Earthoworm-like robots, Robotics, Decision-making, Tactile sensing, Soft robots, Locomotion, Insects, Sensors, Actuators, Robots, Artificial intelligence, Neural networks, Motion planning, Learning algorithms

Maffei, Giovanni, Herreros, Ivan, Sanchez-Fibla, Marti, Friston, Karl J., Verschure, Paul F. M. J., (2017). The perceptual shaping of anticipatory actions Proceedings of the Royal Society B , 284, (1869)

Humans display anticipatory motor responses to minimize the adverse effects of predictable perturbations. A widely accepted explanation for this behavior relies on the notion of an inverse model that, learning from motor errors, anticipates corrective responses. Here, we propose and validate the alternative hypothesis that anticipatory control can be realized through a cascade of purely sensory predictions that drive the motor system, reflecting the causal sequence of the perceptual events preceding the error. We compare both hypotheses in a simulated anticipatory postural adjustment task. We observe that adaptation in the sensory domain, but not in the motor one, supports the robust and generalizable anticipatory control characteristic of biological systems. Our proposal unites the neurobiology of the cerebellum with the theory of active inference and provides a concrete implementation of its core tenets with great relevance both to our understanding of biological control systems and, possibly, to their emulation in complex artefacts.

Keywords: Active inference, Cerebellum, Computational model, Motor control, Perceptual learning

Aviles, A. I., Alsaleh, S., Montseny, E., Sobrevilla, P., Casals, A., (2016). A Deep-Neuro-Fuzzy approach for estimating the interaction forces in Robotic surgery FUZZ-IEEE IEEE International Conference on Fuzzy Systems , IEEE (Vancouver, Canada ) , 1113-1119

Fuzzy theory was motivated by the need to create human-like solutions that allow representing vagueness and uncertainty that exist in the real-world. These capabilities have been recently further enhanced by deep learning since it allows converting complex relation between data into knowledge. In this paper, we present a novel Deep-Neuro-Fuzzy strategy for unsupervised estimation of the interaction forces in Robotic Assisted Minimally Invasive scenarios. In our approach, the capability of Neuro-Fuzzy systems for handling visual uncertainty, as well as the inherent imprecision of real physical problems, is reinforced by the advantages provided by Deep Learning methods. Experiments conducted in a realistic setting have demonstrated the superior performance of the proposed approach over existing alternatives. More precisely, our method increased the accuracy of the force estimation and compared favorably to existing state of the art approaches, offering a percentage of improvement that ranges from about 35% to 85%.

Keywords: Estimation, Force, Machine learning, Robots, Three-dimensional displays, Uncertainty, Visualization

Marbán, Arturo, Casals, Alicia, Fernández, Josep, Amat, Josep, (2014). Haptic feedback in surgical robotics: Still a challenge Advances in Intelligent Systems and Computing ROBOT2013: First Iberian Robotics Conference (ed. Armada, Manuel A., Sanfeliu, Alberto, Ferre, Manuel), Springer International Publishing 252, 245-253

Endowing current surgical robotic systems with haptic feedback to perform minimally invasive surgery (MIS), such as laparoscopy, is still a challenge. Haptic is a feature lost in surgical teleoperated systems limiting surgeons capabilities and ability. The availability of haptics would provide important advantages to the surgeon: Improved tissue manipulation, reducing the breaking of sutures and increase the feeling of telepresence, among others. To design and develop a haptic system, the measurement of forces can be implemented based on two approaches: Direct and indirect force sensing. MIS performed with surgical robots, imposes many technical constraints to measure forces, such as: Miniaturization, need of sterilization or materials compatibility, making it necessary to rely on indirect force sensing. Based on mathematical models of the components involved in an intervention and indirect force sensing techniques, a global perspective on how to address the problem of measurement of tool-tissue interaction forces is presented.

Keywords: Surgical robotics, Haptic feedback, Indirect force sensing, Machine learning, Data fusion, Mathematical models

Antelis, J.M., Montesano, L., Giralt, X., Casals, A., Minguez, J., (2012). Detection of movements with attention or distraction to the motor task during robot-assisted passive movements of the upper limb Engineering in Medicine and Biology Society (EMBC) 34th Annual International Conference of the IEEE , IEEE (San Diego, USA) , 6410-6413

Robot-assisted rehabilitation therapies usually focus on physical aspects rather than on cognitive factors. However, cognitive aspects such as attention, motivation, and engagement play a critical role in motor learning and thus influence the long-term success of rehabilitation programs. This paper studies motor-related EEG activity during the execution of robot-assisted passive movements of the upper limb, while participants either: i) focused attention exclusively on the task; or ii) simultaneously performed another task. Six healthy subjects participated in the study and results showed lower desynchronization during passive movements with another task simultaneously being carried out (compared to passive movements with exclusive attention on the task). In addition, it was proved the feasibility to distinguish between the two conditions.

Keywords: Electrodes, Electroencephalography, Induction motors, Medical treatment, Robot sensing systems, Time frequency analysis, Biomechanics, Cognition, Electroencephalography, Medical robotics, Medical signal detection, Medical signal processing, Patient rehabilitation, Attention, Cognitive aspects, Desynchronization, Engagement, Motivation, Motor learning, Motor task, Motor-related EEG activity, Physical aspects, Robot-assisted passive movement detection, Robot-assisted rehabilitation therapies, Upper limb