Publications

There is a growing interest in developing natural hydrogel-based scaffolds to culture cells in a three-dimensional (3D) millieu that better mimics the in vivo cells' microenvironment. A promising approach is to use hydrogels from animal tissues, such as decellularized extracellular matrices; however, they usually exhibit suboptimal mechanical properties compared to native tissue and their composition with hundreds of different protein complicates to elucidate which stimulus triggers cell's responses. As simpler scaffolds, type I collagen hydrogels are used to study cell behavior in mechanobiology even though they are also softer than native tissues. In this work, type I collagen is mixed with bacterial nanocellulose fibers (BCf) to develop reinforced scaffolds with mechanical properties suitable for 3D cell culture. BCf were produced from blended pellicles biosynthesized from Komagataeibacter xylinus. Then, BCf were mixed with concentrated collagen from rat-tail tendons to form composite hydrogels. Confocal laser scanning microscopy and scanning electron microscopy images confirmed the homogeneous macro- and microdistribution of both natural polymers. Porosity analysis confirmed that BCf do not disrupt the scaffold structure. Tensile strength and rheology measurements demonstrated the reinforcement action of BCf (43% increased stiffness) compared to the collagen hydrogel while maintaining the same viscoelastic response. Additionally, this reinforcement of collagen hydrogels with BCf offers the possibility to mix cells before gelation and then proceed to the culture of the 3D cell scaffolds. We obtained scaffolds with human bone marrow-derived mesenchymal stromal cells or human fibroblasts within the composite hydrogels, allowing a homogeneous 3D viable culture for at least 7 days. A smaller surface shrinkage in the reinforced hydrogels compared to type I collagen hydrogels confirmed the strengthening of the composite hydrogels. These collagen hydrogels reinforced with BCf might emerge as a promising platform for 3D in vitro organ modeling, tissue-engineering applications, and suitable to conduct fundamental mechanobiology studies.

JTD Keywords: 3d cell culture, bacterial cellulose, collagen, composite hydrogels, 3d cell culture, Bacterial cellulose, Cellulose/collagen composite, Collagen, Composite hydrogels, Contraction, Cross-linking, Cytocompatibility, Fibroblasts, Matrix, Mechanical-properties, Reinforcement, Stiffness, Tissue engineering

Isotactic polypropylene (i-PP) nonabsorbable surgical meshes are modified by incorporating a conducting polymer (CP) layer to detect the adhesion and growth of bacteria by sensing the oxidation of nicotinamide adenine dinucleotide (NADH), a metabolite produced by the respiration reactions of such microorganisms, to NAD+. A three-step process is used for such incorporation: (1) treat pristine meshes with low-pressure O2 plasma; (2) functionalize the surface with CP nanoparticles; and (3) coat with a homogeneous layer of electropolymerized CP using the nanoparticles introduced in (2) as polymerization nuclei. The modified meshes are stable and easy to handle and also show good electrochemical response. The detection by cyclic voltammetry of NADH within the interval of concentrations reported for bacterial cultures is demonstrated for the two modified meshes. Furthermore, Staphylococcus aureus and both biofilm-positive (B+) and biofilm-negative (B-) Escherichia coli cultures are used to prove real-time monitoring of NADH coming from aerobic respiration reactions. The proposed strategy, which offers a simple and innovative process for incorporating a sensor for the electrochemical detection of bacteria metabolism to currently existing surgical meshes, holds considerable promise for the future development of a new generation of smart biomedical devices to fight against post-operative bacterial infections.

JTD Keywords: adhesion, bacteria metabolism, behavior, biocompatibility, conducting polymer, electrochemical sensor, hernia repair, in-vivo, liquid, nadh detection, plasma treatment, prevention, reinforcement, sensor, smart meshes, Bacteria metabolism, Polypropylene mesh, Smart meshes

The sample-inefficiency problem in Artificial Intelligence refers to the inability of current Deep Reinforcement Learning models to optimize action policies within a small number of episodes. Recent studies have tried to overcome this limitation by adding memory systems and architectural biases to improve learning speed, such as in Episodic Reinforcement Learning. However, despite achieving incremental improvements, their performance is still not comparable to how humans learn behavioral policies. In this paper, we capitalize on the design principles of the Distributed Adaptive Control (DAC) theory of mind and brain to build a novel cognitive architecture (DAC-ML) that, by incorporating a hippocampus-inspired sequential memory system, can rapidly converge to effective action policies that maximize reward acquisition in a challenging foraging task.

JTD Keywords: Cognitive architecture, Distributed adaptive control, Reinforcement learning, Sample-inefficiency problem, Sequence learning

This book constitutes the proceedings of the 8th International Conference on Biomimetic and Biohybrid Systems, Living Machines 2019, held in Nara, Japan, in July 2019. The 26 full and 16 short papers presented in this volume were carefully reviewed and selected from 45 submissions. They deal with research on novel life-like technologies inspired by the scientific investigation of biological systems, biomimetics, and research that seeks to interface biological and artificial systems to create biohybrid systems.

JTD Keywords: Artificial intelligence, Biomimetics, Computer architecture, Human robot interaction, Human-Computer Interaction (HCI), Humanoid robot, Image processing, Learning algorithms, Mobile robots, Multipurpose robots, Neural networks, Quadruped robots, Reinforcement learning, Robot learning, Robotics, Robots, Sensor, Sensors, Swarm robotics, User interfaces

Machine Learning algorithms (and in particular Reinforcement Learning (RL)) have proved very successful in recent years. These have managed to achieve super-human performance in many different tasks, from video-games to board-games and complex cognitive tasks such as path-planning or Theory of Mind (ToM) on artificial agents. Nonetheless, this super-human performance is also super-artificial. Despite some metrics are better than what a human can achieve (i.e. cumulative reward), in less common metrics (i.e. time to learning asymptote) the performance is significantly worse. Moreover, the means by which those are achieved fail to extend our understanding of the human or mammal brain. Moreover, most approaches used are based on black-box optimization, making any comparison beyond performance (e.g. at the architectural level) difficult. In this position paper, we review the origins of reinforcement learning and propose its extension with models of learning derived from fear and avoidance behaviors. We argue that avoidance-based mechanisms are required when training on embodied, situated systems to ensure fast and safe convergence and potentially overcome some of the current limitations of the RL paradigm.

JTD Keywords: Avoidance, Neural networks, Reinforcement learning

This chapter discusses basic concepts from control theory and machine learning to facilitate a formal understanding of animal learning and motor control. It first distinguishes between feedback and feed-forward control strategies, and later introduces the classification of machine learning applications into supervised, unsupervised, and reinforcement learning problems. Next, it links these concepts with their counterparts in the domain of the psychology of animal learning, highlighting the analogies between supervised learning and classical conditioning, reinforcement learning and operant conditioning, and between unsupervised and perceptual learning. Additionally, it interprets innate and acquired actions from the standpoint of feedback vs anticipatory and adaptive control. Finally, it argues how this framework of translating knowledge between formal and biological disciplines can serve us to not only structure and advance our understanding of brain function but also enrich engineering solutions at the level of robot learning and control with insights coming from biology.

JTD Keywords: Feedback control, Feed-forward control, Supervised learning, Unsupervised learning, Reinforcement, Learning, Classical conditioning, Operant conditioning, Reflex, Anticipatory reflex

A major challenge in cognitive science and AI is to understand how intelligent agents might be able to predict mental states of other agents during complex social interactions. What are the computational principles of such a Theory of Mind (ToM)? In previous work, we have investigated hypotheses of how the human brain might realize a ToM of other agents in a multi-agent social scenario. In particular, we have proposed control-based cognitive architectures to predict the model of other agents in a game-theoretic task (Battle of the Exes). Our multi-layer architecture implements top-down predictions from adaptive to reactive layers of control and bottom-up error feedback from reactive to adaptive layers. We tested cooperative and competitive strategies among different multi-agent models, demonstrating that while pure RL leads to reasonable efficiency and fairness in social interactions, there are other architectures that can perform better in specific circumstances. However, we found that even the best predictive models fall short of human data in terms of stability of social convention formation. In order to explain this gap between humans and predictive AI agents, in this work we propose introducing the notion of trust in the form of mutual agreements between agents that might enhance stability in the formation of conventions such as turn-taking.

JTD Keywords: Cognitive Architectures, Game Theory, Multi-Agent Models, Reinforcement Learning, Theory of Mind