by Keyword: validation

Oliveira LFD, Mallafré-Muro C, Giner J, Perea L, Sibila O, Pardo A, Marco S, (2022). Breath analysis using electronic nose and gas chromatography-mass spectrometry: A pilot study on bronchial infections in bronchiectasis Clinica Chimica Acta 526, 6-13

Background and aims: In this work, breath samples from clinically stable bronchiectasis patients with and without bronchial infections by Pseudomonas Aeruginosa- PA) were collected and chemically analysed to determine if they have clinical value in the monitoring of these patients. Materials and methods: A cohort was recruited inviting bronchiectasis patients (25) and controls (9). Among the former group, 12 members were suffering PA infection. Breath samples were collected in Tedlar bags and analyzed by e-nose and Gas Chromatography-Mass Spectrometry (GC-MS). The obtained data were analyzed by chemometric methods to determine their discriminant power in regards to their health condition. Results were evaluated with blind samples. Results: Breath analysis by electronic nose successfully separated the three groups with an overall classification rate of 84% for the three-class classification problem. The best discrimination was obtained between control and bronchiectasis with PA infection samples 100% (CI95%: 84–100%) on external validation and the results were confirmed by permutation tests. The discrimination analysis by GC-MS provided good results but did not reach proper statistical significance after a permutation test. Conclusions: Breath sample analysis by electronic nose followed by proper predictive models successfully differentiated between control, Bronchiectasis and Bronchiectasis PA samples. © 2021 The Author(s)

JTD Keywords: biomarkers, breath analysis, bronchiectasis, diagnosis, e-nose, fingerprints, gc-ms, identification, lung-cancer, partial least-squares, pseudomonas-aeruginosa, signal processing, validation, volatile organic-compounds, Airway bacterial-colonization, Breath analysis, Bronchiectasis, E-nose, Gc-ms, Signal processing

Dulay, S, Rivas, L, Pla, L, Berdun, S, Eixarch, E, Gratacos, E, Illa, M, Mir, M, Samitier, J, (2021). Fetal ischemia monitoring with in vivo implanted electrochemical multiparametric microsensors Journal Of Biological Engineering 15,

Under intrauterine growth restriction (IUGR), abnormal attainment of the nutrients and oxygen by the fetus restricts the normal evolution of the prenatal causing in many cases high morbidity being one of the top-ten causes of neonatal death. The current gold standards in hospitals to detect this relevant problem is the clinical observation by echography, cardiotocography and Doppler. These qualitative techniques are not conclusive and requires risky invasive fetal scalp blood testing and/or amniocentesis. We developed micro-implantable multiparametric electrochemical sensors for measuring ischemia in real time in fetal tissue and vascular. This implantable technology is designed to continuous monitoring for an early detection of ischemia to avoid potential fetal injury. Two miniaturized electrochemical sensors were developed based on oxygen and pH detection. The sensors were optimized in vitro under controlled concentration, to assess the selectivity and sensitivity required. The sensors were then validated in vivo in the ewe fetus model, by means of their insertion in the muscle leg and inside the iliac artery of the fetus. Ischemia was achieved by gradually obstructing the umbilical cord to regulate the amount of blood reaching the fetus. An important challenge in fetal monitoring is the detection of low levels of oxygen and pH changes under ischemic conditions, requiring high sensitivity sensors. Significant differences were observed in both; pH and pO(2) sensors under changes from normoxia to hypoxia states in the fetus tissue and vascular with both sensors. Herein, we demonstrate the feasibility of the developed sensors for future fetal monitoring in medical applications.

JTD Keywords: electrochemical biosensor, implantable sensor, in vivo validation, ischemia detection, tissue and vascular monitoring, Animal experiment, Animal model, Animal tissue, Article, Blood-gases, Brain, Classification, Controlled study, Diagnosis, Doppler, Early diagnosis, Electrochemical analysis, Electrochemical biosensor, Ewe, Feasibility study, Female, Fetus, Fetus disease, Fetus monitoring, Gestational age, Hypoxemia, Iliac artery, Implantable sensor, In vivo validation, Intrauterine growth restriction, Intrauterine growth retardation, Ischemia detection, Leg muscle, Management, Nonhuman, Oxygen consumption, Ph, Ph and oxygen detection, Ph measurement, Process optimization, Sheep, Tissue and vascular monitoring, Umbilical-cord occlusion

Mallafré‐muro C, Llambrich M, Cumeras R, Pardo A, Brezmes J, Marco S, Gumà J, (2021). Comprehensive volatilome and metabolome signatures of colorectal cancer in urine: A systematic review and meta‐analysis Cancers 13,

To increase compliance with colorectal cancer screening programs and to reduce the recommended screening age, cheaper and easy non‐invasiveness alternatives to the fecal immunochemical test should be provided. Following the PRISMA procedure of studies that evaluated the metabolome and volatilome signatures of colorectal cancer in human urine samples, an exhaustive search in PubMed, Web of Science, and Scopus found 28 studies that met the required criteria. There were no restrictions on the query for the type of study, leading to not only colorectal cancer samples versus control comparison but also polyps versus control and prospective studies of surgical effects, CRC staging and comparisons of CRC with other cancers. With this systematic review, we identified up to 244 compounds in urine samples (3 shared compounds between the volatilome and metabolome), and 10 of them were relevant in more than three articles. In the meta-analysis, nine studies met the criteria for inclusion, and the results combining the case‐control and the pre‐/post‐surgery groups, eleven compounds were found to be relevant. Four upregulated metabolites were identified, 3‐hydroxybutyric acid, L‐dopa, L‐histidinol, and N1, N12‐ diacetylspermine and seven downregulated compounds were identified, pyruvic acid, hydroquinone, tartaric acid, and hippuric acid as metabolites and butyraldehyde, ether, and 1,1,6‐ trimethyl‐1,2‐dihydronaphthalene as volatiles.

JTD Keywords: biomarkers, breast, chromatography, colorectal cancer, diagnosis, markers, meta-analysis, metabolomics, metabonomics, n-1,n-12-diacetylspermine, nucleosides, systematic review, urine, validation, volatilomics, Colorectal cancer, Early-stage, Metabolomics, Meta‐analysis, Systematic review, Urine, Volatilomics

Rodríguez-Pérez, R., Fernández, L., Marco, S., (2018). Overoptimism in cross-validation when using partial least squares-discriminant analysis for omics data: a systematic study Analytical and Bioanalytical Chemistry 410, (23), 5981-5992

Advances in analytical instrumentation have provided the possibility of examining thousands of genes, peptides, or metabolites in parallel. However, the cost and time-consuming data acquisition process causes a generalized lack of samples. From a data analysis perspective, omics data are characterized by high dimensionality and small sample counts. In many scenarios, the analytical aim is to differentiate between two different conditions or classes combining an analytical method plus a tailored qualitative predictive model using available examples collected in a dataset. For this purpose, partial least squares-discriminant analysis (PLS-DA) is frequently employed in omics research. Recently, there has been growing concern about the uncritical use of this method, since it is prone to overfitting and may aggravate problems of false discoveries. In many applications involving a small number of subjects or samples, predictive model performance estimation is only based on cross-validation (CV) results with a strong preference for reporting results using leave one out (LOO). The combination of PLS-DA for high dimensionality data and small sample conditions, together with a weak validation methodology is a recipe for unreliable estimations of model performance. In this work, we present a systematic study about the impact of the dataset size, the dimensionality, and the CV technique used on PLS-DA overoptimism when performance estimation is done in cross-validation. Firstly, by using synthetic data generated from a same probability distribution and with assigned random binary labels, we have obtained a dataset where the true classification rate (CR) is 50%. As expected, our results confirm that internal validation provides overoptimistic estimations of the classification accuracy (i.e., overfitting). We have characterized the CR estimator in terms of bias and variance depending on the internal CV technique used and sample to dimensionality ratio. In small sample conditions, due to the large bias and variance of the estimator, the occurrence of extremely good CRs is common. We have found that overfitting peaks when the sample size in the training subset approaches the feature vector dimensionality minus one. In these conditions, the models are neither under- or overdetermined with a unique solution. This effect is particularly intense for LOO and peaks higher in small sample conditions. Overoptimism is decreased beyond this point where the abundance of noisy produces a regularization effect leading to less complex models. In terms of overfitting, our study ranks CV methods as follows: Bootstrap produces the most accurate estimator of the CR, followed by bootstrapped Latin partitions, random subsampling, K-Fold, and finally, the very popular LOO provides the worst results. Simulation results are further confirmed in real datasets from mass spectrometry and microarrays.

JTD Keywords: Metabolomics, Mass spectrometry, Microarrays, Chemometrics, Data analysis, Classification, Method validation

Rodríguez, R., Cortés, R., Verónica Guamán, A., Pardo, A., Torralba, Y., Gómez, F., Roca, J., Barberà, J.A., Cascante, M., Marco, S., (2018). Instrumental drift removal in GC-MS data for breath analysis: the short-term and long-term temporal validation of putative biomarkers for COPD Journal of Breath Research 12, (3), 036007

Abstract Breath analysis holds the promise of a non-invasive technique for the diagnosis of diverse respiratory conditions including COPD and lung cancer. Breath contains small metabolites that may be putative biomarkers of these conditions. However, the discovery of reliable biomarkers is a considerable challenge in the presence of both clinical and instrumental confounding factors. Among the latter, instrumental time drifts are highly relevant, as since question the short and long-term validity of predictive models. In this work we present a methodology to counter instrumental drifts using information from interleaved blanks for a case study of GC-MS data from breath samples. The proposed method includes feature filtering, and additive, multiplicative and multivariate drift corrections, the latter being based on Component Correction. Biomarker discovery was based on Genetic Algorithms in a filter configuration using Fisher´s ratio computed in the Partial Least Squares – Discriminant Analysis subspace as a figure of merit. Using our protocol, we have been able to find nine peaks that provide a statistically significant Area under the ROC Curve (AUC) of 0.75 for COPD discrimination. The method developed has been successfully validated using blind samples in short-term temporal validation. However, in the attempt to use this model for patient screening six months later was not successful. This negative result highlights the importance of increasing validation rigour when reporting biomarker discovery results.

JTD Keywords: Instrumental shifts, Chemometrics, Biomarker validation

Verschure, P., (2018). A chronology of Distributed Adaptive Control Living machines: A handbook of research in biomimetics and biohybrid systems (ed. Prescott, T. J., Lepora, Nathan, Verschure, P.), Oxford Scholarship (Oxford, UK) , 346-360

This chapter presents the Distributed Adaptive Control (DAC) theory of the mind and brain of living machines. DAC provides an explanatory framework for biological brains and an integration framework for synthetic ones. DAC builds on several themes presented in the handbook: it integrates different perspectives on mind and brain, exemplifies the synthetic method in understanding living machines, answers well-defined constraints faced by living machines, and provides a route for the convergent validation of anatomy, physiology, and behavior in our explanation of biological living machines. DAC addresses the fundamental question of how a living machine can obtain, retain, and express valid knowledge of its world. We look at the core components of DAC, specific benchmarks derived from the engagement with the physical and the social world (the H4W and the H5W problems) in foraging and human–robot interaction tasks. Lastly we address how DAC targets the UTEM benchmark and the relation with contemporary developments in AI.

JTD Keywords: Distributed Adaptive Control, Problem of priors, Symbol grounding problem, Convergent validation, Foraging, brain, Architecture, system

Verschure, P., Prescott, T. J., (2018). A living machines approach to the sciences of mind and brain Living Machines: A Handbook of Research in Biomimetic and Biohybrid Systems (ed. Prescott, T. J., Lepora, Nathan, Verschure, P.), Oxford Scholarship (Oxford, UK) , 15-25

How do the sciences of mind and brain—neuroscience, psychology, cognitive science, and artificial intelligence (AI)—stand in relation to each other in the 21st century? This chapter proposes that despite our knowledge expanding at ever-accelerating rates, our understanding of the relationship between mind and brain is, in some important sense, becoming less and less. An increasing explanatory gap can only be bridged by a multi-tiered and integrated theoretical framework that recognizes the value of developing explanations at different levels, combining these into cross-level integrated theories, and directly contributing to new technologies that improve the human condition. Development of technologies that instantiate principles gleaned from the study of the mind and brain, or biomimetic technologies, is a key part of the validation process for scientific theories of mind and brain. We call this strategy for the integration of science and engineering a Living Machines approach. Following this path can lead not only to better science, and useful engineering, but also a richer view of human experience and of relationships between science, engineering, and art.

JTD Keywords: Convergent validation, Multi-tiered theories, Paradigms in cognitive science, Philosophy of science, Physical models, Reductionism

Padilla, M., Perera, A., Montoliu, I., Chaudry, A., Persaud, K., Marco, S., (2010). Drift compensation of gas sensor array data by orthogonal signal correction Chemometrics and Intelligent Laboratory Systems , 100, (1), 28-35

Drift is an important issue that impairs the reliability of gas sensing systems. Sensor aging, memory effects and environmental disturbances produce shifts in sensor responses that make initial statistical models for gas or odor recognition useless after a relatively short period (typically few weeks). Frequent recalibrations are needed to preserve system accuracy. However, when recalibrations involve numerous samples they become expensive and laborious. An interesting and lower cost alternative is drift counteraction by signal processing techniques. Orthogonal Signal Correction (OSC) is proposed for drift compensation in chemical sensor arrays. The performance of OSC is also compared with Component Correction (CC). A simple classification algorithm has been employed for assessing the performance of the algorithms on a dataset composed by measurements of three analytes using an array of seventeen conductive polymer gas sensors over a ten month period.

JTD Keywords: Gas sensor array, Drift, Orthogonal signal correction, Component correction, Cross-validation, Electronic nose, Data shift