by Keyword: Force estimation
Marban, A., Srinivasan, V., Samek, W., Fernández, J., Casals, A., (2019). A recurrent convolutional neural network approach for sensorless force estimation in robotic surgery Biomedical Signal Processing and Control 50, 134-150
Providing force feedback as relevant information in current Robot-Assisted Minimally Invasive Surgery systems constitutes a technological challenge due to the constraints imposed by the surgical environment. In this context, force estimation techniques represent a potential solution, enabling to sense the interaction forces between the surgical instruments and soft-tissues. Specifically, if visual feedback is available for observing soft-tissues’ deformation, this feedback can be used to estimate the forces applied to these tissues. To this end, a force estimation model, based on Convolutional Neural Networks and Long-Short Term Memory networks, is proposed in this work. This model is designed to process both, the spatiotemporal information present in video sequences and the temporal structure of tool data (the surgical tool-tip trajectory and its grasping status). A series of analyses are carried out to reveal the advantages of the proposal and the challenges that remain for real applications. This research work focuses on two surgical task scenarios, referred to as pushing and pulling tissue. For these two scenarios, different input data modalities and their effect on the force estimation quality are investigated. These input data modalities are tool data, video sequences and a combination of both. The results suggest that the force estimation quality is better when both, the tool data and video sequences, are processed by the neural network model. Moreover, this study reveals the need for a loss function, designed to promote the modeling of smooth and sharp details found in force signals. Finally, the results show that the modeling of forces due to pulling tasks is more challenging than for the simplest pushing actions.
JTD Keywords: Convolutional neural networks, Force estimation, LSTM networks, Robotic surgery
Aviles, A. I., Alsaleh, S. M., Hahn, J. K., Casals, A., (2017). Towards retrieving force feedback in robotic-assisted surgery: A supervised neuro-recurrent-vision approach 
IEEE Transactions on Haptics , 10, (3), 431-443
Robotic-assisted minimally invasive surgeries have gained a lot of popularity over conventional procedures as they offer many benefits to both surgeons and patients. Nonetheless, they still suffer from some limitations that affect their outcome. One of them is the lack of force feedback which restricts the surgeon's sense of touch and might reduce precision during a procedure. To overcome this limitation, we propose a novel force estimation approach that combines a vision based solution with supervised learning to estimate the applied force and provide the surgeon with a suitable representation of it. The proposed solution starts with extracting the geometry of motion of the heart's surface by minimizing an energy functional to recover its 3D deformable structure. A deep network, based on a LSTM-RNN architecture, is then used to learn the relationship between the extracted visual-geometric information and the applied force, and to find accurate mapping between the two. Our proposed force estimation solution avoids the drawbacks usually associated with force sensing devices, such as biocompatibility and integration issues. We evaluate our approach on phantom and realistic tissues in which we report an average root-mean square error of 0.02 N.
JTD Keywords: Computer-assisted surgery, Deep networks, Force estimation, Visual deformation
Aviles, A. I., Marban, A., Sobrevilla, P., Fernandez, Josep, Casals, A., (2014). A recurrent neural network approach for 3D vision-based force estimation IPTA 2014 4th International Conference on Image Processing Theory, Tools and Applications (IPTA) , IEEE (Paris, France) , 1-6
Robotic-assisted minimally invasive surgery has demonstrated its benefits in comparison with traditional procedures. However, one of the major drawbacks of current robotic system approaches is the lack of force feedback. Apart from space restrictions, the main problems of using force sensors are their high cost and the biocompatibility. In this work a proposal based on Vision Based Force Measurement is presented, in which the deformation mapping of the tissue is obtained using the `2−Regularized Optimization class, and the force is estimated via a recurrent neural network that has as inputs the kinematic variables and the deformation mapping. Moreover, the capability of RNN for predicting time series is used in order to deal with tool occlusions. The highlights of this proposal, according to the results, are: knowledge of material properties are not necessary, there is no need of adding extra sensors and a good trade-off between accuracy and efficiency has been achieved.
JTD Keywords: Force estimation, Regularized optimization, Deformable tracking, Recurrent neural network
