Sage Journals: Discover world-class research

Abstract

Various living creatures exhibit embodiment intelligence, which is reflected by a collaborative interaction of the brain, body, and environment. The actual behavior of embodiment intelligence is generated by a continuous and dynamic interaction between a subject and the environment through information perception and physical manipulation. The physical interaction between a robot and the environment is the basis for realizing embodied perception and learning. Tactile information plays a critical role in this physical interaction process. It can be used to ensure safety, stability, and compliance, and can provide unique information that is difficult to capture using other perception modalities. However, due to the limitations of existing sensors and perception and learning methods, the development of robotic tactile research lags significantly behind other sensing modalities, such as vision and hearing, thereby seriously restricting the development of robotic embodiment intelligence. This paper presents the current challenges related to robotic tactile embodiment intelligence and reviews the theory and methods of robotic embodied tactile intelligence. Tactile perception and learning methods for embodiment intelligence can be designed based on the development of new large‐scale tactile array sensing devices, with the aim to make breakthroughs in the neuromorphic computing technology of tactile intelligence.

Keywords

tactile sensing tactile perception tactile learning neuro-morphological computing robotics

1 Introduction

In recent years, many countries have launched brain‐related research projects, aiming at exploring the mechanism of the brain, modeling the structure of the brain and simulating functions of the brain. For these projects, brain‐like artificial intelligence technologies and corresponding devices must be urgently developed, and research on embodiment intelligence must be conducted.

Various living creatures have embodiment intelligence reflected by a collaborative interaction of the brain, body, and environment. The actual behavior of embodiment intelligence is generated by a continuous and dynamic interaction between a subject and the environment through information perception and physical manipulation. Embodiment intelligence can be jointly developed with disembodied intelligence, which emphasizes logic, reasoning, and problem‐solving. Embodiment intelligence and disembodied intelligence complement each other, with each providing an alternative for intelligence breakthrough. Research on embodiment intelligence originated from psychology, which has recently received attention in many fields, such as cognitive science, artificial intelligence, and robotics. Relevant research results have been distributed in the areas of biological development and evolutionary robotics, ubiquitous computing and interface technology, multi‐agent, and artificial life. Rolf Pfeifer, a well‐known researcher from the University of Zurich, Switzerland, clearly described embodiment intelligence by analyzing how a body affects intelligence. He clarified that a body has a profound impact on understanding of the essence of intelligence and studying artificial intelligence systems.

People have always dreamed for robots to have human‐like brain–body collaborative embodiment intelligence, so that they can free humans from trivial work as well as harsh or dangerous environment. The ability of humans to accomplish various highly complex tasks in a dynamic and uncertain environment depends on the interaction between their brain–body collaborative system and the environment and their lifelong learning ability. The use of the brain–body collaborative cognitive mechanism to realize embodied perception and learning has become an important research topic for intelligent robots and an inevitable trend for developing a new generation of intelligent robots.

The physical interaction between a robot and the environment is the basis for realizing embodied perception and learning. In a physical interaction process, tactile information plays a critical role and can be used to ensure safety, stability, and compliance because it can obtain unique information that is difficult to capture using other perception modalities. However, the development of robotic tactile research lags significantly behind other modalities, such as vision and hearing, seriously restricting the development of robotic embodiment intelligence.

This paper presents the current challenges faced in robotic tactile embodiment intelligence and reviews the theory and methods of robotic embodied tactile intelligence. Tactile perception and learning methods for embodiment intelligence can be designed based on the development of new large‐scale tactile array sensing devices, aiming to make breakthroughs in neuromorphic computing technology and tactile intelligence.

The prerequisite of realizing robotic embodied tactile intelligence is obtaining tactile signals during a physical interaction. Tactile perception and learning based on this form the core of embodied tactile intelligence. The computing technology is the foundation of realizing embodied intelligence. Therefore, robotic embodied tactile intelligence involves several important issues, such as sensing, perception, learning, and computing. In recent years, many researchers in several countries have researched this topic from different perspectives. A detailed analysis of the related research, progress, and trends of the abovementioned aspects is presented in the following sections.

2 Embodied tactile sensing

Tactile sensing is the prerequisite of physical interaction and embodied learning among a robot, the environment, and a human. It plays a significant role in many tasks, such as environment detection, dexterous manipulation, and human–robot interaction, as well as in fields such as electronic engineering, instrument measurement, and materials. Tactile sensors are categorized into various types according to their physical principles (Table 1).

Table 1.

Different tactile sensors.

Type of sensor	Reference
Piezoresistive	[1, 2, 3]
Capacitive	[4, 5]
Piezoelectric	[6, 7, 8]
Quantum tunnel effect	[9, 10]
Optical	[11, 12], 13]
Pressure	[11, 12]
Acoustic	[14, 15]
Hybrid	[16, 17]

Robot tactile sensing principles and tactile object recognition have been investigated previously [18, 19]; however, the tactile sensor application has not been discussed for robotic embodied learning tasks. Tactile sensors have been widely deployed in different robot parts (e.g., fingertips, fingers, palms, arms, torsos, legs, and feet; Fig. 1) to study the perception, cognition, learning, and interaction capabilities of robots. The abovementioned research has laid a good foundation for studying robotic embodied tactile intelligence. Robotic tactile sensing in different body parts is described in the subsequent sections.

Fig. 1.

Representative parts for embodied tactile applications.

2.1 Finger tactile sensing

Tactile sensors are widely used in robotic fingers, which can be divided into fingertips and other parts of the finger. A tactile sensor can be integrated at the tip of the robot's finger (i.e., end of the finger) and used to recognize an object’s attributes (e.g., texture, hardness, and shape) by performing different exploratory operations, including sliding, squeezing, pushing, and tapping. Fig. 2 shows some representative work. Winstone et al. [20] integrated a TACTIP tactile sensor that mimics the structure of a human finger on the ELU‐2 Elumation robotic hand, to analyze the texture characteristics of objects. SynTouch Inc. developed a multi‐modal fingertip tactile sensor, called BioTac, which has been integrated into robots (e.g., Shadow, Barret, and Allegro) for tasks such as material classification, object recognition, and slip detection [21]. Schmitz et al. [22] designed a flexible printed circuit board integrated with a capacitive tactile sensing array and developed a fingertip tactile sensor for iCub robots. Koiva et al. [23] designed a rigid three‐dimensional (3D) resistive tactile sensor array using the laser technology, and mounted it on the fingertips of the Shadow robot. Meanwhile, Jara et al. [24] described PPS RoboTouch (Pressure Profile Systems, Inc.) capacitive tactile array installed on four fingertips of the Allegro robotic hand to collect the grasping force for a dexterous manipulation. A fingertip optical tactile sensor, called GelSight, can measure pressure, torque, shear force, and slipping, and is used to estimate object hardness and recognize the object material [25]. The leading author’s group recently installed optical tactile sensors on the fingertips to realize object material recognition and slip estimation [13].

Fig. 2.

Representative work on fingertip tactile sensing. (A) Reproduced with permission from Ref. [20], ©Springer Nature, 2013. (B) Reproduced with permission from Ref. [21], ©IEEE, 2013. (C) Reproduced with permission from Ref., ©IEEE, 2010. (D) Reproduced with permission from Ref. [25], ©IEEE, 2019.

Besides fingertips, tactile sensors are also widely used in other parts of the finger. Fig. 3 shows some representative work. Schmidt et al. [26] proposed the integration of a PPS capacitive tactile array into the gripper of the PR2 robot to explore the characteristics of the object surface to achieve human–robot interaction. To mimic the function of human fingers, Jamali and Sammut [27] used randomly distributed strain gauges and polyvinylidene fluoride (PVDF) films to develop mechanical fingers for identifying different surface textures. Yousef et al. [28] integrated an optical tactile sensor array with 41 contact points into a two‐finger robotic hand, to collect tactile information. Meanwhile, Teshigawara et al. [29] proposed the integration of highly sensitive tactile sensors into a lightweight multifinger robotic hand for slip detection. Heyneman et al. [30] designed a capacitive tactile sensing array and mounted it on Robotiq's grippers to identify different types of slip. Drimus et al. [31] designed an 8 × 8 tactile array based on a piezoresistive conductive rubber and mounted it on a three‐finger Schunk SDH robotic hand to classify deformable objects. In addition, Odhner et al. [32] integrated a barometer‐based tactile sensor array into a finger of an iRobot–Harvard–Yale robotic hand. Suárez‐Ruiz et al. [33] developed a CoRo Lab tactile sensor with a 4 × 7 resolution. They used it to detect the pressure, contact position, and vibration of Robotiq’s fingers. Tenzer et al. [34] proposed the integration of a microelectro mechanical system (MEMS) barometer into the fingers of a RightHand Robotics ReFlex TakkTile hand to feel the force distribution. Moreover, Jamone et al. [35] proposed a magnetic tactile sensor and installed it on a robotic hand for obstacle avoidance and object recognition. Based on the magnetic technology, Paulino et al. [36] proposed a three‐axis tactile sensor and installed it on the fingers of a Vizzy robot to detect normal and shear forces. Funabashi et al. [37] proposed the mounting of uSkin tactile sensors on the fingers of Allegro robots for object recognition. Wilson et al. [38] also indicated that integrating multiple GelSight optical tactile sensors in the fingers of a robotic hand can improve grasping efficiency.

Fig. 3.

Representative work on finger tactile sensing. (A) Reproduced with permission from Ref. [36], ©IEEE, 2017. (B) Reproduced with permission from Ref. [37], ©IEEE, 2018.

Robotic finger tactile sensing is widely used in many scenarios. Many tactile sensors have been equipped on commercial robotic hands (e.g., Shadow, Barret, Schunk SDH, and Robotiq) and robot platforms (e.g., UR, PR2, and iCub) for practical applications (e.g., environmental detection and object attribute recognition).

2.2 Palm tactile sensing

Palm tactile sensing plays an important role in applications such as dexterous manipulation (e.g., grasping and kneading), power grasp, and human–robot interaction (e.g., handshake and flicking). Fig. 4 shows some representative work. Capacitive tactile sensing arrays based on PPS have been installed in the palms of some commercial robotic hands. For example, a capacitive tactile sensor array with 96 contact points was mounted on a Barret Hand, 24 of which were distributed on the palm [39]. The Robonaut robot uses data gloves to detect the force distribution at 19 contact points. Tomo et al. [40] integrated a magnetic silicone tactile detection unit on the palm of a robotic hand. Wang et al. [41] placed four contacting points on the palm of the designed service robot. Pastor et al. [42] integrated a high‐resolution tactile array containing 1400 tactile sensing units on the palm of a robotic hand, which collaboratively worked with the under‐actuated fingers. The palm has a limited contact with the object during the operation; hence, it is greatly affected by the hand’s operating configuration. A palm usually needs to work together with the finger part to implement a task such as object material classification.

Fig. 4.

Representative work on palm tactile sensing. (A) Reproduced with permission from Ref. [39], ©IEEE, 2016. (B) Reproduced with permission from Ref. [41], ©IEEE, 2019. (C) Reproduced with permission from Ref. [42], ©the authors, 2019.

2.3 Arm tactile sensing

During a robotic manipulation, the robotic hand and arm must expand the workspace and increase the dexterity of the operation. Many researchers currently focus on hand tactile sensing, ignoring the importance of arm tactile sensing. Arm tactile sensing plays an important role in human–robot interaction, safe operation, and environment detection. Fig. 5 shows some representative work.

Fig. 5.

Representative work on arm tactile sensing. (A) Reproduced with permission from Ref. [43] , ©IEEE, 2015. (B) Reproduced with permission from Ref. [44], ©IEEE, 2017. (C) Reproduced with permission from Ref. [45], ©IEEE, 2018.

Accordingly, the Institute of Cognitive Systems at the Technical University of Munich designed a hexagonal tactile module to detect temperature, vibration, and light touch to simulate a multi‐modal tactile perception of the human skin. The tactile module was made into a patch and installed on the manipulator from KUKA Robotics Corp to improve its control performance [46]. Bhattacharjee et al. [43] used a robotic arm covered with capacitive tactile sensing arrays to identify a cluttered environment. Albini et al. [44] tiled capacitive tactile sensors on a robotic arm to distinguish various human–robot interactions, such as movement and rotation. Vergara et al. [45] integrated an electronic skin containing 373 tactile sensing units into a UR manipulator to ensure safety when the workspace was shared with other people. Meanwhile, Leboutet et al. [47] used tactile sensing information on an arm to implement compliant operation control. A research team from the Harbin Institute of Technology recently studied a method of deploying tactile sensors on the arms of nursing robots to achieve a safe operation [48]. Research on arm tactile sensing is presently in the beginning stage. Many researchers have adopted a "sleeve"‐style installation for integration into existing robotic arms, which is difficult to adapt to the large extension movement of a robotic arm.

2.4 Torso tactile sensing

In robots, the torso refers to body parts such as the chest, abdomen, and back. Besides the fingers, palms, and arms, torsos also play an important role in the humanoid robot and human–robot interaction. Fig. 6 shows some representative work.

Fig. 6.

Representative work on torso tactile sensing. (A) Reproduced with permission from Ref. [49], ©Taylor & Francis, 2015. (B) Reproduced with permission from Ref. [50], ©IEEE, 2019.

In reality, torso tactile sensing is also likely applicable to the hands, arms, legs, and other parts of the body. Argall et al. [51] summarized the applications of robotic torso tactile sensing in early human–robot interaction systems. In recent years, research on torso tactile sensing has attracted much attention with the continuous development of the large‐scale electronic skin technology. Metta et al. [52] used a capacitive tactile sensing unit to cover the torso of an iCub robot for cognitive ability research. Harada et al. [53] equipped a fingerprint‐like three‐axis tactile force sensor on a robot's body for tactile and slip detection. Mittendorfer et al. [49] imitated the human skin and developed a multi‐modal tactile sensor array, called CellulARSkin, to detect information such as proximity, vibration, temperature, and pressure. A tactile demonstration guides the movement of the entire body to grasp unknown objects. Kaboli et al. [54] further applied CellulARSkin to the torso of a small humanoid robot, NAO. Büscher et al. [55] designed a flexible, stretchable tactile sensor that can seamlessly cover a robot's body for daily human–robot interactions. Ku et al. [50] equipped a humanoid robot with a distributed tactile sensing array to identify 18 social touch patterns (e.g., hit, tap, and pat). Our research team recently installed a tactile array on the back of a service robot to recognize the tactile emotion during the human–robot interaction [56].

Distributed sensing in large‐scale, high‐resolution electronic skin and high‐speed signal interconnection are presently the main bottlenecks for equipping robotic torsos with tactile sensors [57].

2.5 Leg–foot tactile sensing

The research on leg–foot tactile sensing currently focuses on the feet, because they are the areas with the richest contact with the ground. The foot's tactile sensing helps a robot to sense the ground characteristics, adjust gait, improve walking efficiency, and ensure walking stability. Fig. 7 shows some representative work. Shill et al. [58] and Bednarek et al. [59] studied biped and multi‐pedal robots, respectively, in terrain classification, to measure physical interaction information and realize a terrain classification using the tactile sensors integrated into the feet. Wu et al. [60] used a flexible capacitive sensor array on robotic feet to measure the tactile force distribution with the ground and adaptively adjust the gait. Guadarrama et al. [61] studied tactile feedback control for a biped robot when it adapts to an unknown terrain. A research team at Massachusetts Institute of Technology (MIT) realized autonomous high‐speed bounding of Cheetah robots on a complex terrain using tactile intelligence [62]. Meanwhile, the research team from the Harbin Institute of Technology recently studied a terrain classification method using tactile sensors mounted on the bottom of the wheeled robot [63]. A research team in Beijing Nano‐Institute, Chinese Academy of Sciences, designed a smart insole with a novel air pressure‐driven structure to accurately monitor and distinguish various gait patterns [64].

2.6 Advanced tactile sensing for joints

Besides the fingers, palms, arms, torsos, legs, and feet, tactile sensing can also be deployed on tentacles and shoulders in a real robotic system. The current research mainly aims at specific parts of the robot. The resolution is not very high, although a significant progress has been made to develop the robotic tactile sensing technology. A large‐scale distributed tactile sensing system forms the basis for robots to realize embodied perception and learning, while a large gap still exists between the characteristics of the existing array‐type tactile sensors and the sensing ability of the human skin. Although a breakthrough has recently been made in the structure and materials of the electronic skin, achieving high flexibility and elasticity is still challenging. Large‐area tactile sensing arrays generally have poor scalability and are not easy to cut and splice. Manufacturing high‐sensitivity tactile sensing arrays is complicated and expensive, and therefore, are difficult to mass‐produce. Considering these challenges, deploying large-area tactile sensors on any robotic surface has not been realized yet, thereby restricting the development of robotic embodied intelligence.

Fig. 7.

Representative work on torso tactile sensing. (A) Reproduced with permission from Ref. [61], ©the authors, 2019. (B) Reproduced with permission from Ref. [59], ©IEEE, 2019.

The deforming and stretching characteristics of tactile sensors must be considered because of robot kinematics. Traditional tactile sensors are difficult to implement because of wiring constraints. Some tactile sensors are based on an optical measurement and are limited by the imaging distance and field of view. Recently, tomography‐based tactile sensors have been developed [65 –67], which, in principle, can overcome the limitations of wiring, and have received great attention in the field of flexible stretchable electronic skin. Electrical impedance tomography (EIT) is an imaging technique that uses electrodes to measure the voltage or current signals from the boundaries of the electrical conductors and reconstructs the internal conductive distribution. It can be used to implement robotic tactile sensing if a flexible stretchable material is used to develop the electrodes and attached to the robot surface. Tomography‐based tactile sensors are expected to provide a new solution for the development of robotic embodied tactile sensing.

In Ref. [68, 69], rubber materials were used as electrical conductors and EIT was implemented to reconstruct the resistivity distribution. The stretchable electronic skin was designed in Ref. [70, 71], which can adapt to the elbow of a robot with 19 electrodes and was designed into a thin, flexible, and stretchable artificial skin that can be applied on the fore arm and upper arm of the robot to obtain information (e.g., contact position, contact duration, and contact intensity). Furthermore, Pugach et al. [72] and Park et al. [73] used neural networks to implement image reconstruction. Fig. 8 shows some representative work on tactile sensing based on EIT.

Fig. 8.

Representative work on tactile sensing based on the EIT. (A) Reproduced with permission from Ref. [68], ©IEEE, 2007. (B) Reproduced with permission from Ref. [65], ©the authors, 2016.

Compared to traditional tactile sensors, EIT has advantages of low cost, low power consumption, no internal wiring, flexibility, scalability, and repeatability. It has become an important tool for the electronic skin because of its outstanding performance in designing a deformable, soft, flexible, and stretchable multipoint tactile sensor and detecting irregular surfaces and narrow areas. However, EIT has some shortcomings. Besides its higher requirements for electrical conductors, its temporal and spatial resolutions are low. Its current applications on robots are mainly limited to pose detection, security protection, and human–robot interaction. Tactile sensors based on EIT are generally more suitable for a spatial resolution of 10–40 mm and a temporal resolution within 45 Hz. This is obviously not enough for robotic tasks, such as object recognition and robotic grasping [74].

Electrical capacitance tomography (ECT) is another imaging method for visualizing permittivity distribution in the interior of a dielectric object by measuring the capacitance at the boundary, and has been widely used in industrial applications. Muhlbacher‐Karrer et al. [75] described an application of ECT in conductor detection. While two soft‐field imaging modalities (i.e., ECT and EIT) have some similarities, ECT can provide a quantitative measurement. However, note that EIT can provide a qualitative measurement only. For the electronic skin, ECT can realize non‐contact perception and object internal imaging. While non‐contact perception can provide predictive information for a safe operation, internal imaging is beneficial for the tactile perception of complex movements, such as power grasp, embracing, and chest/back contact. In addition, using a capacitor array on a robot enables the performance of imaging diagnosis on key parts of a robot, which is significant in maintaining the robot itself. However, there have been few reports on the application of ECT in the fields of robotic tactile sensing and electronic skin.

3 Embodied tactile perception

Tactile perception is based on the processing of an acquired tactile signal to understand objects and the environment. Essentially, tactile perception can be attributed to the tactile pattern recognition problem. Machine learning has become the mainstream method for solving this problem. This section briefly reviews the progress of tactile features and classification. Active tactile sensing is analyzed as the core technology of robotic embodied tactile perception. Tactile modality usually needs to be combined with other modalities in actual robot systems; hence, the progress of the multi‐modal fusion perception technology, which focuses on tactile modality, is reviewed.

3.1 Tactile feature learning and classification

The pattern recognition methods of robotic tactile perception have been described in detail in [18, 19]. Accordingly, many features have been proposed for tactile perception, including time domain, frequency domain, geometric, and statistical features. Some previous studies have been conducted to convert tactile signals into low‐resolution images. Image features, such as image moments and scale‐invariant feature transform, are then extracted. Many classifiers, including nearest neighbors, naive Bayes, decision trees, support vector machines, and artificial neural networks (ANNs) [76], are applied to tasks of object recognition, slip detection, and grasp quality evaluation. However, most of these classifiers use off‐the‐shelf feature extractors to process tactile signals. They have neither fully explored the inherent characteristics of the tactile signals nor considered the practical application of tactile perception.

Inspired by the sparse coding idea, dictionary‐based representation learning has been recently used to learn middle‐level features through low‐level tactile features. A new classifier has been designed based on structured sparse coding [77], which can effectively describe the dynamic characteristics of tactile signals and is suitable for a few samples of tactile data. Feature coding and classifiers can be adjusted to correspond to specific perception tasks. The method has been applied to tasks such as heterogeneous finger information fusion, tactile adjective attribute identification, and tactile object classification, thereby laying a foundation for the feature and classifier joint design.

With the development of deep learning and its successful applications in the fields of image, video, and speech, some researchers have begun applying deep neural networks to tactile information processing. Madry et al. [78] used unsupervised learning methods to learn the spatio‐temporal features of tactile sequences and obtained the best results for multiple tactile object recognition datasets. Afterward, deep learning methods quickly attracted researchers in the field of tactile perception. To solve the material recognition problem, a tactile sensor with a 4 × 4 resolution was used to collect a 2‐second sequence. A 1500‐dimensional sample was then obtained after 750 Hz sampling. The 1500 × 4 × 4 spatio‐temporal signal was converted into a 1500 × 16 two‐dimensional signal to achieve classification using a conventional neural network (CNN) [79]. Ji et al. [80] proposed the usage of weights of a trained sparse auto‐encoder to initialize those of a CNN for tactile signals, to improve performance and convergence. Zheng et al. [81] further studied full convolutional network learning. A hybrid deep learning structure with a CNN and a recurrent neural network were designed to process sequential tactile information online and solve the tactile emotion recognition problem in the human–robot interaction process [82]. Sohn et al. [83] applied deep learning to large‐scale electronic skin tactile perception. By using various contact forces, the obtained tactile spatio‐temporal sequence information was integrated and a 3D CNN was designed to realize object recognition [42]. CNNs are mostly designed for images; thus, good results can be expected on tactile sensors based on optical measurement. Yuan et al. [84] used a CNN to represent the tactile image obtained by the GelSight sensor. A recursive neural network was then employed to model the object deformation over time. Subsequently, a shape‐independent hardness estimation method based on deep learning was designed. Polic et al. [85] developed a convolutional self‐encoder for feature extraction in optical tactile sensors. Considering the lack of labeled samples, Erickson et al. [86] proposed a method based on semi‐supervised learning for object recognition based on generative adversarial networks. This method enables a robot to estimate the texture of everyday objects with approximately 90% accuracy when 92% of the training data are not labeled. A zero‐shot learning method was recently proposed for robotic tactile fabric recognition [87]. The University of Science and Technology of China used an optical equipment to obtain high‐resolution tactile information and a 50‐layer residual network ResNet to process the diffraction patterns, to improve the sensor performance [88].

Tactile data are more difficult to collect and label than visual and auditory modalities; thus, the performance of the current tactile deep learning is not significant. At present, most of the deep neural networks for tactile information processing are borrowed from the visual field and cannot specifically capture the tactile information characteristics. The shortcomings of deep learning in terms of long time and high energy consumption also restrict its further application in robotic tactile perception. In addition, relying on several manually labeled samples for offline learning, deep learning approaches can hardly be applied to robotic tactile sensing tasks, including online motion, exploration, and contact.

3.2 Active tactile perception

Although progress has been made in tactile feature learning and classification in recent years, most of the studies still collect data based on predefined rules and use pattern recognition methods to process them. Tactile perception can only provide local information on environment characteristics; thus, efficiently achieving a comprehensive perception of the environment is difficult. One of the important characteristics of robotic embodied perception is that it is an active process.

Active tactile perceptions attracted attention in the 1980s [89] for enabling robots to control the position and movement of tactile sensing systems through sensory exploration. The autonomous perception ability of a robot is enhanced with the optimized sensory information obtained from the environment and the reduced number of explorations. Seminara et al. [90] introduced the progress of active tactile perception from the perspective of autonomous learning and safe interaction. The current robotic active tactile perception focuses on two main aspects: shape reconstruction，and recognition and classification.

For shape reconstruction, Strub et al. [91] designed an active exploration strategy by considering the errors between object shape representation and tactile feature localization, simultaneously, to automatically realize the shape representation of objects. Abraham et al. [92] verified that low‐resolution binary contact sensors can be used to achieve shape estimation through effective active exploration. Jamali et al. [93] used Gaussian process classification to effectively sample an object’s surface and proposed an effective active learning strategy for its tactile exploration. Sommer et al. [94] designed an active exploration strategy for multifinger robots to adjust the number of contact points in real time, such that a robot can autonomously adapt to the object shape. Driess et al. [95] improved the traditional discrete point query method in the Gaussian process method through smooth path optimization, to avoid inefficient touch and retraction motion. A method for the fast shape estimation of unknown objects was proposed by introducing tactile uncertainty estimation into the exploration time constraints [96]. Ottenhaus et al. [97] maximized the estimated information gain in information theory to minimize the expected cost of exploration actions. They realized an active reconstruction of the object shapes from the sparse tactile data obtained from the robot fingers. However, limited by the accuracy of tactile sensors and exploration actions, the high‐precision and fast shape reconstruction of complex unknown objects through efficient active tactile exploration remains unsolved.

In recognition and classification, Saal et al. [98] resorted to information theory to analyze the dynamic parameters of bottles filled with different liquids, to estimate uncertainty. They then chose the next best action to quickly learn the dynamic parameters of objects through active perception. Zhang et al. [99] used the Monte Carlo tree search to optimize the grasping motion of a robot and realize active object shape recognition. Lepora et al. [100] established an active Bayesian tactile perception method to analyze the estimated uncertainty of each step of the action. The robotic operation was optimized to obtain more information in the next action. This method was applied on the tactile sensors integrated into the fingertips and tentacles [101]. Martinez‐Hernandez et al. [102, 103] further used multifinger coordination to realize active tactile object shape recognition. Tanaka et al. [104] transformed the active tactile object recognition problem into an optimal control problem. Active tactile exploration was achieved by optimizing the informativeness of the exploration and the compliance of action. It was then combined with the Gaussian process for object shape recognition. With a multivariate Bayesian classifier, Sun et al. [105] proposed a recognition method that fuses tactile information, such as friction and surface roughness, and local geometric features through active exploration. Xu et al. [106] realized the fusion of surface hardness and geometry information using active exploration with a tactile stylus. Kaboli et al. [107] recently designed an active tactile sensing method by modeling an uncertain workspace, so that a robot can actively explore the unknown workspace.

The abovementioned results showed that most of the current application scenarios of active tactile perception are relatively simple. The tactile exploration process is too long to be applied to an open environment. In addition, most of the tactile exploration strategies are based on the uncertainty measurement of the state estimation and are not closely integrated with the perception task. Moreover, most of the current active explorations are limited to robotic hands and arms, and movements from the other parts of the robot are rare.

3.3 Multimodal fusion perception

Although tactile signals play an important role in robotic embodied perception, they have limitations. Tactile perception is usually limited to the local area of contact, and the perception process is limited by the contact process. Many real environments and object properties may not be suitable for measuring signals only by tactile modality. In addition, the service time of tactile sensing is affected by frequent contact with the environment. Real robotic systems are usually equipped with other sensors, such as visual auditory. Visual auditory and tactile modalities complement each other in both perceptual range and characteristics. Therefore, multimodal sensors must be comprehensively leveraged to achieve fusion perception for robotic embodied perception and learning.

Visual and tactile modalities exhibit a big difference. Their format, frequency, and range of information vary greatly. Visual modality is generally more suitable for processing features such as color and shape, while tactile modality is more suitable for processing temperature, hardness, and other characteristics. Both visual and tactile modalities can be used to process surface material. The former generally deals with coarser materials, while the latter deals with finer materials. However, tactile modality can usually obtain information on objects with robot contact, and requires a long exploration process. Meanwhile, visual modality can be used to obtain considerable information on objects in the field of view. The asynchrony and difference in the sensing range of the two modalities bring great challenges to information fusion. Previous studies on visual–tactile fusion mainly focused on 3D object reconstruction [43, 108]. Accordingly, visual–tactile object recognition has attracted attention in recent years. Kroemer et al. [109] studied the use of visually assisted tactile feature extraction and texture classification, indicating that a key problem in visual–tactile fusion is the difficult pairing of visual images with the corresponding tactile data. To solve this problem, a visual–tactile object recognition method was established using weakly paired sparse coding and dictionary learning techniques [110, 111].

Recently, deep learning has been applied to solve visual–tactile fusion problems. For example, Gao et al. [112] used CNNs to study the joint learning method of visual images and tactile data. Zheng et al. [81] improved the network structure. Kerzel et al. [113] proposed a multichannel deep neural network structure and performed a spectrum analysis on the vibration and texture data collected during the exploration, by classifying 32 daily life materials. Yuan et al. [114] investigated the method of fusing an optical image and a tactile image collected by GelSight. Deep adversarial learning was used to solve a weakly paired problem between visual and tactile modalities [115, 116].

When a robot interacts with the environment, sound signals can be easily obtained besides tactile signals. Sound signals have the advantage of being superior to visual or tactile modalities in identifying certain internal states or materials. Similar to visual–tactile fusion, visual‐sound‐tactile modal information presents a more obvious weakly paired characteristic. Strese et al. [117] built a multimodal dataset that contained more than 100 types of surface materials. The problem in visual–sound–tactile heterogeneous fusion material recognition was further addressed using the sparse coding framework [118, 119].

Recently, researchers have focused on realizing multimodal robotic active perception by using visual–sound–tactile modal information. Yang et al. [120] combined passive vision with active touch to provide visual priori for tactile perception. Yuan et al. [121] proposed the use of vision for a robot to find the most suitable grasping point, for obtaining high‐quality tactile signals. However, in these processes, the order of sensor usage is manually preset. The problem of comprehensively scheduling sensors of different modalities to achieve efficient sensing based on task requirements is still challenging. Ferreira et al. [122] used the hierarchical Bayesian framework to analyze the emergence of robotic multimodal active perception. Taniguchi et al. [123] introduced information gain to calculate the uncertainty of different modalities under this framework, which was used to realize the automatic scheduling of visual, auditory, and tactile modalities. However, this study was restricted to very limited environments. An active visual–sound–tactile multimodal scheduling method was recently established by introducing adversarial reinforcement learning. The efficient perception of materials using seven types of modalities was achieved [124].

4 Embodied tactile learning

The abovementioned tactile perception methods include sparse coding and deep learning. While they are common machine learning methods for tactile learning, all of them require labeled tactile data. Tactile samples are more difficult to collect and label than visual and auditory data. The important ability of robotic embodied learning is to comprehensively utilize the association between different modalities to achieve mutual learning and enhancement. At the same time, data samples can be acquired to autonomously and continuously accumulate knowledge during the operation process.

Although cross‐modal perception is a basic ability of human beings, it is always challenging to handle in robotics. Kroemer et al. [109] proposed a cross‐modal learning method based on a joint canonical correlation analysis dimensionality reduction to learn the low‐dimensional representation of tactile data and improve the classification performance of dynamic tactile perception. It uses the visual information of object surfaces with training and tactile information to classify the material of objects during inference. However, the visual–tactile weakly paired problem in the cross‐modal learning process must still be overcome. To solve this problem, the cross‐modal visual–tactile information matching was designed using a joint dictionary learning method [125], and its application was then explored in cognitive robots. The idea of zero‐shot learning was recently introduced. If many samples cannot be obtained, a joint dictionary is used to achieve the transfer from visual features to tactile modality, thereby effectively achieving tactile learning [126].

Using deep neural networks to achieve visual–tactile cross‐modal learning has attracted attention. Yuan et al. [114] used deep convolutional networks to associate visual and tactile properties for fabric recognition. By comparing the embedding vectors, a robot can effectively predict the tactile characteristics of the fabric by looking at an image of an object. Luo et al. [127] used deep neural networks to extract the features of visual images and tactile data. The maximum covariance analysis was used to pair the learned features to learn the cross‐modal visual–tactile shared representation. Takahashi and Tan [128] and Gandarias et al. [129] recently studied how to obtain tactile features from optical images based on self‐encoding networks and CNNs and how to use them to enhance the tactile perception capability. Falco et al. [130] established an active exploration framework to realize cross‐modal visual–tactile object recognition. These results verified the possibility of using visual information to enhance tactile perception. However, existing methods have many limitations on objects and can only handle some simple objects.

While the current study mainly focused on the use of visual modality to enhance tactile learning, tactile data may contain even more information in many cases. Pinto et al. [131] used the tactile data generated by the physical interaction of objects and the environment to automatically generate supervised information for learning, thereby enhancing visual learning, which was further generalized by Xu et al. [132] to realize a multistep dynamic interaction.

Although cross‐modal transfer learning addresses the knowledge transfer between different modalities and considers robot self‐supervised learning, it does not implement knowledge accumulation and update during a real long‐term robotic operation. Whenever a new task is encountered, a robot has to retrain it from scratch and cannot leverage previous knowledge when an unknown scene is learned. An autonomous robot in a dynamic environment must be able to incrementally learn a series of tasks and continuously acquire and update its knowledge base during the learning process. The lifelong learning ability to continuously learn new tasks without forgetting previous tasks has received attention from the machine learning community in recent years. Kirkpatrick et al. [133] indicated the difficulty that lifelong learning is facing and called it the “catastrophic forgetting” problem. A research team from the Institute of Automation, Chinese Academy of Sciences, recently used the idea of orthogonal weights to fully protect the previously acquired information when gradient optimization is performed on deep networks. The team demonstrated a good lifelong learning ability in applications such as handwriting and face recognitions [134]. However, most of the current studies are still limited to single perception modality. Both online dictionary learning and deep learning methods have recently been studied for the lifelong cross‐modal learning for robot material perception [135, 136].

5 Tactile neuromorphic computing

Tactile computing is the foundation of tactile embodied intelligence. As mentioned, most existing robotic tactile sensors have a single contact point. Huge challenges in signal processing would occur when a contact array is densely distributed. Popular deep learning methods also have shortcomings in the tactile perception and learning process, such as insufficient data and large time and energy consumption, which limit their in‐depth application in robotic tactile perception and learning. This issue has motivated the academic community to start investigating neuromorphic computing in line with the cognitive mechanism of the human brain.

Neuromorphic computing is based on the results of neuroscience theories and biological experiments. It integrates cognitive science and information science, refers to biological neural network models and architectures, and uses existing complementary metal oxide semiconductor (CMOS) devices/circuits or new neuromorphic devices to simulate the manner in which biological neurons and synapse process information. An intelligent computing platform with functions of information perception, processing, and learning is built based on the human brain’s neural network. It also has significant advantages in terms of power, energy, and hardware requirements and is expected to become a revolutionary paradigm for dealing with massive real‐time data in the era of big data and artificial intelligence. At present, most specific chips for neuromorphic calculations are based on spiking neural networks (SNNs), which simulate the neuron operation. Unlike deep learning, which mainly imitates the hierarchical learning mechanism of the brain, SNN focuses on simulating brain‐like characteristics from spatio‐temporal dynamics. Its asynchronous event‐driven characteristics significantly reduce the energy consumption. Therefore, it plays an important role in real‐time dynamic applications, such as robotic embodied tactile perception and learning.

Governments and research institutions worldwide have made great investments in promoting “brain‐like computing” since 2004. The development of a series of “brain projects” has advanced research on neuromorphic chips based on the traditional CMOS technology. The representative work includes TrueNorth, Spiking Neural Networks architecture (SpiNNaker), BrainScaleS, Loihi, Tianji, and Darwin.

The University of California, Irvine, used the TrueNorth chip to implement an integrated test of autonomous navigation, decision‐making and autonomous vehicle control. Based on a hybrid heterogeneous brain‐like chip “Tianji”, Tsinghua University developed hardware circuits that combine an ANN based on accurate multi‐bit value processing and computing mechanism and a neuromorphic calculation circuit based on a binary pulse sequence. A neural‐state machine algorithm was used to integrate a multimodal neural computation of vision, speech, and inertial measurement unit sensor information to achieve self‐balancing, dynamic sensing, object detection, tracking, obstacle avoidance, speech understanding, and decision‐making for a self‐driving bicycle. In addition, novel neuromorphic devices, such as memristors, phase‐change memories, ferroelectric devices, and magnetic tunnel junctions, start from the perspective of bionics and simulate neurons and synapses from the device level. They are expected to have significant advantages in power consumption and hardware cost, but are still under the exploratory stage.

In 2006, the University of Manchester, in collaboration with the universities of Cambridge, Sheffield, and Southampton, and supported by three companies (i.e., ARM, Silistix, and Thales), launched the SpiNNaker project, which aimed to simulate up to 1% of the human brain’s neural network in real time [137]. The system integrated 57,600 customized digital packages, each of which incorporated a computer chip with 18 ARM cores and a 128 MB shared local memory chip. A single package could simulate up to 16 000 neurons and 8 million plastic synapses, consuming only 1 W of power. Compared to analog signal neuromorphic computing systems, digital signal neuromorphic computing systems are flexible and can meet the diverse needs of neuromorphic applications. The software development and platform maintenance of this project are supported by the EU‐funded Human Brain Project (HBP), and SpiNNaker, together with the Heidelberg BrainScaleS analogue neuromorphic system, forms a neuromorphic computing platform for HBP projects.

Recently, SpiNNaker‐related research has attracted wide attention from several countries, including the United Kingdom, France, Germany, United States, and Japan. On a SpiNNaker system, a structural plasticity learning function is used to implement a reward‐based synapse sampling algorithm that provides an effective tool for brain‐inspired algorithms [138]. Serrano‐Gotarredona et al. [139] optimized the storage design in the SpiNNaker system based on the “weight sharing” feature of a CNN by implementing a five‐layer CNN for symbol recognition. Mendat et al. [140] developed a Markov Chain Monte Carlo inference algorithm based on graph models using SpiNNaker. A hierarchical temporal memory model was implemented using SpiNNaker, which can be used for anomaly detection in sequence data [141].

The SpiNNaker application is continuously growing. In Tapiador‐Morales [142], a spiking convolution processor was used to extract image features from the Poker‐DVS dataset. The visual classification of four different symbols was also achieved on a SpiNNaker platform. Dominguez‐Morales et al. [143] realized audio sample classification using SpiNNaker. As reported by Gutierrez‐Galan et al. [144], SpiNNaker was used to classify audio information to generate gait for a hexapod robot. Kawasetsu et al. [145] used SpiNNaker to simulate a neural activity in the retina and visual cerebral cortex. Meanwhile, Stromatias et al. [146] used SpiNNaker to implement a deep belief network for handwriting digit recognition. Orchard et al. [147] used SpiNNaker to implement a hierarchical model and the X (HMAX) model for event‐driven object recognition. Haessig et al. [148] used SpiNNaker to recover an optical flow from dynamic visual information and realize stereo estimation. The simultaneous localization and mapping of a mobile robot were realized using SpiNNaker to simulate the navigation and localization mechanism of the rat hippocampus [149]. SpiNNaker was deployed on a mobile platform, which integrated visual processing capabilities. A universal event‐driven neuromorphic robotic computing platform was established [150]. Chen et al. [151] and Rast et al. [152] recently introduced SpiNNaker on a snake‐shaped robot and an iCub humanoid platform, respectively, to realize perception processing.

Although visual and auditory information processing based on neuromorphic has been extensively studied, few studies have been conducted on tactile modality. SNN can be implemented on field programmable gate array (FPGA) to process vibration tactile signals and simulate the fingers of mice for active exploration [153]. Bologna et al. [154] studied the neural decoding mechanism of tactile sensing. A decoding method from the force sensor signals to the spiking signals was designed using spiking neurons. Combined with a robot active detection task, the neural coding mechanism of fine tactile perception was then investigated. Spigler et al. [155] and Lee et al. [156] used the Izhikevich spiking neuron model to characterize the tactile coding and decoding mechanisms under different tasks, such as surface properties and tactile stimuli.

Based on the research of tactile encoding, some researchers have attempted to combine encoding information, especially the Izhikevich spiking neuron model, with machine learning and used it for practical tactile perception tasks. Rongala et al. [157] realized tactile encoding during the texture perception process. The spiking signal features were extracted to achieve texture classification. The abovementioned study indicated the importance of spiking temporal signals for tactile recognition. However, the training can only be implemented offline because of the complex classification calculation. Friedl et al. reported that a heterogeneous population of neurons was used to convert tactile signals into spiking patterns, combined with a band‐pass filter with hidden layers, to extract nonlinear frequency‐domain features [158]. The neuromorphic support vector machine was used for the classification task. Yi et al. [159] discussed the use of nearest‐neighbor classification based on tactile spiking encoding to realize surface roughness recognition. Although such algorithms can be conveniently implemented on a neuromorphic chip similar to SpiNNaker, theoretically, these tasks are only at the software simulation stage. The proposed structure strongly depends on specific tasks and is difficult to generalize. From the perspective of hardware implementation, Rasouli et al. [160] indicated that distributed tactile signal processing can significantly reduce the amount of transmitted information and help achieve portable processing. A neural network processor chip with 128 hidden layer nodes was designed to extract time‐domain features from spiking sequences to achieve texture classification. However, the critical inversion operation is still calculated on a conventional computer. The feasibility of using the extracted spiking signal features was further explored for combination with unsupervised clustering [161] and a sparse coding classifier [162]. These studies realized the neuromorphic construction and integration of tactile sensors, encoding interfaces, and feature extraction, and took an important step for the online application of tactile neuromorphic processing. Bologna et al. [154] utilized a closed‐loop neurobotic system, using an active perception strategy for recognizing the braille input. The system was equipped with a finger having 24 capacitive sensors, mounted on a robotic arm, to identify the object surface texture. It was designed to optimize the finger scanning speed and compensate for the motion execution errors. Both Friedl et al. [158] and Rasouli et al. [160] pointed out the importance of active tactile perception. Research in this area is seriously insufficient.

A series of new brain‐like computing chips represented by SpiNNker was preliminarily verified for processing robotic tactile signals. However, the advantages of the low‐power consumption and efficient processing of brain‐like chips have not been fully realized, because of various restrictions. In addition, existing studies mainly focus on feature extraction, and most classifiers are still based on existing ones. The spatio‐temporal characteristics of tactile signals are neither fully leveraged nor combined with robotic manipulation.

6 Conclusions

In this paper, the robotic tactile technology was analyzed from four perspectives: (1) robotic tactile sensing (sensing), (2) tactile perception (perception), (3) tactile learning (learning), and (4) tactile neuromorphic computing (computing). The following conclusions were drawn.

(1) Sensing: Tactile sensors are currently widely applied on robots, but most of them are limited to specific robotic parts. The wiring problem makes it difficult to adapt to a large‐area signal measurement, thereby significantly restricting research on robotic embodied tactile intelligence. Tomography has effectively overcome the wiring problem and initially showed great performance in the measurement of large‐area tactile signals. Tomography is expected to become an effective approach for solving the problem of large‐area tactile measurement in robotic embodied tactile research. The design of ECT sensors should be the focus in the aspect of “sensing” for embodied tactile tasks.

(2) Perception: Most of the current research on tactile perception is based on existing machine learning. The inherent characteristics of tactile signals have not yet been fully explored. Active tactile sensing, which leverages the tactile signal acquisition and exploration process, is important for robotic embodied tactile intelligence. However, current research results on active tactile perception are still preliminary, and robots still find it difficult to obtain active exploration strategies through autonomous learning, based on task requirements. The problem of robotic multimodal active perception, which is based on tactile perception, should be the focus in the aspect of “perception”.

(3) Learning: Most of the current research on tactile learning is related to “disembodied” learning, where the data collection and learning process are separated. The lifelong learning ability that can adapt to different tasks online is important for solving the robotic embodied tactile learning in an open environment. The use of robotic cross‐modal information should be the focus in the aspect of “learning”, to achieve lifelong self‐supervised learning.

(4) Computing: Current deep learning‐based methods are not suitable for robotic embodied tactile perception and learning tasks. SNN can effectively capture the spatio‐temporal characteristics of signals, and has a more efficient and low‐power computing capability than deep learning methods. It is an important tool for robotic embodied tactile learning. However, the current research can only solve a simple online inferring problem. In the aspect of “computing”, solving the problem of online learning should be the focus for complex embodied tactile tasks.

Footnotes

Conflict of interests

Tengchen Sun is the founder and chairman of Beijing Tashan Technology Co.,Ltd.

Financial support

This work was supported by the National Natural Science Foundation of China under Grant No. 61703284 and Grant No. 61673238.

References

Shimojo

Namiki

Ishikawa

, et al. A tactile sensor sheet using pressure conductive rubber with electrical-wires stitched method. IEEE Sens J. 2004, 4(5): 589–596.

Teshigawara

Shimizu

Tadakuma

, et al. High sensitivity slip sensor using pressure conductive rubber. In 2009 IEEE Sensors, Christchurch, New Zealand, 2009, pp 988–991.

Girão

Ramos

PMP

Postolache

, et al. Tactile sensors for robotic applications. Measurement. 2013, 46(3): 1257–1271.

Muhammad

Oddo

Beccai

, et al. Development of a bioinspired MEMS based capacitive tactile sensor for a robotic finger. Sensor Actuat A: Phys. 2011, 165(2): 221–229.

Schmitz

Maiolino

Maggiali

, et al. Methods and technologies for the implementation of large-scale robot tactile sensors. IEEE Trans Robot. 2011, 27(3): 389–400.

Goger

Gorges

Worn

Tactile sensing for an anthropomorphic robotic hand: Hardware and signal processing. In 2009 IEEE International Conference on Robotics and Automation, Kobe, Japan, 2009, pp 895–901.

Seminara

Capurro

Cirillo

, et al. Electromechanical characterization of piezoelectric PVDF polymer films for tactile sensors in robotics applications. Sensor Actuat A: Phys. 2011, 169(1): 49–58.

Chuang

Wang

, et al. Flexible tactile sensor for the grasping control of robot fingers. In 2013 International Conference on Advanced Robotics and Intelligent Systems. Tainan, China, 2013, pp 135–140.

Dahiya

Metta

Valle

, et al. Tactile sensing—from humans to humanoids. IEEE Trans Robot. 2010, 26(1): 1–20.

10.

Zhang

Liu

Jiang

, et al. Development of a flexible 3-D tactile sensor system for anthropomorphic artificial hand. IEEE Sens J. 2013, 13(2): 510–518.

11.

Johnson

Cole

Raj

, et al. Microgeometry capture using an elastomeric sensor. ACM T Graphic. 2011, 30(4): 1.

12.

Xie

Jiang

Wurdemann

, et al. Magnetic resonance-compatible tactile force sensor using fiber optics and vision sensor. IEEE Sens J. 2014, 14(3): 829–838.

13.

Fang

Sun

Yang

, et al. A dual-modal vision-based tactile sensor for robotic hand grasping. In 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, 2018, pp 4740–4745.

14.

Romano

Hsiao

Niemeyer

, et al. Human-inspired robotic grasp control with tactile sensing. IEEE Trans Robot. 2011, 27(6): 1067–1079.

15.

Jiang

Smith

. Seashell effect pretouch sensing for robotic grasping. In 2012 IEEE International Conference on Robotics and Automation, St Paul, USA, 2012, pp 2851–2858.

16.

Hasegawa

Mizoguchi

Tadakuma

, et al. Development of intelligent robot hand using proximity, contact and slip sensing. In 2010 IEEE International Conference on Robotics and Automation, Anchorage, USA, 2010, pp 777–784.

17.

Yamamoto

Wettels

Fishel

, et al. BioTac: Biomimetic multi-modal tactile sensor. J Robotics Soc Jpn. 2012, 30(5): 496–498.

18.

Kappassov

Corrales

Perdereau

. Tactile sensing in dexterous robot hands — Review. Robot Auton Syst. 2015, 74: 195–220.

19.

Luo

Bimbo

Dahiya

, et al. Robotic tactile perception of object properties: a review. Mechatronics. 2017, 48: 54–67.

20.

Winstone

Griffiths

Pipe

, et al. TACTIP - tactile fingertip device, texture analysis through optical tracking of skin features. In Biomimetic and Biohybrid Systems. Lepora

Mura

Krapp

, Eds. Berlin, Heidelberg: Springer, 2013.

21.

Loeb

Fishel

. Tactile identification of objects using Bayesian exploration. In 2013 IEEE International Conference on Robotics and Automation, Karlsruhe, Germany, 2013, pp 3056–3061.

22.

Schmitz

Maggiali

Natale

, et al. A tactile sensor for the fingertips of the humanoid robot iCub. In 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, Taipei, China, 2010, pp 2212–2217.

23.

Koiva

Zenker

Schurmann

, et al. A highly sensitive 3D-shaped tactile sensor. In 2013 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, Wollongong, USA, 2013, pp 1084–1089.

24.

Jara

Pomares

Candelas

, et al. Control framework for dexterous manipulation using dynamic visual servoing and tactile sensors’ feedback. Sensors (Basel). 2014, 14(1): 1787–1804.

25.

Song

Bhattacharjee

Srinivasa

. Sensing shear forces during food manipulation: resolving the trade-off between range and sensitivity. In 2019 International Conference on Robotics and Automation (ICRA), Montreal, Canada, 2019, pp 8367–8373.

26.

Schmidt

Maël

Würtz

. A sensor for dynamic tactile information with applications in human–robot interaction and object exploration. Robot Auton Syst. 2006, 54(12): 1005–1014.

27.

Jamali

Sammut

. Majority voting: material classification by tactile sensing using surface texture. IEEE Trans Robot. 2011, 27(3): 508–521.

28.

Yousef

Boukallel

Althoefer

. Tactile sensing for dexterous in-hand manipulation in robotics—A review. Sensor Actuat A: Phys. 2011, 167(2): 171–187.

29.

Teshigawara

Tsutsumi

Shimizu

, et al. Highly sensitive sensor for detection of initial slip and its application in a multi-fingered robot hand. In 2011 IEEE International Conference on Robotics and Automation, Shanghai, China, 2011, pp 1097–1102.

30.

Heyneman

Cutkosky

. Biologically inspired tactile classification of object-hand and object-world interactions. In 2012 IEEE International Conference on Robotics and Biomimetics (ROBIO), Guangzhou, China, 2012, pp 167–173.

31.

Drimus

Kootstra

Bilberg

, et al. Design of a flexible tactile sensor for classification of rigid and deformable objects. Robot Auton Syst. 2014, 62(1): 3–15.

32.

Odhner

Jentoft

Claffee

, et al. A compliant, underactuated hand for robust manipulation. Int J Robot Res. 2014, 33(5): 736–752.

33.

Suárez-Ruiz

Galiana

Tenzer

, et al. Grasp mapping between a 3-finger haptic device and a robotic hand. In Haptics: Neuroscience, Devices, Modeling, and Applications. Auvray

Malika

Duriez

Christian

, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2014.

34.

Tenzer

Jentoft

Howe

. The feel of MEMS barometers: inexpensive and easily customized tactile array sensors. IEEE Robot Automat Mag. 2014, 21(3): 89–95.

35.

Jamone

Natale

Metta

, et al. Highly sensitive soft tactile sensors for an anthropomorphic robotic hand. IEEE Sens J. 2015, 15(8): 4226–4233.

36.

Paulino

Ribeiro

Neto

, et al. Low-cost 3-axis soft tactile sensors for the human-friendly robot Vizzy. In 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore, 2017, pp 966–71.

37.

Funabashi

Morikuni

Geier

, et al. Object recognition through active sensing using a multi-fingered robot hand with 3D tactile sensors. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, 2018, pp 2589–2595.

38.

Wilson

Wang

Romero

, et al. Design of a fully actuated robotic hand with multiple Gelsight tactile sensors. arXiv preprint. 2020, arXiv: 2002.02474.

39.

Liu

Guo

Sun

. Object recognition using tactile measurements: kernel sparse coding methods. IEEE Trans Instrum Meas. 2016, 65(3): 656–665.

40.

Tomo

Wong

Schmitz

, et al. A modular, distributed, soft, 3-axis sensor system for robot hands. In 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids), Cancun, Mexico, 2016, pp 454–460.

41.

Wang

Geng

Kang

, et al. Eagle Shoal: A new designed modular tactile sensing dexterous hand for domestic service robots. In 2019 International Conference on Robotics and Automation (ICRA), Montreal, Canada, 2019, pp 9087–9093.

42.

Pastor

Gandarias

García-Cerezo

, et al. Using 3D convolutional neural networks for tactile object recognition with robotic palpation. Sensors (Basel). 2019, 19(24): E5356.

43.

Bhattacharjee

Shenoi

Park

, et al. Combining tactile sensing and vision for rapid haptic mapping. In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany, 2015, pp 1200–1207.

44.

Albini

Denei

Cannata

Enabling natural human-robot physical interaction using a robotic skin feedback and a prioritized tasks robot control architecture. In 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids), Birmingham, UK, 2017, pp 99–106.

45.

Vergara

Borghesan

Aertbelien

, et al. Incorporating artificial skin signals in the constraint-based reactive control of human-robot collaborative manipulation tasks. In 2018 3rd International Conference on Advanced Robotics and Mechatronics (ICARM), Singapore, 2018, pp 280–287.

46.

Mittendorfer

Cheng

. Humanoid multimodal tactile-sensing modules. IEEE Trans Robot. 2011, 27(3): 401–410.

47.

Leboutet

Dean-Leon

Bergner

, et al. Tactile-based whole-body compliance with force propagation for mobile manipulators. IEEE Trans Robot. 2019, 35(2): 330–342.

48.

Liang

Huang

, et al. Soft sensitive skin for safety control of a nursing robot using proximity and tactile sensors. IEEE Sens J. 2020, 20(7): 3822–3830.

49.

Mittendorfer

Yoshida

Cheng

. Realizing whole-body tactile interactions with a self-organizing, multi-modal artificial skin on a humanoid robot. Adv Robotics. 2015, 29(1): 51–67.

50.

Choi

Jang

, et al. Online social touch pattern recognition with multi-modal-sensing modular tactile interface. In 2019 16th International Conference on Ubiquitous Robots (UR), Jeju, Korea, 2019, pp 271–277.

51.

Argall

Billard

. A survey of tactile human–robot interactions. Robot Auton Syst. 2010, 58(10): 1159–1176.

52.

Metta

Natale

Nori

, et al. The iCub humanoid robot: an open-systems platform for research in cognitive development. Neural Netw. 2010, 23(8/9): 1125–1134.

53.

Harada

Kanao

Yamamoto

, et al. Fully printed flexible fingerprint-like three-axis tactile and slip force and temperature sensors for artificial skin. ACS Nano. 2014, 8(12): 12851–12857.

54.

Kaboli

New methods for active tactile object perception and learning with artificial robotic skin. PhD Dissertation, Technische Universität München, Germany, 2017.

55.

Büscher

Kõiva

Schürmann

, et al. Flexible and stretchable fabric-based tactile sensor. Robot Auton Syst. 2015, 63: 244–252.

56.

Wei

Liu

Wang

, et al. Lifelong learning for tactile emotion recognition. 2019, 20(1): 25–41.

57.

Núñez

Navaraj

Polat

, et al. Energy-autonomous, flexible, and transparent tactile skin. Adv Funct Mater. 2017, 27(18): 1606287.

58.

Shill

Collins

Jr Coyle

, et al. Tactile surface classification for limbed robots using a pressure sensitive robot skin. Bioinspir Biomim. 2015, 10(1): 016012.

59.

Bednarek

Wellhausen

, et al. What am I touching? Learning to classify terrain via haptic sensing. In 2019 International Conference on Robotics and Automation (ICRA), Montreal, Canada, 2019, pp 7187–7193.

60.

Huh

Sabin

, et al. Tactile sensing and terrain-based gait control for small legged robots. IEEE Trans Robot. 2020, 36(1): 15–27.

61.

Rogelio Guadarrama Olvera

Leon

Bergner

, et al. Plantar tactile feedback for biped balance and locomotion on unknown terrain. Int J Human Robot. 2020, 17(1): 1950036.

62.

Park

Wensing

Kim

High-speed bounding with the MIT Cheetah 2: Control design and experiments. Int J Robot Res. 2017, 36(2): 167–192.

63.

Bai

Guo

Zheng

. Three-dimensional vibration-based terrain classification for mobile robots. IEEE Access. 2019, 7: 63485–63492.

64.

Lin

Zhang

, et al. A triboelectric nanogenerator-based smart insole for multifunctional gait monitoring. Adv Mater Technol. 2019, 4(2): 1800360.

65.

Visentin

Fiorini

Suzuki

. A deformable smart skin for continuous sensing based on electrical impedance tomography. Sensors (Basel). 2016, 16(11): E1928.

66.

Lee

Kwon

Cho

, et al. Soft nanocomposite based multi-point, multi-directional strain mapping sensor using anisotropic electrical impedance tomography. Sci Rep. 2017, 7: 39837.

67.

Russo

Assaf

Carbonaro

, et al. Touch position detection in electrical tomography tactile sensors through quadratic classifier. IEEE Sens J. 2019, 19(2): 474–483.

68.

Nagakubo

Alirezaei

Kuniyoshi

Y. A

deformable and deformation sensitive tactile distribution sensor. In 2007 IEEE International Conference on Robotics and Biomimetics (ROBIO), Sanya, China, 2007, pp 1301–1308.

69.

Kato

Mukai

Hayakawa

, et al. Tactile sensor without wire and sensing element in the tactile region based on EIT method. In 2007 IEEE Sensors. Atlanta, USA, 2007, pp 792–795.

70.

Tawil

Rye

Velonaki

. Improved image reconstruction for an EIT-based sensitive skin with multiple internal electrodes. IEEE Trans Robot. 2011, 27(3): 425–435.

71.

Silvera Tawil

Rye

Velonaki

. Interpretation of the modality of touch on an artificial arm covered with an EIT-based sensitive skin. Int J Robot Res. 2012, 31(13): 1627–1641.

72.

Pugach

Pitti

Gaussier

. Neural learning of the topographic tactile sensory information of an artificial skin through a self-organizing map. Adv Robotics. 2015, 29(21): 1393–1409.

73.

Park

Lee

Park

, et al. Deep neural network approach in electrical impedance tomography-based real-time soft tactile sensor. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China, 2019, pp 7447–7452.

74.

Silvera-Tawil

Rye

Soleimani

, et al. Electrical impedance tomography for artificial sensitive robotic skin: a review. IEEE Sens J. 2015, 15(4): 2001–2016.

75.

Mühlbacher-Karrer

Padilha Leitzke

Faller

, et al. Non-iterative object detection methods in electrical tomography for robotic applications. COMPEL. 2017, 36(5): 1411–1420.

76.

Cao

Sun

Kotagiri

, et al. Real-time recurrent tactile recognition: momentum batch-sequential echo state networks. IEEE Trans Syst Man Cybern, Syst. 2020, 50(4): 1350–1361.

77.

Liu

Sun

. Robotic Tactile Perception and Understanding. Singapore: Springer Singapore, 2018.

78.

Madry

Kragic

, et al. ST-HMP: Unsupervised Spatio-Temporal feature learning for tactile data. In 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, 2014, 2262–2269.

79.

Baishya

Bauml

Robust material classification with a tactile skin using deep learning. In 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, South Korea, 2016, 8–15.

80.

Fang

Zheng

, et al. Preprocessing-free surface material classification using convolutional neural networks pretrained by sparse Autoencoder. In 2015 IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP), Boston, USA, 2015, pp 1–6.

81.

Zheng

Fang

, et al. Deep learning for surface material classification using haptic and visual information. IEEE Trans Multimedia. 2016, 18(12): 2407–2416.

82.

Hughes

Krauthammer

Correll

Recognizing social touch gestures using recurrent and convolutional neural networks. In 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore, 2017, pp 2315–2321.

83.

Sohn

Chung

Cho

, et al. An extremely simple macroscale electronic skin realized by deep machine learning. Sci Rep. 2017, 7(1): 11061.

84.

Yuan

Zhu

Owens

, et al. Shape-independent hardness estimation using deep learning and a GelSight tactile sensor. In 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore, 2017, pp 951–958.

85.

Polic

Krajacic

Lepora

, et al. Convolutional autoencoder for feature extraction in tactile sensing. IEEE Robot Autom Lett. 2019, 4(4): 3671–3678.

86.

Erickson

Chernova

Kemp

. Semi-supervised haptic material recognition for robots using generative adversarial networks. arXiv preprint. 2017, arXiv:1707.02796.

87.

Wang

Liu

Sun

, et al. Fabric recognition using zero-shot learning. Tinshhua Sci Technol. 2019, 24(6): 645–653.

88.

Hao

Zhu

Dong

. An optical tactile sensor with structural color using deep learning method. In 2019 IEEE International Conference on Mechatronics and Automation (ICMA), Tianjin, China, 2019, pp 707–712.

89.

Goldberg

Bajcsy

. Active touch and robot perception. Cogn Brain Theo. 1984, 7 (2): 199–214.

90.

Seminara

Gastaldo

Watt

, et al. Active haptic perception in robots: a review. Front Neurorobot. 2019, 13: 53.

91.

Strub

Worgotter

Ritter

, et al. Correcting pose estimates during tactile exploration of object shape: a neuro-robotic study. In 4th International Conference on Development and Learning and on Epigenetic Robotics, Genoa, Italy, 2014, pp 26–33.

92.

Abraham

Prabhakar

Hartmann

MJZ

, et al. Ergodic exploration using binary sensing for nonparametric shape estimation. IEEE Robot Autom Lett. 2017, 2(2): 827–834.

93.

Jamali

Ciliberto

Rosasco

, et al. Active perception: Building objects’ models using tactile exploration. In 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids), Cancun, Mexico, 2016, pp 179–185.

94.

Sommer

Billard

. Multi-contact haptic exploration and grasping with tactile sensors. Robot Auton Syst. 2016, 85: 48–61.

95.

Driess

Englert

Toussaint

Active learning with query paths for tactile object shape exploration. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, Canada, 2017, pp 65–72.

96.

Matsubara

Shibata

. Active tactile exploration with uncertainty and travel cost for fast shape estimation of unknown objects. Robot Auton Syst. 2017, 91: 314–326.

97.

Ottenhaus

Kaul

Vahrenkamp

, et al. Active tactile exploration based on cost-aware information gain maximization. Int J Human Robot. 2018, 15(1): 1850015.

98.

Saal

Ting

Vijayakumar

Active estimation of object dynamics parameters with tactile sensors. In 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, Taipei, China, 2010, pp 916–921.

99.

Zhang

Atanasov

Daniilidis

. Active tactile object recognition by Monte Carlo tree search. https://arxiv.org/abs/1703.00095 (accessed 2017).

100.

Lepora

Martinez-Hernandez

Prescott

. Active Bayesian perception for simultaneous object localization and identification. In Robotics: Science and Systems Foundation, 2013.

101.

Lepora

. Biomimetic active touch with fingertips and whiskers. IEEE Trans Haptics. 2016, 9(2): 170–183.

102.

Martinez-Hernandez

Dodd

Prescott

. Feeling the shape: active exploration behaviors for object recognition with a robotic hand. IEEE Trans Syst Man Cybern, Syst. 2018, 48(12): 2339–2348.

103.

Martinez-Hernandez

Dodd

Evans

, et al. Active sensorimotor control for tactile exploration. Robot Auton Syst. 2017, 87: 15–27.

104.

Tanaka

Matsubara

Sugimoto

An optimal control approach for exploratory actions in active tactile object recognition. In 2014 IEEE-RAS International Conference on Humanoid Robots, Madrid, Spain, 2014, pp 787–793.

105.

Sun

Back

Liu

. Combining contact forces and geometry to recognize objects during surface haptic exploration. IEEE Robot Autom Lett. 2018, 3(3): 2509–2514.

106.

Lin

Fan

, et al. Exploring Hardness and Geometry Information through Active Perception. In 2019 WRC Symposium on Advanced Robotics and Automation (WRC SARA), Beijing, China, 2019, pp 2509–2514.

107.

Kaboli

Yao

Feng

, et al. Tactile-based active object discrimination and target object search in an unknown workspace. Auton Robot. 2019, 43(1): 123–152.

108.

Ilonen

Bohg

Kyrki

Fusing visual and tactile sensing for 3-D object reconstruction while grasping. In 2013 IEEE International Conference on Robotics and Automation, Karlsruhe, Germany, 2013, pp 3547–3554.

109.

Kroemer

Lampert

Peters

. Learning dynamic tactile sensing with robust vision-based training. IEEE Trans Robot. 2011, 27(3): 545–557.

110.

Liu

Sun

, et al. Visual–tactile fusion for object recognition. IEEE Trans Automat Sci Eng. 2017, 14(2): 996–1008.

111.

Liu

Sun

, et al. Weakly paired multimodal fusion for object recognition. IEEE Trans Automat Sci Eng. 2018, 15(2): 784–795.

112.

Gao

Hendricks

Kuchenbecker

, et al. Deep learning for tactile understanding from visual and haptic data. In 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, 2016, pp 536–543.

113.

Kerzel

Ali

, et al. Haptic material classification with a multi-channel neural network. In 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, USA, 2017, pp 439–446.

114.

Yuan

Wang

Dong

, et al. Connecting look and feel: associating the visual and tactile properties of physical materials. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, USA, 2017, pp 5580–5588.

115.

Zheng

Liu

Wang

, et al. Cross-modal surface material retrieval using discriminant adversarial learning. IEEE Trans Ind Inf. 2019, 15(9): 4978–4987.

116.

Zheng

Liu

Wang

, et al. Cross-modal material perception for novel objects: a deep adversarial learning method. IEEE Trans Automat Sci Eng. 2020, 17(2): 697–707.

117.

Strese

Schuwerk

Iepure

, et al. Multimodal feature-based surface material classification. IEEE Trans Haptics. 2017, 10(2): 226–239.

118.

Liu

Sun

Fang

, et al. Multimodal measurements fusion for surface material categorization. IEEE Trans Instrum Meas. 2018, 67(2): 246–256.

119.

Liu

Wang

Sun

, et al. Surface material retrieval using weakly paired cross-modal learning. IEEE Trans Automat Sci Eng. 2019, 16(2): 781–791.

120.

Yang

Lepora

. Object exploration using vision and active touch. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, Canada, 2017, pp 6363–6370.

121.

Yuan

Wang

, et al. Active clothing material perception using tactile sensing and deep learning. In 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, 2018, pp 4842–4849.

122.

Ferreira

Tsiourti

Dias

. Learning emergent behaviours for a hierarchical Bayesian framework for active robotic perception. Cogn Process. 2012, 13(Suppl 1): S155–S159.

123.

Taniguchi

Yoshino

Takano

. Multimodal hierarchical dirichlet process-based active perception by a robot. Front Neurorobot. 2018, 12: 22.

124.

Liu

Sun

Zhang

. Robotic material perception using active multimodal fusion. IEEE Trans Ind Electron. 2019, 66(12): 9878–9886.

125.

Liu

Wang

Sun

, et al. Active visual-tactile cross-modal matching. IEEE Trans Cogn Dev Syst. 2019, 11(2): 176–187.

126.

Liu

Sun

Fang

, et al. Cross-modal zero-shot-learning for tactile object recognition. IEEE Trans Syst Man Cybern, Syst. 2018: 1–9.

127.

Luo

Yuan

Adelson

, et al. ViTac: Feature sharing between vision and tactile sensing for cloth texture recognition. In 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, 2018, 2722–2727.

128.

Takahashi

Tan

Deep visuo-tactile learning: estimation of tactile properties from images. In 2019 International Conference on Robotics and Automation (ICRA), Montreal, Canada, 2019, 8951–8957.

129.

Gandarias

Garcia-Cerezo

Gomez-De-gabriel

. CNN-based methods for object recognition with high-resolution tactile sensors. IEEE Sens J. 2019, 19(16): 6872–6882.

130.

Falco

Natale

, et al. A transfer learning approach to cross-modal object recognition: from visual observation to robotic haptic exploration. IEEE Trans Robot. 2019, 35(4): 987–998.

131.

Pinto

Gandhi

Han

, et al. The curious robot: learning visual representations via physical interactions. In Computer Vision – ECCV 2016. Cham: Springer International Publishing, 2016.

132.

Zeng

, et al. DensePhysNet: Learning dense physical object representations via multi-step dynamic interactions. In Robotics: Science and Systems Foundation, 2019.

133.

Kirkpatrick

Pascanu

Rabinowitz

, et al. Overcoming catastrophic forgetting in neural networks. Proc Natl Acad Sci USA. 2017, 114(13): 3521–3526.

134.

Zeng

Chen

Cui

, et al. Continual learning of context-dependent processing in neural networks. Nat Mach Intell. 2019, 1(8): 364–372.

135.

Sun

Liu

Yang

, et al. Multi-modal continual learning using online dictionary updating. IEEE Trans Cogn Dev Syst. 2020: 1.

136.

Zheng

Liu

Sun

. Lifelong visual-tactile cross-modal learning for robotic material perception. IEEE Trans Neural Netw Learn Syst. 2020, in press, DOI 10.1109/TNNLS.2020.2980892.

137.

Furber

Galluppi

Temple

, et al. The SpiNNaker project. Proc IEEE. 2014, 102(5): 652–665.

138.

Yan

Kappel

Neumarker

, et al. Efficient reward-based structural plasticity on a SpiNNaker 2 prototype. IEEE Trans Biomed Circuits Syst. 2019, 13(3): 579–591.

139.

Serrano-Gotarredona

Linares-Barranco

Galluppi

, et al. ConvNets experiments on SpiNNaker. In 2015 IEEE International Symposium on Circuits and Systems (ISCAS), Lisbon, Portugal, 2015, pp 2405–2408.

140.

Mendat

Chin

Furber

, et al. Markov Chain Monte Carlo inference on graphical models using event-based processing on the SpiNNaker neuromorphic architecture. In 2015 49th Annual Conference on Information Sciences and Systems (CISS), Baltimore, USA, 2015, pp 1–6.

141.

Walter

Sandner

Rcohrbein

, et al. Towards a neuromorphic implementation of hierarchical temporal memory on SpiNNaker. In 2017 IEEE International Symposium on Circuits and Systems (ISCAS), Baltimore, USA, 2017, pp 1–4.

142.

Tapiador-Morales

Dominguez-Morales

Gutierrez-Galan

, et al. Live demonstration: Neuromorphic row-by-row multi-convolution FPGA processor-SpiNNaker Architecture for dynamic-vision feature extraction. In 2019 IEEE International Symposium on Circuits and Systems (ISCAS), Sapporo, Japan, 2019, 1–1.

143.

Dominguez-Morales

Jimenez-Fernandez

Rios-Navarro

, et al. Multilayer spiking neural network for audio samples classification using SpiNNaker. In Artificial Neural Networks and Machine Learning – ICANN 2016. Cham: Springer International Publishing, 2016.

144.

Gutierrez-Galan

Dominguez-Morales

Perez-Pena

, et al. Live demonstration: neuromorphic robotics, from audio to locomotion through spiking CPG on SpiNNaker. In 2019 IEEE International Symposium on Circuits and Systems (ISCAS), Sapporo, Japan, 2019, 1–1.

145.

Kawasetsu

Ishida

Sanada

, et al. Live demonstration: A hardware system for emulating the early vision utilizing a silicon retina and SpiNNaker chips. In 2014 IEEE Biomedical Circuits and Systems Conference (BioCAS) Proceedings, Lausanne, Switzerland, 2014, pp 552–555.

146.

Stromatias

Neil

Galluppi

, et al. Live demonstration: Handwritten digit recognition using spiking deep belief networks on SpiNNaker. In 2015 IEEE International Symposium on Circuits and Systems (ISCAS), Lisbon, Portugal, 2015, pp 1901–1901.

147.

Orchard

Lagorce

Posch

, et al. Real-time event-driven spiking neural network object recognition on the SpiNNaker platform. In 2015 IEEE International Symposium on Circuits and Systems (ISCAS), Lisbon, Portugal, 2015, pp 2413–2416.

148.

Haessig

Galluppi

Lagorce

, et al. Neuromorphic networks on the SpiNNaker platform. In 2019 IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS), Hsinchu, China, 2019, pp 86–91.

149.

Galluppi

Conradt

Stewart

, et al. Live Demo: Spiking ratSLAM: Rat hippocampus cells in spiking neural hardware. In 2012 IEEE Biomedical Circuits and Systems Conference, Zhubei, China, 2012, pp 91.

150.

Galluppi

Denk

Meiner

, et al. Event-based neural computing on an autonomous mobile platform. In 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, 2014, pp 2862–2867.

151.

Chen

Bing

Rohrbein

, et al. Toward brain-inspired learning with the neuromorphic snake-like robot and the neurorobotic platform. IEEE Trans Cogn Dev Syst. 2019, 11(1): 1–12.

152.

Rast

Adams

Davidson

, et al. Behavioral learning in a cognitive neuromorphic robot: an integrative approach. IEEE Trans Neural Netw Learn Syst. 2018, 29(12): 6132–6144.

153.

Pearson

Pipe

Mitchinson

, et al. Implementing spiking neural networks for real-time signal-processing and control applications: a model-validated FPGA approach. IEEE Trans Neural Netw. 2007, 18(5): 1472–1487.

154.

Bologna

Pinoteau

Passot

, et al. A closed-loop neurobotic system for fine touch sensing. J Neural Eng. 2013, 10(4): 046019.

155.

Spigler

Oddo

Carrozza

. Soft-neuromorphic artificial touch for applications in neuro-robotics. In 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob), Rome, Italy, 2012, pp 1913–1917.

156.

Lee

Cabibihan

Thakor

. Bio-mimetic strategies for tactile sensing. In 2013 IEEE SENSORS, Baltimore, USA, 2013, pp 1–4.

157.

Rongala

Mazzoni

Oddo

. Neuromorphic artificial touch for categorization of naturalistic textures. IEEE Trans Neural Netw Learn Syst. 2017, 28(4): 819–829.

158.

Friedl

Voelker

Peer

, et al. Human-inspired neurorobotic system for classifying surface textures by touch. IEEE Robot Autom Lett. 2016, 1(1): 516–523.

159.

Zhang

. Recognizing tactile surface roughness with a biomimetic fingertip: a soft neuromorphic approach. Neurocomputing. 2017, 244: 102–111.

160.

Rasouli

Chen

Basu

, et al. An extreme learning machine-based neuromorphic tactile sensing system for texture recognition. IEEE Trans Biomed Circuits Syst. 2018, 12(2): 313–325.

161.

Iskarous

Nguyen

Osborn

, et al. Unsupervised learning and adaptive classification of neuromorphic tactile encoding of textures. In 2018 IEEE Biomedical Circuits and Systems Conference (BioCAS), Cleveland, USA, 2018, pp 1–4.

162.

Nguyen

Osborn

Iskarous

, et al. Dynamic texture decoding using a neuromorphic multilayer tactile sensor. In 2018 IEEE Biomedical Circuits and Systems Conference (BioCAS), Cleveland, USA, 2018, pp 1–4.