The 12th IEEE International Workshop on Advanced Motion Control March 25-27, 2012, Sarajevo, Bosnia and Herzegovina
Towards multimodal Haptics for teleoperation: Design of a Tactile Thermal Display Simon Gallo, Laura Santos-Carreras, Giulio Rognini
Masayuki Hara, Akio Yamamoto, Toshiro Higuchi
Lab of Robotic Systems (LSRO) EPFL Lausanne, Switzerland
[email protected],
[email protected]
Department of precision Engineering University of Tokyo, Japan Hannes Bleuler, LSRO EPFL Lausanne
Abstract — Surgical robotics is among the most challenging applications of motion control. Present and future systems are essentially master-slave systems. Our work focuses on forcefeedback and haptic interfaces. In this context, we study multimodal haptic interfaces, i.e. the fusion of force-feedback, with other tactile information such as temperature or pressure. First results support the proposition that such multimodal haptic devices can help improve surgeon’s dexterity and motion control. In order to strengthen this point, we investigate the psychophysics of thermal perception. This paper presents a device for temperature feedback that can be integrated in a multimodal haptic console. A finger sized tactile temperature display able to generate temperature gradients under the fingertip is presented along with first measurement results.
tools (and rubber hands) likely depends upon the strength of multisensory and sensorimotor congruency and in general, on the quality of multimodal feedback [4][5], i.e. including multiple modalities such as temperature, texture, palpation in a congruent way with visual feedback greatly increases the perception of the surgery tool as embodiment of the surgeon. In this paper, we focus on one aspect of such a multimodal haptic device, the temperature rendering.
Keywords: Haptic device, master console, multimodal haptics, temperature rendering
Advances in technology allow more complex tactile interfaces, the goal is to design them also more intuitive. Experimental results with a telepalpation device have been reported, but to our knowledge, in the domain of robotic surgery, there are still no commercial applications of telepalpation devices including temperature feedback.
I.
INTRODUCTION
Minimally invasive surgery (MIS) has many benefits, it produces small wounds, thus decreasing pain, infection risk and scars for the patient. A much shorter recovery time helps to bring down significantly health care costs. The cost is obviously a greater distance of the surgeon from the patient [1]. The surgeon’s essential tool is his hand, with its high dexterity and very complete multimodal sensing capabilities (force, texture, temperature, pressure). The surgeon can e.g. instinctively discern live from dead tissue and acts accordingly. In present-day robotic surgery, high dexterity is provided by advanced instrumentation, but the multimodal sensing of the hands and the direct patient contact is completely lost. The surgeon acts only relying on visual feedback. Therefore haptic consoles and force feedback devices for robotic surgery are being developed by many research groups [2] [3]. II.
MOTIVATION
In the last decade many studies in the field of cognitive neuroscience have shown that the degree of embodiment for
978-1-4577-1073-5/12/$26.00 ©2012 IEEE
Sensors integrated on the tip of the surgical tools provide data about the point of operation. The haptic device ideally should render the essential properties of the internal organs back to the surgeon’s hands.
Among the important issues for the design of multimodal haptic feedback will be the reliability and intuitiveness. Indications are that they will help alleviate the cognitive overload inherent in other ways of displaying increasing amounts of sensory data [6]. Rendering at the surgeon’s natural sensor array, his hand, seems the most straight forward. We want to produce a temperature and pulse feedback display with high reliability. To do so, it is essential to first extensively analyze the psychophysics of human perception for these two sensorial modalities. Thermal feedback has not been investigated as deeply as other tactile sensations. It has already been integrated to haptic devices [7] but the main trend is to achieve object discrimination using thermal feedback alone [8]. This approach is very challenging as humans need objects with significantly different thermal characteristics to be able to discriminate among them [9]. Gi-Hun Yang, Dong-Soo Kwon (KAIST, Korea) and Lynette Jones (MIT, USA) studied the ability to differentiate
two materials using a multi-fingered thermal display [10]. As a result of this work, the increase of the number of stimulated fingers increases the discrimination ability of the user, due to the thermal spatial summation property of the skin. Nevertheless, poor discrimination percentages when presenting virtual objects with slightly similar thermal properties were also observed. Precise temperature control requires complex models to reproduce the thermal exchange between the skin and an object with the display. For these reasons we are tackling the situation differently, considering temperature as a vector of information, thus hoping to replace the complex realistic feedback by a good knowledge of the psychophysics of human temperature perception and the related illusions [11] in order to render an intuitive thermal feedback. III.
Figure 1: Tactile Temperature Display with Micropump
DEVICE DESCRIPTION
This device features Peltier Elements that transfer heat between their surfaces proportionally to the applied current as described in:
Qco l d ==
s; x
Tco l d X 1-
1
2 x s; x I
2
-
«; x I1T (1) Figure 2: Four Peltier Elements aligned on the water cooling block
Where Qcold is the absorbed heat on the cold side, 1 the current applied, Tcold is the temperature of the cold surface, Sm and R; are the thermal and electrical resistance of the Peltier respectively, and I1T is the temperature difference between the two surfaces. These elements are water cooled on one side in order to maintain a precise temperature control of the other (exposed) surface. There are four "thermal units", refer to fig. 1, consisting of a Peltier element, a thermocouple (Type K) and a copper plate, aligned on the water cooling block. Peltier elements selection and arrangement Peltiers elements have been improving in the last decades. In the eighties they provided a power density of 5 W/cm 2 . This device features four KSAH018 Peltier elements (Komatsu Electronics KELK Ltd, Japan) with a power density of 20 W/cm 2 . This remarkable improvement allows a very small element size of 3.8 x 4.8 W x L mm for a height of 1 mm. There are several possibilities to arrange the Peltiers. A tradeoff between the optimal quantity for finger contact and the complexity of the cable routing is required. Furthermore, there is little information on the number of different temperature stimuli that we are able to discriminate on the same finger. The following reasons led us to adopt a design with four aligned Peltier illustrated in fig. 2, which still presents an high spatial resolution compared to any up to date device, but limits the cable routing and keeps the device small and easy to assemble on any force feedback device.
A.
Temperature sensor selection Two kind of temperature sensors were taken into account, mainly due to their small size: thermocouples and active sensors. Thermocouples provide temperature measure without any supply. However, they provide a relative measure depending on the temperature difference between the hot and cold junctions. Thus, an additional absolute temperature sensor is required. Active sensors are very precise, but need powering. Furthermore, they can be subject to self-heating problems which disqualifies them for precise temperature control. The retained solution for this device was the K-Type NickelchromiumlNickel-aluminum thermocouple (LABFACILITY) with a temperature range going from -75° to +250°C. B.
C. Cooling System
A heat disperser keeping the unexposed side of the Peltier at a constant temperature is essential for an optimal control. In fact, when cooling one side of the Peltier, the other heats up, and if the heat is not evacuated the entire element starts heating, and the control becomes impossible. Two solutions were retained for the heat evacuation, air cooling and liquid cooling. Air cooling can be achieved through natural or forced convection. Forced convection increases the heat exchange allowing a compact heat sink and more precise control. Equation 2 defines the Nusselt Number (NUl) for forced convection, which is the ratio between convective heat transfer and conductive heat
Liquid cooling presents a better heat exchange capability
compared to air cooling. Nevertheless, they require a closed loop system, with a pump and two pipes, inlet and outlet. The micropump (TCS Micropumps, England) able to provide a flow rate of 0.8 l/min and 0.2 bar pressure was used. By varying the voltage from 2 to 4 V, the flow can be proportionally controlled between 0.5 and 0.8 l/min. Leakage and corrosion, absent when using air-cooling, are important issues when a liquid is used. For this reason, water was preferred to other liquids. Furthermore, the use of small pipes limited us to a laminar flow, thus limiting the convection compared to the air design. In this case too several designs were tested, trying to increase the thermal exchange surface. The final design is presented in fig. 4. The input and output of the water cooling are represented in green. Seals between the inlets and the main body ensure a leak free connection. The main cavity is not empty, it is separated in two by a wall, allowing the water to follow a specific route and thus reducing the disturbances and increasing the heat exchange surface and time.
Figure 3: Air cooling, first design on the left and final on the right
D. Electronics Figure 4: Final water cooling design
transfer. Therefore, the higher the NUl the stronger the heat transfer through convection. NUL
hL
== -Il == Fl(Ra)F2(Pr) t
(2)
Where h is the convective heat transfer coefficient, L the characteristic length, kf the thermal conductivity of the fluid, and finally the Reynolds and Prandtl numbers (Ra and Pr) characterizing the flow type and the fluid state respectively. Forced convection is higher for water rather than air, and for a turbulent flow than for a laminar one. For this setup we tested forced convection using compressed air at different pressures. Special attention was given to the design of the exchange chamber. Fig. 3 illustrates a first design where the air was flowing horizontally through the heat sink. Due to its high pressure, the air would not expand to occupy the exchange volume, thus different masks meant to spread the airflow were produced but with little success. The final design has air blown directly from under the heat sink as one can observe in fig. 3. This design forces a maximal contact with the heat sink before the air can leave the device and generates a turbulent flow increasing the heat convection. Different apertures and air pressures were tested.
An electronic board was designed to control the device (see fig. 5). It is made of a DSPIC microcontroller, responsible for the acquisition of the temperature of each thermocouple, the communication with a computer and the generation of the PWM signals used to control the Peltier elements. Four SI9986 Hbridges are used to drive the Peltier modules providing the necessary tension and thus current in both directions (cooling and heating of the exposed Peltier surface). They are commanded with the microcontroller PWM, and output a 6 V signal at a frequency of 115 kHz. The output is filtered through a LC circuit in order to feed a DC signal to the Peltier. Four AD595AQ chips are used to amplify and linearize the thermocouples output current. The output of this chip is then filtered with a RC filter to remove a 50 Hz component. Setup Fig. 6 shows the entire setup. It consists in a 6V and up to 9A power supply (TRACO POWER, Switzerland), the PCB described earlier, the water tank with the submersible water pump, and finally the tactile temperature display. The PCB is connected to the PC through a DART serial connection.
E.
Control Individual proportional voltage control is used for the four Peltier elements using the thermocouples' temperature information as feedback.
F.
temperature varies, to the variations in room temperature, and finally to the less stable behavior of the pump around its minimal voltage input. Air cooling appears to be unable to keep the temperature constant, and increasing the aperture and the pressure does not solve the problem. It is interesting to notice that for positive currents, when the Peltier should be cooling the heat sink, we observe positive temperatures. This is due to the higher resistance of the Peltier to positive currents, see fig. 8, generating self-heating and to the incapacity of the air system to evacuate enough heat. According to these results we have chosen the water cooling system to evacuate heat from the Peltier.
Figure 5: Electronics Board
Figure 6: The setup including the PCB, the water tank, the power source and the display
IV.
B. Peltier control Assessment Fig. 9 illustrates the response of the Peltier element to a command for a step in temperature. Measurements were carried out using the setup described in Chapter III F. The exposed surface of the Peltier was left untouched, thus there was not a user’s finger on the display. One can notice that over a thirty degrees step both the heating and cooling rates decrease, which is normal since the Peltiers behave as a first order system. The cooling rate is significantly inferior to the warming rate since the system equilibrium is closer to cold temperatures. The maximum heating rate recorded was 18.7 °C/sec, the maximum cooling rate of 14.4 °C/sec. The rise and decaying time, see fig. 9, are 3.3 and 9.2 seconds respectively. These times depend on the characteristics of the Peltier, but also to the cooling system which can be easily improved. In fact, the pump is not yet working at maximum flow and the inlet and outlet of the displaying chamber present an important resistance to the flow. In fact, they were designed with an external thread in order to be easily assembled to the chamber and replaced. For this reason they have a significantly smaller internal diameter than the pipe system.
RESULTS
A. Liquid Cooling preliminary Assessment A test was run to compare the two cooling systems. The evaluation was based on the capacity of the cooling system to keep one Peltier side (in contact with the cooling system) at a constant temperature when varying the current through the Peltier, thus its temperature. Measurements were performed for different pressures (0.025, 0.05 and 0.1 Mpa) and inlet apertures (1 and 1.4 mm) for the air-cooling, and different water flows for the water cooling (corresponding to 2, 3 and 4 V as command to the pump). The results can be observed in fig. 7. For the water cooler, measured data shows it can maintain the Peltier surface at a constant temperature, independently of the injected current. For low pump voltage one can notice a high standard deviation, around five degrees. This is probably due to the fact the water for the experiment is recycled, thus its
Figure 7: Measurement of the heat sink temperature for different cooling methods with air inlet aperture 1mm and 1.4 mm
VI.
CONCLUSIONS, OPEN QUESTIONS, FUTURE WORK
A device for temperature rendering has been presented. It is based on Peltier devices, but as opposed to previous designs, it relies on a closed loop fluid circuit with a small tank not too far from the hand. It has been shown that the temperature rendering device is operational as intended. Psychophysical studies are now under way to refine the device, many questions remain open: How many different temperature rendering “pixels” are necessary, how many are possible, what spatial separation is needed? What temperature ranges and what minimal discernible differences can be expected? How large is the person-to-person variability of these data? The full integration has not yet been done, this is among the work foreseen for the future. ACKNOWLEDGMENT Figure 8: The resistance of the Peltier varies with its temperature the current supplied
This research was supported by the Swiss National Science Foundation through the National Centre of Competence in Research Robotics, and partially by the ARAKNES FP-7 project. Special thanks to Fabien Delaloye for the work effectuated in the frame of his master thesis. REFERENCES [1]
Figure 9: System response for step command
V.
ORIGINALITY
This device allows the individual control of every thermal unit. It is thus possible to generate gradients of temperature under the fingertip. Furthermore, it is the first device featuring such a high spatial resolution, due to the small size of each thermal unit, allowing the study of so far uninvestigated thermal cues. The combination of Peltier devices and liquid cooling, as it is proposed here, is new in this context. In this way, there is low inertial mass at the haptic device itself. This is important as we aim at integrating multiple modes of haptic feedback, i.e. fuse force feedback, texture, palpation and temperature. The device allows in principle to combine temperature and palpation information by controlling also the pressure in the closed fluid circuit. A truly multimodal haptic device becomes thus possible.
Ballantyne, GH. Robotic surgery, telerobotic surgery, telepresence, and telementoring . Journal of Surgical Endoscopy. 2002 [2] Hagn, U.; Konietschke, R.; Tobergte, A.; Nickl, M.; Jörg, S.; Kuebler, B.; Passig, G.; Gröger, M.; Fröhlich, F.; Seibold, U.; Le-Tien, L.; AlbuSchäffer, A.; Nothelfer, A.; Hacker, F.; Grebenstein, M.; Hirzinger, G. (2009) DLR MiroSurge - A Versatile System for Research in Endoscopic Telesurgery. In: International Journal of Computer Assisted Radiology and Surgery, 2010. [3] Takahashi, H. and Yonemura, T. and Sugita, N. and Mitsuishi, M. and Sora, S. and Morita, A. and Mochizuki, R. Master manipulator with higher operability designed for micro neuro surgical system. IEEE International Conference on Robotics and Automation, 2008. ICRA 2008. [4] MJ Giummarra, S.J.Gibson, N.Georgiou-Karistianis, J.L.Bradshaw. Mechanisms underlying embodiment, disembodiment and loss of embodiment. Neuroscience Biobehavior Review 32:143–160 (2008). [5] Botvinick M, Cohen J (1998) Rubber hands /`feel/' touch that eyes see. Nature 391:756-756. [6] Ng, J.Y.C. and Man, J.C.F. and Fels, S. and Dumont, G. and Ansermino, J.M. An evaluation of a vibro-tactile display prototype for physiological monitoring (article), Anesthesia \& Analgesia. 2005 [7] M. Bergamasco, A.A. Alessi, and M. Calcara, "Thermal Feedback in Virtual Environments", presented at Presence, 1997, pp.617-629. [8] Akio Yamamoto, Hiroaki Yamamoto, Benjamin Cros, Hironori Hashimoto, and Toshiro Higuchi "Thermal Tactile Presentation Based on Prediction of Contact Temperature" Journal of Robotics and Mechatronics,. 2006 [9] L.A. Jones and M. Berris, "The Psychophysics of Temperature Perception and Thermal-Interface Design", in Proc. Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2002, pp.137-137. [10] G.Hun Yang, L.A. Jones, and D. Soo Kwon, “Use of Simulated Thermal Cues for Material Discrimination and Identification with a Multi-Fingered Display,” Presence, Vol. 17, No. 1, February 2008, pp.29–42 [11] Craig, A.D., and Bushnell, M.C. (1994). The thermal grill illusion: Unmasking the burn of cold pain. Science 265, pp.252–255.
