Pattern identification and perceived stimulus quality as a ... - IEEE Xplore

IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, VOL. 11, NO. 1, MARCH 2003

9

Pattern Identification and Perceived Stimulus Quality as a Function of Stimulation Waveform on a Fingertip-Scanned Electrotactile Display Kurt A. Kaczmarek, Member, IEEE, and Steven J. Haase

Abstract—The effect of stimulation waveform on pattern perception was investigated on a 49-point fingertip-scanned electrotactile (electrocutaneous) display. Waveform variables burst frequency ( ), number of pulses per burst (NPB), and pulse repetition rate (PRR) were varied in a factorial design. Contrast reduction was used to limit performance of perceiving a 1-tactor gap defined within a 3 3 tactor outline square. All three variables accounted for significant variations in performance with higher levels of and NPB and lower levels of PRR, leading to better performance. In addition, we collected qualitative data on each waveform, and the qualitative differences were related to performance (e.g., waveforms perceived as having a more localized sensation were correlated with better pattern identification performance than those waveforms perceived as more broad). We also investigated the effect of stimulation contrast on pattern perception. Index Terms—Electrocutaneous, electrotactile, pattern identification, sensory substitution, tactile display.

I. INTRODUCTION

W

HILE THE USE of computers for information access and processing is increasing very rapidly, persons without useful vision may have difficulty accessing certain kinds of information [1], particularly graphics that do not lend themselves to textual descriptions presented via speech synthesis. Fortunately, much spatial information is readily acquired by the sense of touch, provided that the information is properly formatted and the user is suitably trained. For example, tactile maps, which often take the form of raised-line drawings on stiff paper or plastic, are used by some individuals who are blind for orientation to unfamiliar environments [2]. However, the options for presenting dynamic or updatable tactile information from electronic sources are rather limited at present. Arrays of mechanical pins that protrude from a flat surface and are scanned by the fingers [3], [4] are mechanically complex, power-hungry, and very expensive to manufacture in all but the smallest sizes. Efforts to use small, fingertip-sized arrays in conjunction with fingertip-position tracking [5] to emulate a large virtual array have yielded mixed results. Electrical stimulation of touch via a matrix of electrodes transducing small, controlled electric currents into the skin is

Manuscript received March 21, 2001; revised November 12, 2002. This work was supported by the National Eye Institute under Grant R01-EY10019. K. A. Kaczmarek is with the Department of Orthopedics and Rehabilitation Medicine and the Department of Biomedical Engineering, University of Wisconsin, Madison, WI 53706 USA. S. J. Haase is with the Department of Psychology, Shippensburg University, Shippensburg, PA 17257 USA. Digital Object Identifier 10.1109/TNSRE.2003.810421

called electrotactile, or electrocutaneous, stimulation [6] and has been shown useful for presenting pictorial and graphical information to the abdomen and other cutaneous loci [7]. Research in progress is attempting to use electrotactile display technology to present information to the fingertips, which is a somewhat more difficult problem because of the thick palmar skin [8]. Kaczmarek, Tyler, and Bach-y-Rita [9] demonstrated the ability of participants to accurately identify patterns on a 49-point fingertip-scanned electrotactile array. They also showed that performance was better in comparison to an electrostatic display [10], cf. [11], but not as good as performance on raised-line drawings. The major benefit of an electrotactile display over raised line drawings or similar methods is the dynamic capability of presenting several patterns on the display over a short period of time and the ability to adjust the magnification or other attributes of the display, if necessary. Such capabilities would appear very advantageous for a sensory aid that provides nonvisual access to computer-graphical information or for other applications, such as tactile stimulation of the fingertips in virtual and teleoperated environments. Electrical stimulation of touch produces a wide variety of percept qualities; participants describe these most frequently as tingle, vibration, pressure, pulsation, fizz, pinprick, and buzz [7]. This variety occurs because the electrotactile stimulation excites the primary afferent nerve fibers responsible for cutaneous sensation much differently than do mechanical stimuli, which generally act via specialized mechanoreceptors. The result is that, while the neural mechanisms and sensory correlates of at least elemental mechanical touch stimuli are fairly well characterized [12], the same is not true of electrotactile stimulation. In this case, even the primary afferent response is largely unknown [13]. One practical result of this lack of knowledge is that it has been difficult to generalize the relationship between electrotactile stimulus and the resulting perceptual response, which includes perceived intensity, pattern perception, and sensory quality of the sensation. While much attention has been devoted to the intensive attributes of the electrotactile percept [6], [14], the qualitative and spatial aspects have been largely ignored, with a few notable exceptions. Aiello [15], [16] demonstrated that changes in electrotactile perceived intensity are easier to perceive when stimulus waveform timing is constant and also hypothesized that the delivered energy determines perceived intensity. Kajimoto [17], [18] designed stimulus waveforms and electrodes specifically to elicit different qualities of sen-

1534-4320/03$17.00 © 2003 IEEE

10


sation attributable to the primary mechanoreceptor systems. Similarly, Poletto [19] designed stimulus waveforms that enhance the range of nonpainful stimulus currents by selectively suppressing the response of nociceptive afferent nerve fibers. However, none of these studies considered the effects of waveform timing on spatial pattern perception, which is the object of the present study. Specifically, we investigated how stimulus waveform timing (frequency and arrangement of trains of stimulus pulses) affects identification of small spatial electrotactile patterns scanned by the fingertips, as well as how it affects the perceived quality of the electrotactile stimulation. We also investigated how electrotactile contrast affects pattern identification. II. METHODS A. Preparation

Fig. 1. Electrode array (7

2 7) scanned by the fingertips of participants.

Prior to the start of each experimental session, all participants (Ps) provided informed consent and then washed their hands to remove any debris (e.g., dirt or hand lotion) that might interfere with electrotactile sensation. In addition, at regular intervals, such as in between sets of trials, the participant’s fingertip was cleaned with isopropyl alcohol (70%). The electrode array surface was cleaned with pure ethyl alcohol. Mineral oil was applied to the clean fingertip to improve the ease of scanning the electrode surface and to prevent skin dehydration, which increases skin resistance and can result in potentially interfering electrostatic forces [20], [21] at the electrode–skin interface. B. Electrotactile Stimuli For both experiments, Ps used the distal pad of the left middle finger to actively scan an electrode array consisting of a 7 7 arrangement of 0.89-mm diameter, flat-topped stainless steel electrodes, each surrounded by a 2.36-mm-diameter air gap insulator (Fig. 1). The electrodes were arranged on a square grid with a 2.54-mm interelectrode spacing. A flat stainless steel plate, the top of which was coplanar with the tops of the electrodes, served as the return current path. Ps were instructed to use the finger pad, avoiding both the tip of the finger and the region near the distal metacarpal joint. (Both of these regions are characterized by increased sensitivity to electrotactile stimulation, coupled with a somewhat less controllable percept.) The angle between finger and array surface thus varied from approximately 0 to 45 . Capacitively coupled, positive [22], current-controlled1 pulses (0–6.5 mA) were delivered to each electrode that corresponded to an active element in the displayed pattern; inactive electrodes were effectively disconnected. The waveform diagram in Fig. 2 displays one of the waveforms (waveform C) studied in this experiment. It shows that each active electrode received bursts of three 50- s pulses, where the pulse onsets were separated by 3.33 ms (pulse repetition rate, or PRR 300 Hz) and the burst onsets by 33.3 ms (burst 30 Hz). Only one electrode received current frequency, or at any time; electrodes were pulsed in a raster-scan format. 1The

effective output resistance of the constant-current stimulator was approximately 410 k , while the resistance of the electrode–skin interface was approximately 150–300 k .

Fig. 2. Stimulus waveform C for stimulation electrodes 1, 2, and 49; electrodes are pulsed sequentially in a raster-scan format. Frequency F = 30 Hz, NPB = 3, and PRR = 300 Hz. The pulsewidth for all electrodes is 50 s.

Activation of adjacently pulsed electrodes was staggered by 49 PRR, or in this case, 68.03 s, so that each electrode in the array could be pulsed once before the next pulse in each burst. The main variable of interest in this experiment was how various aspects of the electrical waveform timing might affect pattern perception. We manipulated three variables with two levels 20 or 40 Hz, each in the full factorial experimental design: NPB 2 or 4, and PRR 200 or 400 Hz. Fig. 3 schematically shows these eight waveforms, denoted 1–8, plus a “centered” waveform C, having variable values in the middle of the cube formed by the eight experimental waveforms. C. Participants Fourteen people initially participated in this experiment. Five (3M/2F, age 22–31) were not able to accurately perceive the patterns used in the experimental sessions and did not undertake those trials (two had prior experience and were successful at perceiving a different set of patterns). One participant (F, age 52) did not complete the experimental trials. The remaining eight Ps (2M/6F, age 20–26) contributed data for the main experiment (four had prior experience with electrotactile stimulation). D. Training After receiving familiarization with the nature of the electrotactile stimulation and the experimental apparatus, Ps were

KACZMAREK AND HAASE: PATTERN IDENTIFICATION AND PERCEIVED STIMULUS QUALITY

11

2

Fig. 4. Tactile patterns. The 3 3 outline square was used for waveform familiarization and percept quality ratings. The 3 3 and 4 4 outline squares with gaps and the lines with and without gap were used for training. The 3 3 outline square with gap was used for the pattern perception experiment. The lines were presented with vertical or horizontal orientation. The outline squares with gaps were presented with four orientations, with the gap at top, bottom, left, or right. Open circles represent electrodes with no stimulation or with background stimulation during trials with contrast reduction (main experiment and contrast experiment). Fig. 3. Schematic representation (approximate scale) of the experimental waveforms (1–8) studied, plus a “centered” waveform (C). Waveform variables are F : 20, 30, or 40 Hz; NPB: 2, 3, or 4; PRR: 200, 300, or 400 Hz.

given 12 practice trials of obtaining their sensation threshold ( , defined as a level of stimulation that could just barely be perceived) and four practice trials, obtaining their maximum level without discomfort ( , defined as the strongest level of stimulation that could be experienced without feeling any annoying or uncomfortable sensations). Measurements were taken on waveand form C on a 3 3 outline square pattern (Fig. 4). levels were determined by the method of adjustment whereby the participant adjusted a random offset (30%) knob to the level of intensity defined by the task. Ps were instructed to tweak the knob up or down to obtain the most precise measurement possible before settling on the final value for each trial. These levels were measured to familiarize Ps with electrotactile stimulation and to determine if they were able to produce a suitable dynamic range. In some cases, if the array’s maximum delivered intensity level was perceived as too weak, the participant’s fingertip was hydrated with distilled water for approximately 30 s before the mineral oil was applied. In most cases, this improved the perceived intensity of the stimulation. Ps next received training on identifying simple geometric patterns, also on waveform C. The first set of patterns consisted of 4 4 outline squares with two inactive electrodes on any one of the four sides of the square (Fig. 4). Ps attempted to identify the side with the gap as they freely adjusted the intensity of the pattern using a knob. The patterns were displayed for up to 30 s. After the trial was completed, the experimenter provided feedback on the accuracy of their response. After approximately 12 trials on these patterns, Ps also practiced identifying whether a “line” consisting of a linear array of five electrodes was oriented vertically or horizontally and whether it had an inactive electrode in the middle, third tactor

2

2

2

(i.e., a gap) or not, to teach Ps to identify a single tactor gap in a fairly simple setting (Fig. 4). Ps performed approximately 12 trials on these patterns, again with feedback from the experimenter. Next, Ps obtained practice with feedback on the patterns to be used in the experiment. These patterns were defined within a 3 3 outline square in the center of the 7 7 matrix, similar to the previously practiced 4 4 patterns. However, the “gap” in these patterns was defined by only one inactive electrode on any one of the four sides of the square (Fig. 4). In addition to practice with feedback, Ps ran through at least 12 trials without feedback to determine if they could perform well enough on the task to continue the experiment (at least 50% correct). Finally, Ps were given exposure to each of the eight different waveforms (using a 3 3 outline square) to observe how sensation quality might vary with changing waveform. Here, they were instructed to adjust the intensity to a level that was mod, at a strong enough level that erate (i.e., in between and would provide clear information about the outline square patterns). In addition, was measured three times for each waveform on the full field of 49 electrodes. E. Main Experiment: Pattern Identification and Percept Quality Each block of trials consisted of two practice trials followed by 32 experimental trials. Within each block of 32 experimental trials, Ps received an equal number of the four pattern types (gap-top, bottom, left, or right) and eight waveforms, sampled randomly without replacement. Ps were instructed to emphasize accuracy but that response times would be recorded. They were also informed that they would have a maximum time of 15 s to identify the pattern on each trial. The next sessions (approximately three) comprised the experimental trials. A total of 11 blocks of experimental data

12


were retained. To prevent floor and ceiling effects, blocks of trials where performance was below 40% correct or above 90% correct were omitted from the data analysis and thus were not included in the 11 blocks (with the exception of one participant who it was later realized achieved an average performance 94% in one block of trials). Stimulation current (which could be freely adjusted by participants during the experiment by turning a knob) was recorded at the end of each trial for blocks 9–11. Contrast Reduction: Because most of the eight experimental Ps eventually performed perfectly or near perfectly on identifying the experimental patterns from an earlier pilot study, we found it necessary to reduce performance by activating all nonpattern electrodes (“background electrodes”) to some intensity level below the active pattern electrodes (i.e., contrast reduction). For the experimental trials, the level of background electrodes was set to some proportion of the pattern electrodes so that overall performance within a block of trials would be between 40% and 90%. A slight programming error concerned how the background intensity was calculated for the contrast reduction. For each block of trials, as mentioned, the background current was intended to be a fixed percentage of the current for electrodes defining the pattern. (Recall that Ps could adjust current during trials.) However, the actual percentage varied slightly among the different waveforms. Fortunately, since we recorded the actual current at the end of each trial for blocks 9–11, the actual contrast could be calculated retroactively, and these results are also reported. Sensory Quality: After Ps had completed the first eight blocks of trials within the aforementioned performance range, they were asked to rate each waveform on a number of qualitative dimensions. For this part of the study, a 3 3 outline square (Fig. 4) was presented with no contrast reduction. Ps were instructed to adjust the intensity of the pattern to a level similar to that if they were trying to identify spatial aspects of the pattern. The computer then presented five Likert scale questions in a random order for each of the eight waveforms. Ps had as much time as they needed to rate each question. Waveforms were selected at random, and upon the completion of the five questions, a new waveform was selected. Once each waveform had been rated, the program repeated the process twice more so that each question had three ratings for each waveform. The resulting data were averaged across replications. The five rating scales were Pulsing/Flutter (1)–Buzzing/Vibratory (7) Sharp/Prickly (1)–Dull (7) Broad (1)–Localized (7) Well-defined/Clear (1)–Diffuse/Fuzzy (7) Tingly (1)–Not Tingly (7)

III. RESULTS A. Pattern Identification Fig. 5 summarizes the pattern performance, response time, and qualitative rating data, with the waveforms sorted in order of increasing pattern performance. The bottom panel displays

Fig. 5. Combined-results graph. Bottom to top: Waveform variables, pattern perception, response time, and three percept quality scales. Abscissa is ordered by increasing pattern performance. Error bars are 1 SEM.

6

the average performance levels for identifying the patterns as a function of each waveform. Error bars indicate 1 SEM. Chance performance is 0.25. The relationship between this bottom panel and those above it will be discussed shortly. Fig. 6 displays performance in terms of the levels of the facto, rial design. All main effects were significant: ; the proportion of variance explained ( ) from the , , ANOVA 0.29; NPB ; PRR , , . There was also a significant, but rather weak, interaction between NPB , , . The pattern of and PRR the interaction was such that the effect of PRR is less at higher pulses per burst. We found this effect somewhat interesting in that PRR does not affect the aggregate pulse rate (NPB ) over


13

TABLE I STIMULUS-RESPONSE (S-R) MATRIX FOR PATTERN PERFORMANCE. CELL ENTRIES ARE RESPONSE PROPORTIONS. MAJOR DIAGONAL INDICATES ACCURACY (HIT RATES) FOR EACH PATTERN (E.G., RESPOND TOP GIVEN TOP). BOTTOM ROW INDICATES THE PROBABILITY WITH WHICH EACH RESPONSE CATEGORY WAS SELECTED. d IS AN UNBIASED MEASURE OF PERCEPTUAL SENSITIVITY. IS A MEASURE OF RESPONSE BIAS (HIGHER VALUES INDICATE A TENDENCY TO USE THE RESPONSE CATEGORY LESS)

Fig. 6. Pattern perception as a function of factorial waveform manipulations.

a long period of time (i.e., multiples of the interburst period, or ), as do and NPB.2 We also examined whether there were differences in accuracy as a function of pattern type. For this analysis, we computed the “hit” (e.g., (“gap-left” gap left) and “false alarm” (e.g., (“gap-left” gap right, top, or bottom) rates for each pattern and transformed these into the bias-free measure of discriminability ( ) as was done in similar identification studies [23], [24]. There was a significant effect of gap location on discrim, . A Newman–Keuls post inability, hoc test revealed that the gap-bottom pattern was perceived less accurately than the gap-left and gap-top patterns. Table I displays performance by pattern. An analysis of the signal detection bias measure ( , the ratio of the “hit” to “false alarm” ordinate values from the normal density functions) was also conducted. The analysis revealed only a marginal effect of bias, , . The gap-bottom and gap-left patterns show more conservative bias (less of a tendency to select these patterns on incorrect trials) than the gap-right and gap-top patterns. These results indicate slight differences in the perceptibility of individual patterns but do not affect the general conclusions concerning waveform effects. 2However, when we attempted to isolate this effect by presenting three waveforms varying only in PRR (F was held constant at very close to 20 Hz and NPB constant at 2), there were no significant differences in average performance among the waveforms (200 Hz PRR = 0:714, 300 Hz PRR = 0:75, 400 Hz PRR = 0:675). Perhaps in a more isolated situation, presenting three waveforms that all tend to produce lowered spatial performance, Ps were able to compensate by engaging in more effortful or purposeful scanning, etc. Another possibility is that the difference in PRR at NPB = 2 might only be obtained under lower performance levels. In other words, our attempted replication of this NPB PRR interaction may have failed because the overall level of performance on the waveforms was higher (with averages all near 0.70), compared with the original experiment, where collapsed performance across the 200- and 400-Hz PRR waveforms (at 20 Hz and NPB = 2) was 0.61. If this line of reasoning can be accepted, then the effect may be somewhat limited in scope. Finally, considering the two levels of PRR used in the original experiment, the 95% confidence interval (CI) for the difference between the performance means perf (200 Hz PRR) perf(400 Hz PRR) = 0:097 to 0.176. Thus, the observed difference in the original experiment of 0.123 between 200 Hz PRR and 400 Hz PRR falls within this interval. The 95% CI from the original experiment of perf(200 Hz PRR) perf(400 Hz PRR) = 0:014 to 0.232. While zero is not within this interval, indicating that there is a statistical difference between these means that, in the original experiment, the observed mean difference in the replication of 0.039 falls within this last CI. In sum, there may be a slight effect of PRR, especially at low frequency and pulse repetition rate. The effect merits further investigation, perhaps under a situation using different, more complex spatial patterns.

2

0

0

0

B. Response Time In addition to collecting accuracy data on each trial, we also collected response time (RT) data. Ps were instructed to identify the pattern on each trial as accurately as possible, but if they felt confident that they had identified the pattern, they could respond before the trial deadline, which was set at 15 s. The RT analysis was performed on trials where the identification response was accurate. The fourth panel of Fig. 5 displays the average of median response times3 for each waveform. In terms of the factorial ANOVA, all main effects were significant as in the analysis of accuracy data: burst frequency, , , ; NPB , , ; PRR , , . In addition, there was a marginally sigPRR interaction, , , nificant , qualified by a marginal NPB PRR interac, , . The interaction tion, was such that responding to low PRR waveforms was faster at high frequencies but not the lowest values of and NPB (i.e., waveforms 1 and 2 had very nearly the same median RT, perhaps because of the response deadline of 15 s—a difference between these two patterns could be obscured). The pattern of results in terms of RT was very similar to that of identification, especially regarding the main effects. The waveforms responded to most accurately were identified more quickly in comparison to wave, , forms that were identified less accurately, . In other words, there was no apparent tradeoff between accuracy and speed. C. Sensory Quality For the “pulsing–buzzing” dimension, there was a significant , ; and main effect of frequency, , . Higher and NPB were NPB rated as more buzzing/vibratory. There was also a significant NPB PRR interaction, , . The 3The RT distributions were negatively skewed because of the 15-s time response time limit. Therefore, for each waveform, the median response time was computed for each P . The means of these values are then plotted in Fig. 5.

14


Fig. 7. Typical scatterplot of pattern performance versus percept quality: Broad–localized scale. Each datum represents one participant’s average responses (for both performance and for percept quality) for one waveform.

interaction was such that the buzziness rating was higher (average 4.6) at the lower PRR than at the higher PRR (average 2.9), but only at 20 Hz and 4 pulses/burst (e.g., waveform 3 versus waveform 4). Buzziness ratings as a function of PRR did not differ markedly at any other or NPB levels. For the “broad–localized” dimension, there was a significant , ; and main effect of frequency, , . Higher frequency and NPB NPB waveforms were rated as more localized. For the “clear–fuzzy” dimension, there was a significant main , . Higher NPB waveeffect of NPB forms were rated as more clear. There was a marginal effect of , ; and a marginal NPB frequency, PRR interaction, , . Higher frequencies were rated as more clear; lower PRR were rated as more clear, except at lower NPB, where there was less of a difference of PRR. The “Sharp/Prickly–Dull” and “Tingly–Not Tingly” ratings did not show any clear pattern of differences. One question of interest is whether the qualitative ratings (collected independently from the accuracy data) bear any relationship to identification performance. The top three panels of Fig. 5 show the average rating for each waveform for three of the rating scales. Fig. 7 shows one typical scatterplot of the average rating value (in this case, broad versus localized) by average proportion correct identification. For the three curves in Fig. 5, all of the correlations between qualitative rating and values pattern identification performance (derived from the from the scatterplots similar to Fig. 7) are significant at . In general, the pattern of qualitative data confirm some of the key differences among waveforms. The interesting result is that waveforms rated as more vibratory, clear, and localized yielded higher pattern identification performance. These waveforms tended to have high and high NPB. D. Contrast As mentioned previously, the actual contrast values varied slightly among the waveforms within a block of trials. This could potentially confound the pattern identification versus waveform results. To illustrate, Fig. 8 plots the test current (relative to sensation threshold), and also the contrast

Fig. 8. Average contrast by waveform. Pattern current (relative to sensation threshold I ) is also displayed. Abscissa is ordered by increasing pattern performance as in Fig. 5.

(test/background). Ps tended to select somewhat higher currents for waveforms having a lower aggregate pulse rate, perhaps to compensate for the lower overall stimulation energy. However, in all cases, Ps were free to choose the level they felt was best for maximizing performance. Because of the software error, the contrast varied slightly (1.92, waveform 2; 1.99, waveform 7). We therefore wanted to determine if this contrast difference could possibly account for the difference in pattern ; Waveform identification accuracy [Waveform 2, ], even though the correlation between contrast 7, ). Thus, we and performance was very small ( conducted a follow-up study, very similar to the main factorial waveform experiment. In each block of trials, contrast varied randomly across five levels (generally from 1.11 to 3.33). The levels were chosen individually for each participant to maintain an overall performance level similar to that in the original experiment (approximately 70% correct). Waveform C was used on all trials. Again, Ps adjusted the intensity of the pattern to their preferred level with instructions to maximize accuracy. Each block consisted of 40 trials (eight at each contrast level). A total of four blocks of trials were run. Fig. 9 displays data from seven Ps (six out of seven completed the eight-waveform experiment; one completed a pilot version of the eight-waveform experiment). Each data point is based on average performance from three to seven Ps. (Not all Ps were tested at each contrast level, since Ps differed in their ability to identify patterns as a function of contrast. As mentioned previously, contrast was adjusted for each participant so that average performance across all contrast levels would be near 70% correct.) The slope of the steepest portion of this function (i.e., from contrast 1.11 to 1.67) is 0.94. Therefore, a contrast change from 1.92 to 1.99 (the range recorded in the main experiment) could only explain a 6.6% change in pattern performance, based on this worst-case (steepest slope) analysis. This is a much smaller effect than observed in the main experiment between waveform 2 and waveform 7 ( 30%). We conclude that the observed effect of waveform on performance is mediated primarily by factors other than contrast. Another potentially interesting finding from this contrast study is that the “contrast threshold” can be estimated. That is, if one takes 0.55 as the threshold performance level in a four-alternative forced-choice task (i.e., corresponding to


Fig. 9. Performance-contrast function. The regression line is for only the first three data points, with contrast varying from 1.11 to 1.67 (see text). Error bars are 1 SEM.

6

an identification of 1.00), the contrast level resulting in approximately this level of performance is 1.43.

IV. DISCUSSION The primary goal of this study was to determine the effect of stimulus waveform timing on perception of small spatial pat, which terns. Fig. 5 shows that aggregate pulse rate (NPB is 40 pulses/s for waveforms 1–2, 80 pulses/s for waveforms 3–6, and 160 pulses/s for waveforms 7–8) may be a primary determinant of pattern identification. The specific arrangement of pulses (reflected in PRR) is of secondary importance. There are several possible explanations for this finding. 1) Waveforms with higher values of and NPB feel somewhat more intense (at equal currents) and also have somewhat lower sensory and pain thresholds [25]–[29]. Since stimulus current was adjustable, participants tended to compensate by choosing somewhat higher currents for waveforms with lower and NPB (Fig. 8). However, this compensation may be incomplete; we did not attempt to equalize the perceived intensity of the eight test waveforms. Therefore, the observed performance difference might be mediated by perceived intensity. We consider this unlikely, however, considering previous results showing that the relationship between small pattern identification performance and stimulation current is rather shallow [30]. 2) As mentioned previously, the contrast varied slightly among waveforms. While this unintended effect would appear to have minimal effect on performance (Fig. 9 and related discussion), it raises another important issue, namely, the calculation of the background current. Electrotactile stimulation, unlike other sensory modalities, yields sensory thresholds that are a significant fraction of the maximal level without discomfort [6]. It is therefore possible, when background current is computed as a fixed fraction of pattern current, for the background current to drop below threshold, while the pattern current remains suprathreshold. Therefore, the perceived contrast might vary with pattern current, even if the stimulus contrast (pattern current/background current) remains constant. This also seems an unlikely explanation for the present results, because participants are generally adept at choosing stimulus

15

levels that maximize their performance. However, this area certainly warrants further study. 3) If we consider perception of a patterns presented on pulsed electrodes as a sampled-data system, waveforms with higher and NPB provide more bits of information per second than waveforms with low values of these variables, i.e., the observed effect may be a sampling-rate effect. If we consider even a relatively slow finger scanning speed of 5 cm/s, the distance traversed by the finger between successive bursts of stimulation at a low frequency (20 Hz) is 2.5 mm, approximately the distance between electrodes in our array. In essence, as the finger moves over the electrode array, electrodes might be perceptually displaced or entirely missed in between pulses. Consistent with our results, other studies (e.g., [31]) have found improved pattern identification performance at higher frequencies on vibrotactile stimulator arrays. We therefore consider this the most likely explanation for our observed results, at least within the context of the very simple patterns used in the present study. A secondary goal of this study was to determine if readily describable qualitative attributes of the electrotactile percept correlated with either the stimulus waveform timing, the pattern perception performance, or both. Fig. 5 answers this question affirmatively for the three qualitative scales shown (the other two did not correlate with any of the waveform variables or with pattern performance). It would appear that participants are capable of making accurate judgments about the spatial attributes (clear–fuzzy, broad–localized) of various waveforms and that these judgments are consistent with the expectation that clear, localized percepts will yield the most accurate pattern perception. The significance of the “pulsing–buzzing” dimension is somewhat less clear, at least in its relationship to pattern performance. With reference to the waveform variables, however, it makes sense that a higher aggregate pulse rate would allow individual pulses (barely perceivable on the “pulsing” end of the scale) to fuse into a continuous sensation (on the “buzzing” end of the scale). The origin and significance of the big peak on waveform 6 is unknown. Adequate description and explanation of the relationship between electrotactile waveform timing and sensory quality awaits further research. It is perhaps of consequence that 5 out of the 14 participants initially recruited were unable to adequately perceive the orientation of the small spatial patterns. There are known differences in the abilities of participants to perceive (and learn to perceive) vibrating tactile patterns [32]. We intentionally chose pattern and contrast levels that made the orientation task difficult in order to avoid floor and ceiling effects and, for practical reasons, did not use larger patterns or offer extended training times, both of which typically improve performance. This restricted our study to participants who easily perceived the patterns with minimal training. It remains an open question whether individuals who do not easily perceive spatial tactile patterns will respond similarly to waveform and contrast manipulations. Finally, we should note that the effects of stimulus waveform timing and contrast in this study were measured for only one kind of spatial pattern—one that is very simple but apparently challenges the spatial resolving power of the tactile sense for this kind of tactile display. We cannot be certain exactly how

16


these parameters will affect perception of more complex patterns, particularly in the presence of time and attention constraints imposed by practical use of this technology. However, we have recently shown that the superiority of waveforms with a higher aggregate number of pulses per second also applies for a digit identification task [33], [34]. Experiments investigating the perception of other kinds of pictographic information, such as scatterplots [33], [34], cf. [35], are in progress. ACKNOWLEDGMENT The authors would like to thank M. E. Tyler at the University of Wisconsin—Madison for assistance in instrumentation development. REFERENCES [1] L. H. Boyd, W. L. Boyd, and G. C. Vanderheiden, “The graphical user interface crisis: Danger and opportunity,” Trace R&D Center, University of Wisconsin—Madison, Sept. 1990. [2] R. L. Welsh and B. B. Blasch, Eds., Foundations of Orientation and Mobility, 2nd ed. New York: American Foundation for the Blind, 1997. [3] S. F. Frisken-Gibson, P. Bach-y-Rita, W. J. Tompkins, and J. G. Webster, “A 64-solenoid, four-level fingertip search display for the blind,” IEEE Trans. Biomed. Eng., vol. BME-34, pp. 963–965, Dec. 1987. [4] M. Shinohara, Y. Shimizu, and A. Mochizuki, “Three-dimensional tactile display for the blind,” IEEE Trans. Rehab. Eng., vol. 6, pp. 249–256, Sept. 1998. [5] S. F. Wiker, G. Vanderheiden, S. Lee, and S. Arndt, “Development of tactile mice for blind access to computers: Importance of stimulation locus, object size, and vibrotactile display resolution,” in Proc. Hum. Fact. Soc. 35th Annu. Meeting, 1991, pp. 708–712. [6] K. A. Kaczmarek, J. G. Webster, P. Bach-y-Rita, and W. J. Tompkins, “Electrotactile and vibrotactile displays for sensory substitution systems,” IEEE Trans. Biomed. Eng., vol. 38, pp. 1–16, Jan. 1991. [7] K. A. Kaczmarek and P. Bach-y-Rita, “Tactile displays,” in Virtual Environments and Advanced Interface Design, W. Barfield and T. Furness, Eds. New York: Oxford Univ. Press, 1995, pp. 349–414. [8] R. D. Melen and J. D. Meindl, “Electrocutaneous stimulation in a reading aid for the blind,” IEEE Trans. Biomed. Eng., vol. BME-18, pp. 1–3, Jan. 1971. [9] K. A. Kaczmarek, M. E. Tyler, and P. Bach-y-Rita, “Pattern identification on a fingertip-scanned electrotactile display,” in Proc. 19th Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., Chicago, IL, 1997, pp. 1694–1697. [10] R. M. Strong, “An explorable electrotactile display,” Ph.D. dissertation, Massachusetts Institute of Technology, Cambridge, MA, 1969. [11] H. Tang and D. J. Beebe, “A microfabricated electrostatic haptic display for persons with visual impairments,” IEEE Trans. Rehab. Eng., vol. 6, pp. 241–248, Sept. 1998. [12] K. O. Johnson and S. S. Hsiao, “Neural mechanisms of tactual form and texture perception,” Annu. Rev. Neuroscience, vol. 15, pp. 227–250, 1992. [13] K. A. Kaczmarek, M. E. Tyler, A. J. Brisben, and K. O. Johnson, “The afferent neural response to electrotactile stimuli: Preliminary results,” IEEE Trans. Rehab. Eng., vol. 8, pp. 268–270, June 2000. [14] A. Y. J. Szeto and R. R. Riso, “Sensory feedback using electrical stimulation of the tactile sense,” in Rehabilitation Engineering, R. V. Smith and J. H. Leslie, Jr., Eds. Boca Raton, FL: CRC, 1990, pp. 29–78. [15] G. L. Aiello, “Multidimensional electrocutaneous stimulation,” IEEE Trans. Rehab. Eng., vol. 6, pp. 95–101, Mar. 1998. , “Tactile colors in artificial sensory communication,” in Proc. Int. [16] Symp. Info. Theory Appl., Mexico City, 1998, pp. 82–85. [17] H. Kajimoto, N. Kawakami, T. Maeda, and S. Tachi, “Tactile feeling display using functional electrical stimulation,” in Ninth Annu. Conf. Artificial Reality Telexistence, Tokyo, Japan, 1999. [18] , “Electrocutaneous display as an interface to a virtual tactile world,” in Proc. 2001 IEEE Virtual Reality Conf., Yokohama, Japan, 2001, pp. 289–290.

[19] C. J. Poletto, “Fingertip electrocutaneous stimulation through small electrodes,” Ph.D. dissertation, Case Western Reserve University, 2000. [20] S. Grimnes, “Electrovibration, cutaneous sensation of microampere current,” Acta Phys. Scand., vol. 118, pp. 19–25, 1983. [21] R. M. Strong and D. E. Troxel, “An electrotactile display,” IEEE Trans. Man–Mach. Sys., vol. MMS-11, pp. 72–79, 1970. [22] K. A. Kaczmarek, M. E. Tyler, and P. Bach-y-Rita, “Electrotactile haptic display on the fingertips: Preliminary results,” in Proc. 16th Annu. Int. Conf. IEEE Eng. Med. Biol. Soc.. Baltimore, MD, 1994, pp. 940–941. [23] D. W. Massaro, “Perceptual units in speech recognition,” J. Exp. Psych., vol. 102, pp. 199–208, 1974. [24] J. M. Loomis, “Analysis of tactile and visual confusion matrices,” Percep. Psych., vol. 31, pp. 41–52, 1982. [25] R. H. Gibson, “Electrical stimulation of pain and touch,” in The Skin Senses, D. R. Kenshalo, Ed. Springfield, IL: Charles C. Thomas, 1968, pp. 223–260. [26] K. A. Kaczmarek, “Optimal electrotactile stimulation waveforms for human information display,” Ph.D. dissertation, Dept. Electrical and Computer Engineering, Univ. of Wisconsin—Madison, 1991. [27] G. B. Rollman, “Electrocutaneous stimulation,” in Proc. Conf. Cutan. Comm. Sys. Dev., F. A. Geldard, Ed., 1973, Psychonomic Soc., pp. 38–51. [28] A. Y. J. Szeto, “Relationship between pulse rate and pulse width for a constant-intensity level of electrocutaneous stimulation,” Ann. Biomed. Eng., vol. 13, pp. 373–383, 1985. [29] C. Van Doren, “Contours of equal perceived amplitude and equal perceived frequency for electrocutaneous stimuli,” Percept. Psychophys., vol. 59, pp. 613–622, 1997. [30] K. A. Kaczmarek and S. J. Haase, “Pattern identification as a function of stimulation current on a fingertip-scanned electrotactile display,” IEEE Trans. Neural Sys. Rehab. Eng., to be published. [31] C. H. Rogers, “Choice of stimulator frequency for tactile arrays,” IEEE Trans. Man–Mach. Sys., vol. MMS-11, pp. 5–11, 1970. [32] R. W. Cholewiak and A. A. Collins, “Individual differences in the vibrotactile perception of a ‘simple’ pattern set,” Percept. Psychoph., vol. 59, pp. 850–866, 1997. [33] S. J. Haase and K. A. Kaczmarek, “Perception of correlation via electrotactile displays,” presented at the 43rd Annu. Meeting Psychonom. Soc, Kansas City, MO, 2002. , “Electrotactile perception of scatterplots on the fingertips and ab[34] domen,” in IEEE Trans. Biomed. Eng., submitted for publication. [35] J. H. Flowers, D. C. Buhman, and K. D. Turnage, “Cross-modal equivalence of visual and auditory scatterplots for exploring bivariate data samples,” Hum. Factors, vol. 39, pp. 341–351, 1997.

Kurt A. Kaczmarek (S’82–M’82) received the B.S. degree from the University of Illinois—Urbana in 1982 and the M.S. and Ph.D. degrees from the University of Wisconsin, Madison, in 1984 and 1991, respectively (all in electrical engineering). From 1984 to 1986, he was a Senior Engineer with Baxter International. He is now a Senior Scientist at the University of Wisconsin, Madison, where he has studied the mechanisms and perception of electrical stimulation of touch since 1992. His interests include tactile displays, sensory rehabilitation and augmentation, teleoperation, and virtual environments.

Steven J. Haase received the B.S. degree from the University of Illinois—Urbana in 1987 and the M.S. and Ph.D. degrees from the University of Wisconsin, Madison, in 1990 and 1994, respectively (all in psychology). From 1994 to 1999, he was Assistant Professor of psychology, Gordon College, Barnesville, GA. From 1999 to 2002, he was a Researcher at the University of Wisconsin, Madison, and has been Assistant Professor of psychology at Shippensburg University, Shippensburg, PA, since 2002 . His interests include visual and tactile perception, human–machine interfaces, pattern recognition, theoretical modeling of psychological processes, attention and human information processing, and sensory substitution.