Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

The lips are region of human body that is haptically sensitive as well as fingertips. Lips have not only an acute sense in the intensity of touch but also a relatively high spatial resolution on the location of the touch; it has been reported that its two point discrimination threshold is about 2 mm [1]. This feature of lips suggest that they can be used as an interface based on tactile sensation.

One of typical examples that transmit information through tactile sensation is braille. It is a tactile coding system for blind and low vision, normally using cells of six dots to represent characters and marks. The distance between dots in a cell is usually from 2 mm to 3 mm, and blind people recognize the patterns of the cells relying on the sensitiveness of fingertips. We considered that, since the lips are sensitive similarly to fingertips, tactile presentation of information on lips.

In this study, we developed a novel haptic display for lips using piezo bimorph vibrators (Fig. 1). The interbal of vibrators was designed to be 2 mm based on two point discrimination threshold on the lips; the display was composed of sixteen vibrators considering the width of the lips. The piezo bimorph was employed as the actuator because it is small and simple compared with others such as vibration motor and voice coil motor. This paper describes about design, implementation, and evaluation of the device.

Fig. 1.
figure 1

This picture shows how to use the stimulation unit. Users apply the stimulation unit on one’s lips by having it in one’s hand. In this paper, they apply it on lower lip only.

Full size image

2 Related Works

Many researches on tactile displays have been conducted [2]. Some of them have focused on the haptic interaction using lips and the mouth. Iwata et al. developed “food texture display” [3]. This haptic display was capable of presenting biting force to the teeth. Hashimoto et al. developed “straw-like user interface” [4], that could present the vibration of straw on lips to represent the sensations of drinking. These researches were representing haptic stimulation based on actual food or drink. Samani et al. developed “Kissenger” [5], which was an interactive device that transmit the sensation of contact of kissing. They aimed to augment remote communication.

Various mechanisms to present tactile feedback have been investigated. One of common actuators is piezoelectric element. Poupyrev et al. implemented a tactile apparatus for small touch screens [6, 7]. Taking advantage of smallness of the piezo bimorph, they integrated it into handheld devices. Hayward et al. developed a tactile display that generate distributed lateral skin stretch; the display was constructed from 64 piezoelectric actuators packed into 12 mm \(\times \) 12 mm [8]. Ikei et al. developed a texture display comprising 50 vibratory pins arranged in a 2 mm pitch driven by piezoelectric actuators. Also, authors employed piezoelectric element for our device to attain higher resolution.

Another mechanism that can cause tactile sensation with simple structure is electrical stimulation on the skin. Kajimoto et al. developed a device that can present spatial pattern on the forehead using an array of electrode [9]. Tang et al. investigated electrical stimulation inside a mouth [10]. A drawback of electrical stimulation is that it frequently causes the sensation of pain. We considered that such artifact is undesirabile for our purpose.

3 Device and Control

As stated in Sect. 1, the device was designed based on the constraints opposed by human factors: tactile resolution and the size of the mouth. The interval of vibrators, of stimulation points, was determined to 2 mm considering that the two-point discrimination threshold of lips is approximately 2 mm [1]. Width of the stimulation unit was restricted by the width of the lips, which was about 35 mm, hence the number of stimulation points was decided on sixteen resulting the whole width of the device to 35 mm. A piezo bimorph was employed to meet the requirements on the size and vibration intensity.

An overview of the system is shown in Fig. 2(a). The system consists of the stimulation unit, a control circuit, a regulated power supply, and a PC. The stimulation unit was made by combining piezo bimorphs (BM15015-06HC, RS Inc.) using epoxy adhesive as shown in Fig. 2(b); the size of each piezo bimorphs was 1.5(width) \(\times \) 15(length) \(\times \) 0.6(thickness). It was confirmed beforehand that the vibrator could generate amplitude that is sensible by the lip even when the vibrator is in contact with the lip. Piezo bimorphs were connected to the driver circuit by a flat-ribbon cable.

Fig. 2.
figure 2

This is an overview of the system (a). It is mainly consists of a PC (1), a regulated power supply (2), a control board (3) and a stimulation unit (4). Stimulation unit is array of sixteen piezo bimorphs, bonded by epoxy adhesive. Piezo bimorphs are arranged in 2 mm pitch.

Full size image

The device was controlled by a PC through a microcomputer and a driver circuit (see Fig. 2). The microcomputer (mbed NXP LPC1768, NXP) receives commands from the PC and controls the status of the driver circuit. The driver circuit is composed of H-bridge circuits for bipolar operation; actually the circuit was implemented using a serial to parallel converter with push-pull outputs (HV57908, Microchip), and using two outputs as a pair formed each H-bridge circuit. The source voltage of the driver circuit was 60 volt; although a regulated power supply was used in out current implementation, it will be easy to replace it by battery with booster circuit because the power consumption of the bimorph cell is low (Fig. 3).

Fig. 3.
figure 3

This diagram shows the control flow of the our display. The control board receives a serial data from PC and drives piezo bimorphs of the stimulation unit.

Full size image

4 Evaluation of Tactile Perception

In this study, we conducted two basic tests to evaluate tactile perception using vibrotactile stimuli generated by our device. These tests were carried on to confirm the two-point discrimination threshold and the point localization on lips. At the two-point discrimination test, we evaluated that how much resolution users can identify when they are stimulated at the same time. On the other hand, we evaluated it when they are stimulated multiple points in sequence. Through these tests, we explored what information and how stimulation approach is efficient for the user on this system at the point localization test.

4.1 Common Test Procedure

First, we explained how to use this device to participants. They applied the stimulation unit to the lower lip holding by own hand (as shown in Fig. 1). Also, contact of the device to the lip was confirmed by checking that the user can feel the vibration of the piezo bimorphs of all channels. Next, we gave participants time for practice up to 5 min before starting the test. We stimulated and confirm the answer with participants. In addition, each piezo bimorphs were numbered from right to left in relation to the user as channel 1 to 15 for the convenience of explanation in this paper.

4.2 Two-Point Discrimination Threshold

The first test was conducted to confirm the two-point discrimination threshold on lips. The test procedure was based on constant method; stimulate one or two channels selected randomly from comparison channels and let subjects answer one-point or two-points.

Concrete procedure of this evaluation was as follows:

  1. 1.

    Stimulate the standard channel and comparison channel for 1 s (one of the comparison stimulus is same as standard one: that is one-point).

  2. 2.

    Answer, “Stimulus is one-point/two-point.”

  3. 3.

    Repeat 10 set for all of the comparison stimulus the step (1) to (2)

Stimulation range is from 0 mm (one-point) to 10 mm at intervals of 2 mm. Standard stimulus was Ch.7. And also, comparison stimuli were Ch.8 to Ch.12. We didn’t stimulate whole of lips supposing the actual use of this device. Also, we adopt three frequencies: 50, 100, 200 Hz. 50 Hz is the frequency that FA-II fire most frequently. 100 Hz is a resonance frequency of the piezo bimorph. 200 Hz is same as 50 Hz on FA-I. Five volunteers (four males and one female, in ages from 22 to 24) participated this test.

Result of this test is shown in Fig. 4. 75 % discrimination threshold was about 8 mm at minimum when vibrating frequency was 50 Hz. It was much the same at 100 Hz. The most effective frequency differed by every participant. We only told participants, “Please wear the stimulus unit, so that you can feel the vibration well”. We think that this caused a large individual difference in the test’s results.

Fig. 4.
figure 4

The result of the two-point discrimination test. A curve is psychometric function that fitting average of all participants’ answers to the cumulative distribution function. Stars are average of each stimulus, and error bar shows standard deviation of them. Vertical lines show PSE, and also dash lines show the JND.

Full size image

4.3 Point Localization

The second basic test was conducted to confirm the point localization on lips, how much correctness users can recognize the point of stimuli. To evaluate this perception, we used constant method; vibrate standard and comparison channels selected randomly in sequence and let subjects answer relative point of the comparison channel.

This test was carried out in the following procedure:

  1. 1.

    Stimulate the standard channel for 500 ms.

  2. 2.

    1 s interval.

  3. 3.

    Stimulate the comparison channel for 500 ms.

  4. 4.

    Answer, “Comparison is left/right/same.”

  5. 5.

    Repeat 10 sets for all of the comparison stimulus the step (1) to (4)

The frequency of vibration was fixed to 100 Hz, as it achieved relatively better results in the two-point discrimination test. We conducted this test for three standard stimuli, Ch.5, Ch.8, and Ch.12, to confirm the difference between center and side of lips. Comparison stimuli were 2 mm to 8 mm away from the right and left of the standard stimulus. We recorded the ratio of the answer “left.” In the computation, we treated the answer “same” as 0.5. Five volunteers (four males and one female, ranging in the age group 22 to 24) participated this test.

Result of this test is displayed in Fig. 5. The JND is around 3 mm at each condition. Participants were able to identify the small shift of stimuli point as compared to two-point discrimination. In the previous test, two-point discrimination threshold was about 6 mm. However, 75 % threshold value is around 3 mm in this test. We also confirmed the difference of this perception between center and end of lips. Results of the test displayed that the difference was not significant.

Fig. 5.
figure 5

The result of the point localization test. A curve is psychometric function that fits average of all participants’ answers to the cumulative distribution function. Stars are average of each stimulus, and error bar displays standard deviation. Vertical lines displays PSE, and dashed lines displays the JND.

Full size image

5 Vibrating Pattern Identification

This test was conducted to investigate the users’ identification capability for multiple vibration points on their lips. The aim of this test was to see how efficiently users can identify the stimulation patterns that are presented by our device. Vibration patterns were presented using three and four piezo bimorphs, i.e., 3 and 4 bit. In addition, this test was conducted in two conditions: simultaneous stimulation of multiple points (at the same time) or in sequences.

5.1 Simultaneous Stimulation

The test was carried out as follows:

  1. 1.

    Stimulate some channels that are randomly chosen from the prepared stimuli patterns for 1 s.

  2. 2.

    Participants marked answers on their answer sheet to denote where the stimulation point vibrated.

  3. 3.

    Repeat 10 sets for all the stimulus patterns from step (1) to (2)

In the 3-bit test, seven vibration patterns from a combination of three stimulation points were used except for all the off pattern points. Regarding the 4-bit test, however, four stimulation points make fifteen different vibrating patterns. We chose 10 patterns out of these fifteen and excluded some symmetrical ones (see Fig. 6). Channels 1, 8, and 15 were used for the 3-bit test, while channels 1, 6, 11, and 15 were used for the 4-bit test. These channels were selected so that the stimulation points were equally spaced. Six male volunteers and one female volunteer, ranging in the age group 22 to 24 years old participated in this test.

Fig. 6.
figure 6

This figure shows the vibrating patterns used in the pattern recognition test. Every participants answered which vibrator was ON, looking this figure.

Full size image

The result of the tests is displayed in Fig. 7. Regarding the 3-bit test (see Fig. 7(a)), average correct answer ratio was 87 % although considerable variation among the patterns was observed. For example, correct answer ratio of pattern 5 was 63 %. Average correct answer ratio of the 4-bit test was 67 %. The result suggests that the test was considerably difficult for the participants.

Fig. 7.
figure 7

This bar graph shows the results of the pattern identification test at 3bit(a) and 4bit(b). Errorbars show standard deviations.

Full size image

5.2 Stimulation in Sequence

Additionally, we conducted the test that vibrates piezo bimorphs in sequence instead of multiple point stimuli at the same time. The test procedure was same as the previous one. The duration of stimulation was 500 ms per channel. Only 4-bit test was conducted in this condition.

The result of this test is displayed in Fig. 8. All participants recorded the higher correct answer rate than the previous test. One of them answered perfectly to all the questions. T-test calculation displays significantly higher average correct answer ratio at 5 % significance level as compared to previous test.

Fig. 8.
figure 8

This bar graph displays the results of the pattern identification test with stimuli in sequence (orange bar) as compared to previous results (blue bar). The average correct answer ratio increased significantly (\(p=0.05\)). (Color figure online)

Full size image

6 Discussion

In the 2PD test, a large individual difference was observed. We think that this resulted in a large individual difference in the test’s results. In this paper, users placed the stimulation unit on their lips by holding it in their hand (Fig. 1). Additionally, we bonded sixteen piezo bimorphs with epoxy adhesive. This fabrication approach makes the stimulation unit rigid. Therefore, the system was not able to adapt to the shape of the lips. This led to difference in stimulation intensity between the channels. We thought these factors made it difficult to sense the multiple point stimuli correctly by the participants. In future works, these issues will be addressed and the stimulation unit will be made adaptable to the shape of the users’ lips.

We confirmed through the pattern recognition test that users can recognize the three piezo bimorph’s vibration at the same time with 87 % correct answer ratio. Users can recognize the vibrating piezo bimorphs in sequence when piezo bimorphs increases to four. Bitrates calculated from test conditions were 3 bit/s and 2 bit/s. During the 3-bit test in sequence, the duration of stimulation was 1 s. Therefore, the estimated bitrate was 3 bit/s. During the 4-bit test at the same time, the duration of stimulation was 500 ms per channel. Therefore, the estimated bit rate was 2 bit/s. In this test, we asked participants to recognize the pattern only. We did not limit information to characters, figures, and so on. If the presented information is limited, the recognition rate will rise. For example, users get a hint of some word by complementing it themselves if this system is used to present the word.

7 Conclusion and Future Works

In this study, we developed a novel tactile display using the piezo bimorph. We confirmed that the system is capable of representing the vibrotactile stimulation using the evaluation of two-point discrimination and point localization. This system can present some information on the three simulation points from the pattern identification results.

Future work is to make the stimulation unit easily and certainly wearable. It is heavy to hold it in one’s mouth because of the flat-ribbon cable. We can use both hands for any exertions by reducing its weight. Additionally, we aim to make the stimulation unit by substituting soft material for epoxy adhesive. This is because epoxy adhesive makes the stimulation unit rigid and it cannot be adapted according to the shape of lips.