Keywords

1 Introduction

In recent years, the performance of computers has improved and systems that interact with virtual reality (VR) are researched and developed. A Haptic interface is a device that lets a user touch, feel, and manipulate objects in the VR world created on the computers. When a user interacts with the virtual object using a haptic interface, the haptic interface measures the position and orientation of the end effector as input information from the user. The haptic interface also generates the force that is calculated by a physics simulation, spring-damper mode, etc. to the end effector. the user can feel the haptic sensation through the end effector. The stability range of the impedance (Z-width) is one of the most significant properties of a haptic interface. The wide stability range enables us to interact with a variety of virtual objects such as very soft objects, very stiff objects, etc. In the haptic control loop, the state of the user who operates the end effector influences the stability and fidelity of the system. The impedance of the user differs between when the user strongly grasps the end effector and when the user weakly grasps it. The change of grasping force causes the change of dynamics of the whole system. The viscosity of the user who strongly grasps end effector absorbs extra energy generated by the haptic interface that displays the very stiff virtual object. In addition, the effect of the dissipation of the energy is considered to be more effective if the viscosity of the device is sufficiently smaller than the viscosity of the user.

In order to measure a user state and feed back to a system, electroencephalogram, electromyogram, electrocardiograph, etc. have been presented. The human interfaces using them have also been researched and developed [1,2,3,4,5,6]. However, they have some problems of requiring a large-scale device to use them, usability, etc.

In this research, we aimed for the high definition haptic interface that measures the grasping force of the user and estimates the user impedance in real time. The system also controlled the coupling impedance between the haptic interface and the virtual object based on the grasping force of the user as biofeedback for both stability and fidelity.

2 Impedance Control Haptic Interface

An impedance control haptic interface measures the position and orientation of the end effector and displays the force and torque to the end effector. The user can feel haptic sensation through the end effector. Many such devices have been developed.

2.1 String Based Haptic Interface

A string based haptic interface SPIDAR-G (Fig. 1) is an impedance control haptic interface [7, 8]. The device consists of high precision coreless direct current (DC) motors with rotary encoder, pulleys, strings, grip, and frame. The DC motor with rotary encoder is installed in the frame. The eight strings are connected to the end effector, and each string is driven by DC motor with pulley. The device measures the length of strings with the rotary encoder and calculates the 6 degrees of freedom (DOF) of the position and orientation of the end effector. By controlling the tensions of all strings with the DC motor, the force and torque of 6DOF are displayed to the end effector. The user can feel haptic sensation by grasping the end effector.

Fig. 1.
figure 1

String based haptic interface SPIDAR-G (6 degrees of freedom, impedance control)

Full size image

2.2 Virtual Coupling

Colgate etc. [9] introduced virtual coupling, which is a connection between a haptic interface and a virtual object by virtual impedance (Fig. 2). The virtual coupling between an impedance haptic interface and a virtual object is given by the Eqs. (1, 2). Where f h (s) is the force to the end effector in frequency domain, f e (s) is the force to the object, v h (s) is the velocity of the end effector, and v e (s) is the velocity of the object. The coupling impedance Z c (s) consists of stiffness K and viscosity B. The property of the virtual coupling depends on the coupling impedance which enables the coupling between the haptic interface in the real world and the virtual object in the virtual world. In order to increase the fidelity of the operation, it is necessary to increase the coupling impedance. However, the coupling impedance is increased much higher, the passivity of the system is not maintained and the system becomes unstable. Because the virtual stiffness is not a passive element in the temporal and spatial discrete system by digital computer. It is important to increase the coupling impedance. However, we have to decrease the coupling impedance to maintain stability. Both fidelity and stability of haptic interface have been discussed in various viewpoints [10,11,12,13]. In this research, we aimed to achieve both fidelity and stability by controlling the coupling impedance dynamically based on biofeedback.

$$ f_{h} (s) = - f_{e} (s) = Z_{c} (s)(v_{e} (s) - v_{h} (s)) $$
(1)
$$ Z_{c} (s) = K/s + B $$
(2)
Fig. 2.
figure 2

Virtual coupling

Full size image

3 Proposed System

3.1 Stability of Haptic Interface

Focus on energy generated by the virtual stiffness used in the virtual coupling. The force calculated by the virtual stiffness is discretely increased and decreased by squeezing and releasing in the temporal and spatial discrete system. There is a difference between the charged energy and the discharged energy. The discharged energy is greater than the charged energy. It means that energy conservation low is not satisfied in the virtual coupling in the discrete system. The higher stiffness is displayed the more extra energy is generated. The extra generated energy is absorbed by the viscosity of haptic interface and the user. However, the amount of the extra generated energy is greater than the amount of consumption by the viscosity of haptic interface and the user, the whole system becomes active and can be unstable.

3.2 System Configuration

The higher coupling impedance provides the smaller error between the VR object and the end effector. It is also possible to display force and torque close to the real object and to directly operate the virtual object. However, if the impedance is always very high to increase the fidelity, the extra energy generated by the impedance and the system can be unstable.

We propose a method to control the coupling impedance according to the impedance of user that is estimated from the grasping force (Fig. 3) and a mapping between the grasping force and the coupling impedance (Fig. 4). If the grasping force is weak, the system reduces the coupling impedance within the range of maintaining interaction in VR world to achieve high stability. If the grasping force is strong, the system increases the coupling impedance within the range of maintaining stability in the system to achieve high fidelity.

Fig. 3.
figure 3

Dynamic virtual coupling with biofeedback

Full size image
Fig. 4.
figure 4

Relationship between grasping force and coupling impedance

Full size image

3.3 Prototype of End Effector

In order to design the end effector that allows measurement of the grasping force without deteriorating performance of haptic sensation, it must be lightweight, compact, and rigid. The end effector receives force not only from a user’s hand but also from the tension of all strings in SPIDAR-G. The proposed end effector needs to separate these two forces. We designed the new end effector for SPIDAR-G (Fig. 5). The end effector consists of a main plate, two sheet force sensors and two half sphere grips. The main plate is connected to all strings from DC motors in SPIDAR-G. Each force sensor on the center of the main plate only measures the vertical force from the half sphere grip. The new end effector measures the grasping force that is generated by user holding the both half sphere grips by using Eqs. (4, 5, 6, 7) where f sens0 and f sens1 are the sensor values of the force sensors on both sides of the main plate, f grasp0 and f grasp1 are the grasping forces to the end effector, F dispZ is the vertical force from strings to the main plate and f grasp is the grasping force.

$$ f_{sens0} = f_{grasp0} + F_{dispZ} $$
(3)
$$ f_{sens1} = f_{grasp1} - F_{dispZ} $$
(4)
$$ f_{grasp} = (f_{grasp0} + f_{grasp1} )/2 $$
(5)
$$ f_{grasp} = (f_{sens0} + f_{sens1} )/2 $$
(6)
Fig. 5.
figure 5

Prototype of the end effector

Full size image

The values of force sensors are converted into voltages by the analog amplifier circuit. The voltage is taken into the PC by the A/D converter on the haptic controller via USB2.0. The sensing frequency synchronize with the control frequency of the haptic interface.

4 Experiment

4.1 Measurement of Grasping Force

The grasping force of the user interacting with a virtual object in virtual world by the haptic interface SPIDAR-G was measured.

A rigid body of a sphere and a rigid surface were placed in the virtual world (Fig. 6). The virtual world was simulated by Open Dynamics Engine [14] that is a real-time physics simulator of rigid body. In this measurement, the simulation frequency was about 333 Hz and the update frequency of the haptic interface was 1 kHz. The coupling impedance that was reasonable value to interact with the virtual object stably was K = 1200 N/m and B = 20 N/ms−1 in this experiment. The subject was five adult males. The measurement was as follows:

  1. Step1.

    A subject moved the haptic cursor without coupling objects in free space.

  2. Step2.

    The subject moved the coupling object in free space.

  3. Step3.

    The subject moved the coupling object to the virtual surface.

Fig. 6.
figure 6

VR world

Full size image

The grasping force of the subject was measured at each step. The result is shown in the Table 1. The grasping force on the Table 1 is the average of the grasping force of 5 subjects during each step. The results show that the minimum grasping force was measured when the subject moved the haptic cursor without coupling objects in free space, the slightly increasing grasping force was measured when the subject moved the coupling object in free space, and the maximum grasping force that was much greater than the others was measured when the subject moved the coupling object to the surface. The state of the subject in the manipulation caused the change of the grasping force. The subjects adjusted their grasping force according to the situation. It was suggested that the grasping force, which is adjusted by capability of the user, enables biofeedback.

Table 1. Average of grasping force.
Full size table

4.2 Measurement of Coupling Impedance Range

In order to control the coupling impedance by the grasping force as biofeedback, the controllable range of coupling impedance was measured. The upper limit of the range was measured when a subject grasped maximum force and interacted with a virtual object stably. The lower limit of the range was measured when the end effector with weight fell down and became stable on the virtual surface. Because the lower limit of the impedance assures the maximum coupling impedance without grasp by a subject on the end effector and the interaction with a virtual object stably. The stability of the system is specially disturbed by the update frequency of the system. Especially in the VR world that is executed in a PC with non-real-time operating system, the update frequency is disturbed by computational load. In order to show that the proposed method is an effective method even if the update frequency of the VR world is changed, we measured impedance at the simulation update frequency 1 kHz, 500 Hz, and 333 Hz, respectively. The update frequency of the haptic interface was 1 kHz in this measurement.

Figure 7 shows experiment result of measurement of coupling impedance range. The upper limit of the impedance is indicated by a solid line and the lower limit of the impedance is indicated by a dotted line. At each update frequency in the VR world, increasing of viscosity B enables increasing of stiffness K, and increasing of viscosity B exceeding a certain value causes decreasing stiffness K sharply.

Fig. 7.
figure 7

Coupling stiffness range

Full size image

4.3 Measurement of Maximum Impedance by Biofeedback

Based on the coupling impedance range in Fig. 7, the relationship between the grasping force and the coupling stiffness was implemented. In this study, the viscosity coefficient was fixed at B = 5 N/ms−1 where the stiffness K was high at all the measured frequencies in Fig. 7. The relationship between the normalized grasping force and the coupling stiffness corresponded by the linear function shown in Fig. 8. The proposed system estimated the appropriate coupling impedance by using the coupling stiffness range in Fig. 7 and the relationship between the grasping force and coupling stiffness in Fig. 8. We examined whether the stiffness K in the proposed system is adjusted by biofeedback in the several control frequencies and it is possible to increase the stiffness K compared with the traditional system which the stiffness is fixed in. In the traditional system, the stiffness was fixed maximum stiffness that enabled interaction with the virtual object. In the proposed system, the stiffness was controlled by biofeedback. The other experiment setups were the same as the former experiment.

Fig. 8.
figure 8

Relationship between grasping force and coupling stiffness

Full size image

Table 2 shows the results that are the maximum impedance of the proposed system and the static impedance of the traditional system, and improvement rate between the proposed system and the traditional system. The result in Table 2 indicates that the proposed system achieves higher stiffness than the traditional system in all simulation update frequencies. The improvement rates in each control frequency are 16.7%, 21.0%, and 21.6%, respectively.

Table 2. Maximum stiffness and improvement rate.
Full size table

This is because the maximum stiffness in the traditional system is not necessarily the maximum stiffness that becomes stable while the virtual object contact on the virtual surface. The traditional system that uses the fixed coupling impedance reduces the maximum stiffness that allows user to stably interact with the virtual object at all times. The proposed system that uses the dynamic controlled coupling impedance based on biofeedback achieves the maximum stiffness higher than the traditional system in the all conditions. When the virtual object that is operated by a user touches the other virtual object, the grasping force of the user is increased. The increase of the grasping force is caused by the increase of the impedance of the user, which enables the maximum stiffness higher than the traditional system.

5 Conclusion

This paper addressed dynamic control of coupling impedance between the haptic interface and a virtual object by using biofeedback based on the grasping force of user improved both fidelity and stability with haptic interface SPIDAR-G. The end effector that enabled measurement of the grasping force of a user for SPIDAR-G was designed and implemented. The grasping force was measured at moving the haptic cursor without coupling object in free space, moving the coupling object in free space, and moving the coupling object to the surface. It was suggested that the grasping force, which is adjusted by capability of the user, enables biofeedback. The impedance range that was dynamically controlled was measured in several update frequencies. The dynamic control of the coupling stiffness, which proportioned the grasping force was implemented. It was found that the maximum stiffness of the virtual coupling improved by about 20%. The validity of biofeedback based on the grasping force has been shown by the development of the proposed system.

In the future, the control method of dynamic coupling such as viscosity coefficient, mapping coefficient also need to be discussed in more detail. To further verify the results, we will carry out subjective evaluation by users.