Abstract
Noninvasive brain stimulation has the potential to revolutionize the way we interact with virtual environments by inducing tactile sensations. In this study, we conducted a virtual hand illusion experiment to investigate how the induced embodiment in virtual reality using brain stimulation differs from that using traditional haptic devices. The haptic glove was used to provide vibrotactile feedback, while the noninvasive brain stimulation was achieved through transcranial magnetic stimulation (TMS) to elicit hand movement and sensation. We compared the proprioceptive drifts resulting from the changes in ownership and body representation. Both the TMS and vibrotactile methods showed a similar tendency of moving toward the virtual hand, indicating the successful induction of the illusion of embodiment. The drifts were 21.9 mm and 30.7 mm for the vibrotactile and the TMS-evoked perception, respectively. Questionnaire results about ownership significantly favored the synchronous condition of visual and haptic sensations compared to the asynchronous condition. Our results demonstrate the feasibility of using TMS-based virtual reality to enhance the illusion effect, offering new possibilities for improved interactions in virtual environments.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
1 Introduction
The field of virtual reality (VR) is fast expanding owing to the recent growth in consumer-grade VR devices and the increase in noncontact interactions caused by the COVID-19 pandemic in early 2020. Various applications, such as entertainment, education (Radianti et al. 2020; Xie et al. 2021), manufacturing (Choi et al. 2015), and rehabilitation with VR therapy (Lohse et al. 2014; Vourvopoulos et al. 2019; Heinrich et al. 2021) have been implemented in virtual environments. To provide a more immersive VR experience (Ventura et al. 2022), several studies have used a combination of visual sensations and other induced sensations (Bau 2012; Won et al. 2015; Kreimeier et al. 2019). For example, a performance metric with both tactile and visual feedback is better than one with only visual feedback in a task-related experiment (Kreimeier et al. 2019).
Botvinick and Cohen first reported the rubber hand Illusion (RHI) experiment in which people felt a rubber hand in front of them belonged to their own bodies after the rubber hand and their blinded hands were brushed synchronously (Botvinick and Cohen 1998). Recently, researchers have reported that the RHI can be induced with a virtual hand as a fake hand (Slater 2009; Sanchez-Vives et al. 2010; Yuan and Steed 2010; Ma and Hommel 2013). The virtual hand illusion (VHI) in VR environments leads to the experience of embodiment in the virtual hand. There are three components of VR embodiment: (1) the feeling that the body belongs to the person (sense of ownership) (Kilteni et al. 2012), (2) the perception of being the agent of some behavior (sense of agency) (Kalckert and Henrik Ehrsson 2012), and (3) the feeling that one’s body is located at the same position as a virtual body (sense of self-location) (Jeannerod 2003; Serino et al. 2013). These three ideas are connected and also separate from one another. The sensation of ownership is related to the feeling of experiencing an external object, such as a rubber hand, as if it were a part of one’s own body. The sense of agency is experiencing the sensation of actively moving the object under one’s own control. The sense of localization is related to our ability to find ourselves and other objects in space, as well as the spatial sensation of being inside a body. The VHI can be induced by controlling the delay between the movement of the user with the data glove and the movement of the virtual hand on a screen, even if tactile stimulation is absent (Sanchez-Vives et al. 2010). A combination of visual and other sensations, referred to as “multisensory feedback”, is expected to provide enriched information (Gürkök and Nijholt 2012). The VHI with haptic feedback is more effective in inducing embodiment in VR. Haptic feedback is merged into the VR, which increases the realism of the simulation (Blake and Gurocak 2009).
The objective of this research is to evaluate the effectiveness of TMS-induced illusion in comparison to the illusion achieved with widely used haptic gloves. Our ultimate aim is to employ TMS to enhance the overall VR experience by providing additional sensory feedback. Our preliminary tests suggested that TMS can be effectively utilized in applications where both touch and proprioception are dominant, such as VR baseball hitting games. Haptic gloves deliver vibrotactile feedback, while TMS induces sensations like hand twitching, which are often described as sudden involuntary movement and are more related to proprioception than touch. Current proprioception-generating devices are bulky and inconvenient, posing challenges that are unlikely to be resolved in the near future. Therefore, we are exploring TMS as a viable alternative.
In this study, the effects of embodiment in a VHI experiment were compared using two different types of feedback. First, we used the wearable haptic glove which generates mechanical vibration to the skin surface. Second, a perception of the hand was induced using transcranial magnetic stimulation (TMS), which is a kind of noninvasive brain stimulation (Groppa et al. 2012). The former is an established approach for providing a more immersive experience in VR applications by allowing users to feel tactile sensations in hands while the latter is rarely tested. The latter, however, has great potential if it succeeds, as it provides the sensation without wearing anything in hands. Receptors related to somesthesia, including body touch, pain, and position of limbs, can be divided into cutaneous receptors and proprioceptors. The vibrotactile feedback from the haptic glove is received via cutaneous receptors for skin tactile sensation. However, the sensation of involuntary hand movement by TMS over the primary motor cortex is accommodated by the proprioception of the muscle spindle located in the corresponding muscle (Amassian et al. 1989). Therefore, we compare the effect of the combination of sensory and visual information by the other two receptors on the illusion.
Stimulation in the specific region of the brain is difficult because the brain is not exposed. Our robotized TMS with a neuronavigational system (Shin et al. 2023) made it possible to perform this task with high accuracy and repeatability. Thus, perception by TMS over the primary motor cortex in the VHI was possible (Bassolino et al. 2018), as shown in Fig. 1. Unlike vibrotactile gloves, which can only be applied to the hand, the controlled cortical excitability caused by TMS has a wide range of VR applications without being limited to a certain body part (Groppa et al. 2012).
To evaluate the embodiment of the VHI experiment, we measured a quantifiable drift and questionnaire. Proprioceptive drift is the relative difference in the perceived position of one’s own hand between the baseline and after the VHI. In the VHI experiment, the experimenter could freely construct a virtual hand intended to be placed at a certain offset distance from the real hand. However, the participant may not perceive the virtual hand as the experimenter intended because the virtual and real hands exist in different spaces. To determine how the participant recognized the configuration, we measured not only the perceived real hand position, but also the virtual hand position. A questionnaire about the sense of ownership and self-location was administered after the experiments. It was hypothesized that the synchronized combination of visual feedback and vibrotactile or perception induced by TMS would improve the embodiment of the virtual hand compared with the asynchronized condition. It was expected that perceived ownership and self-location would be significantly reduced in the asynchronous condition compared with the synchronous condition.
2 Methods
2.1 Participants
A crossover design was used with eight healthy male volunteers 27–32 years old (mean = 29.4, standard deviation (SD) = 1.9). The inclusion criteria included no history of neurological or psychiatric illness. All but one of the participants were right-handed (mean = 66, SD = 59.4, range: −40 to 100) as assessed by the Edinburgh Inventory (Oldfield 1971). The study was approved by the Institutional Review Board of Hanyang University Hospital (HYUIRB-202310-021) and performed in accordance with the principles of the Declaration of Helsinki. Informed consent was obtained from all the participants. All subjects were informed about the risks of the illusion or side effects of TMS and the monetary rewards of this experiment, and they completed consent forms before participating in the experiment. They were naïve with respect to the purpose of VHI, which had a single-blind design. The participants were unaware of the experiment’s goal and had never taken part in a RHI study before. To minimize the impact on experimental results, we did not disclose the purpose of the experiment to participants. While this approach allows participants to make intentional variations in their measurements based on personal expectations, we requested minimal deviation during repeated measurements. Additionally, we excluded instances with large deviations from the experimental results, as these could introduce significant bias, particularly if caused by intentions other than the experimental protocol. To avoid a carryover effect, the interval between the synchronous and asynchronous conditions was set to more than one week.
2.2 Materials and procedure
All participants were seated in front of a table with an armrest cushion for the right hand. The cushion was intentionally positioned to the left of the actual right hand, closer to the central body. There were two reasons for using this setup. First, the VR environment was configured to enhance the illusion effect. The strength of the illusion decreased when a rubber hand was placed far from the real hand (Costantini and Haggard 2007; Lloyd 2007; Preston 2013; Kalckert and Ehrsson 2014a), such as when the virtual hand was positioned to the right side of the actual hand. Second, it was possible to avoid the visual congruency in which the position of the virtual hand appears similar to that of the real hand before the illusion. The distance between the virtual and real hands must exist to validate the experience of the illusory feeling.
The participants observed an artificial environment similar to the real environment, including the right hand on the cushion through an HTC Vive Flow HMD device with a resolution of 1600 × 1600 pixels per eye and a field of view of 100°. To avoid collisions between the HMD device and exterior cover of the TMS coil, a compact Vive Flow HMD device was used, as shown in Fig. 2 (c). The VR scene was developed using Unity3D, and the details of the scene, as well as the experimental view, are shown in the Supplementary Video. Instructions on how to position the self-localization of the real or virtual hand with a measurement device during the VHI were displayed in VR scene. If the experimenter intervened in the VHI to explain the procedure, the strength of the illusion could be influenced by interference in the illusion (Schwind et al. 2019). Because there was no additional investigation after the experiment, the setup of the instruction was time-efficient and practical. When the participant performed the task according to the instructions, one could step into the next step independently. The experimental procedure is illustrated in Fig. 2.
Because a distance error between the virtual and real hands inevitably exists, the positions of the real and virtual hands were recorded using a measurement device before a combination of visual and stimulation step. In Fig. 2, the term “pre” represents the baseline of drifts. The proprioceptive drift was calculated by the difference between “post” and “pre”. The positions were recorded three times to obtain reliable measurements, as shown in Fig. 2 (a). In the traditional RHI experimental setup, covering the actual hand with a box or cloth was replaced in this experiment environment by naturally obscuring the real hand while wearing HMD device. During the combination of sensory information, visual and vibrotactile feedback or perception induced by TMS was generated, as depicted in Fig. 2 (d and e). The participants observed the movement of the index finger of the virtual hand that mimicked their own hand movement. They were instructed to gaze at the index finger in VR scene. Other sensory information included two types of feedback: vibrotactile feedback with a haptic glove and perception induced by TMS over the primary motor cortex. Unlike for the haptic glove, an initial preparation step was required to find the motor cortex corresponding to the index finger. The TMS-evoked perception is explained in detail in another subsection.
Each stimulation was repeated 12 times for approximately 2 min. The stimulation was randomly generated within an interval of less than 3 s to minimize the attenuation of the illusion. Unlike the manual stroking method, the programmable stimulation methods used can reduce the effect of synchronous delay between visual and other information because it avoids human error (Pritchard et al. 2016). Thus, these programmable methods result in improved experimental control. After the combination of sensory feedback, the two self-localizations were measured in the same manner as before.
2.3 Vibrotactile feedback
To generate tactile sensations in the VHI, we used an AvatarVR (Neurodigital technologies 2017) glove-type device, which is the most common and reliable vibrotactile feedback technology. The gloves with programmable low-latency linear resonance actuators (vibrotactile motors) provide interaction with the fingers according to the vibrotactile feedback. We stimulated the right index finger of the participants at the planning timing. We created a virtual pager and positioned it beneath the target finger to provide visual effects along with vibrotactile feedback. We provided feedback through color changes and slight shaking effects of the virtual pager according to each condition. Furthermore, participants progressed to the next stage of the VR scene through predefined gestures with the haptic glove.
2.4 TMS-evoked perception
When a single-pulse of TMS, a representative noninvasive brain stimulation, can generate hand movement by activating the primary motor cortex with appropriate intensity. Previous studies applying offline repetitive TMS over the left extrastriate body area (Wold et al. 2014) or inferior parietal lobule (Kammers et al. 2009b) modulated behavioral assessments of the RHI. In this study, we use the single-pulse TMS stimulation because it is to compare with vibrotactile using indirect sensory production rather than using the effect of brain plasticity through repeated brain stimulation. Perception of the hand by TMS was conducted using ALTMS (REMED, South Korea) connected to a figure-of-eight coil.
However, because the location of a hand in the motor cortex varies among individuals, the procedure to detect the region is time-consuming (Meincke et al. 2016) and imprecise. Even though the target was manually identified, illusion in the VR experiment decreases after a given length of time without automatic preparation by a robot. Moreover, if the participant’s head position changes during the experiment, it may deviate from the stimulation focus of TMS. However, to ensure the successful induction of TMS-evoked perception, it is crucial to remain within the stimulation point, namely the primary motor cortex.
To improve the repeatability and accuracy of the stimulation, a TMS positioning robot was developed in a previous study and transferred to a company, as shown in Fig. 3 (b) (Shin et al. 2019) (BSR-V1, PITCHLAB, South Korea). The robot can adjust the position and orientation of the coil above the human head and maintain the stimulation location through movement compensation. In order to find the optimal location for activation of right first dorsal interosseous (FDI), we detected the position of stimulation by moving the coil using the robot, starting at a location 5 cm forward and 1 cm laterally from the vertex of the head. Through automatic TMS coil positioning and retargeting, evoked hand movements can be manipulated efficiently. This article does not focus on the details of the positioning robot.
The position of the TMS coil was recorded with a reflective marker using a tracking system when the motor-evoked potential (MEP) amplitude in the electromyography (EMG) sensor attached to the right FDI muscle was \(\:\ge\:\)50 \(\:{\upmu\:}\text{V}\) (Groppa et al. 2012) as demonstrated in Fig. 3 (b). The intensity of stimulation was set at 100% of the motor threshold (MT) during the VHI. The mean of intensity was 72% of the maximum output of the TMS stimulator. Each recorded location of the participants was repeatably available in the VHI using the robotic TMS. It was assumed that the primary motor cortex was stimulated during the VHI, the EMG data were not recorded, and the experimenter observed the hand movement visually.
For a general HMD device with a head strap, it is difficult to place the coil over the scalp because the strap and the exterior cover of the coil collide. The head strap provides difficult tangential contact between the scalp and the TMS coil, which is required for effective stimulation. Therefore, a compact HMD device without a head strap was used as depicted in Fig. 2 (c).
The flowchart and communication between VR scene and other devices were constructed as shown in Fig. 3 (a).
2.5 Virtual reality scene
A scene of the VHI based on Unity3D consisted of a virtual room with a table, monitor, virtual hand, and armrest cushion, which were displayed to the participant from a first-person perspective, as shown in Fig. 2 (b). All objects in the scene were generated with the same scale as the real environment. Especially, a virtual hand model similar to a realistic male hand in skin and shape was used, as shown in Fig. 4. All objects, except the virtual hand, were aligned with respect to the body of the participant (Preston 2013). Although the virtual hand approximately matched the real hand, the position of the virtual hand was not calibrated to prove the evidence that the participant perceived the virtual hand to be located in the middle line.
For visual feedback during the TMS condition, the movement of the virtual index finger was vertically animated. When applying the vibrotactile method to the subjects, we provided a visual effect where a virtual pager’s red light appeared under the index finger of the subject. The time interval between visual feedback and other sensory feedback (e.g., vibrotactile and TMS-evoked perception) was randomly alternated less than 1 s in the asynchronous condition. To minimize experimenter intervention, subjects received instructions directly through the VR scene.
2.6 Self-localization device
To assess self-localization in previous research, a ruler, keyboard, optical, or electromagnetic tracking system was typically used (Tsakiris and Haggard 2005; Kammers et al. 2009a; Sanchez-Vives et al. 2010; Kalckert and Ehrsson 2014b; Wold et al. 2014; Fuchs et al. 2016; Erro et al. 2018). In the existing method, the participants may be dependent on the VR environment during self-localization with the keyboard (Tsakiris 2010). When recording the location of the hand recognized by the change of the pointer in response to keyboard input, the accuracy may vary depending on the distance between the pointer and participant. The SDs of the traditional method were compared with those of the keyboard (Bassolino et al. 2018) and our developed localization method. The average SDs of the two methods were 10.28 and 3.94 mm. The results of the pilot experiment obtained using the device were more accurate than those obtained using the traditional method. In the case of an optical tracking system based on an additional marker, the marker could be occluded owing to the table or the body. The optical tracking system offers high accuracy in three-dimensional positioning, but it may not be robust against various sources of interference. However, our developed guided measurement device was suitable for robustly measuring the location under a table without occlusion. A self-localization device was developed to measure the drift that the participants perceived as the positions of their real or virtual index fingers before and after the VHI. By moving the handle of the device with the other hand, the right finger position was localized without visual information.
The device consists of a brushless motor with a relative encoder (Maxon motor, Switzerland), ball screw, and force/torque sensor (Robotus, South Korea), as shown in Fig. 5. The resolution of the device was approximately 0.01 mm using the ball screw and the encoder. Because the ball screw, which translates rotational motion to linear motion, inevitably experiences friction, the motion was compensated for using the force/torque sensor to control the device comfortably. We implemented a simple control loop to optimize the tradeoff between precision and comfortable movement with gains \(\:{K}_{d},{K}_{f}\), as shown in Fig. 5.
Utilizing instructional cues embedded in the VR environment, the position of each participant’s non-dominant hand was systematically measured through three repetitions. After each measurement, the handle autonomously returned to its initial position. To minimize measurement deviations, participants had the flexibility to choose their measurement times. Ensuring precision in capturing user intent, position values were recorded during periods of stable handle position and consistent force variations. The procedure about the device can also be found in the Supplementary Video.
The strength of the illusion decreased when a rubber hand was vertically placed far from the real hand (Kalckert and Ehrsson 2014a). The device was attached under the table to grasp and control the device easily using their left hands as shown in Fig. 2 (a) and Supplementary Video. Thus, they could intuitively localize their right index fingers and virtual index fingers where they perceived them with no need to attach additional markers.
In addition, it was not necessary to consider the problem of outlier removal for the data that were positioned by the participant through the limited range of the linear guide. The drift in the lateral direction can be validated more directly by using 1D information rather than 3D information.
2.7 Questionnaire
After the experiment, a questionnaire was conducted to explore the participants’ level of illusion. The questionnaire items were selected and modified from a standard nine-item questionnaire (Botvinick and Cohen 1998). Three items of the traditional questionnaire were skipped to concentrate on the other six questions regarding the illusion in VR. The statements are listed in Table 1.
Questions 1–3 are supposed to indicate “ownership” in the presence of the illusion. Question 1 (Q1) attests to an illusory localization of sense, and Question 2 (Q2) refers to a causal link between visual and vibrotactile or TMS-evoked perception. Question 3 (Q3) directly denotes the illusion of ownership. Questions 4–6 are considered “self-location” questions with the location of the body. The questionnaire was scored on a seven-point Likert scale ranging from − 3 (“strongly disagree”) to 3 (“strongly agree”).
Not only a description of the procedure but also the questionnaire was displayed in view of the VR monitor. To avoid interference from the illusion effect, how to evaluate the questionnaire score was explained before the experiment. Thus, the experimental procedure was more time-efficient and practical because an additional block to survey the questionnaire after the VHI experiment was skipped. The details can be found in the Supplementary Video.
2.8 Statistical analysis
Data were tested for normality using the Shapiro–Wilk test (p > 0.05). The difference in each variable drift between the synchronous and asynchronous conditions was analyzed using a paired t-test. We also compared the differences between the two stimulation methods using Welch’s t-test. For the paired t-test and Welch’s t-test, p < 0.05 was deemed significant. Possible correlations between the drifts and questionnaire scores were explored using Pearson’s product-moment correlation. All data were analyzed using RStudio software for R.
3 Results
3.1 Proprioceptive drift
The drift was analyzed in evaluating the illusion effect in the VHI experiments. To calculate the drift of the localization, two positions were measured in which the participants placed the handle of the self-localization device into their right and virtual index fingers themselves before and after. To account for any potential variations, we systematically logged the position three times during each trial, allowing us to obtain precise and reliable measurements.
Figure 6 shows the proprioceptive drifts for each condition. The averages of the proprioceptive drifts in the synchronous condition during the combination of visual and vibrotactile feedback and TMS-evoked perception were 21.9 mm and 30.74 mm, respectively. In the synchronous condition, each method exhibits deviations of 7.15 mm and 2.97 mm, respectively. A positive higher value indicates that the drift was toward the virtual hand from the real hand. This means that the illusion of ownership was induced by the VHI.
The dataset was validated for homogeneity of variance using the Shapiro-Wilk test, p > 0 0.05. A paired t-test was performed to investigate the difference in proprioceptive drift toward the virtual hand between the asynchronous and synchronous conditions with TMS-evoked perception and vibrotactile feedback, revealing significant differences, t = − 16.308, p = 0.0001 and t = − 4.2345, p = 0.0133. We conducted a one-way analysis of variance (ANOVA) to examine differences in proprioceptive drift among four groups, which revealed a difference F = 4.405, p = 0.05. The Bonferroni-corrected pairwise t-test comparisons revealed that vibrotactile and TMS-evoked perception showed similar values only in the synchronous condition group, p = 0.15.
3.2 Questionnaire
To provide diverse analyses based on each stimulus combination, we conducted a questionnaire with each participant immediately after the conclusion of every experiment. The subjective experience of the illusion was measured using a questionnaire, as shown in Figs. 7 and 8. The first three components are thought to correlate with the feeling of ownership, whereas the others indicate a feeling of self-location. The participants experienced a feeling of ownership after the synchronous condition with the vibrotactile feedback. The differences in ownership between the synchronous and asynchronous conditions through the vibrotactile and TMS-evoked perception were seen with p = 0.0399 and p = 0.0004, respectively. And, the feelings of self-location with the two stimulation methods (vibrotactile feedback and TMS-evoked perception) revealed no significant differences between the synchronous and asynchronous conditions (t = 1.9144, p = 0.099 and t = 1.8791, p = 0.0997 respectively). Pearson correlations between the responses to the questionnaire and drifts were also computed. For the TMS-evoked perception condition, the correlation between the score for Q2 and proprioceptive drift was significant (r = 0.895, p = 0.0005). The scores of Q3 was significantly correlated (r = 0.7431, p = 0.0138, respectively) with proprioceptive drift in the vibrotactile feedback condition.
4 Discussion & conclusion
VHI was induced using an HMD for visual stimulus and two modalities: a haptic glove and TMS brain stimulation for somatosensation. The illusion effects of vibrotactile feedback and TMS-evoked perception were compared. Both successfully led to the illusion effect. Repeated TMS-evoked perception is difficult to induce because of the technical challenge of positioning the coil continually at the desired brain region during the VHI experiment. This difficulty was resolved by adopting a robotic TMS. The illusion of embodiment in the VHI experiment was evaluated by measuring proprioceptive drift and through questionnaires about the sense of ownership. Using a custom self-localization device, the drift that participants perceived as the positions of their index finger or virtual index finger before and after the VHI was precisely measured. To maintain illusion in the VR environment, the questionnaire was included in the same VR environment. In this study, both the questionnaire and proprioceptive drift were used as measures of illusion. Previous research has analyzed the reliability of proprioceptive drift for measuring embodiment (Rohde et al. 2011). Rohde found the results that dissociation of proprioceptive drift and the feeling of ownership. The result showed that prolonged asynchronous stroking seems to interfere with visual-proprioceptive integration of the visual location of the rubber hand with the proprioceptive location on the own hand. In (Rohde et al. 2011), the analysis particularly focused on differences in changes between ownership and proprioceptive drift based on the frequency of drift measurements. However, proprioceptive drift is a measurement closely related to self-location, one of the elements of embodiment. In (Kilteni et al. 2012), the use of proprioceptive drift to measure body ownership can be seen as addressing two fundamental aspects of embodiment: self-location, which involves locating a body part; and body ownership, as the task requires localizing one’s own hand. Therefore, proprioceptive drift can be considered as a means to assess self-location, a component of embodiment.
In the synchronous condition, both stimulation methods showed a similar tendency for proprioceptive drift to move toward the virtual hand. The proprioceptive drift with TMS-evoked perception showed a larger value than that with vibrotactile feedback (p < 0.05). We performed a one-way ANOVA to investigate differences in proprioceptive drift across four groups, which indicated a significant difference (F = 4.405, p = 0.05). Bonferroni-corrected pairwise t-tests showed that vibrotactile and TMS-evoked perception were similar only in the synchronous condition group (p > 0.05). The results of the questionnaire on ownership yielded significantly higher values for the synchronous than for the asynchronous condition through the vibrotactile and TMS-evoked feedback (p < 0.05 and p < 0.0005, respectively), reconfirming that synchrony is a critical factor in inducing the illusion of embodiment. The experiment indicated that TMS-evoked sensation can be adopted in VR for better illusion through this brain illusion. While vibrotactile devices, such as haptic gloves, have been widely adopted in VR, the TMS-evoked sensation in VR has not been extensively examined. Because TMS shows a comparable illusion effect, it should be tested and evaluated further as a novel modality for a better illusion effect in VR. Therefore, further research and technical improvements are both needed, such as research on TMS-evoked perception corresponding to various body parts (e.g., the leg), the effect in VR when combined with other modalities (e.g., sound), and miniaturization of TMS devices (e.g., head-mounted TMS).
If we could develop a technology that enables us to feel tactile sensations in VR through brain stimulation without the need for wearing any haptic devices, it would offer several benefits. Firstly, it would eliminate the need for users to wear bulky and potentially uncomfortable haptic gloves or other devices, making the VR experience more natural and immersive. This would allow for greater freedom of movement and a more seamless integration between the virtual and physical world. Secondly, brain stimulation technology could offer a more precise and accurate simulation of touch sensations than haptic devices, which may be limited in their ability to replicate the complexity of real-world tactile experiences. Additionally, this technology could provide new opportunities for individuals who are unable to use haptic devices due to physical disabilities or limitations. It could also enable new forms of communication, entertainment, and therapeutic applications in VR that are not currently possible with haptic devices. Overall, the development of brain stimulation technology for tactile sensations in VR has the potential to revolutionize the way we interact with virtual environments and unlock new possibilities for human-machine interaction. However, significant research and development are needed to make this technology a reality.
Data availability
The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.
References
Amassian VE, Cracco RQ, Maccabee PJ (1989) A sense of movement elicited in paralyzed distal arm by focal magnetic coil stimulation of human motor cortex. Brain Res 479. https://doi.org/10.1016/0006-8993(89)91640-5
Bassolino M, Franza M, Bello Ruiz J et al (2018) Non-invasive brain stimulation of motor cortex induces embodiment when integrated with virtual reality feedback. Eur J Neurosci 47. https://doi.org/10.1111/ejn.13871
Bau O (2012) REVEL: Tactile Feedback Technology for Augmented reality. 31:1–11. https://doi.org/10.1145/2185520.2185585
Blake J, Gurocak HB (2009) Haptic glove with MR brakes for virtual reality. IEEE/ASME Trans. Mechatron. 14:606–615. https://doi.org/10.1109/TMECH.2008.2010934
Botvinick M, Cohen J (1998) Rubber hands feel touch that eyes see. Nature 391. https://doi.org/10.1038/35784
Choi S, Jung K, Noh S, Do (2015) Virtual reality applications in manufacturing industries: Past research, present findings, and future directions. https://doi.org/10.1177/1063293X14568814
Costantini M, Haggard P (2007) The rubber hand illusion: sensitivity and reference frame for body ownership. Conscious Cogn 16:229–240. https://doi.org/10.1016/j.concog.2007.01.001
Erro R, Marotta A, Tinazzi M et al (2018) Judging the position of the artificial hand induces a visual drift towards the real one during the rubber hand illusion. Sci Rep 2–11. https://doi.org/10.1038/s41598-018-20551-6
Fuchs X, Riemer M, Diers M et al (2016) Perceptual drifts of real and artificial limbs in the rubber hand illusion. Sci Rep 6:24362. https://doi.org/10.1038/srep24362
Groppa S, Oliviero A, Eisen A et al (2012) A practical guide to diagnostic transcranial magnetic stimulation: report of an IFCN committee. Clin Neurophysiol 123:858–882. https://doi.org/10.1016/j.clinph.2012.01.010
Gürkök H, Nijholt A (2012) Brain-computer interfaces for multimodal interaction: a survey and principles. Int J Hum Comput Interact 28. https://doi.org/10.1080/10447318.2011.582022
Heinrich C, Cook M, Langlotz T, Regenbrecht H (2021) My hands? Importance of personalised virtual hands in a neurorehabilitation scenario. Virtual Real 25:313–330. https://doi.org/10.1007/s10055-020-00456-4
Jeannerod M (2003) The mechanism of self-recognition in humans. Behav Brain Res 142:1–15. https://doi.org/10.1016/S0166-4328(02)00384-4
Kalckert A, Ehrsson HH (2014a) The spatial distance rule in the moving and classical rubber hand illusions. Conscious Cogn 30. https://doi.org/10.1016/j.concog.2014.08.022
Kalckert A, Ehrsson HH (2014b) The moving rubber hand illusion revisited: comparing movements and visuotactile stimulation to induce illusory ownership. Conscious Cogn 26:117–132. https://doi.org/10.1016/j.concog.2014.02.003
Kalckert A, Henrik Ehrsson H (2012) Moving a rubber hand that feels like your own: a dissociation of ownership and agency. Front Hum Neurosci. https://doi.org/10.3389/fnhum.2012.00040
Kammers MPM, de Vignemont F, Verhagen L, Dijkerman HC (2009a) The rubber hand illusion in action. Neuropsychologia 47:204–211. https://doi.org/10.1016/j.neuropsychologia.2008.07.028
Kammers MPM, Verhagen L, Dijkerman HC et al (2009b) Is this hand for real? Attenuation of the rubber hand illusion by transcranial magnetic stimulation over the inferior parietal lobule. J Cogn Neurosci 21:1311–1320. https://doi.org/10.1162/jocn.2009.21095
Kilteni K, Groten R, Slater M (2012) The sense of embodiment in virtual reality. Presence Teleoperators Virtual Environ 21:373–387. https://doi.org/10.1162/PRES_a_00124
Kreimeier J, Hammer S, Friedmann D et al (2019) Evaluation of different types of haptic feedback influencing the task-based presence and performance in virtual reality. In: Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments. ACM, New York, NY, USA, pp 289–298. https://doi.org/10.1145/3316782.3321536
Lloyd DM (2007) Spatial limits on referred touch to an alien limb may reflect boundaries of visuo-tactile peripersonal space surrounding the hand. 64:104–109. https://doi.org/10.1016/j.bandc.2006.09.013
Lohse KR, Hilderman CGE, Cheung KL et al (2014) Virtual reality therapy for adults Post-stroke: a systematic review and Meta-analysis Exploring virtual environments and Commercial games in Therapy. 9. https://doi.org/10.1371/journal.pone.0093318
Ma K, Hommel B (2013) The virtual-hand illusion: effects of impact and threat on perceived ownership and affective resonance. Front Psychol 4:1–6. https://doi.org/10.3389/fpsyg.2013.00604
Meincke J, Hewitt M, Batsikadze G, Liebetanz D (2016) Automated TMS hotspot-hunting using a closed loop threshold-based algorithm. NeuroImage 124:509–517. https://doi.org/10.1016/j.neuroimage.2015.09.013
Neurodigital technologies (2017) AvatarVR. https://www.neurodigital.es/
Oldfield RC (1971) The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9:97–113. https://doi.org/10.1016/0028-3932(71)90067-4
Preston C (2013) The role of distance from the body and distance from the real hand in ownership and disownership during the rubber hand illusion. Acta Psychol (Amst) 142:177–183. https://doi.org/10.1016/j.actpsy.2012.12.005
Pritchard SC, Zopf R, Polito V et al (2016) Non-hierarchical influence of visual form, touch, and position cues on embodiment, agency, and presence in virtual reality. Front Psychol 7. https://doi.org/10.3389/fpsyg.2016.01649
Radianti J, Majchrzak TA, Fromm J, Wohlgenannt I (2020) Computers & Education a systematic review of immersive virtual reality applications for higher education: design elements, lessons learned, and research agenda. Comput Educ 147:103778. https://doi.org/10.1016/j.compedu.2019.103778
Rohde M, Luca M, Ernst MO (2011) The rubber hand illusion: feeling of ownership and proprioceptive drift do not go hand in hand. PLoS ONE 6. https://doi.org/10.1371/journal.pone.0021659
Sanchez-Vives MV, Spanlang B, Frisoli A et al (2010) Virtual hand illusion induced by visuomotor correlations. PLoS ONE 5:1–6. https://doi.org/10.1371/journal.pone.0010381
Schwind V, Knierim P, Haas N, Henze N (2019) Using presence questionnaires in virtual reality. In: Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/3290605.3300590
Serino A, Alsmith A, Costantini M et al (2013) Bodily ownership and self-location: components of bodily self-consciousness. Conscious Cogn 22:1239–1252. https://doi.org/10.1016/j.concog.2013.08.013
Shin H, Ryu W, Cho S et al (2019) Development of a spherical positioning Robot and Neuro-Navigation System for Precise and repetitive non-invasive brain stimulation. Appl Sci 9:4561. https://doi.org/10.3390/app9214561
Shin H, Jeong H, Ryu W et al (2023) Robotic transcranial magnetic stimulation in the treatment of depression: a pilot study. Sci Rep 13:1–11. https://doi.org/10.1038/s41598-023-41044-1
Slater M (2009) Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philos Trans R Soc Lond B Biol Sci 364:3549–3557. https://doi.org/10.1098/rstb.2009.0138
Tsakiris M (2010) My body in the brain: a neurocognitive model of body-ownership. Neuropsychologia. 48:703–712. https://doi.org/10.1016/j.neuropsychologia.2009.09.034
Tsakiris M, Haggard P (2005) The rubber hand illusion revisited: visuotactile integration and self-attribution. J Exp Psychol Hum Percept Perform 31:80–91. https://doi.org/10.1037/0096-1523.31.1.80
Ventura S, Cebolla A, Latorre J et al (2022) The benchmark framework and exploratory study to investigate the feasibility of 360-degree video-based virtual reality to induce a full body illusion. Virtual Real 26:323–332. https://doi.org/10.1007/s10055-021-00567-6
Vourvopoulos A, Pardo OM, Lefebvre S (2019) Effects of a brain-computer interface with virtual reality (VR) Neurofeedback: a pilot study in chronic stroke patients. 13:1–17. https://doi.org/10.3389/fnhum.2019.00210
Wold A, Limanowski J, Walter H, Blankenburg F (2014) Proprioceptive drift in the rubber hand illusion is intensified following 1 HzTMS of the left EBA. Front Hum Neurosci 8:1–6. https://doi.org/10.3389/fnhum.2014.00390
Won AS, Bailenson J, Lee J, Lanier J (2015) Homuncular flexibility in virtual reality. J Comput Commun 20:241–259. https://doi.org/10.1111/jcc4.12107
Xie B, Liu H, Alghofaili R et al (2021) A review on virtual reality skill training applications. 2:1–19. https://doi.org/10.3389/frvir.2021.645153
Yuan Y, Steed A (2010) Is the rubber hand illusion induced by immersive virtual reality? In: Proceedings - IEEE Virtual Reality. https://doi.org/10.1109/VR.2010.5444807
Acknowledgements
This work was supported in part by the Commercialization Promotion Agency for R&D Outcomes (COMPA) and in part by the National Research Foundation of Korea (NRF) grant funded by the Ministry of Science and ICT (MSIT) (No. 2022-URE-08 and No. 2022R1A4A3033961) and in part by the research fund of Hanyang University, South Korea (HY-202400000000561).
Author information
Authors and Affiliations
Contributions
HS designed the project. HS, GL and JL performed the experiment and analyzed the data. HS wrote the paper in consultation with SL and JP. All authors discussed the results and commented on the manuscript.
Corresponding authors
Ethics declarations
Ethics approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. The study was approved by the Institutional Review Board of Hanyang University Hospital (HYUIRB-202310-021).
Conflict of interest
The authors declare no competing interests.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Supplementary Material 1
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Shin, H., Lee, J., Lee, G. et al. Effect of vibrotactile feedback and perception induced by transcranial magnetic stimulation over the primary motor cortex in virtual hand illusion. Virtual Reality 28, 159 (2024). https://doi.org/10.1007/s10055-024-01052-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10055-024-01052-6