Health Education VR

Abstract

We developed a full-body virtual-presence system in which users crawl to travel within the human body, and gain insight into the structures of digestive organs through visual, tactile, and auditory sensations. Digestive organs (esophagus, stomach, duodenum, small intestine, large intestine, rectum) are created in detail with 3-D images that users could look around using a head-mounted display, and an appropriate feeling of touch is presented to the abdomens and thighs of users by vibrotactile devices. Pressure sensors detect the crawling action of users so that they can travel in the digestive organs while hearing realistic 3-D sounds. Participants felt as if their body turned into digested food and was moving inside another person’s body. Participants also felt as if their bodies became smaller. Thus, this system gives users the sensation of a reduced body size and provides a telepresence at improbable locations such as the stomach. Participants enjoyed the experience and were interested in the human body structure and function. This system can serve as an interactive, educational experience by simulating the contents of the human digestive system.

1 Introduction

Health education is important to gain necessary knowledge about one’s health and make decisions about health problems. In particular, understanding one’s own body structure is the basis for obtaining various kinds of health knowledge. However, the inner human body, such as the digestive organs, is usually a mysterious space for ordinary people. We can neither see nor touch digestive organs in daily life if we are not physicians. Learning about digestive organs through textbooks may not be interesting to most people because the organs cannot be easily visualized. Anatomical models of the human body have been used to visualize organs. Recently, virtual reality (VR) contents that provide experience and opportunities of looking around from inside the human body have been provided. With these VR contents, you can travel and look around inside the kidneys [1]. You can also get inside of Björk’s mouth when she is singing [2]. In addition, an augmented reality (AR) system was developed to help visualize the anatomical organs and bones of an actual human body [3]. Using the system, people can view the outer surface of organs such as the lungs, heart, and stomach. These VR and AR systems involve an active and interactive experience, but they are limited to visual experience and serve as a realistic image-based anatomical model of the human body. People cannot touch organs. To understand a living and moving body, however, it will be effective to experience it not only through visual but also tactile sensation. In this study, we aimed to develop a full-body telepresence system in which users crawl to travel within the human body and study structures of digestive organs by visual, tactile, and auditory sensations.

Synesthesia Suit [4] and Teslasuit [5] used methods of presenting tactile stimuli to the whole body. In this study, a similar method of presenting multiple oscillators by manipulating the time difference and intensity difference was used, but the tactile stimuli were simply presented using only four oscillators. Users looked around the inside of the digestive tract, which was 3-D modeled on a head-mounted display (HMD), and traveled the human body by crawling. Vibration stimuli were presented to the abdomen and thighs, and a heartbeat sound was presented according to the user’s current position inside the body. Crawling was used because we thought that active movement would increase user enjoyment. We expect that experiencing the process of being digested in the body as food would help us to understand the digestive function as our affairs. We also expected that the experience of combining visual, auditory, and tactile sensations would enhance reality and lead the good learning effect about the inner human body.

2 System Overview

The developed system consisted of a head-mounted display (HMD; Oculus DK2, 960 (width) × 1024 (height) pixel for each eye, 90 × 110 degrees of visual angle), four vibrotactile transducers (AcouveLab Vp408), a set of headphones (Bose quiet comfort 2), a computer (Dell XPS8900), a USB pre-amplifier (BEHRINGER FCA1616), a power amplifier (BEHRINGER EPQ304), four pressure sensors (Interlink electronics FSR408), and a microcontroller (Arduino Uno Rev3) (see Fig. 1). The computer (DELL XPS8900, Core i7-6700 2.4 GHz, 16 GB RAM, GPU AMD Radeon R9-370) controlled the entire system with software developed using Unity 5.

Fig. 1.

Schematic of the apparatus

3 Contents

Users traveled through the digestive system of the human body as if they became food. At the beginning of the simulation, users were eaten by a person, and taken into his mouth (see Fig. 2). Then, the scene changed into the inside of the human body and users moved forward through digestive organs by crawling on a mat embedded with pressure sensors (see Fig. 3). They could look around inside of the digestive organs and move at their own pace. Some 3-D visual images showing each digestive organ (esophagus, stomach, duodenum, small intestine, large intestine, rectum) were presented to match the users’ forward movement (see Fig. 4). The vibrator presented the tactile stimuli of rubbing the body, and sound accompanying movement was also presented from the headphones. The visual images and vibrations were different for different digestive organs, and they could learn their properties through vision and touch. 3-D sounds were arbitrarily created rather than relying on actual sounds because we could ascertain what the inside of organs sounded like. However, the sounds were made to enhance the different sensations of digestive organs. Moreover, simulated heartbeats were presented as auditory and tactile stimuli. The amplitude of the sound of heartbeats and touch increased as the user got closer to the heart. Users did not only move but also changed their appearance because they were food being digested, and they were able to see it from a subjective viewpoint (see Fig. 4).

Fig. 2.

In the first scene, users were eaten.

Fig. 3.

Crawling inside of a digestive organ.

Fig. 4.

Schematic of the experiences

3.1 Modeling of Organs

Digestive organs (esophagus, stomach, duodenum, small intestine, large intestine, rectum) were created in detail with 3-D models based on medical literature. However, the model did not reproduce details such as intestinal flora and villus and emphasized the surface shape and texture (see Fig. 5). The animation represented the peristalsis of the digestive organs and reproduced the specific movement of each digestive organ. We also reproduced the following movement to reproduce the characteristic phenomena occurring in each digestive organ.

Fig. 5.

Sample images of digestive organs. (a) Esophagus, (b) Stomach, (c) Duodenum, (d) Small intestine, (e) large intestine, (f) rectum.

Esophagus.

Peristalsis is performed to transport the ingested food, causing the mouth to contract and the stomach to relax.

Stomach.

First, large peristalsis is performed from the greater curvature to the pylorus. After that, the pyloric sphincter opens, and the pylorus performs fine peristalsis to send food to the duodenum. The inside of the organ is filled with gastric juice.

Duodenum.

Bile is secreted from Vater’s papilla. Moreover, digestive activity due to bile takes place, and feedback such as tactile stimulation starts to change.

Small Intestine.

The segmentation contractions (movements that produce a constriction in the small intestine to create segmentation) and pendulum movements (movements in which the intestinal tract expands and contracts like a bellows) are performed to absorb nutrients from food. In addition, liquefaction of food due to digestion occurs.

Large Intestine.

Performs the same movements as the small intestine, but performs only peristaltic movements as it approaches the anus. In addition, the solidification of digests starts and it becomes a stool.

Rectum.

Stools are excreted by relaxation of the anal sphincter and contraction of the rectum wall.

3.2 Detection of Creeping Movement

To detect the creeping movement of the arms, two tape-type pressure sensors (Interlink Electronics FSR408) were placed in parallel on the left half and the right half of the mat, for each front and rear, for a total of four (see Fig. 6a). A similar locomotion device was proposed for navigating in a virtual world [6]. Two pressure sensors were bonded on two thick mats (50 × 41 × 6cm), which were independent on the left and right, and thin cushioning materials (1.5 cm thick) were pasted on them (see Fig. 6b, c). We put another thin cushioning material further forward as a mark to place the user’s wrist. A thin, slippery material (Lycra® stretch mat) was placed on the surface to make the arm easier to move (see Fig. 6c). Users were told to move their arm from the upper protrusion to the lower protrusion.

Fig. 6.

Apparatus to detect crawling motion. (a) whole picture, (b) side view, (c) details.

The pressure sensor could measure the pressure applied anywhere on the tape between 0.1 and 10 kg. Users put the forearm on the two pressure sensors located in front and rear. The right and left arms were measured independently to determine the crawling movements. The outputs of the front and rear sensors were normalized to values from 0 to 1. If the front sensor position is 1 and the rear sensor position is 2, the center of gravity position M can be calculated by the following formula.

$$ M = \frac{{{\text{Front}}\,{\text{sensor}}\,{\text{value}} \times 1 + {\text{Rear}}\,{\text{sensor}}\,{\text{value}} \times 2}}{{{\text{Front}}\,{\text{sensor}}\,{\text{value}} + {\text{Rearsensor}}\,{\text{value}}}} $$

(1)

The value of M ranges from 1 to 2. If the value of M is 1, the center of gravity is located directly above the front sensor, and if the value of M is 2, the center of gravity is located directly above the rear sensor. The value of M indicates where the center of gravity lies between the front and rear sensors. Crawling was detected by the sensors for the users’ left and right hands independently, through a sequence in which the center of gravity is closer to the front sensor than the midpoint (M < 1.5), followed by the center of gravity moving to close to the rear sensor (M > 1.5). The judgment of the position of the center of gravity was made at 75 Hz frequency, but in to stabilize the operation, the average value of the immediately preceding 15 frames (200 ms) was used. Each time it was judged as having performed a creeping movement, users experienced moving forward a preset distance in each digestive organ. During this, the movement accompanying the viewpoint change (i.e., optic flow and vibration) was presented. As an example, Fig. 7 shows the movement path in the duodenum. When the creeping movement was detected with either the left or right arm, users could move forward. There was no movement or swing to the left or right.

Fig. 7.

Moving path in the duodenum.

3.3 Presentation of Visual, Haptic, and Auditory Stimuli

The 3-D models of the digestive organs described above were presented as visual stimuli on the HMD with binocular disparities as 3-D images, and users could look around scenes by moving their heads while both stationary and moving. Head motions were monitored by the HMD’s gyro sensor sampling at 1 kHz, and reflected on visual images at 75 Hz.

Tactile stimuli were presented on users’ abdomens and thighs by the vibrotactile transducers through the multi-channel audio pre-amplifier and the power amplifier for two s per movement (see Fig. 8). The vibration of the abdominal vibrator gradually increased for 1 s and reached the maximum value, and then decreased linearly in the next 1 s. The thigh vibrator started to vibrate at the maximum value after 1 s, and attenuated over 2 s. This vibration gave a feeling of movement to users. The vibration frequency changed according to the digestive organs experienced (see Fig. 9). The digested food, which is the user is initially solid and somewhat hard. Then, it becomes soft as a result of digestion in the stomach and small intestine. It hardens as a stool while it moves from the latter half of the large intestine o the anus (see Fig. 3). High-frequency vibration was presented when digested food is hard, and low-frequency vibration was presented when digested food is soft, considering digestive organ characteristics. For example, in the small intestine, a tactile stimulus as if crawls on mud was presented because the food was digested and becomes like porridge.

Fig. 8.

Vibration locations.

Fig. 9.

Presented vibration patterns while moving in each digestive organ.

At the same time, the same frequency 3-D sound was presented through the headphones. In addition, the heartbeat sound was always presented. Distance from the heart to the current user position was calculated, and the beating sound was presented louder when closer and lower when far away.

4 Experience and Impression

We performed a preliminary demonstration at IVRC 2016 (The 24th International collegiate Virtual Reality Contest) [7]. About 100 people participated. Participants could move through the esophagus, stomach, duodenum, small intestine, large intestine, and rectum, and learn about them in a fun visual and tactile experience. Almost every participant was able to experience it to the end in about 5–10 min. They felt as if their own body became small and were moving inside of a person’s bodily organs. Thus, this system would give us sensations of a change in one’s own body size and a telepresence at improbable spaces.

Many participants reported the impressions that the experience was fun and they felt like actually moving inside the internal organs. There were also impressions or opinions such as “I was interested in the body structure through this system,” “I want to experience the internal organs other than the digestive organs,” and “It would be interesting to have feedback when eating and excreting.” These responses suggest that our VR system might contribute to the improvement of learning motivation. Also, some opinions evaluate elaborateness such as “reproducing the movement of peristalsis accurately and it can be used as learning content”. Also, some people opined that asking for further elaboration such as “reproducing small parts such as intestinal flora will make it more suitable for learning”. On the other hand, there was an opinion that “I felt less resistance because it was more deformed than the actual internal organs.” It will be necessary to balance the elaboration of models and sensory stimuli according to individual interests and knowledge levels. This system may serve as an interactive educational program of the inner human body.

5 Conclusion

The users enjoyed the experience and gave us the opinion that our VR system would be useful for learning. This suggests the possibility that our system is effective as educational content. In the education field, it is pointed out that linking knowledge with students’ experiences facilitate their deeper understanding of learning contents. By applying multi-sensory stimuli, our system can provide a subjective experience of becoming the food that is digested and traveling inside the human body. This may help people to link their experience to knowledge of the structure and function of the internal organs of the human body, and encourage intellectual curiosity to human body and health. However, verification of the learning effect is a future topic.