Virtual co-embodiment
MindMeld Cyclers (2023)
A Collaborative Motion Mode Using Virtual Co-embodiment
Key Words
Virtual reality · Co-embodiment · Serious game · Collaboration · Attention · virtual therapy · User Centered Computing
Abstract
This project introduced a new collaborative game mode using virtual co-embodiment (VCE). Virtual co-embodiment is a novel collaborative mode in virtual environments (VEs) where users jointly control the same avatar. Unlike separate avatar control, co-embodiment aggregates user inputs either through weighting or distinct body segments.
This game mode showing potential applications in skills training, tele-rehabilitation and multiplayer entertainment. According to recent studies, collaboration may increase game engagement and attention. This game aimed to explore a new design paradigm for VCE and the first experimental insight into the relationship between VCE and attention. Participants performed a bicycle simulation under individual, weighted, and segmented co- embodiment conditions. A superimposed go/no-go task (often used in ADHD research) measured attention by gaze behavior, game performance and GEQ questionnaire.
Research
Related work in collaborative serious games
Virtual Co-embodiment
Different from previous collaboration where each user controls their avatar solely, virtual co-embodiment was defined as a situation where two users control the same avatar. This concept was proposed in 2020. It consists of two configuration methods: weighted-average-based and body-part-segmented based. In the first scenario, each entity involved is assigned a percentage of control to the joint avatar. This percentage is called "weight" and the sum of these weights is 100%. Considering the second scenario, in this situation, each user controls different body parts such as limbs to control the avatar simultaneously.
Collaborative Games for ADHD
Serious games for ADHD (SGADs) have been one of the successful non-medication methods. Yulia et al. developed a full body embodied game PigScape which helped with cognitive impulsivity training. The group of Gongsook utilized a virtual time simulation game to train time perception of ADHD group. Combining chess and VR together, Rodrigo Yanguas’s group developed The Secret Trail of Moon (TSTM) for cognitive treatment for ADHD.
Game Engagement
Typically, engagement represents attentional level in a gaming context, which is indicated by immersion and flow.
Definitional agreement regarding how to label subjective experience during video game-playing has not yet been achieved (Wirth et al., 2007). In this paper, the term ‘engagement’ will be used as a generic indicator of game involvement. Other relevant and more technical terms include immersion, presence, flow, psychological absorption, and dissociation. These can be conceptualized as representing a progression of ever-deeper engagement in game-playing.
Core Gameplay
Co-embodied Cycling Simulation
In the game, participants will be asked to perform a simulated cycling task. As shown in the figure, the two controllers are respectively fixed at the handlebars of the simulated bicycle model, collecting bicycle steering and speed data through suspended sensors. When the user turns the controller in the real world, the bicycle in the virtual world will correspond and generate steering to simulate the experience of cycling. At the same time, users need to control the speed of the bicycle movement with the trigger buttons on the controllers.
"This research aims to investigate the effect of the user experience of proposed collaborative mode in virtual environment (VE) by using both virtual co-embodiment and serious game design, and demonstrating the positive impact of the design on the field of collaborative mode in the context of Serious Games (SGs).."
Two Game Modes
Weighted-average-based
virtual co-embodiment
The control level of the avatar body was equitably distributed between the participant and the experimenter, with both parties having a 50% share. This shared control allowed them to jointly manipulate the speed and direction of the bike using a shared avatar.
Body-part-segmented
virtual co-embodiment
The allocation of control between the participant and experimenter was randomly assigned to the avatar's left and right hands, symbolizing their control over speed and direction, respectively.
Level Design
In the game, players engaged in both a bicycling simulation and a superimposed Go/No-go task under different co-embodiment conditions. By examining performance on both components simultaneously, the dual-task method provided insight into how co-embodiment may have influenced participants’ embodied experience and attentional capacities.
A go/no-go task was employed as the concurrent attention assessment component of the dual-task paradigm. Variants of this go/no-go task are commonly used in widely accepted neuropsychological batteries to evaluate attention and executive functions. Based on this task paradigm, the game was divided into two alternating phases completed five times:
Phases
Technical Foundation
Set up - Space
The virtual co-embodiment environment was established using two HTC Vive head-mounted displays (HMDs), each paired with an HTC Vive controller and HTC Pro Eye (to collect gaze behavior). This allowed participants to simultaneously inhabit the same virtual avatar by connecting to the same network. The co-embodiment system was developed in Unity (version 2021.3.11f1c1). Through the Netcode multiplayer networking plugin for Unity, participants could jointly control an anthropic virtual avatar from a first-person perspective.
Set up - Equipment
As shown in the figure, the two controllers are respectively fixed at the handlebars of the simulated bicycle model, collecting bicycle steering and speed data through suspended sensors. When the user turns the controller in the real world, the bicycle in the virtual world will correspond and generate steering to simulate the experience of cycling. At the same time, users need to control the speed of the bicycle movement with the trigger buttons on the controllers.
Development Process
Stage 1:
Literature Review
[1] H. Liang and X. Dong, “Enhancing cognitive ability through a VR serious game training model mixing Piaget’s epistemological methodology and Lumosity concept,” Vis. Comput., vol. 38, no. 9–10, pp. 3487–3498, Sep. 2022, doi: 10.1007/s00371-022-02552-9.
[2] M. Crepaldi et al., “Antonyms: A Computer Game to Improve Inhibitory Control of Impulsivity in Children with Attention Deficit/Hyperactivity Disorder (ADHD),” Information, vol. 11, no. 4, p. 230, Apr. 2020, doi: 10.3390/info11040230.
[3] Y. A. Sekhavat, “Collaboration or battle between minds? An attention training game through collaborative and competitive reinforcement,” Entertain. Comput., vol. 34, p. 100360, May 2020, doi: 10.1016/j.entcom.2020.100360.
[4] A. Schena et al., “IAmHero: Preliminary Findings of an Experimental Study to Evaluate the Statistical Significance of an Intervention for ADHD Conducted through the Use of Serious Games in Virtual Reality,” Int. J. Environ. Res. Public. Health, vol. 20, no. 4, p. 3414, Feb. 2023, doi: 10.3390/ijerph20043414.
[36] E. Ruth Baker, Q. Liu, and R. Huang, “A View from the Start: A Review of Inhibitory Control Training in Early Childhood,” in Inhibitory Control Training – A Multidisciplinary Approach, S. Palermo and M. Bartoli, Eds., IntechOpen, 2020. doi: 10.5772/intechopen.88700.
[37] D. Roth and M. E. Latoschik, “Construction of the Virtual Embodiment Questionnaire (VEQ),” IEEE Trans. Vis. Comput. Graph., vol. 26, no. 12, pp. 3546–3556, Dec. 2020, doi: 10.1109/TVCG.2020.3023603.
[38] K. Poels, Y. de Kort, and IJsselsteijn WA, “Game Experience Questionnaire: development of a self-report measure to assess the psychological impact of digital games,” Eindh. Tech. Univ. Eindh., p. 46, 2007.
[39] J. H. Brockmyer, C. M. Fox, K. A. Curtiss, E. McBroom, K. M. Burkhart, and J. N. Pidruzny, “The development of the Game Engagement Questionnaire: A measure of engagement in video game-playing,” J. Exp. Soc. Psychol., vol. 45, no. 4, pp. 624–634, Jul. 2009, doi: 10.1016/j.jesp.2009.02.016.
[40] E. R. Høeg et al., “Buddy biking: a user study on social collaboration in a virtual reality exergame for rehabilitation,” Virtual Real., vol. 27, no. 1, pp. 245–262, Mar. 2023, doi: 10.1007/s10055-021-00544-z.
[41] M. J. Johnson, R. C. V. Loureiro, and W. S. Harwin, “Collaborative tele-rehabilitation and robot-mediated therapy for stroke rehabilitation at home or clinic,” Intell. Serv. Robot., vol. 1, no. 2, pp. 109–121, Apr. 2008, doi: 10.1007/s11370-007-0010-3.
[42] J. R. Lewis, “The System Usability Scale: Past, Present, and Future,” Int. J. Human–Computer Interact., vol. 34, no. 7, pp. 577–590, Jul. 2018, doi: 10.1080/10447318.2018.1455307.
[43] A. Bangor, P. T. Kortum, and J. T. Miller, “An Empirical Evaluation of the System Usability Scale,” Int. J. Hum.-Comput. Interact., vol. 24, no. 6, pp. 574–594, Jul. 2008, doi: 10.1080/10447310802205776.
[44] E. McAuley, T. Duncan, and V. V. Tammen, “Psychometric Properties of the Intrinsic Motivation Inventory in a Competitive Sport Setting: A Confirmatory Factor Analysis,” Res. Q. Exerc. Sport, vol. 60, no. 1, pp. 48–58, Mar. 1989, doi: 10.1080/02701367.1989.10607413.
Stage 2:
Prototype in Unity
Stage 3:
3D Printing
Stage 4:
Programming
Stage 5:
Animation, IK & Eye-tracking
Here is the full paper!
MindMeld Cyclers: A new collaborative game mode using virtual co-embodiment
Full paper is here!
MindMeld Cyclers: A new collaborative game mode using virtual co-embodiment
Click Here
Experiment
Collaborative Game & Virtual Reality
Summary
An experiment was conducted to investigate the impact of different game modes on user perception within the proposed multiplayer game framework implemented on the virtual co-embodiment platform. Participants learned to familiarize themselves with both the game rules and the skill for collaboratively operating the same avatar by using two distinct control modes (Weight-Based Mode and Body-Based Mode). The research protocol received approval from the Ethics Committee of Xian Jiaotong-Liverpool University.
Participants
Twenty-four participants from the university campus participated in the experiment [14 males, 10 females, average age = 23.5 ± 3.4 (SD)]; they were recruited from among both students and staff. They were uninformed about the study’s objectives, possessed normal or corrected-to-normal vision, and provided written and informed consent. Pairings were established between participants and members of the experimental team, with whom they had no prior interactions.
Measurement
The measured data (dependent variables) considered the objective performance and subjective measurements (questionnaires). In terms of objective performance, we gauged the time participants spent completing the entire task in each phase. Additionally, we assessed the participant’s attentional level during the No go phase of the game using gaze movement data recorded by an eye-tracker. Concerning subjective measures, we devised questionnaires pertaining to the sense of embodiment, engagement, motivation, and social presence, aligning with the game experience.
Procedure
In the tutorial phase, guidance from the experimenter was sought for the donning of Head-Mounted Displays (HMDs), along with a demonstration and explanation of relevant operations. During the tutorial for the dual task, participants received explicit instructions for the Fast and Focus (Go/No-go task) tasks. They also underwent a hands-on practice session for speed and direction control using the bicycle, guided by the experimenter, and completed the eye movement test and calibration. In the case of Co-embodiment, participants immersed themselves in virtual co-embodiment, experiencing the two modes it entailed. Subsequently, they performed the dual task twice during the main experiment phase under predetermined conditions, namely Weight-Based Co-embodiment, Body-Based Co-embodiment, or Alone. A 30-second resting interval was implemented between trials to mitigate potential participant fatigue. Participants were required to remove their HMDs for a 60-second break upon completing a game mode (two trials), during which they answered relevant questionnaires covering Virtual Embodiment, engagement, motivation, and social presence (as described in the following section). Following the completion of all experimental procedures, participants underwent an interview regarding their user experience throughout the entire process. Additionally, participants were requested to provide demographic information, including their age and prior VR experience. The overall duration of the experimental session was approximately 50 minutes.
Final view
Collaborative Game & Virtual Reality
