Ruiming Wang, CAMD graduate student and AR/VR developer and designer, has spent several months working in Northeastern’s Center for Cognitive & Brain Health, located in the Interdisciplinary Science and Engineering Complex (ISEC). Ruiming, who is now completing his last semester in the Master of Science in Game Science and Design program, has been playing a key role in research about how virtual reality (VR) can help with psychological testing and examination.

He has been working alongside, and learning from, the project leads: Northeastern’s Art Kramer, Director of the Center for Cognitive & Brain Health; Charles Hillman, Professor of Psychology and Associate Director of the Center for Cognitive & Brain Health; and Rachel Hopman, a Postdoctoral Research Fellow within the center. These researchers are using VR equipment to test one’s cognitive ability to adapt to an unfamiliar environment and determine the effects of environmental exposures during physical activity.

Ruiming has taken the lead on the software development side of this project. He worked as a full-time co-op in this lab last summer, and then extended the work part-time through the fall and now, spring.

“This was my first co-op position, and it has been an incredibly valuable and rewarding experience,” Ruiming said. “Before this role, I had no idea about the intersections of AR/VR and cognitive research, but now, I am so much more aware of the different ways I can use my software development skills.”

His role in the Center for Cognitive & Brain Health research project has been to build the virtual environment, which is a wilderness landscape — including grassy land, hills, trees, and other elements you may typically find outdoors. Using the cross-platform game engine Unity, Ruiming developed this nature scene from scratch, as well as built the map that helps the subjects navigate through the wilderness scene, a key component of the study.

The map is especially important because the subject’s ability to use it can provide important information about their cognitive functioning. To see the map, as well as the landscape they are navigating through, participants are completely immersed in the virtual scene by wearing the VIVE Pro VR headset. It is equipped with noise cancelling technology and built-in speakers, which contribute to the immersive experience.

Combined with this state-of-the-art VR headset, participants also use the Virtuix Omni treadmill, a virtual reality motion platform that tracks real-time physical speed and stride-length so that participants can move their bodies and feel like they are making actual progress in the simulated landscape. Participants, who can walk and run at 360 degrees on this platform, are secured in a harness that allows them to make these physical movements (facilitated by wearing frictionless covers over their shoes) without risking their own safety.

In his position, Ruiming has had the opportunity to fully understand and experiment with the capabilities of this cutting-edge AR/VR technology. He has also had the ability to experience, first-hand, what the virtual world he develops actually would look like to a participant. Beyond his work in the lab, Ruiming is currently working on his thesis project including eye-tracking and biometrics.

Because of these opportunities, Ruiming plans to continue developing software that has a positive impact on society.

“I really hope to continue using my skillset and master’s degree to be helpful and contribute to important research,” Ruiming concluded. “My goal after graduation is to focus on ways to pair software development and usability with making the world a better place.”