Research

My dissertation investigates spatial learning in virtual environments. I look into how human cognitive capabilities interact with various virtual reality settings and how to establish novel virtual-navigation interventions that enhance learning, memory, and performance. Specifically, my research contribution includes two related areas: immersion in the sense of technical features, and geographic scale or environment visibility from single locations.

Immersion

One area of my work focuses on the effect of immersion on spatial learning. Immersion is a technical term describing the characteristics of a VR system such as the field of regard that measures what can be seen by physically rotating the eyes, head, and body. Although head-mounted displays are attractive to users such as inducing a higher sense of presence than low-immersion desktop computers, we know relatively little about whether higher immersion levels increase or decrease spatial learning outcomes. My study takes advantage of the flexible stimuli manipulation in VR systems by integrating navigation with well-established metrics measuring spatial knowledge acquisition, such as estimates of distance, direction judgments, and cognitive mapping, into a cross-platform spatial learning paradigm that can be assessed via, for example, HTC Vive, mobile VR, and desktop computers. This spatial learning paradigm has been extended to probe the educational role of immersion in place-based disciplines such as geosciences.

The ground-level perspective of using teleportation on a desktop screen.
Teleportation using the hand controller in an HTC Vive headset.
Onsite pointing task on a desktop screen.
Model-building task using the HTC Vive.
Publications

Zhao, J., Sensibaugh, T., Bodenheimer, B., McNamara, T. P., Nazareth, A., Newcombe, N., Minear, M., and Klippel, A. (2020). Desktop versus immersive virtual environments: Effects on spatial learning. Spatial Cognition & Computation, 3(3), 1–36.
Full paper

Zhao, J., LaFemina, P., Carr, J., Sajjadi, P., Wallgrun, J. O., & Klippel, A. (2020). Learning in the Field: Comparison of Desktop, Immersive Virtual Reality, and Actual Field Trips for Place-Based STEM Education. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (pp. 893–902). Atlanta, GA: IEEE.
Full paper

Klippel, A., Oprean, D., Zhao, J., Wallgrün, J. O., LaFemina, P., Jackson, K., Gowen, E. (2019) Immersive Learning in the Wild: A Progress Report. In: Beck D. et al. (eds) Immersive Learning Research Network. iLRN 2019. Communications in Computer and Information Science, vol 1044. Springer, Cham.
Full paper

Geographic Scale

The second area of my research centers on geographic scale, defined as the spatial extent visually accessible from a single viewpoint, and its impact on spatial learning in environmental space (i.e., which requires movement to apprehend). Investigating the relationship between the human body and its environment is a critical component in understanding the process of acquiring spatial memories. However, few empirical evaluations have looked at how the relative visual accessibility of an environment affects human-space interaction. This project aims at using immersive technologies to establish novel virtual-navigation interventions that foster rapid comprehension of large-scale environments regardless of users’ background.

Change of geographic scale at a single position within a virtual maze. Left: ground perspective (4.5ft/1.4m above ground); the blue flag was the only visible landmark. Right: elevated perspective (17.5ft/5.3m above ground); both the blue flag and Big Ben could be seen from a single viewpoint. The spatial extent visually accessible from a single viewpoint was controlled by hedges along both sides of the path.
Left: six participants in the same session using Oculus Go headsets to learn the virtual maze. Right: participant’s view in the onsite pointing task.
Publications

Zhao, J., Ma, X., Simpson, M., Sajjadi, P., Wallgrün, J. O., & Klippel, A. (2021). Reference frames and geographic scale: Understanding their relationship in environmental learning. Cartography and Geographic Information Science, 3(4), 1–15.
Full paper

Zhao, J., Simpson, M., Wallgrün, J. O., Sajjadi, P., & Klippel, A. (2020). Exploring the Effects of Geographic Scale on Spatial Learning. In Cognitive Research: Principles and Implications, 5(14), 1–18.  
Full paper

Zhao, J., Simpson, M., Wallgrun, J. O., Sajjadi, P., & Klippel, A. (2020). Extended Realities – How Changing Scale Affects Spatial Learning. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) (pp. 601–602). Atlanta, GA: IEEE.
Full paper

Zhao, J. & Klippel, A. (2019). Scale – Unexplored Opportunities for Immersive Technologies in Place-based Learning. 2019 IEEE Virtual Reality Conference.  
Full paper

Graduate Research (2014-2016)

Visualizing Distant Landmarks on Mobile Phones

My interest in geographic information science had a specific focus during my master’s program: developing navigation systems that make a person more spatial oriented by visualizing off-screen landmarks on mobile screens. In this study, off-screen landmarks were displayed on the edge of the mobile screen to support sense of direction during navigation. Our design embedded the concepts of direction and distance which not only contributed to a person’s spatial orientation but also decreased the frequency of interaction with the mobile screen such as zooming or panning. I implemented two approaches to visualize off-screen landmarks on a mobile screen: using interval icons that changed in size continuously according to the distance, and using ordinal icons that represented different ranges of distance.

Approach of displaying distant landmarks on a small screen. Distant/Off-screen landmarks are re-projected onto the buffered area and resized in terms of their distance to the user (larger distance would reduce the size of edge proxies). The user location is indicated by a human icon in the center of the mobile screen.
Credit: Dr. Rui Li
Publications

Li, R., & Zhao, J. (2017). Off-Screen Landmarks on Mobile Devices: Levels of Measurement and the Perception of Distance on Resized Icons. KI – Künstliche Intelligenz, 7(27), 1–9.   
Full paper

Zhao, J., & Li, R. (2016). Visualizing distance objects on mobile phones: Choice of resizable icons. In S. M. Freundschuh (Ed.), Conference Proceedings, AutoCarto2016. The 21st International Research Symposium on Computer-based Cartography and GIScience, Albuquerque, New Mexico, USA. September 14-16, 2016. (pp. 239–250). CaGIS.   
Full paper

Zhao, J. (2016). Supporting spatial orientation: Using resizable icons to visualize distant landmarks on mobile phones (Master’s dissertation). University at Albany, State University of New York, Albany, N.Y.   
Full paper