Demonstration

While not universal, there are demonstrable benefits from the use of immersive technologies in many geo-related areas, including agent-based modeling, geographic training, climate change, hazard control and management, human-environmental interaction, and geospatial simulations involving both the physical and human dimensions. As a spatial cognition researcher with GIScience background, I hope to utilize cutting-edge technologies and cognitive concepts to empower geospatial applications so that they can serve as an authentic and scalable testbed for both theoretical and practical questions about human-space interaction.

Visualizing Environmental Data: Immersive Workbench for Earth Sciences

Remotely sensed and terrestrial geospatial data sets are increasing in availability and quantity. Historically, these data sets have been analyzed and quarried on 2D desktop computers. We developed an immersive workbench that allows users to visualize, manipulate, and make quantitative observations of multi-source geospatial datasets in a virtual environment. The workbench offers a suite of geometric measurement tools and allows for the direct manipulation of the geo-referenced data sets, providing embodied experiences of field sites across Earth and beyond. So far, the workbench has been implemented for the Thrihnukagigur volcano in Iceland, Taroko National Park in Taiwan, and the Speulderbos forest in the Netherlands.

Immersive workbench for the Thrihnukagigar Volcano (In collaboration with Dr. Peter LaFemina).
Immersive workbench for the Speulderbos Forest LiDAR data (in collaboration with the Laboratory of Geo-information Science and Remote Sensing, Wageningen University & Research).
Publications

Wallgrün, J. O., Huang, J., Zhao, J., Brede, B., Lau, A., & Klippel, A. (2021). Embodied digital twins of forest environments. In GIScience 2021 Short Paper Proceedings. 11th International Conference on Geographic Information Science. September 27-30: Vol. 2021. Poznań (p. Poland (Online).). eScholarship, University of California.
Full paper

Klippel, A., Sajjadi, P., Zhao, J., Wallgrün, J. O., Huang, J., & Bagher, M. M. (2021). EMBODIED DIGITAL TWINS FOR ENVIRONMENTAL APPLICATIONS. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, V-4–2021, 193–200.
Full paper

Zhao, J., Wallgrün, J. O., LaFemina, P. C., Normandeau, J., & Klippel, A. (2019): Harnessing the power of immersive virtual reality – visualization and analysis of 3D earth science data sets. In Geo-spatial Information Science, pp. 1–14.
Full paper

Zhao, J., Lafemina, P., Wallgrun, J. O., Oprean, D., & Klippel, A. (2017). IVR for the geosciences. 2017 IEEE Virtual Reality Workshop on K-12 Embodied Learning through Virtual & Augmented Reality (KELVAR).
Full paper

Geoscience Virtual Field Trip: Taking Students to Remote Locations

While the number of field trips offered is typically relatively small, they are an integral part of many academic disciplines. Educators agree that field trips, as a critical aspect of place-based learning, require more support in the form of resources. We integrated 360-degree images, audio scripts, Structure-from-Motion models, videos, and 2D texts and images into a consistent work frame. The integration was the first step to simulate field trip experiences for Penn State undergraduate students and beyond. We also conducted a series of systematic evaluations as part of an introductory geoscience laboratory course. In light of the COVID-19 pandemic with students taking classes online, we created a WebGL adaptation of the virtual field trips that users could run in a web browser using their personal computers (https://sites.psu.edu/virtualfieldtrips/).

Measuring the thickness of rock layers in the immersive virtual field trip. Left: an overview of the outcrop model. Right: a student using the HTC Vive for the virtual field experience.
Geoscience Virtual Field Trip (in collaboration with the Department of Geosciences, the Pennsylvania State University).
Publications

Zhao, J., Wallgrün, J. O., Sajjadi, P., LaFemina, P. C. & Klippel, A. (2021, under review). Longitudinal effects in the effectiveness of educational virtual field trips. Manuscript submitted to the Journal of Educational Computing Research.

Sajjadi, P., Zhao, J., Wallgrun, J. O., La Femina, P., & Klippel, A. (2021). Influence of HMD Type and Spatial Ability on Experiences and Learning in Place-based Education. In Conference Proceedings of the 2021 7th International Conference of the Immersive Learning Research Network (iLRN) (pp. 289–296).
Full paper

Zhao, J., LaFemina, P., Carr, J., Sajjadi, P., Wallgrün, J. O., & Klippel, A. (2020). Learning in the Field: Comparison of Desktop, Immersive Virtual Reality, and Actual Field Trips for Place-Based STEM Education. 2020 IEEE Virtual Reality Conference (pp. 893–902).
Full paper

Sajjadi, P., Zhao, J., Wallgrün, J. O., Furman, T., LaFemina, P., Fatemi, A., Zidik, Z. E., & Klippel, A. (2020). The Effect of Virtual Agent Gender and Embodiment on the Experiences and Performance of Students in Virtual Field Trips. 2020 IEEE International Conference on Teaching Assessment, and Learning for Engineering (TALE) (pp. 221-228).
Full paper

Klippel, A., Zhao, J., Sajjadi, P., Wallgrun, J. O., Bagher, M. M., & Oprean, D. (2020). Immersive Place-based Learning – An Extended Research Framework. In 2020 IEEE Virtual Reality Workshop on K-12 Embodied Learning Through Virtual and Augmented Reality (KELVAR) (pp. 449–454).
Full paper

Klippel, A., Zhao, J., Oprean, D., Wallgrün, J. O., Stubbs, C., La Femina, P., & Jackson, K. L. (2019). The value of being there: Toward a science of immersive virtual field trips. Virtual Reality, 1(4), 24.
Full paper

Zhao, J. & Klippel, A. (2019). Scale – Unexplored Opportunities for Immersive Technologies in Place-based Learning. 2019 IEEE Virtual Reality Conference (pp. 155–162).
Full paper

Klippel, A., Zhao, J., Jackson, K. L., LaFemina, P., Stubbs, C., Oprean, D., Wetzel, R., Wallgrün, J. O., & Blair, J. (2019): Transforming Earth Science Education Through Immersive Experiences. Delivering on a Long Held Promise. In Journal of Educational Computing Research 10 (2).  
Full paper

Klippel, A., Oprean, D., Zhao, J., Wallgrün, J. O., LaFemina, P., Jackson, K., Gowen, E. (2019) Immersive Learning in the Wild: A Progress Report. In: Beck D. et al. (eds) Immersive Learning Research Network. iLRN 2019. Communications in Computer and Information Science, vol 1044. Springer, Cham.
Full paper

Klippel, A., Zhao, J., Oprean, D., Wallgrun, J. O., & Chang, J. S.-K. (2019). Research Framework for Immersive Virtual Field Trips. In 2019 IEEE Virtual Reality Workshop on K-12 Embodied Learning Through Virtual and Augmented Reality (KELVAR) (pp. 1612–1617).
Full paper

L2 Acquisition in VR: Immersive Paradigms for Second Language Learning

Learning a second language (L2) presents a significant challenge to many adults. Immersive VR affords real-life immersion and shows promise as a particularly useful L2 learning platform. We developed a virtual zoo and a fully interactive virtual kitchen environment as the first step to position second language (Chinese) in an authentic context. Evaluation of these immersive learning environments’ effectiveness empirically shows that students who struggle with language learning to start with particularly benefitted from using immersive technologies.

Virtual zoo (left) and virtual kitchen (right).
A virtual reality zoo for second language learning (Credit: The Brain, Language, and Computation Laboratory, the Hong Kong Polytechnic University).
A virtual reality kitchen for second language learning (credit: The Brain, Language, and Computation Laboratory, the Hong Kong Polytechnic University).
Publications

Li, P., Legault, J., Klippel, A., & Zhao, J. (2020). Virtual reality for student learning: Understanding individual differences. Human Behaviour and Brain, 28–36.
Full paper

Legault, J., Zhao, J., Chi, Y.-A., Chen, W., Klippel, A., & Li, P. (2019). Immersive Virtual Reality as an Effective Tool for Second Language Vocabulary Learning. Languages, 4(1), 13.
Full paper

Protorypes

Virtual Brain: Learning the Parts of the Brain in Virtual Reality
In collaboration with The Brain, Language, and Computation Laboratory, the Hong Kong Polytechnic University.
Embodied Learning of Moon Phases Using a Virtual Earth-Moon System
In collaboration with Dr. Julia D. Plummer.
Manipulating a Wood Piece From Ishi Wildness, CA Through Hand Gestures
In collaboration with Dr. Alan Taylor.