2 Understanding Immersive Technologies and Their Use in Language Education

When describing immersive technologies, two closely related terms, immersion, and presence, are usually considered together. Both terms have roots in human-computer interaction or HCI, although different HCI scholars will have different definitions for it depending on the area of HCI they are studying. According to Hein, Wienrich, and Latoschik (2021), immersion is “what the technology delivers in all sensory and tracking modalities and that it can be objectively assessed” (p. 119). It is an objective term, in the sense that it can be observed from the outside, therefore it is measurable. Presence, on the other hand, is a subjective concept, because it is about how individuals react to various immersive technologies and cannot be viewed or observed from outside. Presence is about individual choices, preferences, perceptions, and experiences; therefore, it can be different from person to person (*).

Immersion and presence concepts are important in that they help us define augmented and virtual reality technologies. Milgram et al. (1994) viewed immersive technologies on a continuum, ranging from augmented reality (AR), less or low immersive technology, to virtual reality (VR), most and highly immersive technology (See Figure 2.1 for traditional illustration of MR continuum). In other words, augmented reality refers to the enhancement of real-life objects and environments with computer images, while virtual reality is the total immersion in the virtual world, real or imagined. When AR and VR technologies are blended, combining real and virtual realities, it is called mixed reality (MR) technologies. In recent years, another term, extended reality or XR is also commonly used. Lee et al. (2021) describe XR not as a specific technology, but as a broader category of technological advancements that enhance human experience. While XR is the latest of the immersive technologies, due to the convergence of artificial intelligence (AI) with the XR technologies, much more advanced novel applications are also expected to emerge soon, possibly extending the MR continuum even further.

Figure 2.1. Milgram et al. (1994) Reality-Virtuality continuum

In the past, the concept of immersion in the context of language learning usually referred to either dual language programs, which allowed learners to have increased language experiences, or study-abroad programs, where learners lived in the target language country surrounded by language and culture (Blyth, 2018). Today, however, immersion with the use of AR and VR technologies aims to provide virtual immersion experiences that simulate authentic environments. In this chapter, the use of immersive technologies in the context of learning languages and cultures is discussed, to demonstrate affordances as well as limitations of such technologies for language education.

Degree of Immersion

Before discussing the pros and cons of mixed reality technologies, it is important to understand the differences between various levels of immersion, because the immersion level determines the kind of virtual technology that would be selected for educational use. In the literature, it is becoming common to categorize virtual realities under three categories: nonimmersive, semi or low immersive, and immersive. Most desktop-based games that manipulate virtual worlds using a mouse are considered non-immersive reality. Most of these applications are so widespread in daily life that they are usually not considered part of immersive technologies. AR and 360 videos experienced via PC or tablet computers are semi-immersive technologies and are viewed as a partial immersion into virtual worlds. Fully immersive technologies require wearable technology such as an AR or VR glass or head mount set. While highly immersive, head-mounted VR experiences are also the most expensive VR tools.

Figure 2.2. Level of Immersion

 Benefits and Challenges of Immersive Technologies

Some of the reported benefits of using AR and VR technologies in language education are increased immersion and presence (Davis, 2023; Wang, Petrina & Feng, 2018), active participation and interaction (Peixoto et.al., 2021), reduction of learning anxiety and stress (Lowell & Yan, 2023) and increased use of authentic materials (Parmaxi & Demetriou, 2019). Table 2.1. summarizes some of the other research studies and their results to illustrate the benefits. The following summaries are intended as a quick review, rather than an extensive list.

Table 2.1. Benefits of AR and VR use in language education

Many current studies were positive on the use of immersive technologies for teaching and learning, but as with any emerging technology, AR and VR have also disadvantages. For instance, it has been reported that AR and VR present some technical difficulties, such as low-quality images that can cause physical discomfort (Dolgunsöz, Yildirim & Yildirim, 2018); extraneous cognitive load (Bacca-Acosta, et al, 2022), accessibility issues (Lowell & Yan, 2023), and low-level perceived usefulness of the technology by the students (Repetto et. al., 2021).

Immersive Technology Applications for Language Learning

There are many immersive technology applications, as seen in Table 2.2, for language instruction. These tools can be categorized into four groups. The first category includes virtual reality platforms, which can be used for entertainment or educational purposes. It should be emphasized that these platforms are not created specifically with students or language learners in mind. All the platforms in this category have a high level of commercial purpose. Therefore, they need significant customization and planning on the educators’ part. The second group of tools is created for public use, but they have great educational benefits, such as Google Street View or Google Arts and Culture. The third group includes tools for educational purposes, while the last group is specific to language instruction.

Table 2.2. Immersive Technology Applications and Tools

Immersive Technologies and Intercultural Competence: Research Gap

Due to the increased interest in immersive technologies, many language scholars have been conducting systematic literature reviews to better understand the cumulative impact of AR and VR technologies. Three of the recent comprehensive reviews reveal similar results. For instance, in a study conducted by Huang, Cheng, and Xie (2021) VR technologies are mostly used in teaching vocabulary (28%), followed by speaking (18%) and writing and culture (10% each). Another study revealed that most technology integration studies focused on cognitive learning, such as teaching vocabulary or speaking skills, and affective domain goals, such as motivation, satisfaction, or speaking anxiety and discomfort (Hein, Wienrich, & Latoschik, 2021). Finally, Parmaxi and Demetriou (2019) found that vocabulary learning (23.9%), reading (12.7%), speaking (9.9%) writing (8.5%) are the most used activities with the mobile AR. These reviews show that language educators mostly use immersive technologies for vocabulary acquisition and practicing speaking skills.

These literature reviews also demonstrated that often students are active in immersive learning activities, while instructors have a more observant role. Another important outcome of similar literature review studies is that more research has been done in the last decade to study the impact of augmented reality technologies and language learning, compared to virtual reality technologies and language learning. This may be due to ease of use, availability, and low cost of AR technologies. Higher education was also found to be the most active implementation and research field, while K-12 schools clearly lagged. Parmaxi and Demetriou (2019) analyzed the languages used in their reviews, concluding that 63% of the research studies used English as a Second Language or ESL, 14% used Mandarin, while all other languages were in single digits.

Regarding intercultural competence or teaching of culture, only 5% of the studies used immersive technology (Hein, Wienrich & Latoschik, 2021), while Parmaxi and Demetriou (2019) reported 4.2% usage. These results show a very significant research gap in an area full of potential, especially for languages other than English and Mandarin.

References

Bacca-Acosta, T., J., Fabregat, R., Kinshuk, & Guevara, J. (2022). Scaffolding in immersive virtual reality environments for learning English: An eye-tracking study. Educational Technology Research and Development, 70 (1), 339–362.

Blyth, C. (2018). Immersive technologies and language learning. Foreign Language Annals, 51(1), 225–232.

Davis, R. (2023). Leveraging the Power of Presence for Learning Design. In Cherner, T., & Fegely, A. (Eds). Bridging the XR Technology-to-Practice Gap: Methods and Strategies for Blending Extended Realities into Classroom Instruction. Association for the Advancement of Computing in Education. (p. 13-24), (AACE).

Dolgunsöz, E., Yıldırım, G., Yıldırım, S. (2018). The effect of virtual reality on EFL writing performance. Journal of Language Linguistic Studies. 14, 278–292.

Hein, R. M., Wienrich, C., & Latoschik, M. E. (2021). A systematic review of foreign language learning with immersive technologies (2001-2020). AIMS Electronics and Electrical Engineering, 5(2): 117–145. Available at https://opus.bibliothek.uni-wuerzburg.de/opus4-wuerzburg/frontdoor/deliver/index/docId/26881/file/10.3934_electreng.2021007.pdf.

Huang, Zou, D., Cheng, G., & Xie, H. (2021). A systematic review of AR and VR enhanced language learning. Sustainability (Basel, Switzerland), 13(9), 4639. https://doi.org/10.3390/su13094639.

Lee, M. J. W., Georgieva, M., Alexander, B., Craig, E., & Richter, J. (2021). State of XR & immersive learning outlook report 2021. Walnut, CA: Immersive Learning Research Network.

Legault, J., Zhao, J., Chi, Y.-A., Chen, W., Klippel, A., & Li, P. (2019). Immersive virtual reality as an effective tool for second language vocabulary learning. Languages, 4(1), 13.

Lowell, V. L., & Yan, W. (2023). Facilitating foreign language conversation simulations in Virtual Reality for authentic learning. In Cherner, T., & Fegely, A. (Eds). Bridging the XR Technology-to-Practice Gap: Methods and Strategies for Blending Extended Realities into Classroom Instruction. Association for the Advancement of Computing in Education. (p. 119-134), (AACE).

Milgram, P., Takemura, H., Utsumi, A., and Kishino, F. (1994). Augmented Reality: A class of displays on the reality-virtuality continuum. Proc. SPIE 2351, 282–292. DOI: 10.1117/12.197321.

Palomeque, C., & Pujolà, J.-T. (2018). Managing multimodal data in virtual world research for language learning. ReCALL (Cambridge, England), 30(2), 177–195.

Peixoto, Pinto, R., Melo, M., Cabral, L., & Bessa, M. (2021). Immersive Virtual Reality for foreign language education: A PRISMA Systematic Review. IEEE Access, 9, 48952–48962.

Xue, Y., & Wang, J. (2021). English listening teaching device and method based on Virtual Reality technology under wireless sensor network environment. Journal of Sensors, 2021, 1–11.

Wang, Petrina, S., & Feng, F. (2017). VILLAGE—Virtual immersive language learning and gaming environment: Immersion and presence. British Journal of Educational Technology, 48(2), 431–450.