JCMC: The Discussion Section

Nicole Ellison 0:02
ICA presents.

Welcome to JCMC: The Discussion Section, a production of the International Communication Association Podcast Network. I'm Nicole Ellison, the editor-in-chief of the Journal of Computer-Mediated Communication. I'm very excited to be here today with two authors of a recent JCMC publication entitled "People, Places, and Time: A Large Scale Longitudinal Study of Transformed Avatars and Environmental Context in Group Interaction in the Metaverse". Broadly, the paper looks at how changes in avatar appearance and environment shapes social behavior and user experience. So with us today, we have Eugy Han. Eugy is a SGF Fellow and third-year PhD candidate in the Department of Communication at Stanford University, where she also works with Professor Jeremy Bailenson at the Virtual Human Interaction Lab. Joining Eugy is Kristine Nowak. Dr. Nowak is a professor in the Department of Communication and Director of the Human-Computer Interaction Lab at the University of Connecticut. This year Dr. Novak is a Fulbright scholar and visiting professor in the Department of Engineering at the University of Palermo. Maybe we could start out with a quick summary about the paper for those of our listeners who haven't had a chance to read it yet. Eugy, do you want to tell us a little bit about what you did and found in the study?

Eugy Han 1:33
So, this JCMC paper covers two studies and it was done with my brilliant colleagues, Mark Miller, Cyan DeVeaux, Hanseul Jun, Kristine Nowak, who's here with us today, Jeff Hancock, Nilam Ram, and Jeremy Bailenson. We conducted these studies during a course offered here at Stanford where around 100 students took part in the first course and around 170 took part in the second course. We ran the study over a period of eight weeks for each course. For our first study, we focused on self representation. So, what happens if we transform what you look like when you're interacting with others inside VR? Students were either in an avatar that looked and felt like themselves or they were in a uniform avatar that allowed everyone to look the same. In our second study, we focused on environmental context. So, what happens if you're in a world that's outdoors surrounded by nature or indoors in a room that feels very constraining? Students were in 1 of 192 possible virtual worlds that were either indoors or outdoors and either panoramic or constrained. For the study, we sent each student in the course a Meta Quest 2 Headset and using these headsets students met inside this social VR platform called Engage, which allows people to interact with one another in real time as avatars. Students met in groups for about half an hour each week for around eight weeks total and they engaged in discussion and took part in some physical activities. During this time, we collected various forms of data before and after the course, during the VR sessions, and right after each VR session as well. And we collected self-report data on measures such as: how much presence there were feeling, how connected they felt to their group members, how photographically real things looked. We also collected motion data as well as verbal data. Then we took this data and then we build a linear growth model that would show us how these various measures changed over time and how they changed for different types of people as they grew more familiar with using VR. In study one, where we manipulated avatar appearance, many of the measures that we collected increased over time. So entitativity, or groupness, presence, enjoyment, and realism all increased over time. That helped us realize that time plays a really critical role in how people's experience in VR evolves. People adapt to the medium and they're able to really reap the benefits, such as presence and enjoyment. We also found that when students were in avatars that looked and felt like themselves, there was increased synchrony, which is similarities in moment-to-moment nonverbal behaviors between a pair of participants. Meanwhile, if students were in a uniform avatar, they reported feeling lower self-presence and perceived the virtual environment, and others, as being less photorealistic but at the same time they also reported feeling greater enjoyment. In study two, where we manipulated the environment, we found that as there was more visible space, there was more nonverbal synchrony and people reported feeling more restored by the environment. They reported a greater sense of self and spatial presence, enjoyment, and realism. In other words, the more space you could see, the more benefits there are. And similarly, when environments were outdoors, students reported feeling more restored by the environment and felt greater enjoyment, which is very in line with previous research findings on the restorative properties of nature. Before these studies, during the height of the pandemic, there was this great interest in using VR as a tool to host remote activities and courses. So, for part of Kristine's class, we ran a smaller and slightly different version of what I just described in these two large long studies. Kristine sent Pico Neo headsets, we ran a study, we collected measures and so on. What was really key here was the qualitative findings that really helped us prepare for the JCMC studies. In these qualitative findings, we found that it's really critical to provide a lot of training time to allow students to grow accustomed to the medium before investigating how responses to VR changes. One of the pillars of our JCMC paper was the critical role time plays and how people's behaviors change over time. A second thing that we learned was that task type and content is important. We tried our best to choose activities that would reduce cognitive load and also encourage social interaction. So, this pilot study was really helpful in helping us choose what kind of tasks we wanted to do in VR. The last thing is also how long we wanted our sessions to be. Should they be short, 10 minutes? Or should they be really long, like hours long? In our JCMC studies, our sessions were 30 minutes long. We were able to choose this time, not only because of previous research findings on people becoming motion sick after the 25-minute mark, but also from Kristine's pilot study where we realized 45 minutes might be too long but if it's too short then students would not really be able to become familiar with it.

Nicole Ellison 6:23
There are some differences with men versus women in VR spaces, nausea and things like that. Did you see any of those differences among your student participants?

Eugy Han 6:32
One of our colleagues Dave Markowitz and Gabby Harari are using the data that we collected to see individual differences and that paper will delve deeper into not only gender differences, but also ethnicity differences if they exist in our current data set, and also differences in simulator sickness.

Kristine Nowak 6:50
I just wanted to sort of highlight the fact that the way someone reacts to a virtual reality experience, the third time they're in VR is different from the first time they're in VR, which is different from the fifth time they're in VR. Many of us have been doing these studies where we bring people in and we compare their experience in a head-mounted display to the video. These studies, comparing modalities, suggests that that method is not a fair test. You're comparing somebody who's been watching television and playing video games for hundreds of hours. I've spent a lot of time in VR and still it takes me a few minutes. As Eugy said, the importance of training someone was our big takeaway from that first paper. You have to learn how to VR before you can use VR effectively or before you can effectively test what happens. There's more immersion, there's more enjoyment, there's more presence after a couple of times you use it. And it's only then that you're really able to assess what's going on in the system.

Nicole Ellison 7:44
Can you tell us a little bit about the backstory of this paper and how it came to be?

Kristine Nowak 7:48
Jeremy and I have had some successful collaborations for a number of years working on VR studies. We were planning to do a study in the spring of 2020. In 2021, we were talking about how we were going to open our VR labs and so we quickly realized that wasn't going to happen. Eventually, we had an idea to buy some head mounts and send them home with the students who were also very tired. They were still learning online and he was doing this very important work on Zoom fatigue. And so, we were trying to see if being able to put people in head-mounted displays over time might relax some of that. One of the questions we had from previous work, how do people learn in virtual reality? And how do you measure it? You're in these immersive experiences, it's experiential learning, it's not really recall and memorization. Because we sent them home with the students, we were able to collect this data over time. One of the cool things was we got the opportunity to do something that I as a scientist wouldn't have necessarily thought to do or thought to look at, which is to see this effect that happens when somebody has a head mount for four or five weeks in a row, where you can do multiple experiences and actually measure how their perspective of the modality and of the experiences changes, both over time and then also how it changes as a result of the different tasks. And then he had the idea to have his students in his summer class do it exclusively in VR, using Eugy and some of what we had learned.

Nicole Ellison 9:19
Hearing you talk about the difference in being able to capture longitudinal data over time versus just that snapshot reminded me a lot of the Bob Kraut HomeNet studies, from 25 years ago now. As we become used to these technologies, they become domesticated and they're shaping our interactions. Without that longitudinal view you wouldn't be able to see that.

Kristine Nowak 9:42
That first time is not representative of when they put their head mount on and they're doing something that they've done a million times. Twenty years from now, is everybody going to be VR experienced and therefore not have the same, "Oh my gosh, this is cool" reaction?

Nicole Ellison 9:59
Augmented reality for pedagogy would be also very useful. I think we really are still figuring out what kinds of tasks this technology is good for.

Kristine Nowak 10:10
Part of the experience of education is making friends, and doing things together, and having shared experiences, which we are very good at. Is VR very good at allowing you to understand how much CO2 is in the ocean? I don't know. Can it show you what it looks like when there's too much? And maybe you remember that? Or can it help you have a shared experience with a friend that helps you develop that sense of community, and involvement, and connection, which is central to the learning experience? You learn better when you're in a safe environment surrounded by people you know. I don't think it's good for things where you're trying to get people to memorize facts. But if what you want is to provide an environment and experiences or shared learning or team bonding, I think it's very, very good for that.

Eugy Han 10:52
I think I remember Jeremy saying that, "If it's not broken, don't fix it," when we were thinking about what can we do inside VR without forcing them to be in VR for the sake of being in VR. Jeremy was really adamant on not doing 2D lectures inside VR. Doing things that you could easily do really well on Zoom inside VR, like a 2D screen inside a 3D environment made no sense. There were other little takeaways that really helped us understand the power of VR really lies in the social-interaction aspect of it.

Kristine Nowak 11:22
Most VR researchers would agree that we want to save it for the thing that it does really well. And that is providing shared experiences, immersive experiences, experiential learning. When you're in immersive VR, we have less control over what they're paying attention to. When you design something, you want to create it in a way that forces people to pay attention to the thing that you want them to pay attention to. So if you're trying to teach them, then you don't want this big immersive environment where they're looking at things that are beyond what you want them to learn.

Nicole Ellison 11:53
What kinds of advice would you have for researchers or teachers who are interested in bringing VR into the research lab or the classrooms?

Eugy Han 12:04
The first thing would be time; seeing how much training you want, whether you want to be collecting data at that time, or whether you want your students to have ample time to adjust to the medium. The second thing that we haven't really discussed was providing a lot of technological support. A concern that some instructors have brought up in the past where they say, "I'm the only person running this course, I don't have the resources for having multiple people help me." I think the best advice would be as long as there's someone at least for the first few moments of training, where it might be overwhelming for just one person to take on all the workload of 30 plus students interacting with you at the same time. So, having that technical support, at least in the first few weeks would be great. Over time, as students become more familiar with the medium that drops off. And the third thing is, just having a lot of flexibility for yourself and the students. We had a lot of technical issues like software updates and we had very specific weird bugs we didn't really realize was a problem until we came across it. So, having some flexibility for both the instructor side and the student side was really nice and to created a collaborative effort between the students and the instructors to troubleshoot together.

Kristine Nowak 13:12
Eugy points out some really important pieces there, that the technical problems were greater than I thought they would be. The students would show up and have forgotten to charge the head-mounted displays or it wasn't charged enough, and so we're halfway through the class and the student gets kicked out. And then students would have trouble because there would be a software update the night before. We had a number of people, but it still wasn't enough. Students would get very frustrated, their head mount wasn't working. There was another time where we had three students that were trying to use it too close and they overloaded the Wi-Fi of the room because they use a lot of Wi-Fi. There were technical difficulties that were unanticipated. We told the students, "Listen, we're trying this out, we're all struggling with it." And the student would say, "Oh, I had this thing happen." And we gave them time to do that. We made it totally flexible. If they tried, they got credit for it. We just said, "We're all in this together and we're having fun," and the students largely loved it. They were so grateful for the opportunity to do it that they kind of rolled with it. Which I was really gratified by because I didn't know how the students were going to handle this frustration. That continued to be the case where they just rose above it and were excited to be a part of this and they managed it. So, you have to go into it with the expectation that the technology is not going to work the way you think it is, the software is not going to work, and students are going to not have the thing charged. And you just have to say, "Okay, well that's where we are today and let's do our best and try to learn what we can and be together in this experience."

Nicole Ellison 14:39
Three years in the future, do you think that all these technical and other challenges will still be there? As technologies become more accessible to the general population, do you think that the level of enthusiasm and buy-in from students will be lessened?

Kristine Nowak 15:00
Well I mean, I've been doing VR research for 20 years and the need to have a lot of technical support has not changed. Will that change as it becomes more popular and/or will people just be better at figuring out how to troubleshoot? Maybe. But I will say that that piece of it isn't any different. COVID students, mostly seniors who had most of their college years online, were just so happy to have something that wasn't zoom to do and they thought it was such a cool experience. Will that maintain in the future? Probably not at that level. But certainly, I think anytime you can give students the ability to do something hands-on where they actually get to try it, and use it, and do it, instead of reading about it, the students tend to react really positively to that, and then get very excited about it.

Eugy Han 15:43
I can think of three other factors that will play an increasing role in the future, for both researchers and teachers. So, the first one would be the type of content that's available. There just isn't a lot of VR content that's out there right now. They're very hard to find and they serve very specific needs that might not align with what exactly you're looking for. So, it's not really sustainable long-term, in that you can't reuse the same content over and over for each class. But I think as more developers jump into this field and more people are invested in building the best type of content out there, that might change. I think that might dictate how people use VR in their classrooms. The second would be the structure of the class. Our class was done synchronously where everyone was in the classroom together. But there have been talks of people interested in MOOCs or asynchronous classes, where people can just go inside the classroom and whoever's there would be there. The last factor is how much VR doses you want in your class and the type of learning you want students to engage in. We really prioritized experiential learning. So, learning by being in the space and interacting with it was really valuable for us. But I think different instructors would have different priorities. VR might just be a one-time use case in their classes, it might be a more of a everyday class type of thing where you are constantly inside VR. So, depending on what kind of learning is going to take place in that classroom and also how often that will take place would also shape how people use VR in classrooms.

Kristine Nowak 17:16
Yeah, that brings up a really good point. As we're moving back into studies, do we keep people in groups and have them do these shared experiences? Or do we have them go back to that individual-level experience? So, if we're bringing people into VR and having them do some experience by themselves then that's not necessarily giving them that shared experience that VR can do the best. So, I think as people are considering how to design their studies or their classes, that's an important part to at least think about. Is this something that's best as a solo experience? Or should this be a group level experience? And if so, what does that mean for my design, and my results, and all those complicating things?

Nicole Ellison 17:56
What other research projects are on the horizon as an extension or continuation?

Kristine Nowak 18:03
So, one of the projects that Eugy and I had talked about doing is the idea of how best to train people before they go into VR. It's really difficult to bring people in and have them do VR once and then try to get them to come back. That's very expensive. The other complicating factor is you can't have somebody come in to VR and do a 15 or 10-minute training and then also do a 20-minute study, because they're going to have nausea. So, how long does that training need to be? Can it be them watching a video of someone else being trained? What exactly does that training need to look like? The other piece of it is, how we measure what is learned, how we measure and understand what is happening when people are in virtual reality, what they're getting out of it, what they're learning, what they're feeling.

Eugy Han 18:46
These studies produced a lot of data and I think we'll be eating at it for years to come. Everyone's tackling a different aspect of the data and extensions of the project. Some of my colleagues have been looking at how people talk in VR and what they talk about. So, Cyan DeVeaux, who is also a co-author on this paper, has been leading a paper on that. And Mark, who's also another co-author on this paper, has been looking more at personal identifiability of the students that were involved. He's also looked at the motion side and the non-behavioral aspect of the data. One thing that I've been looking at personally, is the creative outcomes. So understanding how creativity is expressed and how it can be understood inside a VR context. I think there's a lot of exciting things that can be done with the formats of the data that we do already have. There are so many directions that we can take into, which is really exciting.

Nicole Ellison 19:39
JCMC: The Discussion Section is a production of the International Communication Association Podcast Network. Our producer is Sharlene Burgos. Our executive producer is DeVante Brown. The theme music is by Nicholas Rowe. Please check the show notes in the episode description to learn more about me, the articles we discussed, and JCMC: The Discussion Section overall. Thanks for listening!