Enterprise AI Analysis: Research Article Summary
Unveiling User Preferences for Virtual Agents in Augmented Reality
Dive into our comprehensive analysis of user perceptions on virtual agent position, gaze, and gender, and discover key insights for designing effective AR experiences.
Executive Impact Summary
Our research provides actionable insights for optimizing virtual agent design in augmented reality, highlighting critical user preferences and behavioral patterns.
Key Findings at a Glance
Our study revealed significant insights into user perceptions of virtual agents in Augmented Reality, specifically concerning gender and position.
0
Participants
0
Gender Bias (Male Favored)
0
Nova (Preferred Agent)
0
Angel (Least Favored)
Deep Analysis & Enterprise Applications
Select a topic to dive deeper, then explore the specific findings from the research, rebuilt as interactive, enterprise-focused modules.
Introduction
Augmented reality (AR) headsets have become increasingly popular in recent years. They have caught the attention and interest of a wide audience since they offer a unique and immersive way to experience digital content in the real world. AR devices, such as the Microsoft Hololens 2 [40] and Magic Leap 2 [24], allow users to place or superimpose virtual objects, artifacts, or holograms over what is seen in the real world. This enhances how users see and interact with their surroundings. These AR headsets are becoming popular not just because they are appealing, but also because they have the potential to revolutionize sectors like health [1] and education [14]. AR devices offer versatile user interactions, going beyond keyboards and mice with natural gestures and speech. Additionally, as AR technology continues to advance, the concept of virtual agents has emerged as another interactive modality. Virtual agents interpret users' speech, movements, or both using sensors, but may lack a physical form or representation. Research has shown that giving a virtual agent a physical presence, whether as a human avatar or another form of embodiment, can be beneficial [2, 7, 9, 33, 52]. For example, embodied conversational agents (ECAs) enhance a dialogue system experiences by adding a social aspect to the interaction with the virtual agent [44, 48]. Further research, such as Wang et al.'s [54] study, has explored users' preferences towards different embodied agents' features in AR environments, revealing a preference for human-like agents. Building on the findings about visual representation preferences, the design attributes of virtual agents, such as gender, position, and gaze direction have also attracted attention in research. While studies like Wang et al. [54] have shown preferences for human-like agents in augmented reality settings, they did not explore gender dynamics. Existing literature presents mixed findings regarding gender preferences for virtual agents, with some studies favoring female representations [15, 34, 45, 58], others favoring male representations [35], and some indicating no significant difference [12, 21]. This diversity of findings highlights the need to investigate the role of gender alongside other design attributes in shaping user perceptions and preferences towards embodied agents. In terms of positioning and gaze direction, a significant portion of research has been dedicated to positioning virtual agents in AR in ways that ensure they do not obstruct objects and are in harmony with the environment [23, 56]. However, these studies often do not emphasize a human-centric approach, such as positioning agents side-by-side with users, as one would in real-life human interactions [19]. Our study investigates whether an AR virtual agent's position relative to the user, gaze direction, and gender affects the users' behaviors and perceptions of the agent. Our hypothesis is that the side-by-side agent configuration, where the agent stands or sits beside the user, mirrors a more natural human interaction, similar to how people typically assist one another [5]. In contrast, conventional EVAS have been designed to face the user when interacting with them [15, 22, 54]. To this end, we aim to answer the following question: What is the impact of an AR virtual agent's position, gaze direction, and gender on the users' behaviors and perceptions during an AR activity? In this paper, we present the results and findings from studying users' interactions with AR virtual agents while completing four sets of find-the-hidden-object puzzles (see section 3.3). Each participant engaged with four uniquely positioned agents, as shown in Figure 1. Participants were randomly split into two groups: half interacted with male versions of these agents, while the other half interacted with their female counterparts.
Related Work
This section, we present the domain of embodied virtual agents, highlighting both the advancements and open questions in research surrounding virtual agent positioning and gender. Embodiment has emerged as a highly effective approach for presenting interactive virtual agents to users, contributing to enhanced user-agent interactions and immersive experiences [20, 28, 29, 36]. Researchers have established contextual design guidelines for creating embodied virtual agents (EVAs) [3, 46], focusing on agents that can engage users in a more human-like manner. Wang et al. [54] conducted research in the realm of EVAs' appearance within headset-enabled AR environments. By studying various methods to generate the visual attributes of these agents, they aimed to determine the ideal design choices based on users' preferences, behaviors, and perceptions. The findings highlighted the significance of agent design in fostering rapport and trust between users and virtual entities, revealing the importance of creating visually appealing and relatable virtual agents. Trust is also a crucial factor that underpins user-agent interactions, particularly in the context of EVAs. Morinezhad and Solovey [42] explored the intricate dynamics of trust in agents, emphasizing its relative nature. The level of trust users placed in EVAs is contingent upon both their past interaction experiences with such agents and the agents' current behavior. Their research outlined that integrating cooperative behavior into EVAs fosters higher levels of perceived trustworthiness, enhancing the overall user experience. In the field of education, EVAs have shown promising outcomes. Grivokostopolou et al. investigated the impact of embodied pedagogical agents in virtual learning environments [17]. Their study demonstrated that the presence of embodied agents significantly improved students' learning experiences, leading to heightened engagement and more effective knowledge-creation processes. These findings demonstrate the potential of EVAs as valuable educational tools. In summary, embodiment stands as a compelling and effective paradigm for virtual agent interaction. Studies in this area have elucidated key design considerations for crafting EVAs that align with users' preferences and contextual requirements. The appearance of embodied agents can play a pivotal role in fostering rapport and trust with users. Additionally, understanding the dynamic nature of users' trust in EVAs, as well as the positive impact of cooperative behavior, can contribute to more successful user-agent interactions. Virtual agent positioning refers to the placement and orientation of an agent with respect to the user and entities within the virtual environment. When designing virtual agents for AR environments, it is crucial to consider the positioning of the agent to simulate natural human interactions. For example, Kendon [19] discussed the spacing and orientation in human co-present interactions, shedding light on how people naturally adjust their positioning based on their interpersonal connections. He introduced various positioning arrangements (e.g., circular, side-by-side) and discussed how the group maintained spatial and orientational relationships while leaving room for group activities. These findings provided valuable insights into guiding the design of human-agent interactions. Various factors can impact user preferences for agent positioning, including the agent's position regarding the objects [23, 56], the user [47, 56], and the user's objective in the task [49, 56]. These studies computed the position of a virtual agent as the outcome of a model incorporating the user and environmental factors. Lang et al. [23] employed scene semantics to guide the positioning of a virtual agent in mixed reality. The position of their agent is computed using a cost function of the positions of the key objects in a 3D environment. Users favored the agent's positioning (e.g., location, orientation, and overall impression) than placing it randomly within a 30-degree angle facing the user or directly in front of the user. Ye et al. [56] proposed an algorithm optimizing the position and orientation of a virtual reality agent to assist users in room navigation. They modeled agent positions as quadratic functions of the distance between the user and the agent, while agent orientations were represented by polynomial functions based on the agent's position. In various contexts (i.e., school gym, museum, and factory), their approach outperformed baselines in which the agent's position was fixed relative to the object in the room (i.e., regardless of the user's position) or relative to the user (i.e., ignoring the object's position). The authors noted that automatically positioned agents could reduce the workload of human guides who relied on virtual agents for navigation assistance. Similarly, Techasarntikul et al. [49] suggested the positioning for a virtual agent in AR during a museum-guided tour to assist users in viewing the paintings. They found that users did not mind when the agent obstructed the painting but wanted them to be within the same field of view so that they did not have to trace the agent pointing to the image. Researchers also reported the effects of agent positioning on users' perceptions through monitoring physiological signals. Suzuki et al. [47] proposed a virtual agent that adjusted its position in real-time according to the arousal levels of users' heart rates. They found that the agent's irregular movements increased users' arousal levels, thereby suggesting the importance of employing a constant movement method to reach an appropriate position to suppress arousal. However, monitoring users' physiological signals for real-time agent position adjustment can be costly. As described in this section, prior works have shown that the optimal positioning of virtual agents have a positive effect on user preference and experience in navigation tasks. As AR technology emerges to be used in shared tasks in which the user and the agent are facing the screen together, it is essential to understand how various agent positions impact users' perceptions and preferences in such context. Our approach extends prior work by placing the agent beside the user when users complete tasks to simulate a human-human interaction, like prior work noted [19], comparing to placing the agent in the user's field of view that was found to be favored by users [54]. We provide insights into user perceptions of agent positions on co-presented shared tasks. Gender has received considerable attention in diverse contexts related to virtual agents, as researchers have investigated its influence in various studies [6, 21, 27, 35]. A virtual agent's gender, whether explicitly embodied or implied through voice or behavior, can influence and impact users' preferences and interactions. For instance, Lee et al. [27] conducted a study examining human interactions with Siri, a voice-only assistant. Their findings with this intelligent virtual agent revealed that participants displayed a stronger sense of trust when their gender aligned with the gender of Siri's voice. Other prior work has shown that users' preferences for the gender of virtual agents exhibit variability, with some results reporting participants' preferences towards female agents [15, 34, 45, 58], others for male agents [35], and some having no particular preference at all [12, 21]. Many factors can influence users' preferences regarding a virtual agent's gender like cultural norms [6] and the task at hand given gender stereotypes in society [15]. Forlizzi et al. [15] conducted a study to explore the relationship between the visual features of embodied agents and the tasks they perform. The study revealed that, in general, female agents received higher ratings compared to male agents. Additionally, the results indicated that there is a significant correlation between the task assigned to the agent and the preferred gender. This suggests that individuals tend to prefer agents whose appearance aligns with the gender stereotypes typically associated with the given task. Additionally, the authors suggested that if users do not have a clear expectation of an agent's gender for a task, the agent should just be the same gender as the user. Krämer et al. [22] investigated the effects of the agent's rapport and gender on participants' performance, effort, and motivation on a mathematical task. The researchers found that establishing rapport with virtual agents can improve participants' performance in mathematical tasks. Contrary to their initial hypothesis, they found that virtual agents of the opposite sex are most successful at boosting participants' performance. In summary, gender significantly impacts virtual agent design and user interactions, with varying preferences observed. While consensus on the ideal gender is lacking, aligning gender with tasks can be advantageous. Studies highlight gender's role in enhancing trust, performance, and motivation. However, most research focuses on credibility and likability. Hence, additional exploration of gender's influence on task-oriented virtual agents and user perceptions is necessary.
Methodology
We employed a mixed design, using agent gender as a between-subjects variable and agent representations as a within-subjects variable. To alleviate the order effect, we used a counterbalanced design for the representation of the agent and the gender of the agent. Every participant interacted with a fixed-gender agent in four different positions. This mixed design was chosen to examine both the main effects of agent gender and position, as well as their potential interaction. Prior research has shown that agent gender can significantly influence user perceptions, with users associating gendered agents with specific roles and characteristics [15, 25, 26, 43]. By examining both gender and position together, we aimed to investigate whether these factors interact to influence how users perceive virtual agents. For this study, we recruited a total of 48 adults aged between 18 and 34 years (mean = 22, SD = 3.56 years). Users were recruited from a local university (28 self-identified as male, 20 as female). To investigate how positioning and gender influence participants' perceptions of the agents during tasks, we presented participants with four different embodied agent configurations. Figure 1 depicts these configurations. Angel represented a human-sized agent standing beside the participant. Kai represented a human-sized agent sitting beside the participant. Nova was a small-sized agent on the table facing the participant. Drew was also a small-sized agent on the table but was oriented toward the screen. It is important to note that certain agent configurations, such as smaller agents positioned beside the participant, either sitting or standing, or larger agents positioned atop the desk in front of the participant, were not tested in our study. This decision was influenced not only by findings from Wang et al. [54], which suggested a preference for miniature embodied agents over full-sized embodied agents, but also by the rationale that such placements would hinder effective interaction. Along these lines, agents in these configurations would either fall outside the participant's field of view, lack the ability to directly interact with the screen or participant, or represent exaggerated scenarios of users interacting with agents. These configurations were hence deemed unrealistic and were therefore excluded in our study. Each agent was available in both male and female variants, with their physical attributes remaining consistent across genders and only varying in positions. All four agents had non-verbal communication capabilities; for example, they would gaze back when the participant looked at them or make gestures toward the participant when the agent talked. The female agents each had a unique female voice, and the male agents each had a unique male voice, all sourced from Microsoft's Azure Text-to-Speech service [41]. Additionally, each agent was uniquely clothed, effectively distinguishing them from the others. The agents were developed using the Unity editor [50] and Microsoft Mixed Reality Toolkit [37]. We used the Microsoft Rocketbox Avatar library [38] to create fully-rigged characters, and the agents were displayed on a Microsoft HoloLens 2 headset [40]. The Unity application served only as a client to display the agents and did not control the agents' dialogues, which were manipulated by the game server that ran on the same PC. The dialogue interactions were controlled using the "Wizard of Oz" technique, where predetermined responses for the virtual agent were selected in real time by the researcher pressing a designated key on a keyboard. We sent predefined hints and responses to the server, which were then sent to the HoloLens 2 to be uttered by the agent. The hints and responses for any particular puzzle were the same across all agent configurations. We asked participants to play a game called "find-the-hidden-object". The game was inspired by Geven et al. [16] and Wang et al. [54], in which they asked people to find objects while engaging with an agent for hints. Our find-the-hidden-object game was displayed on a 23-inch/58-cm touchscreen connected to a workstation. The game was implemented in C# using the Windows Presentation Foundation framework (WPF) [39] and established a connection with the game server on the same PC, so the agents knew which puzzle the user was currently on. The task in the game was to solve puzzles from the I-Spy children's book series [31, 32]. Each puzzle is an illustration of a scene with multiple objects, including a target object hidden in the scene (as shown in Figure 2). Participants solved puzzles and tapped on the screen once they found the object. The agents acted as assistants during the game and provided verbal hints when participants played the game. Participants were prompted to verbally ask for hints when they felt stuck. The agent could respond to utterances such as "Can you give me a hint?", and then request participants to specify the type of hint that they wanted. Notably, the agents' responses were played through the HoloLens 2 integrated speakers, with the audio spatially rendered based on the agent's location in the real world. The following types of hints were available for every puzzle: object, color, shape, context, location. Other natural conversations were also supported, such as greetings and "you're welcome". All agents, female and male, shared the same verbal responses.
Results
In this section, we present the results of quantitative and qualitative analysis of participants' responses to the questionnaires for each agent, and the results of the quantitative analysis of participants' gaze data when interacting with the agents. When conducting an analysis of variance (ANOVA), we first tested for normality using a Shapiro-Wilks test. If the data distribution was not normal, we applied an Aligned Rank Transform (ART) [55] to the data before conducting a mixed-model ANOVA, which included random effects to account for repeated measures. For qualitative responses from the questionnaires, our team transformed the responses into written notes and then conducted an interpretation session of the notes. To organize and categorize the notes, we used a bottom-up approach with an affinity diagram to group related content [10]. Since most our quantitative results were not normally distributed, we initially conducted an ART-ANOVA with three independent variables: agent gender, participant gender, and agent position, aiming to explore potential influences or interaction effects of participant gender on our results. However, since we observed no interaction effect, and participant gender was a variable we did not control, we proceeded with an ART-ANOVA involving only two independent variables: agent gender and agent position. To further explore significant main effects, we conducted pairwise post-hoc comparisons using Tukey HSD corrections to control for multiple comparisons. Contrary to our initial expectations, the ART-ANOVA revealed no statistically significant difference in any of the six metrics of the ARQ questionnaire associated with agent position (Fig. 3). However, a significant main effect of agent gender was observed for three dimensions: Appropriate (F1,46 = 5.29, p < 0.05, η² = 0.10 ), Willingness (F1,46 = 7.54, p < 0.01, η² = 0.14), and Likability (F1,46 = 8.9, p < 0.01, η² = 0.16) (Fig. 4). Despite these differences, we did not observe any order or interaction effects between the agent position and the gender of the virtual agents. To gain deeper insights into the gender-based preferences across the three significant scales identified in our study, we conducted pairwise post-hoc comparisons with Tukey HSD corrections. The analysis showed that, consistently, the male gender was favored the most. Specifically, participants rated male agents higher for Likability (p < 0.01), expressed greater Willingness to interact with male agents (p < 0.01), and perceived male agents as more Appropriate (p < 0.05) While our initial analysis did not find any statistically significant results for the influence of agent position on user perceptions, the statistically significant differences observed in gender prompted us to investigate this aspect further. We divided our data into two groups: female agents and male agents. Subsequently, we conducted separate RM-ANOVA tests for each group, allowing us to delve deeper into the gender-specific differences in agent position. The analysis of male agents revealed no statistically significant difference in position on user perceptions. Helpfulness (F3,69 = 0.45, n.s.), Personal (F3,69 = 0.95, n.s.), Trustworthiness (F3,69 = 1.01, n.s.), Appropriate (F3,69 = 0.1, n.s.), Willingness (F3,69 = 1.00, n.s.), and Likability (F3,69 = 0.33, n.s.). However, it was significant in the position of the female agents on the Personal dimension of the ARQ questionnaire (F3,69 = 2.74, p < 0.05). Post-hoc tests revealed that the small-size agent facing the participants was rated more personal than the full-size side-by-side agent standing next to the participants (p < 0.05). We did not find significance in the Personal dimension for the female agent between other position settings. Our analysis did not uncover any significant interaction effects between the agent position and the gender of the virtual agents. To examine the post-study questions regarding participants' most-liked and least-liked agents, we added up each mention as votes, similar to the method used in a study by Wang et al. [54]. Fractional votes were considered when participants could not decide on a single agent. Post-study audio data from 7 participants was not included due to technical issues with the recording equipment. All the remaining 41 valid votes were added up and divided between each of the four agents, disregarding gender differences. In general, Nova (small desk agent facing the participant), was the most liked, with a total of 16.5 votes. Kai (human-size agent sitting beside the participant) came in second, then Drew (small desk agent facing the screen), and finally Angel (human-size agent standing beside the participant) in last place, with just 3 votes. Coincidentally, Angel was also the least-liked agent, with 25 votes, followed by Nova. Kai came in third place with Drew following closely behind. In order to gain a more comprehensive understanding of how the aggregate preferences for the most- and least-liked options contributed to a broader consensus, we employed a net scoring approach. This involved subtracting the number of least-liked votes from the count of most-liked votes. The outcome of this analysis revealed that Kai received the highest net score of 9.5 points, closely followed by Nova with a score of 8 points. In stark contrast, Angel emerged as the most polarizing option, garnering a net score of -22, indicative of significant divergence in the participants' general subjective preferences. To achieve a better insight into the preferences that the participants expressed in their responses, we analyzed the open-ended questions. In total, we collected 1152 responses from our 48 participants. The team discarded the blank or no comments, and then proceeded to group the responses into related clusters using a bottom-up approach with affinity diagrams, finding 13 overarching themes. The themes, which encompass comments about different agent characteristics, are: location, size, gaze and body orientation, realness or human-likeness, gestures, appearance, interactivity and responsiveness, voice and tone, helpfulness, encouraging nature, artificial or robot-likeness, unhelpfulness, and pressuring nature. We also looked at the recurrence of comments for each theme across the different agents. All agents received virtually the same amount of positive comments regarding themes such as helpfulness, human-likeness, appearance, and encouraging nature. The agents also received similar number of negative comments in contrasting themes like pressuring nature and unhelpfulness. The following list contains the most outstanding positive and negative themes for each of our agents: Male Agents Male agents received more positive comments about the agents' interactivity and responsiveness. In this regard, participants felt that male agents were quick to respond, e.g., 'he [the agent] is so accurate about what I ask and responding within no time' (P17). Participants also made remarks about the agent being good at understanding natural speech interaction, e.g.: 'regular speech without specific commands' (P9). On the other hand, male agents received more negative comments in themes such as robot-likeness and appearance. Participants perceived male agents as 'scary' and 'creepy', with comments like: 'it was creepy looking and obviously robotic sounding' (P39), and 'I could be perceiving it this way because I already found it creepy to look at' (P33). Female Agents In the female agent grouping, the outstanding theme was both positive and negative comments about her voice and tone and how that reflected on the perceived agent's attitude. Participants' opinions regarding her voice and tone ranged from: 'she spoke very clearly' (P12) to 'the virtual assistant's tone did not match the praise it was giving' (P16). Female agents also netted the most negative comments regarding location and gaze direction, almost doubling the negative comments for male agents on this two themes. This happened even though female and male agents were placed in the same exact four locations with the same gaze direction. Finally, the unhelpfulness theme was also more present in female agent remarks. Participants mentioned that female agents were limited in the help they could give, with comments such as: 'I did not like how her hint system made me feel limited in what I could ask her' (P8), or that they considered the female agent to be 'not intelligent enough' (P44). Gaze Interaction In addition to analyzing our questionnaire data, we wanted to see if different agent positions and gaze directions would result in differences in gaze interactions from the participants. For all our agents, we looked at the number of times participants looked at the agent and the duration of each gaze (in seconds). Gaze estimation was based on head orientation derived from the HoloLens 2, a method similar to that used in virtual reality systems [4]. I identified gaze by checking if the participant's head orientation intersected with the designated area (referred to as a "hitbox") around the agent's body. A single gaze event was recorded when the user directed their gaze towards the agent and then shifted their gaze away. We conducted an ART-ANOVA to analyze both gaze occurrences and gaze time per trial for agent position and agent gender. Our results revealed no statistically significant effect of gender on gaze time per trial (F1,48 = 1.17, n.s.) or gaze occurrences per trial (F1,48 = 0.68, n.s.). However, for position, our results revealed a statistically significant effect of agent position on both gaze occurrences per trial (F3,138 = 7.31, p < 0.001) and gaze time per trial (F3,138 = 12.06, p < 0.001). Post-hoc analysis with Tukey HSD corrections revealed that, for gaze occurrences, Nova, the small-size agent facing the participants, was looked at more times per trial than Drew, the small-size agent facing the screen (p < 0.01), Angel, the human-size agent standing beside the participants (p < 0.05), and Kai, the human-size agent sitting beside participants (p < 0.001).
Discussion
The primary purpose of our research was to identify if the positioning and gaze direction of an agent in an AR setting would have any main effect on users' perceptions. While prior work has explored the placement of virtual agents in various contexts, their emphasis has been on positioning the virtual agents concerning the surrounding environment, making sure agents are not in the way of objects and are environmentally aware [11, 23, 56, 57]. In addition to placing the agent in the environment, our approach extends prior work by placing the agent beside the user to simulate a more human-like behavior. The results suggest that while positioning an agent beside a user might seem more human-like and natural, it does not necessarily lead to increased likability, reduced cognitive workload, or affect user perceptions whatsoever. Interestingly, Nova agents (small desk agent facing the participant) garnered considerably more positive feedback regarding their location compared to other agents. Observations for both Male Nova and Female Nova, such as 'It's easy to see him now' (P45) and 'I liked having her on the desk so that she was easy to glance at while playing' (P38), highlighted the participants' preference for Nova's positioning, particularly appreciating its placement within their field of view. This preference also aligns with our results which revealed Nova was the most looked-at agent, with participants gazing at Nova significantly more frequently and for longer durations per trial compared to the other agents. These findings are consistent with prior work by Wang et al. [54], where participants favored smaller, desk-level agents for their novelty and reduced uncanniness compared to larger agents, and with Techasarntikul et al. [49], who revealed the users' preference for museum virtual guide agents to be within a user's immediate line of sight in museum AR setups were preferred. Nova's placement directly in front of participants ensured constant visibility and eliminated the need for users to significantly adjust their head or gaze. Additionally, Nova's front-facing orientation likely enhanced its prominence and accessibility, making it easier for participants to establish and maintain visual contact compared to the other agents. However, Nova agents received more negative comments regarding gestures than the other agents. In our implementation, gesturing was triggered only when the agents spoke under these three conditions: when users found a hidden object (praising the user), when users pressed their fingers on a wrong object (indicating that the selection was incorrect), or when users asked a question (to give more detail, to offer a hint, or to indicate that the agent was incapable of answering that particular inquiry). The gestures consisted of simple arm and hand movements, emphasizing the agent's spoken message. It is possible that Nova's positioning (facing the participant and within their field of view), made the gestures more noticeable. In contrast, both Kai and Angel were always outside of the participant's field of view, while Drew's hand movements might have been less distracting due to their arms being obscured by the agent's body angle, as shown in Figure 1. Building on our observations, our gaze duration and frequency data suggest that all agents garnered more attention than Drew, the desk agent facing the screen. To understand this result, we need to consider that Drew was located near the participants' field of view, similar to Nova. One possible explanation for why participants' gaze was less engaging with Drew is related to the agent's body orientation and gaze direction. Our study had Drew's body facing the screen. However, we had all the agents turn their heads around and face the participants directly when observed, triggered by calculating the intersection between the hitbox and gaze. In Drew's case, this meant that if a participant pulled their head away from the screen when facing Drew, the agent would turn their head around in an almost unnatural way to look at the user directly. This phenomenon was evidenced by some responses from the open questions about Drew, such as: 'I found it a bit strange how she was looking at me over her shoulder when she could have rotated herself a bit to look at me straight on' and 'I wish she would have faced me directly instead of looking at me over her shoulder' (P12), and 'Creepy how it's staring at me' (P50). It is likely that some participants were unconsciously trying to avoid looking at Drew agents, or only looked at them for the least possible amount of time, to escape from the agents' unnatural head turn and gaze. The secondary purpose of our study was to identify if agent gender would have any main effect on user perceptions of virtual agents. The results of our study indicate a gender bias in the perception of virtual agents, as male agents were rated higher across dimensions of appropriateness, willingness, and likability. When analyzing the open-ended feedback, we can see a male gender preference in some contexts. For example, female agents received the most amount of negative comments regarding their location, gaze direction, and helpfulness, even though they were the same across all agent positions and genders. Our findings align with prior work on the influence of gender on people's perceptions, be it human-human interactions or human-agent interactions. For example, Nass et al. [43] revealed gender stereotypical reactions to computers based on voice output. Both male and female users rated a male-voiced computer as more adept in technical subjects while rating a female-voiced computer as more knowledgeable about love and relationships. Similarly, Lee et al. [26] found that male-voiced computers had a greater influence on user decisions and were perceived as more socially attractive and trustworthy. In our study and task context, these findings could explain why participants felt the male agents were more appropriate and likable than their female counterparts, as the guidance-oriented nature of the task may have subconsciously aligned with societal stereotypes that associate masculinity with competence, authority, and problem-solving capabilities. This preference for male agents also persisted among female participants, likely reflecting the internalization of these societal norms and stereotypes regardless of their own gender [30] Prior research has also highlighted the tendency of individuals to link gender with expertise. Typically, users align more with masculine topics when communicated by a male agent and resonate more with feminine topics when a female agent addresses them [25]. In interactions with on-screen agents, users tend to favor female agents for roles traditionally associated with women, like librarians or matchmakers, while male agents are preferred for roles often linked to men, such as athletic trainers [15]. In the context of our study, this association between gender and specific roles or tasks might provide insight into our findings. The task in our study, a 'find the hidden object' game, along with the agent's role of offering hints or guidance, might subconsciously be associated with the male gender. This could be one reason why participants showed a preference for male agents in our experiment. Given the findings of our study, combined with evidence from prior work, it is clear that gender biases and gender stereotypes can influence user's perceptions of virtual agents in human-computer interactions. These stereotypes can be understood in the context of the Social Role Theory (SRT) [13]. According to SRT, societal roles, often categorized by gender, influence our behaviors, expectations, and perceptions. As people often observe men and women in specific roles, they associate certain behaviors and attributes with each gender. Over time, these perceptions become gender stereotypes. In the context of our study, these biases and stereotypes are not just superficial, but they may improve or worsen the efficacy of virtual agents in completing certain tasks and engaging with users. While this suggests that designers should take into account gender stereotypes when designing these agents, we would argue that instead, designers should use the opposite gender roles in hopes that increasing visibility will result in breaking down these gender roles and stereotypes, similar to what researchers (e.g., [8]) advocate to reduce gender bias in fields dominated by one sex.
Limitations and Future Work
In our study, we explored the effect of agent gender and position on user perceptions. However, we found some limiting factors in our study. One notable limitation was the HoloLens 2 restricted field of view, which might have led to certain agents being more consistently visible to participants than others, particularly those situated on the desk as opposed to those beside the participants. Additionally, the demographics of our participants may not represent a broader or more diverse audience, given all were recruited from a university setting. Future work could study a diverse population to uncover findings that are more widely applicable. Another limitation involves the design of our agents. Some participants felt some agents had unnatural head turns and gazes, which could have influenced their perceptions. Finally, another aspect to consider pertains to gender dynamics in our study. The chosen task could have influenced participants' perceptions regarding gender and position. Our results suggest a potential gender bias that may be linked to this specific task. Simultaneously, the study lacked control over participant gender as a variable, which could have influenced the results. Future studies should examine whether these findings remain consistent when using gender-less agents, non-gendered agents, non-human anthropomorphized agents, or androgynous agents, while also controlling for participant gender to ensure a more balanced and representative sample.
Conclusion
In this paper, we presented findings from a study examining user perceptions and interactions with virtual agents in AR, focusing on agent positioning and gender. Our data showed that while agent positioning did not significantly alter user perceptions, gender biases were evident, with male agents consistently outperforming their female counterparts in areas like appropriateness, willingness, and likability. From our qualitative analysis, Nova, the small desk agent facing the participant, emerged as the preferred agent due to its positioning, while Angel, the human-size agent standing beside participants, was the least favored, often described as intrusive, despite certain positive feedback. We also delved into how gender biases and stereotypes influenced user perceptions, even in the context of AR. These insights offer guidance and highlight the importance of addressing gender biases for the design of virtual agents in AR settings.
Acknowledgments
The authors thank the reviewers of current and prior versions of this paper whose comments helped us significantly improve the paper. Special thanks to Dr. Rohith Venkatakrishnan for his invaluable feedback and insightful suggestions. We also thank the Ruiz HCI Lab for their valuable feedback. This work is partially supported by the National Science Foundation Grant Award #IIS-1750840. Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect these agencies' views.
Gender Bias in Virtual Agent Perception
Male Agents Preferred
Participants consistently rated male virtual agents higher in terms of appropriateness, willingness to interact, and likability, showing a clear gender bias in AR interactions.
Experimental Methodology Overview
Participant Recruitment (48 Adults)
→
HoloLens 2 Training & Game Intro
→
Hidden Object Game with Virtual Agents (4 configurations)
→
Post-Task Questionnaires (ARQ, I-PANAS-SF, NASA-TLX)
→
Gaze Data Analysis
→
Post-Study Interview & Qualitative Analysis
→
Gender and Position Impact Assessment
| Agent Configuration | Perceived Likability | Gaze Occurrence (Relative) | Key Qualitative Feedback |
|---|---|---|---|
| Nova (Small, Desk, Facing Participant) | Most Liked (16.5 votes) | Highest |
|
| Drew (Small, Desk, Facing Screen) | Third Most Liked | Least |
|
| Kai (Human-size, Sitting Beside Participant) | Second Most Liked (9.5 net score) | Moderate |
|
| Angel (Human-size, Standing Beside Participant) | Least Liked (25 negative votes) | Moderate |
|
The Impact of Agent Positioning on User Experience
Scenario: A study participant was tasked with finding hidden objects in an AR environment, assisted by a virtual agent.
Challenge: The agent was positioned standing next to the participant (Angel configuration), which was intended to simulate a natural human interaction. However, the participant reported feeling the agent was 'intrusive' and had a 'scary demeanor,' leading to a sense of being 'coerced to hurry' during the task.
Solution: Qualitative feedback indicated that agents like Nova (small desk agent facing the participant) were preferred due to being 'easy to see' and 'within their field of view,' suggesting that smaller, less intrusive agents enhance user comfort and focus in AR tasks.
Outcome: While human-like side-by-side positioning might seem intuitive, user perceptions in AR can be complex. Optimal agent placement considers factors like field of view, perceived intrusiveness, and task relevance, often favoring less dominant configurations to avoid negative user experiences.
Calculate Your Potential AI ROI
Estimate the efficiency gains and cost savings your enterprise could achieve by implementing intelligent virtual agents, optimized with our research findings.
Estimated Annual Savings
$0
Annual Hours Reclaimed
0
Your AI Implementation Roadmap
Leverage our research to design and deploy AI virtual agents that truly resonate with your users. Here's how we guide you:
Phase 1: Discovery & Strategy
We analyze your specific AR application and user base to align virtual agent design with our findings on user perceptions and preferences.
Phase 2: Agent Prototyping & Testing
Develop and test agent configurations (position, gaze, gender) based on optimal user engagement and task performance identified in our research.
Phase 3: Integration & Optimization
Seamlessly integrate refined virtual agents into your AR environment, with ongoing monitoring and adjustments for peak user experience.
Phase 4: Performance Monitoring & Iteration
Continuous evaluation of agent performance and user feedback to ensure long-term success and adaptability to evolving user needs.
Ready to Transform Your AR Experience?
Unlock the full potential of Augmented Reality with AI virtual agents designed for intuitive user interaction and optimal engagement.
Ready to Get Started?
Book Your Free Consultation.
Let's Discuss Your AI Strategy!
Lets Discuss Your Needs
Select a Date
Sun
Mon
Tue
Wed
Thu
Fri
Sat