Researchers at Cornell University have raised concerns about the implications of augmented reality (AR) glasses, which allow wearers to search for information, manipulate images, or even record conversations. They emphasize that this technology creates a significant power imbalance between the wearer and the person being observed.
While most current research on AR glasses focuses on enhancing the user experience, a team from the Cornell Ann S. Bowers College of Computing and Information Science and Brown University investigated the effects of this technology on interpersonal interactions. Their findings revealed that although wearers felt less anxious using the devices, the experience was quite different for those on the receiving end.
Jenny Fu, a doctoral student in information science, presented the study, titled “Negotiating Dyadic Interactions through the Lens of Augmented Reality Glasses,” at the 2023 ACM Designing Interactive Systems Conference in July.
AR glasses overlay virtual images and text in the user’s field of vision, creating a mixed-reality experience. As technology progresses, these glasses are becoming increasingly similar in appearance to regular eyewear. This raises concerns about the potential for wearers to secretly record others or generate deepfakes using their likeness.
In this study, Fu and co-author Malte Jung, an associate professor of information science, collaborated with doctoral student Ji Won Chung and associate professor Jeff Huang from Brown University, as well as independent extended reality designer Zachary Deocadiz-Smith. They observed five pairs of participants—one wearing AR glasses and one not—during a discussion on a desert survival scenario. The wearer used Spectacles, an AR glasses prototype from Snap Inc., equipped with a camera and five custom filters that transformed the appearance of the non-wearer into a deer, cat, bear, clown, or pig-bunny.
After the activity, participants engaged in a design session to suggest improvements for AR glasses for both wearers and non-wearers. They were also interviewed about their experiences.
While wearers found the playful filters helped reduce their anxiety during the exercise, non-wearers reported feeling disempowered and uncomfortable, as they had no insight into what the wearer was seeing or doing. They expressed frustration that the filters compromised their control over their appearance. The potential for the wearer to record them without consent further heightened their sense of vulnerability.
However, non-wearers were not entirely passive. Some questioned the wearer about their view and even attempted to evade the filters, which provided them with a measure of agency in this mixed-reality context. “I think that’s the biggest takeaway I have from this study: I’m more powerful than I thought I was,” Fu reflected.
Another challenge identified was that the Spectacles had tinted lenses to allow the wearer to see virtual images, which hindered social interaction quality. Fu noted, “There is no direct eye contact, which makes people very confused, because they don’t know where the person is looking. That diminishes the quality of the conversation, as the glasses obscure all nonverbal interactions.”
To foster more positive experiences for both wearers and non-wearers, study participants recommended that designers incorporate a projection display and a recording indicator light on AR glasses, so people nearby can be aware of what the wearer is viewing and recording.
Fu also suggests that designers test their glasses in social environments and engage in participatory design processes like the one conducted in their study. Furthermore, she encourages considering video interactions as a valuable data source.
By involving non-wearers in the design of the emerging mixed-reality landscape, the researchers believe they can help ensure a more equitable and transparent experience for everyone involved.
