In an online marketplace like Airbnb, host profiles can mean the difference between a booked room and a vacant one. Too peppy, too long, too many exclamation points? Language is critical in a user’s search for trust and authenticity, crucial factors in any online exchange.
With so much at stake, should Airbnb hosts rely on an algorithm to write their profiles for them?
That depends, according to new research from Cornell and Stanford University. If everyone uses algorithmically generated profiles, users trust them. However, if only some hosts choose to delegate writing responsibilities to artificial intelligence, they are likely to be distrusted. Researchers billed this the “replicant effect” – a nod to the movie “Blade Runner.”
“Participants were looking for cues that felt mechanical versus language that felt more human and emotional,” said Maurice Jakesch, a doctoral student in information science at Cornell Tech and lead author of “AI-Mediated Communication: How the Perception that Profile Text was Written by AI Affects Trustworthiness,” which will be presented at the ACM Conference on Human Factors in Computing Systems, May 4-9 in Glasgow, Scotland.
“They had their own theories of what an AI-generated profile would look like,” Jakesch said. “If there were serious spelling mistakes in a profile, they would say it’s more human, whereas if someone’s profile looked disjointed or senseless, they assumed it was AI.”
AI stands to revolutionize natural language technologies and how humans interact with each other. People experience some AI-mediated communication already: Gmail scans the content of our arriving emails and generates suggested, one-click “smart replies,” and a new generation of writing aids not only fixes our spelling errors but polishes our writing style.
“We’re beginning to see the first instances of artificial intelligence operating as a mediator between humans, but it’s a question of: ‘Do people want that?’” Jakesch said. “We might run into a situation where AI-mediated communication is so widespread that it becomes part of how people evaluate what they see online.”
In the study, researchers sought to explore whether users trust algorithmically optimized or generated representations, particularly in online marketplaces. The team conducted three experiments, enlisting hundreds of participants on Amazon Mechanical Turk to evaluate real, human-generated Airbnb profiles. In some cases, participants were led to believe that some or all of the profiles were generated through an automated AI system. Participants were then asked to give each profile a trustworthiness score.
When told they were viewing either all human-generated or all AI-generated profiles, participants didn’t seem to trust one more than the other. They rated the human- and AI-generated profiles about the same.
But that changed when participants were informed they were viewing a mixed set of profiles. Left to decide whether the profiles they were reading were written by a human or an algorithm, users distrusted the ones they believed to be machine-generated.
“The more participants believed a profile was AI-generated, the less they tended to trust the host, even though the profiles they rated were written by the actual hosts,” the authors wrote.
What’s so bad about using AI-mediated communication? As one study participant put it, AI-generated profiles “can be handy but also a bit lazy. Which makes me question what else they’ll be lazy about.”
As AI becomes more commonplace and powerful, foundational guidelines, ethics and practice become vital, the researchers said. Their findings suggest there are ways to design AI communication tools that improve trust for human users. For starters, Jakesch said, companies could add an emblem on all text produced by AI, as some media outlets already use on algorithm-created content. Design and policy guidelines and norms for using AI-mediated communication are worth exploring now, he added.
“The value of these technologies will depend on how they are designed, how they are communicated to people and whether we’re able to establish a healthy ecosystem around them,” he said. “It comes with a range of new ethical questions: Is it acceptable to have my algorithmic assistant generate a more eloquent version of me? Is it okay to for an algorithm to tell my kids goodnight on my behalf? These are questions that need to be discussed.”
The paper was co-authored with Mor Naaman, associate professor of information science at the Jacobs Technion-Cornell Institute at Cornell Tech; Xiao Ma, a doctoral student in information science at Cornell Tech; and Megan French and Jeffrey T. Hancock of Stanford.
Louis DiPietro is communications coordinator for the Department of Information Science.