Why DeepMind isn't rolling out its new AI chatbot - and what it means for responsible AI

Couldn't attend Transform 2022? Check out all the summit sessions in our on-demand library now! Look here.

DeepMind's new AI chatbot, Sparrow, is hailed as an important step towards creating safer and less biased machine learning systems, thanks to its reinforcement learning application based on participants' contributions to the human research for training.

The UK subsidiary of Google's parent company, Alphabet, claims that Sparrow is a "dialog agent that is helpful and reduces the risk of dangerous and inappropriate responses". The agent is designed to "talk with a user, answer questions, and search the internet using Google when it's useful to seek evidence to inform their answers."

But DeepMind sees Sparrow as a research-based proof-of-concept model that isn't ready for deployment, said Geoffrey Irving, security researcher at DeepMind and lead author of the paper featuring Sparrow.

"We haven't deployed the system because we believe it has many biases and flaws of other types," Irving said. “I think the question is, how do you weigh the pros of communication — like communicating with humans — against the cons? I tend to believe in the security needs of talking to humans… I think this is a tool for that in the long run.

Event

MetaBeat 2022

MetaBeat will bring together thought leaders to advise on how metaverse technology will transform the way all industries communicate and do business on October 4 in San Francisco, CA.

register here

Irving also noted that he would not yet comment on the possible path for enterprise apps using Sparrow - whether it will ultimately be more useful for general digital assistants such as Google Assistant or Alexa, or for specific vertical applications.

"We're not nearly there," he said.

DeepMind tackles dialogue difficulties

One of the biggest challenges with any conversational AI is dialogue, Irving said, because there's so much context to consider.

"A system like DeepMind's AlphaFold is built into a clear scientific task, so you have data like what the folded protein looks like, and you have a rigorous notion of what the answer is - for example, have- you got the right shape,” he said. But in general cases, "you're dealing with soft questions and humans - there won't be a full definition of success."

To solve this problem, DeepMind turned to a form of reinforcement learning based on human feedback. He used the preferences of paid study participants (using a crowdsourcing platform) to train a model on how helpful a response was.

To ensure that the model's behavior is safe, DeepMind has determined an initial set of rules for the model, such as "don't make threatening statements" and "don't make hateful or insulting comments", as well as rules around potentially dangerous advice and other rules informed by existing work on the harms of language and expert consultation. A separate "rules pattern" has been trained to indicate when Sparrow's behavior violates one of the rules.

Bias in the "Human Loop"

Eugenio Zuccarelli, innovation data scientist at CVS Health and researcher at the MIT Media Lab, pointed out that there could always be a bias in the "human loop"; after all, what might be offensive to one person might not be. to another.

Furthermore, he added, rule-based approaches can create stricter rules, but lack scalability and flexibility. "It's hard to encode all the rules we can think of, especially as time goes by, these can change, and running a system based on fixed rules could...