“What is so impressive is that PaLM has never seen parallel sentences between Bengali and English,” Pichai said at that event. “It was never explicitly taught to answer questions or translate at all. The model brought all of its capabilities together to answer questions correctly in Bengali, and we can extend the technique to more languages and other complex tasks.”
Jason Post, a Google spokesperson, told BuzzFeed News that the company had never claimed that it didn’t train PaLM in Bengali. “While the PaLM model was trained on basic sentence completion in a wide variety of languages (including English and Bengali), it was not trained to know how to 1) translate between languages, 2) answer questions in Q&A format, or 3) translate information across languages while answering questions,” Post said in a statement. “It learned these emergent capabilities on its own, and that is an impressive achievement.”
Emily M. Bender, a University of Washington professor and researcher who wrote a Twitter thread about the 60 Minutes segment, took issue with Manyika’s comments. The program’s ability to translate “all of Bengali,” Bender told BuzzFeed News, is an “unscoped, unsubstantiated claim.”
“What does ‘all of Bengali’ actually mean?” Bender tweeted. “How was this tested?” She also wrote that Manyika’s statement ignored or hid the fact that Bengali texts are in the training data.
Bender tweeted that the term “‘emergent properties’ seems to be the respectable way of saying AGI,” which stands for artificial general intelligence, a hypothetical technology that can learn on its own and perform tasks better than humans. “It’s still bullshit,” she said.
Mitchell was equally blunt on Twitter. “Maintaining the belief in ‘magic’ properties, and amplifying it to millions (thanks for nothin @60Minutes!) serves Google’s PR goals,” Mitchell tweeted. “Unfortunately, it is disinformation.”
Several other people in the tech space also publicly criticized CBS and Google: