What we learned about AI and deep learning in 2022

Check out all the Smart Security Summit on-demand sessions here.

Now is a great time to discuss the implications of advances in artificial intelligence (AI). 2022 has seen some exciting advancements in deep learning, especially in generative models. However, as the capabilities of deep learning models increase, so does the confusion surrounding them.

On the one hand, advanced models such as ChatGPT and DALL-E show fascinating results and feel like thinking and reasoning. On the other hand, they often make mistakes that prove they lack some of the basic elements of intelligence that humans have.

The scientific community is divided on what to do with these advances. At one end of the spectrum, some scientists have gone so far as to say that sophisticated patterns are sensitive and should be attributed to personality. Others have suggested that current deep learning approaches will lead to artificial general intelligence (AIG). Meanwhile, some scientists have studied the failures of current models and point out that, while useful, even the most advanced deep learning systems suffer from the same type of failures as earlier models.

It is in this context that the online AGI #3 debate was held on Friday, moderated by Montreal AI President Vincent Boucher and AI researcher Gary Marcus. The conference, which included talks from scientists from different backgrounds, discussed lessons learned from cognitive science and neuroscience, the path to common sense reasoning in AI, and suggested architectures that can help bridge the next AI stage.

On-Demand Smart Security Summit

Learn about the essential role of AI and ML in cybersecurity and industry-specific case studies. Watch the on-demand sessions today.

"Deep learning approaches can provide useful tools in many areas," said linguist and cognitive scientist Noam Chomsky. Some of these apps, such as automatic transcription and text auto-completion, have become tools we rely on every day.

"But beyond usefulness, what do we learn from these approaches about cognition, thought, especially language?" said Chomsky. “[Deep learning] systems make no distinction between possible and impossible languages. The more systems are improved, the deeper the failure becomes. They will do even better with impossible languages ​​and other systems."

This flaw is evident in systems like ChatGPT, which can produce text that is grammatically correct and consistent, but logically and factually flawed. Conference presenters provided many examples of such flaws, such as large language models unable to sort sentences by length, making serious mistakes on simple logical problems, and making false and inconsistent statements. /p>

According to Chomsky, current approaches to advancing deep learning systems, which rely on adding training data, creating larger models, and using "intelligent programming", do not will only exacerbate the errors made by these systems.

"In short, they tell us nothing about language and thought, about cognition in general, or what it is to be human or any other flight of fancy in contemporary discussion," said said Chomsky.

Marcus said that a decade after the 2012 deep learning revolution, tremendous progress has been made, "but some challenges remain."

He presented four key asps...