Simple artificial intelligence models that aim to replicate how humans speak may provide clues to how humans actually process language...
Work published in Proceedings of the National Academy of Sciences has shown that a range of models that are optimised to predict the next word from previous text, just like those used for predictive text on smartphones, have shown remarkable similarities to human brains.
The models in question are known as ‘neural networks’. Though their basic structure is based loosely on the neurons within the brain, they have been developed without trying to closely imitate the brain’s computations, not least because these computations are not well understood. These simple models are formed of a collection of nodes, or neurons, which are interconnected with varying degrees of strength. Information can be passed through these networks and they can be optimised to perform certain tasks, like predicting the next word in a string.
In this study, researchers measured the neural patterns produced by such artificial neural networks when presented with strings of text. These responses were then compared to patterns in the human brain as subjects underwent brain scans.
Researchers performed this comparison for 43 state-of-the-art language models and found that the best next-word prediction models exhibited patterns that closely resembled those seen in the brain. Models that were not optimised for next-word prediction did not show this similarity, suggesting that next-word prediction may play a special role in the underlying computations present in the brain.
Ev Fedoronko from MIT, a senior author of the study, expressed their surprise at the findings, “Just a few years ago, I would not have predicted that we would get here. Even just a few years ago, if you remember trying to translate something with Google Translate, it was pretty abysmal. Suddenly, within the last 5 years, these tasks are being done really, really well. So, they’re good enough to the point where I, as a language scientist, think ‘OK, this model is capturing something really useful about language statistics. Let’s see if we can leverage them somehow to try to understand how human brains might solve language.’”
While these links are exciting, these models are unlikely to include the right model of the brain. As Ev pointed out, they are all overly simplistic. However, by comparing many different models, the researchers could ask what makes some better than others, and make inferences about the brain’s underlying mechanisms. The hope is that this could push the boundaries of the understanding of how the brain works.
“Before these models came along, we were at a point where we could characterise these language brain regions. We could say, ‘OK, they’re highly specialised for language, they pay attention to both word meanings and structure, to how words combine into larger units, but how do we actually start making guesses about the computations that go on and how would we test it?’. These models provide just the way to do that.”
Comments
Add a comment