Artificial neural networks, inspired by the human brain’s biological neurons, have given rise to transformers about six years ago. These models can generate near-human-like text, as seen in AI systems like ChatGPT.
While artificial neural networks draw inspiration from the brain, it hasn’t been evident how transformers could be replicated using biological components.
Researchers from MIT, the MIT-IBM Watson AI Lab, and Harvard Medical School hypothesize that the brain may use a combination of neurons and astrocytes (non-neuronal brain cells) to perform computations similar to transformers.
Astrocytes, previously considered as mere supporting cells, play a crucial role in regulating processes like blood flow. Their computational role, however, has been unclear.
Astrocytes, when connected to two neurons, form what is known as tripartite synapses. These connections can store and integrate information, acting as a type of memory buffer, which can be crucial for computations similar to the self-attention mechanism in transformers.
The researchers developed a mathematical model based on the roles and interactions of neurons and astrocytes, which behaves similarly to a transformer.
The study offers insights that could fuel further neuroscience research on the human brain’s operations.
This bridge between AI and neuroscience is termed by Dmitry Krotov, a member of the research team, as “neuroscience for AI and AI for neuroscience.”
The team aims to compare their model’s predictions with biological experiments to refine or even challenge their hypothesis.
Given the memory-buffer role of astrocytes, the team speculates that these cells might be pivotal for long-term memory, warranting further exploration.
Quote: “For a lot of reasons, astrocytes are extremely important for cognition and behavior, and they operate in fundamentally different ways from neurons. My biggest hope for this paper is that it catalyzes a bunch of research in computational neuroscience toward glial cells, and in particular, astrocytes,” said Leo Kozachkov, the lead author.
The study was published in PNAS.
