This repository contains the code to train and evaluate TRIBE v2, a multimodal model for brain response prediction
Introducing TRIBE v2: AI Model Predicts Human Brain Responses | AI at Meta posted on the topic | LinkedIn
Today we're introducing TRIBE v2, a foundation model trained to predict how the human brain responds to almost any sight or sound. Building on our Algonauts 2025 award-winning architecture, TRIBE v2 draws on 500+ hours of fMRI recordings from 700+ people to create a digital twin of neural activity. It enables zero-shot predictions for new subjects, languages, and tasks, consistently outperforming standard modeling approaches. We’re releasing the model, codebase, paper, and an interactive demo to help researchers advance neuroscience, apply brain insights to build better AI, and use computational simulation to speed up breakthroughs in neurological disease diagnosis and treatment. Try the demo and learn more here: https://go.meta.me/tribe2 | 175 comments on LinkedIn
🚨 We're very happy to introduce TRIBE v2: a foundation model of the human brain's responses to sight, sound, and language. Leveraging 1,000+ hours of fMRI across 720 subjects, it generalizes… | Stéphane d'Ascoli | 16 comments
🚨 We're very happy to introduce TRIBE v2: a foundation model of the human brain's responses to sight, sound, and language. Leveraging 1,000+ hours of fMRI across 720 subjects, it generalizes zero-shot to new stimuli, tasks and people, finetunes efficiently, and enables in-silico experiments. ❓How does it work? Stemming from our v1, which won the Algonauts 2025 challenge, TRIBE v2 combines video, audio, and language embeddings to predict brain activity for any brain, then adapts to each individual. Key results: 📊 High-quality predictions — TRIBE v2 predicts brain activity across cortical and subcortical regions, significantly better than standard linear models, with a log-linear scaling law and no plateau in sight. 🎯 Zero-shot generalization — Without retraining, the predictions of TRIBE v2 are more correlated with group-averaged brain responses than almost any individual fMRI scan! A short finetuning step vastly improves over linear models trained, from scratch, on each individual. 🧪 In-silico experiments — Can we do useful experiments with TRIBE v2? Yes: classic vision and language paradigms replicate in-silico. It zero-shot recovers the FFA, PPA, EBA, VWFA, Broca's lateralization, and syntactic responses in STG — all without training on these artificial tasks. 🔍 Interpretability & multimodality — ICA on the weights rediscovers known functional networks (auditory, language, motion, default mode, visual) from naturalistic data alone. Ablating modalities further maps how vision, audition, and language integrate, with the largest gains at the temporo-parietal-occipital junction. 🧠 This effort is a step toward a foundation model of the human brain. Much remains to be understood, but we hope this opens a path for neuroscience, AI, and medical research alike. All code, weights, and a live demo are open — find it useful or mistaken in some conditions? Let us know, new test cases can only help improving this effort. 📄 Paper: https://lnkd.in/e7cbunJp 💻 Code: https://lnkd.in/ebwBVuJp ▶️ Demo: https://lnkd.in/eEUVxP4S 🤗 Model: https://lnkd.in/e2T8nPJP Joint work with Jérémy RAPIN, Yohann Benchetrit, Teon Brooks, Katie Begany, Joséphine Raugel, Hubert Banville and Jean-Rémi King. 🙏 Special thanks to Elisa Cascardi, Diego Marcos, Dominic Giardini, AI at Meta, and the open-source and neuroscience communities (in particular Lune Bellec and Bertrand Thirion for the amazing Courtois NeuroMod and IBC datasets) | 16 comments on LinkedIn
Introducing TRIBE v2: A Predictive Foundation Model Trained to Understand How the Human Brain Processes Complex Stimuli
Understanding how the human brain processes the world around us is one of the greatest open challenges in neuroscience. Breakthroughs here could transform how we understand and treat neurological conditions affecting hundreds of millions of people — and improve AI systems by directly guiding their development from neuroscientific principles.
✨🧠 Tribe v2, our latest model of human brain responses to sound, sight and language can now be (partly) explored on your phone📱: ▶️demo: https://lnkd.in/eT6uFMnm 📄paper: https://lnkd.in/eDbFsmHx… | Jean-Rémi King | 10 comments
✨🧠 Tribe v2, our latest model of human brain responses to sound, sight and language can now be (partly) explored on your phone📱: ▶️demo: https://lnkd.in/eT6uFMnm 📄paper: https://lnkd.in/eDbFsmHx 💻code: https://lnkd.in/eTtKHu_7 | 10 comments on LinkedIn
Foundation Model Predicts Brain Responses to Visual and Auditory Stimuli | Elisa Cascardi posted on the topic | LinkedIn
Thrilled to share this work with the world! Today, we're releasing a foundation model that predicts how the human brain responds to almost any sight or sound -- and replace the need for human scans to significantly fast-track neuroscience and clinical research. 🧠 With this model, we can simulate brain responses to advance our understanding of the brain -- without the need for costly human brain scans 🌐 By using improved understanding of how efficient our brains perceive the world around us, we can guide the development of more advanced AI systems 👩⚕️ With computer-simulated experimentation, we can now speedup clinical research to diagnose neurological diseases and find treatments faster We've open sourced the model and code for researchers to use and build on, and an interactive demo for you to learn more -- see below! 📄 Paper: https://lnkd.in/e7cbunJp 💻 Code: https://lnkd.in/ebwBVuJp ▶️ Demo: https://lnkd.in/eEUVxP4S 🤗 Model: https://lnkd.in/e2T8nPJP So thrilled to be a part of this team with Stéphane d'Ascoli Jean-Rémi King Jérémy RAPIN Yohann Benchetrit Teon Brooks Katelyn Begany Joséphine Raugel Hubert Banville and for the great teamwork with Diego Marcos Dominic Giardini bringing this research to life! #neuroscience #AI #aiforscience #opensource #neuroAI
Introducing TRIBE v2: A Predictive Foundation Model Trained to Understand How the Human Brain Processes Complex Stimuli | Keith Doelling
This is some very cool work by some awesome colleagues Jean-Rémi King, and Teon Brooks! Seriously not enough good things can be said about how cool it is. You should enjoy it and play with it. And kudos to Meta for open sourcing it. At the same time, I'm already seeing posts about how the model will replace fMRI experiments as researchers will simulate how the brain "really works" instead of running costly experiments. I think this goes WELL beyond what its creators intend. We are already seeing that use of AI in science allows you to explore charted ideas more thoroughly and much more rapidly but slows us down in finding novel ideas (https://lnkd.in/eMR2akqt). At the same time, there is growing concern that LLM performance will collapse as they are increasingly trained on their own output (https://lnkd.in/eavgfyuY). Leaving neuroscience to AI simulations risks following the same fate, where we generate seemingly new findings without gaining new meaning. A mechanistic understanding of how the brain works (if that is still your goal) will be found at the margins, in errors and idiosyncrasies of neural function. What TRIBE provides is a super useful and cool instantiation of our current understanding on how and where neural activity is instantiated in the brain. But it won't help us make groundbreaking new findings of how neural circuits lead to cognition and behavior. Experiments on real human brains, may be costly, but they will always be necessary!
Introducing TRIBE v2: A Predictive Foundation Model Trained to Understand How the Human Brain Processes Complex Stimuli | Keith Doelling
This is some very cool work by some awesome colleagues Jean-Rémi King, and Teon Brooks! Seriously not enough good things can be said about how cool it is. You should enjoy it and play with it. And kudos to Meta for open sourcing it. At the same time, I'm already seeing posts about how the model will replace fMRI experiments as researchers will simulate how the brain "really works" instead of running costly experiments. I think this goes WELL beyond what its creators intend. We are already seeing that use of AI in science allows you to explore charted ideas more thoroughly and much more rapidly but slows us down in finding novel ideas (https://lnkd.in/eMR2akqt). At the same time, there is growing concern that LLM performance will collapse as they are increasingly trained on their own output (https://lnkd.in/eavgfyuY). Leaving neuroscience to AI simulations risks following the same fate, where we generate seemingly new findings without gaining new meaning. A mechanistic understanding of how the brain works (if that is still your goal) will be found at the margins, in errors and idiosyncrasies of neural function. What TRIBE provides is a super useful and cool instantiation of our current understanding on how and where neural activity is instantiated in the brain. But it won't help us make groundbreaking new findings of how neural circuits lead to cognition and behavior. Experiments on real human brains, may be costly, but they will always be necessary!
#neuroscience #ai #python #opensource #neuroai | Jean-Rémi King
⚡ We're happy to release NeuralSet: a fast, simple, and scalable Python framework for Neuro-AI. Already supports: 🧠 fMRI, EEG, MEG, iEEG, spikes… recordings 💬 text, 🔊 audio, ▶️ video, 🏞️ image… embeddings 📦 `pip install neuralset` 💻 Code: https://lnkd.in/eamwxBUY 📄 Paper: https://lnkd.in/epbreyDy Made possible thanks to: Hubert Banville, Katie Begany, Corentin Bel, Yohann Benchetrit, Teon Brooks, Marlène Careil, Simon Dahan, Stéphane d'Ascoli, Alexandre Défossez, Linnea Evanson, PhD, Pablo J. Diego Simón, Julien Gadonneix, Sophia Houhamdi, Shubh Khanna, Jarod Lévy, Pierre Orhan, Antoine Ratouchniak, Joséphine Raugel, Andrea Elena Santos Revilla, Alexis Thual, Lucy (Mingfang) Zhang, Jérémy RAPIN #Neuroscience #AI #Python #OpenSource #NeuroAI
#neuroscience #ai #python #opensource #neuroai | Jean-Rémi King | 34 comments
⚡ We're happy to release NeuralSet: a fast, simple, and scalable Python framework for Neuro-AI. Already supports: 🧠 fMRI, EEG, MEG, iEEG, spikes… recordings 💬 text, 🔊 audio, ▶️ video, 🏞️ image… embeddings 📦 `pip install neuralset` 💻 Code: https://lnkd.in/eamwxBUY 📄 Paper: https://lnkd.in/epbreyDy Made possible thanks to: Hubert Banville, Katie Begany, Corentin Bel, Yohann Benchetrit, Teon Brooks, Marlène Careil, Simon Dahan, Stéphane d'Ascoli, Alexandre Défossez, Linnea Evanson, PhD, Pablo J. Diego Simón, Julien Gadonneix, Sophia Houhamdi, Shubh Khanna, Jarod Lévy, Pierre Orhan, Antoine Ratouchniak, Joséphine Raugel, Andrea Elena Santos Revilla, Alexis Thual, Lucy (Mingfang) Zhang, Jérémy RAPIN #Neuroscience #AI #Python #OpenSource #NeuroAI | 34 comments on LinkedIn
#neuroscience #ai #python #opensource #neuroai | Jean-Rémi King | 34 comments
⚡ We're happy to release NeuralSet: a fast, simple, and scalable Python framework for Neuro-AI. Already supports: 🧠 fMRI, EEG, MEG, iEEG, spikes… recordings 💬 text, 🔊 audio, ▶️ video, 🏞️ image… embeddings 📦 `pip install neuralset` 💻 Code: https://lnkd.in/eamwxBUY 📄 Paper: https://lnkd.in/epbreyDy Made possible thanks to: Hubert Banville, Katie Begany, Corentin Bel, Yohann Benchetrit, Teon Brooks, Marlène Careil, Simon Dahan, Stéphane d'Ascoli, Alexandre Défossez, Linnea Evanson, PhD, Pablo J. Diego Simón, Julien Gadonneix, Sophia Houhamdi, Shubh Khanna, Jarod Lévy, Pierre Orhan, Antoine Ratouchniak, Joséphine Raugel, Andrea Elena Santos Revilla, Alexis Thual, Lucy (Mingfang) Zhang, Jérémy RAPIN #Neuroscience #AI #Python #OpenSource #NeuroAI | 34 comments on LinkedIn
nubrain - A foundation model for neural decoding
Building the world's largest dataset of human brain activity.
Neuropeek — Accelerating the Neuro-AI convergence
Where brain data becomes shared knowledge. A community-driven platform federating neuroscience datasets, models, and tools to accelerate discovery.
Introducing Gemma 4 12B: a unified, encoder-free multimodal model
An overview of Gemma 4 12B, a model designed to bring high-performance multimodal intelligence directly to your laptop.
DANDI
DANDI: Distributed Archives for Neurophysiology Data Integration The DANDI platform is supported by the BRAIN Initiative for publishing, sharing, and processing neurophysiology data. The archive accepts cellular neurophysiology data including electrophysiology, optophysiology, and behavioral time-series, and images from immunostaining experiments. The platform is now available for data upload and distribution. The storage of data in the archive is also supported by the Amazon Opendata program. The data in the archive can be browsed using the Data Portal. For detailed instructions on how to interact (view, upload, download, process) with DANDI click here.
