







On going beyond message-passing based graph neural networks with physics-inspired “continuous” learning models
How PhysicsGraph Adaptive Learning Works
Learn how our platform adapts to your knowledge level and optimizes your study time.
Graph Structure of Neural Networks
View recent discussion. Abstract: Neural networks are often represented as graphs of connections between neurons. However, despite their wide use, there is currently little understanding of the relationship between the graph structure of the neural network and its predictive performance. Here we systematically investigate how does the graph structure of neural networks affect their predictive performance. To this end, we develop a novel graph-based representation of neural networks called relational graph, where layers of neural network computation correspond to rounds of message exchange along the graph structure. Using this representation we show that: (1) a "sweet spot" of relational graphs leads to neural networks with significantly improved predictive performance; (2) neural network's performance is approximately a smooth function of the clustering coefficient and average path length of its relational graph; (3) our findings are consistent across many different tasks and datasets; (4) the sweet spot can be identified efficiently; (5) top-performing neural networks have graph structure surprisingly similar to those of real biological neural networks. Our work opens new directions for the design of neural architectures and the understanding on neural networks in general.
Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Science
View recent discussion. Abstract: Through the success of deep learning in various domains, artificial neural networks are currently among the most used artificial intelligence methods. Taking inspiration from the network properties of biological neural networks (e.g. sparsity, scale-freeness), we argue that (contrary to general practice) artificial neural networks, too, should not have fully-connected layers. Here we propose sparse evolutionary training of artificial neural networks, an algorithm which evolves an initial sparse topology (Erdős-Rényi random graph) of two consecutive layers of neurons into a scale-free topology, during learning. Our method replaces artificial neural networks fully-connected layers with sparse ones before training, reducing quadratically the number of parameters, with no decrease in accuracy. We demonstrate our claims on restricted Boltzmann machines, multi-layer perceptrons, and convolutional neural networks for unsupervised and supervised learning on 15 datasets. Our approach has the potential to enable artificial neural networks to scale up beyond what is currently possible.
Neural Networks Are Already Clean
A working hypothesis about neurons, generalization, and the ecology of circuits.

AI-Newton: A Concept-Driven Physical Law Discovery System without Prior Physical Knowledge
While current AI-driven methods excel at deriving empirical models from individual experiments, a significant challenge remains in uncovering the common fundamental physics that underlie these models -- a task at which human physicists are adept. To bridge this gap, we introduce AI-Newton, a novel framework for concept-driven scientific discovery. Our system autonomously derives general physical laws directly from raw, multi-experiment data, operating without supervision or prior physical knowledge. Its core innovations are twofold: (1) proposing interpretable physical concepts to construct laws, and (2) progressively generalizing these laws to broader domains. Applied to a large, noisy dataset of mechanics experiments, AI-Newton successfully rediscovers foundational and universal laws, such as Newton's second law, the conservation of energy, and the universal gravitation. This work represents a significant advance toward autonomous, human-like scientific discovery.

AEGNN: Asynchronous Event-based Graph Neural Networks
The best performing learning algorithms devised for event cameras work by first converting events into dense representations that are then processed using standard CNNs. However, these steps discard both the sparsity and high temporal resolution of events, leading to high computational burden and latency. For this reason, recent works have adopted Graph Neural Networks (GNNs), which process events as ``static" spatio-temporal graphs, which are inherently "sparse". We take this trend one step further by introducing Asynchronous, Event-based Graph Neural Networks (AEGNNs), a novel event-processing paradigm that generalizes standard GNNs to process events as ``evolving" spatio-temporal graphs. AEGNNs follow efficient update rules that restrict recomputation of network activations only to the nodes affected by each new event, thereby significantly reducing both computation and latency for event-by-event processing. AEGNNs are easily trained on synchronous inputs and can be converted to efficient, "asynchronous" networks at test time. We thoroughly validate our method on object classification and detection tasks, where we show an up to a 11-fold reduction in computational complexity (FLOPs), with similar or even better performance than state-of-the-art asynchronous methods. This reduction in computation directly translates to an 8-fold reduction in computational latency when compared to standard GNNs, which opens the door to low-latency event-based processing.

How neural networks learn distributed representations
Deep learning’s effectiveness is often attributed to the ability of neural networks to learn rich representations of data.

Understanding the inner workings of neural networks
New research on that gives us a deeper understanding of why networks make the decisions they do

Making thousands of open LLMs bloom in the Vertex AI Model Garden
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Introducing beginners to the mechanics of machine learning – Miriam Posner
Every year, I spend some time introducing students to the mechanics of machine learning with neural nets. I definitely don’t go into great depth; I usually only have one class for this. But I try to unpack at least some of the major concepts, so that ML isn’t quite such a black box.
Are biological systems poised at criticality?
Many of life's most fascinating phenomena emerge from interactions among many elements--many amino acids determine the structure of a single protein, many genes determine the fate of a cell, many neurons are involved in shaping our thoughts and memories. Physicists have long hoped that these collective behaviors could be described using the ideas and methods of statistical mechanics. In the past few years, new, larger scale experiments have made it possible to construct statistical mechanics models of biological systems directly from real data. We review the surprising successes of this "inverse" approach, using examples form families of proteins, networks of neurons, and flocks of birds. Remarkably, in all these cases the models that emerge from the data are poised at a very special point in their parameter space--a critical point. This suggests there may be some deeper theoretical principle behind the behavior of these diverse systems.

Neural Computer: A New Machine Form Is Emerging
A research essay on Neural Computer: how it differs from agents, world models, and conventional computers; what runtime and CNC would mean; what current prototypes already show; and how software and hardware might change.
Deep Learning - The Biggest Data Science Breakthrough of the Decade - O'Reilly Media Free, Live Events
Machine learning and AI have appeared on the front page of the New York Times three times in recent memory: 1) When a computer beat the world's #1 chess player 2) When Watson beat the world's best Jeopardy players 3) When deep learning algorithms won a chemo-informatics Kaggle...

OpenAI peeks into the “black box” of neural networks with new research
We do not understand" how LLMs work, admits OpenAI in quest to make them interpretable.

Neural network implemented with light instead of electrons
Can do basic character, object recognition.

The spelled-out intro to neural networks and backpropagation: building micrograd