Neuroscience-based algorithms make for better networks
When it comes to developing efficient,
robust networks, the brain may often know best. Researchers from
Carnegie Mellon University and the Salk Institute for Biological Studies
have, for the first time, determined the rate at which the developing
brain eliminates unneeded connections between neurons during early
childhood.
Though
engineers use a dramatically different approach to build distributed
networks of computers and sensors, the research team of
computer scientists
discovered that their newfound insights could be used to improve the
robustness and efficiency of distributed computational networks. The
findings, published in
PLOS Computational Biology, are the latest
in a series of studies being conducted in Carnegie Mellon's Systems
Biology Group to develop computational tools for understanding
complex biological systems while applying those insights to improve computer algorithmNetwork structure is an important topic for both biologists and computer scientists. In biology, understanding how the
network
of neurons in the brain organizes to form its adult structure is key to
understanding how the brain learns and functions. In computer science,
understanding how to optimize network organization is essential to
producing efficient interconnected systemBut the processes the brain and network engineers use to learn the optimal network structure are very different.
Neurons create networks through a process called pruning. At birth and throughout
early childhood,
the brain's neurons make a vast number of connections—more than the
brain needs. As the brain matures and learns, it begins to quickly prune
away connections that aren't being used. When the brain reaches
adulthood, it has about 50 to 60 percent less synaptic connections than
it had at its peak in childhood.
In sharp contrast, computer science and engineering networks are
often optimized using the opposite approach. These networks initially
contain a small number of connections and then add more connections as
needed.
"Engineered networks are built by adding connections rather than
removing them. You would think that developing a network using a pruning
process would be wasteful," said Ziv Bar-Joseph, associate professor in
Carnegie Mellon's Machine Learning and Computational Biology
departments. "But as we showed, there are cases where such a process can
prove beneficial for engineering as well."
The researchers
first determined key aspects of the pruning process by counting the
number of synapses present in a mouse model's somatosensory cortex over
time. After counting synapses in more than 10,000 electron microscopy
images, they found that synapses were rapidly pruned early in
development, and then as time progressed, the pruning rate slowed.
The results of these experiments allowed the team to develop an
algorithm for designing computational networks based on the brain
pruning approach. Using simulations and theoretical analysis they found
that the neuroscience-based algorithm produced networks were much more
efficient and robust than the current engineering methods.
In the networks created with pruning, the flow of information was
more direct, and provided multiple paths for information to reach the
same endpoint, which minimized the risk of network failure.
"We took this high-level algorithm that explains how neural
structures are built during development and used that to inspire an
algorithm for an engineered network," said Alison Barth, professor in
Carnegie Mellon's Department of Biological Sciences and member of the
university's BrainHubSM initiative. "It turns out that this
neuroscience-based approach could offer something new for computer
scientists and engineers to think about as they build networks."
As a test of how the algorithm could be used outside of neuroscience,
Saket Navlakha, assistant professor at the Salk Institute's Center for
Integrative Biology and a former postdoctoral researcher in Carnegie
Mellon's Machine Learning Department, applied the algorithm to flight
data from the U.S. Department of Transportation. He found that the
synaptic pruning-based algorithm created the most efficient and robust
routes to allow passengers to reach their destinations.
"We realize that it wouldn't be cost effective to apply this to
networks that require significant infrastructure, like railways or
pipelines," Navlakha said. "But for those that don't, like wireless
networks and sensor networks, this could be a valuable adaptive method
to guide the formation of networks."
In addition, the researchers say the work has implications for
neuroscience. Barth believes that the change in pruning rates from
adolescence to adulthood could indicate that there are different
biochemical mechanisms that underlie pruning.
"Algorithmic neuroscience is an approach to identify and use the
rules that structure brain function," Barth said. "There's a lot that
the brain can teach us about computing, and a lot that computer science
can do to help us understand how neural networks function."
As the birthplace of artificial intelligence and cognitive
psychology, Carnegie Mellon has been a leader in the study of brain and
behavior for more than 50 years. The university has created some of the
first cognitive tutors, helped to develop the Jeopardy-winning Watson,
founded a groundbreaking doctoral program in neural computation, and
completed cutting-edge work in understanding the genetics of autism.
Building on its strengths in biology,
computer science,
psychology, statistics and engineering, CMU recently launched
BrainHubSM, a global initiative that focuses on how the structure and
activity of the
brain give rise to complex behaviors.
Read more at:
http://phys.org/news/2015-07-neuroscience-based-algorithms-networks.html#jCp
The researchers
first determined key aspects of the pruning process by counting the
number of synapses present in a mouse model's somatosensory cortex over
time. After counting synapses in more than 10,000 electron microscopy
images, they found that synapses were rapidly pruned early in
development, and then as time progressed, the pruning rate slowed.
The results of these experiments allowed the team to develop an
algorithm for designing computational networks based on the brain
pruning approach. Using simulations and theoretical analysis they found
that the neuroscience-based algorithm produced networks were much more
efficient and robust than the current engineering methods.
In the networks created with pruning, the flow of information was
more direct, and provided multiple paths for information to reach the
same endpoint, which minimized the risk of network failure.
"We took this high-level algorithm that explains how neural
structures are built during development and used that to inspire an
algorithm for an engineered network," said Alison Barth, professor in
Carnegie Mellon's Department of Biological Sciences and member of the
university's BrainHubSM initiative. "It turns out that this
neuroscience-based approach could offer something new for computer
scientists and engineers to think about as they build networks."
As a test of how the algorithm could be used outside of neuroscience,
Saket Navlakha, assistant professor at the Salk Institute's Center for
Integrative Biology and a former postdoctoral researcher in Carnegie
Mellon's Machine Learning Department, applied the algorithm to flight
data from the U.S. Department of Transportation. He found that the
synaptic pruning-based algorithm created the most efficient and robust
routes to allow passengers to reach their destinations.
"We realize that it wouldn't be cost effective to apply this to
networks that require significant infrastructure, like railways or
pipelines," Navlakha said. "But for those that don't, like wireless
networks and sensor networks, this could be a valuable adaptive method
to guide the formation of networks."
In addition, the researchers say the work has implications for
neuroscience. Barth believes that the change in pruning rates from
adolescence to adulthood could indicate that there are different
biochemical mechanisms that underlie pruning.
"Algorithmic neuroscience is an approach to identify and use the
rules that structure brain function," Barth said. "There's a lot that
the brain can teach us about computing, and a lot that computer science
can do to help us understand how neural networks function."
As the birthplace of artificial intelligence and cognitive
psychology, Carnegie Mellon has been a leader in the study of brain and
behavior for more than 50 years. The university has created some of the
first cognitive tutors, helped to develop the Jeopardy-winning Watson,
founded a groundbreaking doctoral program in neural computation, and
completed cutting-edge work in understanding the genetics of autism.
Building on its strengths in biology,
computer science,
psychology, statistics and engineering, CMU recently launched
BrainHubSM, a global initiative that focuses on how the structure and
activity of the
brain give rise to complex behaviors.
Read more at:
http://phys.org/news/2015-07-neuroscience-based-algorithms-networks.html#jCp
The researchers
first determined key aspects of the pruning process by counting the
number of synapses present in a mouse model's somatosensory cortex over
time. After counting synapses in more than 10,000 electron microscopy
images, they found that synapses were rapidly pruned early in
development, and then as time progressed, the pruning rate slowed.
The results of these experiments allowed the team to develop an
algorithm for designing computational networks based on the brain
pruning approach. Using simulations and theoretical analysis they found
that the neuroscience-based algorithm produced networks were much more
efficient and robust than the current engineering methods.
In the networks created with pruning, the flow of information was
more direct, and provided multiple paths for information to reach the
same endpoint, which minimized the risk of network failure.
"We took this high-level algorithm that explains how neural
structures are built during development and used that to inspire an
algorithm for an engineered network," said Alison Barth, professor in
Carnegie Mellon's Department of Biological Sciences and member of the
university's BrainHubSM initiative. "It turns out that this
neuroscience-based approach could offer something new for computer
scientists and engineers to think about as they build networks."
As a test of how the algorithm could be used outside of neuroscience,
Saket Navlakha, assistant professor at the Salk Institute's Center for
Integrative Biology and a former postdoctoral researcher in Carnegie
Mellon's Machine Learning Department, applied the algorithm to flight
data from the U.S. Department of Transportation. He found that the
synaptic pruning-based algorithm created the most efficient and robust
routes to allow passengers to reach their destinations.
"We realize that it wouldn't be cost effective to apply this to
networks that require significant infrastructure, like railways or
pipelines," Navlakha said. "But for those that don't, like wireless
networks and sensor networks, this could be a valuable adaptive method
to guide the formation of networks."
In addition, the researchers say the work has implications for
neuroscience. Barth believes that the change in pruning rates from
adolescence to adulthood could indicate that there are different
biochemical mechanisms that underlie pruning.
"Algorithmic neuroscience is an approach to identify and use the
rules that structure brain function," Barth said. "There's a lot that
the brain can teach us about computing, and a lot that computer science
can do to help us understand how neural networks function."
As the birthplace of artificial intelligence and cognitive
psychology, Carnegie Mellon has been a leader in the study of brain and
behavior for more than 50 years. The university has created some of the
first cognitive tutors, helped to develop the Jeopardy-winning Watson,
founded a groundbreaking doctoral program in neural computation, and
completed cutting-edge work in understanding the genetics of autism.
Building on its strengths in biology,
computer science,
psychology, statistics and engineering, CMU recently launched
BrainHubSM, a global initiative that focuses on how the structure and
activity of the
brain give rise to complex behaviors.
Read more at:
http://phys.org/news/2015-07-neuroscience-based-algorithms-networks.html#jCp
The researchers
first determined key aspects of the pruning process by counting the
number of synapses present in a mouse model's somatosensory cortex over
time. After counting synapses in more than 10,000 electron microscopy
images, they found that synapses were rapidly pruned early in
development, and then as time progressed, the pruning rate slowed.
The results of these experiments allowed the team to develop an
algorithm for designing computational networks based on the brain
pruning approach. Using simulations and theoretical analysis they found
that the neuroscience-based algorithm produced networks were much more
efficient and robust than the current engineering methods.
In the networks created with pruning, the flow of information was
more direct, and provided multiple paths for information to reach the
same endpoint, which minimized the risk of network failure.
"We took this high-level algorithm that explains how neural
structures are built during development and used that to inspire an
algorithm for an engineered network," said Alison Barth, professor in
Carnegie Mellon's Department of Biological Sciences and member of the
university's BrainHubSM initiative. "It turns out that this
neuroscience-based approach could offer something new for computer
scientists and engineers to think about as they build networks."
As a test of how the algorithm could be used outside of neuroscience,
Saket Navlakha, assistant professor at the Salk Institute's Center for
Integrative Biology and a former postdoctoral researcher in Carnegie
Mellon's Machine Learning Department, applied the algorithm to flight
data from the U.S. Department of Transportation. He found that the
synaptic pruning-based algorithm created the most efficient and robust
routes to allow passengers to reach their destinations.
"We realize that it wouldn't be cost effective to apply this to
networks that require significant infrastructure, like railways or
pipelines," Navlakha said. "But for those that don't, like wireless
networks and sensor networks, this could be a valuable adaptive method
to guide the formation of networks."
In addition, the researchers say the work has implications for
neuroscience. Barth believes that the change in pruning rates from
adolescence to adulthood could indicate that there are different
biochemical mechanisms that underlie pruning.
"Algorithmic neuroscience is an approach to identify and use the
rules that structure brain function," Barth said. "There's a lot that
the brain can teach us about computing, and a lot that computer science
can do to help us understand how neural networks function."
As the birthplace of artificial intelligence and cognitive
psychology, Carnegie Mellon has been a leader in the study of brain and
behavior for more than 50 years. The university has created some of the
first cognitive tutors, helped to develop the Jeopardy-winning Watson,
founded a groundbreaking doctoral program in neural computation, and
completed cutting-edge work in understanding the genetics of autism.
Building on its strengths in biology,
computer science,
psychology, statistics and engineering, CMU recently launched
BrainHubSM, a global initiative that focuses on how the structure and
activity of the
brain give rise to complex behaviors.
Read more at:
http://phys.org/news/2015-07-neuroscience-based-algorithms-networks.html#jCp
