Abstract
In the past decade, we have witnessed the rise of deep learning to dominate
the field of artificial intelligence. Advances in artificial neural networks
alongside corresponding advances in hardware accelerators with large memory
capacity, together with the availability of large datasets enabled researchers
and practitioners alike to train and deploy sophisticated neural network models
that achieve state-of-the-art performance on tasks across several fields
spanning computer vision, natural language processing, and reinforcement
learning. However, as these neural networks become bigger, more complex, and
more widely used, fundamental problems with current deep learning models become
more apparent. State-of-the-art deep learning models are known to suffer from
issues that range from poor robustness, inability to adapt to novel task
settings, to requiring rigid and inflexible configuration assumptions. Ideas
from collective intelligence, in particular concepts from complex systems such
as self-organization, emergent behavior, swarm optimization, and cellular
systems tend to produce solutions that are robust, adaptable, and have less
rigid assumptions about the environment configuration. It is therefore natural
to see these ideas incorporated into newer deep learning methods. In this
review, we will provide a historical context of neural network research's
involvement with complex systems, and highlight several active areas in modern
deep learning research that incorporate the principles of collective
intelligence to advance its current capabilities. To facilitate a
bi-directional flow of ideas, we also discuss work that utilize modern deep
learning models to help advance complex systems research. We hope this review
can serve as a bridge between complex systems and deep learning communities to
facilitate the cross pollination of ideas and foster new collaborations across
disciplines.
Users
Please
log in to take part in the discussion (add own reviews or comments).