Misc,
Statistical field theory for neural networks
(2019)cite arxiv:1901.10416.
Abstract
These notes attempt a self-contained introduction into statistical field
theory applied to neural networks of rate units and binary spins. The
presentation consists of three parts: First, the introduction of fundamental
notions of probabilities, moments, cumulants, and their relation by the linked
cluster theorem, of which Wick's theorem is the most important special case;
followed by the diagrammatic formulation of perturbation theory, reviewed in
the statistical setting. Second, dynamics described by stochastic differential
equations in the Ito-formulation, treated in the Martin-Siggia-Rose-De
Dominicis-Janssen path integral formalism. With concepts from disordered
systems, we then study networks with random connectivity and derive their
self-consistent dynamic mean-field theory, explaining the statistics of
fluctuations and the emergence of different phases with regular and chaotic
dynamics. Third, we introduce the effective action, vertex functions, and the
loopwise expansion. These tools are illustrated by systematic derivations of
self-consistency equations, going beyond the mean-field approximation. These
methods are applied to the pairwise maximum entropy (Ising spin) model,
including the recently-found diagrammatic derivation of the
Thouless-Anderson-Palmer mean field theory.
