We present a work-in-progress snapshot of learning with a 15 billion
parameter deep learning network on HPC architectures applied to the largest
publicly available natural image and video dataset released to-date. Recent
advancements in unsupervised deep neural networks suggest that scaling up such
networks in both model and training dataset size can yield significant
improvements in the learning of concepts at the highest layers. We train our
three-layer deep neural network on the Yahoo! Flickr Creative Commons 100M
dataset. The dataset comprises approximately 99.2 million images and 800,000
user-created videos from Yahoo's Flickr image and video sharing platform.
Training of our network takes eight days on 98 GPU nodes at the High
Performance Computing Center at Lawrence Livermore National Laboratory.
Encouraging preliminary results and future research directions are presented