Efficient Recurrent Neural Networks using Structured Matrices in FPGAs
Z. Li, S. Wang, C. Ding, Q. Qiu, Y. Wang, and Y. Liang. (2018)cite arxiv:1803.07661Comment: To appear in International Conference on Learning Representations 2018 Workshop Track.
Abstract
Recurrent Neural Networks (RNNs) are becoming increasingly important for time
series-related applications which require efficient and real-time
implementations. The recent pruning based work ESE suffers from degradation of
performance/energy efficiency due to the irregular network structure after
pruning. We propose block-circulant matrices for weight matrix representation
in RNNs, thereby achieving simultaneous model compression and acceleration. We
aim to implement RNNs in FPGA with highest performance and energy efficiency,
with certain accuracy requirement (negligible accuracy degradation).
Experimental results on actual FPGA deployments shows that the proposed
framework achieves a maximum energy efficiency improvement of 35.7$\times$
compared with ESE.
Description
Efficient Recurrent Neural Networks using Structured Matrices in FPGAs
%0 Generic
%1 li2018efficient
%A Li, Zhe
%A Wang, Shuo
%A Ding, Caiwen
%A Qiu, Qinru
%A Wang, Yanzhi
%A Liang, Yun
%D 2018
%K FPGA
%T Efficient Recurrent Neural Networks using Structured Matrices in FPGAs
%U http://arxiv.org/abs/1803.07661
%X Recurrent Neural Networks (RNNs) are becoming increasingly important for time
series-related applications which require efficient and real-time
implementations. The recent pruning based work ESE suffers from degradation of
performance/energy efficiency due to the irregular network structure after
pruning. We propose block-circulant matrices for weight matrix representation
in RNNs, thereby achieving simultaneous model compression and acceleration. We
aim to implement RNNs in FPGA with highest performance and energy efficiency,
with certain accuracy requirement (negligible accuracy degradation).
Experimental results on actual FPGA deployments shows that the proposed
framework achieves a maximum energy efficiency improvement of 35.7$\times$
compared with ESE.
@misc{li2018efficient,
abstract = {Recurrent Neural Networks (RNNs) are becoming increasingly important for time
series-related applications which require efficient and real-time
implementations. The recent pruning based work ESE suffers from degradation of
performance/energy efficiency due to the irregular network structure after
pruning. We propose block-circulant matrices for weight matrix representation
in RNNs, thereby achieving simultaneous model compression and acceleration. We
aim to implement RNNs in FPGA with highest performance and energy efficiency,
with certain accuracy requirement (negligible accuracy degradation).
Experimental results on actual FPGA deployments shows that the proposed
framework achieves a maximum energy efficiency improvement of 35.7$\times$
compared with ESE.},
added-at = {2018-03-22T08:36:53.000+0100},
author = {Li, Zhe and Wang, Shuo and Ding, Caiwen and Qiu, Qinru and Wang, Yanzhi and Liang, Yun},
biburl = {https://www.bibsonomy.org/bibtex/2d040bbcd5b50384f64d84fea78e580da/jk_itwm},
description = {Efficient Recurrent Neural Networks using Structured Matrices in FPGAs},
interhash = {ac89790ebb0e5e0d6d0cd0cf002b30b7},
intrahash = {d040bbcd5b50384f64d84fea78e580da},
keywords = {FPGA},
note = {cite arxiv:1803.07661Comment: To appear in International Conference on Learning Representations 2018 Workshop Track},
timestamp = {2018-03-22T08:36:53.000+0100},
title = {Efficient Recurrent Neural Networks using Structured Matrices in FPGAs},
url = {http://arxiv.org/abs/1803.07661},
year = 2018
}