We consider the problem of distributed feature quantization, where the goal
is to enable a pretrained classifier at a central node to carry out its
classification on features that are gathered from distributed nodes through
communication constrained channels. We propose the design of distributed
quantization schemes specifically tailored to the classification task: unlike
quantization schemes that help the central node reconstruct the original signal
as accurately as possible, our focus is not reconstruction accuracy, but
instead correct classification. Our work does not make any apriori
distributional assumptions on the data, but instead uses training data for the
quantizer design. Our main contributions include: we prove NP-hardness of
finding optimal quantizers in the general case; we design an optimal scheme for
a special case; we propose quantization algorithms, that leverage discrete
neural representations and training data, and can be designed in
polynomial-time for any number of features, any number of classes, and
arbitrary division of features across the distributed nodes. We find that
tailoring the quantizers to the classification task can offer significant
savings: as compared to alternatives, we can achieve more than a factor of two
reduction in terms of the number of bits communicated, for the same
classification accuracy.
Description
[1911.00216] On Distributed Quantization for Classification
%0 Generic
%1 hanna2019distributed
%A Hanna, Osama A.
%A Ezzeldin, Yahya H.
%A Sadjadpour, Tara
%A Fragouli, Christina
%A Diggavi, Suhas
%D 2019
%K classification machinelearning
%T On Distributed Quantization for Classification
%U http://arxiv.org/abs/1911.00216
%X We consider the problem of distributed feature quantization, where the goal
is to enable a pretrained classifier at a central node to carry out its
classification on features that are gathered from distributed nodes through
communication constrained channels. We propose the design of distributed
quantization schemes specifically tailored to the classification task: unlike
quantization schemes that help the central node reconstruct the original signal
as accurately as possible, our focus is not reconstruction accuracy, but
instead correct classification. Our work does not make any apriori
distributional assumptions on the data, but instead uses training data for the
quantizer design. Our main contributions include: we prove NP-hardness of
finding optimal quantizers in the general case; we design an optimal scheme for
a special case; we propose quantization algorithms, that leverage discrete
neural representations and training data, and can be designed in
polynomial-time for any number of features, any number of classes, and
arbitrary division of features across the distributed nodes. We find that
tailoring the quantizers to the classification task can offer significant
savings: as compared to alternatives, we can achieve more than a factor of two
reduction in terms of the number of bits communicated, for the same
classification accuracy.
@misc{hanna2019distributed,
abstract = {We consider the problem of distributed feature quantization, where the goal
is to enable a pretrained classifier at a central node to carry out its
classification on features that are gathered from distributed nodes through
communication constrained channels. We propose the design of distributed
quantization schemes specifically tailored to the classification task: unlike
quantization schemes that help the central node reconstruct the original signal
as accurately as possible, our focus is not reconstruction accuracy, but
instead correct classification. Our work does not make any apriori
distributional assumptions on the data, but instead uses training data for the
quantizer design. Our main contributions include: we prove NP-hardness of
finding optimal quantizers in the general case; we design an optimal scheme for
a special case; we propose quantization algorithms, that leverage discrete
neural representations and training data, and can be designed in
polynomial-time for any number of features, any number of classes, and
arbitrary division of features across the distributed nodes. We find that
tailoring the quantizers to the classification task can offer significant
savings: as compared to alternatives, we can achieve more than a factor of two
reduction in terms of the number of bits communicated, for the same
classification accuracy.},
added-at = {2019-11-04T18:54:38.000+0100},
author = {Hanna, Osama A. and Ezzeldin, Yahya H. and Sadjadpour, Tara and Fragouli, Christina and Diggavi, Suhas},
biburl = {https://www.bibsonomy.org/bibtex/2a5ff2639a489f403926e2d7c54df7272/cpankow},
description = {[1911.00216] On Distributed Quantization for Classification},
interhash = {65e5d1679838c810c10776b3ad7ee454},
intrahash = {a5ff2639a489f403926e2d7c54df7272},
keywords = {classification machinelearning},
note = {cite arxiv:1911.00216},
timestamp = {2019-11-04T18:54:38.000+0100},
title = {On Distributed Quantization for Classification},
url = {http://arxiv.org/abs/1911.00216},
year = 2019
}