Distributed Representations of Words and Phrases and their Compositionality
, , , , and .
Advances in neural information processing systems, page 3111--3119. (2013)

The recently introduced continuous Skip-gram model is an efficient method for learning high-quality distributed vector representations that capture a large number of precise syntactic and semantic word relationships. In this paper we present several improvements that make the Skip-gram model more expressive and enable it to learn higher quality vectors more rapidly. We show that by subsampling frequent words we obtain significant speedup, and also learn higher quality representations as measured by our tasks. We also introduce Negative Sampling, a simplified variant of Noise Contrastive Estimation (NCE) that learns more accurate vectors for frequent words compared to the hierarchical softmax. An inherent limitation of word representations is their indifference to word order and their inability to represent idiomatic phrases. For example, the meanings of Canada''and" Air''cannot be easily combined to obtain" Air Canada''. Motivated by this example, we present a simple and efficient method for finding phrases, and show that their vector representations can be accurately learned by the Skip-gram model."
  • @schwemmlein
  • @becker
  • @ghagerer
  • @thoni
  • @scheuerpflug
  • @albinzehe
  • @stdiff
  • @dblp
  • @sxkdz
  • @mnoukhov
  • @jaeschke
  • @ven7u
  • @dallmann
  • @kde-alumni
  • @wangxinxi
  • @tobias.koopmann
  • @tengmf
  • @hauptmann
  • @tschumacher
  • @hotho
  • @benwesterath
This publication has not been reviewed yet.

rating distribution
average user rating0.0 out of 5.0 based on 0 reviews
    Please log in to take part in the discussion (add own reviews or comments).