bookmarks (hide)59
- In this article, I am going to show you how to choose the number of principal components when using principal component analysis for dimensionality reduction. In the first section, I am going to give you a short answer for those of you who are in a hurry and want to get something working. Later, I am going to provide a more extended explanation for those of you who are interested in understanding PCA.
- Perplexity is a useful metric to evaluate models in Natural Language Processing (NLP). This article will cover the two ways in which it is normally defined and the intuitions behind them. A language…
- At the very beginning of the tutorial, I’ll explain the dimensionality of a dataset, what dimensionality reduction means, the main approaches to dimensionality reduction, the reasons for dimensionality reduction and what PCA means. Then, I will go deeper into the topic of PCA by implementing the PCA algorithm with the Scikit-learn machine learning library. This will help you to easily apply PCA to a real-world dataset and get results very fast.
- As data scientists, we spend a lot of our time doing exploratory data analysis (EDA), cleaning data and making sure the data we use to generate insights is of good quality. Have you ever found…
- Why are single-shot methods like SSD and YOLO less accurate than region-proposal methods like R-CNN and how does focal loss fix this problem?
- When the downstream applications only care about the direction of the word vectors (e.g. they only pay attention to the cosine similarity of two words), then normalize, and forget about length. However, if the downstream applications are able to (or need to) consider more sensible aspects, such as word significance, or consistency in word usage (see below), then normalization might not be such a good idea.
- Comparing machine learning methods and selecting a final model is a common operation in applied machine learning. Models are commonly evaluated using resampling methods like k-fold cross-validation from which mean skill scores are calculated and compared directly. Although simple, this approach can be misleading as it is hard to know whether the difference between mean skill scores is real or the result of a statistical fluke.
- There are currently few datasets appropriate for training and evaluating models for non-goal-oriented dialogue systems (chatbots); and equally problematic, there is currently no standard procedure for evaluating such models beyond the classic Turing test. The aim of our competition is therefore to establish a concrete scenario for testing chatbots that aim to engage humans, and become a standard evaluation tool in order to make such systems directly comparable.
- I have been working with LangChain applications for quite a while now and as you might know there is always something new to learn in the GenAI universe. So a couple of weeks ago I was going through…
- One of the key enablers of the ChatGPT magic can be traced back to 2017 under the obscure name of reinforcement learning with human feedback(RLHF). Large language models(LLMs) have become one of the most interesting environments for applying modern reinforcement learning(RL) techniques. While LLMs are great at deriving knowledge from vast amounts of text, RL can help to translate that knowledge into actions. That has been the secret behind RLHF.
- Definition of NLP coherence scores, in particular intrinsic UMass measure and PMI. Human judgment not being correlated to perplexity (or likelihood of unseen documents) is the motivation for more work trying to model the human judgment. This is by itself a hard task as human judgment is not clearly defined; for example, two experts can disagree on the usefulness of a topic. One can classify the methods addressing this problem into two categories. \textit{Intrinsic} methods that do not use any external source or task from the dataset, whereas \textit{extrinsic} methods use the discovered topics for external tasks, such as information retrieval [Wei06], or use external statistics to evaluate topics.
- In this article, we will explore how we can use Llama2 for Topic Modeling without the need to pass every single document to the model. Instead, we will leverage BERTopic, a modular topic modeling technique that can use any LLM for fine-tuning topic representations.
- In natural language understanding (NLU) tasks, there is a hierarchy of lenses through which we can extract meaning — from words to sentences to paragraphs to documents. At the document level, one of the most useful ways to understand text is by analyzing its topics. The process of learning, recognizing, and extracting these topics across a collection of documents is called topic modeling. In this post, we will explore topic modeling through 4 of the most popular techniques today: LSA, pLSA, LDA, and the newer, deep learning-based lda2vec.
- Uniform Manifold Approximation and Projection (UMAP) is a dimension reduction technique that can be used for visualisation similarly to t-SNE, but also for general non-linear dimension reduction. The algorithm is founded on three assumptions about the data The data is uniformly distributed on Riemannian manifold; The Riemannian metric is locally constant (or can be approximated as such); The manifold is locally connected. From these assumptions it is possible to model the manifold with a fuzzy topological structure. The embedding is found by searching for a low dimensional projection of the data that has the closest possible equivalent fuzzy topological structure.
- We introduce Vicuna-13B, an open-source chatbot trained by fine-tuning LLaMA on user-shared conversations collected from ShareGPT. Preliminary evaluation using GPT-4 as a judge shows Vicuna-13B achieves more than 90%* quality of OpenAI ChatGPT and Google Bard while outperforming other models like LLaMA and Stanford Alpaca in more than 90%* of cases. The cost of training Vicuna-13B is around $300. The code and weights, along with an online demo, are publicly available for non-commercial use.
- In mathematics, the Wasserstein or Kantorovich–Rubinstein metric or distance is a distance function defined between probability distributions on a given metric space M {\displaystyle M} M. Intuitively, if each distribution is viewed as a unit amount of "dirt" piled on M {\displaystyle M} M, the metric is the minimum "cost" of turning one pile into the other, which is assumed to be the amount of dirt that needs to be moved times the mean distance it has to be moved. Because of this analogy, the metric is known in computer science as the earth mover's distance.
- In recent years, the real-world impact of machine learning (ML) has grown in leaps and bounds. In large part, this is due to the advent of deep learning models, which allow practitioners to get state-of-the-art scores on benchmark datasets without any hand-engineered features. Given the availability of multiple open-source ML frameworks like TensorFlow and PyTorch, and an abundance of available state-of-the-art models, it can be argued that high-quality ML models are almost a commoditized resource now. There is a hidden catch, however: the reliance of these models on massive sets of hand-labeled training data. These hand-labeled training sets are expensive and time-consuming to create — often requiring person-months or years to assemble, clean, and debug — especially when domain expertise is required. On top of this, tasks often change and evolve in the real world. For example, labeling guidelines, granularities, or downstream use cases often change, necessitating re-labeling (e.g., instead of classifying reviews only as positive or negative, introducing a neutral category). For all these reasons, practitioners have increasingly been turning to weaker forms of supervision, such as heuristically generating training data with external knowledge bases, patterns/rules, or other classifiers. Essentially, these are all ways of programmatically generating training data—or, more succinctly, programming training data. We begin by reviewing areas of ML that are motivated by the problem of labeling training data, and then describe our research on modeling and integrating a diverse set of supervision sources. We also discuss our vision for building data management systems for the massively multi-task regime with tens or hundreds of weakly supervised dynamic tasks interacting in complex and varied ways. Check out the our research blog for detailed discussions of these topics and more!
- Basic knowledge of XLNet to understand the difference between XLNet and BERT intuitively
- When a word appears in different contexts, its vector gets moved in different directions during updates. The final vector then represents some sort of weighted average over the various contexts. Averaging over vectors that point in different directions typically results in a vector that gets shorter with increasing number of different contexts in which the word appears. For words to be used in many different contexts, they must carry little meaning. Prime examples of such insignificant words are high-frequency stop words, which are indeed represented by short vectors despite their high term frequencies ...
publications (hide)166
- (2023)
- Annals of Applied Statistics, (2007)
- Proceedings of the Association for Computational Linguistics (ACL), page 271--278. Association for Computational Linguistics, (2004)
- (2017)cite arxiv:1706.04902.
- ACM Computing Surveys (CSUR), 54 (6): 1--35 (2021)
- Proceedings of the 2004 SIAM International Conference on Data Mining, page 333--344. Lake Buena Vista, FL, Society for Industrial and Applied Mathematics, (April 2004)
- (June 2023)
- CoRR, (2014)
- COLT, page 257-269. Omnipress, (2010)
- volume 1112 of Communications in Computer and Information Science, Springer, (2019)
- IEEE Trans. Med. Imaging, 35 (5): 1313-1321 (2016)
- IEEE Trans. Med. Imaging, 35 (5): 1313-1321 (2016)
- ACL, page 299-309. Association for Computational Linguistics, (2017)
- Leeds Metropolitan University, (April 2007)
- Koli Calling, page 40:1-40:2. ACM, (2020)
- ACL (1), page 388-397. Association for Computational Linguistics, (2017)
- 2015 IEEE Frontiers in Education Conference (FIE), page 1-9. IEEE Computer Society, (2015)
- (2019)cite arxiv:1905.09418Comment: ACL 2019 (camera-ready).
- NAACL-HLT (1), abs/1904.01596, page 2970-3005. Association for Computational Linguistics, (2019)
- Language resources and evaluation, 39 (2-3): 165-210 (2005)
- LAW@COLING, page 120-128. The Association for Computer Linguistics, (2014)
- PLOS ONE, 17 (9): 1-34 (September 2022)
- APSIPA, IEEE, (December 2014)
- LREC, European Language Resources Association (ELRA), (2016)
- DH, page 121-125. ACM, (2018)
- Proceedings of the 2019 SIAM International Conference on Data Mining, Society for Industrial and Applied Mathematics, (May 2019)
- ENTER, page 365-380. Springer, (2018)
- (2023)
- (2023)
- AAAI Workshops, volume WS-18 of AAAI Workshops, page 731-734. AAAI Press, (2018)
- Neurocomputing, (2017)
- NIPS, page 1800-1808. (2011)
- arXiv preprint arXiv:1810.04805, (2018)
- ECML/PKDD (2), volume 11907 of Lecture Notes in Computer Science, page 681-696. Springer, (2019)
- CoRR, (2020)
- EMNLP '08: Proceedings of the Conference on Empirical Methods in Natural Language Processing, page 254--263. Morristown, NJ, USA, Association for Computational Linguistics, (2008)
- EMNLP '08: Proceedings of the Conference on Empirical Methods in Natural Language Processing, page 254--263. Morristown, NJ, USA, Association for Computational Linguistics, (2008)
- IEEE Trans. Vis. Comput. Graph., 24 (1): 142-151 (2018)
- EMNLP, page 830-836. Association for Computational Linguistics, (2018)
- Omnipress, (2010)
- Conference Proceedings of "3rd International Conference on Advanced Research Methods and Analytics", page 211-219. Valencia, Spain, Editorial Universitat Politècnica de València, (July 2020)
- WWW, page 155-164. ACM, (2014)
- Number 3 Article, 16 (3): 5--6 (2011)
- (2018)Online; accessed 29-June-2021.
- Knowledge Representation for Health Care Process-Oriented Information Systems in Health Care Extraction & Processing of Rich Semantics from Medical Texts, (2017)
- Proceedings of the Conference on Natural Language Processing (KONVENS), page (89-99). Vienna, Austria, (2018)
- Advances in Neural Information Processing Systems 18, 18, page 147. Cambridge, MA, MIT; 1998, (2006)
- Universitat Pompeu Fabra, (2019)
- Journal of Marketing Management, 31 (9-10): 1141--1157 (2015)
- Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), page 133--142. (2019)
