About Word embeddings: Difference between revisions
From Algolit
(Created page with "{| |- | Type: || Algoliterary exploration |- | Technique: || word-embeddings |- | Developed by: || Algolit |} "''Meaning is this illusive thing that were trying to capture''"...") |
|||
| Line 11: | Line 11: | ||
<br> | <br> | ||
| − | Word embeddings are used to represent words as inputs to machine learning. The words become vectors in a multi-dimensional space, where nearby vectors represent similar meanings. With word embeddings, you can compare words by (roughly) what they mean, not just exact | + | Word embeddings are used to represent words as inputs to machine learning. The words become vectors in a multi-dimensional space, where nearby vectors represent similar meanings. With word embeddings, you can compare words by (roughly) what they mean, not just exact letter matches. |
| − | Successfully training word vectors requires starting from hundreds of gigabytes of input text. Fortunately, various machine-learning groups have already done this and provided pre-trained word embeddings that one can download. Two very well-known datasets of pre-trained English word embeddings are word2vec, pre-trained on Google News data, and [http://www.algolit.net/index.php/The_GloVe_Reader GloVe], pre-trained on the [http://www.algolit.net/index.php/Common_Crawl Common Crawl] of web pages. | + | Successfully training word vectors requires starting from hundreds of gigabytes of input text. Fortunately, various machine-learning groups have already done this and provided pre-trained word embeddings that one can download. Two very well-known datasets of pre-trained English word embeddings and stand alone implementations are word2vec ((a single layered neural network), pre-trained on Google News data, and [http://www.algolit.net/index.php/The_GloVe_Reader GloVe] (distributional semantic model), pre-trained on the [http://www.algolit.net/index.php/Common_Crawl Common Crawl] of web pages. |
The term has only recently entered the vocabulary of machine learning, with the expansion of the deep learning community. In computational linguistics the expression 'distributional semantic model' is sometimes preferred. Other terms include 'distributed representation', 'semantic vector space', or 'word space'. | The term has only recently entered the vocabulary of machine learning, with the expansion of the deep learning community. In computational linguistics the expression 'distributional semantic model' is sometimes preferred. Other terms include 'distributed representation', 'semantic vector space', or 'word space'. | ||
| − | |||
| − | |||
Revision as of 16:12, 25 October 2017
| Type: | Algoliterary exploration |
| Technique: | word-embeddings |
| Developed by: | Algolit |
"Meaning is this illusive thing that were trying to capture" (Richard Socher in CS224D Lecture 2 - 31st Mar 2016 (Youtube))
Word embeddings are used to represent words as inputs to machine learning. The words become vectors in a multi-dimensional space, where nearby vectors represent similar meanings. With word embeddings, you can compare words by (roughly) what they mean, not just exact letter matches.
Successfully training word vectors requires starting from hundreds of gigabytes of input text. Fortunately, various machine-learning groups have already done this and provided pre-trained word embeddings that one can download. Two very well-known datasets of pre-trained English word embeddings and stand alone implementations are word2vec ((a single layered neural network), pre-trained on Google News data, and GloVe (distributional semantic model), pre-trained on the Common Crawl of web pages.
The term has only recently entered the vocabulary of machine learning, with the expansion of the deep learning community. In computational linguistics the expression 'distributional semantic model' is sometimes preferred. Other terms include 'distributed representation', 'semantic vector space', or 'word space'.