Command Palette
Search for a command to run...
Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks
Kai Sheng Tai; Richard Socher; Christopher D. Manning
Abstract
Because of their superior ability to preserve sequence information over time, Long Short-Term Memory (LSTM) networks, a type of recurrent neural network with a more complex computational unit, have obtained strong results on a variety of sequence modeling tasks. The only underlying LSTM structure that has been explored so far is a linear chain. However, natural language exhibits syntactic properties that would naturally combine words to phrases. We introduce the Tree-LSTM, a generalization of LSTMs to tree-structured network topologies. Tree-LSTMs outperform all existing systems and strong LSTM baselines on two tasks: predicting the semantic relatedness of two sentences (SemEval 2014, Task 1) and sentiment classification (Stanford Sentiment Treebank).
Code Repositories
Benchmarks
| Benchmark | Methodology | Metrics |
|---|---|---|
| semantic-similarity-on-sick | Bidirectional LSTM (Tai et al., 2015) | MSE: 0.2736 Pearson Correlation: 0.8567 Spearman Correlation: 0.7966 |
| semantic-similarity-on-sick | LSTM (Tai et al., 2015) | MSE: 0.2831 Pearson Correlation: 0.8528 Spearman Correlation: 0.7911 |
| semantic-similarity-on-sick | Dependency Tree-LSTM (Tai et al., 2015) | MSE: 0.2532 Pearson Correlation: 0.8676 Spearman Correlation: 0.8083 |
| sentiment-analysis-on-sst-2-binary | 2-layer LSTM [tai2015improved] | Accuracy: 86.3 |
| sentiment-analysis-on-sst-2-binary | Consistency Tree LSTM with tuned Glove vectors [tai2015improved] | Accuracy: 88.0 |
| sentiment-analysis-on-sst-5-fine-grained | Constituency Tree-LSTM | Accuracy: 51.0 |
Build AI with AI
From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.