Syntax

Learning is about generalization. Language learning is about generalization to sentences not seen before. One aspect of this is syntacic generalization,

Papers:

  • Representations of Syntax [MASK] Useful: Effects of Constituency and Dependency Structure in Recursive LSTMs
  • Overestimation of Syntactic Representation in Neural Language Models
  • Syntactic Data Augmentation Increases Robustness to Inference Heuristics
  • A Systematic Assessment of Syntactic Generalization in Neural Language Models
  • Does Syntax Need to Grow on Trees? Sources of Hierarchical Inductive Bias in Sequence-to-Sequence Networks
Paper Task Metric Models Findings
R verb-object agreement accuracy sequence & tree LSTMs Tree structure & augmentation help
O syntactic priming perplexity change sequence LSTM, n-gram Task too easy for baselines
S entailment inference accuracy BERT Augmentation helps
A 6 tasks accuracy GPTs, sequence RNNs, n-gram… GPTs > RNNs, model > data
D question formulation accuracy (first word) sequence & tree RNNs Tree structure helps, not augmentation*
Header1 Header2 Header3
cell1 cell2 cell3
cell4 cell5 cell6
cell1 cell2 cell3
cell4 cell5 cell6
Foot1 Foot2 Foot3

My takes

def what?
  42
end
Name Lunch order Spicy Owes
Joan saag paneer medium $11
Sally vindaloo mild $14
Erin lamb madras HOT $5

There are multiple syntax highlighting themes to choose from. Here’s one of them:

// All the code you will ever need
var hw = "Hello World!"
alert(hw);

My math is so rusty that I barely remember the quadratic equation: $-b \pm \sqrt{b^2 - 4ac} \over 2a$

Written on July 7, 2020