Timothy Dozat

I'm a Ph.D. student in the Linguistics Department at Stanford University. I work in the Natural Language Processing Group, advised by Chris Manning. I'm a member of the team that designed the first version of the Universal Dependencies standard and released the EWT dependency corpus. My dissertation is on building better neural dependency parsers, emphasizing simple architectures and statistically and empirically motivating additional complexity.

Email: [t+last name]@stanford.edu



  • Stanford's Graph-based Neural Dependency Parser at the CoNLL 2017 Shared Task
    Timothy Dozat, Peng Qi, Christopher D. Manning.
    CoNLL conference paper, 2017.
    Abstract: The neural dependency parser submitted by Stanford to the CoNLL 2017 Shared Task on parsing Universal Dependencies. Our system uses relatively simple LSTM networks to produce part of speech tags and labeled dependency parses from segmented and tokenized sequences of words. We include a character-based LSTM word representation in addition to pretrained and token-based representations. Our system was ranked first according to all relevant metrics for the system.
    [paper] [poster] [presentation] [code] [models]
  • Deep Biaffine Attention for Neural Dependency Parsing.
    Timothy Dozat, Christopher D. Manning.
    ICLR conference paper, 2017.
    Abstract: Builds off recent work in dependency parsing using neural attention in a simple graph-based dependency parser with biaffine classifiers. Our parser is the highest-performing graph-based parser on standard treebanks for six different languages, gettin SotA or near SotA performance on all of them. We also show which hyperparameter choices had a significant effect on parsing accuracy, allowing us to achieve large gains over similar approaches.
    [paper] [poster] [code] [model]
  • Incorporating Nesterov Momentum into Adam.
    Timothy Dozat.
    ICLR workshop paper, 2016.
    Abstract: Demonstrates how to simplify Nesterov momentum so that it can be straightforwardly incorporated into a minimally modified variant of the popular Adam optimizer, and supports the modification with empirical evidence from several different domains.
    [CS229 paper] [poster]
  • A Gold Standard Dependency Corpus for English.
    Natalia Silveira, Timothy Dozat, Marie-Catherine de Marneffe, Sam Bowman, Miriam Connor, John Bauer, Christopher Manning.
    LREC, 2014.
    Abstract: A gold standard annotation of syntactic dependencies in the English Web Treebank corpus using the Stanford Dependencies standard. We show that training a dependency parser on a mix of newswire and web data improves performance on the web data without greatly hurting performance on newswire text. In response to the challenges encountered by annotators in the EWT corpus, we revised and extended the Stanford Dependencies standard.
  • Universal Stanford dependencies: A cross-linguistics typology.
    Marie-Catherine de Marneffe, Natalia Silveira, Timothy Dozat, Christopher Manning.
    LREC, 2014.
    Abstract: An improved taxonomy to capture grammatical relations across languages that revists the now de facto standard Stanford dependency representation, including morphologically rich ones, emphasizing the lexicalist stance of Stanford Dependencies. We show how existing dependency schemes for several languages map onto the universal taxonomy proposed here and close with consideration of practical implications of dependency representation choices for NLP applications, in particular parsing.
  • More constructions, more genres: Extending Stanford Dependencies.
    Marie-Catherine de Marneffe, Miriam Connor, Natalia Silveira, Sam Bowman, Timothy Dozat, Christopher Manning.
    DepLing, 2013.
    Abstract: This addresses two limitations the Stanford Dependencies annotation scheme has suffered from: First, the scheme has not offered explicit analyses of more difficult syntactic constructions; second, it did not focus on constructions that are rare in newswire but very frequent in more informal texts. Here, we propose dependency analyses for several linguistically interesting constructions and extend the scheme to provide better coverage of modern web data.