Hello! I'm Ben Newman, a current senior at Stanford studying Computer Science working with the Stanford NLP group and the Stanford Internet Observatory. My interests include in computer science, cognitive science, linguistics, education, and misinformation.
I'm interested in understanding how NLP systems learn and process language and the role that our systems play in society at large. I've worked on projects analyzing models' abilities to extrapolate, evaluating their ability to communicate, and tracking misinformation in the run-up to the 2020 election. At Stanford Splash I co-teach courses in Introductory Linguistics and Computing Fundamentals.
Refining Targeted Syntactic Evaluation
Benjamin Newman, Kai Siang-Ang, Julia Gong, John Hewitt
How should we evaluate the syntactic understanding of our NLP models? We build off of a body of work that uses minimal pair for evaluation and argue that we should be evaluating models' likely behavior and systematicty. We adapt minimal pair evaluation to address these goals, finding the models prefentially conjugate verbs they deem likely.
The EOS Decision and Length Extrapolation
Benjamin Newman, John Hewitt, Percy Liang and Chris Manning
Blackbox NLP@EMNLP 2020 (Outstanding Paper)
Why do sequence models struggle to extrapolate? For many reasons, but the decision to train models with End of Sequence tokens at the end of training examples is one of them. We investigate and visualize the effect that this decision has on neural models' extrapolative abilities.
Communication-based Evaluation for Natural Language Generation
Benjamin Newman, Reuben Cohn-Gordon, and Christopher Potts
Society for Computation in Linguistics@LSA 2020
Do n-gram overlap metrics like BLEU capture whether the models are successful communicators? Not really, so we created our own way of evaluating communicative effectiveness based on the Rational Speech Acts framework.
Conducted during CS224U and the Center for the Study of Language and Information (CSLI) summer internship.
A coalition of research groups that was tracking misinformation in the run-up to the 2020 US election. [site]
Representations from pretrained language models likely incorporate syntax, but can we access it without training supervised probes? [pdf]
CS229: Machine Learning. Final Project (2019).
What we can learn about people's use of swears by looking at their word2vec and GLOVE embeddings? [pdf]
Hiding secret messages in HTML zero-width space characters. Demo here!