Ben Newman

blnewman at stanford.edu

Github | Resume | Google Scholar

Hello! I'm Ben Newman, a current senior at Stanford studying Computer Science working with the Stanford NLP group. My interests include in computer science, cognitive science, linguistics, education, and misinformation.

I'm interested in understanding how NLP systems learn and process language and the role that our systems play in society at large. I've worked on projects analyzing models' abilities to extrapolate, evaluating their ability to communicate, and tracking misinformation in the run-up to the 2020 election. At Stanford Splash I co-teach courses in Introductory Linguistics and Computing Fundamentals.

Publications

Refining Targeted Syntactic Evaluation

Benjamin Newman, Kai Siang-Ang, Julia Gong, John Hewitt

NAACL 2021

[pdf] [code] [cite] [blog]


How should we evaluate the syntactic understanding of our NLP models? We build off of a body of work that uses minimal pair for evaluation and argue that we should be evaluating models' likely behavior and systematicty. We adapt minimal pair evaluation to address these goals, finding the models prefentially conjugate verbs they deem likely.

The EOS Decision and Length Extrapolation

Benjamin Newman, John Hewitt, Percy Liang and Chris Manning

Blackbox NLP@EMNLP 2020 (Outstanding Paper)

[pdf] [code] [cite]


Why do sequence models struggle to extrapolate? For many reasons, but the decision to train models with End of Sequence tokens at the end of training examples is one of them. We investigate and visualize the effect that this decision has on neural models' extrapolative abilities.

Communication-based Evaluation for Natural Language Generation

Benjamin Newman, Reuben Cohn-Gordon, and Christopher Potts

Society for Computation in Linguistics@LSA 2020

[pdf] [code] [cite]


Do n-gram overlap metrics like BLEU capture whether the models are successful communicators? Not really, so we created our own way of evaluating communicative effectiveness based on the Rational Speech Acts framework.

Conducted during CS224U and the Center for the Study of Language and Information (CSLI) summer internship.

Projects

EIP: Election Integrity Partnership

A coalition of research groups that was tracking misinformation in the run-up to the 2020 US election. [site]

Unsupervised Recovery of Tree Metrics from Learned Representations

Representations from pretrained language models likely incorporate syntax, but can we access it without training supervised probes? [pdf]

CS229: Machine Learning. Final Project (2019).

English-Chinese Name Machine Transliteration Using Search and Neural Networks

What's your name in Chinese? Name translations differ from standard MT as they are based in phoenetics and lack large corpera. We explore two approaches here. [pdf] [code]

CS221: Artificial Intelligence: Principles and Techniques: Final Project with Julia Gong (2018).

Using POMDPs to Learn Language in a Spatial Reference Game

How can you teach computational agents to follow directions without defining what each instruction means? POMDPs! [pdf] [code]

CS238: Decision Making under Uncertainty: Final Project with Suvir Mirchandani and Levi Lian (2018)

Swear Analysis

What we can learn about people's use of swears by looking at their word2vec and GLOVE embeddings? [pdf]

Linguist 130A: Semantics and Pragmatics: Final Project with Julia Gong (2018)

Zero Width Space Encrypter

Hiding secret messages in HTML zero-width space characters. Demo here!

Stanford Student Space Initiative (SSI)

Class Notes

Stanford Splash Slides: In a Manner of Speaking...
LINGUIST 21N: Linguistic Diversity and Universals

Contact

blnewman@stanford.edu