Sasha Rush on NLP research, engineering, and education

Tue, 04 Oct 2022 07:00:00 -0700

Coding puzzles, practices, and education, structured prediction, the culture of Hugging Face, large models, and the energy of New York.

Transcript: https://web.stanford.edu/class/cs224u/podcast/rush/

Sasha's website

Sasha on Twitter

Sasha on the Humans of AI podcast

Sasha on The Thesis Review Podcast with Sean Welleck

Sasha on the Talking Machines Podcast

Sasha interviewed by Sayak Paul

Hugging Face

PyTorch

The Annotated Transformer

The Annotated Alice

The Annotated S4

Sasha and Dan Oneață's declarative graphics library Chalk

Drawing Big Ben in Chalk

OpenNMT

Ken Shan

Blog post by Ken and Dylan Thurston

Edward Z. Yang

Stuart Shieber

Literate programming

Soumith Chintala

Lua Torch

TensorFlow

Graham Neubig

Chris Dyer

DyNet

JAX

jax.vmap

Matt Johnson

Finale Doshi-Velez, whose undergrad ML course inspired and informed Sasha's

Tensor Puzzles

GPU Puzzles

A tweet that Chris added to his CV

Adam Paszke

Dougal MacLaurin

Dex

Named Tensor notation

Named Tensors in PyTorch

TorchDim

Mini Torch

Torch-Struct

Sarah Hooker's paper 'The hardware lottery'

Jacob Andreas

Kevin Ellis

Hugging Face transformers library

Hugging Face datasets library

Hugging Face diffusers library

Hugging Face evaluate library

scikit-learn

Big Science blog

BLOOM

The Technology Behind BLOOM Training

CRFM

Eleuther

T0 and PromptSource

Washington Post: Big Tech builds AI with bad data. So scientists sought better data

The bet: Is Attention All You Need?

Democratizing access to large-scale language models with OPT-175B

Epic OPT-175 Logbook

Google's PaLM

United's shares plunge 76% on bogus bankruptcy report

Imagen

Albert Gu

Bell Labs