Stanford EE Computer Systems Colloquium

Available as Zoom Live Stream

May 18, 2022 4PM-6PM Pacific

live streamed, YouTube



Dennis Abts, Groq*

Oskar Mencer, Maxeler/Groq

Dataflow for convergence of AI and HPC - GroqChip!


This talk provides a journey through Dataflow history, arriving at the recent convergence of AI and HPC, with the novel Groq architecture, the tensor streaming processor (TSP), combining elements from traditional dataflow architectures, coupled with a powerful programming model built around producer-consumer stream programming. We describe the processing elements and their SIMD spatial microarchitecture capable of efficiently exploiting dataflow locality in deep learning models. Starting with over 400,000 arithmetic units, we describe the stream programming model with each on-chip functional unit (eg. vector processor, matrix unit, etc) are consuming tensor inputs from “streams” flowing on the chip, and producing output tensors that can be “chained” together to avoid writing intermediate results back to main memory. The combination of stream programming and dataflow locality along with deterministic execution provides the compiler with a simple abstraction of the underlying hardware components so that the compiler can orchestrate the arrival of input data (operands) with the instructions operating on them. We extend this simple “deterministic execution” model from a single chip to a distributed scale-out system that operates in lock-step across the entire distributed parallel computer, providing the illusion of a large single-core synchronous system. The Groq parallelizing compiler leverages this simple programming model to auto-scale the number of TSPs (processing elements) to grow with your deep learning models during the build-test-and-learn cycle to build robust numerical computations in production.

* Oskar Mencer contributed to this presentation and was prepared to step in if Dennis Abts were to be indisposed and unable to give the talk. We are, sad to say, still in the midst of a pandemic; health issues trump everything:w. Many thanks to everyone who worked to make this happen.