John C. Duchi

John C. Duchi


A little about me: I am an assistant professor of Statistics and Electrical Engineering at Stanford University. I completed my PhD in computer science at Berkeley in 2014. My research interests are a bit eclectic, and they span computation, statistics, optimization, and machine learning; if you like any of these, we can probably find something interesting to chat about. At Berkeley, I worked in the Statistical Artificial Intelligence Lab (SAIL) under the joint supervision of Michael Jordan and Martin Wainwright. I obtained my master's degree (MA) in statistics in Fall 2012. I was also an undergrad and a masters student at Stanford University, where I worked with Daphne Koller in her research group, DAGS. I also spend some time at Google Research, where I had (and continue to have) the great fortune to work with Yoram Singer. (Here is a slightly more formal bio in the third-person.)

Curriculum Vitae: [pdf]

Contact info: [Visit]

Please note: regretfully, I am unable to respond to most inquiries regarding openings for graduate and postdoctoral positions in my group. Admissions to Stanford are handled at a department-wide level, not by me individually, so I am unable to comment on your suitability for graduate school or work with me. If you are already a Stanford student or have been admitted to Stanford, feel free to contact me about interests we may share.


Teaching

CS/Stats 229: Machine learning (Spring 2016)

Statistics 311/Electrical Engineering 377: Information Theory and Statistics (Winter 2016)

Electrical Engineering 364b: Convex Optimization II (Spring 2015)

Statistics 311/Electrical Engineering 377: Information Theory and Statistics (Fall 2014)


Publications

Clicking the publication title will give an abstract and publication information.

Preprints/In Preparation

Statistics of Robust Optimization: a Generalized Empirical Likelihood Approach, John Duchi, Peter Glynn, Hongseok Namkoong. [pdf]

Local Minimax Complexity of Stochastic Convex Optimization, Yuancheng Zhu, Sabyasachi Chatterjee, John C. Duchi, John Lafferty. [pdf]

Minimax Optimal Procedures for Locally Private Estimation, John C. Duchi, Michael I. Jordan, Martin J. Wainwright. [pdf]

Information Measures, Experiments, Multi-category Hypothesis Tests, and Surrogate Losses, John C. Duchi, Khashayar Khosravi, and Feng (Frank) Ruan. [pdf]

Asynchronous Stochastic Convex Optimization, John C. Duchi and Sorathan Chaturapruek. [pdf]

Privacy and Statistical Risk: Formalisms and Minimax Bounds, Rina F. Barber and John C. Duchi. [pdf]

Distance-based and continuum Fano inequalities with applications to statistical estimation, John C. Duchi, and Martin J. Wainwright. [pdf]

Oracle Inequalities for Computationally Adaptive Model Selection, Alekh Agarwal, Peter L. Bartlett, and John C. Duchi. [pdf]

Books, book chapters, and lecture notes

Introductory Lectures on Stochastic Convex Optimization, John C. Duchi. Park City Mathematics Institute, Graduate Summer School Lectures, July 2016.

Information Theory and Statistics, John C. Duchi. Lecture Notes for Statistics 311/Electrical Engineering 377, Stanford University. 2015.

Journal Articles

Optimal rates for zero-order optimization: the power of two function evaluations, John C. Duchi, Michael I. Jordan, Martin J. Wainwright, and Andre Wibisono. IEEE Transactions on Information Theory 61(5): 2788--2806, 2015. [pdf, slides]

Divide and Conquer Kernel Ridge Regression: A Distributed Algorithm with Minimax Optimal Rates, Yuchen Zhang, John C. Duchi, and Martin J. Wainwright. Journal of Machine Learning Research (to appear), 2015. [pdf]

Privacy Aware Learning, John C. Duchi, Michael I. Jordan, and Martin J. Wainwright. Journal of the Association for Computing Machinery, 2014, to appear. [pdf]

The Asymptotics of Ranking Algorithms, John C. Duchi, Lester Mackey, Michael I. Jordan. Annals of Statistics 41(5):2292--2323, 2013. [pdf]

Communication-Efficient Algorithms for Statistical Optimization, Yuchen Zhang, John C. Duchi, and Martin Wainwright. Journal of Machine Learning Research 14(Nov):3321--3363, 2013. [pdf]

The Generalization Ability of Online Algorithms for Dependent Data, Alekh Agarwal and John C. Duchi. IEEE Transactions on Information Theory (2013). [pdf]

Ergodic Mirror Descent, John C. Duchi, Alekh Agarwal, Mikael Johansson, Michael I. Jordan. SIAM Journal on Optimization (SIOPT), 2012. [pdf]

Randomized Smoothing for Stochastic Optimization, John Duchi, Peter L. Bartlett, and Martin Wainwright. SIAM Journal on Optimization (SIOPT), 2012. [pdf]

Dual Averaging for Distributed Optimization: Convergence and Network Scaling, John Duchi, Alekh Agarwal, and Martin Wainwright. IEEE Transactions on Automatic Control (March 2012). [pdf]

Adaptive Subgradient Methods for Online Learning and Stochastic Optimization, John Duchi, Elad Hazan, and Yoram Singer. Journal of Machine Learning Research (JMLR 2011). [pdf]

Efficient Online and Batch Learning using Forward Backward Splitting, John Duchi and Yoram Singer. Journal of Machine Learning Research (JMLR 2009) and Neural Information Processing Systems (NIPS 2009). [pdf]

Ph.D. Thesis

Multiple Optimality Guarantees in Statistical Learning, John C. Duchi. Ph.D. Thesis, Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, 2014. [pdf]

Conference Proceedings

Estimation, Optimization, and Parallelism when Data is Sparse, John C. Duchi, Michael I. Jordan, and Brendan McMahan. Neural Information Processing Systems (NIPS 2013). [pdf]

Information-theoretic lower bounds for distributed statistical estimation with communication constraints, Yuchen Zhang, John C. Duchi, Michael I. Jordan, and Martin Wainwright. Neural Information Processing Systems (NIPS 2013). [pdf]

Local Privacy and Minimax Bounds: Sharp Rates for Probability Estimation, John C. Duchi, Michael I. Jordan, and Martin Wainwright. Neural Information Processing Systems (NIPS 2013). [pdf coming soon]

Local Privacy and Statistical Minimax Rates, John C. Duchi, Michael I. Jordan, and Martin Wainwright. 54th Annual Symposium on Foundations of Computer Science (FOCS 2013). [pdf]

Divide and Conquer Kernel Ridge Regression, Yuchen Zhang, John C. Duchi, and Martin Wainwright. Conference on Learning Theory (COLT 2013). [pdf]

Privacy Aware Learning, John C. Duchi, Michael I. Jordan, and Martin Wainwright. Neural Information Processing Systems (NIPS 2012). [pdf]

Communication-Efficient Algorithms for Statistical Optimization, Yuchen Zhang, John C. Duchi, and Martin Wainwright. Neural Information Processing Systems (NIPS 2012). [pdf]

Finite Sample Convergence Rates of Zero-Order Stochastic Optimization Methods, John C. Duchi, Michael I. Jordan, Martin Wainwright, and Andre Wibisono. Neural Information Processing Systems (NIPS 2012). [pdf]

Randomized Smoothing for (Parallel) Stochastic Optimization, John Duchi, Peter L. Bartlett, and Martin Wainwright. International Conference on Machine Learning (ICML 2012) . Presented but not included in proceedings. [pdf]

Distributed Delayed Stochastic Optimization, Alekh Agarwal and John Duchi. Neural Information Processing Systems (NIPS 2011). [pdf] [Long pdf]

Ergodic Subgradient Descent, John Duchi Alekh Agarwal, Mikael Johansson, Michael I. Jordan. Allerton Conference on Communications, Control, and Computing 2011. [pdf]

Oracle Inequalities for Computationally Budgeted Model Selection, Alekh Agarwal, John Duchi, Peter L. Bartlett, Clement Levrard. Conference on Learning Theory (COLT 2011). [pdf]

Distributed Dual Averaging in Networks, John Duchi, Alekh Agarwal, and Martin Wainwright. Neural Information Processing Systems (NIPS 2010). [pdf]

On the Consistency of Ranking Algorithms, John Duchi, Lester Mackey, and Michael Jordan. International Conference on Machine Learning (ICML 2010). [pdf] Winner of best student paper award.

Adaptive Subgradient Methods for Online Learning and Stochastic Optimization, John Duchi, Elad Hazan, and Yoram Singer. Conference on Learning Theory (COLT 2010). [pdf]

Composite Objective Mirror Descent, John Duchi, Shai Shalev-Shwartz, Yoram Singer, Ambuj Tewari. Conference on Learning Theory (COLT 2010). [pdf]

Efficient Learning using Forward Backward Splitting, John Duchi and Yoram Singer. Neural Information Processing Systems (NIPS 2009). [pdf]

Boosting with Structural Sparsity, John Duchi and Yoram Singer. International Conference on Machine Learning (ICML 2009). [pdf] [Long pdf]

Efficient Projections onto the L1-Ball for Learning in High Dimensions, John Duchi, Shai Shalev-Shwartz, Yoram Singer, and Tushar Chandra. International Conference on Machine Learning (ICML 2008). [pdf]

Constrained Approximate Maximum Entropy Learning of Markov Random Fields, Varun Ganapathi, David Vickrey, John Duchi, and Daphne Koller. Conference on Uncertainty in Artificial Intelligence (UAI 2008). [pdf]

Projected Subgradient Methods for Learning Sparse Gaussians, John Duchi, Stephen Gould and Daphne Koller. Conference on Uncertainty in Artificial Intelligence (UAI 2008). [pdf]

Using Combinatorial Optimization within Max-Product Belief Propagation, John Duchi, Danny Tarlow, Gal Elidan, and Daphne Koller. Neural Information Processing Systems (NIPS 2006). [pdf]