Research
My research interests lie at intersection of convex optimization and deep learning. Currently, I focus on the understanding of neural network through the lens of convex optimization. Previously, I also developed efficient second order randomized algorithms to train neural networks.


Demystifying Batch Normalization in ReLU Networks: Equivalent Convex Optimization Models and Implicit Regularization
Tolga Ergen, Arda Sahiner, Batu Ozturkler, John Pauly, Morteza Mardani, Mert Pilanci
PDF
neural networks, convex analysis


Implicit Convex Regularizers of CNN Architectures: Convex Optimization of Two and ThreeLayer Networks in Polynomial Time
Tolga Ergen, Mert Pilanci
ICLR 2021 (Spotlight Presentation)
PDF
convolutional neural networks, convex optimization, deep learning


Vectoroutput ReLU Neural Network Problems are Copositive Programs: Convex Analysis of Two Layer Networks and Polynomialtime Algorithms
Arda Sahiner, Tolga Ergen, John Pauly, Mert Pilanci
ICLR 2021
PDF
neural networks, convex analysis, nonconvex optimization


Convex Programs for Global Optimization of Convolutional Neural
Networks in PolynomialTime
Tolga Ergen, Mert Pilanci
NeurIPS 2020 Workshop on Optimization for Machine Learning (Oral Presentation)
PDF
convolutional neural networks, convex optimization, deep learning


Neural Networks are Convex Regularizers: Exact Polynomialtime Convex Optimization Formulations for Twolayer Networks
Mert Pilanci, Tolga Ergen
ICML 2020
PDF
neural networks, convex analysis, nonconvex optimization


Revealing the Structure of Deep Neural Networks via Convex Duality
Tolga Ergen, Mert Pilanci
arXiv
deep neural networks, convex analysis, nonconvex optimization


Convex Geometry of TwoLayer ReLU Networks: Implicit Autoencoding and Interpretable Models
Tolga Ergen, Mert Pilanci
AISTATS 2020
PDF
neural networks, convex analysis, nonconvex optimization


Exact and Relaxed Convex Formulations for Shallow Neural Autoregressive Models
Vikul Gupta, Burak Bartan, Tolga Ergen, Mert Pilanci
ICASSP 2021
PDF
generative models, neural networks, convex optimization


Convex Geometry and Duality of Overparameterized Neural Networks
Tolga Ergen, Mert Pilanci
arXiv
neural networks, convex analysis, nonconvex optimization


A Novel Distributed Anomaly Detection Algorithm Based on Support Vector Machines
Tolga Ergen, Serdar Kozat
Digital Signal Processing
PDF
support vector machines, distributed optimization


Convex Duality and Cutting Plane Methods for Overparameterized Neural Networks
Tolga Ergen, Mert Pilanci
NeurIPS 2019 Workshop on Optimization for Machine Learning
PDF
neural networks, convex analysis, nonconvex optimization


Random Projections for Learning Nonconvex Models
Tolga Ergen, Mert Pilanci
NeurIPS 2019 Workshop on Beyond First Order Methods in Machine Learning
PDF
randomized algorithms, nonconvex optimization


Convex Optimization for Shallow Neural Networks
Tolga Ergen, Mert Pilanci
ALLERTON 2019
PDF
neural networks, convex optimization


EnergyEfficient LSTM Networks for Online Learning
Tolga Ergen, Ali Mirza, Serdar Kozat
IEEE TNNNLS
PDF
recurrent neural networks, online learning, nonconvex optimization


Unsupervised Anomaly Detection with LSTM Neural Networks
Tolga Ergen, Serdar Kozat
IEEE TNNLS
PDF
recurrent neural networks, support vector machines, nonconvex optimization


TeamOptimal Online Estimation of Dynamic Parameters over Distributed Tree Networks
Fatih Kilic, Tolga Ergen, Muhammed Sayin, Serdar Kozat.
Signal Processing
PDF
online learning, distributed optimization


Online Training of LSTM Networks in Distributed Systems for Variable Length Data Sequences
Tolga Ergen, Serdar Kozat
IEEE TNNLS
PDF
recurrent neural networks, distributed optimization


Efficient Online Learning Algorithms Based on LSTM Neural Networks
Tolga Ergen, Serdar Kozat
IEEE TNNLS
PDF
recurrent neural networks, online learning


A Highly Efficient Recurrent Neural Network Architecture for Data Regression
Tolga Ergen, Emir Ceyani
IEEE SIU 2018
PDF
recurrent neural networks, online learning


A Novel Anomaly Detection Approach Based on Neural Networks
Tolga Ergen, Mine Kerpicci
IEEE SIU 2018
PDF
neural networks, nonconvex optimization


Computationally Efficient Online Regression via LSTM Neural Networks
Tolga Ergen, Serdar Kozat
EUSIPCO 2017
PDF
recurrent neural networks, online learning


An Efficient Bandit Algorithm for General Weight Assignments
Kaan Gokcesu, Tolga Ergen, Selami Ciftci, Serdar Kozat
IEEE SIU 2017
PDF
adversarial multi armed bandit, nonconvex optimization


Neural Networks Based Online Learning.
Tolga Ergen, Serdar Kozat
IEEE SIU 2017
PDF
neural networks, online learning


Novelty Detection Using Soft Partitioning and Hierarchical Models
Tolga Ergen, Kaan Gokcesu, Mustafa Simsek, Serdar Kozat
IEEE SIU 2017
PDF
online learning, nonconvex optimization


Online Distributed Nonlinear Regression via Neural Networks
Tolga Ergen, Serdar Kozat
IEEE SIU 2017
PDF
neural networks, distributed optimization


(Fall 2019, 2020, 2021) EE269, Signal Processing for Machine Learning, TA
(Winter 2020, 2021) EE270, Large Scale Matrix Computation, Optimization and Learning, TA
(Spring 2020, 2021) EE364B, Convex Optimization II, TA


(Fall 2016, 2017, Spring 2018) EEE424, Digital Signal Processing, TA
(Spring 2018) EEE102, Introduction to Digital Circuit Design, TA

Reviewer for NeurIPS, ICML, IEEE TNNLS, IEEE SPL

