Research
My research interests lie at intersection of convex optimization and deep learning. Currently, I focus on the understanding of neural network through the lens of convex optimization. Previously, I also developed efficient second order randomized algorithms to train neural networks.
|
Publications
|
|
Demystifying Batch Normalization in ReLU Networks: Equivalent Convex Optimization Models and Implicit Regularization
Tolga Ergen, Arda Sahiner, Batu Ozturkler, John Pauly, Morteza Mardani, Mert Pilanci
PDF
neural networks, convex analysis
|
|
Implicit Convex Regularizers of CNN Architectures: Convex Optimization of Two- and Three-Layer Networks in Polynomial Time
Tolga Ergen, Mert Pilanci
ICLR 2021 (Spotlight Presentation)
PDF
convolutional neural networks, convex optimization, deep learning
|
|
Vector-output ReLU Neural Network Problems are Copositive Programs: Convex Analysis of Two Layer Networks and Polynomial-time Algorithms
Arda Sahiner, Tolga Ergen, John Pauly, Mert Pilanci
ICLR 2021
PDF
neural networks, convex analysis, non-convex optimization
|
|
Convex Programs for Global Optimization of Convolutional Neural
Networks in Polynomial-Time
Tolga Ergen, Mert Pilanci
NeurIPS 2020 Workshop on Optimization for Machine Learning (Oral Presentation)
PDF
convolutional neural networks, convex optimization, deep learning
|
|
Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networks
Mert Pilanci, Tolga Ergen
ICML 2020
PDF
neural networks, convex analysis, non-convex optimization
|
|
Revealing the Structure of Deep Neural Networks via Convex Duality
Tolga Ergen, Mert Pilanci
arXiv
deep neural networks, convex analysis, non-convex optimization
|
|
Convex Geometry of Two-Layer ReLU Networks: Implicit Autoencoding and Interpretable Models
Tolga Ergen, Mert Pilanci
AISTATS 2020
PDF
neural networks, convex analysis, non-convex optimization
|
|
Exact and Relaxed Convex Formulations for Shallow Neural Autoregressive Models
Vikul Gupta, Burak Bartan, Tolga Ergen, Mert Pilanci
ICASSP 2021
PDF
generative models, neural networks, convex optimization
|
|
Convex Geometry and Duality of Over-parameterized Neural Networks
Tolga Ergen, Mert Pilanci
arXiv
neural networks, convex analysis, non-convex optimization
|
|
A Novel Distributed Anomaly Detection Algorithm Based on Support Vector Machines
Tolga Ergen, Serdar Kozat
Digital Signal Processing
PDF
support vector machines, distributed optimization
|
|
Convex Duality and Cutting Plane Methods for Over-parameterized Neural Networks
Tolga Ergen, Mert Pilanci
NeurIPS 2019 Workshop on Optimization for Machine Learning
PDF
neural networks, convex analysis, non-convex optimization
|
|
Random Projections for Learning Non-convex Models
Tolga Ergen, Mert Pilanci
NeurIPS 2019 Workshop on Beyond First Order Methods in Machine Learning
PDF
randomized algorithms, non-convex optimization
|
|
Convex Optimization for Shallow Neural Networks
Tolga Ergen, Mert Pilanci
ALLERTON 2019
PDF
neural networks, convex optimization
|
|
Energy-Efficient LSTM Networks for Online Learning
Tolga Ergen, Ali Mirza, Serdar Kozat
IEEE TNNNLS
PDF
recurrent neural networks, online learning, non-convex optimization
|
|
Unsupervised Anomaly Detection with LSTM Neural Networks
Tolga Ergen, Serdar Kozat
IEEE TNNLS
PDF
recurrent neural networks, support vector machines, non-convex optimization
|
|
Team-Optimal Online Estimation of Dynamic Parameters over Distributed Tree Networks
Fatih Kilic, Tolga Ergen, Muhammed Sayin, Serdar Kozat.
Signal Processing
PDF
online learning, distributed optimization
|
|
Online Training of LSTM Networks in Distributed Systems for Variable Length Data Sequences
Tolga Ergen, Serdar Kozat
IEEE TNNLS
PDF
recurrent neural networks, distributed optimization
|
|
Efficient Online Learning Algorithms Based on LSTM Neural Networks
Tolga Ergen, Serdar Kozat
IEEE TNNLS
PDF
recurrent neural networks, online learning
|
|
A Highly Efficient Recurrent Neural Network Architecture for Data Regression
Tolga Ergen, Emir Ceyani
IEEE SIU 2018
PDF
recurrent neural networks, online learning
|
|
A Novel Anomaly Detection Approach Based on Neural Networks
Tolga Ergen, Mine Kerpicci
IEEE SIU 2018
PDF
neural networks, non-convex optimization
|
|
Computationally Efficient Online Regression via LSTM Neural Networks
Tolga Ergen, Serdar Kozat
EUSIPCO 2017
PDF
recurrent neural networks, online learning
|
|
An Efficient Bandit Algorithm for General Weight Assignments
Kaan Gokcesu, Tolga Ergen, Selami Ciftci, Serdar Kozat
IEEE SIU 2017
PDF
adversarial multi armed bandit, non-convex optimization
|
|
Neural Networks Based Online Learning.
Tolga Ergen, Serdar Kozat
IEEE SIU 2017
PDF
neural networks, online learning
|
|
Novelty Detection Using Soft Partitioning and Hierarchical Models
Tolga Ergen, Kaan Gokcesu, Mustafa Simsek, Serdar Kozat
IEEE SIU 2017
PDF
online learning, non-convex optimization
|
|
Online Distributed Nonlinear Regression via Neural Networks
Tolga Ergen, Serdar Kozat
IEEE SIU 2017
PDF
neural networks, distributed optimization
|
 |
(Fall 2019, 2020, 2021) EE269, Signal Processing for Machine Learning, TA
(Winter 2020, 2021) EE270, Large Scale Matrix Computation, Optimization and Learning, TA
(Spring 2020, 2021) EE364B, Convex Optimization II, TA
|
|
(Fall 2016, 2017, Spring 2018) EEE424, Digital Signal Processing, TA
(Spring 2018) EEE102, Introduction to Digital Circuit Design, TA
|
Reviewer for NeurIPS, ICML, IEEE TNNLS, IEEE SPL
|
|