Author: Viraj Shah

Posted on

Weekly Seminar – 04/06/2018 – Solving Linear Inverse Problems using GAN priors

This week, Viraj will be presenting the paper ” Solving Linear Inverse Problems Using GAN Priors: An Algorithm with Provable Guarantees” available at: https://arxiv.org/abs/1802.08406

Abstract:

In recent works, both sparsity-based methods as well as learning-based methods have proven to be successful in solving several challenging linear inverse problems. However, sparsity priors for natural signals and images suffer from poor discriminative capability, while learning-based methods seldom provide concrete theoretical guarantees. In this work, we advocate the idea of replacing hand-crafted priors, such as sparsity, with a Generative Adversarial Network (GAN) to solve linear inverse problems such as compressive sensing. In particular, we propose a projected gradient descent (PGD) algorithm for effective use of GAN priors for linear inverse problems, and also provide theoretical guarantees on the rate of convergence of this algorithm. Moreover, we show empirically that our algorithm demonstrates superior performance over an existing method of leveraging GANs for compressive sensing.

Time: 12.10 – 1.00 PM, Friday, April 06.

Venue: 2222 Coover Hall
Slides are available here.
Code and Poster are available on Viraj’s website here: http://virajshah.me/

 

Posted on

Weekly Seminar – 03/23/2018 and 03/30/2018 – Robust Subspace Clustering

For these two weeks, Praneeth will be presenting the paper “Robust Subspace Clustering” which is available at “https://arxiv.org/abs/1301.2603”

This chalk on blackboard talk will mostly focus on the algorithm and an overview of the theoretical results presented in the paper.

Time: 12.10 – 1.00 PM, Friday, March 23.
Venue: 2222 Coover Hall
Notes for the first session is available here.
Notes for the second session is here.
Posted on

Weekly Seminar – 03/02/2018 – Graph Convolutional Neural Networks

This week’s speaker for the DSRG seminar series is Rahul Singh. He will talk about Graph Convolutional Neural Networks. The abstract and references are as follows.
——————————————————
Abstract:

While classical CNNs have been very successful when dealing with signals such as speech, images, or video, in which there is an underlying Euclidean structure (regular grids), recently there has been a growing interest in trying to apply CNNs on non-Euclidean geometric data. Some examples of such data include

– In social networks, the characteristics of users can be modeled as signals on the vertices of the social graph
– In sensor networks, the sensor reading are modeled as time-dependent signals on the vertices
– In genetics, gene expression data are modeled as signals defined on the regulatory network

The non-Euclidean nature of such data implies that there are no such familiar properties as common system of coordinates, vector space structure, or shift-invariance. Consequently, basic operations like convolution and shifting that are taken for granted in the Euclidean case are even not well defined on non-Euclidean domains.

The talk will be about the recent efforts made towards the generalization of CNNs from low-dimensional regular grids to high-dimensional irregular domains such as graphs.”
——————————————————-

References:

Michaël Defferrard et al., “Convolutional neural networks on graphs with fast localized spectral filtering.” In Advances in Neural Information Processing Systems, pp. 3844-3852. 2016.

Thomas Kipf and Max Welling, ” Semi-supervised classification with graph convolutional networks.”  In Proceedings of International Conference on Learning Representations, 2017.

Michael Bronstein et al., “Geometric deep learning: going beyond euclidean data.” IEEE Signal Processing Magazine 34.4 (2017): 18-42.
——————————————————

Venue:2222 Coover Hall
Time: 12.00 – 1.00PM Friday, March 2nd.
Posted on

Weekly Seminar – 02/23/2018 – Consensus-based distributed stochastic gradient descent method for fixed topology networks

This week Zhanhong Jiang of Mechanical Engineering will speak about Distributed Deep Learning Algorithms.
——————————————————
The abstract:
The focus of this talk is towards developing novel distributed deep learning algorithms in order to solve challenging learning problems in various domains such as robotic networks. Specifically, I will present a consensus-based distributed stochastic gradient descent method for fixed topology networks. While some previous work has been done on this topic, the data parallelism and distributed computation are still not sufficiently explored. Therefore, the proposed method can be used to tackle such issues.
—————————————————–

Venue: 2222 Coover Hall

Time: 12.10 – 1.00PM Friday, 23 February.

Posted on

Weekly Seminar – 02/16/2018 – Interpreting the Deep Learning Models

This week’s speaker for the DSRG seminar series is Aditya Balu from Mechanical Engineering.
———————————

Abstract:

Deep learning models are very accurate to give good prediction. However, unlike the traditional shallow models, interpreting such model’s performance is very difficult. In this talk, I shall discuss several approaches which people have used for interpreting the models. Also, we shall discuss about application of such methods on different domains such as manufacturing etc.
——————————-

Location: 2222 Coover Hall

Time: 12.00 Noon, Feb. 16th.

Notes: Slides for this Seminar are available here.

Posted on

Weekly Seminar – 02/02/2018 – Symmetry-Breaking Convergence Analysis

Gauri Jagatap from Dr. Hegde’s research group presented the paper “Symmetry-Breaking Convergence Analysis of Certain Two-layered Neural Networks with ReLU nonlinearity”. You can find the paper here: https://openreview.net/forum?id=Hk85q85ee

————————————————-
Abstract:

In this paper, the authors use dynamical system to analyze the nonlinear weight dynamics of two-layered bias-free networks in the form of g(x; w) = \sum_{j=1}^K \sigma(w_j x), where \sigma(.) is ReLU nonlinearity. The input x is assumed to follow Gaussian distribution.

The authors show that for K = 1 (single ReLU), the nonlinear dynamics can be written in close form, and converges to w* with probability at least (1-\epsilon)/2, if random weight initializations of proper standard derivation (1/\sqrt{d}) are used, verifying empirical practice.

For networks with many ReLU nodes (K >= 2), they apply our closed form dynamics and prove that when the teacher parameters w*_j ‘s form an orthonormal basis,
(1) a symmetric weight initialization yields a convergence to a saddle point and
(2) a certain symmetry-breaking weight initialization yields global convergence to w* without local minima.

They claim that this is the first proof that shows global convergence in nonlinear neural network without unrealistic assumptions on the independence of ReLU activations.
————————————————-

Location: 2222 Coover Hall

Time: 12.10 – 1 PM, Friday February 2nd.
Please find the notes of the talk here.