Sahil Sidheekh

ECSS 3.214, Erik Jonsson School of Engineering & Computer Science, UTD

I am an AI researcher trying to make machines capable of representing knowledge while reasoning about uncertainties in this increasingly complex world.

Presently, I am a 1^st year Ph.D. student working at the StARLing Lab headed by Dr. Sriraam Natarajan. I work broadly in the domains of Neuro-Symbolic AI, Probabilistic Generative Models and Meta-learning. I also explore novel applications that can benefit from these reasearch areas, especially problems that can impact society for the better. Previously, I completed by undergraduate studies majoring in Computer Science and Engineering from IIT Ropar, where I was fortunate to be a part of the LSAIML team headed by Dr. Narayanan C. K.

I am passionate about building robust and efficient systems that can reason probabilistically and make interpretable decisions. I enjoy connecting dots and innovating ideas that span multiple disciplines. I have a strong academic background in engineering and machine learning. If you would like to collaborate, please feel free to reach out through email .

news

Aug 22, 2022	I have joined the StARLing Lab at UT Dallas as a `Ph.D. Student`. Excited !
May 16, 2022	Our work *VQ-Flows: Vector Quantized Local Normalizing Flows* has been accepted to `UAI 2022`
Jan 19, 2022	Our work *Machine Learning Methods Trained on Simple Models can Anticipate Crtitical Transitions in Complex Systems* has been accepted to the `Royal Society Open Science` Journal.
Oct 24, 2021	Our work *Task Attended Meta-Learning for Few-Shot Learning* has been accepted to the `NeurIPS 2021 Workshop on Metalearning`
May 10, 2021	Our work *On Characterizing GAN Convergence Through Proximal Duality Gap* has been accepted to `ICML 2021`

selected publications

On Characterizing GAN Convergence Through Proximal Duality Gap

Sidheekh, Sahil, Aimen, Aroof, and C.Krishnan, Narayanan

In Proceedings of the 38th International Conference on Machine Learning (ICML) 2021

Abs HTML PDF Code

Despite the accomplishments of Generative Adversarial Networks (GANs) in modeling data distributions, training them remains a challenging task. A contributing factor to this difficulty is the non-intuitive nature of the GAN loss curves, which necessitates a subjective evaluation of the generated output to infer training progress. Recently, motivated by game theory, duality gap has been proposed as a domain agnostic measure to monitor GAN training. However, it is restricted to the setting when the GAN converges to a Nash equilibrium. But GANs need not always converge to a Nash equilibrium to model the data distribution. In this work, we extend the notion of duality gap to proximal duality gap that is applicable to the general context of training GANs where Nash equilibria may not exist. We show theoretically that the proximal duality gap is capable of monitoring the convergence of GANs to a wider spectrum of equilibria that subsumes Nash equilibria. We also theoretically establish the relationship between the proximal duality gap and the divergence between the real and generated data distributions for different GAN formulations. Our results provide new insights into the nature of GAN convergence. Finally, we validate experimentally the usefulness of proximal duality gap for monitoring and influencing GAN training.
Task Attended Meta-Learning for Few-Shot Learning

Aimen, Aroof, Sidheekh, Sahil, Ladrecha, Bharat, and Krishnan, Narayanan Chatapuram

In Fifth Workshop on Meta-Learning at the Conference on Neural Information Processing Systems (NeurIPS) 2021

Abs HTML PDF Code

Meta-learning (ML) has emerged as a promising direction in learning models under constrained resource settings like few-shot learning. The popular approaches for ML either learn a generalizable initial model or a generic parametric optimizer through episodic training. The former approaches leverage the knowledge from a batch of tasks to learn an optimal prior. In this work, we study the importance of a batch for ML. Specifically, we first incorporate a batch episodic training regimen to improve the learning of the generic parametric optimizer. We also hypothesize that the common assumption in batch episodic training that each task in a batch has an equal contribution to learning an optimal meta-model need not be true. We propose to weight the tasks in a batch according to their "importance" in improving the meta-model’s learning. To this end, we introduce a training curriculum motivated by selective focus in humans, called task attended meta-training, to weight the tasks in a batch. Task attention is a standalone module that can be integrated with any batch episodic training regimen. The comparisons of the models with their non-task-attended counterparts on complex datasets like miniImageNet and tieredImageNet validate its effectiveness.
Stress Testing of Meta-learning Approaches for Few-shot Learning

Aimen, Aroof, Sidheekh, Sahil, Madan, Vineet, and Krishnan, Narayanan C.

In AAAI Workshop on Metalearning and Co-Hosted Challenge 2021

HTML PDF
On Duality Gap as a Measure for Monitoring GAN Training

Sidheekh, Sahil, Aimen, Aroof, Madan, Vineet, and Krishnan, Narayanan C.

In International Joint Conference on Neural Networks (IJCNN) 2021

Abs HTML PDF Code

Generative adversarial network (GAN) is among the most popular deep learning models for learning complex data distributions. However, training a GAN is known to be a challenging task. This is often attributed to the lack of correlation between the training progress and the trajectory of the generator and discriminator losses and the need for the GAN’s subjective evaluation. A recently proposed measure inspired by game theory - the duality gap, aims to bridge this gap. However, as we demonstrate, the duality gap’s capability remains constrained due to limitations posed by its estimation process. This paper presents a theoretical understanding of this limitation and proposes a more dependable estimation process for the duality gap. At the crux of our approach is the idea that local perturbations can help agents in a zero-sum game escape non-Nash saddle points efficiently. Through exhaustive experimentation across GAN models and datasets, we establish the efficacy of our approach in capturing the GAN training progress with minimal increase to the computational complexity. Further, we show that our estimate, with its ability to identify model convergence/divergence, is a potential performance measure that can be used to tune the hyperparameters of a GAN.