Sahil Sidheekh

Selected Publications

1. UAI 2023 (Oral)

   Probabilistic Flow Circuits: Towards Unified Deep Models for Tractable Probabilistic Inference

   Sahil Sidheekh, Kristian Kersting, and Sriraam Natarajan

   In The 39th Conference on Uncertainty in Artificial Intelligence (UAI) 2023

   Abstract PDF Code Poster

   We consider the problem of increasing the expressivity of probabilistic circuits by augmenting them with the successful generative models of normalizing flows. To this effect, we theoretically establish the requirement of decomposability for such combinations to retain tractability of the learned models. Our model, called Probabilistic Flow Circuits, essentially extends circuits by allowing for normalizing flows at the leaves. Our empirical evaluation clearly establishes the expressivity and tractability of this new class of probabilistic circuits

2. UAI 2022

   VQ-Flows: Vector Quantized Local Normalizing Flows

   Sahil Sidheekh, Chris Barton Dock, Tushar Jain, Radu Balan, and Maneesh Kumar Singh,

   In The 38th Conference on Uncertainty in Artificial Intelligence (UAI) 2022

   Abstract HTML PDF Poster

   Normalizing flows provide an elegant approach to generative modeling that allows for efficient sampling and exact density evaluation of unknown data distributions. However, current techniques have significant limitations in their expressivity when the data distribution is supported on a low-dimensional manifold or has a non-trivial topology. We introduce a novel statistical framework for learning a mixture of local normalizing flows as "chart maps" over the data manifold. Our framework augments the expressivity of recent approaches while preserving the signature property of normalizing flows, that they admit exact density evaluation. We learn a suitable atlas of charts for the data manifold via a vector quantized auto-encoder (VQ-AE) and the distributions over them using a conditional flow. We validate experimentally that our probabilistic framework enables existing approaches to better model data distributions over complex manifolds.

3. ICML 2021

   On Characterizing GAN Convergence Through Proximal Duality Gap

   Sahil Sidheekh, Aroof Aimen, and Narayanan C. Krishnan

   In Proceedings of the 38th International Conference on Machine Learning (ICML) 2021

   Abstract HTML PDF Code

   Despite the accomplishments of Generative Adversarial Networks (GANs) in modeling data distributions, training them remains a challenging task. A contributing factor to this difficulty is the non-intuitive nature of the GAN loss curves, which necessitates a subjective evaluation of the generated output to infer training progress. Recently, motivated by game theory, duality gap has been proposed as a domain agnostic measure to monitor GAN training. However, it is restricted to the setting when the GAN converges to a Nash equilibrium. But GANs need not always converge to a Nash equilibrium to model the data distribution. In this work, we extend the notion of duality gap to proximal duality gap that is applicable to the general context of training GANs where Nash equilibria may not exist. We show theoretically that the proximal duality gap is capable of monitoring the convergence of GANs to a wider spectrum of equilibria that subsumes Nash equilibria. We also theoretically establish the relationship between the proximal duality gap and the divergence between the real and generated data distributions for different GAN formulations. Our results provide new insights into the nature of GAN convergence. Finally, we validate experimentally the usefulness of proximal duality gap for monitoring and influencing GAN training.

4. NeurIPS-W 2021

   Task Attended Meta-Learning for Few-Shot Learning

   Aroof Aimen, Sahil Sidheekh, Bharat Ladrecha, and Narayanan Chatapuram Krishnan

   In Fifth Workshop on Meta-Learning at the Conference on Neural Information Processing Systems (NeurIPS) 2021

   Abstract HTML PDF Code

   Meta-learning (ML) has emerged as a promising direction in learning models under constrained resource settings like few-shot learning. The popular approaches for ML either learn a generalizable initial model or a generic parametric optimizer through episodic training. The former approaches leverage the knowledge from a batch of tasks to learn an optimal prior. In this work, we study the importance of a batch for ML. Specifically, we first incorporate a batch episodic training regimen to improve the learning of the generic parametric optimizer. We also hypothesize that the common assumption in batch episodic training that each task in a batch has an equal contribution to learning an optimal meta-model need not be true. We propose to weight the tasks in a batch according to their "importance" in improving the meta-model’s learning. To this end, we introduce a training curriculum motivated by selective focus in humans, called task attended meta-training, to weight the tasks in a batch. Task attention is a standalone module that can be integrated with any batch episodic training regimen. The comparisons of the models with their non-task-attended counterparts on complex datasets like miniImageNet and tieredImageNet validate its effectiveness.

5. AAAI-W 2021

   Stress Testing of Meta-learning Approaches for Few-shot Learning

   Aroof Aimen, Sahil Sidheekh, Vineet Madan, and Narayanan C. Krishnan

   In AAAI Workshop on Metalearning and Co-Hosted Challenge 2021

   HTML PDF