publications

2023

1. UAI 2023 (Oral)

   Probabilistic Flow Circuits: Towards Unified Deep Models for Tractable Probabilistic Inference

   Sahil Sidheekh, Kristian Kersting, and Sriraam Natarajan

   In The 39th Conference on Uncertainty in Artificial Intelligence (UAI) 2023

   Abstract PDF Code Poster

   We consider the problem of increasing the expressivity of probabilistic circuits by augmenting them with the successful generative models of normalizing flows. To this effect, we theoretically establish the requirement of decomposability for such combinations to retain tractability of the learned models. Our model, called Probabilistic Flow Circuits, essentially extends circuits by allowing for normalizing flows at the leaves. Our empirical evaluation clearly establishes the expressivity and tractability of this new class of probabilistic circuits

2022

1. UAI 2022

   VQ-Flows: Vector Quantized Local Normalizing Flows

   Sahil Sidheekh, Chris Barton Dock, Tushar Jain, Radu Balan, and Maneesh Kumar Singh,

   In The 38th Conference on Uncertainty in Artificial Intelligence (UAI) 2022

   Abstract HTML PDF Poster

   Normalizing flows provide an elegant approach to generative modeling that allows for efficient sampling and exact density evaluation of unknown data distributions. However, current techniques have significant limitations in their expressivity when the data distribution is supported on a low-dimensional manifold or has a non-trivial topology. We introduce a novel statistical framework for learning a mixture of local normalizing flows as "chart maps" over the data manifold. Our framework augments the expressivity of recent approaches while preserving the signature property of normalizing flows, that they admit exact density evaluation. We learn a suitable atlas of charts for the data manifold via a vector quantized auto-encoder (VQ-AE) and the distributions over them using a conditional flow. We validate experimentally that our probabilistic framework enables existing approaches to better model data distributions over complex manifolds.

2. RSOS 2022

   Machine learning methods trained on simple models can predict critical transitions in complex natural systems

   Smita Deb, Sahil Sidheekh, Christopher F. Clements, Narayanan C. Krishnan, and Partha S. Dutta,

   Royal Society Open Science Journal, 2022

   Abstract HTML PDF Code

   1. Sudden transitions from one stable state to a contrasting state occur in complex systems ranging from the collapse of ecological populations to climatic change, with much recent work seeking to develop methods to predict these unexpected transitions from signals in time series data. However, previously developed methods vary widely in their reliability, and fail to classify whether an approaching collapse might be catastrophic (and hard to reverse) or non-catastrophic (easier to reverse) with significant implications for how such systems are managed. 2. Here we develop a novel detection method, using simulated outcomes from a range of simple mathematical models with varying nonlinearity to train a deep neural network to detect critical transitions - the Early Warning Signal Network (EWSNet). 3. We demonstrate that this neural network (EWSNet), trained on simulated data with minimal assumptions about the underlying structure of the system, can predict with high reliability observed real-world transitions in ecological and climatological data. Importantly, our model appears to capture latent properties in time series missed by previous warning signals approaches, allowing us to not only detect if a transition is approaching but critically whether the collapse will be catastrophic or non-catastrophic. 4. The EWSNet can flag a critical transition with unprecedented accuracy, overcoming some of the major limitations of traditional methods based on phenomena such as Critical Slowing Down. These novel properties mean EWSNet has the potential to serve as a universal indicator of transitions across a broad spectrum of complex systems, without requiring information on the structure of the system being monitored. Our work highlights the practicality of deep learning for addressing further questions pertaining to ecosystem collapse and have much broader management implications.

3. 

   Attentive Contractive Flow: Improved Contractive Flows with Lipschitz-constrained Self-Attention

   Avideep Mukherjee, Badri Narayan Patro, Sahil Sidheekh, Maneesh Singh, and Vinay P. Namboodiri,

   Preprint, Under Review 2022

   Abstract

   Normalizing flows provide an elegant method for obtaining tractable density estimates from distributions by using invertible transformations. The main challenge is to improve the expressivity of the models while keeping the invertibility constraints intact. We propose to do so via the incorporation of localized self-attention. However, conventional self-attention mechanisms don’t satisfy the requirements to obtain invertible flows and can’t be naively incorporated into normalizing flows. To address this, we introduce a novel approach called Attentive Contractive Flow (ACF) which utilizes a special category of flow-based generative models - contractive flows. We demonstrate that ACF can be introduced into a variety of state of the art flow models in a plug-and-play manner. This is demonstrated to not only improve the representation power of these models (improving on the bits per dim metric), but also to results in significantly faster convergence in training them. Qualitative results, including interpolations between test images, demonstrate that samples are more realistic and capture local correlations in the data well. We evaluate the results further by performing perturbation analysis using AWGN demonstrating that ACF models (especially the dot-product variant) show better and more consistent resilience to additive noise.

2021

1. ICML 2021

   On Characterizing GAN Convergence Through Proximal Duality Gap

   Sahil Sidheekh, Aroof Aimen, and Narayanan C. Krishnan

   In Proceedings of the 38th International Conference on Machine Learning (ICML) 2021

   Abstract HTML PDF Code

   Despite the accomplishments of Generative Adversarial Networks (GANs) in modeling data distributions, training them remains a challenging task. A contributing factor to this difficulty is the non-intuitive nature of the GAN loss curves, which necessitates a subjective evaluation of the generated output to infer training progress. Recently, motivated by game theory, duality gap has been proposed as a domain agnostic measure to monitor GAN training. However, it is restricted to the setting when the GAN converges to a Nash equilibrium. But GANs need not always converge to a Nash equilibrium to model the data distribution. In this work, we extend the notion of duality gap to proximal duality gap that is applicable to the general context of training GANs where Nash equilibria may not exist. We show theoretically that the proximal duality gap is capable of monitoring the convergence of GANs to a wider spectrum of equilibria that subsumes Nash equilibria. We also theoretically establish the relationship between the proximal duality gap and the divergence between the real and generated data distributions for different GAN formulations. Our results provide new insights into the nature of GAN convergence. Finally, we validate experimentally the usefulness of proximal duality gap for monitoring and influencing GAN training.

2. NeurIPS-W 2021

   Task Attended Meta-Learning for Few-Shot Learning

   Aroof Aimen, Sahil Sidheekh, Bharat Ladrecha, and Narayanan Chatapuram Krishnan

   In Fifth Workshop on Meta-Learning at the Conference on Neural Information Processing Systems (NeurIPS) 2021

   Abstract HTML PDF Code

   Meta-learning (ML) has emerged as a promising direction in learning models under constrained resource settings like few-shot learning. The popular approaches for ML either learn a generalizable initial model or a generic parametric optimizer through episodic training. The former approaches leverage the knowledge from a batch of tasks to learn an optimal prior. In this work, we study the importance of a batch for ML. Specifically, we first incorporate a batch episodic training regimen to improve the learning of the generic parametric optimizer. We also hypothesize that the common assumption in batch episodic training that each task in a batch has an equal contribution to learning an optimal meta-model need not be true. We propose to weight the tasks in a batch according to their "importance" in improving the meta-model’s learning. To this end, we introduce a training curriculum motivated by selective focus in humans, called task attended meta-training, to weight the tasks in a batch. Task attention is a standalone module that can be integrated with any batch episodic training regimen. The comparisons of the models with their non-task-attended counterparts on complex datasets like miniImageNet and tieredImageNet validate its effectiveness.

3. AAAI-W 2021

   Stress Testing of Meta-learning Approaches for Few-shot Learning

   Aroof Aimen, Sahil Sidheekh, Vineet Madan, and Narayanan C. Krishnan

   In AAAI Workshop on Metalearning and Co-Hosted Challenge 2021

   HTML PDF
4. IJCNN 2021

   On Duality Gap as a Measure for Monitoring GAN Training

   Sahil Sidheekh, Aroof Aimen, Vineet Madan, and Narayanan C. Krishnan

   In International Joint Conference on Neural Networks (IJCNN) 2021

   Abstract HTML PDF Code

   Generative adversarial network (GAN) is among the most popular deep learning models for learning complex data distributions. However, training a GAN is known to be a challenging task. This is often attributed to the lack of correlation between the training progress and the trajectory of the generator and discriminator losses and the need for the GAN’s subjective evaluation. A recently proposed measure inspired by game theory - the duality gap, aims to bridge this gap. However, as we demonstrate, the duality gap’s capability remains constrained due to limitations posed by its estimation process. This paper presents a theoretical understanding of this limitation and proposes a more dependable estimation process for the duality gap. At the crux of our approach is the idea that local perturbations can help agents in a zero-sum game escape non-Nash saddle points efficiently. Through exhaustive experimentation across GAN models and datasets, we establish the efficacy of our approach in capturing the GAN training progress with minimal increase to the computational complexity. Further, we show that our estimate, with its ability to identify model convergence/divergence, is a potential performance measure that can be used to tune the hyperparameters of a GAN.

5. 

   Learning Neural Networks on SVD Boosted Latent Spaces for Semantic Classification

   Sahil Sidheekh

   Computing Research Repository, ArXiv . 2021

   Abstract HTML PDF Code

   The availability of large amounts of data and compelling computation power have made deep learning models much popular for text classification and sentiment analysis. Deep neural networks have achieved competitive performance on the above tasks when trained on naive text representations such as word count, term frequency, and binary matrix embeddings. However, many of the above representations result in the input space having a dimension of the order of the vocabulary size, which is enormous. This leads to a blow-up in the number of parameters to be learned, and the computational cost becomes infeasible when scaling to domains that require retaining a colossal vocabulary. This work proposes using singular value decomposition to transform the high dimensional input space to a lower-dimensional latent space. We show that neural networks trained on this lower-dimensional space are not only able to retain performance while savoring significant reduction in the computational complexity but, in many situations, also outperforms the classical neural networks trained on the native input space.

2020

1. 

   Scale Invariant Fast PHT based Copy-Move Forgery Detection

   Aroof Aimen, Amandeep Kaur, and Sahil Sidheekh,

   In 11th International Conference on Computing, Communication and Networking Technologies (ICCCNT) 2020

   Abstract HTML PDF

   Copy-Move forgery is a type of image forgery wherein a patch from the image is copied and pasted on the same image either to increase the occurrence of a particular object or to conceal some important detail in the image. This paper addresses the issue of copy-move forgery using the block-based method of feature extraction. In block-based methods of feature extraction, PHT is one of the competing solutions, but it is not much robust to scaling. This paper proposes Scale-Invariant Fast PHT (SIFPHT) algorithm to detect the copy-move forgery which uses Fast PHT [1] for extracting the features from the blocks. Fast PHT has a higher convergence rate than the traditional PHT, and the results prove that the speed-up of almost 4 is attained for detecting the forgery. Moreover, the Fast PHT features so obtained from the blocks are normalized before comparison due to which the scaled forged segments are also identified. Further, Fast K-Means clustering is used to estimate the similarity in the blocks and hence detect the copy-move forgery.