More Like this

1. Compositional sparsity of learnable functions

   https://doi.org/10.1090/bull/1820  

   Poggio, Tomaso; Fraser, Maia (July 2024, Bulletin of the American Mathematical Society)

   Neural networks have demonstrated impressive success in various domains, raising the question of what fundamental principles underlie the effectiveness of the best AI systems and quite possibly of human intelligence. This perspective argues that compositional sparsity, or the property that a compositional function have “few” constituent functions, each depending on only a small subset of inputs, is a key principle underlying successful learning architectures. Surprisingly, all functions that are efficiently Turing computable have a compositional sparse representation. Furthermore, deep networks that are also sparse can exploit this general property to avoid the “curse of dimensionality”. This framework suggests interesting implications about the role that machine learning may play in mathematics. 

   more » « less
2. Learnable Filters for Geometric Scattering Modules

   https://doi.org/10.1109/TSP.2024.3378001  

   Tong, Alexander; Wenkel, Frederik; Bhaskar, Dhananjay; Macdonald, Kincaid; Grady, Jackson; Perlmutter, Michael; Krishnaswamy, Smita; Wolf, Guy (January 2024, IEEE Transactions on Signal Processing)

   We propose a new graph neural network (GNN) module, based on relaxations of recently proposed geometric scattering transforms, which consist of a cascade of graph wavelet filters. Our learnable geometric scattering (LEGS) module enables adaptive tuning of the wavelets to encourage band-pass features to emerge in learned representations. The incorporation of our LEGS-module in GNNs enables the learning of longer-range graph relations compared to many popular GNNs, which often rely on encoding graph structure via smoothness or similarity between neighbors. Further, its wavelet priors result in simplified architectures with significantly fewer learned parameters compared to competing GNNs. We demonstrate the predictive performance of LEGS-based networks on graph classification benchmarks, as well as the descriptive quality of their learned features in biochemical graph data exploration tasks. Our results show that LEGS-based networks match or outperforms popular GNNs, as well as the original geometric scattering construction, on many datasets, in particular in biochemical domains, while retaining certain mathematical properties of handcrafted (non-learned) geometric scattering. 

   more » « less
3. Most Neural Networks Are Almost Learnable

   Daniely, Amit; Srebro, Nati; Vardi, Gal (January 2024, Advances in neural information processing systems)
4. Learnable-pattern scanning based deep compressed imaging

   https://doi.org/10.1117/12.2577998  

   Zhang, Kangning; Hu, Junjie; Yang, Weijian (March 2021, High-Speed Biomedical Imaging and Spectroscopy VI)

   Goda, Keisuke; Tsia, Kevin K. (Ed.)

   We present a new deep compressed imaging modality by scanning a learned illumination pattern on the sample and detecting the signal with a single-pixel detector. This new imaging modality allows a compressed sampling of the object, and thus a high imaging speed. The object is reconstructed through a deep neural network inspired by compressed sensing algorithm. We optimize the illumination pattern and the image reconstruction network by training an end-to-end auto-encoder framework. Comparing with the conventional single-pixel camera and point-scanning imaging system, we accomplish a high-speed imaging with a reduced light dosage, while preserving a high imaging quality. 

   more » « less
5. Learnable Bernoulli Dropout for Bayesian Deep Learning

   Shahin Boluki, Randy Ardywibowo (January 2020, Proceedings of Machine Learning Research)

   null (Ed.)