More Like this

1. Learning constraints from demonstrations with grid and parametric representations

   https://doi.org/10.1177/02783649211035177  

   Chou, Glen; Berenson, Dmitry; Ozay, Necmiye (September 2021, The International Journal of Robotics Research)

   We extend the learning from demonstration paradigm by providing a method for learning unknown constraints shared across tasks, using demonstrations of the tasks, their cost functions, and knowledge of the system dynamics and control constraints. Given safe demonstrations, our method uses hit-and-run sampling to obtain lower cost, and thus unsafe, trajectories. Both safe and unsafe trajectories are used to obtain a consistent representation of the unsafe set via solving an integer program. Our method generalizes across system dynamics and learns a guaranteed subset of the constraint. In addition, by leveraging a known parameterization of the constraint, we modify our method to learn parametric constraints in high dimensions. We also provide theoretical analysis on what subset of the constraint and safe set can be learnable from safe demonstrations. We demonstrate our method on linear and nonlinear system dynamics, show that it can be modified to work with suboptimal demonstrations, and that it can also be used to learn constraints in a feature space. 

   more » « less
2. On Approximate Thompson Sampling with Langevin Algorithms

   Mazumdar, Eric; Pacchiano, Aldo; Ma, Yian; Jordan, Michael; Bartlett, Peter (January 2020, Proceedings of the 37th International Conference on Machine Learning)

   Daumé III, Hal; Singh, Aarti (Ed.)

   Thompson sampling for multi-armed bandit problems is known to enjoy favorable performance in both theory and practice. However, its wider deployment is restricted due to a significant computational limitation: the need for samples from posterior distributions at every iteration. In practice, this limitation is alleviated by making use of approximate sampling methods, yet provably incorporating approximate samples into Thompson Sampling algorithms remains an open problem. In this work we address this by proposing two efficient Langevin MCMC algorithms tailored to Thompson sampling. The resulting approximate Thompson Sampling algorithms are efficiently implementable and provably achieve optimal instance-dependent regret for the Multi-Armed Bandit (MAB) problem. To prove these results we derive novel posterior concentration bounds and MCMC convergence rates for log-concave distributions which may be of independent interest. 

   more » « less
3. The cognitive instability aspect of impulsivity predicts the ERN: An ERP study

   https://doi.org/10.1016/j.ijpsycho.2025.113206  

   Folivi, Folly; Denaro, Chandlyr M; Hartley, Alan A; Bukach, Cindy M; Couperus, Jane W; Reed, Catherine L (August 2025, International Journal of Psychophysiology)

   Introduction: Defined as a tendency to act without thinking or considering the consequences, impulsivity may affect the ability to detect and respond to errors. However, impulsivity is a multidimensional construct with attentional, motor and non-planning components, among others. Not all aspects of impulsivity may relate to error monitoring. In this event-related potential (ERP) study we used an individual difference approach with a large sample of healthy young adults (n = 261) and the flanker task to explore what specific facets of impulsivity were predictive of error monitoring as indexed by the error-related negativity (ERN). Methods: The Barratt Impulsiveness Scale (BIS-11) was used to measure impulsivity and its subcomponents. A visual flanker task was administered to elicit the commission of errors and the associated ERN. Results: BIS-11 total scores did not correlate with ERN amplitude. Using an exploratory strategy, we first regressed scores for six previously identified components of impulsivity on the ERN, finding a significant coefficient for cognitive instability. Because internal consistency was low, we next conducted a principal component analysis of the 30 BIS-11 items; three factors emerged: planning, impetuosity and cognitive instability. When the three scale scores were regressed on ERN amplitudes, only cognitive instability ("racing thoughts") was predictive, associating greater cognitive instability with reduced ERN amplitudes. Conclusions: Increases in the cognitive instability aspect of impulsivity predicts reduced ERN amplitudes, which may be related to individual differences in the motivational salience of errors. 

   more » « less
4. On the Variety and Veracity of Cyber Intrusion Alerts Synthesized by Generative Adversarial Networks

   https://doi.org/10.1145/3394503  

   Sweet, Christopher Ryan; Moskal, Stephen; Yang, Shanchieh (April 2020, ACM Transactions on Management Information Systems)

   Many cyber attack actions can be observed but the observables often exhibit intricate feature dependencies, non-homogeneity, and potential for rare yet critical samples. This work tests the ability to model and synthesize cyber intrusion alerts through Generative Adversarial Networks (GANs), which explore the feature space through reconciling between randomly generated samples and the given data that reflects a mixture of diverse attack behaviors. Through a comprehensive analysis using Jensen-Shannon Divergence (JSD), conditional and joint entropy, and mode drops and additions, we show that the Wasserstein-GAN with Gradient Penalty and Mutual Information (WGAN-GPMI) is more effective in learning to generate realistic alerts than models without Mutual Information constraints. The added Mutual Information constraint pushes the model to explore the feature space more thoroughly and increases the generation of low probability yet critical alert features. By mapping alerts to a set of attack stages it is shown that the output of these low probability alerts has a direct contextual meaning for cyber security analysts. Overall, our results show the promising novel use of GANs to learn from limited yet diverse intrusion alerts to generate synthetic ones that emulate critical dependencies, opening the door to data driven network threat models. 

   more » « less
5. Neural representational geometry underlies few-shot concept learning

   https://doi.org/10.1073/pnas.2200800119  

   Sorscher, Ben; Ganguli, Surya; Sompolinsky, Haim (October 2022, Proceedings of the National Academy of Sciences)

   Understanding the neural basis of the remarkable human cognitive capacity to learn novel concepts from just one or a few sensory experiences constitutes a fundamental problem. We propose a simple, biologically plausible, mathematically tractable, and computationally powerful neural mechanism for few-shot learning of naturalistic concepts. We posit that the concepts that can be learned from few examples are defined by tightly circumscribed manifolds in the neural firing-rate space of higher-order sensory areas. We further posit that a single plastic downstream readout neuron learns to discriminate new concepts based on few examples using a simple plasticity rule. We demonstrate the computational power of our proposal by showing that it can achieve high few-shot learning accuracy on natural visual concepts using both macaque inferotemporal cortex representations and deep neural network (DNN) models of these representations and can even learn novel visual concepts specified only through linguistic descriptors. Moreover, we develop a mathematical theory of few-shot learning that links neurophysiology to predictions about behavioral outcomes by delineating several fundamental and measurable geometric properties of neural representations that can accurately predict the few-shot learning performance of naturalistic concepts across all our numerical simulations. This theory reveals, for instance, that high-dimensional manifolds enhance the ability to learn new concepts from few examples. Intriguingly, we observe striking mismatches between the geometry of manifolds in the primate visual pathway and in trained DNNs. We discuss testable predictions of our theory for psychophysics and neurophysiological experiments. 

   more » « less