More Like this

1. Variational Optimization on Lie Groups, with Examples of Leading (Generalized) Eigenvalue Problems

   Tao, Molei; Ohsawa, Tomoki (August 2020, Proceedings of Machine Learning Research)

   Chiappa, Silvia; Calandra, Roberto (Ed.)

   The article considers smooth optimization of functions on Lie groups. By generalizing NAG variational principle in vector space (Wibisono et al., 2016) to general Lie groups, continuous Lie-NAG dynamics which are guaranteed to converge to local optimum are obtained. They correspond to momentum versions of gradient flow on Lie groups. A particular case of SO(𝑛) is then studied in details, with objective functions corresponding to leading Generalized EigenValue problems: the Lie-NAG dynamics are first made explicit in coordinates, and then discretized in structure preserving fashions, resulting in optimization algorithms with faithful energy behavior (due to conformal symplecticity) and exactly remaining on the Lie group. Stochastic gradient versions are also investigated. Numerical experiments on both synthetic data and practical problem (LDA for MNIST) demonstrate the effectiveness of the proposed methods as optimization algorithms (\emph{not} as a classification method). 

   more » « less
2. Learning problem decomposition-recomposition with data-driven chunky parsons problem within an intelligent logic tutor.

   Shabrina, P; Mostafavi, B; Tithi, D; Chi, M; Barnes, T (July 2023, Springer)

   Problem decomposition into sub-problems or subgoals and recomposition of the solutions to the subgoals into one complete solution is a common strategy to reduce difficulties in structured problem solving. In this study, we use a datadriven graph-mining-based method to decompose historical student solutions of logic-proof problems into Chunks. We design a new problem type where we present these chunks in a Parsons Problem fashion and asked students to reconstruct the complete solution from the chunks. We incorporated these problems within an intelligent logic tutor and called them Chunky Parsons Problems (CPP). These problems demonstrate the process of problem decomposition to students and require them to pay attention to the decomposed solution while they reconstruct the complete solution. The aim of introducing CPP was to improve students’ problem-solving skills and performance by improving their decomposition-recomposition skills without significantly increasing training difficulty. Our analysis showed that CPPs could be as easy as Worked Examples (WE). And, students who received CPP with simple explanations attached to the chunks had marginally higher scores than those who received CPPs without explanation or did not receive them. Also, the normalized learning gain of these students shifted more towards the positive side than other students. Finally, as we looked into their proof-construction traces in posttest problems, we observed them to form identifiable chunks aligned with those found in historical solutions with higher efficiency. 

   more » « less
3. Leader Stochastic Gradient Descent for Distributed Training of Deep Learning Models

   Teng, Y.; Gao, W.; Chalus, F.; Choromanska, A. E.; Goldfarb, D.; Weller, A. (January 2019, Advances in neural information processing systems)

   We consider distributed optimization under communication constraints for training deep learning models. We propose a new algorithm, whose parameter updates rely on two forces: a regular gradient step, and a corrective direction dictated by the currently best-performing worker (leader). Our method differs from the parameter-averaging scheme EASGD in a number of ways:(i) our objective formulation does not change the location of stationary points compared to the original optimization problem;(ii) we avoid convergence decelerations caused by pulling local workers descending to different local minima to each other (ie to the average of their parameters);(iii) our update by design breaks the curse of symmetry (the phenomenon of being trapped in poorly generalizing sub-optimal solutions in symmetric non-convex landscapes); and (iv) our approach is more communication efficient since it broadcasts only parameters of the leader rather than all workers. We provide theoretical analysis of the batch version of the proposed algorithm, which we call Leader Gradient Descent (LGD), and its stochastic variant (LSGD). Finally, we implement an asynchronous version of our algorithm and extend it to the multi-leader setting, where we form groups of workers, each represented by its own local leader (the best performer in a group), and update each worker with a corrective direction comprised of two attractive forces: one to the local, and one to the global leader (the best performer among all workers). The multi-leader setting is well-aligned with current hardware architecture, where local workers forming a group lie within a single computational node and … 

   more » « less
4. Kashiwara–Vergne solutions degree by degree

   https://doi.org/10.5802/crmath.750  

   Dancso, Zsuzsanna; Halacheva, Iva; Laplante-Anfossi, Guillaume; Robertson, Marcy (June 2025, Comptes Rendus. Mathématique)

   We show that solutions to the Kashiwara–Vergne problem can be extended degree by degree. This can be used to simplify the computation of a class of Drinfel’d associators, which under the Alekseev–Torossian conjecture, may comprise all associators. We also give a proof that the associated graded Lie algebra of the Kashiwara–Vergne group is isomorphic to the graded Kashiwara–Vergne Lie algebra. 

   more » « less
5. Well-posedness of mean field games master equations involving non-separable local Hamiltonians

   https://doi.org/10.1090/tran/8760  

   Ambrose, David; Mészáros, Alpár (January 2023, Transactions of the American Mathematical Society)

   In this paper we construct short time classical solutions to a class of master equations in the presence of non-degenerate individual noise arising in the theory of mean field games. The considered Hamiltonians are non-separable andlocalfunctions of the measure variable, therefore the equation is restricted to absolutely continuous measures whose densities lie in suitable Sobolev spaces. Our results hold for smooth enough Hamiltonians, without any additional structural conditions as convexity or monotonicity. 

   more » « less