More Like this

1. pyAMPACT: A Score-Audio Alignment Toolkit for Performance Data Estimation and Multi-modal Processing

   Devaney, Johanna; McKemie, Daniel; Morgan, Alexander (June 2025, 2025 International Computer Music Conference)

   pyAMPACT (Python-based Automatic Music Performance Analysis and Comparison Toolkit) links symbolic and audio music representations to facilitate score-informed estimation of performance data from audio. It can read a range of symbolic formats and can output note-linked audio descriptors/performance data into MEI-formatted files. pyAMPACT uses score alignment to calculate time-frequency regions of importance for each note in the symbolic representation from which it estimates a range of parameters from the corresponding audio. These include frame-wise and note-level tuning-, dynamics-, and timbre-related performance descriptors, with timing-related information available from the score alignment. Beyond performance data estimation, pyAMPACT also facilitates multi-modal investigations through its infrastructure for linking symbolic representations and annotations to audio. 

   more » « less
2. Practical Cross-Modal Manifold Alignment for Robotic Grounded Language Learning

   https://doi.org/10.1109/CVPRW53098.2021.00177  

   Nguyen, Andre T.; Richards, Luke E.; Kebe, Gaoussou Youssouf; Raff, Edward; Darvish, Kasra; Ferraro, Francis; Matuszek, Cynthia (June 2021, IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2021)

   We propose a cross-modality manifold alignment procedure that leverages triplet loss to jointly learn consistent, multi-modal embeddings of language-based concepts of real-world items. Our approach learns these embeddings by sampling triples of anchor, positive, and negative data points from RGB-depth images and their natural language descriptions. We show that our approach can benefit from, but does not require, post-processing steps such as Procrustes analysis, in contrast to some of our baselines which require it for reasonable performance. We demonstrate the effectiveness of our approach on two datasets commonly used to develop robotic-based grounded language learning systems, where our approach outperforms four baselines, including a state-of-the-art approach, across five evaluation metrics. 

   more » « less
3. Global hypocoercivity of kinetic Fokker-Planck-Alignment equations

   https://doi.org/10.3934/krm.2022005  

   Shvydkoy, Roman (January 2022, Kinetic and Related Models)

   In this note we establish hypocoercivity and exponential relaxation to the Maxwellian for a class of kinetic Fokker-Planck-Alignment equations arising in the studies of collective behavior. Unlike previously known results in this direction that focus on convergence near Maxwellian, our result is global for hydrodynamically dense flocks, which has several consequences. In particular, if communication is long-range, the convergence is unconditional. If communication is local then all nearly aligned flocks quantified by smallness of the Fisher information relax to the Maxwellian. In the latter case the class of initial data is stable under the vanishing noise limit, i.e. it reduces to a non-trivial and natural class of traveling wave solutions to the noiseless Vlasov-Alignment equation.The main novelty in our approach is the adaptation of a mollified Favre filtration of the macroscopic momentum into the communication protocol. Such filtration has been used previously in large eddy simulations of compressible turbulence and its new variant appeared in the proof of the Onsager conjecture for inhomogeneous Navier-Stokes system. A rigorous treatment of well-posedness for smooth solutions is provided. Lastly, we prove that in the limit of strong noise and local alignment solutions to the Fokker-Planck-Alignment equation Maxwellialize to solutions of the macroscopic hydrodynamic system with the isothermal pressure. 

   more » « less
4. Interspecific Variation of Calcaneal Morphology in Gorillas

   Harper, Christine M.; Ruff, Christopher B.; Sylvester, Adam D. (February 2019, American journal of physical anthropology)

   The foot plays a prominent role in weight-bearing suggesting it may reflect locomotor variation. Despite the immense amount of foot research, the calcaneus has been relatively understudied. Here we analyzed the entire calcaneal shape of Gorilla gorilla gorilla (n=41), Gorilla beringei graueri (n=17) and Gorilla beringei beringei (n=8) to understand how morphology relates to locomotor behavior. Calcanei were surface scanned and external shape analyzed using a three-dimensional geometric morphometric sliding semilandmark analysis. Semilandmarks were slid to minimize the bending energy of the thin plate spline interpolation function relative to the updated Procrustes average. Generalized Procrustes Analysis was used to align landmark configurations and shape variation was summarized using a principal components analysis. Procrustes distances between species were calculated and resampling statistics were run to test for group differences. All subspecies demonstrate statistically different morphologies (p<0.005 for pairwise comparisons). G. b. graueri separates from other subspecies based on posterolateral morphology, with G. b. graueri demonstrating an elongated peroneal trochlea, and thus more bone superiorly than G. g. gorilla. Compared to G. b. beringei, G. b. graueri has less bone inferiorly near the tuberosity. Cuboid and posterior talar facet shapes correlate with arboreality. G. b. beringei (most terrestrial) has a flatter cuboid facet and a more transversely oriented/relatively smaller posterior talar facet than G. g. gorilla (most arboreal) and G. b. graueri represents an intermediate morphology. These differences demonstrate a relationship between calcaneal shape and locomotor behavior and suggest that G. b. graueri may load its foot differently from the other subspecies. This project was supported by NSF grant # BCS - 1824630. 

   more » « less
5. The Shape of Sound: a Geometric Morphometrics Approach to Laryngeal Functional Morphology

   https://doi.org/10.1007/s10914-019-09466-9  

   Borgard, Heather L; Baab, Karen; Pasch, Bret; Riede, Tobias (May 2019, Journal of Mammalian Evolution)

   Diversification of animal vocalizations plays a key role in behavioral evolution and speciation. Vocal organ morphology represents an important source of acoustic variation, yet its small size, complex shape, and absence of homologous landmarks pose major challenges to comparative analyses. Here, we use a geometric morphometric approach based on geometrically homologous landmarks to quantify shape variation of laryngeal cartilages of four rodent genera representing three families. Reconstructed cartilages of the larynx from contrast-enhanced micro-CT images were quantified by variable numbers of three-dimensional landmarks placed on structural margins and major surfaces. Landmark sets were superimposed using generalized Procrustes analysis prior to statistical analysis. Correlations among pairwise Procrustes distances were used to identify the minimum number of landmarks necessary to fully characterize shape variation. We found that the five species occupy distinct positions in morphospace, with variation explained in part by phylogeny, body size, and differences in vocal production mechanisms. Our findings provide a foundation for quantifying the contribution of vocal organ morphology to acoustic diversification. 

   more » « less