More Like this

1. Typological gaps in iambic nonfinality correlate with learning difficulty

   https://doi.org/10.3765/amp.v9i0.5175  

   Pater, Joe; Prickett, Brandon (August 2022, Proceedings of the Annual Meetings on Phonology)

   This paper discusses gaps in stress typology that are unexpected from the perspective of a foot-based theory and shows that the patterns pose difficulties for a computationally implemented learning algorithm. The unattested patterns result from combining theoretical elements whose effects are generally well-attested, including iambic footing, nonfinality, word edge alignment and a foot binarity requirement. The patterns can be found amongst the 124 target stress systems constructed by Tesar and Smolensky (2000) as a test of their approach to hidden structure learning. A learner with a Maximum Entropy grammar that uses a form of Expectation Maximization to deal with hidden structure was found to often fail on these unattested languages. 

   more » « less
2. Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning

   Nye, Maxwell; Tessler, Michael Henry; Tenenbaum, Joshua B; Lake, Brenden M (July 2021, ArXivorg)

   null (Ed.)

   Human reasoning can often be understood as an interplay between two systems: the intuitive and associative (“System 1”) and the deliberative and logical (“System 2”). Neural sequence models—which have been increasingly successful at performing complex, structured tasks—exhibit the advantages and failure modes of System 1: they are fast and learn patterns from data, but are often inconsistent and incoherent. In this work, we seek a lightweight, training-free means of improving existing System 1-like sequence models by adding System 2-inspired logical reasoning. We explore several variations on this theme in which candidate generations from a neural sequence model are examined for logical consistency by a symbolic reasoning module, which can either accept or reject the generations. Our approach uses neural inference to mediate between the neural System 1 and the logical System 2. Results in robust story generation and grounded instruction-following show that this approach can increase the coherence and accuracy of neurally-based generations. 

   more » « less
3. Long sequence Hopfield memory

   https://doi.org/10.1088/1742-5468/ad6427  

   Tahir_Chaudhry, Hamza; Zavatone-Veth, Jacob A; Krotov, Dmitry; Pehlevan, Cengiz (October 2024, Journal of Statistical Mechanics: Theory and Experiment)

   Abstract Sequence memory is an essential attribute of natural and artificial intelligence that enables agents to encode, store, and retrieve complex sequences of stimuli and actions. Computational models of sequence memory have been proposed where recurrent Hopfield-like neural networks are trained with temporally asymmetric Hebbian rules. However, these networks suffer from limited sequence capacity (maximal length of the stored sequence) due to interference between the memories. Inspired by recent work on Dense Associative Memories, we expand the sequence capacity of these models by introducing a nonlinear interaction term, enhancing separation between the patterns. We derive novel scaling laws for sequence capacity with respect to network size, significantly outperforming existing scaling laws for models based on traditional Hopfield networks, and verify these theoretical results with numerical simulation. Moreover, we introduce a generalized pseudoinverse rule to recall sequences of highly correlated patterns. Finally, we extend this model to store sequences with variable timing between states’ transitions and describe a biologically-plausible implementation, with connections to motor neuroscience. 

   more » « less
4. Using Sequence Analysis to Identify Conversational Motifs in Supportive Interactions

   https://doi.org/10.1177/02654075211066618  

   Solomon, Denise Haunani; Jones, Susanne; Brinberg, Miriam; Bodie, Graham D.; Ram, Nilam (January 2022, Journal of Social and Personal Relationships)

   This study demonstrates how sequence analysis, which is a method for identifying common patterns in categorical time series data, illuminates the nonlinear dynamics of dyadic conversations by describing chains of behavior that shift categorically, rather than incrementally. When applied to interpersonal interactions, sequence analysis supports the identification of conversational motifs, which can be used to test hypotheses linking patterns of interaction to conversational antecedents or outcomes. As an illustrative example, this study evaluated 285 conversations involving stranger, friend, and dating dyads in which one partner, the discloser, communicated about a source of stress to a partner in the role of listener. Using sequence analysis, we identified three five-turn supportive conversational motifs that had also emerged in a previous study of stranger dyads: discloser problem description, discloser problem processing, and listener-focused dialogue. We also observed a new, fourth motif: listener-focused, discloser questioning. Tests of hypotheses linking the prevalence and timing of particular motifs to the problem discloser’s emotional improvement and perceptions of support quality, as moderated by the discloser’s pre-interaction stress, offered a partial replication of previous findings. The discussion highlights the value of using sequence analysis to illuminate dynamic patterns in dyadic interactions. 

   more » « less
5. Clusters of acidic and hydrophobic residues can predict acidic transcriptional activation domains from protein sequence

   https://doi.org/10.1093/genetics/iyad131  

   Kotha, Sanjana R; Staller, Max Valentín (July 2023, GENETICS)

   Kaplan, C (Ed.)

   Abstract Transcription factors activate gene expression in development, homeostasis, and stress with DNA binding domains and activation domains. Although there exist excellent computational models for predicting DNA binding domains from protein sequence, models for predicting activation domains from protein sequence have lagged, particularly in metazoans. We recently developed a simple and accurate predictor of acidic activation domains on human transcription factors. Here, we show how the accuracy of this human predictor arises from the clustering of aromatic, leucine, and acidic residues, which together are necessary for acidic activation domain function. When we combine our predictor with the predictions of convolutional neural network (CNN) models trained in yeast, the intersection is more accurate than individual models, emphasizing that each approach carries orthogonal information. We synthesize these findings into a new set of activation domain predictions on human transcription factors. 

   more » « less