Search for: All records

Computational models of distributional semantics can analyze a corpus to derive representations of word meanings in terms of each word’s relationship to all other words in the corpus. While these models are sensitive to topic (e.g., tiger and stripes) and synonymy (e.g., soar and fly), the models have limited sensitivity to part of speech (e.g., book and shirt are both nouns). By augmenting a holographic model of semantic memory with additional levels of representations, we present evidence that sensitivity to syntax is supported by exploiting associations between words at varying degrees of separation. We find that sensitivity to associations at three degrees of separation reinforces the relationships between words that share part-of-speech and improves the ability of the model to construct grammatical sentences. Our model provides evidence that semantics and syntax exist on a continuum and emerge from a unitary cognitive system. 

We present analysis of existing memory models, examining how models represent knowledge, structure memory, learn, make decisions, and predict reaction times. On the basis of this analysis, we propose a theoretical framework that characterizes memory modelling in terms of six key decisions: (1) choice of knowledge representation scheme, (2) choice of data structure, (3) choice of associative architecture, (4) choice of learning rule, (5) choice of time variant process, and (6) choice of response decision criteria. This framework is both descriptive and proscriptive: we intend to both describe the state of the literature and outline what we believe is the most fruitful space of possibilities for the development of future memory models. 

We explore replacing the declarative memory system of the ACT-R cognitive architecture with a distributional semantics model. ACT-R is a widely used cognitive architecture, but scales poorly to big data applications and lacks a robust model for learning association strengths between stimuli. Distribu- tional semantics models can process millions of data points to infer semantic similarities from language data or to in- fer product recommendations from patterns of user prefer- ences. We demonstrate that a distributional semantics model can account for the primacy and recency effects in free recall, the fan effect in recognition, and human performance on it- erated decisions with initially unknown payoffs. The model we propose provides a flexible, scalable alternative to ACT- R’s declarative memory at a level of description that bridges symbolic, quantum, and neural models of cognition. 

This paper improves on several aspects of a sieve-based event ordering architecture‚ CAEVO (Chambers et al.‚ 2014)‚ which creates globally consistent temporal relations between events and time expressions. First‚ we examine the usage of word embeddings and semantic role features. With the incorporation of these new features‚ we demonstrate a 5% relative F1 gain over our replicated version of CAEVO. Second‚ we reformulate the architecture’s sieve-based inference algorithm as a prediction reranking method that approximately optimizes a scoring function computed using classifier precisions. Within this prediction reranking framework‚ we propose an alternative scoring function‚ showing an 8.8% relative gain over the original CAEVO. We further include an in-depth analysis of one of the main datasets that is used to evaluate temporal classifiers‚ and we show that in spite of the density of this corpus‚ there is still a danger of overfitting. While this paper focuses on temporal ordering‚ its results are applicable to other areas that use sievebased architectures. 

We present a novel fine-tuning algorithm in a deep hybrid architecture for semi-supervised text classification. During each increment of the online learning process‚ the fine-tuning algorithm serves as a top-down mechanism for pseudo-jointly modifying model parameters following a bottom-up generative learning pass. The resulting model‚ trained under what we call the Bottom-Up-Top-Down learning algorithm‚ is shown to outperform a variety of competitive models and baselines trained across a wide range of splits between supervised and unsupervised training data.