An official website of the United States government
Sign language recognition and translation technologies have the potential to increase access and inclusion of deaf signing communities, but research progress is bottlenecked by a lack of representative data. We introduce a new resource for American Sign Language (ASL) modeling, the Sem-Lex Benchmark. The Benchmark is the current largest of its kind, consisting of over 84k videos of isolated sign productions from deaf ASL signers who gave informed consent and received compensation. Human experts aligned these videos with other sign language resources including ASL-LEX, SignBank, and ASL Citizen, enabling useful expansions for sign and phonological feature recognition. We present a suite of experiments which make use of the linguistic information in ASL-LEX, evaluating the practicality and fairness of the Sem-Lex Benchmark for isolated sign recognition (ISR). We use an SL-GCN model to show that the phonological features are recognizable with 85% accuracy, and that they are effective as an auxiliary target to ISR. Learning to recognize phonological features alongside gloss results in a 6% improvement for few-shot ISR accuracy and a 2% improvement for ISR accuracy overall. Instructions for downloading the data can be found at https://github.com/leekezar/SemLex.
more »
« less
Strict Locality in Syntax
Lexical selection, functional hierarchies, and adjunct ordering are arguably distinct parts of syntax, yet are surprisingly similar in their computational properties. All three fall within the formal class strictly local (SL) and thus are maximally simple. Many phonological patterns are also SL, motivating a more detailed comparison. Towards this end, I develop a model based on com- mand strings (Graf & Shafiei 2019) which allows syntactic and phonological grammars to be visualized using finite-state automata. Using this model, I show that the same basic patterns allowed within SL occur in both domains.
more »
« less
- Award ID(s):
- 1845344
- PAR ID:
- 10496690
- Publisher / Repository:
- Proceedings of CLS 2023
- Date Published:
- Journal Name:
- Proceedings of CLS 2023
- Format(s):
- Medium: X
- Location:
- University of Chicago, Chicago, IL
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Statistical learning (SL) is typically assumed to be a core mechanism by which organisms learn covarying structures and recurrent patterns in the environment, with the main purpose of facilitating processing of expected events. Within this theoretical framework, the environment is viewed as relatively stable, and SL ‘captures’ the regularities therein through implicit unsupervised learning by mere exposure. Focusing primarily on language— the domain in which SL theory has been most influential—we review evidence that the environment is far from fixed: it is dynamic, in continual flux, and learners are far from passive absorbers of regularities; they interact with their environments, thereby selecting and even altering the patterns they learn from. We therefore argue for an alternative cognitive architecture, where SL serves as a subcomponent of an information foraging (IF) system. IF aims to detect and assimilate novel recurrent patterns in the input that deviate from randomness, for which SL supplies a baseline. The broad implications of this viewpoint and their relevance to recent debates in cognitive neuroscience are discussed.more » « less
-
This study examines the acoustic realizations of American English intervocalic flaps in the TIMIT corpus, using the landmark-critical feature-cue-based framework. Three different acoustic patterns of flaps are described: (i) both closure and release landmarks present, (ii) only the closure landmark present, and (iii) both landmarks deleted. The patterns occur consistently across several phonological and morphological conditions but vary with sociolinguistic factors, including speaker dialect and gender. This method of analysing speech at the level of acoustic landmarks and other individual cues to distinctive features contributes to a deeper understanding of how speakers and listeners employ systematic variation in phonetic detail in speech processing.more » « less
-
Objective: This study aimed to evaluate lenition, a phonological process involving consonant weakening, as a diagnostic marker for differentiating Parkinson’s Disease (PD) from Atypical Parkinsonism (APD). Early diagnosis is critical for optimizing treatment outcomes, and lenition patterns in stop consonants may provide valuable insights into the distinct motor speech impairments associated with these conditions. Methods: Using Phonet, a machine learning model trained to detect phonological features, we analyzed the posterior probabilities of continuant and sonorant features from the speech of 142 participants (108 PD, 34 APD). Lenition was quantified based on deviations from expected values, and linear mixed-effects models were applied to compare phonological patterns between the two groups. Results: PD patients exhibited more stable articulatory patterns, particularly in preserving the contrast between voiced and voiceless stops. In contrast, APD patients showed greater lenition, particularly in voiceless stops, coupled with increased articulatory variability, reflecting a more generalized motor deficit. Conclusions: Lenition patterns, especially in voiceless stops, may serve as non-invasive markers for distinguishing PD from APD. These findings suggest potential applications in early diagnosis and tracking disease progression. Future research should expand the analysis to include a broader range of phonological features and contexts to improve diagnostic accuracy.more » « less
-
This paper is the first of a pair that aims to classify a large number of the type I I II quantum subgroups of the categories C ( s l r + 1 , k ) \mathcal {C}(\mathfrak {sl}_{r+1}, k) . In this work we classify the braided auto-equivalences of the categories of local modules for all known type I I quantum subgroups of C ( s l r + 1 , k ) \mathcal {C}(\mathfrak {sl}_{r+1}, k) . We find that the symmetries are all non-exceptional except for four cases (up to level-rank duality). These exceptional cases are the orbifolds C ( s l 2 , 16 ) Rep ( Z 2 ) 0 \mathcal {C}(\mathfrak {sl}_{2}, 16)^0_{\operatorname {Rep}(\mathbb {Z}_{2})} , C ( s l 3 , 9 ) Rep ( Z 3 ) 0 \mathcal {C}(\mathfrak {sl}_{3}, 9)^0_{\operatorname {Rep}(\mathbb {Z}_{3})} , C ( s l 4 , 8 ) Rep ( Z 4 ) 0 \mathcal {C}(\mathfrak {sl}_{4}, 8)^0_{\operatorname {Rep}(\mathbb {Z}_{4})} , and C ( s l 5 , 5 ) Rep ( Z 5 ) 0 \mathcal {C}(\mathfrak {sl}_{5}, 5)^0_{\operatorname {Rep}(\mathbb {Z}_{5})} . We develop several technical tools in this work. We give a skein theoretic description of the orbifold quantum subgroups of C ( s l r + 1 , k ) \mathcal {C}(\mathfrak {sl}_{r+1}, k) . Our methods here are general, and the techniques developed will generalise to give skein theory for any orbifold of a braided tensor category. We also give a formulation of orthogonal level-rank duality in the type D D - D D case, which is used to construct one of the exceptionals. We uncover an unexpected connection between quadratic categories and exceptional braided auto-equivalences of the orbifolds. We use this connection to construct two of the four exceptionals. In the sequel to this paper we will use the classified braided auto-equivalences to construct the corresponding type I I II quantum subgroups of the categories C ( s l r + 1 , k ) \mathcal {C}(\mathfrak {sl}_{r+1}, k) . This will essentially finish the type I I II classification for s l n \mathfrak {sl}_n modulo type I I classification. When paired with Gannon’s type I I classification for r ≤ 6 r\leq 6 , our results will complete the type I I II classification for these same ranks. This paper includes an appendix by Terry Gannon, which provides useful results on the dimensions of objects in the categories C ( s l r + 1 , k ) \mathcal {C}(\mathfrak {sl}_{r+1}, k) .more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
