More Like this

1. Good-enough language production

   https://doi.org/10.1016/j.tics.2022.01.005  

   Goldberg, Adele E.; Ferreira, Fernanda (April 2022, Trends in Cognitive Sciences)
2. Is Bearing Witness Enough?

   https://doi.org/10.1111/aman.13078  

   Hernandez, Christopher (September 2018, American Anthropologist)
3. Open is not enough

   https://doi.org/10.1038/s41567-018-0342-2  

   Chen, Xiaoli; Dallmeier-Tiessen, Sünje; Dasler, Robin; Feger, Sebastian; Fokianos, Pamfilos; Gonzalez, Jose Benito; Hirvonsalo, Harri; Kousidis, Dinos; Lavasa, Artemis; Mele, Salvatore; et al (February 2019, Nature Physics)
4. Attention is not enough

   Miller, Joshua and (January 2022, Proceedings of the Annual Meeting of the Cognitive Science Society)
5. Attention Is Not Enough

   Miller, Joshua; Naderi, Shawheen; Millinax, Channing B.; Phillips, Joshua L. (July 2022, Proceedings of the Annual Conference of the Cognitive Science Society)

   The human ability to generalize beyond interpolation, often called extrapolation or symbol-binding, is challenging to recreate with computational models. Biologically plausible models incorporating indirection mechanisms have demonstrated strong performance in this regard. Deep learning approaches such as Long Short-Term Memory (LSTM) and Transformers have shown varying degrees of success, but recent work has suggested that Transformers are capable of extrapolation as well. We evaluate the capabilities of the above approaches on a series of increasingly complex sentence-processing tasks to infer the capacity of each individual architecture to extrapolate sentential roles across novel word fillers. We confirm that the Transformer does possess superior abstraction capabilities compared to LSTM. However, what it does not possess is extrapolation capabilities, as evidenced by clear performance disparities on novel filler tasks as compared to working memory-based indirection models. 

   more » « less