More Like this

1. On effects that do occur versus effects that can be made to occur

   https://doi.org/10.3389/frsps.2023.1193349  

   Brauer, Markus; Kennedy, Kevin R (August 2023, Frontiers in Social Psychology)

   Following Douglas Mook's lead we distinguish between research on “effects that can be made to occur” and research on “effects that do occur” and argue that both can contribute to the advancement of knowledge. We further suggest that current social psychological research focuses too much on the former type of effects. Given the discipline's emphasis on innovation, many published effects are shown to exist under very specific circumstances, i.e., when numerous moderator variables are set at a particular level. One often does not know, however, how frequently these circumstances exist for people in the real world. Studies on effects that can be made to occur are thus an incomplete test of most theories about human cognition and behavior. Using concrete examples, this article discusses the shortcomings of a field that limits itself to identifying effects that might—or might not—be relevant. We argue that it is just as much a scientific contribution to show that a given effect actually does occur as it is to provide initial evidence for a new effect that could turn out to be important. The article ends with a series of suggestions for researchers who want to increase the theoretical and practical relevance of their research. 

   more » « less
2. Do students think that objects have a true definite position?

   https://doi.org/10.1103/PhysRevPhysEducRes.20.010154  

   Stump, Emily M; Hughes, Mark; Holmes, N G; Passante, Gina (June 2024, Physical Review Physics Education Research)

   Previous research on student thinking about experimental measurement and uncertainty has primarily focused on students’ procedural reasoning: Given some data, what should students calculate or do next? This approach, however, cannot tell us what beliefs or conceptual understanding leads to students’ procedural decisions. To explore this relationship, we first need to understand the range of students’ beliefs and conceptual understanding of measurement. In this work, we explored students’ philosophical beliefs about the existence of a true value in experimental measurement. We distributed a survey to students from 12 universities in which we presented two viewpoints on the existence of a true definite position resulting from an experiment, asking participants to indicate which view they agreed with more and asking them to explain their choice. We found that participants, both students and experts, varied in their beliefs about the existence of a true definite position and discussed a range of concepts related to quantum mechanics and the experimental process to explain their answers, regardless of whether or not they agreed with the existence of a true value. From these results, we postulate that students who exhibit similar procedural reasoning may hold widely varying philosophical views about measurement. We recommend that future work investigates this potential relationship and whether and how instruction should attend to these philosophical views in addition to students’ procedural decisions. 

   more » « less
3. On the generalization of learning algorithms that do not converge.

   N. Chandramoorthy, A. Loukas (November 2022, Neural Information Processing Systems (NeurIPS))

   Generalization analyses of deep learning typically assume that the training converges to a fixed point. But, recent results indicate that in practice, the weights of deep neural networks optimized with stochastic gradient descent often oscillate indefinitely. To reduce this discrepancy between theory and practice, this paper focuses on the generalization of neural networks whose training dynamics do not necessarily converge to fixed points. Our main contribution is to propose a notion of statistical algorithmic stability (SAS) that extends classical algorithmic stability to non-convergent algorithms and to study its connection to generalization. This ergodic-theoretic approach leads to new insights when compared to the traditional optimization and learning theory perspectives. We prove that the stability of the time-asymptotic behavior of a learning algorithm relates to its generalization and empirically demonstrate how loss dynamics can provide clues to generalization performance. Our findings provide evidence that networks that “train stably generalize better” even when the training continues indefinitely and the weights do not converge. 

   more » « less
4. What would jiminy cricket do? towards agents that behave morally

   Hendrycks, Dan; Zhu, Christine; Mazeika, Mantas; Navarro, Jesus; Song, Dawn; Zou, Andy; Li, Bo; Patel, Sahil; Steinhardt, Jacob (January 2022, Advances in neural information processing systems)
5. Do Machine Learning Models Produce TypeScript Types that Type Check?

   Ming-Ho Yee; Arjun Guha (January 2023, European Conference on Object Oriented Programming (ECOOP))

   Type migration is the process of adding types to untyped code to gain assurance at compile time. TypeScript and other gradual type systems facilitate type migration by allowing programmers to start with imprecise types and gradually strengthen them. However, adding types is a manual effort and several migrations on large, industry codebases have been reported to have taken years. In the research community, there has been significant interest in using machine learning to automate TypeScript type migration. Existing machine learning models report a high degree of accuracy in predicting individual TypeScript type annotations. However, in this paper we argue that accuracy can be misleading, and we should address a different question: can an automatic type migration tool produce code that passes the TypeScript type checker? We present TypeWeaver, a TypeScript type migration tool into which one can plug in an arbitrary type prediction model. We evaluate TypeWeaver with three models from the literature: DeepTyper (a recurrent neural network), LambdaNet (a graph neural network), and InCoder (a general-purpose, multi-language transformer that supports fill-in-the-middle tasks). Our tool automates several steps that are necessary to use a type prediction model, including (1) importing types for a project’s dependencies; (2) migrating JavaScript modules to TypeScript notation; (3) inserting predicted type annotations into the program to produce TypeScript when needed; and (4) rejecting non-type predictions when needed. We evaluate TypeWeaver on a dataset of 513 JavaScript packages, including packages that have never been typed before. With the best type prediction model, we find that only 21% of packages type check, but more encouragingly, 69% of files type check successfully. 

   more » « less