More Like this

1. Development of a questionnaire on teachers' knowledge of language as an epistemic tool

   https://doi.org/DOI: 10.1002/tea.21666  

   Gavin W. Fulmer, Jihyun Hwang (August 2020, Journal of research in science teaching)

   null (Ed.)

   We report on the development of a new instrument for measuring teachers' knowledge of language as an epistemic tool in science classes. Language is essential for science learning, as all learning requires the use of language to constitute one's own ideas and to engage with others' ideas. Teachers with knowledge of language as an epistemic tool can recognize the ways that language allows students to generate and validate knowledge for themselves, rather than to replicate canonical knowledge transmitted by other sources.We used a construct‐driven development approach with iterations of domain analysis, item revision, teacher feedback, expert review, and item piloting to address the content, substance, and structure aspects of validity. Data from 158 preservice and in‐service teachers on 27 preliminary items were collected. Findings from Rasch measurement modeling indicate a single dimension fits the items well and can distinguish teachers of higher and lower knowledge. We revised and selected 15 items for an updated instrument. This contributes to ongoing measurement projects and provides a potential instrument for future, broader use by the field to gauge teachers' knowledge of language as an epistemic tool. 

   more » « less
2. Visual (dis)Confirmation: Validating Models and Hypotheses with Visualizations

   https://doi.org/10.1109/IV-2.2019.00032  

   Choi, In Kwon; Raveendranath, Nirmal Kumar; Westerfield, Jared; Reda, Khairi (July 2019, 23rd International Conference in Information Visualization – Part II)

   Data visualization provides a powerful way for analysts to explore and make data-driven discoveries. However, current visual analytic tools provide only limited support for hypothesis-driven inquiry, as their built-in interactions and workflows are primarily intended for exploratory analysis. Visualization tools notably lack capabilities that would allow users to visually and incrementally test the fit of their conceptual models and provisional hypotheses against the data. This imbalance could bias users to overly rely on exploratory analysis as the principal mode of inquiry, which can be detrimental to discovery. In this paper, we introduce Visual (dis) Confirmation, a tool for conducting confirmatory, hypothesis-driven analyses with visualizations. Users interact by framing hypotheses and data expectations in natural language. The system then selects conceptually relevant data features and automatically generates visualizations to validate the underlying expectations. Distinctively, the resulting visualizations also highlight places where one's mental model disagrees with the data, so as to stimulate reflection. The proposed tool represents a new class of interactive data systems capable of supporting confirmatory visual analysis, and responding more intelligently by spotlighting gaps between one's knowledge and the data. We describe the algorithmic techniques behind this workflow. We also demonstrate the utility of the tool through a case study. 

   more » « less
3. End-user software customization by direct manipulation of tabular data

   https://doi.org/10.1145/3426428.3426914  

   Litt, Geoffrey; Jackson, Daniel; Millis, Tyler; Quaye, Jessica (January 2020, Onward! 2020: Proceedings of the 2020 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software)

   null (Ed.)

   Customizing software should be as easy as using it. Unfortunately, most customization methods require users to abruptly shift from using a graphical interface to writing scripts in a programming language. We introduce data-driven customization, a new way for end users to extend software by direct manipulation without doing traditional programming. We augment existing user interfaces with a table view showing the structured data inside the application. When users edit the table, their changes are reflected in the original UI. This simple model accommodates a spreadsheet formula language and custom data-editing widgets, providing enough power to implement a variety of useful extensions. We illustrate the approach with Wildcard, a browser extension that implements data-driven customization on the web using web scraping. Through concrete examples, we show that this paradigm can support useful extensions to many real websites, and we share reflections from our experiences using the tool. Finally, we share our broader vision for data-driven customization: a future where end users have more access to the data inside their applications, and can more flexibly repurpose that data as part of everyday software usage. 

   more » « less
4. TiQi: A natural language interface for querying software project data

   https://doi.org/10.1109/ASE.2017.8115714  

   Lin, Jinfeng; Liu, Yalin; Guo, Jin; Cleland-Huang, Jane; Goss, William; Liu, Wenchuang; Lohar, Sugandha; Monaikul, Natawut; Rasin, Alexander (October 2017, Automated Software Engineering (ASE), 2017 32nd IEEE/ACM International Conference on)

   Software projects produce large quantities of data such as feature requests, requirements, design artifacts, source code, tests, safety cases, release plans, and bug reports. If leveraged effectively, this data can be used to provide project intelligence that supports diverse software engineering activities such as release planning, impact analysis, and software analytics. However, project stakeholders often lack skills to formulate complex queries needed to retrieve, manipulate, and display the data in meaningful ways. To address these challenges we introduce TiQi, a natural language interface, which allows users to express software-related queries verbally or written in natural language. TiQi is a web-based tool. It visualizes available project data as a prompt to the user, accepts Natural Language (NL) queries, transforms those queries into SQL, and then executes the queries against a centralized or distributed database. Raw data is stored either directly in the database or retrieved dynamically at runtime from case tools and repositories such as Github and Jira. The transformed query is visualized back to the user as SQL and augmented UML, and raw data results are returned. Our tool demo can be found on YouTube at the following link:http://tinyurl.com/TIQIDemo. 

   more » « less
5. EvidenceBot: A Privacy-Preserving, Customizable RAG-Based Tool for Enhancing Large Language Model Interactions

   https://doi.org/10.1145/3696630.3728607  

   Khan, Nafiz Imtiaz; Filkov, Vladimir (June 2025, Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering)

   Large Language Models (LLMs) have become pivotal in reshaping the world by enabling advanced natural language processing tasks such as document analysis, content generation, and conversational assistance. Their ability to process and generate human-like text has unlocked unprecedented opportunities across different domains such as healthcare, education, finance, and more. However, commercial LLM platforms face several limitations, including data privacy concerns, context size restrictions, lack of parameter configurability, and limited evaluation capabilities. These shortcomings hinder their effectiveness, particularly in scenarios involving sensitive information, large-scale document analysis, or the need for customized output. This underscores the need for a tool that combines the power of LLMs with enhanced privacy, flexibility, and usability. To address these challenges, we present EvidenceBot, a local, Retrieval-Augmented Generation (RAG)-based solution designed to overcome the limitations of commercial LLM platforms. Evidence-Bot enables secure and efficient processing of large document sets through its privacy-preserving RAG pipeline, which extracts and appends only the most relevant text chunks as context for queries. The tool allows users to experiment with hyperparameter configurations, optimizing model responses for specific tasks, and includes an evaluation module to assess LLM performance against ground truths using semantic and similarity-based metrics. By offering enhanced privacy, customization, and evaluation capabilities, EvidenceBot bridges critical gaps in the LLM ecosystem, providing a versatile resource for individuals and organizations seeking to leverage LLMs effectively. 

   more » « less