Abstract Inspirational stimuli are known to be effective in supporting ideation during early-stage design. However, prior work has predominantly constrained designers to using text-only queries when searching for stimuli, which is not consistent with real-world design behavior where fluidity across modalities (e.g., visual, semantic, etc.) is standard practice. In the current work, we introduce a multi-modal search platform that retrieves inspirational stimuli in the form of 3D-model parts using text, appearance, and function-based search inputs. Computational methods leveraging a deep-learning approach are presented for designing and supporting this platform, which relies on deep-neural networks trained on a large dataset of 3D-model parts. This work further presents the results of a cognitive study ( n = 21) where the aforementioned search platform was used to find parts to inspire solutions to a design challenge. Participants engaged with three different search modalities: by keywords, 3D parts, and user-assembled 3D parts in their workspace. When searching by parts that are selected or in their workspace, participants had additional control over the similarity of appearance and function of results relative to the input. The results of this study demonstrate that the modality used impacts search behavior, such as in search frequency, how retrieved search results are engaged with, and how broadly the search space is covered. Specific results link interactions with the interface to search strategies participants may have used during the task. Findings suggest that when searching for inspirational stimuli, desired results can be achieved both by direct search inputs (e.g., by keyword) as well as by more randomly discovered examples, where a specific goal was not defined. Both search processes are found to be important to enable when designing search platforms for inspirational stimuli retrieval.
more »
« less
Riker: Mining Rich Keyword Representations for Interpretable Product Question Answering
This work studies product question answering (PQA) which aims to answer product-related questions based on customer reviews. Most recent PQA approaches adopt end2end semantic matching methodologies, which map questions and answers to a latent vector space to measure their relevance. Such methods often achieve superior performance but it tends to be difficult to interpret why. On the other hand, simple keyword-based search methods exhibit natural interpretability through matched keywords, but often suffer from the lexical gap problem. In this work, we develop a new PQA framework (named Riker) that enjoys the benefits of both interpretability and effectiveness. Riker mines rich keyword representations of a question with two major components, internal word re-weighting and external word association, which predict the importance of each question word and associate the question with outside relevant keywords respectively, and can be jointly trained under weak supervision with large-scale QA pairs. The keyword representations from Riker can be directly used as input to a keyword-based search module, enabling the whole process to be effective while preserving good interpretability. We conduct extensive experiments using Amazon QA and review datasets from 5 different departments, and our results show that Riker substantially outperforms previous state-of-the-art methods in both synthetic settings and real user evaluations. In addition, we compare keyword representations from Riker and those from attention mechanisms popularly used for deep neural networks through case studies, showing that the former are more effective and interpretable.
more »
« less
- Award ID(s):
- 1815674
- NSF-PAR ID:
- 10106775
- Date Published:
- Journal Name:
- Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
- Page Range / eLocation ID:
- 1389 to 1398
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Answering complex questions about textual narratives requires reasoning over both stated context and the world knowledge that underlies it. However, pretrained language models (LM), the foundation of most modern QA systems, do not robustly represent latent relationships between concepts, which is necessary for reasoning. While knowledge graphs (KG) are often used to augment LMs with structured representations of world knowledge, it remains an open question how to effectively fuse and reason over the KG representations and the language context, which provides situational constraints and nuances. In this work, we propose GreaseLM, a new model that fuses encoded representations from pretrained LMs and graph neural networks over multiple layers of modality interaction operations. Information from both modalities propagates to the other, allowing language context representations to be grounded by structured world knowledge, and allowing linguistic nuances (e.g., negation, hedging) in the context to inform the graph representations of knowledge. Our results on three benchmarks in the commonsense reasoning (i.e., CommonsenseQA, OpenbookQA) and medical question answering (i.e., MedQA-USMLE) domains demonstrate that GreaseLM can more reliably answer questions that require reasoning over both situational constraints and structured knowledge, even outperforming models 8x larger.more » « less
-
more » « less
Abstract Political and social scientists have been relying extensively on keywords such as hashtags to mine social movement data from social media sites, particularly Twitter. Yet, prior work demonstrates that unrepresentative keyword sets can lead to flawed research conclusions. Numerous keyword expansion methods have been proposed to increase the comprehensiveness of keywords, but systematic evaluations of these methods have been lacking. Our paper fills this gap. We evaluate five diverse keyword expansion techniques (or pipelines) on five representative social movements across two distinct activity levels. Our results guide researchers who aim to use social media keyword searches to mine data. For instance, we show that word embedding-based methods significantly outperform other even more complex and newer approaches when movements are in normal activity periods. These methods are also less computationally intensive. More importantly, we also observe that no single pipeline can identify little more than half of all movement-related tweets when these movements are at their peak mobilization period offline. However, coverage can increase significantly when more than one pipeline is used. This is true even when the pipelines are selected at random.
-
There are a variety of urgent calls for institutional initiatives and actions to transform engineering education. For a transformational change to occur, the initiatives must alter the culture of the institutions (Eckel, Hill, and Green, 1998). In this work in progress, we detail the methods used to conduct a scoping literature review (ScR) concerning the current state of the literature surrounding institutional culture and transformational change in engineering education at institutions of higher learning in the United States. As institutional culture and transformational change are currently underexplored topics in the engineering education literature, we investigated the larger body of computer science and engineering literature in the United States. Once completed, this study aims to reveal the current trends, theories, and potential gaps in the literature regarding these topics. Arksey and O’Malley’s methodology for conducting scoping reviews informed the development of our scoping review protocol, which similarly includes five stages: (1) identify the research questions, (2) identify relevant studies, (3) select relevant studies, (4) chart the data, and (5) collate, summarize, and report results (Arksey and O’Malley, 2005). University librarians who specialize in conducting systematic reviews aided in the refinement of this protocol. From the research question and aim of the study, three main inclusion criteria were created: (1) the literature must discuss both organizational culture and transformational change, (2) discussion of transformational change must describe the institution where the change happened, and (3) the literature must emphasize the agents of transformational change. Additional inclusion and exclusion criteria were created in collaboration with both the librarians and reviewers. These criteria guided the search for existing literature in the following online databases: Elsevier (Engineering Village – Compendex and Engineering Village – INSPEC), ProQuest (ERIC and Education Database), Scopus, and Web of Science. These six databases were selected as they often include publications relevant to the field of engineering education. After the search was conducted, the inclusion and exclusion criteria were turned into questions to inform a three-step screening process (title, abstract, and full text) used by reviewers to determine whether a publication was eligible for the study. Reviewers were assigned to review papers through Covidence, a cloud-based systematic literature review management platform. There are currently two primary reviewers and a third additional reviewer to resolve any conflicts or disagreements if they should arise. Before each review cycle, the inclusion and exclusion criteria are revisited, revised, and agreed upon by the three reviewers. This screening process is performed iteratively, allowing for critical reflection at each stage to drive the resulting findings by the reviewers in consultation with content matter experts. We are currently conducting our first round of screening in the study selection (third stage) of the scoping review protocol. After the removal of duplicates, 999 publications were found by searching in the six selected databases. This number is expected to be further reduced with each step of the screening process. When this scoping review is complete, the resulting publication will contain an analysis of the literature and synthesis of our findings, and present the prominent themes, theories, and potential gaps in the literature. This publication is expected to unite disparate lines of research on institutional culture and transformational change, challenge the assumptions in the field, and change the way engineering education views transformational change.more » « less
-
Current textual question answering (QA) models achieve strong performance on in-domain test sets, but often do so by fitting surface-level patterns, so they fail to generalize to out-of-distribution settings. To make a more robust and understandable QA system, we model question answering as an alignment problem. We decompose both the question and context into smaller units based on off-the-shelf semantic representations (here, semantic roles), and align the question to a subgraph of the context in order to find the answer. We formulate our model as a structured SVM, with alignment scores computed via BERT, and we can train end-to-end despite using beam search for approximate inference. Our use of explicit alignments allows us to explore a set of constraints with which we can prohibit certain types of bad model behavior arising in cross-domain settings. Furthermore, by investigating differences in scores across different potential answers, we can seek to understand what particular aspects of the input lead the model to choose the answer without relying on post-hoc explanation techniques. We train our model on SQuAD v1.1 and test it on several adversarial and out-of-domain datasets. The results show that our model is more robust than the standard BERT QA model, and constraints derived from alignment scores allow us to effectively trade off coverage and accuracy.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3292500.3330985
