An official website of the United States government
A practical and well-studied method for computing the novelty of a design is to construct an ordinal embedding via a collection of pairwise comparisons between items (called triplets), and use distances within that embedding to compute which designs are farthest from the center. Unfortunately, ordinal embedding methods can require a large number of triplets before their primary error measure — the triplet violation error — converges. But if our goal is accurate novelty estimation, is it really necessary to fully minimize all triplet violations? Can we extract useful information regarding the novelty of all or some items using fewer triplets than classical convergence rates might imply? This paper addresses this question by studying the relationship between triplet violation error and novelty score error when using ordinal embeddings. Specifically, we compare how errors in embeddings produced by Generalized Non-Metric Dimensional Scaling (GNMDS) converge under different sampling methods, for different numbers of embedded items, sizes of latent spaces, and for the top K most novel designs. We find that estimating the novelty of a set of items via ordinal embedding can require significantly fewer human-provided triplets than is needed to converge the triplet error, and that this effect is modulated by the type of triplet sampling method (random versus uncertainty sampling). We also find that uncertainty sampling causes unique converge behavior in estimating most novel items compared to non-novel items. Our results imply that in certain situations one can use ordinal embedding techniques to estimate novelty error in fewer samples than is typically expected. Moreover, the convergence behavior of top K novel items motivates new potential triplet sampling methods that go beyond typical triplet reduction measures.
more »
« less
Novelty and the Structure of Design Landscapes: A Relational View of Online Innovation Communities
Design artifacts in online innovation communities are increasingly becoming a primary source of innovation for organizations. A distinguishing feature of such communities is that they are organized around design artifacts, not around people. The search for novel innovations thus equates to a search for novel designs. This is not a trivial problem since the novelty of a design is a function of its relationship to other designs, and this relationship changes as each design is added. These relations between artifacts affect both consumption and production. Moreover, these relations form a landscape whose structure affects the emergence of novelty. We find evidence for our theorizing using an analysis of over 35,000 Thingiverse design artifacts. This work identifies the differential effects of different forms of novelty, visual and verbal, on subsequent innovation, and identifies the differential effects of different degrees of structure in the landscape on novelty.
more »
« less
- PAR ID:
- 10302566
- Date Published:
- Journal Name:
- MIS quarterly
- Volume:
- 46
- Issue:
- 3
- ISSN:
- 0276-7783
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)With increasing challenges to health care in the foreseeable future, novel technology solutions are increasingly needed. Meanwhile, biomedical engineers are increasingly asked to develop user-centered solutions (i.e., desired by the end users). Nevertheless, the importance of user-centeredness is often neglected in the innovation process. It remains unclear about the interplay between thinking of solution novelty and desirability in addition to feasibility, and thus it is challenging for biomedical engineering educators to balance the teaching of the above two aspects in a BME design curriculum. This study aims to develop a preliminary version of a user-centered innovation potential assessment instrument applicable to diverse biomedical engineering design projects. The assessment instrument was adapted from File and Purzer (2014)’s definition of innovation potential (1) feasibility (2) viability (3) desirability and (4) novelty. Among these aspects, we focused on assessing feasibility, desirability and novelty, which can be quantified and assigned to each design idea proposed by the students. As the first attempt, we targeted students’ innovation potential in the design prototyping phase. To validate our preliminary development, we gave an in-class design task for smart pill dispenser to 30+ pairs of senior students enrolled in the BME capstone design course. To assess the design ideas, the instructor and his teaching assistant (two of the authors on the paper) applied a thematic analysis. We first identified patterns from the submitted design ideas by extracting key attributes including dispenser’s portability, tracking/reminding capability, safety, and easy to use. We then estimated the frequency and novelty of these key attributes appearing in each design idea and converted each of them to a 5-point scale. Finally, we calculated a composite score for user-centered innovation potential by multiplying the scales on feasibility, desirability and novelty. We believe this study has added value to improving our understanding of user-centered innovation potential in an undergraduate biomedical engineering curriculum. With further development and scaled-up validation, we may be able to use the instrument to provide insights into developing teaching interventions for stimulating user-centered innovative potentials among biomedical engineers.more » « less
-
Molecular search is important in chemistry, biology, and informatics for identifying molecular structures within large data sets, improving knowledge discovery and innovation, and making chemical data FAIR (findable, accessible, interoperable, reusable). Search algorithms for polymers are significantly less developed than those for small molecules because polymer search relies on searching by polymer name, which can be challenging because polymer naming is overly broad (i.e., polyethylene), complicated for complex chemical structures, and often does not correspond to official IUPAC conventions. Chemical structure search in polymers is limited to substructures, such as monomers, without awareness of connectivity or topology. This work introduces a novel query language and graph traversal search algorithm for polymers that provides the first search method able to fully capture all of the chemical structures present in polymers. The BigSMARTS query language, an extension of the small-molecule SMARTS language, allows users to write queries that localize monomer and functional group searches to different parts of the polymer, like the middle block of a triblock, the side chain of a graft, and the backbone of a repeat unit. The substructure search algorithm is based on the traversal of graph representations of the generating functions for the stochastic graphs of polymers. Operationally, the algorithm first identifies cycles representing the monomers and then the end groups and finally performs a depth-first search to match entire subgraphs. To validate the algorithm, hundreds of queries were searched against hundreds of target chemistries and topologies from the literature, with approximately 440,000 query–target pairs. This tool provides a detailed algorithm that can be implemented in search engines to provide search results with full matching of the monomer connectivity and polymer topology.more » « less
-
Recent breakthroughs in deep-learning (DL) approaches have resulted in the dynamic generation of trace links that are far more accurate than was previously possible. However, DL-generated links lack clear explanations, and therefore non-experts in the domain can find it difficult to understand the underlying semantics of the link, making it hard for them to evaluate the link's correctness or suitability for a specific software engineering task. In this paper we present a novel NLP pipeline for generating and visualizing trace link explanations. Our approach identifies domain-specific concepts, retrieves a corpus of concept-related sentences, mines concept definitions and usage examples, and identifies relations between cross-artifact concepts in order to explain the links. It applies a post-processing step to prioritize the most likely acronyms and definitions and to eliminate non-relevant ones. We evaluate our approach using project artifacts from three different domains of interstellar telescopes, positive train control, and electronic healthcare systems, and then report coverage, correctness, and potential utility of the generated definitions. We design and utilize an explanation interface which leverages concept definitions and relations to visualize and explain trace link rationales, and we report results from a user study that was conducted to evaluate the effectiveness of the explanation interface. Results show that the explanations presented in the interface helped non-experts to understand the underlying semantics of a trace link and improved their ability to vet the correctness of the link.more » « less
-
null (Ed.)Design researchers have long sought to understand the mechanisms that support creative idea development. However, one of the key challenges faced by the design community is how to effectively measure the nebulous construct of creativity. The social science and engineering communities have adopted two vastly different approaches to solving this problem, both of which have been deployed throughout engineering design research. The goal of this paper was to compare and contrast these two approaches using design ratings of nearly 1000 engineering design ideas paired with a qualitative study with expert raters. The results of this study identify that while these two methods provide similar ratings of idea quality, there was a statistically significant negative relationship between these methods for ratings of idea novelty. Qualitative analysis of recordings from expert raters’ think aloud concept mapping points to potential sources of disagreement. In addition, the results show that while quasi-expert and expert raters provided similar ratings of design novelty, there was not significant agreement between these groups for ratings of design quality. The results of this study provide guidance for the deployment of idea ratings in engineering design research and evidence for the development and potential modification of engineering design creativity metrics.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.25300/MISQ/2022/15059
