An official website of the United States government
Abstract The crowdsourcing literature has shown that domain experts are not always the best solvers for complex system design problems. Under certain conditions, novices and specialists in adjacent domains can provide novel solutions at lower costs. Additionally, the best types of solvers for different problems are dependent on the architecture of complex systems. The joint consideration of solver assignment and system decomposition, referred to as solver-aware system architecting (SASA), expands traditional system architecting practices by considering solver characteristics and contractual incentive mechanisms in the design process and aims to improve complex system design and innovation by leveraging the strengths of domain experts, crowds, and specialists for different parts of the problem. The joint consideration of problem decomposition and solver assignment decisions in SASA renders the design space exponentially more complex. Therefore, new computationally efficient and mathematically rigorous methods are needed to explore this high-dimensional space and extract reliable heuristics. To address this need, this paper presents a computational approach using a Markov decision process (MDP) formulation, Q-learning, and Gaussian mixture models. Together, these techniques explore the large space of possible solver–module assignments by modeling the sequential nature of solver assignment decisions, capturing these temporal dependencies, thereby enabling optimization for long-term expected rewards, and analyzing reward distributions. The approach identifies heuristics for solver assignment based on the designer’s preference for cost-performance trade-off through the parameterized reward function. The approach is demonstrated using a simple and idealized golf problem, which has characteristics similar to design problems, including how the problem is decomposed into interdependent modules and can be solved by different solvers with different strengths that interact with the module type. The results show that the proposed approach effectively elicits a rich set of heuristics applicable in various contexts for the golf problem and can be extended to more complex systems design problems.
more »
« less
Extracting Design Process Heuristics: A Systematic Computational Approach
Abstract Engineering design relies heavily on heuristics, yet there is a lack of systematic methods for identifying and validating design heuristics. This paper introduces a computational approach to representing engineering design problems that involve decomposition and assignment decisions, facilitating systematic extraction of generalizable heuristics. We model design processes using a Markov Decision Process (MDP) framework, characterizing problems through attributes of the problem space, solver capabilities, and trade-offs embedded within preference functions. Reinforcement learning methods are employed to learn optimal policies, from which we extract inclusionary and exclusionary heuristics using Gaussian Mixture Models. The effectiveness of the approach is demonstrated through two case studies: solver-aware system architecting (SASA) for a robotic arm design and sequential information acquisition in parametric design optimization. The results highlight the context-dependent nature of learned heuristics, demonstrating how problem complexity, designer preferences, and solver characteristics influence their selection.
more »
« less
- PAR ID:
- 10667616
- Publisher / Repository:
- American Society of Mechanical Engineers
- Date Published:
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Heuristics are essential for addressing the complexities of engineering design processes. The goodness of heuristics is context-dependent. Appropriately tailored heuristics can enable designers to find good solutions efficiently, and inappropriate heuristics can result in cognitive biases and inferior design outcomes. While there have been several efforts at understanding which heuristics are used by designers, there is a lack of normative understanding about when different heuristics are suitable. Towards addressing this gap, this paper presents a reinforcement learning-based approach to evaluate the goodness of heuristics for three sub-problems commonly faced by designers: (1) learning the map between the design space and the performance space, (2) acquiring sequential information, and (3) stopping the information acquisition process. Using a multi-armed bandit formulation and simulation studies, we learn the suitable heuristics for these individual sub-problems under different resource constraints and problem complexities. Additionally, we learn the optimal heuristics for the combined problem (i.e., the one composing all three sub-problems), and we compare them to ones learned at the sub-problem level. The results of our simulation study indicate that the proposed reinforcement learning-based approach can be effective for determining the quality of heuristics for different problems, and how the effectiveness of the heuristics changes as a function of the designer’s preference (e.g., performance versus cost), the complexity of the problem, and the resources available.more » « less
-
Abstract This paper introduces an educational multiplayer web-based game to teach solver-aware systems architecting (SASA) heuristics, along with preliminary findings regarding its educational value. SASA is a framework for leveraging the relative strengths of domain experts, crowds, and specialists to design innovative complex systems by pairing technical problem decomposition with solver assignment decisions. This new line of thinking calls for a paradigm shift from the typical approach to system development, as it greatly increases the complexity of the problem space by expanding the set of actions designers could pursue by introducing solver-assignment decisions in addition to the already vast technical decomposition decisions. To that end, heuristics could provide an effective mechanism to navigate this landscape by leading towards satisficing solutions in a cost and time-efficient manner. The game was piloted on 30 participants. It first introduced the basic SASA concepts, and then asked participants increasingly more difficult system architecting problems that ranged from subproblem-level (module) heuristics to system-level problems with two counterbalancing objectives. Findings suggest that the game could be an effective tool for higher-order learning, as by the end of the game, all participants exhibited acceptable command of using SASA heuristics for making system-level tradeoffs. Future work could leverage the game’s rich data collection mechanism to investigate biases and background factors that influence designer learning outcomes.more » « less
-
Abstract Design heuristics are traditionally used as qualitative principles to guide the design process, but they have also been used to improve the efficiency of design optimization. Using design heuristics as soft constraints or search operators has been shown for some problems to reduce the number of function evaluations needed to achieve a certain level of convergence. However, in other cases, enforcing heuristics can reduce diversity and slow down convergence. This paper studies the question of when and how a given set of design heuristics represented in different forms (soft constraints, repair operators, and biased sampling) can be utilized in an automated way to improve efficiency for a given design problem. An approach is presented for identifying promising heuristics for a given problem by estimating the overall impact of a heuristic based on an exploratory screening study. Two impact indices are formulated: weighted influence index and hypervolume difference index. Using this approach, the promising heuristics for four design problems are identified and the efficacy of selectively enforcing only these promising heuristics over both enforcement of all available heuristics and not enforcing any heuristics is benchmarked. In all problems, it is found that enforcing only the promising heuristics as repair operators enables finding good designs faster than by enforcing all available heuristics or not enforcing any heuristics. Enforcing heuristics as soft constraints or biased sampling functions results in improvements in efficiency for some of the problems. Based on these results, guidelines for designers to leverage heuristics effectively in design optimization are presented.more » « less
-
Problem solvers vary their approaches to solving problems depending on the context of the problem, the requirements of the solution, and the ways in which the problems and material to solve the problem are represented, or representations. Representations take many forms (i.e. tables, graphs, figures, images, formulas, visualizations, and other similar contexts) and are used to communicate information to a problem solver. Engagement with certain representations varies between problem solvers and can influence design and solution quality. A problem solver’s evaluation of representations and the reasons for using a representation can be considered factors in problem-solving heuristics. These factors describe unique problem-solving behaviors that can help understand problem solvers. These behaviors may lead to important relationships between a problem solver’s decisions and their ability to solve a problem and overall quality of the solution. Therefore, we pose the following research question: How do factors of problem-solving heuristics describe the unique behaviors of engineering students as they solve multiple problems? To answer this question, we interviewed 16 undergraduate engineering students studying civil engineering. The interviews consisted of a problem-solving portion that was followed immediately by a semi-structured retrospective interview with probing questions created based on the real time monitoring of the problem-solving interview using eye tracking techniques. The problem-solving portion consisted of solving three problems related to the concept of headloss in fluid flow through pipes. Each of the three problems included the same four representations that were used by the students as approaches to solving the problem. The representations are common ways to present the concept of headloss in pipe flow and included two formulas, a set of tables, and a graph. This paper presents a set of common reasons for why decisions were made during the problem-solving process that help to understand more about the problem-solving behavior of engineering students.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Conference Paper:
- https://doi.org/10.1115/DETC2025-168784
