When selecting ideas or trying to find inspiration, designers often must sift through hundreds or thousands of ideas. This paper provides an algorithm to rank design ideas such that the ranked list simultaneously maximizes the quality and diversity of recommended designs. To do so, we first define and compare two diversity measures using determinantal point processes (DPP) and additive submodular functions. We show that DPPs are more suitable for items expressed as text and that a greedy algorithm diversifies rankings with both theoretical guarantees and empirical performance on what is otherwise an NP-Hard problem. To produce such rankings, this paper contributes a novel way to extend quality and diversity metrics from sets to permutations of ranked lists. These rank metrics open up the use of multi-objective optimization to describe trade-offs between diversity and quality in ranked lists. We use such trade-off fronts to help designers select rankings using indifference curves. However, we also show that rankings on trade-off front share a number of top-ranked items; this means reviewing items (for a given depth like the top ten) from across the entire diversity-to-quality front incurs only a marginal increase in the number of designs considered. While the proposed techniques are general purpose enough to be used across domains, we demonstrate concrete performance on selecting items in an online design community (OpenIDEO), where our approach reduces the time required to review diverse, high-quality ideas from around 25 h to 90 min. This makes evaluation of crowd-generated ideas tractable for a single designer. Our code is publicly accessible for further research.
Skip Nav Destination
Article navigation
January 2018
Research-Article
Ranking Ideas for Diversity and Quality
Faez Ahmed,
Faez Ahmed
Department of Mechanical Engineering,
University of Maryland,
College Park, MD 20742
e-mail: faez00@umd.edu
University of Maryland,
College Park, MD 20742
e-mail: faez00@umd.edu
Search for other works by this author on:
Mark Fuge
Mark Fuge
Department of Mechanical Engineering,
University of Maryland,
College Park, MD 20742
e-mail: fuge@umd.edu
University of Maryland,
College Park, MD 20742
e-mail: fuge@umd.edu
Search for other works by this author on:
Faez Ahmed
Department of Mechanical Engineering,
University of Maryland,
College Park, MD 20742
e-mail: faez00@umd.edu
University of Maryland,
College Park, MD 20742
e-mail: faez00@umd.edu
Mark Fuge
Department of Mechanical Engineering,
University of Maryland,
College Park, MD 20742
e-mail: fuge@umd.edu
University of Maryland,
College Park, MD 20742
e-mail: fuge@umd.edu
Contributed by the Design Theory and Methodology Committee of ASME for publication in the JOURNAL OF MECHANICAL DESIGN. Manuscript received December 15, 2016; final manuscript received September 21, 2017; published online November 9, 2017. Assoc. Editor: Katja Holtta-Otto.
J. Mech. Des. Jan 2018, 140(1): 011101 (11 pages)
Published Online: November 9, 2017
Article history
Received:
December 15, 2016
Revised:
September 21, 2017
Citation
Ahmed, F., and Fuge, M. (November 9, 2017). "Ranking Ideas for Diversity and Quality." ASME. J. Mech. Des. January 2018; 140(1): 011101. https://doi.org/10.1115/1.4038070
Download citation file:
Get Email Alerts
DeepJEB: 3D Deep Learning-Based Synthetic Jet Engine Bracket Dataset
J. Mech. Des (April 2025)
Design and Justice: A Scoping Review in Engineering Design
J. Mech. Des (May 2025)
Related Articles
Multiple Surrogate-Assisted Many-Objective Optimization for Computationally Expensive Engineering Design
J. Mech. Des (May,2018)
Sequential Multi-Objective Optimization for Lubrication System of Gasoline Engines With Bilevel Optimization Structure
J. Mech. Des (February,2017)
Large-Scale Needfinding: Methods of Increasing User-Generated Needs From Large Populations
J. Mech. Des (July,2015)
A Novel Sequential Multi-Objective Optimization Using Anchor Points in the Design Space of Global Variables
J. Mech. Des (December,2016)
Related Proceedings Papers
Related Chapters
A Learning-Based Adaptive Routing for QoS-Aware Data Collection in Fixed Sensor Networks with Mobile Sinks
Intelligent Engineering Systems through Artificial Neural Networks, Volume 20
A Collaborative Framework for Distributed Multiobjective Combinatorial Optimization
International Conference on Computer and Computer Intelligence (ICCCI 2011)
A Multi-Objective Optimization Model for Developing Emergency Evacuation Plans (PSAM-0466)
Proceedings of the Eighth International Conference on Probabilistic Safety Assessment & Management (PSAM)