Abstract
Human subjects are often used in design theory and methods studies to understand how designers behave and link behaviors with design outcomes. We highlight that much of our past work and resultant knowledge in design theory and methodology (DTM) has used data from samples composed mainly of White men. We argue, therefore, that the tools, methods, and practices conceived through research intended for universal use may not be as generalizable as intended. We support this assertion through a quantitative assessment of past DTM literature. We definitively demonstrate a historic lack of diversity in human-subjects research in DTM through a critical review of 109 published works. Our findings suggest that age, gender, and race/ethnic background are underreported. This work is a call to action for researchers to reflect on who are the beneficiaries of design theory and methods research, how participant background might impact the findings, and what strategies can be employed to fulfill our promise of a more equitable design community and impact.
Issue Section:
Design Theory and Methodology
References
1.
Benitez
, M.
, 2011
, “Resituating Culture Centers within a Social Justice Framework: Is There Room for Examining Whiteness?” Culture Centers in Higher Education: Perspectives on Identity, Theory, and Practice
, 1st ed., L.D.
Patton
, ed., Routledge
, New York
, pp. 119
–134
.2.
Mejia
, J. A.
, Revelo
, R. A.
, Villanueva
, I.
, and Mejia
, J.
, 2018
, “Critical Theoretical Frameworks in Engineering Education: An Anti-Deficit and Liberative Approach
,” Educ. Sci.
, 8
(4
), p. 158
. 3.
Blosser
, E.
, 2020
, “An Examination of Black Women’s Experiences in Undergraduate Engineering on a Primarily White Campus: Considering Institutional Strategies for Change
,” J. Eng. Educ.
, 109
(1
), pp. 52
–71
. 4.
Intemann
, K.
, 2009
, “Why Diversity Matters: Understanding and Applying the Diversity Component of the National Science Foundation’s Broader Impacts Criterion
,” Soc. Epistemol.
, 23
(3–4
), pp. 249
–266
. 5.
Mays
, D. C.
, 2022
, “How Diversity Makes Better Engineering Teams How Diversity Makes Better Engineering Teams How Diversity Makes Better Engineering Teams
,” J. AWWA
, 114
(7
), pp. 62
–67
. 6.
Woolley
, A. W.
, Chabris
, C. F.
, Pentland
, A.
, Hashmi
, N.
, and Malone
, T. W.
, 2010
, “Evidence for a Collective Intelligence Factor in the Performance of Human Groups
,” Science (1979)
, 330
(6004
), pp. 686
–688
. 7.
Page
, S.
, 2008
, The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies-New Edition
, Princeton University Press
, Princeton, NJ
.8.
Anderson
, E.
, 2015
, “The White Space
,” Sociol. Race Ethnicity
, 1
(1
), pp. 10
–21
. 9.
Tsui
, A. S.
, Egan
, T. D.
, and O'Reilly
, C. A.
III, 1992
, “Being Different: Relational Demography and Organizational Attachment
,” Adm. Sci. Q.
, 37
(4
), p. 549
. 10.
Eastman
, M. G.
, Miles
, M. L.
, and Yerrick
, R.
, 2019
, “Exploring the White and Male Culture: Investigating Individual Perspectives of Equity and Privilege in Engineering Education
,” J. Eng. Educ.
, 108
(4
), pp. 459
–480
. 11.
Cabay
, M.
, Bernstein
, B. L.
, Rivers
, M.
, and Fabert
, N.
, 2018
, “Chilly Climates, Balancing Acts, and Shifting Pathways: What Happens to Women in STEM Doctoral Programs
,” Soc. Sci.
, 7
(2
), p. 23
. 12.
Cross
, K. J.
, Mendenhall
, R.
, Clancy
, K. B. H.
, Imoukhuede
, P.
, and Amos
, J.
, 2021
, “The Pieces of Me: The Double Bind of Race and Gender In Engineering
,” J. Women Minor Sci. Eng.
, 27
(3
), pp. 79
–105
. 13.
Chesler
, N. C.
, Barabino
, G.
, Bhatia
, S. N.
, and Richards-Kortum
, R.
, 2010
, “The Pipeline Still Leaks and More Than You Think: A Status Report on Gender Diversity in Biomedical Engineering
,” Ann. Biomed. Eng.
, 38
(5
), pp. 1928
–1935
. 14.
Sharpe
, D.
, and Poets
, S.
, 2017
, “Canadian Psychology Department Participant Pools: Closing for the Season?
,” Can. Psychol.
, 58
(2
), pp. 168
–177
. 15.
Lupton
, D. L.
, 2019
, “The External Validity of College Student Subject Pools in Experimental Research: A Cross-Sample Comparison of Treatment Effect Heterogeneity
,” Pol. Anal.
, 27
(1
), pp. 90
–97
. 16.
Peng
, A.
, Menold
, J.
, and Miller
, S. R.
, 2022
, “Crossing Cultural Borders: A Case Study of Conceptual Design Outcomes of U.S. and Moroccan Student Samples
,” ASME J. Mech. Des.
, 144
(3
), p. 032301
. 17.
Carberry
, A. R.
, Baker
, D.R.
, Dori
, Y.J.
, Mevarech
, Z.R.
, and Baker
, D.R.
, 2018
, Cognition, Metacognition, and Culture in STEM Education. Innovations in Science Education and Technology
, Vol. 24
, Springer, Cham
, Cham, Switzerland
, pp. 217
–239
. 18.
Bennett
, M. J.
, and Hammer
, M.
, 2017
, “A Developmental Model of Intercultural Sensitivity,” The International Encyclopedia of Intercultural Communication
, Vol. 1
, Y.Y.
Kim
, and K.L.
McKay-Semmler
, eds., John Wiley & Sons
, Hoboken, NJ
.19.
Jansson
, D. G.
, and Smith
, S. M.
, 1991
, “Design Fixation
,” Des. Stud.
, 12
(1
), pp. 3
–11
. 20.
Viswanathan
, V.
, and Linsey
, J.
, 2011
, “Design Fixation in Physical Modeling: An Investigation on the Role of Sunk Cost
,” Proceedings of the ASME Design Engineering Technical Conference
, Washington, DC
, Aug. 28–31
, Vol. 9, pp. 119
–130
.21.
Neroni
, M. A.
, Vasconcelos
, L. A.
, and Crilly
, N.
, 2017
, “Computer-Based ‘Mental Set’ Tasks: An Alternative Approach to Studying Design Fixation
,” ASME J. Mech. Des.
, 139
(7
), p. 071102
. 22.
Leahy
, K.
, Daly
, S. R.
, McKilligan
, S.
, and Seifert
, C. M.
, 2020
, “Design Fixation From Initial Examples: Provided Versus Self-Generated Ideas
,” ASME J. Mech. Des.
, 142
(10
), p. 101402
. 23.
Jia
, M.
, Jiang
, S.
, Hu
, J.
, and Qi
, J.
, 2023
, “Toward Understanding Sources and Influences of Design Fixation: A Focus on Example Stimuli and Background of Novice Designers
,” ASME J. Mech. Des.
, 145
(5
), p. 051402
. 24.
Koronis
, G.
, Chia
, P. Z.
, Siang
, J. K. K.
, Silva
, A.
, Yogiaman
, C.
, and Raghunath
, N.
, 2019
, “An Empirical Study on the Impact of Design Brief Information on the Creativity of Design Outcomes With Consideration of Gender and Gender Diversity
,” ASME J. Mech. Des.
, 141
(7
), p. 071102
. 25.
Mirabito
, Y.
, and Goucher-Lambert
, K.
, 2022
, “Factors Impacting Highly Innovative Designs: Idea Fluency, Timing, and Order
,” ASME J. Mech. Des.
, 144
(1
), p. 011401
. 26.
Zheng
, X.
, Ritter
, S. C.
, and Miller
, S. R.
, 2018
, “How Concept Selection Tools Impact the Development of Creative Ideas in Engineering Design Education
,” ASME J. Mech. Des.
, 140
(5
), p. 052002
. 27.
Zheng
, X.
, and Miller
, S. R.
, 2021
, “Out in the Field Versus Inside in the Lab: A Comparison of Design Professionals’ Concept Screening Practices (DETC2019-97564)
,” ASME J. Mech. Des.
, 143
(4
), p. 041401
. 28.
Deo
, S.
, Blej
, A.
, Kirjavainen
, S.
, and Holtta-Otto
, K.
, 2021
, “Idea Generation Mechanisms: Comparing the Influence of Classification, Combination, Building on Others, and Stimulation Mechanisms on Ideation Effectiveness
,” ASME J. Mech. Des.
, 143
(12), p. 121403
. 29.
Olteteanu
, A. M.
, and Shu
, L. H.
, 2018
, “Object Reorientation and Creative Performance
,” ASME J. Mech. Des.
, 140
(3
), p. 031102
. 30.
Daly
, S. R.
, Seifert
, C. M.
, Yilmaz
, S.
, and Gonzalez
, R.
, 2016
, “Comparing Ideation Techniques for Beginning Designers
,” ASME J. Mech. Des.
, 138
(10
), p. 101108
. 31.
Goucher-Lambert
, K.
, Gyory
, J. T.
, Kotovsky
, K.
, and Cagan
, J.
, 2020
, “Adaptive Inspirational Design Stimuli: Using Design Output to Computationally Search for Stimuli That Impact Concept Generation
,” ASME J. Mech. Des.
, 142
(9
), p. 091401
. 32.
Mccomb
, C.
, Lapp
, S.
, Jablokow
, K.
, and Mccomb
, C.
, 2019
, “KABOOM: An Agent-Based Model for Simulating Cognitive Style in Team Problem Solving
,” Des. Sci.
, 5
(13
), pp. 1
–32
. 33.
McComb
, C.
, Cagan
, J.
, and Kotovsky
, K.
, 2015
, “Lifting the Veil: Drawing Insights About Design Teams From a Cognitively-Inspired Computational Model
,” Des. Stud.
, 40
(1
), pp. 119
–142
. 34.
Lauff
, C. A.
, Kotys-Schwartz
, D.
, and Rentschler
, M. E.
, 2018
, “What Is a Prototype? What Are the Roles of Prototypes in Companies?
,” ASME J. Mech. Des.
, 140
(6
), p. 061102
. 35.
Camburn
, B.
, Dunlap
, B.
, Gurjar
, T.
, Hamon
, C.
, Green
, M.
, Jensen
, D.
, Crawford
, R.
, Otto
, K.
, and Wood
, K.
, 2015
, “A Systematic Method for Design Prototyping
,” ASME J. Mech. Des.
, 137
(8
), pp. 81102
–81103
. 36.
Starkey
, E. M.
, Menold
, J.
, and Miller
, S. R.
, 2019
, “When Are Designers Willing to Take Risks? How Concept Creativity and Prototype Fidelity Influence Perceived Risk
,” ASME J. Mech. Des.
, 141
(3
), p. 031104
. 37.
Rodríguez-Calero
, I.
, Daly
, S. R.
, Burleson
, G.
, and Sienko
, K. H.
, 2023
, “Prototyping Strategies to Engage Stakeholders During Early Stages of Design: A Study Across Three Design Domains
,” ASME J. Mech. Des.
, 145
(4
), p. 041413
. 38.
Nelson
, J.
, and Menold
, J.
, 2021
, “Between Worlds: Exploring the Links Between the Tangible and Intangible Costs of Prototypes
,” ASME J. Mech. Des.
, 143
(3
), p. 031402
. 39.
Menold
, J.
, Jablokow
, K.
, and Simpson
, T.
, 2019
, “The Prototype for X Framework: Assessing Impact on Self-Reported Prototyping Behavior of Student Designers
,” ASME J. Mech. Des.
, 141
(4
), p. 042001
. 40.
Liedtka
, J.
, 2011
, “Learning to Use Design Thinking Tools for Successful Innovation
,” Strateg. Leadersh.
, 39
(5
), pp. 13
–19
. 41.
Yilmaz
, S.
, Daly
, S. R.
, Seifert
, C. M.
, and Gonzalez
, R.
, 2016
, “Evidence-Based Design Heuristics for Idea Generation
,” Des. Stud.
, 46
(1
), pp. 95
–124
. 42.
National Academy of Sciences, National Academy of Engineering, and Institute of Medicine
, 2010
, Expanding Underrepresented Minority Participation: America's Science and Technology Talent at the Crossroads
, National Academies Press
, Washington, DC
, pp. 1
–269
.43.
National Center for Science and Engineering Statistics
, 2001
, Science and Engineering Degrees, by Race/Ethnicity of Recipients: 1990–98
.44.
Cox
, T. H.
, and Blake
, S.
, 1991
, “Managing Cultural Diversity: Implications for Organizational Competitiveness
,” Acad. Manage. Perspect.
, 5
(3
), pp. 45
–56
. 45.
Berdahl
, J. L.
, 1996
, “Gender and Leadership in Work Groups: Six Alternative Models
,” Leadersh. Q.
, 7
(1
), pp. 21
–40
. 46.
Babaria
, P.
, Abedin
, S.
, Berg
, D.
, and Nunez-Smith
, M.
, 2012
, “‘I’m Too Used to It’: A Longitudinal Qualitative Study of Third Year Female Medical Students’ Experiences of Gendered Encounters in Medical Education
,” Soc. Sci. Med.
, 74
(7
), pp. 1013
–1020
. 47.
Toh
, C. A.
, Strohmetz
, A. A.
, and Miller
, S. R.
, 2016
, “The Effects of Gender and Idea Goodness on Ownership Bias in Engineering Design Education
,” ASME J. Mech. Des.
, 138
(10
), pp. 101105
. 48.
Concannon
, J. P.
, and Barrow
, L. H.
, 2009
, “A Cross-Sectional Study of Engineering Students’ Self-Efficacy by Gender, Ethnicity, Year, and Transfer Status
,” J. Sci. Educ. Technol.
, 18
(2
), pp. 163
–172
. 49.
Schmader
, T.
, Johns
, M.
, and Forbes
, C.
, 2008
, “An Integrated Process Model of Stereotype Threat Effects on Performance
,” Psychol. Rev.
, 115
(2
), pp. 336
–356
. 50.
Logel
, C. R.
, Walton
, G. M.
, Spencer
, S. J.
, Peach
, J.
, and Mark
, Z. P.
, 2012
, “Unleashing Latent Ability: Implications of Stereotype Threat for College Admissions
,” Educ. Psychol.
, 47
(1
), pp. 42
–50
. 51.
Bassett-Jones
, N.
, 2005
, “The Paradox of Diversity Management, Creativity and Innovation
,” Creat. Innov. Manag.
, 14
(2
), pp. 169
–175
. 52.
Pawley
, A. L.
, 2017
, “Shifting the ‘Default’: The Case for Making Diversity the Expected Condition for Engineering Education and Making Whiteness and Maleness Visible
,” J. Eng. Educ.
, 106
(4
), pp. 531
–533
. 53.
Hankerson
, D.
, Marshall
, A. R.
, Booker
, J.
, El Mimouni
, H.
, Walker
, I.
, and Rode
, J. A.
, 2016
, “Does Technology Have Race?
” Conference on Human Factors in Computing Systems – Proceedings
, San Jose, CA
, May 7–12
, pp. 473
–485
.54.
Rode
, J. A.
, 2011
, “Reflexivity in Digital Anthropology
,” Conference on Human Factors in Computing Systems – Proceedings
, Vancouver, BC
, May 7–12
, pp. 123
–132
.55.
Lee
, W.
, 2015
, “How Tech's Lack of Diversity Leads to Racist Software,” SFGate. https://www.sfgate.com/business/article/How-tech-s-lack-of-diversity-leads-to-racist-6398224.php, Accessed December 14, 2023.56.
Holmes
, A. G. D.
, 2020
, “Researcher Positionality – A Consideration of Its Influence and Place in Qualitative Research – A New Researcher Guide.
,” Shanlax Int. J. Educ.
, 8
(4
), pp. 1
–10
. 57.
NSF – National Science Foundation
. “Diversity and STEM: Women, Minorities, and Persons With Disabilities 2023 | NSF – National Science Foundation.” https://ncses.nsf.gov/pubs/nsf23315/report/science-and-engineering-degrees-earned#degrees-earned-by-women-in-broad-s-e-fields, Accessed December 11, 2023.58.
Zelenski
, J. M.
, Rusting
, C. L.
, and Larsen
, R. J.
, 2000
, “Consistency in the Time of Experiment Participation and Personality Correlates: A Methodological Note
,” Pers. Individ. Diff.
, 34
(4
), pp. 547
–558
. 59.
Bourke
, B.
, 2016
, “Meaning and Implications of Being Labelled a Predominantly White Institution
,” College & University
, 91
(3
), p. 12
.60.
Rutgers Graduate School of Education Center for MSIs
, 2023
, List of Minority Serving Institutions
.61.
MSI Data Project
, 2023
, DATA | MSI Data Project
.62.
Simpson
, J. C.
, 2001
, “Segregated by Subject: Racial Differences in the Factors Influencing Academic Major Between European Americans, Asian Americans, and African, Hispanic, and Native Americans
,” J. Higher Educ.
, 72
(1
), pp. 63
–100
. 63.
Humphreys
, S.
, and Freeland
, R.
, 1992
, Retention in Engineering: A Study of Freshman Cohorts
, Regents of the University of California
, Berkeley, CA
.64.
Crenshaw
, K.
, 2013
, “Demarginalizing the Intersection of Race and Sex: A Black Feminist Critique of Antidiscrimination Doctrine, Feminist Theory and Antiracist Politics,” Feminist Legal Theories
, K.
Maschke
, ed., Routledge
, New York
, pp. 23
–51
.Copyright © 2024 by ASME
You do not currently have access to this content.
