Transforming graduate education in physics to be more equitable and diverse

2023

1. 

   Making graduate admissions in physics more equitable

   Nicholas T. Young, Kirsten Tollefson, and Marcos D. Caballero

   Physics Today, Jul 2023

   Abs HTML PDF

   The admission of graduate students to post-bachelor’s physics programs is a complex and challenging system. Any graduate director, faculty member, or graduate student can recount their own vivid experience with that complicated and, quite often, opaque process.3 At Michigan State University, we set out to understand the admissions process, determine how it was functioning, and make changes to it. We hoped to admit more diverse candidates to our program, evaluate them more equitably and holistically, and, ultimately, create a more inclusive program where each student is valued and supported in their studies. We are far from our ideal collective vision but feel that our progress toward that vision is nonetheless important to share with our colleagues.

2022

1. 

   Rubric-based holistic review: A promising route to equitable graduate admissions in physics

   Nicholas T. Young, K. Tollefson, Remco G. T. Zegers, and Marcos D. Caballero

   Phys. Rev. Phys. Educ. Res., Nov 2022

   Abs HTML PDF

   As systematic inequities in higher education and society have been brought to the forefront, graduate programs are interested in increasing the diversity of their applicants and enrollees. Yet, structures in place to evaluate applicants may not support such aims. One potential solution to support those aims is rubric-based holistic review. Starting in 2018, our physics department implemented a rubric-based holistic review process for all applicants to our graduate program. The rubric assessed applicants on 18 metrics covering their grades, test scores, research experiences, noncognitive competencies, and fit with the program. We then compared faculty’s ratings of applicants by admission status, sex, and undergraduate program over a three-year period. We find that the rubric scores show statistically significant differences between admitted and nonadmitted students as hoped. We also find that differences in rubric scores based on sex or undergraduate program reflected known systematic inequities such as applicants from smaller and less prestigious undergraduate universities scoring lower on the physics GRE and women performing more volunteer work in academia. Our results then suggest rubric-based holistic review as a possible route to making graduate admissions in physics more equitable.

2021

1. PRPER

   Physics Graduate Record Exam does not help applicants “stand out”

   Nicholas T. Young, and Marcos D. Caballero

   Physical Review Physics Education Research, Nov 2021

   Abs HTML

   One argument for keeping the physics Graduate Record Exam (GRE) is that it can help applicants who might otherwise be missed in the admissions process stand out. In this work, we evaluate whether this claim is supported by physics graduate school admissions decisions. We used admissions data from five Ph.D.-granting physics departments over a 2-year period (N=2537) to see how the fraction of applicants admitted varied based on their physics GRE scores. We compared applicants with low GPAs to applicants with higher GPAs, applicants from large undergraduate universities to applicants from smaller undergraduate universities, and applicants from selective undergraduate institutions to applicants from less selective undergraduate institutions. We also performed a mediation and moderation analysis to provide statistical rigor and to better understand the previous relationships. We find that for applicants who might otherwise have been missed (e.g., have a low GPA or attended a small or less selective school), having a high physics GRE score did not seem to increase the applicant’s chances of being admitted to the schools. However, having a low physics GRE score seemed to penalize otherwise competitive applicants (i.e., applicants with mid to high GPAs). Thus, our work suggests that the physics GRE does not, in fact, help applicants who might otherwise be missed stand out.

2. JEDM

   Predictive and Explanatory Models Might Miss Informative Features in Educational Data

   Nicholas T. Young, and Marcos D. Caballero

   Journal of Educational Data Mining, Nov 2021

   Abs HTML

   We encounter variables with little variation often in educational data mining (EDM) due to the demographics of higher education and the questions we ask. Yet, little work has examined how to analyze such data. Therefore, we conducted a simulation study using logistic regression, penalized regression, and random forest. We systematically varied the fraction of positive outcomes, feature imbalances, and odds ratios. We find the algorithms treat features with the same odds ratios differently based on the features’ imbalance and the outcome imbalance. While none of the algorithms fully solved how to handle imbalanced data, penalized approaches such as Firth and Log-F reduced the difference between the built-in odds ratio and value determined by the algorithm. Our results suggest that EDM studies might contain false negatives when determining which variables are related to an outcome. We then apply our findings to a graduate admissions dataset. We end by proposing recommendations that researchers should consider penalized regression for datasets on the order of hundreds of cases and should include more context about their data in publications such as the outcome and feature imbalances.

2019

1. PERC

   Using Machine Learning to Understand Physics Graduate School Admissions

   Nicholas T. Young, and Marcos D. Caballero

   In 2019 Physics Education Research Conference Proceedings, Nov 2019