Implementation of Artificial Intelligence in Writing Letters of Recommendation

Abstract

INTRODUCTION: Letters of recommendation (LoRs) play a critical role in the residency application process, yet their subjectivity raises concerns about reliability. Artificial intelligence (AI) offers a standardized alternative to human-generated LoRs. This study examines the quality of AI-generated LoRs compared to traditional LoRs. METHODS: Faculty at West Virginia University School of Medicine rated LoRs for residency candidates without knowledge that AI-generated letters were submitted. Candidate quality was controlled using pre-interview Thalamus (SJ MedConnect, Inc. dba ThalamusGME, Santa Clara, CA, US) scores. Independent sample t test and Kendall's W test assessed differences in ratings and interrater agreement. RESULTS: AI-generated LoRs scored higher than traditional LoRs (4.14 vs. 3.29, p < 0.0001). For lower-quality candidates, AI LoRs significantly outperformed traditional LoRs (4.17 vs. 2.85, p < 0.0001). In higher-quality candidates, AI LoRs also scored higher (4.13 vs. 3.63, p = 0.006). Kendall's W test demonstrated interrater concordance (p < 0.05). CONCLUSIONS: AI-generated LoRs were rated superior to traditional LoRs, particularly for lower-quality candidates. These findings suggest AI could serve as an adjunct to enhance LoR quality and standardization. Further research is needed to explore ethical considerations and AI's broader applicability in the letter-writing process.

Ähnliche Arbeiten