Department of Medicine Fellowship Recruitment Toolkit

The purpose of this toolkit is to collect, align, and share best practices in fellowship recruitment. This allows the department to advance strategic goals and ensure that our workforce is best able to serve the healthcare of our communities.

Preparation and Time Allocation

• Review documents for updates.
• Avoid recycling documents to save time.
• Provide time to review candidates and their application materials.
• Ensure time for interviews/ranking, including time for rubric/evaluation directly after the interview.

Search and Screen Committee

Provides different perspectives.

• Assemble a committee with a variety of representation.
• Prepare and allot time for the goals, interview questions, rubrics, evaluations, rankings, etc.

Unconscious Beliefs

• Engage in ongoing learning, unlearning, and relearning to minimize the impact of our unconscious beliefs on decision-making.
  • Normalize unconscious beliefs: everyone holds unconscious beliefs about other people or groups of people based on attitudes, associations, and stereotypes.
  • Implement strategies for mitigating unconscious beliefs while reviewing applications, interviewing candidates, and any other areas where unconscious beliefs shows up.
  • Mitigate for a particular unconscious beliefs through stereotype replacement. Replace stereotypes with positive experiences from individuals who identify from that group.
  • Strive for progress versus perfection.

Holistic Review

AAMC Holistic Review: A flexible, mission-driven approach to recruit and assess an individual’s competencies by considering their experiences, attributes, and academic metrics in order to select applicants who will best contribute to the program’s unique goals, learning environment, and the practice of medicine. 

• Holistic Review Core Principles

Screening

The screening process must be strategic to avoid unintentional barriers preventing all qualified candidates to be fully considered.    

• Rely upon objective strategies in making decisions to move a candidate forward.
• Stop periodically to review your evaluation criteria and screening rubrics to ensure their application in the process is aligned with the fellowship requirements.
• Provide support for every decision to advance or eliminate a candidate.
• Redacted application screening is the practice of intentionally making anonymous personally and/or demographic information from application materials to reduce the effect that unconscious beliefs have on assessing candidates.
  • NOTE: We encourage the use of redacted/anonymous application screening versus blind screening.

Objective Language

• For example:
  • Use they/them/theirs.
  • Use technical skills instead of hard skills
  • Use people skills instead of soft skills
• Use phrases that are broad unless the focus is on a particular identity group.
• Use language that centers the candidates.
• Avoid references to “fit” and/or “goodness of fit” in the process. Instead, focus on qualifications and skills.

Unconscious Beliefs

• See bullet points from PHASE 2: APPLICATION REVIEW
• AAMC: Be aware of your unconscious bias
  • Interviewers can help mitigate their unconscious beliefs through:  
    • Awareness of personal beliefs.
    • Deliberate thinking and decision-making.
    • Perspective taking.
  • Increasing standardization to help reduce the impact of unconscious beliefs. Implement the following enhancements:
    • Clearly define evaluation criteria.
    • Use a scoring rubric.
    • Ensure variety in interviewers.
    • Train interviewers. 

Interview Recommendations

• AAMC: Best Practices for Creating & Conducting Interviews
  • Select a virtual interview format
  • Develop a structured interview: Structured interviews involve standardized procedures, including predefined questions and established scoring rules, which enhance reliability and validity.
    • Sample structured interview

Structured Interviews

• AAMC: Conducting Effective Structured Residency and Fellowship Selection Interviews: Preparation, Execution, and Follow-Up
  • Before - tips for preparing
  • During - tips for conducting the interview
  • After - post interview tips
• UW-Madison Office of Human Resources Diversity, Equity, and Inclusion Interview Questions [PDF]

• AAMC: Developing Structured Residency and Fellowship Interviews: Content and Evaluation Related Components

  • Choosing competencies to evaluate
  • Behavioral and situational questions
  • Evaluation-related components

  Maintain a balance between the number of competencies you want to assess, the number of questions needed to assess them, and the amount of time you have available for each interview.

Evaluation

• AAMC: Evaluation-Related Components: Use rating scales to evaluate candidate’s responses. Rating scales will increase interviewers’ ability to compare candidates because they were evaluated on a common scale.
• SMPH HR Resources: Interview Evaluation Rubric (with rating benchmarks)
• SMPH HR Resources: SMPH HR Qualtrics Library

Other Interview Resources

• Bias in Recruitment and Hiring Modules
• Understanding and Addressing Microaggressions

Best Practices

• Mitigate unconcious beliefs in fellowship ranking by using these strategies:
  • Structured Interviews (See bullet points from PHASE 3: INTERVIEW/EVALUATION - Structured Interviews)
  • Unconscious Beliefs awareness (See bullet points from PHASE 2: APPLICATION REVIEW - Unconscious Beliefs)
  • Standardized Rubrics (See bullet points from PHASE 3: INTERVIEW/EVALUATION - Evaluation)
  • Anonymous Review (See bullet points from PHASE 3: INTERVIEW/EVALUATION - Screening)
  • Varied Review Panels (See bullet points from PHASE 1: PRE-RECRUITMENT - Search & Screen Committee)
  • Regular Calibration (See bullet points from PHASE 2: APPLICATION REVIEW - Screening)
  • Feedback and Reflection (See bullet points from PHASE 2: APPLICATION REVIEW - Screening)

Phase 1: Pre-Recruitment

• SMPH Human Resources: Search & Screen Committee guidelines

Phase 2: Application Review

• SMPH Human Resources: Unconscious Bias Learning
• UW Health/SMPH web training (six modules): Bias in Recruitment and Hiring
• Harvard University: Implicit Bias Test
• AAMC: Unconscious Bias Resources for Health Professionals
  • Virtual seminar: What You Don’t Know: The Science of Unconscious Bias and What to Do about it in the Search and Recruitment Process
    • Facilitator's Guide [PDF]
  • AAMCNews article on unconscious bias in academic medicine
• AAMC: Holistic Review
• NIH:
  • Evidence-Based Inclusive Graduate Medical Education Recruitment Strategies
  • Everyone Looks the Same in ERAS! Differentiating Between an Increasingly Homogenous Pool of Graduate Medical Education Applicants
• Science Direct: Utilization of Core Values and Branding to Recruit the Best Trainees for Your Program
• SMPH Human Resources:
  • Screen Applications
  • Applicant Screening Spreadsheet [spreadsheet]
  • Application Evaluation Criteria Rubric [Word doc]
  • Qualtrics Library (survey templates)

Phase 3: Interviews/Evaluations

• Key Considerations: Interviews in GME: Where Do We Go From Here?
• AAMC:
  • Literature supporting use of structured interviews
  • Behavioral and situational questions
• SMPH HR Resources:
  • Candidate Interviews
  • Interview Questions
• UW-Madison Office of Human Resources Diversity, Equity, and Inclusion Interview Questions [PDF]

Phase 4: Ranking/Offer

• The Match National Resident Matching Program

Resources: Web-based Trainings

• Race, Racism, and Antiracism  
• Why Lead with Race?
• Understanding and Addressing Microaggressions
• Embedding Equity Tools
• Multicultural Interactive Video
• Disability and Ableism  
• Bias in Healthcare