Enter the matrix: Scientist self-assessments that make sense!

  • June 27, 2018
iJOBS Blog

Edited by: Aminat Saliu Musah You approach the office of your program director with sweaty palms and little scientific progress. You have been dreading this progress meeting since your graduate program assistant asked you to fill out a doodle poll a month ago. As you enter their cluttered office you wonder about how you compare your progress to other graduate students and your own personal career and skills development. When will I graduate? What, if anything in science, am I good at? When will I be ready for the next step in my career progression? Like many unclear things in graduate school, self-assessments of strengths and weaknesses can be difficult. Graduate school is an apprenticeship and, unfortunately, some of us have advisors who believe no feedback is good feedback. While this “sink or swim” model has been successful when faculty positions were plentiful, if you read a few of the articles on the iJOBS blog it becomes clear that there are no longer enough faculty positions for the number of PhD scientists produced. How is it decided that certain trainees are better equipped than others for a successful scientific career? Often, that is a factor outside of the trainee’s control and is determined by being in the right lab at the right time. In a recent article entitled “Point of view: Competency based assessment for the training of PhD students and early-career scientists,” several graduate and post-doctoral program leaders lament the lack of a national framework for assessing scientific development and future scientific career success. They describe how under the current American system “The success of a scientist is highly variable and depends on a number of factors; the trainee’s supervisor or research adviser, the institution and/or graduate program, and the organization or agency funding the trainee.” In contrast to other systems of PhD education the American system does not include a comprehensive assessment to determine the scientific potential of recently graduated or graduating scientists. This is problematic because there can be discrepancies between the quality of PhD programs between peer American institutions. The authors were able to identify 10 important core competencies that can be used to evaluate the scientific development of career PhD scientists and their development programs. Ten Core Competencies for the PhD Scientist:  

  1. Broad Conceptual Knowledge of a Scientific Discipline- This refers to a scientist’s ability to engage in scientific conversations across a discipline their discipline (biology, chemistry…)
  2. Deep Knowledge of a Specific Field- Encompasses the historical context, current state of the art, and relevant experimental approaches for a specific field, such as immunology or nanotechnology.
  3. Critical Thinking Skills- Focuses on elements of the scientific method, such as designing experiments and interpreting data.
  4. Experimental Skills- Includes identifying appropriate experimental protocols, designing and executing protocols, troubleshooting, lab safety, and data management.
  5. Computational Skills- Encompasses relevant statistical analysis methods and informatics literacy.
  6. Collaboration and Team Science Skills- Includes openness to collaboration, self- and disciplinary awareness, and the ability to integrate information across disciplines.
  7. Responsible Conduct of Research (RCR) and Ethics- Includes knowledge about and adherence to RCR principles, ethical decision making, moral courage, and integrity.
  8. Communication Skills- Includes oral and written communication skills as well as communication with different stakeholders.
  9. Leadership Skills- Includes the ability to formulate a research vision, manage group dynamics and communication, organize and plan, make decisions, solve problems, and manage conflicts.
  10. Survival Skills- Includes a variety of personal characteristics that sustain science careers, such as motivation, perseverance, and adaptability, as well as participating in professional development activities and networking skills.

These core competencies are very broad, however, they give a potential scientist guidance towards developing successful skills and habits. You might find that some important skills like business and management are left out of these competencies. To address these issues, the authors included sub-competencies that fit under several of these categories. How is this assessment different than other administered tests? Rather than using a self-assessment to measure a student’s proficiency the authors propose several observable measures of a student’s effectiveness. To evaluate the student’s effectiveness, the authors propose using a “milestone” model for student evaluation, whereby when a student achieves these milestones they have “progressed” to the next level. The authors then choose these proficiency levels for each student: novice, advanced beginner, competent, proficient, and expert. You can see from the figure, as a student progresses, the following table would have to be filled out for each student: Table 3 from paper including milestones One criticism I had is that author’s do not list specific examples of how each competency can be represented and rely on the student and advisor to define what determines progress in each of these categories. To address this problem conversations should happen between the students and their supervisor or course director to identify areas of improvement as well as areas of strength. These competencies can be used in conjunction with course curriculum changes to develop fantastic graduate programs. The competencies can provide the student with clear personal objectives as well as providing feedback to the program directors on areas that students could improve upon. I think that implementing these competencies will help PhD granting institutions recruit higher quality students and post-docs by providing aspiring career scientists with a professional development structure. Adding in these objectives will give PhD students a greater chance at succeeding during and after graduate school. This article discusses many of the frustrations I hear and have noticed at my own graduate institution. Graduate students struggle with advisors who have no clear plan, developing their own research project without prior experience of creating a project, and graduate programs that have no clear path for scientific development. I know that Rutgers has been taking steps to address it by passively having us fill out the AAAS Individual Development Plan (IDP). The IDP, when used properly, can help graduate students and their PIs track thesis progress as well as professional development progress. Ultimately it is still up to the individual student to determine, plan and navigate their competencies for career objectives. I think that progressive graduate programs can and will use these guidelines to mentor and develop students so that the students can reach their career goals and the graduate programs can produce fantastic scientists. In the future students will, hopefully, enter program director meetings with a positive outlook and leave with a clear sense of direction.