Nosek et al., 2010, Cumulative and career-stage citation impact of social-personality psychology programs and their members

citation

Nosek, B. A., Graham, J., Lindner, N. M., Kesebir, S., Hawkins, C. B., Hahn, C., Schmidt, K., Motyl, M., Joy-Gaba, J . A., Frazier, R., & Tenney, E. R. (2010). Cumulative and career-stage impact of social-personality psychology programs and their members. Personality and Social Psychology Bulletin, 36, 1283-1300. [Request paper]

Considerations of inclusion criteria for who is in a social-personality program

Our starting point was identifying primary members of social-personality psychology programs, not social psychologists. These are highly related, but not perfectly related. Both are fuzzy categories. When we started to refine our inclusion criteria by searching programs, we found remarkable (i.e., unexpected) variation in how social programs define themselves, and who comprises them. In retrospect, it is obvious – the definitions of social-personality psychology and a social-personality psychologist are fuzzy.

Here are some inclusion criteria we considered, and our assessment of the problems of using each as a singular criterion:

1) Whether the person’s subfield expertise is social-personality (Problem: What counts as social-personality is not consensually shared. For example, some would not count judgment and decision-making, others would. Some would not count health psychology, others would.)

2) Whether the individual publishes in social psychology journals. (Problem: What counts as a social psychology journal? To narrow to a set that everyone agrees are prototypes [e.g., JPSP, PSPB, JESP] leaves out some faculty and people that are social psychologists by every other criterion except that they do not publish much, or in top journals.)

3) Whether the individual got their PhD from a social psych program or social psychologist advisor (Problems: Pushes the definition of social psychologist back a level. Also, interests/expertise develop and change over time.)

4) Whether the person is a primary supervisor of graduate students admitted to the social area. (Problems: Some primary members don’t serve as primary advisor. Some departments don’t require the graduate students to specify the subarea. Some junior scholars have not attracted or graduated a student yet.)

5) Whether the individual is funded at least 50% by the Psychology department (Problems: Being funded by the department does not clarify whether the person is in the social-personality program. Funding is infrequently by area/subdiscipline within a department.)

6) Whether the person spends at least 50% of his/her time with the area (Problems: Very hard to clarify. Some people are easily seen as “primary” contributors even if they are below this percentage. [For example, Bobbie Spellman at UVa is ½ in the law school and ½ in Psychology, she also is a member of the Social and Cognitive areas. Simultaneously, she is easily defined as a primary member of the Social area by all other criteria.] Some areas are complex admixtures - e.g., Social, Abnormal, Cognitive - making it difficult to parse the social-personality core even if the person is a full-time member.)

7) Whether the person is a member of SPSP (Problem: There are lots of members of SPSP who are not easily identified as members of their social psychology programs by any other criteria. There are people who are not members of SPSP, but viewed by the program as being essential contributors to the area.)

8) Whether the person is identified on the department website as a member of the social-personality area (Problem: Departments vary in their strategy for identifying area members. For example, some include people that have interests in social-personality even if they have no involvement with the area or discipline otherwise. Also, many department websites may be several years out of date.)

9) Whether the person has a profile on the social psychology network website (http://socialpsychology.org/). (Problem: Not all relevant people have posted a profile at SPN for a variety of reasons that are unrelated to being a social-personality psychologist.)

In sum, because of the wide variety of instantiations of social-personality programs, any single criterion (or subset of criteria) generates the membership that violates intuitions and/or self-identities of at least a subset of the graduate programs. As such, we opted for an approach that maximizes respect for Departments’ and Areas’ self-definitions of their area and its members. We began with Department websites to generate a list of the members that the program suggested were primary members of the Social Psychology program. Identifying “primary” involved consideration of the criteria above.

Next, we contacted a member of EVERY department to confirm or revise our list. In those contacts, we (a) described our project goal, (b) gave context about clarifying the primary members of their social-personality area, and (c) showed them the list that we generated ourselves. If they offered corrections that still fit within our general criteria, we edited our list accordingly. [Exceptions would be clear violations, such as suggesting addition of a faculty member that was not a member of the program on August 1, 2009.]

Importantly, our criteria should NOT be considered the definitive word on inclusion criteria or how to define areas. These decisions have some arbitrariness. It will be useful to examine convergence and divergence of indicators across a variety of methods for defining inclusion criteria.

Citations Home page, Brian Nosek's Website