Industrial and Organizational Psychology - Research Methods

Research Methods

As described above, I–O psychologists are trained in the scientist–practitioner model. I–O psychologists rely on a variety of methods to conduct organizational research. Study designs employed by I–O psychologists include surveys, experiments, quasi-experiments, and observational studies. I–O psychologists rely on diverse data sources including human judgments, historical databases, objective measures of work performance (e.g., sales volume), and questionnaires and surveys.

I–O researchers employ both quantitative and qualitative research methods. Quantitative methods used in I–O psychology include both descriptive statistics and inferential statistics (e.g., correlation, multiple regression, and analysis of variance). More advanced statistical methods employed by some I–O psychologists include logistic regression, multivariate analysis of variance, structural equation modeling, and hierarchical linear modeling (HLM; also known as multilevel modeling). HLM is particularly applicable to research on team- and organization-level effects on individuals. I–O psychologists also employ psychometric methods including methods associated with classical test theory (CTT), generalizability theory, and item response theory (IRT). In the 1990s, a growing body of empirical research in I–O psychology was influential in the application of meta-analysis, particularly in the area of the stability of research findings across contexts. The most well-known meta-analytic approaches are those associated with Hunter & Schmidt, Rosenthal, and Hedges & Olkin. With the help of meta-analysis, Hunter & Schmidt advanced the idea of validity generalization, which suggests that some performance predictors, specifically cognitive ability tests (see especially Hunter and Hunter & Schmidt ) have a relatively stable and positive relation to job performance across all jobs. Although not unchallenged, validity generalization has broad acceptance with regard to many selection instruments (e.g. cognitive ability tests, job knowledge tests, work samples, and structured interviews) across a broad range of jobs.

Qualitative methods employed in I–O psychology include content analysis, focus groups, interviews, case studies, and several other observational techniques. I–O research on organizational culture research has employed ethnographic techniques and participant observation to collect data. One well-known qualitative technique employed in I–O psychology is John Flanagan's Critical Incident Technique, which requires "qualified observers" (e.g., pilots in studies of aviation, construction workers in studies of construction projects) to describe a work situation that resulted in a good or bad outcome. Objectivity is ensured when multiple observers identify the same incidents. The observers are also asked to provide information about what the actor in the situation could have done differently to influence the outcome. This technique is then used to describe the critical elements of performance in certain jobs and how worker behavior relates to outcomes. Most notably, this technique has been employed to improve performance among aircraft crews and surgical teams, literally saving thousands of lives since its introduction. An application of the technique in research on coping with job stress comes from O'Driscoll & Cooper.

I–O psychologists sometimes use quantitative and qualitative methods in concert. For example, when constructing behaviorally-anchored rating scales (BARS), a job analyst may use qualitative methods, such as critical incidents interviews and focus groups to collect data bearing on performance. Then the analyst would have SMEs rate those examples on a Likert scale and compute inter-rater agreement statistics to judge the adequacy of each item. Each potential item would additionally be correlated with an external criterion in order to evaluate its usefulness if it were to be selected to be included in a BARS metric.