For example, the key aspects of understanding job performance are technical (e.g., knowledge and skills), interpersonal (e.g., teamwork and communication), and intrapersonal (e.g., time management and emotional control). Armed with this understanding, I-O psychologists can work with organizations to decide which activities to invest in, such as recruiting, hiring (my research area), training, teamwork, and job redesign.

What is interesting — and the main point of this blog post — is how I-O psychologists can contribute in vital ways in understanding, predicting, and creating pathways between education and the workforce. I-O psychologists have made extensive use of the O*NET Content Model (Department of Labor) as a way to compare job characteristics and job requirements across occupational categories that cover most of the US labor market.

The O*NET Content Model organizes hundreds of variables that describe 923 occupational titles covering most of the US labor market. Notice that the structure of the model aligns well with the three aspects of performance above. Worker Characteristics — work styles, values, and interpersonal demands — line up with the interpersonal and intrapersonal skills that increasingly define good work. Worker Requirements line up with necessary technical skills for an occupation. Experience Requirements reflect the educational investments (training and certification) needed to enter into a given occupation. Specific variables within the Content Model are organized below.

In parallel, educational researchers have been deeply involved in developing a wide variety of skill taxonomies highly relevant to current and future workforce needs (here is a comprehensive overview by ETS, 2023). Harvard’s Explore SEL project even crosswalks these taxonomies, so you can view how frameworks are related (using different words for similar ideas) and unique (tapping new-yet-important skills).

If I-O psychologists could work more closely and more often with educational researchers to align educational taxonomies with O*NET’s descriptors, both fields would benefit: educators would gain clearer targets for what to teach, and employers would gain a workforce whose preparation better maps onto the skills that jobs and organizations actually demand.

In thinking about relevant data and research behind building strong education-to-work pathways, the golden spike at Promontory Point comes to mind, where educational expertise can meet I-O psychology expertise, and a shared, evidence-based vocabulary will connect what students learn to what work requires. This is of course not easy because the number and nature of jobs change as the economy and skills base evolve (hello, AI). But if credentialing (diplomas, certifications, technical training) are better aligned with job descriptions, we will have done better than we have in the past. Educational policymakers and, ultimately, each school and classroom will be put on a stronger track for building the competencies the labor market rewards, doing so with some agility as the supply and demand for skill sets change over time. Helping people understand and pursue greater possibilities in work and in life pays off at multiple levels: individuals, families, and communities, which in turn build a vibrant society and a competitive national economy.

Thank you for keeping me on your radar, and please stay tuned.

• First, the Part 1 blog was about a technical (math) clarification for calculating validity differences in selection systems. Although Schmidt & Hunter (1998) correctly calculated utility differences between two selection systems as the difference in multiple R values (ignoring the cost of testing, etc.), they did not calculate validity differences correctly. Validity differences have to be calculated as the difference between R² values. You can then report that as ∆R², or take the square root and report ∆R, or do whatever other transformation you would like to make—so long as it is based on this ∆R² value. My post did not intend to take a stand on what you “should” report; instead, the goal was very modest, to focus solely on how to calculate a validity difference correctly.
  
  Let me reinforce the point with a specific numerical example: Say that a selection system has a multiple R of .20, and after adding more predictors, the multiple R goes up to .30. The difference in utility is a function of validity and is indeed simply R = .30 - .20 = .10 (S&H had this correct). But .10 is not the validity difference, where instead you have to subtract the respective R² values: ∆R² = .09 - .04 = .05. If you then want to report the R metric instead, you would take the square root of .05 to get R = .22 (…and not .10).
  
  This isn’t new stuff, by the way! (a) In hierarchical linear regression, if you want to report ∆R, you cannot just subtract multiple Rs from each stage of the regression; you would have to calculate ∆R² and then take the square root. (b) Similarly, if you want the average of two standard deviations, you first must square them (to get variances), then average them, and then take the square root.

• Second, given that the Part 1 blog was so narrow and “mathy” in scope, it was decidedly not a commentary on two recent and related papers by Sackett et al. (2022, 2023)1 that update the data and the statistical approach of S&H. Those two papers generated numerous responses2, and two Sackett et al. replies to those responses3. As far as I can tell, none of this recent work performs the calculation the Part 1 blog focused on. However, given the timing of these Sackett et al. papers, I probably should have stated that I was not focused on them—so I’ll do that now.

OK, whew! Now in my next post, I will turn to Part 2, “More Adventures in Meta-Analysis,” which will be much more expansive (and less “mathy”) than Part 1 was. Yes, here I will refer to the Sackett et al. papers and responses, but in the context of a broader meta-analysis idea. Please stay tuned!

The general conclusion of this article is hard to dispute—so long as the employment tests in question are conceptually and psychometrically well supported. However, I will put myself at variance with S&H in two ways. The first way is narrow and statistical; it is covered here (Part 1). The second way is much broader and will be covered in a forthcoming blog post (Part 2).

At Variance with the Variance

Appealing to fair use2, let’s take a look at the often-referred-to Table 1 (S&H, p. 266):

As my wonderful PowerPoint® flourishes indicate above, the second and third columns of numbers would be correct if these referred to utility — not validity. When estimating utility (e.g., predicting the dollar value of selecting employees), then word on the street since 1949 is that you should directly subtract the validity coefficients (correlations, multiple Rs) of each selection system from one another (Brogden, 1949, p. 183)3:

The left side above is the change in the value of a selection system M, when moving to using selection system a, from selection system b. On the right side, the yellow highlight is the difference in the systems’ respective validities (or multiple Rs). Thus, to obtain R_d(utility) the difference in utility that is due to validity, simply calculate:

But in the context of validity, some different math is needed, contrary to the footnotes in Tables 1 and 2 of S&H, both of which state:

This is incorrect. If you want to express an increase in the validity of a selection system in terms of a multiple R, one cannot simply take the difference between the multiple Rs for each system, as we (correctly) did above for utility estimation. Instead, the difference in R²s for each system (the variance components) need to be subtracted from one another. You could then simply express the difference as an R² as well (similar to the ∆R² in multiple linear regression)—or take the square root to report the difference as a multiple R. The need to focus on variances in validity estimation is why I am ‘at variance’ with S&H. 😎

Where R_d(validity) is the multiple R for the difference in validities for two selection systems a and b, where R²_a > R²_b. Use of this formula results in validity gains that are much higher than those reported by S&H (again, S&H should be reporting their results as utility gains, not validity gains).

Stay tuned for Part 2, the broader reason that I’m ‘at variance’ with S&H (insert suspenseful Halloween music here, for one because it might take at least that long for me to post again…).