How do I know if my chosen psychological assessments or organizational surveys ‘make a difference’ in my organization?

Before we look closer at this question, Figure 1 shows the types of recruitment tools/assessments which define the assessment market.

Figure 1: Typical Assessment types comprising the assessment market

Figure 2 shows the broad employee assessment domains in which psychological and assessment tools are used.

“Before-After” evidence comes in several flavours and timescales

I’m using that “before-after” phrasing to clarify how causality might be most clearly addressed. That is, by comparing relevant outcomes before the use of the chosen assessments/recruitment/workforce survey tools to the outcomes observed during and after their deployment, one might infer that any differences in outcomes could reflect the deployment impact (given the exclusion of other factors potentially impacting outcomes). So, what might constitute empirical evidence?

Quantifiable cost savings/hiring efficiencies

The typical efficiency statement associated with this kind of evidence looks something like: the average time to hire went from months to weeks and saved an estimated 20,000 hours of work from the recruitment team. Recruiters spent 80% less time reviewing applications.

The problem of course is that from such statements, we are actually none the wiser as to the magnitude of financial benefit, nor exactly how it was realised. It may be that such information is commercially sensitive or that it’s just too complex to calculate what the saved time and efficiencies amount to financially. That complexity is exemplified in one of Cognadev’s technical reports entitled: The ROI of the Gallup Q12: Assessing the true value of high-cost HR interventions.

In addition, no information is provided in such statements as to how many of the new-selection-process employees fail to perform as expected over a certain timescale, leave voluntarily, or compare in terms of churn/ retention rate to previous cohorts While upfront costs and efficiencies may look positive, it is possible these are negated by increasing costs incurred over time.

My suspicion is that these efficiency statements are similar to the claims made by those who used utility analysis to show the significant financial benefits of using psychometric testing. The usually huge expected monetary gains computed using the standard Brogden-Cronbach-Gleser utility model remain unobservable financial projections, as explained in a sensible overview chapter by Philip L. Roth, Philip Bobko & Hunter Mabo (2001) and demonstrated computationally by Sturman (2000), in which the abstract states:

“This paper argues that most utility analysis (UA) applications are flawed because they employ an overly simplistic formula. This piece reviews the literature on UA adjustments and demonstrates that the adjustments have a large impact on resulting estimates. These results imply that human resources programs do not invariably yield positive returns; rather, intervention success is contingent on program-specific, organizational, and environmental factors.”

Share Price/Total Shareholder Return

It is tempting to examine the share price of an organization over time, pre-and-post psychometrics deployment, but this is well-nigh impossible as a method of attributing company performance to the use of psychological assessment or workplace surveys. Why? Because so many other factors external to HR workforce interventions can affect such global indices. The recent COVID pandemic is a reminder just how powerful the effects of these external socio-economic conditions can impact any organization, positively or negatively.

Increased profitability and perceived financial value of the organization

This requires careful financial analytics of what is usually a years-long initiative to facilitate significant change and focus among the executive members of leadership teams within an organization, so as to impact corporate profitability, market share, and even the value itself of an organization as perceived by shareholders and investors. A detailed Cognadev case-study blog shows what this looks like in practice, and the clear evidence of financial impact which was achieved:

“A marked difference was found between the results of the two due-diligence procedures that were conducted within the 3-year time frame. The transformational initiative was gauged to have contributed to a significant improvement in the performance of the executive team and the organisation as a whole. This translated into an increment of approximately €3bn to the net asset value of the organisation, which had a significant positive impact on the value of its shares and, therefore, the final acquisition consideration of the organisation.”

This is probably one of the most influential forms of evidence that has clear C-Suite credentials, yet how many senior HR executives or even C-Suite recruitment consultants do the necessary hard yards to acquire such evidence in support of their strategies/decision-making?

Event-based evidence – Accident Rates/Safety, more measurably efficient/productive employees

At a more specific level of analysis, evidence of impact is relatively straightforward when deploying assessments which are targeted to produce clearly specified and measurable outcomes (such as a decrease in workplace/site accidents within an infrastructure or mining organization). Because the outcome is directly measurable and usually mandatory, existing recording systems make such comparative analytics straightforward. Likewise recorded counter-productive/adverse workplace behaviours, or even personal performance targets which are directly measurable (sales, product assembly, etc.).

However, in every case, there has to be a ‘before’ criterion against which the new assessment impact can be compared. For example, it’s of no value to institute a new assessment or AI video interview which is designed to help select more ‘safe’ employees if you have no idea of the current safety incident rates, severity, and characteristics.

Conclusion

The extent to which psychological assessment solutions can contribute to the effectiveness of people decisions and organisational viability, is of critical importance for corporate users as well as for the talent management industry. However, given the dynamic and interactive effects involved, a rigorous evaluation of the quality and appropriateness of assessment practices within specific contexts may be required to determine the associated financial, organisational and personal benefits involved.