You are Assessing Talent the Wrong Way. This is How you can Avoid it (1/2)
In this second issue, we dive deeper into waterfall selection processes and show what science says about them. Spoiler: you might be missing out on great talent.
Hey, we are Ramón Rodrigáñez and Andrea Marino, Co-Founders at Nova, the Global Top Talent Network.
Welcome to Talent First, our monthly newsletter where those who believe that talent is the most important resource in the economy get together to learn and discuss attracting, hiring, developing, and retaining talent.
Summary
Talent Assessment 1/2 - Why most selection processes (particularly "massive" ones with hundreds of candidates) are wrong and how holistic assessments can be 10x better (more accurate, faster, and cheaper).
Novas open for a change - The high potentials of Nova who are now open to a career change but don't disclose it elsewhere. You can contact them directly for free.
⭕ Talent Assessment 1/2
Today, we want to discuss a topic that is crucial to the success of any organization: talent assessment. In our previous newsletter, we introduced the concept of "Holistic Talent Assessment" as the most accurate way to assess candidates. Today, we will dive deeper into the concept and unveil the science and data behind it.
DISCLAIMER: The "holistic" methodology makes particular sense for massive selection processes where the volume of candidates is high, especially if the number of candidates is significantly higher than the number of hires to be made, such as in Graduate Programs.
Although the "Holistic Talent Assessment" is still useful in search/headhunting processes with few candidates, those have a different nature and typically require a different process. Moreover, the more past experience counts in a process (i.e. a senior role), the less of a problem the traditional or waterfall way of interviewing is.
1. Waterfall assessment, the traditional way
For many years, organizations have used a traditional waterfall approach to assess candidates, which looks more or less like this:
Screening. The first step is typically a CV screening, where we try to understand if the candidate has the right experience and track record. Some companies, such as management consulting firms or investment banks, would include university grades in this initial screening.
Testing. Successful candidates of the initial phase will be invited to a second phase, typically consisting of some kind of assessment or test. Those are normally IQ-based (inductive/deductive reasoning), but sometimes they may also be skills-based tests (i.e. does the candidate know how to write code or use Excel?). Here we try to test the candidate’s intellectual capacity (to learn, to solve problems, etc.) as well as the candidate’s soft skills.
Interviewing. Successful candidates of the second phase will be invited to a final phase, which may have one or several rounds of interviews to assess competencies (i.e. soft skills) and cultural fit (i.e. does this candidate match with the company and the specific team).
Of course, not all processes are equal, and some selection processes may involve group dynamics, skill tests, and other testing phases. But, overall, selection processes look like a sales/conversion funnel:
2. Why Waterfall Assessments are fundamentally wrong
Waterfall or funnel assessment processes are simply the way the industry has always worked, but that does not mean they are the best or the right way to do it.
In the end, when you want to hire someone for your organization, you do not look for the person with the best CV or who is best at a test. You want the best, period. This means the person who:
Has the right experiences (E)
Has the right hard skills (H)
Has the right competencies and soft skills (S)
Is the best cultural match (C)
If you want to apply a quantitative measure, you would be doing some kind of "weighted average" with the score of all 4 items. For instance, if you believe all are equally important (which is unlikely), your calculation of the final score of a candidate would be:
Total Score = 25% x E + 25% x H + 25% x S + 25% x C
As a hiring manager or person responsible for this vacancy, you want to hire the candidate with the highest Total Score. Of course, your weighting may vary a lot, and you may include some "thresholds" (i.e., a minimum of 80/100 in experience) but all in all, the calculation would look like some kind of weighted average.
The main problem that waterfall processes have is that, by making each part of the selection process a go/no-go and measuring it independently, 2 types of mistakes appear:
False positives: candidates arriving at the final stages who are not among the best.
For instance, candidates who are poor or average on a competence level and cultural fit but who have great CVs and tests (high E and H, low S and C) reach the final steps. This means you might be spending time interviewing the wrong people.
False negatives: candidates who are among the best who are not given the opportunity and are rejected early in the process.
For instance, outstanding candidates on the interview and test but who maybe did not spend time preparing their CVs (maybe because nobody taught them how to: low E, high H, S, and C) would not be given the chance. This means you might be missing out on great talent for your company.
However, there is a better way to assess talent that can provide more accurate results and help organizations make better hiring decisions. We are talking about a “Holistic Talent Assessment”.
3. What the science says about assessments
There are many theories of what is talent and how to measure it. At Nova, we have our own, but there are certainly a lot of different opinions around it.
One of the most famous pieces of research in the complex world of talent assessment was done about 20 years ago by Frank L- Schmidt from the University of Iowa: “The Validity and Utility of Selection Methods in Personnel Psychology”, which summarizes the practical and theoretical implications of 85 years of research in personnel selection and their validity for predicting job performance.
The main conclusions are the following:
GMA (General Mental Ability) or “IQ tests” are the single best predictors of job performance for entry-level jobs. That means that, in general, GMA is the best predictor of performance with a 0.51 validity (goes up to 0.58 for professional-managerial roles).
Work sample tests (i.e. a problem to be solved at home by the candidate) and structured employment interviews are also relevant (0.51-0.54), but there is less research about them, and are only valid for experienced workers.
The combination of measures correlates better with performance. Combining a test with a structured interview or with a “task-at-home” is the best predictor of performance.
Combining these measures correlates higher with job performance (0.63)
This research clearly supports the case for holistic assessments, which combine several measures of the candidate, such as interviews, tests, and CV, to deliver a more accurate perspective than any of those on their own.
4. Introducing Holistic Talent Assessment
We understand "Holistic Talent Assessment" to be a talent measure that considers a candidate's background, IQ, hard skills, competencies, and cultural fit all at the same time before deciding whether or not to continue with them. This type of assessment helps reduce both false positive and false negative mistakes and can dramatically increase the accuracy of selection processes.
In particular, this approach can help identify candidates with potential who may not have the exact experience or qualifications listed on their resume but who could be exceptional performers thanks to superior IQ, cultural fit, and/or soft skills.
To create a "Holistic Talent Assessment," organizations need to design a process where candidates go through multiple stages of the selection process before being discarded:
Resume (i.e., background and experiences)
Testing/”Task-at-home” (i.e., IQ and/or hard skills)
Interview (i.e., competencies and cultural fit)
You may think this sounds beautiful but practically impossible. "I do not have the time or resources to interview hundreds of candidates," we often hear. And that is totally right. It does not make sense to interview 100 people to recruit 2.
But technology is here to help. If you make the process asynchronous and the interviews are recorded (video interviews), you save hundreds of hours in coordination and can scale a process where you actually see many candidates in action. Also, you may want to take shortcuts and only look at the best X candidates from steps 1 and 2 and their interviews. That would already yield massive improvements in accuracy.
Finally, you will most likely want to conduct a final live interview (online or remote) before issuing offers. And that makes total sense. In a recorded video, you never get to fully meet the person. Our proposal is to simply take the very best from the holistic talent assessment and have them go through that live interview as the 4th and last step in the selection process.
5. The math around Holistic Talent Assessment
At this point, you are most likely convinced that assessing talent in a holistic way is more accurate, but you probably still believe it is not practical and time-consuming. We want to prove you wrong. What if we told you that a holistic assessment can be both more accurate AND less costly?
We will do the math together in our next issue. Compared to a Waterfall Assessment, Holistic Assessment…
… can be cheaper (~16% in our case study)
… while reducing false positives/negatives in the final rounds (~44% in our case study)
… and ensuring you do not miss out on the best candidates who do not reach the final round in a waterfall assessment (~17% on average in our case study)
(1/2) To be continued.
👀 Novas open for a change
And now, like every newsletter from now on, we want to introduce you to +50 Novas (i.e. top talented, pre-vetted individuals) who are passively looking for new job opportunities. You can reach out to them with your open vacancies for free!
PS: At Nova, our mission is to become the Talent Agents of the most talented people in Business and Tech. We have created a merit-based access community of+18.000 pre-vetted, high-potential individuals who trust us to help them achieve their full potential through networking, development, and career acceleration opportunities.
Every month, we share the anonymized profiles of Novas who are now open for a change so that HR Managers and Founders can contact them directly and avoid wasting time on spamming candidates who are not ready for a change on LinkedIn.
Thanks for reading Talent First.
If you liked this issue, don't forget to hit ❤️
We would be grateful if you share it by email or social media with other people who might like it or who might be looking to hire top talent.
__
Andrea & Ramón
Co-Founders at Nova