How do you measure the effectiveness and validity of your competency-based interviews?

How do you get these skills? Schedule a job interview with the hiring manager and HR. Ask structured questions and take notes. You can then distil the competencies and criteria. This will also show whether your job profile is sufficient or needs to be revised (does the job description cover everything?)

It is crucial to establish from the outset the competencies and behaviors required for success in the job. A blueprint for the perfect match should therefore be created and then replicated. Subsequently, these must be evaluated with a uniform scale and a uniform structure in the interview, otherwise it can lead to biases.

To identify the right candidate from day 1, gather insights from successful incumbents & their managers, & direct reports about the behaviors that contributed to their early success. Ask them to share anecdotal stories that highlight these key behaviors. These real-life examples will help decode the nuances of the behaviors necessary for success in starting the job. Avoid job descriptions—they often contain outdated or overly broad criteria. Real world inputs & stories provide actionable, relevant insights that paint a clearer idea of what it takes to excel in the role from the start. This approach will ensure a more accurate and effective hiring process. Limit it to no more than five competencies plus the behaviors of the company values.

When scoring candidate responses, prioritize those who have demonstrated the required behaviors within the past 24 months. Candidates who have shown these behaviors in two distinct experiences should receive a higher score than those who have only done it once recently. It's not just about "past behavior predicts future behavior"; instead, "recent and frequent behavior predicts future actions." Additionally, audit your notes to ensure accuracy and that the candidate conveyed what you think you heard. This approach focuses on recency and frequency and enhances reliability and validity in identifying the best fit for the role.

The important core is not only the correct question but how you evaluate the answers and what you take from them. That's why I advocate moving away from socially desirable self-disclosure towards further testing of what is said

Consider integrating a digital platform that allows for real-time scoring and feedback during the interview. This can enhance objectivity and provide immediate insights for decision-making. It also improves note taking, documentation and transparancy. Finally, it enables you to measure the effectiveness and validity of your competency-based interviews

When crafting behavioral questions, remember they don't end in a question mark. Most questions you see—those involving who, what, why, when, where, and how—aren't true behavioral questions as they lead the candidate toward specific answers. The 5Ws are probing question only. A behavioral question doesn't explain what the company is looking for or mention a desired outcome. It prompts candidates to share specific experiences and actions that reveal their behavior. For instance, "Describe a recent situation where you had to deal with a significantly challenging coworker" invites candidates to share detailed stories and allows you to assess relevant behaviors without leading them to a particular response.

Implement a "Bar Raiser" methodology. One measure is a periodic peer review system where interviewers assess each other's interview techniques and scoring. This can foster continuous improvement and consistency.

References can be tricky due to legal restrictions. To get around this, ask candidates for references from within the stories they share during the interview. I call it the 'honesty factor.' If a candidate has fabricated the story, they will likely hesitate, change the narrative, or appear uncomfortable when asked for a specific reference. Don't rely on the references provided upfront by the candidate; those are usually pre-selected. Instead, seek out real-time references embedded in their anecdotes. It is the best way to validate the behavior. This approach helps verify the authenticity of their experiences and offers a more reliable assessment of their fit for the role.