12:24 pm | 0 recommendations | Be the first to comment

The Four Best Methods for Selecting Top-Performing Employees

The following post is a continuation from the “Hiring with Insight” article series, which was designed to help high-growth small and medium-sized companies overcome problems associated with the early stages of company development. Full article, with graphics and tables, can be viewed at http://blog.hireinsightselect.com.

• Selection methods must be valid and reliable in order to help you accurately predict candidate success.
• Only use methods that demonstrate a reliability of at least 0.40 and a validity of at least 0.21.
• Four methods, rarely used by SMEs, greatly exceed these validity and reliability criteria.
• Two of these methods, used together, give you the best chance of predicting success at the lowest cost.

In the last edition of Hiring with Insight, we discussed the three most common hiring methods used by SMEs and why they are so ineffective at identifying top talent. According to the reaction we received from readers, it is clear that many of you have been grappling with the problem of finding top people and are very frustrated with traditional means of exploring candidate skills and qualifications.

Fortunately, there are at least four different modern approaches available to you that will greatly increase your ability to predict performance, despite the fact that they are rarely used by small and medium-sized companies. These are methods that researchers have explored in great detail, over many years, in studies that have evaluated the performance of hundreds and thousands of job candidates. Over the next few pages, we will provide a description of these methods, discuss why they are helpful, and explain how you can cost-effectively implement them in your hiring practices.

A Measure of Quality

If you were asked to find the best long-distance runner in a group, what would you do? Very likely, you would have everyone “perform the job”, in terms of running as far as they could for as long as possible. You could then clearly see which people possessed the most endurance and easily determine who you would want on your next marathon team. How simple the role of hiring manager would be if we could just give such a task to all of our job candidates and accurately predict their future performance!

Unfortunately, most jobs are much more complex than this example would illustrate, so you are forced to find ways to predict success without having candidates actually do the work. This includes assessing various qualities that we assume relate to actual performance on the job, such as the responses that candidates provide during an interview. However, how do we know for sure that such qualities relate to performance and that our way of assessing them is in fact accurate?

Validity

The first is to explore validity, or the degree to which a selection method predicts job success. Researchers and test publishers use a correlation coefficient to determine validity, which tells us the degree to which a certain method is able to predict actual job performance. This number can fall between 0.00 and 1.00, and a number closer to 1.00 is preferred as it means that a measure is more highly related to job success.

For example, our marathon running test may have a validity coefficient close to 1.00, because it is such an excellent measure of endurance in that context. However, selecting top runners based on hair colour would likely have a validity coefficient of 0.00 because it should have absolutely no relationship to one’s ability to finish a marathon. Basically, it would be the same as flipping a coin to choose candidates - or only slightly worse than depending on a resume.

Reliability

The second is reliability, or the degree to which a method gives us consistent results. For example, if we give accounting candidates a math test on Monday and then again on Friday, we need to know that each would receive a similar score - otherwise we would have to assume that the test is defective, as math skills are very unlikely to change within a week.

Again, a correlation coefficient is used to determine reliability. A test that gives the exact same result over time would have a reliability coefficient of 1.00, while one that gives completely random results over time would have a coefficient of 0.00.

With that short introduction to validity and reliability, you have a much better idea of how to determine the quality of the selection methods you are currently using. For information on the quality of specific tests and measures, please ask your provider or review the research manual they have provided (any well-designed instrument will have this documentation).

Quick Tip

A validity coefficient is

• Excellent if larger than 0.35,
  
• Good if between 0.21 and 0.35,
  
• Poor if less than 0.21.
  

A reliability coefficient is

• Excellent if larger than 0.75,
  
• Good if between 0.40 and 0.75,
  
• Poor if less than 0.40.
  

- U.S. Department of Labor, Employment and Training Administration, 2000

The Four Best Hiring Methods

In the last edition of this articles series, we included a table with some blank rows, which was intended to get you wondering about what methods could be so much better than the typical resume, reference check and unstructured interview. These results were uncovered by researchers who reviewed a large number of studies to find out which methods were the most reliable and valid measures of job performance.

Their findings are not opinion, nor are they gathered from a single study that may have provided a biased account of the results. To give you an idea of the scope of this work, the validity coefficients for the different approaches to interviewing were provided by Wiesner and Cronshaw, whose research review included over 51,000 people! That said, let’s talk about the selection methods hidden in the table.

Structured Interviews (Validity of 0.63)

As mentioned in the last article, traditional unstructured interviews are notorious for being generally accepted practice, particularly among SMEs, yet also quite ineffective at predicting future job performance. When leading researchers and practitioners realized this, they began developing a more structured approach, the best of which had the following characteristics:

• Questions were developed from an objective job analysis. This ensured that all questions were directly related to job tasks and problems.
• The same job-related questions were asked of all candidates.
• Quality of the response was judged using standard rating scales. Scores were given based on whether candidates’ answers demonstrated a certain behaviour or ability (e.g., if candidate displays the ability to manage complex strategic projects, they receive a score of “5” for “Planning”).

The results from validity studies on this approach to interviewing were very impressive, with some asserting that hiring managers could predict up to 40% of job candidates’ future performance by their responses during the interview. Of course, a number of factors would affect this, including interviewer training and ability, and the quality of the job analysis conducted and subsequent questions developed.

Work Samples (Validity of 0.54)

According to the research, the second most effective method for selecting new employees is the work sample. In essence, it involves having candidates perform a task or series of tasks that are done on the job and are critical to success. For example, a firefighter candidate may be provided with required equipment and asked to manage a small controlled fire. Or an accounting candidate may be provided with fake company financial records and asked to balance the books. The hiring manager would give the same test to all candidates and score them according to a standard set of rating scales, developed through a job analysis.

Although work samples are highly valid, they also can be difficult and expensive to develop; in most cases, they have to be customized for a particular job in a specific company. Work samples may be particularly difficult to develop for knowledge workers, considering that much of the work involves tasks and actions that are not easily observed and rated.

Ability Tests (Validity of 0.53)

There are three generally-accepted types of ability tests, including those that measure cognitive, physical and sensory/perceptual, and psychomotor abilities. Although the latter two types tend to be expensive and time consuming, and require specialized testing equipment, the former cognitive ability tests are typically cost effective and easy to administer - most come in the form of paper-and-pencil or online tests that can be completed in under an hour.

Even better, well-researched tests of cognitive ability have been found to relate very highly to job performance in a wide range of roles, particularly as the complexity of the job increases. For example, Hire Insight uses well-developed tests of verbal and numerical reasoning ability to assess candidates for managerial and professional roles because such scores relate very well to an employee’s ability to effectively solve problems, learn and benefit from training, adapt to changing issues and circumstances, and even helps determine the employee’s level of job satisfaction (Campbell, Dunnette, Lawaler & Weick, 1970; Gottfredson, 1986).

Assessment Centres (Validity of 0.43)

The last selection method that demonstrates a high relationship to job performance is the assessment centre. Typically, this involves candidates performing a series of tasks that mimic those done on the job, and having their performance rated by a trained assessment team. For example, they may take part in a Leaderless Group Discussion, then have to manage issues found in a simulated In-Basket. Some of these exercises can be purchased off-the-shelf, although they are typically custom designed for a particular purpose.

Assessment centres may also include other methods that help increase the validity and information gained from the exercises, including ability tests, interviews and personality questionnaires - it really depends on the type of role and job requirements at a specific organization. A typical assessment centre would require candidates to perform exercises over 2-3 days and involve a team of assessors who would rate and write reports for each candidate.

Creating a Selection Program for the SME

So, we simply need to design a selection program that includes all four of the best methods, right? No, definitely not. As you read through the description of each method above, you very likely identified a couple of key problems for the SME - time and expense. While a larger more established organization may be able to invest $100,000 and months of time in developing and administering work samples or assessment centres, investing those resources would be unrealistic for a quickly growing smaller company.

Fortunately, the two least expensive and time-consuming methods can also together be the best ways to predict success, with many HR leaders stating that a combination of structured interviews and job-relevant cognitive ability tests is the ideal means of evaluating future talent. For example, Hunter and Hunter found that a test of cognitive ability has a better cost-benefit ratio than any other single test method - used with a well-developed structured interview, any small and medium-sized business should be able to understand each candidate’s potential and make much better predictions about future success.

Without a doubt, traditional selection methods need to make way for more modern approaches, at the very least because the former provide few benefits for the time and expense that must be invested. Of course now that you know that change is needed, you are likely wondering how to begin. In upcoming editions of this article series, we will be moving from the conceptual to the practical, discussing the steps that you can take to develop and improve a hiring process that will help you consistently select the best talent available.

MeetingPlace

In anticipation of the next edition of this newsletter, we would greatly appreciate hearing more about your hiring problems and frustrations so that we can make comments and recommendations that are most helpful and practical to you and your company. As further incentive, a select few participants will be provided with a free full assessment for them or one of their team members.

www.hireinsightgroup.com/survey.html

Featured Candidate Profile

Benjamin K.

Ben was the latest candidate assessed by Hire Insight and added to the pre-assessed candidate database, to be viewed by companies hiring for marketing managers.

As with all successful candidates, he participated in a very intensive but informative assessment process and did exceptionally well compared to other managers.
He was certainly one of the most gifted problem-solvers we had evaluated, scoring better than 99% of the 8,436 people who had completed a test of numerical reasoning skills and better than 90% on a test of verbal reasoning. His capacity for learning and adapting to new situations was certainly a factor in helping him be successful as a business owner and marketing executive, particularly for the years he spent in international business development, providing consulting services for foreign companies entering European markets.

One of the key factors that stood out for Ben was his drive to compete and achieve - to work extremely hard to ensure that his team and company were outperforming other organizations. This included coming up with novel approaches and ideas to market a product, but also performing his due diligence in ensuring that a solution was in fact ideal. This also meant taking time to gather relevant data and think through the impact of the decision.

As an added bonus for this article, we have decided to include Ben's full eight-page assessment report (http://www.hireinsightgroup.com/s61.pdf). It includes an executive summary of his strengths and accomplishments, feedback provided by former superiors, peers, direct reports and clients, as well as his responses to the behaviour-based interview that help support the results of his testing and assessment. If you have any questions or comments about this article or Hire Insight recruitment and assessment programs, please feel free to contact us.

06:52 pm | 0 recommendations | Be the first to comment

Costly and Ineffective Hiring Methods used by Small and Medium-Sized Businesses

The following post is from the "Hiring with Insight" article series, which was designed to help high-growth small and medium-sized companies overcome problems associated with the early stages of company development (full article, with graphics and tables, can be accessed at http://blog.hireinsightselect.com).

• High-growth SMEs are particularly sensitive to the actions and decisions made by employees.
• SMEs are at particular risk of making poor hiring decisions, compared to larger organizations.
  
• Three of the hiring methods most commonly used by SMEs do not help predict job success.
  
• Resume reviews, reference checks and unstructured interviews are among the worst ways to identify talent.

As a key decision-maker for a high-growth Small or Medium-sized Enterprise (SME), you understand how critical it is to make good hiring decisions. This is not to say that selecting the right people is not important for other organizations - in fact, it is one of the most vitally important aspects of running any business. However, in comparison to those at a more stable and established company, you know that individual contributors at a smaller firm can have an astounding impact on its continued performance.

Unfortunately, such organizations are also especially sensitive to bad hiring decisions, when the people selected do not have the qualities or abilities needed for success and generally fail to fit in with the culture of the team and company at large. And, in most cases, these poor decisions were made because hiring managers were unable to accurately predict success using more conventional selection methods.

This is not simply an assumption. Catano et al. reviewed the most commonly used selection instruments for their book, Recruitment and Selection in Canada, and found that SMEs were more likely, than larger organizations, to use traditional methods that did not accurately predict performance in a role. Possibly because such organizations do not have the same level of HR support, or are not as experienced with more sophisticated methods, they are very much at risk of making very costly mistakes when it comes to hiring. A change is needed.

The first step towards improvement is identifying the most ineffective methods that you may be using today and how they are costing you much more time, money and frustration than is necessary. For that reason, we will devote our attention in the remainder of this article to discussing the primary methods used by SMEs to recruit, screen and assess candidates. In the next edition of the Hiring with Insight Newsletter, we will provide information on more sophisticated methods that are not only less expensive, but will greatly improve the accuracy of your hiring decisions.

“Another key to the HOT 50’s success: they know the importance of top-notch employees. Some 90% report that the ability to attract and retain good workers has been critical to their growth, while 70% say they could have grown even faster were there not a shortage of qualified people. Moreover, while a bad hire is a major inconvenience for an established business, it can be lethal for a startup. That’s why the HOT 50 strive to find and keep top staff.”
- PROFIT magazine, October 2007

Resumes

By far the most frequently used screening tool, the resume or CV has for many years been the primary method companies use to identify appropriate candidates. In fact, researchers have found that 94% of SMEs relied on resume information when making selection decisions; our own experience suggests that in many cases it is the only method that they use before investing the considerable time and expense required to conduct final interviews. Worse, however, is that resume screening is also one of the most invalid and unreliable ways to predict performance. Our review of the research literature suggests three primary reasons for this.

Resumes are Inconsistent

If you have ever screened applicant resumes, you know how frustrating it can be, considering the differences between each in terms of the format chosen, writing skill, and the individual style and type of information included. Many include information that does not relate to the job at all and simply confuses what could be a clear and concise account of past experience.

Of course, this makes it extremely difficult for hiring managers, who are most interested in quickly finding out whether applicants have the basic job-related qualifications required. Unfortunately for applicants, this frustration can result in recruiters and hiring managers spending very little time on each resume, likely overlooking important information, and employing “short cuts” that often unintentionally weed out very capable individuals. For example, automated keyword searching was designed to find specific skills in a sea of inconsistent resumes, and is notorious for overlooking quality applicants who have described their background in a unique way.

Resumes Often Contain False Information

Resumes are basically a subjective self-report on what applicants feel will best “sell” their capabilities. Of course, in trying to persuade hiring managers that they are capable, many applicants exaggerate skills and experiences, leave out undesirable facts, and even outright lie about their qualifications.

In fact, Mike Aamodt, Professor of Psychology at Radford University, conducted a review of 15 studies on the subject and found that an average of 25% of resumes contain false information. This is extremely high, considering the weight that is typically placed on resume data, but should not be surprising - many people will embellish on a resume to ensure their spot at the interview table, knowing that such information is rarely verified.

Resume Information is Invalid

Perhaps the worst reason to rely on resume information alone is that the type of data typically included simply does not relate to performance on the job. When Hunter & Hunter conducted extensive research on the validity of different components of the resume - Experience, Education and Academic Achievement - they found that neither related particularly well to performance.

If we really thought about it, these research results make complete sense. Having ten years of experience in a job does not mean that one performed it very well - we all know senior employees who are not very good at their job, just as we know very talented and eager junior hires who consistently perform beyond our expectations.

The validity results for education are even less impressive - something that Steve Jobs (co-founder, Chairman and CEO of Apple computer...and college drop-out) would certainly agree with!

Reference Checks

Upwards of 72% of SMEs depend on standard reference checking to help determine applicant capabilities. The idea is that feedback from those who have worked with the applicant will help us confirm our opinions and uncover key problems that the individual may have failed to mention in a resume or interview.

Of course, if you’ve conducted traditional reference checks, you’ve likely found that the information gathered is not always helpful, particularly when it comes to finding out why not to hire a person. And the research to date would support that assertion, with most showing that past ratings provided by references generally do not relate to ratings given to employees once hired. Again, our review uncovered key reasons to avoid traditional reference checks, at least for anything beyond confirming basic background information.

References are Subjective

Like the resume, information gathered through reference checks is subjective; opinions provided that may not actually represent true performance or ability.

Yet many hiring managers still spend a considerable amount of time questioning references, who may not even be qualified to provide such information, have the skills to do so, or have known the candidate in enough of a capacity to predict success.

Reference Information has Low Variance

All of us have provided future employers with a list of references and likely none of us has chosen people who would provide negative feedback. To do so would put us at a serious disadvantage compared to other applicants.

The problem, of course, is that hiring managers end up with very positive information on every candidate and little to no information that would help them identify potential problems. It would be the same as trying to choose the best student in the class, when teachers give everyone the same gold star.

Reference checking can be useful if done correctly. Here are a few tips on increasing their effectiveness:

• Make sure that those providing references know that their comments are anonymous.
• Ask open-ended questions that are clearly related to job performance.
• Keep it positive. Instead of asking about weaknesses, ask about ways to help candidates reach their potential.
• Focus on key themes across raters, as opposed to “one-off” comments made by an individual.
• Use references to confirm information gathered by other methods to increase the validity of your entire hiring process.

Unstructured Interviews

The third most frequent method used by SMEs is the interview. I should be clear that I am referring to the commonly-used unstructured interview; more recent advancements in question development and interviewer training can help increase the validity of more structured interviews, although many companies still rely on the former outdated approach.

Research supports the need for improvement in the way SMEs interview potential hires. There are four main reasons why the traditional interview does not relate well to future performance.

Interviews are Not Standardized

Most traditional interviews are really just conversations between a candidate and hiring manager, with very little in the way of standard or consistent questions. This means that candidates’ chances of getting a job can be completely dependant on the interviewer’s mood and type of questions asked on a particular day.

Interviewers are also placed in a difficult situation, particularly when they have to compare candidates and decide who performed best. With different questions asked, and different answers given, it is nearly impossible to tell who is best suited to the job.

Interview Questions are Often Unrelated to the Job

Of the most common questions asked in an unstructured interview, many are unrelated to job performance. For example, asking applicants about their hobbies, or where they see themselves in ten years, typically result in answers that provide little value, and could even open you up to legal attack should unsuccessful candidates question job relevancy.

Individuals can Develop their Interview Skills

The key problem with standard interview techniques, as it relates to skill and practice, is that we are at risk of hiring people who do well when interviewed, despite their true ability on the job. For example, modest people are at a disadvantage in an unstructured interview, as they may not volunteer positive information that could be used to predict success.

More savvy candidates gain training in interviewing techniques, and are able to provide stock answers that are sure to impress an interviewer, regardless of accuracy. This is why many traditional interview questions are ineffective.

Some Interviewers are Biased

Studies by Springbett, and by Tucker and Rowe, demonstrate how quickly interviewers can form impressions, with many saying that they determine whether to hire within the first four to nine minutes of the interview. This is certainly not time enough to base decisions on fact, so we must assume that many are skewed by personal feelings and biases.

Without proper training, interviewers are at risk of making many types of errors in judgement related to certain biases. Many tend to choose candidates who are more like them, despite true ability and qualifications. Or they guide the interview favourably, and misinterpret a candidate’s answers, for those who impressed them early-on. In future, we will be devoting an entire article to the interview, and providing suggestions on how to develop a more valid and structured approach.

 

Clearly, using some of the more traditional approaches to selection is not going to help you identify the best candidates. In the next edition of this article series, we will be discussing more modern techniques that are highly related to success and providing tips on how to integrate them into your current hiring process.