Thursday, February 28, 2013

Do unconventional interview questions work?

Whether today's ever-more-polymorphous interviews succeed in identifying better employees is an unanswered question. The use of peculiar questions and arbitrary tests may seem to go against one of the few rock-solid percepts of today's human resources profession. This holds that any method of selecting job candidates should be as closely related to the work as possible. Most HR people place the most faith in work sampling, where the candidate is asked to perform or simulate work similar to which he'd be doing if hired. Statistical studies of work sampling (a famous one was done by AT&T from 1956 to 1965) showed impressive predictive ability.

The usual justification for "creative thinking" riddles and personality assessments is that they test broad, general abilities, not tied to a specific set of skills. Whether they do that is hard to say. What's certain is that "pet" questions take on a talismanic quality for some interviewers. Just as athletes don't change their shirt during a wining streak, interviewers keep asking the same questions because of a few remembered instances where it supposedly "worked". That fact that many of the most admired, innovative companies use such interview questions seems to speak for itself ("You can't argue with success").

It is far from clear that either season holds water. The human resources profession is full of customary practices of no demonstrable value. The psychologies Daniel Kahneman tells the tale of a test once used by the Israeli military to identify candidates for officer training. A group of either recruits, stripped of insignia, was instructed to carry a telephone pole over a wall without letting it touch the wall or the ground. The point was to observe who tool charge (the "natural leaders") and who fell meekly into place behind them (the "followers"). "But the trouble was that, in fact, we could not tell", Kahneman said. "Every month or so we had a 'statistics day', during which we would get feedback from the officer-training school, indicating the accuracy of our ratings of candidates' potential. The story was always the same: our ability to predict performance at the school was negligible. But the next day, there would be another batch of candidates to be taken to the obstacle field, where we would face them with the wall and see their true natures revealed."

Similar tactics are alive and well throughout corporate America. In today's overheated job market, a common test is to seat a group of candidates for the same job around a conference table for a "group discussion". They know that only one will get the job. The discussion becomes a little reality show, with the recruiter quietly noting who takes charge. It's doubtful that it works any better than the Israeli army test did.

Proving that a hiring technique works - or that it doesn't work - is a complex exercise in statistics. Were once to demand that a hiring criterion be 100 percent reliable, employers would have to hand out jobs at random. There aren't any 100 percent reliable criteria - not work history, not grades, not anything. Hiring is always a game of chance. Many job seekers complain that some talented people do poorly on today's unconventional interview questions - ergo no one should use them in deciding whom to hire. This isn't a compelling argument for the reason given above. But psychological studies indicate that people are apt to view almost any criterion as "unfair" when it's used to decide who's hired or promoted. The sense of unfairness is greater when the criterion is unfamiliar. A traditional job interview is a conversation. The job offer or rejection comes days or weeks later, affording a certain emotional distance. Creative-thinking questions often bring the rejection right into the interview, right in your face. If you fail, you generally know you've failed. That feels worse than a rejection days later. Admittedly, this attitude may not make sense, but when have emotions ever had to make sense?

William Poundstone. (2012). Punked and Outweirded. In: Are you smart enough to work at Google?. New York: Little, Brown and Company. p48-50.

1 comment:

  1. I think unconventional interview questions work just as well as an unconventional resume does - that is, they don't work at all. But to each their own, and if it works for you, it works for you!