Do unconventional interview questions work?

Whether today's ever-more-polymorphous interviews succeed in identifying better employees is an unanswered question. The use of peculiar questions and arbitrary tests may seem to go against one of the few rock-solid percepts of today's human resources profession. This holds that any method of selecting job candidates should be as closely related to the work as possible. Most HR people place the most faith in work sampling, where the candidate is asked to perform or simulate work similar to which he'd be doing if hired. Statistical studies of work sampling (a famous one was done by AT&T from 1956 to 1965) showed impressive predictive ability.

The usual justification for "creative thinking" riddles and personality assessments is that they test broad, general abilities, not tied to a specific set of skills. Whether they do that is hard to say. What's certain is that "pet" questions take on a talismanic quality for some interviewers. Just as athletes don't change their shirt during a wining streak, interviewers keep asking the same questions because of a few remembered instances where it supposedly "worked". That fact that many of the most admired, innovative companies use such interview questions seems to speak for itself ("You can't argue with success").

It is far from clear that either season holds water. The human resources profession is full of customary practices of no demonstrable value. The psychologies Daniel Kahneman tells the tale of a test once used by the Israeli military to identify candidates for officer training. A group of either recruits, stripped of insignia, was instructed to carry a telephone pole over a wall without letting it touch the wall or the ground. The point was to observe who tool charge (the "natural leaders") and who fell meekly into place behind them (the "followers"). "But the trouble was that, in fact, we could not tell", Kahneman said. "Every month or so we had a 'statistics day', during which we would get feedback from the officer-training school, indicating the accuracy of our ratings of candidates' potential. The story was always the same: our ability to predict performance at the school was negligible. But the next day, there would be another batch of candidates to be taken to the obstacle field, where we would face them with the wall and see their true natures revealed."

Similar tactics are alive and well throughout corporate America. In today's overheated job market, a common test is to seat a group of candidates for the same job around a conference table for a "group discussion". They know that only one will get the job. The discussion becomes a little reality show, with the recruiter quietly noting who takes charge. It's doubtful that it works any better than the Israeli army test did.

Proving that a hiring technique works - or that it doesn't work - is a complex exercise in statistics. Were once to demand that a hiring criterion be 100 percent reliable, employers would have to hand out jobs at random. There aren't any 100 percent reliable criteria - not work history, not grades, not anything. Hiring is always a game of chance. Many job seekers complain that some talented people do poorly on today's unconventional interview questions - ergo no one should use them in deciding whom to hire. This isn't a compelling argument for the reason given above. But psychological studies indicate that people are apt to view almost any criterion as "unfair" when it's used to decide who's hired or promoted. The sense of unfairness is greater when the criterion is unfamiliar. A traditional job interview is a conversation. The job offer or rejection comes days or weeks later, affording a certain emotional distance. Creative-thinking questions often bring the rejection right into the interview, right in your face. If you fail, you generally know you've failed. That feels worse than a rejection days later. Admittedly, this attitude may not make sense, but when have emotions ever had to make sense?

---

William Poundstone. (2012). Punked and Outweirded. In: Are you smart enough to work at Google?. New York: Little, Brown and Company. p48-50.

Beat 2013 like a boss

Annually, at this time of a year, we gather together looking back at the challenging year that just passed by and looking forward to a successful year ahead.

Changes are a crucial part of life. Hate it or love it, no one can deny that variation in life constantly face us head on, define us and make us who we are. And 2012 was such a milestone, full of changes. The world economy was still trying to keep its feet on the ground. Our belief was shaken. We have never been through that much up and down in our history.

Summarizing the full of event 2012 is a non-trivial task. I am afraid that the meaning of events that we have been through exceed the expression power of words and only the very people who had been there, done that can understand. In my best attempt, I would like to retrospect what happened to us and the difficulties that we faced and gear up for the coming year.


That web team was restructured into 3 functional teams was probably the most significant course of events. That was to reflect the fact that as the company grew up, we were on the edge of having to trade the dynamic nature for stability. The only way to keep us flexible yet manageable was to split up based on our field of expertise. Smaller teams allowed us to move faster, specialize in a few things that mattered and keep shipping.

If 2011 was a year of inception, of putting thing together, 2012 was the time we took bold moves to catch up with the world. We had chance to observe a spectacular raise of JavaScript (Backbone in particular) and Erlang in our work. We were also taking steps to master our tool kit. There had been more pull requests for Warp last year that ever. Finally good use of CSS framework and preprocessor enabled us to share the burden on designers' shoulders.

The fruit of that effort was reflected on the projects we took. In 2011, BnB was the only star project where multiple platforms were integrated for a world-class solution. In 2012, we were tackling complicated projects requiring multiple subsystems or possessing sophisticated business logic with confidence. Since Brendon left, we have been living without an official CTO, and the world didn't end yet.


Web continued to be the major revenue stream of the company. The number of technologies that we used, processes that we put into practice and projects that we shipped were steadily raising. However, in order to achieve the growth, we have made a number of decisions that eventually ironically prevented us from growing. Many of these decisions were probably right at the time, and on their own merit would not have caused us trouble. But in this case, the sum is much greater and more damaging than single pieces.

For example the decision to split up the team solved the problem of balancing flexibility and stability. At that time we overlooked the need of cross-team communication and cross-project overseeing. This reduced our ability to allocate resources to mission-critical projects. In general, lack of internal corporation slowed us down.

In order to retain our shipping pace, quality was sometimes traded for speed. The amount of technical debts stacked up, evolved into GigaDB and the likes, and gave us much headache.

And in order to keep the revenue stream that supported our working environment, we failed to allocate sufficient resource in PM. And even when people had the ball, the initiatives to take up PM works, every data about project and customer was jammed up in one single bottle neck.


However. Despite all those troubles and an unstable macro economy of 2012, we managed to grow so much horizontally and vertically. It doesn't matter which team you are in, developer, designer or administrator, applause yourself, you deserve that.


The end of lunar new year is drawing near and Tet can be felt in every breath. Yet in the last few days, a news has spread among us, shock us and demonstrate that event giants can collapse. To prepare for 2013 whose challenges are expected to be even more ferocious, we need to learn from last year, make up resolution for coming months and most importantly, better ourselves.

As a human being, try to fulfill your potential. Say "fuck you" to the status quo. Learn another language, explore another text editor, write a blog. Don't limit yourselves on what you can learn and want to do.

As a staff, we want to continue build up this great place with you. Lets put extra emphasize on code quality, make fewer GigaDB or Solar. Let us know you are growing and allow us to learn from your growth via TechTalk and HackDay. And finally better our ability to run business, from communication to estimation and even PM.

I wish you all a successful year ahead.