I see how this one interview question is expansive and sets up a line of followup questions.
I don't see why it's a good question. Presume you get an good answer to the question and every followup question. Now tell me this: how much more do you know about how capable the candidate is of doing the job you're hiring for? And, were these questions the fastest, most accurate way of learning that?
I submit: no. "What were the biggest challenges you faced"? How do I assess the answer to that question? How do I compare it with answers from other candidates? How do I track, over time, which challenges correlate to strong performance in the actual role we're hiring for? "Explain your manager's style and whether you liked it"? Come on.
Don't virtually all of these questions ask me to read tea leaves, to get a sense of whether I "like" or "respect" the candidate? That's not the same thing as learning whether the candidate will do a good job.
Oh, and good luck hiring anyone right out of college this way.
I'd also argue that my answer to this question won't actually help you hire me. My biggest accomplishment: I opened the door for [list of people] who then walked through it. It's useless information if you want to hire me - I'm a statistician ("data scientist"), not a guidance counselor.
(Of course, in a real interview I'd have a good answer for what you wanted to hear. I'm just saying that an honest answer would be useless.)
It also biases you against people with many small accomplishments as opposed to one grand one. But people who can handle 20 different small tasks on a schedule are also valuable. Your organization probably needs more of them than me (I'm the one big thing sort of person).
I haven't been using the proposed question but if I was and the candidate told me that his accomplishment is to solve many issues and keep the product bug-free that would be an accomplishment I'd appreciate as well.
I've managed a team and there are those who will find a grand solution to a thorny problem or find a solution to something you didn't think of as a problem and there are others who will plow on to fix countless bugs and make the system better for it. Both are needed, and others as well along the spectrum.
I believe that knowing your strengths and playing to them would be a way to get the best for both sides. Unless of course the interviewee is looking to change course completely.
The only interview questions I've ever found at all reliable--in the sense of providing good answers to your questions about comparison & assessment--are coding problems. As in, here's a toy problem, you have 45 minutes/3 hours to write some code to solve it, on this actual computer with actual Google etc.
That actually tests a set of skills, which puts it well ahead of various interview questions which test nothing of relevance. But it's also a limited set of skills, some of which a recent graduate might reasonably lack but might be reasonably be expected to obtain quickly (if the right kind of smart, committed, a quick study etc).
The problem is I don't know really know how to test for anything but current ability to write reasonable code for a not-too-large problem. The more experience I get interviewing, the more decisions feel arbitrary to me. And I'm pretty skeptical when other people claim to have it figured out.
Oh, and good luck hiring anyone right out of college this way.
I think it is a fantastic question to ask someone out of college. It shows those people who are passionate about something and went beyond the minimal requirements for their degree to achieve it. A passionate student could answer this question in a number of ways; he could point out an open source project that he published on Github that has users, some research that he did with a professor that got published, or even a class project that got him interested in the current position he is applying to. I mean, John Resig would talk about the projects he did as an undergraduate that lead to jQuery[1].
If you dig into the details of a candidate's answers to this question, you can find skills that they might have forgotten to put on their resume. Such as the fact that they know git or Matlab. You can ask them whether they used design patterns in their project and use that as a launching point for a short quiz on design patterns.
You do lose a bit of control of the interview process when you ask an open-ended question like this. A candidate can go off on a tangent about something irrelevant to the job. That is a risk. But it could turn out that you learn more about the candidate from his answers to this question than the puzzle problems that you might have planned to give him.
There might be nothing in the industry that gets under my skin quite as effectively as engineers making hiring decisions based on the perceived "passion" of candidates.
"Passion" doesn't matter.
What matters in a job is effectiveness and competence. Effectiveness gets things done. Competence ensures that what's getting done isn't going to backfire and create more work down the line.
In our common usage, "passion" describes the coder who stays until 9:00PM on a Friday hacking on things. But that property doesn't tell a team anything about whether that coder is the most effective use of headcount dollars. Passionate coders stay late working on compilers for obscure languages, or rewriting ugly but working code into trendy new languages and frameworks. Even if they're building important capabilities for a team's core product, they're doing it in a way that is basically impossible to schedule, on timelines that are impossible to reason about, because "passion" is idiosyncratic.
Passion (about what? quality? new technology? learning? building things? these are all different things) doesn't hurt anything. It's not a bad thing. It might even be a good thing. But it is not a crucial attribute of a new hire on most teams. If you need passion, consider whether you're hiring cofounders, not teammates.
This is to say nothing of my total lack of faith in the ability of engineers to assess "passion". For any given engineer, candidates that happen to do things in their spare time that the interviewing engineer happens to enjoy or appreciate are "passionate". Meanwhile, a supremely competent professional who, given a spec, can reliably estimate the time to execute that spec and then actually nail that schedule, that person is a 9-5er, because they don't have strong opinions about Node.js.
•Shudder•.
Almost every idea that engineers have about hiring is broken.
It's this line -- "given a spec, can reliably estimate the time to execute that spec" -- that, I would guess, separates the types of jobs I've experienced from yours. I've never been given a spec that was worth the wiki page it was written on. In most of the jobs I've experienced, "effectiveness" (get done what you're given) and "competence" (don't cause extra work for others) are barely a minimum bar to success; to be successful, colleagues also need "insight" (look past the documented requirement to the reasons behind it) and "creativity" (design a solution that solves the documented problem and three related undocumented ones).
You're right, "passion" isn't one of the things to hire for. At best, in my experience, it's highly correlated with the breadth of experience that fuels "creativity" and "effectiveness" - at least, in the sense that it causes people to be persistent against hard and unclear problems. It's fair to ask whether this should be required for teammates or just co-founders -- but even in my most enterprise-y gigs, I've rarely seen a team that could succeed without people with passion helping drive the work.
Passion is critical. I wouldn't hire someone if I didn't think they were passionate about building good products. Other passions can tell me something interesting about them, but that's the one that counts since that's what I would be hiring them to do.
Maturity is also critical. And passion and maturity are not at all mutually exclusive.
If there are misconceptions about passion floating around the memespace, then you are certainly right to challenge them, but let's not throw the baby out with the bathwater.
I once watched management totally destroy a developer's passion.
As bad as management's actions were for me, the absolute worst thing was watching the change in this guy. Before, he would constantly seek out every way to improve the customer experience and go above and beyond. He turned into a guy who literally said "I don't give a shit."
Almost every person I've ever worked and every job with has been passionate about doing their best, unless actively stopped from doing it. The one or two who didn't could not have been filtered out by any kind of interview question because they knew how to bluff their way through the process. But you can tell after working with them for a few months that they just try to do the minimum necessary to not get fired.
An attempt to recruit for passion is a bad sign. You can't recruit for good morale, either.[1] Either your company is a nice place to work or it isn't. Either your company will bring out the passion that almost all developers have or it will destroy it.
[1] Yes, there are assholes who destroy morale, just like there are total clock-punchers. Sometimes you can tell by having a normal human conversation, and a normal human conversation should of course be part of any interview.
If you are are trying to find the liars, assholes, frauds, and professional slackers, you need to realize that
1) they have you vastly outmatched in this contest unless maybe you are bringing in psychologists trained in this,
2) there is a social cost from treating your prospective candidates as if they are one of these things.
This is a good question. I had to think about it a bit -- at the risk of not posting a reply in time for you to see it.
I think it's because building great products isn't just a matter of execution. It's also, to some extent, an art.
I have trouble imagining someone being effective and competent with absolutely no passion. I will grant, though, that there are competent people who are more passionate, and those who are less so. I think a successful organization could have a blend of people across the passion scale, but I don't think one could build a successful product business without at least a few people who are passionate about what they're doing. Maybe in other kinds of businesses you could; I don't know.
If you can, in your hiring process, get a decent prediction of effectiveness and competence, and stipulating that passion does not connote effectiveness and competence, is it smart to screen for passion?
Yes, I think it is. Given two candidates of equal competence, I'd naturally prefer the one with the greater passion for the particular kind of work the job would entail. Does that really seem strange?
Of course it gets trickier when the more competent one seems the less passionate. Then it's a question of how much to weight the two dimensions (along with whatever others one is considering). I have no magic formula here; I think it depends on the circumstances.
There seems to be a subtext here that work sample tests give no indication of passion. I don't think that's true; when I've given people programming exercises, I think the craftsmanship of the code (is it well-modularized, well-documented, nicely indented, etc.) is indicative of their passion. Competence, to me, is more about whether the code works.
So yes, I'm using craftsmanship as a proxy for passion, so I suppose next you'll ask, why not just go for craftsmanship? The point we seem to be coming to is that passion is difficult or impossible to measure directly, so why not stick to qualities that are more measurable?
I don't think everything important is easy to measure.
I'm just feeling we are using the words in different ways. I'm not a native English speaker but for me effectiveness is a term that includes passion. You can be competent in your work but effective at deflecting it (it's painful to work with such people), I believe that if you are passionate about doing the work it will also mean your effectiveness is in the right direction.
Because the kid in the basement down the street you're competing with -- the one you don't even know exists until it's way too late -- is effective, competent, and passionate.
Passion is continuing to learn, because you realize that what you learned in school is not the end, but the beginning. Passion is continuing to learn, and having an opinion about "node.js or some other technologies" because using the right tool for the right job is important. Passion is being given a spec, and can not only reliably execute it, but can also offer ways to improve that spec, and not just be a coder without an opinion.
Thus, the only way to be effective and competent is to have a passion for what you do. Otherwise, if you limit yourself to 9 to 5, you'll be treated as such. And it works like that in many industries, and in those industries, those with a passion will routinely do better, whereas those that don't, will stick to the 9 to 5 places.
> Almost every idea that engineers have about hiring is broken.
Almost every idea anyone has about interviewing is wrong.
Interviews are a terrible way to recruit. It's amazing to me that STEM companies with money and scientists still use interviewing in recruitment processes and haven't found something else.
There is the "on the job interview" where a candidate is invited to do the actual job for a day or half day. For obvious reasons this method tends to be used for minimum wage positions, or for places where you're recruiting people who you know might interview poorly. But it seems to be a great way to recruit people.
But I'm not just talking about blind, unfocused and overly enthusiastic passion. I'm talking about the passion that drives people to go beyond the call of duty, not because it is part of the job but because it is what they like to do. Passion got me the job I have now. Although I was also more than merely competent, the passion drove me to achieve that competence in the first place.
Even if they're building important capabilities for a team's core product, they're doing it in a way that is basically impossible to schedule, on timelines that are impossible to reason about, because "passion" is idiosyncratic.
Yeah, passion is a bit of an x-factor, isn't it? By itself, it is meaningless. You may get something unexpectedly great or unexpectedly bad. But, using myself as an example again, I was able to get a project at work done well ahead of schedule because I had already been working on something related in my spare time.
Passion (about what? quality? new technology? learning? building things? these are all different things) doesn't hurt anything. It's not a bad thing. It might even be a good thing. But it is not a crucial attribute of a new hire on most teams. If you need passion, consider whether you're hiring cofounders, not teammates.
Inasmuch as someone is proud of a project they did and can talk enthusiastically about it, and that project is related to work currently going on at the company they are applying to, I think a passionate candidate can be better long term than another candidate that is initially more competent and more effective. If experience is the main reason that candidate A is more competent and effective than candidate B, then candidate B may eventually surpass candidate A as a better employee simply because passion drives candidate B to keep honing his skills in his free time. I think this is especially true for college graduates or other junior candidates who may not have anything else to go by but their projects.
I'd agree with the parent...there are jobs where passion is key, like a tech evangelist for instance, or a product leader, a salesman, or a founder. But for an engineer, while of course creativity and trying to get better at the trade are important, so are not getting burned out, going through periods of shitty tasks keeping a professional attitude, make pragmatic choices when needed, without getting blinded by coolness or just curiosity.
These skills can come with experience, but can also be personal traits. An employee that doesn't code the week-end but focuses on his job while on site and takes pride in being professional and reliable is a solid asset for any company. In the long term he might be a corner stone of your team, balancing the ones that are half dead because they didn't sleep at night to work on the projects they really invest themselves into.
Yeah, I can see how too much passion can burn you out. I'm borderline obsessive about some of the things I do in my spare time. The thing is that I really like doing them but taking a break from too much intellectual stimulation is probably a good thing. I'm probably not old enough to know but it may be more necessary for older workers. Probably also the reason I can handle it relatively well is because I don't have a wife and/or kids. Again, all the more reason to take advantage of passion when you find it in the younger folks!
Because some people cap at "I managed to code the most lines of code in June, 2006 among my seven colleagues, even though I was just hired on the tenth of the month and didn't know the code yet. That was quite a month."
whereas others cap at, "well, I did manage to build the world's first and largest bitcoin currency exchange and reach volumes of about a million a day before we had to shut down."
when you find out they did it by themselves, that is a huge difference. like two orders of magnitude FTE's.
it's like, if some guy spearheaded and came up with the whole idea behind the new Mac Pro, in an era of stagnant desktops, the whole mechanics and central idea (with the heat core). This question lets you know this.
So this one interview question allows you to make the decision between hiring someone who has only worked in the profession for a month and hiring someone who wrote a whole bitcoin exchange. Faint praise.
The big issue is, what does this style of interviewing do for you with a real-world candidate selection challenge --- say, 3 developers, all similar on paper?
There is no reason the best candidate necessarily has the best story. The story of your "best accomplishment" has as much to do with random chance as it does with capability. This question demands that an interview somehow de-confound the luck element and the skill element.
And to what end? Even if you got a clean reading on this question, somehow equalized between the candidates, I ask again: do you really know a lot more about how the candidate will actually perform on the job? Because that is almost the only question that matters in candidate selection.
You misread my example, no beginner writes a ton of LOC, but yes. I don't think it's faint praise - some hiring managers would manage to find a reason to not hire the second guy, when the first one had been "employee of the month" already and that's good enough for them.
Seems to me a person with a single 1000-unit achievement would look better than a person with 1000 10-unit achievements if you ask about their single biggest achievement.
Maybe I coded an iphone app. Meanwhile you coded an iphone app, and fixed bugs in a huge legacy codebase, and did hiring and code reviews and mentored new employees, and contributed to open source projects, and introduced your company to source control and automated testing, and masterminded a trivial code change that made the company a million dollars a year, and you spend your free time running a programming club at a local orphanage.
If we're competing for a job and we're asked about our single biggest achievement, we'll both say iphone app. An employer who wants to accurately gauge who is the better employee should probably dig deeper.
Really? You wouldn't say, 'Saved the company a million dollars a year' or 'Introduced source control and automated testing into the company' ? To my mind those are much more impressive achievements than an iphone app (at least assuming it's a typical CRUD app.)
Say I saved a million dollars a year by adjusting a configuration setting, no coding required. That might go over well with a business type, but if the interviewer is all about the programming, it doesn't demonstrate technical mastery. There will be no scope for talking about the technical challenges, the languages and databases I used, what I learned from the experience, or how much I enjoyed it and grew as a person and a professional.
Say I introduced source control and automated testing at a medium sized company. Again, the technical work is limited - you can install Jenkins with apt-get, the real challenges are political and educational, changing 'official' workflows and 'supported' software and convincing people to sink time into using the tools for long enough that they can see the benefits which are not immediately apparent. If I'm being interviewed by a young guy at a startup they might think they have no politics so my skills will be useless - or worse that I'll introduce politics. Or it just won't seem like much of an achievement, after all the six of us at this startup switched from bzr to git in an afternoon.
Needless to say, if the original job description says "wanted: someone to teach us source control and automated testing" that would be a different matter :)
I can also come up with a lot of "technical jargon words" and then just throw out at the interviewers. I can bullshit so much to cover up the mess I created. I can be really stupid and took 3 months to figure out the problem I was facing.
Though this question can be a testimony to see whether the candidate is articulate at describing things. People who have trouble to describe problems to another person is usually not a good co-worker to have (no matter how competent he or she is). You will just waste so much time going to put his or her words into complete picture. or the new co-worker will work on his own problem the whole day and no team work.
What you need going into this interview is a solid understanding of your corporate culture -- what behaviors you value. Maybe you work for a bank, and the most important things for you are integrity and conservatism. Or, maybe you work for an early stage start-up, and calculated risk-taking and team spirit are most important.
In either case, as you walk through these questions, you ask about the how's and why's of the person's behavior. Doing so is the best way to figure out if the person is going to act in ways that you would want and expect them to. Again, the significance of the biggest thing itself is small, it's how and why they achieved it, gleaned through the 20-odd follow-up questions, that will get you to the answer.
You mention college hiring -- yes, that's a different beast. A lot of companies have success with paid internships the summer before graduation. You can also ask some of these behavioral questions, but unfortunately our educational system doesn't include enough collaborative, project-based work to get a good baseline. Nonetheless, a college student who has done significant summer project work, or who has worked on an extracurricular project, should have some data to extract.
No, asking questions in a face-to-face interview is not the best way to figure out if a person is going to do what you want them to. It is not a very effective way of doing that at all. To be meaningful at all, it requires an interviewer with many of the same skills as a psychologist, and a candidate who can represent themselves effectively in an extraordinarily hostile situation. Even if you have both, you're still not even in the vicinity of the power of a work-sample test.
Job interviews suck, and I feel like people who have been hiring in this industry for a long time recognize at some level that they're essentially random functions, made workable only by dev teams willingness to put up with employee churn and extreme inefficiency.
How long does a work-sample test take? I'm typically hiring people to work on on-going projects over a period of months or years; the fact that candidate A does better on a 2-hour or 2-day task than candidate B may not give me that much information about things like how effective they will be at sharing their own knowledge with the rest of the team (or taking guidance from more senior members of the team), developing a good working rapport with their QA teammates so they can get to the root of any problems as efficiently as possible, convincing the sysadmins that the candidate's proposed deployment tool is the right way to do it (or gracefully agreeing to use the sysadmins' tool of choice, either because it's actually better, or just because the benefit of the sysadmins' familiarity with it outweighs any technical benefits of the other tool), etc. Unless I'm hiring someone to sit off in a corner developing their own solo project, I'm not convinced that "work sample" tests are that informative.
I don't agree that this question is a silver bullet, but I think it's valuable to see what sort of achievment the candidate considers significant, to ask follow-up questions about how they handled problems along the way to that achievment, etc.
The question helps reduce the power assymetry of the interview, by letting the candidate choose something they're interested in talking about and presumably familiar with. I find that people seem more relaxed when they get into the flow of describing something they're interested in, which lets me explore various aspects of their working style with as little stress to them as possible.
Job interviews do suck, but the alternatives I've seen proposed suck at least as bad, though in different ways.
In our case? Call it ~6 hours, over the course of several (potentially discontiguous) days, offsite and at the candidate's leisure.
Work sample testing has been extremely powerful for us. In particular: it has enabled us to screen candidates for aptitude instead of experience, and the result has been a series of hires with no prior experience in our (very specialized) field who have completely blown us the fuck away, like the former line-of-biz insurance company .NET guy who, a month or so in, looked at Rails for the first time and found a serious security hole (which now has a CVE), or the other line-of-biz .NET guy who now runs our crypto review board and co-wrote our crypto challenges, or the math major we hired out of college who ripped through the MSP430 CTF in a couple days (and also has a Rails CVE).
We picked up a guy who had done a couple months of Android development after college, and then moved to what I think (from his resume) was a devops job --- I never asked him in the whole course of our hiring process what that job was about, so I don't know. Six months later, he implemented a CRI elliptic curve DSA exploit that involved BKZ lattice reduction and an FFT filter step. The lattice reduction steps on that attack took something like SIX HOURS to run, and when they finished, you still had to get the FFT step working to figure out if you got the BKZ step right.† Months later and I'm blown away just thinking about how he could have gotten that working. In case you're wondering, he kicks ass on the more boring parts of our work too.
Some other things to know about that guy:
* No prior crypto experience
* No prior software security experience
* Less than 2.5 years dev experience across both jobs
* From flyover country
* Physically uncomfortable during the interview (something noted by interviewers on his feedback)
How compelling do you think his "greatest achievement so far" story would have been? We weren't dumb enough to ask, so we'll have to guess.
Before we did work-sample testing, overcommunication about hiring process, demoted the importance of phone screens, and standardized face-to-face testing, we never would have hired these people. Any process that filters outs Seans, Alexes, Peters, and Ryans over things like "passion" or "what is your greatest achievement" is a stupid fucking process. But there it is, enshrined as the default process for our whole industry. And here's a thread full of people arguing on its behalf.
Yes, I think someone is going to make a lot of money correcting that market inefficiency.
† If you're wondering, number of other people on our team who could have helped him debug lattice reduction C++ code: zero.
If this comment shows up hereafter on every "developers need passion" and hiring thread, I'll up vote it. every. single. time. Alternately, a blog post expanding on the thesis, if we could draft you to write it...
Wait -- there are two questions here. One is whether developers need passion, and the other is whether an interview process is a good way to tell whether they have it (and the other qualities required -- I haven't seen anyone say that passion is sufficient). I think most of us are speaking to the former -- I certainly have been.
I agree that work sample tests can be quite valuable.
Why would you expect anything rigorous in such a nebulous business as predicting someone's performance in the environment he's never been doing things that he's probably never done before (even if he's done similar things)? I don't think you can have any rigor here. Do you have a proposed rigorous solution?
Yes, I have written long and specific comments on alternative approaches on HN before and won't bore you with a recap, except to note that I sketched out what I meant by "rigor" in the comment rooting this thread.
I understand what you mean by rigor but I'm not sure that your quest for rigor in that sense would yield better results. I could be very wrong, of course, my experience is nothing but a relatively small list of anecdotes. But if you indeed have a rigorous way of assessing candidates, I certainly look forward for you to creating the best recruiting agency ever, becoming a billionaire and transforming this field forever (not being sarcastic, btw, I'd like so see some innovation in this field, I'm just not sure how "rigorous" can work here, but if you do - I hope you are right and I am wrong).
So you ding candidates for lacking an ability (carrying a time-bounded conversation with a stranger who is judging them for a new job) that has nothing whatsoever to do with what they'll actually be doing on the job, and this is something you're happy to tell us?
Job interviews are hostile. In fact, they're one of the most hostile experiences most of us have to deal with. I watch candidates in face-to-face interviews very carefully now, because this is a problem I am studying, and when you stop and consider candidates not as gamepieces but instead as human beings, you'll be amazed at what you notice. For instance, a majority of the candidates I've sat down with in interviews have, subtly but noticeably, been physically shaking.
Job interviews are like a game of Jeopardy! where one of the players is being filmed and broadcast on live television and the other player is playing from their couch and has been given an answer book, and the poor televised shmoe is judged on the same curve as the person on the couch.
So I think you are reading way to much into the comment. It was more meant to probe an expert for his approach to sussing out viable teammates or employees.
If I as someone interviewing for a QA role ask them a question such as 'Tell me about the QA work you are most proud of.' And they spend the next 20 minutes not talking about anything QA related it's a red flag. And an important data point to me when doing a retro with my peers. Perhaps that recent scenario is a reflection of the folks my company is able to attract in this area.
Conversely, if I am interviewing with you and I ask you, what's the most interesting thing your team has done and how did you help them accomplish it.' And you go into great detail about a deep area in your field that's part of the product your team works on its useful and important information when I need to make a decision. However if you are only interested and talking about me, my technical (or lack of - I know very little about security) understanding and you brush off conversational attempts to gain information that will help me judge the book by its cover it's usefully info when I need to make a decision.
I am not sure why your tone was so hostile? I did not mean to imply anything by my question. I am merely looking for other ways that people have found works for them.
Assuming two candidates are both qualified to perform a job, and one of them gels and converses with team (aka gets along with), then that person should get the job. The best teams get the best work done, and good teams have amazing communication - also assuming that everyone on said team can actually do the work without bring the others down. A good teammate is a multiplier on total team productivity and efficiency - like in sports and the military. Everyone wishes it wasn't the case, but talent alone doesn't cut it.
First, you're assuming that the interview process reliably distinguishes between "qualified" and "easy to talk to", but in practice not only do engineering interviews not do that, but in confounding the two they also drastically over-weigh the "easy to talk to" attribute.
Second, you're assuming that the way a candidate interacts with your team in an interview is an accurate representation of how they'll gel with the team down the road. Some of the shyest people I've interviewed have gone on to be the ones I actually end up talking to all day. Part of that is because interviews are hostile and make people shy, and part of it is that different people take different amounts of time to open up to strangers.
Finally, you're describing the "gut check" interview process used by almost every engineering team since the 1990s. We have evidence for how well that process works. Answer: not well.
As an aside: the idea of hiring candidates based on how well team members "get along" with them is repellant to me, as is "culture fit", since they're basically ways of making sure you have a team full of people who listen to roughly the same music, drink roughly the same beer, play roughly the same X360 games, and like roughly the same technology stacks. But that's an emotional argument; the real argument is that it doesn't work.
The ability for people to gel doesn't guarantee any of the following: listening to the same music, drinking roughly the same beer, playing the same games, and liking the same technology stacks. What I am arguing for is the part after someone's ability to accomplish the work and be creative has been shown. I think you underestimate how many people actually have the smarts and creative ability to do just about all software engineering tasks.
I think I do the opposite: I think I value candidate potential more than you, because you propose screening based on how cordial or fluent a single interview with their prospective team is, and I don't buy for a second that you get a realistic picture of how well someone will communicate or gel with that team in the course of a single adversarial interview.
How many tech interviews are one session long anymore (especially for the good ones)? Like I said in my post above this one, I'm arguing for the point after candidate potential has been acknowledged. If you know a candidate can do his/her potential job well - then I think good team fit is the obvious next step. If nobody wants to work with a potential candidate - then it will clearly be a detriment to a team (short term and most likely long term as well).
I think the problem is that everyone on HN thinks things are binary, when practically every scenario in this universe is not. You can't just eschew potential team fit because you want a rigorous hiring process that depends only on candidate qualification. The opposite holds true as well. Here are the two things you need for an awesome team:
I don't know how you get the same music and beer from that, but preferring the same technology stacks may have a value in the team. Less time spent arguing religion = more time spent doing actual work.
>>> the real argument is that it doesn't work.
It's not an argument, it's a statement. It would become argument if it would be supported with something that is not of equal weight with the contrary claim "no, it actually works quite well". Everybody has their anecdotes, do you have more than that?
Let me understand you completely before I respond.
Is your argument here that conventional engineering job interviews...
(the sort where candidates interview with a series of engineers from the team all of whom ask different sets of questions and seek to gauge from their answers previous accomplishments, working style, technical competence, and team compatibility)
... are generally an effective way for teams to screen candidates?
Yes, it generally is. In my experience this is proven by the fact that close to 100% of working engineers are hired that way, and tech industry didn't collapse yet. But if you have better way, that is provably more efficient - I'd be very interested to hear about it (of course, if you're interested to tell me).
Ask any developer if they've ever had the experience of hiring someone who blew them away in an interview and turned out to be either completely ineffective or, worse, slapdash and incompetent on the job. Then, ask how long that poor-fit developer managed to remain on the job before being shown the door.
Or, ask an engineer how many times over the course of their career they've worked with "worthless" teammates, or people who should never have been hired. I've worked across a decent slice of the industry these past 19 years, and from what I can tell, this story is universal.
Another strong piece of evidence on my side is software quality, or the pervasive lack thereof.
Another piece of evidence is fizzbuzz.
Another piece of evidence is the (harmful) trend of demanding that candidates work on a contract-to-permanent basis as part of the recruiting process; what is this other than an attempt at work-sample testing that also happens to exploit candidates and repel the best of them?
I agree with you that close to 100% of working engineers are hired this way. But that doesn't prove anything; it's just as likely, perhaps more likely given the evidence, that what it shows is that development teams can ship marketable software despite terrible inefficiencies.
If you think that conventional hiring does a good job of selecting effective and competent programmers to the median ISV (and note here I'm stipulating ISVs, the best-case scenario for your argument, not line-of-business software teams at F500s), fine. I don't feel like I know a lot of smart engineering managers who are comfortable with the predictive power of their interview systems.
It's interesting that you bring up sports and the military as examples. I suspect that neither of those emphasize team interviews to determine cultural fit to anywhere near the extent that the tech industry does.
I posted a reply elsewhere on some similar points, but I'll go into more detail here.
What I like to ask is what project or achievement people are most proud of. That forces them to use their own subjective judgment rather than attempt to cast about for some sort of objective definition of "significant".
There are many reasons why I love this question.
First off, it puts things squarely into the realm of the real instead of the abstract. It can tell you a lot about whether or not you're dealing with someone who gets things done or just someone who "does work".
From there it's a quick jumping off point into the specific skills and competencies of the candidate, viewed through the lens of productive work. It's a way to see what skills people have in sort of a sideways way, because you see what skills they actually use, rather than the skills they tell you they have.
Perhaps most importantly, it tells you about a person's priorities and values. It tells you the sort of things they think are important. Did they do something that actually delivered value to anyone? Did they do something that required overcoming obstacles or did they just cruise along and build something "cool"? Or something that was technologically sophisticated but useless in practical terms? Do they value titles and money over real accomplishments where something of lasting importance or value is built and delivered to users?
It also gives an opportunity to explore a lot of issues that are hard to suss out otherwise. For example, does the candidate have initiative or do they just follow along? Do they strive to improve the way things work or just tolerate the status quo? That can be evidenced if the project they describe was something they took on of their own accord or was handed to them, whether it was an internal improvement project, and whether they had to fight against the establishment to get things done.
So there you go. Values, skillset, competencies, initiative, and ability to execute. You won't necessarily get all of that out of a candidate every time you ask such a question, but there's no such thing as a perfect interviewing technique.
For someone straight out of college or without much experience of course you have to modify how you interview them, but that's always going to be true, and sometimes this same question can still be valuable, just not in exactly the same way.
P.S. Also, just having a good set of questions or a good process to go through is not enough to make someone a good interviewer. Interviewing is hard, easily one of the hardest things in the profession, and only a small subset of folks who do it are actually good at it. I don't know exactly how to make a bad interviewer into a good one, that's a subject that could fill volumes and volumes. But I think this particular kind of question is, on par, much more helpful than not and a step in the right direction.
I don't see why it's a good question. Presume you get an good answer to the question and every followup question. Now tell me this: how much more do you know about how capable the candidate is of doing the job you're hiring for? And, were these questions the fastest, most accurate way of learning that?
I submit: no. "What were the biggest challenges you faced"? How do I assess the answer to that question? How do I compare it with answers from other candidates? How do I track, over time, which challenges correlate to strong performance in the actual role we're hiring for? "Explain your manager's style and whether you liked it"? Come on.
Don't virtually all of these questions ask me to read tea leaves, to get a sense of whether I "like" or "respect" the candidate? That's not the same thing as learning whether the candidate will do a good job.
Oh, and good luck hiring anyone right out of college this way.