I've never understood the use of puzzles and tests for hiring. I'm notoriously bad at them, but I consider myself to be a very solid programmer, with 13 years professional web development experience, and a major open source project under my belt.
I think one of the reasons I don't do well on these is because my brain doesn't seem to work well for hypotheticals. I need real world use cases or else it's exceedingly difficult for me to work out a solution for a problem that is too abstract.
Maybe I'm unique amongst other programmers with my level of experience. It's possible that other programmers love doing puzzles like this, but I tend to get discouraged if I'm looking at a company, and they use tests and puzzles to gauge my capabilities.
Especially (and I wouldn't accuse Facebook of this, but it happens much too often), if those tests and puzzles are substitutes for looking at the massive amounts of code I've written.
The Facebook puzzles are pretty easy. That being said, I hate puzzles too. I think it's pretty degrading to ask a well-established researcher/engineer to do that kind of stuff. On interviews I typically ask people to give a talk about their previous projects in detail. A lot of nuances arise during these talks that allow you to quite easily judge the level of the candidate.
Why is it degrading? I would think it would be perfectly acceptable for companies to ensure that their new hires (experienced or not) pass their basic heuristics. This comes in the form of tests, resume checks, social compatibility, etc.
(I ask this from the perspective of a young, inexperienced developer.. so excuse the thoughtlessness!)
I don't know... I would personally feel a bit weird asking, for example, an experienced compiler developer with a CS PhD to reverse a linked list, or words in a string, or something like that.
So get them to solve something more difficult than reversing a linked list. A good programmer will enjoy a good puzzle, especially if it is new to them.
I think the attitude of a lot of companies, especially the ones hiring fresh grads, is that the ability to think is more important than the ability to code (the latter of which can be taught far more easily than the former).
I don't mean to imply that you are unable to think, of course - the puzzles are surely an imperfect heuristic - but that's my understanding of their aim.
How are these puzzles hypothetical? They look better written in specification than specs in the real world:
(selected few and summarized...)
Hoppity Hop!
Input: filename as arg from command prompt, file contains a number
Output: Some variation on the word "Hoppity" repeated
Simon Says
the gist: Connect to some server, call some api's in a certain order.
Battleship - write a client that connects to some server that plays battleship...
Find Sophie - read some data,build some data structures/run algorithm,output result
Input: filename and specifications of the file format
Output: result
Some test simple programming skills (Hoppity Hop is basically the Fogbugz for-loop question + no brainer file parsing) and the harder ones seem to involve knowledge of (not too tough) algorithms.
If the questions were more like "How would you move Mt. Fuji", then that's certainly hypothetical.
Hoppity Hop is basically the Fogbugz for-loop question + no brainer file parsing
Why not give me a real world use case, with a code base to work off of? Put up a codebase on github, provide a set of user stories and requirements to satisfy (ie, "add a real time search using this data set and code base which works cross-browser and degrades in IE"), and ask me to add it, thus basically mirroring a normal development process.
As someone who has had to interview programmers before, that would tell me infinitely more than their ability to re-iterate the word "Hoppity" on the command line.
Because the people you want most aren't going to waste a ton of time jumping through hoops? The nice thing about puzzles is that they're self-contained and can be understood and solved in a few hours. When I was applying for a Facebook internship, I wound up spending ~6 hours over a couple of days finishing two of the puzzles. This already seemed like a significant time investment. A problem that involved "a real world use case, with a code base to work off of" would necessarily take a lot longer to understand and implement. I was pretty busy that month, and if the puzzle questions had been more time-consuming I might have skipped Facebook and applied to other companies instead. (As it happened I wound up at Google)
Puzzle-based interview questions are meant to be a rough cut to weed out the people who are clearly unqualified for the position. They're not meant as a substitute for more in-depth discussion of skills, work experience, etc. If the filter is too time-consuming, it's going to weed out some of the best people who are confident they'll get offers from other companies. And a large company like Facebook doesn't necessarily want to test specific skills because they may not have yet decided which position you'd be hired for, and in any event you're likely to work on multiple projects during your time with the company. For entry-level positions (which is presumably what these questions are designed for), it's better to hire the smartest guy you can find and train him on specific technologies than to hire a guy who already has a specific skillset and then discover he's a one-trick pony.
That's confusing to me, 6 hours is more than enough time to implement a small but significant feature. I think it's possible you're assuming the request would be a particularly large one.
"Why not give me a real world use case, with a code base to work off of?"
Because they want you to be good enough to do it without that.
(I'm saying this as an about to be new grad who is studying these subjects to not have that much help and have to do it on a whiteboard in an interview
and also working on these puzzles and similar ones from a lot of other companies)
As someone with a lot of industry experience: It's great to know how people solve puzzles and tackle algorithms. It's even better to know how they can fit those solutions into a bigger picture, how they structure, comment and document their code, and how they interpret a use case and user stories.
Dealing with things in the abstract, stripping it of context, interface, interaction, and real world requirement, seems to be an incomplete way to understand the capabilities of the person you're potentially hiring.
Unless, of course, you're hiring for the position of Junior Abstract Puzzle Solver. In which case, by all means, Godspeed.
The reality, for better or for worse, is that a lot of the companies that are using puzzles like this prefer false negatives over false positives. In other words, they would rather pass over the occasional excellent engineer than accidentally hire a mediocre engineer. This is the case at Google, Amazon et al, and likely at Facebook and others as well. It very well might be the case that you're just an unfortunate casualty of this sort of hiring practice.
I think you hit the nail on the head, but here's my issue: It's not so much companies like those that have a problem. I've been recruited by all the big guys on the strength of my open source work (save for Facebook, but that's probably due to a major conflict of interest, considering what I'm working on), but the real problem arises when you have smaller companies latching on to this approach, without the recruiting power of a Google or an Amazon. So now I get contacted by a small startup who all of a sudden wants me to take a quiz because they fancy themselves the next Google.
Please don't ask me to review your "massive amounts of code" - I guarantee you will not come out ahead in that evaluation, if I even have time to properly evaluate it (according to Code Complete, you can properly review code in a high-level language at about 100-500 lines per hour).
If you consider the fact that:
- It is unlikely I am well-versed in the problem domain of your code,
- I am unlikely to be aware of external design constraints or coding pressures,
- I am probably going to randomly select the worst section of code to review,
- I have no context for how the code evolved
the chances that I walk away with a negative impression are high enough that it should give you pause. If I do manage to walk away with a positive impression of your code, it will be offset by the perception that you believe your time is more valuable than mine (why else would you send me a massive amount of code and ask me to review it to determine your suitability for a position) - and you still haven't answered the other big question I need answered which is whether you are a good "fit" for the team from a personality/cultural perspective.
If you really want to impress me, go ahead and send me a list of your projects and contributions and then slice a relevant sample of code from it. Describe what the problem was and how you solved it so I can understand the context. Try to make it relevant to the position, but failing that describe how you could utilize that experience in the relevant domain. Keep it small enough that I can review it in under an hour or two.
Is it a lot more work for you? Absolutely, but if you do this I guarantee you I will look at your code and will likely call you for an interview because you are demonstrating a high level of competence and professionalism while still respecting my time and needs.
You're assuming that I would "send you" massive amounts of code. I wouldn't. I'd have a link to my github on my resume, along with a description of the projects I've worked on.
It's presumptuous that I would be asking you to do a full code review (my Appleseed project is tens of thousands of lines of code, for instance), and that I wouldn't recognize the impracticality of such a request.
I would find it problematic, then, if you responded with tests and puzzles, without any interest or questions about my code. Or if the test and puzzles couldn't be skipped once I mentioned my experience level and accomplishments. This is often what interviewers have done, and I find it to be a good indicator of a workplace I'm not interested in working in.
The best workplaces let their developers take some time to comb through the best parts and the worst parts, and have a conversation about what I've done, my goals, how I did things, etc.
Asking me to look at code on github is not that different from just sending me all the code (other than the fact that you aren't cluttering my inbox with it). What code do I look at? Why is it interesting to me? I understand your point about having a discussion about your code, but you are again asking me to find those interesting bits that are relevant to the position I want to fill. You are familiar with the code base already - it is not difficult for you in your cover letter to indicate that I might be interested in x,y,z in project foo because they demonstrate a,b,c, and I can find the code on github. Now I know what to look for, and I am very likely to go and look for it. That may lead me to explore some other things and will lead to an interesting interview. Even if I don't consider the code interesting and relevant, it still lets me ask about why YOU thought it was. If you don't point me to that, I may just flip through a couple of methods and never find anything that interests me enough to talk about it.
And for the record, I don't like "puzzles" either - even if I know the "trick" to the puzzle, sometimes my brain doesn't work right and I can't think of it after a long day of interviews. I would much prefer to be evaluated on how I work in a real environment, seeing my real code and my approach to real problems as opposed to impossible problems whose solution hinges on an obscure bit of semantic parsing in order to arrive at the solution. But as an interviewer, I don't have unlimited time to evaluate candidates so if you want me to do something, it helps to make it as easy as possible for me to do it.
In the end, I don't disagree with you, but I can understand interviewers that don't make the effort if you don't also make an effort.
it is not difficult for you in your cover letter to indicate that I might be interested in x,y,z in project foo because they demonstrate a,b,c, and I can find the code on github.
I think it's fair to assume that after my years in the industry, that I would not simply hand you a link with no context, nor that I would be applying for a job which didn't relate to the work I had done, and I don't feel these arguments apply to the situation I presented.
But as far as I'm concerned, my preference would be to hire based on established code and real world tasks, not abstract puzzles. Not only are puzzles like these faddish, I've met too many programmers who excelled at puzzles, but were severely lacking in more mundane (but all too necessary) departments, so in terms of what kind of things I would use to weed people out, I'd do a set of real world tasks like the one I mentioned above, well before I'd ever give a list of clever puzzles to solve.
Assuming the applicant didn't contribute to open source, which would always be a huge plus in my book.
Most of the time anyone with solid engineering background, industry experience, programming contest medals, or entrepreneurial streak would have no problem landing an interview.
However, that leaves a whole lot of candidates out of the picture. Faced with limited recruiting resources, puzzles are about the next best thing to serve as a recruiting filter. Plus some people send them in without being told to, which is usually a plus, as it implicates coding in their spare time is something they enjoy doing.
These FB puzzles test your algorithmical capabilities; and being able to develop new algorithms is not always necessary (though always useful) for a programmer. So you can have lots of programming experience and still not be too good at algorithms.
But it is important for Facebook - so that's what they test.
Well, its certainly more fair than giving puzzle/coding questions during an interview. This way you are under no pressure and can work on your own machine in your own time.
I think one of the reasons I don't do well on these is because my brain doesn't seem to work well for hypotheticals. I need real world use cases or else it's exceedingly difficult for me to work out a solution for a problem that is too abstract.
Maybe I'm unique amongst other programmers with my level of experience. It's possible that other programmers love doing puzzles like this, but I tend to get discouraged if I'm looking at a company, and they use tests and puzzles to gauge my capabilities.
Especially (and I wouldn't accuse Facebook of this, but it happens much too often), if those tests and puzzles are substitutes for looking at the massive amounts of code I've written.