Method 1:

  How many ten penny nails in a pound?
  Who makes better screwdrivers, Craftsman or Snap-on? Why?
  What's your favorite toolbox? Why?
  How can you tell when wood has been treated?
  Why should you measure twice, cut once?
  Which end of the hammer do you hold?

  Show me something you built. Describe what you did.
  Cut this piece of wood according to these specs.

Both. The first method provides better protection against talented bullshitters, the second provides better insight into the candidate's work.

There is nothing wrong with examining the level of details somebody has at their fingertips. If they don't have the specific detail immediately handy, watching (and perhaps collaborating) with the person to find detail-type info gives you insight into whether they have a suitable user-interface when you need information from them.

Assuming that your skill level is above that of a talented bullshitter it shouldn't be that hard to drill deeper in to the previous projects with specifics. You don't necessarily need to even know the field, if you're a competent programmer you should at least get a feeling of how skilled/technical the other person is. And I'm willing to bet that that information will be a lot more reliable than the "what package is Foo in ?".

IMO using the first approach you concede that you aren't skilled enough to meaningfully evaluate the previous work/ skills, which is fine for contracting small work on your home, but you should consider third party evaluation/insurance service for anything big. In a interview scenario using this approach tells me I'm being interviewed by technically illiterate person that can't or won't use someone with more technical ability to do the evaluation, and both can't/won't options tell me that this company probably isn't for me.

Only to those who confuse a test on details with ignorance of a field.

There is a wonderful scene in The Seven Samurai. The leader is testing the prospective candidates one-by-one as they walk into the "interview" room. There is a junior fighter hidden behind the doorway who attacks each prospective samurai as he walks through the door.

Most of the fighters dispatch the junior fighter without much ado, and forgive the "interviewer" understanding that he must test each one of them. There is one candidate who stands outside the door before walking in and says (paraphrased): Please, skip the attack that I know you have planned; It is a waste of energy. He does this in such a way that it is clear he will dispatch the junior fighter even before fighting.

Then there is one candidate who is unaware and gets whacked over the head.

The scene is a wonderful summary of interviewing candidates. (And a great movie too, fwiw.)

There are insightful details that you only catch by working with something, eg. If I ask you what will the following python code print :

    def default():
      print("using default arguments [1, 3]")
      return [1, 3]

    def append_five(a=default()):
      a.append(5)
      return a

    print(append_five())
    print(append_five([1, 3]))
    print(append_five())
    print(append_five([9, 7]))

But this is not the same as asking what package contains what class, and similar petty details that you think are good questions because you looked them up in the documentation before the interview. I would expect many good programmers to miss that and bad ones get it right. Knowing the answer to the above question vs. knowing the name of the Java clusterfuck naming convention IMO gives a very different level of insight in to the experience of a programmer.

But really, this question only tells you whether a candidate is lying about Python experience on their résumé. If the candidate answers, "I have no idea. But I have 5 years of Ruby experience, and many years of Perl before that, and I can be competent in Python within a month or so," this question tells you nothing either way.

I've always been delighted to hire a really good programmer. I'm usually not too picky about which languages they're really good at.

But I agree completely, which is why I said you should be able to evaluate the previous work trough the interview, even if you don't know the specific technology (eg. If I was interviewing someone for a Django/Python gig and he has RoR experience ~10-15 minutes of interviewing about a previous project that he thinks is representative would be enough to figure out if he's winging it or he knows what he's talking about, it's cool if someone misses something but he should be able to reason about things on the fly, eg. in the above gotcha I expect you to figure out what's happening when I tell you the result if you didn't get it right away).

I'm just saying - some details can be useful, eg. if I knew Ruby or Rails I could ask about a few similar details that separate the men from the boys - but asking what namespace is foo located in isn't going to provide useful information and puts you in the PHB class of people who don't realize that their lack of skill is preventing them from creating good evaluation tools and can't delegate the responsibility to someone skilled - the kind of environment I would want avoid at all costs.

Also the language was just the first thing that came to mind, you can use a common library/framework/tool quirk, etc. Anything that shows familiarity with the required tools.

A better example of nano-questions would be "What is the A312X131/22 3-1/2 inch nail used for?" or "How do the Black & Decker DS321 and Rockwell RK7866 belt sanders compare? When would you use one over the other?"

And I think "how do you inherit a class" is the "end of the hammer" equivalent for a language if you claim to have been using it continuously for the past two years. I'm much less happy about the "what package" questions, but if they're meant to simply get you to name a few major packages I think they're fine. You might get the package question wrong, but I expect you to get it wrong intelligently ("what package" questions were reasonable 12 years ago when programmers dealt with them daily, but eclipse does all that for you now).

I think of these as "what color is the door" questions. If you tell me you've been working at Company X, I expect you to be able to tell me the color of the front door, or where the bathroom is, or something about the place. Not that I care, it's completely irrelevant to your skills, I just want to find out if you're really the person you claim to be before I waste time with a serious interview.

Because my time is precious, and I want to spend it with the serious candidates.

Another option would be to email the candidate a chunk of average code while you're on the phone with them, then say "walk me through the code and tell me what it's doing."

"I think of these as "what color is the door" questions. If you tell me you've been working at Company X, I expect you to be able to tell me the color of the front door, or where the bathroom is, or something about the place. Not that I care, it's completely irrelevant to your skills, I just want to find out if you're really the person you claim to be before I waste time with a serious interview."

That's what references are for. FWIW, I can't remember the color of the doors to ANY place I've worked, except for our current office which has quite a peculiar door. And besides, how are you going to know that I'm not bullshitting when I say the door was green and was right beside the elevator?

The problem with nano questions is that they assume that these few tiny tidbits of trivial and often overlooked factoids are representative of knowledge on the whole, and thus are prone to false negatives. They are heavily biased towards what YOU tend to notice, not necessarily what others tend to notice.

Besides, if you have the slightest clue, your references will all speak favorably of you. It's a test with very poor specificity.

Show me something you built. Describe what you did.

When the person describes what they did that opens up questions around materials, methods, and technicalities. Just having them tell you what a certain kind of nail is would have been useless without also demonstrating they used it properly when needed.

But neither method would have overcome your friend's lack of expertise.

If you realise you made a poor choice of screwdriver it is trivial to replace it with a different one and carry on working almost immediately since it's operation is basically the same.

Choosing your language or platform poorly and running into issues later once you have an app close to release or already running in production is going to cause much more difficult problems.

Essentially our tools end up embedded inside our product itself.

It's close to impossible to reliably compare programming to any other profession simply because there is nothing else quite like it.

Making a general contracting mistake such as accidentally compromising the structural integrity of a building by making a poor choice on where to run the plumbing can easily result in a lot of work having to be re-done, at significant cost.

Some of the models (for hydrodynamic research) they produced in teak and other hardwoods where works of art.

Insert obvious parallel about hiring managers who can and can't program.

Once you've established you can talk to this guy (and he can talk to you) you'd ask him to show of his skills.

The proper method is:

Here are the plans and specifications. Bids close in two weeks. Please include qualifications and references with your submittal.

  IDE dependency? Perhaps, but that isn’t necessarily a bad
  thing since that is representative of the tools they will
  be using in the office.

I have come to believe that a decent programmer can and should be able to work well enough without an IDE. Presence of an IDE should be a sufficient condition not a necessary condition and I see so much of the latter.

IDEs are a crutch and there is a minimum level of "walking" that I expect from you without a crutch. If you have to reach for a crutch even for the simplest tasks you will earn my suspicion that you are not as comfortable as the other guy in more complicated operations. As for the claim that IDEs are a representative tool, it need not be true, more for a start up pushing the envelope. Who knows which new or non mainstream language/technology will be most suited for the job. There may not be any mature tooling available yet. I would want my co-hacker to be able to cope.

To give an example, there was this guy who claims he knows scala, but loathes working on a scala project because apparently there are no mature debugger for scala in eclipse. For him a eclipse plugin is a necessary condition.

I have seen too many programmers who program in the following mode - let me type something in the IDE. No errors detected ?, good. Let me test it on a few cases. Didn't work ? let me bump up the loop bound by one, no cookie ? let me decrement it by 2, didnt work, let me change the sign of this expression from positive to negative -...ad infinitum. I am more scared when such a code passes a few half assed test cases and the author cannot explain why the fix works.

I am not saying IDE are universally bad, quite the contrary. I advocate IDEs, but under two scenarios - (i) you are learning a new language (ii) you are an expert in that language. It is the overly populated middle ground, where it is used as a substitute for (a) thinking about your code or (b) knowing the language, that worries me.

There are many times I have to log into remote systems and all I have are nano or vi. Not even emacs. If I have IDE-dependancy, I'm up a creek without a paddle.

I use an IDE--and for many languages, ex. Java, Scala, and C#, would say I have to--because my time is worth something to me.

For the languages you've mentioned. Java and C# do you really have any other option, but to use an IDE? Its like, goes without saying and assumed that you are and every body is programming with an IDE.

In case of Java. You must know eclipse today is nearly a part of the language and not a separate tool.

The assumption that it would be Eclipse, however, is hilarious.

You may be used to a workflow that a particular IDE emphasizes but that doesn't mean that they couldn't muddle through with an editor + command line tools if required.

Sure you might have to read some more documentation but I would argue that you are still using what is in essence an "IDE" just without the "I" since you are using a more loosely coupled set of tools.

At it's core an IDE is usually just a text editor with a few buttons that run scripts anyway. Whether you happen to call them from BASH or a GUI is largely inconsequential.

There may be people who are hopelessly lost without an IDE but more than likely it's just that it's that IDE they learned at school/their last job because they were focusing more on learning the code than learning Unix commands.

People are naturally afraid of things that are unfamiliar but given time and a push in the right direction anybody capable of writing a program should be able to pick up these skills.

  but that doesn't mean that they couldn't muddle through
  with an editor + command line tools if required.

There is, unfortunately, an un-ignorable set of IDE users who cannot be bothered into reading documentation about the language (and I am not talking about support tools manuals). They feel entitled that the IDE should take care of those "details". For this crowd, IDE becomes a necessity and my claim is that its an accidental or an artificial necessity. Not having an IDE may slow you down but it should not stop you, and that is exactly what I have seen time and again.

I am fine with them using any of their preferred tools but they should have some idea of whats going on underneath, hence my two use cases for an IDE. If you know what you are doing, by all means use an IDE.

Plus sometimes it is nice to have a bit of "magic" to do some of the work for you when you have something you need done on a deadline and don't have time to grok the details of everything. It can be useful to go back and try and grok some of the details later but you won't usually grasp this until you experience the pain of working with an incomplete understanding first hand.

I have also met programmers who are almost the complete opposite of IDE addicts. Usually people who come from a PHP/Javascript background and have become used to doing everything using EditPad and debugging with echo statements. It's often just as difficult to make these people realise that they could save time and gain more power by looking into more sophisticated tools whether they are parts of an IDE or command line tools.

But Java is special, try to code Java without an IDE. It's possible, but it's PITA. Typing org.something.somethingElse.somethingEvenMoreElse.ClassINeed and looking it up for every class you need will kill you. Even importing org.something * from every library won't help, you still need to know subpackages path to each class, and most libraries have quite huge depth and branching factor :).

A good programmer can put together a system with nothing but a stack of note cards, a pencil and a good eraser.

Since good programmers can work in really archaic and restrictive environments should they? Of course not! You should give them the best tools for the job, they will work quicker. However, that doesn't mean they can't be amazingly productive with primitive tools (see the 1970s for instance).

- Can't work without an integrated debugger (i.e. can't diagnose a core, doesn't know how to attach a gdb to a running process, etc...)

- Can't diagnose problems outside of the development sandbox.

- Will demand access to expensive debugging tools instead of throwing some logging in.

- Doesn't code experimentally. Won't try things. Will ship the first version that works.

- Can't generate a simple test case for a bug, instead hammering away on a giant 200-file project someone else wrote.

- Often can't write a single-file, simple program from scratch AT ALL.

The analogy to a crutch is very apt. Yes, you can do great work with IDEs, and I'm sure people do. But 90% of the time an IDE user is using it only because it's all they know, and they're too lazy or unskilled to learn something different.

A real programmer uses a pen.

I think the problem is that unless you're talking about using nano or notepad to code versus something else then you're trying to make your IDE sound superior over someone elses choice. When most people talk about being superior using an editor over an IDE, they usually mean using vim/emacs editor. The problem is that these days vim/emacs are IDEs. They may be lighter weight versus some other IDEs, but they are IDEs in functionality, plugins, and language support.

I'm particularly interested in this right now because I've just moved from a C++ project to a Scala project, which meant moving from vim to an IDE.

Of course I see a difference, but it's a difference of degree, not kind. I should clarify that by "text editor" I mean something above the level of ed or nano (though I'm happy to make a similar argument for these too): language-aware features in vim or Emacs such as auto-indenting, syntax highlighting and ctags support are not "transferable across languages" unless someone does the work to implement them: witness Emacs's lack of C++11 support.

Expecting e.g. auto-completion to work cross-language in vim but claiming that debugging in Eclipse in inherently language specific flies in the face both of experience (I personally have used debuggers in Eclipse for Python, Scala, and Java, and I know others exist) and also common sense; both are fully extensible via code, and while Eclipse's support for Java is better than the other languages, it's not tied to it in the way that e.g. the Python REPL is tied to Python. I think you'll find that Emacs's support for various languages differs in quality too.

Also IDEs like eclipse are faily language agnostic, of course the tool support is better for some languages rather than others.

I use eclipse for PHP but I only really use it as a text editor (along with some of the features that help me navigate the code easily), all the debugging etc is done server-side anyway.

Let me help you understand how absurd your statement is: "I have come to believe that a decent builder can and should be able to work well enough without power tools."

claims he knows scala .. there are no mature debugger for scala in eclipse. For him a eclipse plugin is a necessary condition.

What's wrong with that? If your favorite editor is vim with a host of preferred customization and plugins, and I tell you that at my company we only allow the use of TextPad, are you going to be happy about that? How is it wrong for a developer to prefer working from within a particular environment? I hear people talk about muscle memory and the power of vim, yet when the table is turned they cry about crutches and ineptitude.

Also keep in mind that optimal decision making requires that prior probabilities be taken into account together with the costs of false posotives, cost of false negatives and the cost of acquiring evidence. Evidence does change posterior beliefs but but Bayes' rule essentially dictates that stronger claims (i.e. low prior) require stronger evidence.

I dont hate an IDE I advocate it. Circumstances matter.

I do not assume, I verify and learn from past experience. You may find this helpful http://en.wikipedia.org/wiki/Posterior_probability

I have no problems about users preferring one tool over another. I re-iterate , "preference" is different from necessity.

I am not sure why I am re-repeating myself. Apparently it does not make a difference and dont expect that it would even now. Perhaps an integrated reading/comprehension environment is called for.

This is also one of my favourite questions to ask when I'm interviewing someone. Unfortunately most of the time the only response I get is confused stares. Maybe this is just a UK culture thing but it feels like many people are not prepared to display critical thinking on a job interview.

I think any experienced programmer can rant for a least 5 minutes about his favorite language. Also, I think that experienced programmers can quickly find valid criticism even in languages that they don't use all day.

And I think thats global. Most people aren't critics and will hop onto any prepared argument they can find. Thats why I hate most discussions about software.

Every now and then you get a candidate whose eyes lit up and you can't get them to shut up (in a good way). And that is the story about how I discovered judy arrays.

Also if one cannot understand the philosophy behind the language that they are using and express why they like that philosophy and what benefits it brings, they probably aren't using the language right.

However, ask anyone who is great at what they do and you got yourself an hour long discussion.

Haskell is too restrictive, OCaml too verbose and inflexible, Java lacks closures, Javascript has weird equality semantics, C lacks garbage collection, Python lacks static typing, Clojure has too many `)))`s, Ruby is inconsistent, ... I could go on.

In the end, I just settled on a language that I have to use at work (currently Python), but I'm wishing for something better to come along...

Now, if only MS would open-source their implementation, most of these arguments would become void.

Couple that with the future of .NET itself not looking terribly bright thanks to WinRT, and I'm personally getting more and more concerned about the viability of .NET, and implicitly Mono. I'd dearly love to be wrong, but I don't think I am.

- This is one of the things about Microsoft that infuriates me. I've heard from people in Microsoft that "developers have told us they want C++"--well no shit, really, when you've jerked around .NET as much as you have, people are going to assume there's little future for the platform. Self-fulfilling prophecy.

I saw the same pattern begin with .NET; you had to keep playing catchup and buying into the New Hotness. I realized that this seemed to be a Microsoft strategy: obsolete the old knowledge and watch the developers pay money for the new hotness. I'm not saying that MS needs to be static, but there appears to me to be a deep seated instability in their approach, which implies pretty heavy retraining every five years. Contrast this against C++, which has been more or less standard since the mid-90s, especially for the common cases.

I get tired of running on a tech treadmill. Learning how to crap out the same concept in yet another language (much the same as the ones I already have used) has nil interest for me. I am tired of wasting my life figuring out the subtleties of a Java clone, or a C++ clone, or a Perl clone, or... etc.

I look for a stable, long-term language, on fairly stable core technologies, so I can build really amazing new technology without throwing it out and restarting due to the New Hotness. I personally have picked Common Lisp for my long-term code and projects. Others might pick C++, Perl 5, or Fortran. I look over new languages and technologies as they come out and see if they have something meaningfully innovative that really merits learning and/or switching. Some do. Most don't. Haskell looks like it has a good future right now.

(OpenGL is a festering mess, though. DirectX is a wonderful API in comparison.)

The lack of closures in Java means I have to go write an interface and implement that single method (or go implement Runnable). It creates line bloat which can separate code out too far and makes it harder to read, and so on. The lack of a feature is a weakenss.

Nothing wrong with that obviously, but asking a question like "what is your favourite language" is never going to yield fair results across the board

PS: One of my favorite stories of such a person goes something like this. Coder X was fired after failing to get a 6 month project finished after a full year. Greg, took two 40 hour weeks to read the documentation and slap something together to keep the project from slipping, but made a simple mistake which ended up causing significant problems in production a few days later. After fixing the problem he asked the QA people why they put it into production that quickly while missing such a basic issue was not discovered. Their response, we stopped testing your code a few years ago and this is the first problem that's showed up. The most interesting part of this story is if Programmer X had got his code to work after say 11 month he would no have been fired. Which means the company would employ someone that's 3% as effective as this guy.

Just out of curiosity: How would you react if my answer is: Shell?

That said, I very much agree with your sentiments and it does require effort to write resilient, maintainable scripts.

Nevertheless, for my day to day work it's probably the most useful "language".

A 'weakness' implies some kind of flaw, whereas a limitation is something the language wasn't specifically designed to do or specialise in.

I know how Javascript works, scoping, hoisting blah blah blah, but I haven't written a for loop, or used an actual getByTagName or whatever in a long long time ... I told the interviewer this, and they seemed cool with that, they asked me to write in pseudo code and I did that, then they asked me to convert it to javascript ... hunh?

Well, I started going through line by line and started doing just that, asking them to refresh my memory about the syntax of stuff, even how to write a for loop (yup, its amazing what spending years using jQuery.each or $.each will do to you :P). Anyway, we concluded and though I was annoyed at how out of practice I was, I thought I did okay.

Well the recruiter who set it up, called me back (very nicely I might add) and gave me the feedback on the interview. The interviewer, (who had been very nice to me too btw), had eviscerated me, writing that I displayed a lack of Javascript fundamentals ... saying I didn't even know how to write a for loop.

I don't blame them, because we clearly shouldn't have been talking in the first place, they were obviously looking for a hard core Javascript guy ... and not person who could just build complex front end UIs & interactions using backbone/ember/spine/jquery/whatever, which is what I was more interested in.

But it also got me thinking about how some engineers fixate on syntax, and use it in judging other programmers, and I realized that in some circumstances it is pretty appropriate.

For example, if you're looking for a specialized dev, then that kind of stuff probably does matter; in that, it can help you spot a star very quickly ... but I also think that over reliance on it could let you miss out on people who could easily specialize to the level you want, but might not have that immediate level of familiarity with the language. But when you have to go through 100's of candidates, is that something you're willing to take the time to look out for? Should you?

Sorry for the rambling, just been thinking about it for a long time now.

That is, knowledge of language over libraries indicates generality (if you know the language and its idioms well, you should pick up most libraries fairly easily); knowledge of libraries over language indicates specialisms, because you end up a little out of your depth when not wading through the library's DSLs.

It's pretty natural to forget the first syntax if you haven't used it in a while because there's no real standard for it across languages.

I've done, C, C++, C# & Java so the syntax wasn't unfamiliar to me at all, and I probably could have recalled it if I had taken 2 minutes to do it, I just didn't think it would be a big deal to ask so I could get on with the code.

I don't think the current state of the art in software development has yet advanced to the point where we can just black-box away all of the entire "compilation" stuff such that it never affects the "system" stuff. I really would like that to be the case, because it would eliminate a lot of unnecessary complexity in software development.

I think asking a limited number of compiler-level questions (less than 5) in an interview doesn't take up a lot of time, and can allow you to get an idea of how much actual experience the candidate has with the language as well as dealing with nitty-gritty problems that come up while you're coding. The value and time spent are both small, so the value/time ratio is probably in the same ballpark as any other question you might ask.

Nano questions are bad, because you are really asking if candidate has had the exact same experience that you had. Some did, some didn't, it doesn't matter. Anybody will solve the problem within minutes, most probably googling the most promising lines of the stacktrace, and ctrl+clicking around in Eclipse (or grepping the source tree, whatever).

Even auto-importing packages in IDE can cause problems - sometimes it imports from the wrong package. Is it enough reason to ask questions about "which package contains Hibernate Session class?" or even "which package contains ArrayList class?". I don't think so. Being aware of the problems good abstractions can cause (there's always a few problems) is IMHO enough, no need to remember everything you can google in 5 seconds.

EDIT: or maybe you meant that you just check if people know casting can throw ClassCastException, with that I'm OK.

I have worked for some nice, amiable smart folk who interview like crazy people. It’s the whole > IT skills < social skills thing.

But I empathise - I really don’t like interviews which are like that, especially when they’re usually a game to try and make the interviewer seem smarter than the interviewee.

"Here's a coding challenge, go home, write the code to solve this, come back tomorrow and tell me three things: - How you did it, - Why you pick this solution, - How it could be improved"

by doing this I could see how he writes code, if he's just a google copy&paster, and what's his skills in algorithm optimization etc...and specially, how he/she thinks.

Quite frankly I don't care if someone doesn't know what Polymorphism is right from his head. I didn't knew what polymorphism was until a few months ago, and yet I was applying the same principles in a lot places in my code. That's the downside of being a self-taught you don't get to know a lot of theory, but on my work I know I'm doing it right. (Also because polymorphism is a very abstract thing to be honest....)

In a nutshell, get developers who can show you their code and can it explain it, don't rely on theory, you are not hiring a college professor.

I do agree that nano-questions are not useful except in situations where you get a resume with keyword bonanza: You know the kind, ones with every language,OS in the first page listed so that they will pass through the corporate recruiter filter. For those, nano-questions trip up but that will give me an indication to dig a little deeper into their IDE usage pattern. If they show some proficiency in it, (for e.g., tell me the command in Eclipse to find all the references to a particular method, if the answer is search in files, .. :)).

Code reviews are there to find problems with the code that are not found by trivial static verifiers and/or IDEs. This requires a level of understanding of the language and experience in reading (instead of writing) code, but has nothing to do with regurgitating keyword definitions.

Better in that case to just give a code fragment and make them spot the (algorithmic, logic, security, not spelling) errors.

was trying to point out that it is easy have developers write code that can easily be O(n2) vs O(n log n) because this did not use the right kind of the maps when they were initially building the code. Or I can see developers using HashTables vs HashMaps interchangeably and not see the threading impact. In these cases, some basic questions about their understanding of the classes seem to be a good indicator. Would you consider that a nano-question? I was..

These questions might seem pretty lame, but as an interviewer it does tell you something about the candidate's level of experience with a language.

If you've worked with a language for any decent amount of time. There are things that you really should know without having to look it up all the time.

This is especially true if you mention on your resume that you're an expert at something. I'm definitely going to ask you a difficult question about it and I'll expect you to answer in some depth.

If you mention you're a Java pro but can't tell me what the 'synchronized' keyword does for example. I'd be concerned.

However if you are trying to hire some kind of junior-like engineer who has never had a job before, i can see why questions like this could be asked, but i still dont think it's the best way to gather knowledge about the person you are interviewings level of expertise.

If you have an applicant who has a github full of great code or experience working for a good software company you can probably skip this.

However if you get a candidate without much on paper but who insists they are a Java expert but doesn't know what the extends keyword does then there is something wrong somewhere.

The problem with asking for definitions for words though is that I personally often forget exactly what these are if I ever learned the correct term in the first place.

For example I knew how to override methods and the differences between the type system in PHP and the one in Java long before I knew what polymorphism or "dynamic typing" meant.

The problem is: people lie.

It's the old thing where if you want to hire a juggler, don't you want to see them juggle?

It is a huge turn off for a lot of people and indicates that the interviewer is either too lazy to make the question interesting, or not technical enough to generate an actual good question. Either way it is a red flag.

If you want to ask difficult questions, be my guest. But "what package is class X in?" is not difficult, just foolish.

But I look upon them as a kind of FizzBuzz. I know for a fact that while you can look them up in Google in 500 nanoseconds, experienced people doing hands-on work are going to be able to answer 2/3 nano-questions instantly. A few such questions sprinkled in the opening part of the interview are useful for weeding out the AbstractArchitectureAstronautFactoryFactories from the Programmers.

I’m not talking about esoterica, e.g. “What is protected inheritance in C++?” Lots of practitioners might say, correctly, that they never use it and can easily look it up if you need to know how it works. And sometimes answering esoterica correctly has negative correlation: You end up hiring people who are SmartButTooBusyPlayingWithCoolIdeasToGetStuffDone (I’m a prize example of this). But if a programmer says he’s working with jQuery and I ask him to explain the difference between $(‘.foo.bar’), $(‘.foo,.bar’) and $(‘.foo .bar’), I don’t want to hear from him that he can look it up on the Internet, a jQuery programmer knows the answer.

It’s acceptable for someone to say, “I don’t know jQuery, but I’m a smart guy, I can figure it out on the job by looking stuff up,” but if he says he’s been using it to build awesome HTML5 applications, I want to ask a few of these questions to make sure that I’m interviewing the person described on the resumé. A working programmer will know the answer to all of them, they shouldn’t be hard. Interview pressure will cause some fuzz, so I don’t expect perfect, maybe get 2/3 or 3/5 right and then we can talk about composition vs. inheritance or whatever other pet “Start a conversation with a smart person” question.

I want to be incredibly clear about this: I don’t think that there is some strong correlation between memorizing things and being a good programmer. I don’t think programmers should rush out and memorize trivia just to “pass” an interview. I look stuff up all the time in my job. But I do think that if the questions are easy and obvious to the working programmer, a few of them are an appropriate FizzBuzz for weeding out the folks who have falsified their hands-on experience.

By and large, interview questions are a touchy subject because they serve as a kind of metric for valuing people. So when I say I think it’s cool to ask a few of these questions, some folks are going to think that I say that I measure their worth by whether they know a handful of bits of information. That’s not the case. I’m not saying you’re unworthy if you don’t memorize things. But I am saying that if you claim to have a certain type of experience, there are some questions that are going to be dead easy for you to answer, and those questions are going to help us get to the good part of the interview very quickly.

p.s. List vs Set is a funny question. I can’t imagine anyone with a degree being ignorant of the difference regardless of the programming language! It’s also a good lead-in to a more experiential question: “Okay, can you describe a time you’ve used a set of some kind? What were you trying to accomplish? ..."

That's not what he's complaining about, though, he said that question was perfectly fair game. Because it is something that, as a Java programmer, you actually need to know to do your job. Similarly, the difference between the jQuery selectors is relevant, because those things actually do different things. I'd argue that even your joke example, "What is protected inheritance in C++?" is still relevant, if esoteric, because it's knowledge that could potentially be relevant to how code works and what it does.

You truly never need to know what package List or File is in, though, in the real world. If you're programming Java like 99% of the other people out there (i.e. you're not the type of masochist that thinks they can be productive in a boilerplate-heavy language like Java with nothing but a text editor), your IDE will fill in those blanks, and quite literally hide the import statements from you so you never need to think about them again (unless you mis-selected the class, in which case the fast solution is still not to remember the actual package name, but to delete the offending import statement and "Optimize imports" again, this time picking more carefully rather than mindlessly selecting the first one in the list...).

That said, most Java experienced programmers would probably know that stuff like List is somewhere in java.util, and stuff like File is going to be in java.io, just from having auto-completed them so many times, so it could be a very weak filter against people that are lying about their experience. But I still don't think it's a useful question, at all - if they get it right, it very weakly confirms that they're not lying, and if they get it wrong it's nothing more than a minor eyebrow-raiser, suggesting that possibly they've exaggerated their familiarity (though again, any practicing Java programmer could be very good but still be weak at the "Name That Package!" game, because it's 100% irrelevant to the job). The thing is, you'd get far more information either way by just having them rant about Java in general, or asking them to talk about what pieces of the standard library they use the most, etc.

On any other day, if the choice of eclipse or equivalents(IntelliJ etc) is not available it practically makes zero sense to deal with monstrosity and needless complexity Java has with a ordinary text editor. Even if that is Emacs. Emacs is still a text editor.

On any given day Java development makes sense only if done on a IDE. And that's nearly every Java programmer works. Exceptions exist, but they only prove the rule.

Having said all that, These days eclipse really does 90% of the coding. You job is only to tell eclipse what needs to be auto completed.

Emacs has working autocomplete from Java, especially with Semantic. It's far away from being as smart as something like IntelliJ, but it works well enough. Most functions and arguments you remember after a while. I can type much faster than I can think, so less autocomplete doesn't slow down. In fact, not having to watch if the autocomplete gets it right frees the brain to think ahead more. Maybe it's because I'm used to this.

Import statements are one of those things where you have to switch to Google, though.

I am now planning to shift to ST2.

Even on an editor like ST2 which is awesome anyway. You get to see what an editor is designed for. Its supposed to make 'Text' manipulation easy. So that is what it is, after everything.

So Emacs with all its power is still a Text editor, albeit a very powerful one.

Every time I work on Text editor I focus on reading manuals and understanding stuff no matter which editor that is. But with Java,IntelliJ and Eclipse its different, I just throw in whatever little is there on my mind and just wait for the autocomplete to do the magic.

Some of my colleagues go a step far. They just can't write any Java code without an IDE and I see that's the case with nearly every Java programmer I've met so far! Even in interviews. Its sort of like if you are talking of Java development its taken for granted that its is happening on an IDE. You never know, a few days from now the Java language specification my include a IDE specification.

According to me tools and languages must enable you think more than you write. Unfortunately in case of Java(and now growingly Python) you just can't work without an IDE.

Yes, this is often a good use case for avoiding tools like Eclipse (IntelliJ is somewhat better), sometimes the multi-language pain is just too great. Especially if you already have to set up a complicated build by hand and interact with several other systems, the IDE can no longer gain you very much.

For pure Java, though...I don't know. I use Emacs all the time for other stuff, but I just feel so naked when coding Java within it (I've tried a few times, but always end up going back), maybe I'm just too used to all the help I get from IntelliJ...

I have experience working in good IDEs. For many years I did .NET development, and Visual Studio is very, very good.

However, over time I realized that even with all their awesome features, powerful IDEs were a net-loss to my productivity. They work best from a single machine and that made me a lot less likely to work remotely (I have done my best work in the middle of the night, from home). Their less responsive UIs made it easier for me to get distracted. Their slow startup times exacerbated my natural proclivity to procrastinate.

I realized: My working memory is fine for most projects, and emacs' text-based autocomplete fills in the rest.

All I'm saying is: don't assume that the way you work, the tools you prefer, or the tradeoffs you make are universal. There are a lot of programmers out there, and a whole lot of tools.

(If you really care, I have an interview about my setup here: http://aaron.boodman.usesthis.com/)

I just think that it's a good sign that you can tolerate a lot of pain. :)

Nano-questions, on the other hand, I don't see as useful at all. I spent 5 years writing enterprise Java applications, and I couldn't tell you what package List and File are in without looking them up. I've also been away from Java for a few years and have been using other languages, so I couldn't tell you for sure if the "inheritance" keyword is colon, parenthesis, "extends", "subclass", "specializes", or whatever. So in this case I'd get 0 out of 3, thus weeding out a proven, experienced, enterprise Java architect. Even if I had known the answers, I'd consider it a major red flag that such questions were even asked, as it would bring into question the interviewer's ability to think creatively (and no creative person wants an uncreative boss).

The examples in the article have no high-level component at all. I don't know what package File is in off the top of my head. I've never needed to commit it to memory, since the IDE will look it up, and if that doesn't work I can look it up myself. It's not useful information. Your question, on the other hand, is useful information.

For example, you could ask how to switch two variable's values in C++ with the aim of finding out whether they know about the swap function (and you can then go on to ask about template specialization for extra credit).

This would be a better question than the ones in the OP because it's not just trivia and it's not something that your IDE will do for you.

Many people will claim to know a programming language on their CV because they spent a week trying it out 3 years ago, figuring that they can just pick it up quickly if they get the job (this might be somewhat true for the 5th or 6th language, but probably not for the 2nd). And these are the same people who will then program in that language for years without ever producing any quality code because they don't realize that there is room for improvement after you know all the keywords.

After the 10th question asking me to debug 30 lines of obfuscated code in my head while a 3 minute countdown ticked away, I called it a day and decided these were people I didn't want to work for.

Precisely. I'll take someone who can think this way before someone who can rattle-off all of the minutiae about a particular language. I've run across too many programmers who don't have a clue about project organization, MVC, data representation, optimization, etc. Yet, they can pass minutiae-filled tests about a particular language.

Unless you've been programming in a single language for an extended period of time you will not have encyclopedic knowledge about that language and its libraries.

Get 15 to 20 languages under your belt and the effect is more pronounced.

I know exactly what it takes to write a number of sort algorithms, genetic solvers, neural networks, real-time embedded OS and more. No, I can't rattle off exactly how to write it in the language of the day. When switching to a language I haven't touched for a while it takes me two to four weeks to "task switch". I surround myself with reference books, use the IDE and any available online resource. Somewhere during that period I start to rock. I get the job done and produce clean and efficient code, fast.

I would probably fail the kind of questioning the article describes. Yet I've been solely responsible for large projects using languages spanning from assembler to Forth, C, C++, Verilog, Lisp and, lately, Objective-C.

Bad programmer! No doughnut!

Put another way, the problem is less the specificity of the questions and more the ease with which you can expect to earn them.

I haven't used Swing/AWT in two years, and I couldn't really tell you what inherits from what. But after a weekend of breaking through the rust, I'd get back into it, so it's of no consequence to any potential employer.

However, some language-specific questions are very telling. For example:

    How many arguments does type() take in Python?[1]

(Of course, it's possible that you know that and have simply forgotten, or were confused by the phrasing, etc. - no question is perfect, but after enough of these, you start to piece together a picture of the candidate).

The point is, the former question is something that you can reasonably learn during a training period and which asks little more than face value. As for the latter question - you all know that now, so you can also learn it during a reasonable training period. But actually applying that knowledge (which comes with the followup questions) is something that would require a significant level of familiarity with the subject in question, which is what they are really trying to get at.

[1]This may not be the best wording for the question, since it tips the interviewee off to the trick beforehand, but I can imagine a way to present this question so that it wouldn't.

For example, "how would you implement a least recently used cache in Java" can provide a wide array of answers and tells me a lot about the person's knowledge of the SDK and overall algorithm design. The details of their solution and the ensuing discussion tells me a lot more, no matter what their level, than whether they simply know that LinkedHashMap.removeEldestEntry exists.

I really don't give a stuff that your Boost/STL trick question is missing a semi colon, which in reality would result in pages and pages of false compiler error messages.

I really don't want to try an second guess the compiler and tell you where the missing semi colon should go.

I use the compiler to detect my mistakes. I read the compiler output error messages, understand the message and fix my mistakes.

But I don't pretend to be a compiler. I'm a software developer. I use the tools available to me to write software.

A sportswriter who claimed an expertise in basketball wouldn't feel it unfair to be asked a few trivia questions in an interview. If he didn't know how may winning seasons the Chicago Bulls had with Michael Jordan then maybe he isn't as expert in basketball as he claims.

Where these questions are unfair is when the applicant is more of a generalist. They have worked on very diverse projects in multiple paradigms and multiple languages. Their skill lies not in knowing how to use their tools, but in approaching new tools and quickly understanding how to use them. If you are a Java shop and someone comes into the interview with little Java experience then asking questions about Java specific type enforcement is going to leave them a little flustered.

Cater the interview to the applicant, not necessarily to the position. Then evaluate your understanding of the applicant with respect to the position and see how well they fit.

I use C# almost every day and if you quizzed me about what namespaces certain types are in, I'd probably get it wrong. That's not because I haven't been using it regularly for years (I have), it's because that shit isn't worth wasting brain cells on.

Even worse, though, the modern programmer ends up using a lot of languages. I have, in the past few years, had cause to write C#, Python, C++, C, Perl, JavaScript, ActionScript, PHP, SQL, Lua, Objective C, and Ruby. All for work. Each one has its own syntax, semantic quirks, standard library, etc. Out of all of the important things for someone to understand about a language - why would you care about something as unimportant as the exact namespace/package path of a particular type? Does that matter? Would you rather know whether I know the exact namespace Adobe's JSON parser lives in, or would you like to find out whether I'm familiar with the ins and outs of the ActionScript runtime?

Honestly, if you think the kind of minutiae this article mentions is a good interview question, there's no way I'd want to work for you, because you don't care about things that matter - culture, candidates' desire and ability to learn, candidates' willingness to teach others, etc.

Sure there is. In a set, you can't have repeated values. PHP arrays can. Of course, you can implement a set using a PHP array.

My point, which was apparently lost, is that the underlying implementation of these structures is the same in PHP.

As the post says find the Person who is the best fit for your team/company. Everything else can and will be learned on the job if you get the right candidate.

Can't argue there. I haven't done a ton of interviewing in my career so far, but it is important to me to find out how the other person thinks. Being able to cite function names and package identifiers from rote is not important, at all.

Coupled with that, maybe some questions would be in order about how and when you're legally allowed to use the stuff you find using Google (MIT license? GPL? Creative Commons? How do I have to attribute this stuff? Do I have to release the source?).

I don't think I've ever written an import statement by hand, it really, really isn't important to know.

What's a namespace? Uhmmm, HashTable? I don't use collections.

Also, if they can't name some namespaces I'll usually ask them what classes they're familiar with for accessing a database or opening a file. If you can't name parts of the framework you use on a daily basis by namespace of class name, you're not going to be a very productive programmer.

If knowing this stuff isn't important, than I'm an expert Java programmer too. I just can't name any of the packages or classes, but I can tell you all about inheritance.