They always tell you when you are interviewing you should never bitch or gossip about your old job. Articles like this are the inverse of that, the employeer is subtly bitching about every bad hire they've ever made. This is a giant red flag that whoever is doing the hiring sucks at their job.
I've had a few interview process that have included some or all of these 6 steps, and I've turned down every one of their offers at the end. Not because I don't have the time to work on a sample project,or because I can't cobble together and isolate some sample code for them to take a look at, and not because I don't know how to play the interview game.
It is because your interview process takes too damn long. One or more phone screens, one or more in person interviews, then a request for a sample project or source code, then a request for references, then a meeting with team, etc. That takes months, with weeks of down time in between.
If you can't figure out if a person is a good fit socially and is a competent programmer in less time that, you suck at hiring and shouldn't be doing it.
I know this is tangential to your main point but I actually like and expect some bitching when I'm interviewing someone, and frankly don't see how you could do an honest interview without some bitching. The supposed motivation for the no-bitching rule is that the interviewer thinks you'll complain about their company in two years. Well yeah, no shit, if you get fired or quit in two years, you're not going to run around telling people what a pleasure it was to work there, and that the breakup was mutual.
The people who seem to say every place they worked was good, had a good team, had no problems, etc, always seem to be mediocre. The interviewee who's like "Oh my god, they didn't even use source control when I got there. I setup a subversion server on my desktop." shows that they take charge. Horror stories about bug tracker shenanigans are always good for a laugh. "Well 'critical' was getting used too much, so they created 'ultra-critical'. After all the bugs ended up at that status they added 'yesterday'." Funny, but also shows they understand the concept of prioritizing. The people who bitch, and (more importantly) take the steps to make things better are the people I want to work with.
Of course it can be overdone, and I've had people who bitched so much or were so caustic that they got a pass.
I guess maybe things would be different if I was working in Fortune 500 IT-land. Maybe then I wouldn't want someone who complains about Fortune 500 IT-land shenanigans, and just want someone who will just implement the code written by the Grand Architects on Mount High. But it's like the rule about wearing a suit and covering up any tattoos. I will never wear a suit to an interview again. If that's a pass, that's a company I don't want to work for. Same with an interviewer who would be outraged that you complained about the company you're actively trying to leave. Or that you had the gall to take a year off of your professional life to tour Europe.
It's probably a bit extreme to say he's "bitching".
This sort of thing is quite useful to those of us who have an involvement in hiring, but don't have a lot of experience.
I've been involved in hiring a couple of people, though the process I go through usually involves:
a) Read a code sample, if available.
b) Face-to-face interview (telephone if face-to-face is difficult).
c) Hire/No-Hire
The interview usually involves talking about our company and the interviewee talking about their previous experience. The hiring decision is usually based on "I like the feel of this guy", with some weighing up of their skill set, the job role, and also how interested they would be in working for us.
I've found this works for the company I work for. We don't hire large numbers of people, so I don't think this approach could scale - as an example, the largest block of CVs for a position that I've seen is about 10. I don't know how it would work if we got 100 CVs.
He's saying negative things about former employees. So much for symmetry. Interesting that employees cannot prudently make negative remarks about former employers even in general terms, but employers can make negative remarks about former employees. If a negative remark would lead a prospective employer to wonder if a candidate would eventually say the same things about him, why wouldn't a prospective employee think that a prospective employer who made remarks about bad hires would not eventually say the same things about him?
I think it depends when and where you discuss a negative experience. Sure, not a good idea in an interview.
But what about doing a personal reflection, say a blog post or something on your own time about the observations and lessons learned with regards to things like culture or development philosophy directly related to that company? As long as you do it in a way that doesn't break any non-disclosure agreements why should you be punished for it and do you really want to work for someone who would look negatively on you for expressing what you have learned from a bad experience? Even more important - perhaps the employer doen't know how bad the culture is inside or worse they do know and rely on you not ever saying anything to continue recruiting others similar to you?
Then, during the interview you can find a way to bring a summary of the negative experience into the conversation without it looking like you want to trash or burn your former employer or manager. There are many opportunities for this, a common question one gets asked is to describe what you have learned from mistakes on the job. This is open and general and one could easily work programming specific or soft skill lessons learned into the discussion that do not paint your former employer in a positive light. And if your former employer has not earned the right to a positive reference you should not be required to give it to them just because you want to continue working.
I've started flatly turning down offers from one large company in particular because of this. It didn't help that they only wanted to conduct their two or more 1+ hour phone screens during banker's hours.
dangrossman has the right range: they wouldn't even go as late as 4pm, or as early as 9am, which suggests to me they're only looking for people that are out of work or otherwise desperate enough to take what they can get.
While I can definitely see how it could take a long time, the last time I went through the hiring process, it included every step but #4 and was completed in about 1.5 weeks.
I suspect that length of time it takes to complete a hire is probably a good proxy for measuring bureaucracy in an organization. A low-bureaucracy team (like StackExchange?) can get through all these steps pretty quickly if they want to.
More importantly: if you place so insanely high value on that ONE new programmer you are hiring and you can obviously abso-frakking-lutely not afford to have picked a single "bad apple" then that says a LOT about the situation of your company, your management and what state the projects are in.
All these steps feel like bordering Stasi-methods (hey, why not lie-detector test the applicant while you are at it!) but I'm sorry, this is really extremely over-doing it in contrast to the other extreme, the "oh you have a cool tshirt and unwashed hair you must be a genius!" hiring that was done in the dotcom bubble.
Er, for many, many employers (like mine) with very small dev shops -- I'm talking 2 devs here -- a single bad hire DOES have an extremely deleterious effect on everyone and everything else. Not everyone works at an organization with so many devs that they can afford to have an incompetent 10%. If you'd like I can regale you with my personal experience on just how damaging a bad hire can be.
All steps SHOULD be taken to avoid bad hires, and ... I'm sorry I really just cannot believe I'm even having to say this. Where the hell do you WORK? What do you do for a living? I seriously can't imagine anyone I know holding the belief that it's no big deal to hire a bad programmer.
I'll echo this. We also have just two devs currently (soon to be four!) and we are still finding issues that were caused by the previous third developer, who was let go several months ago. Some of these bugs have brought the production system down for one or more of our clients. I'm quite certain that if he has gone through Jeff's process he would have been filtered out prior to being hired.
If your company is that small then I agree you need to be more careful but you also need to find someone who basically wants to work at a startup, any company with just 2 dev's isn't a "normal job", even if its not technically considered a startup you would still need to approach hiring as if you are.
I wasn't talking about holding on to a bad dev for 5 years... I was talking about the possibility of hiring a "bad" dev and seeing how they do in a real work situation for a couple of months maybe. Typically it should become very clear pretty damn quickly, in a matter of weeks. If you cannot afford that to happen then you have no business opening a dev position and you are VERY likely better off with trying to make it with the devs you currently have available. I seriously doubt those almost-Stasi methods will ultimately get you a 100% sure pick - so you will ALWAYS have to deal with the possibility of having picked a bad one, but those methods will very likely drive the real good ones away because they couldn't care less and usually they have TONS of other options available.
Do you really expect a potential employee to implement a customer-requested feature you are getting paid for for free, just for the mercy of being allowed to apply at your small 2-men shop? Really? You expect your employees to be productive and billable from the first hour they work for you? Sorry, good devs would never ever work for you... they have way better and more rewarding and realistic options available.
Nobody worth your time or money works for free - or do you give your customers a "freebie" without ANY strings attached?
I've interviewed at a lot of places, and some of the things these companies do is breathtaking ... that's when they're interested in you. They try to schedule you into 3-4 hour interviews like you don't have a job to do that day. They send you coding assignments that take 4-8 hours to do. Interviewers get your resume 15 minutes before they walk into the room with you, then ask you to rehash your entire background (because ... you know ... that shit isn't in the resume in front of you already).
Then there's the default position some of them take that 90% of the people they interview are idiots, so they're just going to treat you as such until you prove them wrong.
I had one firm set me up for an interview, and then cancel it without actually telling me, until I emailed about 2 hours to the time of the interview wondering what was up etc etc
But that's just the half of it, as soon as some of them decide you're that worthless programmer who didn't pass their pet brainteaser/trick question, they won't even do you the courtesy of telling you that you're out of the running. The recruiter who was all so chummy with you and positive about your prospects goes AWOL and you get that dreaded automatically generated rejection email.
If you felt good enough about me to actually talk to me on the phone to get me to interview, why can't you even email me to cut me? Just man up and do it like the movie Moneyball. "Sorry x, you didn't make the cut. Good luck in your search". I'm a professional, I can take it.
Its all rather exasperating, because we both want the same thing in the end.
"One of the marks of a professional programmer is knowing how to organize software so that the complexity remains manageable as the size increases. Even among professionals there are large differences in ability. The programmers who can effectively manage 100,000-line projects are in a different league than those who can manage 10,000-line projects. ... Writing large buggy programs is hard. ... Writing large correct programs is much harder."
Jeff Atwood's metrics will help you filter out engineers whose complexity ceiling is <1k lines -- StackOverflow answers, whoopee -- but that's not a terribly hard thing to interview for. Much harder to interview for the very best, the mythical 10x productivity programmers[2], those who can handle 100k LOC, 1M, or more. Perhaps this is the difference between an experienced non-expert and a real expert[3].
In my experience not a lot of employers care about this, perhaps because their challenges aren't those of complexity-in-scale, or perhaps because complexity hasn't bit them hard enough yet, or perhaps because they are "unconsciously incompetent"[4]. About the only hiring signal I've identified for this is interest in functional programming -- languages like Clojure and Scala exist precisely to raise the ceiling of complexity a human can handle[6] -- and as such I'm trying to learn this stuff and trying to find people via the community who care to hire engineers with these skills. Unfortunately my own bias may be blinding me, you never know which side of Dunning-Kruger[5] you're on until it's too late.
If you care about these things: I'd love to know who you are and what you're working on, email me.
Engineering is a combination of technology and art. Therefore, it is impossible to have a simple process for interviewing engineers that magically works. Otherwise, such posts would not have been brought up so many times in HN.
Also, some people are good at people evaluation, some are not. An excellent engineer could be a lousy interviewer. On the other hand, a mediocre engineer is lack of the ability to evaluate a top engineer within an hour. Therefore, the employer must first know who are his/her best interviewers. If s/he can't identify her/his own people, I highly doubt s/he is able to identify good candidates.
The hiring signals I pay attention to are one's curiosity and one's depth in understanding in at least one programming language or one computer science topic, no matter if that topic or programming language has any relevance to the position to be filled. Here is the rationale. To be a very good engineer, one needs both internal motivation and intelligence. Both of the two signals speak for internal motivation. A deep understanding on any topic shows the candidate is sufficiently intelligent. Such a candidate, even if s/he knows little about the languages and/or the frameworks the position requires, s/he would learn it very fast and would be very good at it. To be realistic, it is really not hard to learn a programming language and a framework.
I think software has a lot of reusable principles/concepts. And these days syntax is really skin. Programming languages get used a lot because their ecosystem.
You are definitely very correct to demand perfection in at least one walk of our profession. Because those concepts get reused nearly everywhere.
Another thing that I don't understand is rejecting people merely because they don't know answers to some questions from the algorithm and data structures text book. Software engineering today is so much about so many things.
Above all I would say productivity and passion is the only factors I would use to judge people today. Because those factors decide nearly every other factor.
> man, this got 2 downvotes in 2 minutes, cmon guys i put a lot of thought into this!
That really sucks because there is some very good stuff in here.
One thing that I find strange is to assume that those that are really good at this would have the time to maintain a stackoverflow profile. Likely they're too busy raking in the $. On the other hand start-ups likely can't afford those guys anyway.
Until they drown in a 150K loc tangle that makes spaghetti look structured.
>>Much harder to interview for the very best, the mythical 10x productivity programmers[2], those who can handle 100k LOC, 1M, or more. Perhaps this is the difference between an experienced non-expert and a real expert[3].
Only that the 10x productive programmer deosn't quite really have time in his day job to memorize and master arcane facts and puzzles.
And since it turns out the big Web brands are all about this facts and puzzles in their initial rounds. This means they are missing out on nearly all 10x productive programmers.
well, you have to figure that the true 10x programmers are probably smart enough to figure out how to attract and close other 10x programmers. so either a) the puzzle-people aren't 10x programmers so who cares, or b) they are, and we don't understand their process because their process isn't designed to find us. i mean, i tend to go with (a), just sayin.
Jane Street Capital (world-class FP shop) is famous for two things: a) having a ridiculously hard interview, and b) hiring the best of the best. their interviews are blogged all over google. they ask everything from "three bags - apples, oranges, mixed; all mislabeled; how many guesses do you need to label them properly" to "implement a perl-style regular expression parser. on the whiteboard. in 45 minutes."[1]
i'm not sayin' i dig the puzzles, but I've read a few of their whitepapers, they are definitely better engineers than I am. and if i ever decide to interview again (once upon a time i insta flunked a phone interview), you can bet i will be a godly puzzle master. no matter the outcome, i bet i learn something about why they ask the questions they do.
But so far the best people in the industry I have seen so far have no time for puzzles and facts. They have better things to do and build. And they are busy enough and generally have enough of the actual work, to keep themselves busy enough to be adding meaningful value.
I now follow the same principle. I've cut down on my social network interactions. Once a day email communication. No more fact hunting. No more evaluation of every single open source software that gets released. No more worrying about language wars. No more text book reading etc.
I plan my actual work in a GTD model. I execute in pomodoro style. All aligned towards getting work done and keeping me busy and productive all day.
I have received amazing results. I've developed new hobbies like music. I spend meaningful time on things like meditation and exercise in whatever spare time remains.
I am for sure not going back to any puzzle and algorithm shopping any more. My experience reveals provided I can remain a order of magnitude more productive then others everything else will get automatically get taken care of.
On another note, the work environment is a crowded open space with people constantly yelling to each other. They advertise a 50 hour work week with no lunch break. My impression is that your base salary is just over $100k, without any indication of bonuses for software engineers.
If (emphasis on if) that is true, the only reason I can see people going there is a false sense of elitism.
I'm very interested in this topic too. Aside from functional programming, I think examining how existing systems were designed is another good way to improve. I'm taking a class[1] where we read a bunch of classic papers on systems design, and I think it's been helpful. I just did a bit of googling, and it appears that MIT has a similar course[2] with a free online textbook -- I'm going to check it out later.
(I haven't architected a really large codebase before, so take all this with a grain of salt...)
> it appears that MIT has a similar course[2] with a free online textbook
lol. you're looking for SICP[1] and if you're interested in this stuff, and you haven't read it yet, you should a) buy it and burn all your other books so as not to distract you, b) google "sicp site:news.ycombinator.com" to convince yourself i'm not full of hot air, and c) email me so i can get you in on my online participate-as-you-have-time discussion group which is not yet organized but we will start in a week or two ;)
I am interested in functional programming, because it's interesting. Nothing to do with managing complexity. And I have a few friends interested in exotic declarative languages becasue of "coolness factor", not because they need such languages in their job.
I also have a few experiences with writing hobby projects in functional languages, which never went anywhere, and what I have finished, I did using the simplest tools I had on my hand (turbo pascal, python, java, javascript).
So I think interest in functional languages isn't really good predictor of anything.
Yeah, but give me a language (e.g. - Perl, etc) that lets me make closures, instead of anonymous inner classes that implement an interface with exactly one method, any day. Fascist OOP can lead to some real code bloat, and I suspect that is starting to dawn on the leadership of this industry.
A bit tangential, but it's something I am reminded of day in and out trudging through Java code at work, and one of the "quick payback" reasons I'd like any new guy to understand FP.
(oh, and Python and JavaScript are, at least in a "compatible style", functional programming languages)
Good luck hiring someone who is merely unhappy in their current job.
If someone is in a full-time job, it is hard enough to find time to go to interview. Good luck finding a free week to do a side project for you. They will self-select out of your selection pool, yet they are very valuable possibilities. Ditto for a probation period, why would someone with a job go work for 3 months for the possibility of maybe getting a different job?
Yet that said, I don't think this is bad advice. A company will see many candidates that could have worked out, the challenge is making sure you don't wind up with a promising looking dud. A high rate of false negatives are OK, so long as you don't get a false positive.
the challenges you describe are, i think, why networking is really the only viable way to find the best talent - self-selection can work in your favor, if you're doing it right.
One of the leaders here in the Philadelphia tech community, Kyle Burton, talks about networking on HN here[1] - he is running a Clojure startup and has had little difficulty hiring due to the time he spent cultivating the local community. he hosts his own clojure meetup which it appears he can hire from at will. edit: i just emailed him to belatedly ask permission to quote him, maybe he will chime in here.
"I was actively recruited to the team. I personally believe that this was because I made a conscious effort in the fall of 2008 to start being much more active in the community at large. I started doing more visibile things on the internet: a blog, putting up a sandbox on github. I also started speaking at local user groups (god bless them for listening to my first talks and my horrible, horrendous presentation skills). ...
"By 'recruited into' I mean that not in the sense that either side used a recruiter. I mean I was approached through my network because of the effort I put into building the netowrk and the effort I put into building a personal brand (as slimy as that sounds, it is working). ...
"Actively working at networking has been wonderful. I can't recommend it enough. I was involved in starting a "tech breakfast" meeting that takes place 1x a month. The local groups (esp volunteering to speak), the breakfast sessions, and buying people lunch (an hour of interesting conversation is totally worth the $10, and every time it has come back to me) has gone a long way to building a local network."
I'm agreed on the week of work, since as you say, no one in a proper job is going to take a week of holiday to come and work for someone who may or may not employ them.
Probation however is a standard clause on every UK employment contract I've seen, usually with a 3-6 month period, during which either side can get out without penalty if it isn't working out.
As much as people like to deride #1 (here and elsewhere) it's one of the most useful filters (in terms of filter value for effort spent). It's not a positive filter (in that a candidate who passes won't necessarily be a great programmer) but it is a great negative filter (a candidate who fails almost certainly isn't).
#2 I have mixed feelings about. A lot of people work on things where the source code (even the product) isn't visible. Having an SO profile is semi-useful but lack of one doesn't really mean anything. Github only makes sense if you work on open source (and frankly I'm tired of the incredibly naive "Github is the new CV" postings that pop up every few weeks).
#5 is what I really object to. At least in this case you've gone through the initial filters. Some companies use an audition project as an initial filter before you've ever spoken to anyone (you know, to see if you'd be a cultural fit and so on). I have zero time for this.
The problem with an audition project is that anyone truly good won't have to jump through those kinds of hoops. They'll have their pick of offers as is. Job seeking through a traditional (non-network) route is relatively low percentage such that there's only so much time worth investing in any particular position.
I'm happy to prove to you know I have a solid enough foundation in data structures and algorithms, I can program my way out of a paper bag and I can problem solve and communicate effectively but as soon as you ask me to spend a week--even a day--on some audition project, forget it.
Honestly, hiring isn't that hard. You just need people who are good at hiring to do it. I've done 10 or so "lunch interviews" at Google this year. This isn't an interview per se. It's simply taking someone to lunch in between their onsite interviews. It's a chance for the candidate to unwind, ask some questions, etc. But there is no feedback element to the process.
Of the 10 after 10 minutes I predicted 8 wouldn't get hired, 1 would and 1 I wasn't sure about. Turns out the "yes" declined an offer and the "maybe" didn't get one and I was otherwise right. And this is simply from going to lunch with them.
My formula is:
1. Filter early. "Hello world" and other simple programming tests;
2. Establish technical foundation and problem-solving ability (with one algorithm-related question that should be coded); and
3. Otherwise ascertaining personality/cultural fit. This you should get just from talking to someone for 10-15 minutes.
It's really not that hard. You just need someone who can actually do it to do it.
#2 is indeed a mixed bag. Hopefully the interviewer realized that only source code samples are a good positive indicator of programming prowess, the rest are fluff, nice to have, but fluff none the less.
The best programmer (I estimate he is the 10x at least) that I know, has no SO profile, no tumblr, no twitter. Pretty sure, he has not committed any patches to open source projects in recent years. He comes at work, gets more done in 5 hours than a team of 4-5 people, then he leaves to go home to his family. He is the one who can handle 100k project with ease (and probably 1million project). His knowledge of code and the ability to travel move up/down abstraction chain is astounding, he knows more about my source code than me, and he only has glanced on it a few times.
If he was looking for a new job, he would not be able to show the closed source he is working on, as it is a large embedded project.
Granted this is an edge case, but there are programmers like him out there.
Actually there are a lot of programmers like these. I have a relative of mine, who has around 20+ experience in embedded software/hardware areas.
He has made tons of cash, built amazing stuff. Run and sold a start up and is currently running a small company. He has enough cash to retire, but he is one of the folks to whom retirement doesn't mean not working. He doesn't have a blog, his idea of social network is probably his linkedin profile, nothing more. And he doesn't harp and tweet about every single discovery he is making every two hours(which is pretty common today). He just works like crazy, gets work done, makes his money as silently as he can.
There are a lot of folks like these, but they are hidden, they keep to themselves. Make enough money doing what they like. And are generally at places they want to work at. You can't really interview such people. Because they choose you, you don't get to choose them.
So clearly common interviewing techniques are not for guys like these. And most of these guys don't work at big Web giants.
Sorry, we are now stretching the term "portfolio". Your link is what I would call an item in your resume, with a quick summary about which you would be ready to talk during the interview.
I think when Jeff Atwood says "portfolio" he means not just things you'd put on your resume, but source code and publicly available articles you've written, and for some reason a twitter account.
But not just "I developed that closed source project, over there." That's what everyone has on their resume.
-- Ninja Edit---
Actually I just looked at Jeff's other article, and apparently what he calls a "portfolio", is what I call a well written resume.
I have a relative of mine, who has around 20+ experience in embedded software/hardware areas
How exactly is he going to put up an online portfolio, or a link to a website, when he works in embedded software/hardware. Not everything programming related is done for the web!
Besides, your link that you have provided means nothing since I can't see your source code... unless you are going for a job as a front end coder of course.
Sounds exactly like a senior sw engineer I work with. The guy is incredibly productive and an excellent mentor. He hates social networks, doesn't have a github account, and all his best code (and mine) are locked up in private IP that can't be thrown into a portfolio.
I'd hope that not having a portfolio isn't a deal breaker in Jeff's hiring approach. My co-worker would be able to give any interviewer a very interesting and detailed discussion about past projects.
Absolutely there are competent programmers with no hobby code out there. I know, I was one of them when I applied for my current gig.
But if a candidate with the skills for the job really wants it very much, then merely not having any dusty code laying around readily at had should do absolutely nothing to prevent them from being able to deliver a sample, given a few days' notice.
For my part, when they asked for sample code, I said, "Hey, I usually leave my work at work, but if you give me a day I can whip something up for you." And I spent the evening grinding out code, and was able to deliver a tarball the next day.
I agree with your point on #2. A github or stack overflow profile is a great measure for graduates, but not as a general filter for experienced professionals.
as soon as you ask me to spend a week--even a day--on some audition project, forget it.
Note that he proposes the audition project to be "a regular consulting gig with an hourly rate", spent on-site doing real work.
You don't have time to earn money doing real work? Then why are you applying?
And even for a "truly good" professional who has his pick of a dozen offers, I could see this as a good idea, since they have just as much reason to be interested in the company's "real" work environment than the company has to be interested in their "real" work performance.
You don't have time to earn money doing real work? Then why are you applying?
False dichotomy. I'm already earning money doing real work. I'm not applying so that you can cut into my family time. I'm applying because I want work that's more interesting or more money or who knows what. If you really want to know, ask, instead of assuming ;)
EDIT: Clarifying the "more interesting work" expression.
For many people, their current job might not allow this (this is true especially for those who are on visas). Also, many people might not want their current employers to know about their job search, and using a week of their vacation time for an audition project with uncertain reward, is just not practical for them.
I think the audition project is a good idea for people who don't have restrictions like the above, but should not be used to reject those for whom it doesn't make sense.
My only real qualm with this is whether to bill them my consulting rate or to put up with something well below the salary I would expect for ongoing stuff.
But if it's short and interesting, it usually falls into "I don't care" and stays there.
Agreed. These "assault course" style programmer interviews are getting more ridiculous - and seem nine-parts interviewer ego stroking to one-part ascertaining candidate quality.
> I'm happy to prove to you know I have a solid enough foundation in data structures and algorithms, I can program my way out of a paper bag and I can problem solve and communicate effectively but as soon as you ask me to spend a week--even a day--on some audition project, forget it.
I think it's about balance, an who you are talking to. If i were hiring a young programmer without that much prior experience, having some hobby projects for a portfolio, that seems to be a nice fit, but i want some confirmation, the idea of an audition project seems very nice to me.
And to take an extreme example, if you are hiring Peter Norvig (lucky you), you won't even think about that audition project.
I never hired anybody, just seems the way i would do things. Also on my first computing gig, i had an audition project, and i found it very nice. I was young and happy to prove myself (still am :))
#1. Yep, great negative filter. And if I was looking for a job, I would be happy to do this.
#2. Hard. Very good programmers will often be very busy and never get a chance to open source any of their work.
I have so many open source things I'd like to work on, but I am full time employed and when I am done with work I go running, reading books, eating out, anything other than sitting in front of a computer. And I think I'm a better programmer for it.
This is why I find claims that this proves "passion" about programming very dubious.
Now I do happen to have a source forge profile, from an old job that paid me to work on an open source but not free product. It was a critical piece of infrastructure and my job was to do some very tricky things to it.
Every time I've given people a link to my profile and my patches or commit history on that project, they realize they in no way shape or form have the time or energy to grok this huge, old and complex C++ code base, to be able to evaluate my changes to it. Oops!
I end up giving them trivial python scripts I've written in an afternoon.
Also, a twitter account is definite positive? Really?
#3. Yes! As a candidate ask lots of questions and carefully listen to the answers. It's not just you who is auditioning for them!
#4. Who doesn't do this?
#5. Oh boy. If that only worked. I've worked often with contractors who have been very impressive. We almost often make then a great offer to join our team as full time employees. They always easily decline with a smile.
Because they are happy being hired guns, and don't want to go back to being employees.
Now suppose a candidate is willing and able to do this, but what happens when some other company makes them a great offer which doesn't require a week long contracting gig first?
I've been given great offers hours after walking out of the first in-person interview. Unless I am passionately, madly in love with the idea of working for your company, I will not bother. I would bother if I had no other offers, but are you hiring the best then?
And what if the candidate is looking to switch jobs, but professional ethics prevent him from slacking at this current time intensive job? Would they do your contracting thing? What if they don't, but they are in fact far and away the best developer?
I've been in that exact siltation, I've taken vacation days to do in-person interviews. But I would not take a week long contracting job just to get to the face to face interview.
#6. Unless sales is part of the job I wouldn't want "pitching". Personal chemistry with the team you'd be joining is important, but pitching does not have to be part of that evaluation.
Having visited the Google SF campus once (and knowing a number of Googlers), I picked up on a certain monoculture. This makes me question whether you were objectively picking out good and bad candidates, versus intuitively deciding whether someone will fit in the Google clique.
> deciding whether someone will fit in the Google clique
That can be shockingly important. You can be a great developer, but still not a good fit for a company. As an example, I saw several employees top-out at relatively low points at MSFT but then go to Google and have heard back that they have done shockingly well. I would bet that the reverse is also true.
Some of the best interviewers I've had on my loops were people who had a great grasp of whether a person was not only smart, but whether they would be successful long-term (e.g. if this entire division got killed, would they still do great work elsewhere in the company?).
While I can't characterize hire/no-hire decisions for old legal reasons, I can certainly say that many of the people who struggled at MSFT but excelled at Google favored closed-door remote-work style hacking over personal interaction code-design. One style is not necessarily better than the other, but for people who can only work in one mode or the other, they will find themselves career-limited in the long term if they go someplace that does not fit with their work style.
>versus intuitively deciding whether someone will fit in the Google clique.
Which I thing is also very important. Imagine making your current colleagues work with someone that just does not fit! everybody might become miserable.
Also, one point I did not see mentioned, and have not seen in any interview I have taken is code reading. If I were looking for a colleague to work in my project, I would give him two or three pages of code from my system which I think are "interesting" and ask him to tell me what it does and some other stuff he observes.
It is most likely that reading the current code is something he/she will be doing at least in the beginning, I would like to make shure she knows how to read code (extra points if it is a language she does not write in, but can grasp the idea of what happens).
If someone asked me to write a "Hello world" program in an interview I'd get up and leave (or hang up the phone). It's an insult. This would be very clear sign that I was talking to the wrong people.
A lot has to happen before a formal interview takes place. If you have to ask a guy (or gal) to write a "hello world" program it probably means that you have absolutely no clue who you are talking to and have done zero homework pre-interview.
You know what I want to see? Bring your laptop to the interview. Let's connect it to a projector and take me through some code that you've written.
Inside of 30 minutes I should know what kind of a programmer you are. In 60 minutes I probably have enough data to decode your DNA and tell you what you are going to die from.
What the hell are you going to learn from printf("Hello world")?
I don't mind being asked "Hello World" or fizz-buzz, so long as things ramp up quickly from there.
If I'm interviewing for a senior position, and the interviewer insists on slogging through a bunch of trivia questions about algorithms and data structures, or, $DIETY forbid, trick questions, then that's usually an indication I'm wasting my time.
But I won't ever blame somebody for starting with the assumption that I might be a complete idiot, because I've seen too many people that could talk a good talk, but somehow couldn't code anything to save their life.
Aye, I'll do a FizzBuzz style question, but it totally changes the dynamic of the interview. Now the onus is on them to sell me the role. And it's going to be uphill.
Despite what I said up above, I have to admit it sometimes has that effect on me as well--although in almost all those cases FizzBuzz is followed up with more questions that aren't much more difficult.
I have to disagree with you. This isn't supposed to insult you and it can be prior to the in person interview.
I did an interview once and asked them to interchange the value of two variables in any language they wanted. In the end maybe 2 out of the 20 applicants that past our HR dept checks got through this.
This one only one question, I there was much harder questions on my test but none that I or another person on the team couldn't answer in less than 1 minute. It was a disaster and I ended getting forced to hire someone from that pool anyway. That hire didn't mesh well at all.
>I did an interview once and asked them to interchange the value of two variables in any language they wanted. In the end maybe 2 out of the 20 applicants that past our HR dept checks got through this.
That sounds insane!?! What kind of background did these guys have that let them througt the HR process?
No, but the post I refered to specifically mentioned a HR department and the coding problem was very simple leading me to believe they had absolutely no coding experience.
I guess my point is that there has to be something wrong with the hiring process if 2 out of 20 applicants can't swap the values of two variables.
Either that or there's something very seriously wrong with CS programs. How can someone walk into an interview with a degree and fail something like that?
If the quality of candidates is that bad, then, OK, maybe I'm wrong about the "hello world" test. In my reality someone wouldn't even get an interview without evidence of prior non-trivial work. Hence my rejection of the "hello world" test.
I also recognize that if you are hiring massive numbers of people you need to mechanize things a bit. I couldn't see sitting down with someone, their laptop and a projector for an hour under those circumstances.
I posit that some of it is just people lying. I had someone send in a huge perl script he claimed to be his own work yet failed that variable swapping test. Basically our setup was HR gets a bunch of resumes and scans for minimum requirements (degree, prior experience) then we get them. You get really good looking resumes that you need to prove somehow. That's where that basic litmus test comes in.
Things like Github improve this a bit although I'm not even a huge user of public github repos (something I'm working on fixing since I realize my online profile is a bit sparse at the moment). I don't it's perfect either as a lot of very good candidates don't have time or are shy to show they work in public. Heck I love working on my professional project in my spare time and can't show that code publicly most of the time.
> Either that or there's something very seriously wrong with CS programs
I've been involved in hiring. I've seen top uni CS grads fail (not borderline - train wreck) a programming exercise not much more complicated than fizzbuzz. And a variation: cheat on the exercise. I've interviewed people who clearly were not the authors of the code sample they'd submitted.
We administer the test online and the candidate can take it on their own time. A decent programmer can easily complete it in less than 20 minutes, a good one in five, and judging from the proportion of people who get it wrong, I'd argue it's a good filter to make sure we're not wasting each others time. And I think it goes both ways: It costs us very little to evaluate a response, so we can filter very lightly at the top of the funnel. The first round of interviews are much more costly, so if they were the first point of contact, we'd have to filter much harder on CVs, which isn't always fair to candidates who didn't take the beaten path (and excessively fair to candidate with good degrees).
It should be mentioned that we skip the programming exercise if a candidate come through a personal recommendation.
The way I read that, 2 out of 20 succeeded rather than failed. This doesn't greatly surprise me. Rockstar salaries have attracted so many idiots and frauds that we have to sift a massive number of candidates to find any that aren't useless. And the only evidence I could accept is someone I know and trust saying they personally saw you get through non-trivial work. Anyone else needs to show me they can solve a toy problem they didn't come in prepared for.
I once interviewed a CS graduate who couldn't even tell me what an integer was. I was tempted to write a letter of complain to the university in question.
I'll add to that. I am far more interested in how a programmer thinks and how they might attack and evolve a problem. For example, let's say that they wrote some GA code. Show me the first version. Explain it. Then show me the evolution of the code and how (and why) you optimized it and what results you got. Tell me where you made mistakes and why. Tell me what you'd do differently next time.
That sort of thing would tell me a lot more about someone than the procedure delineated in the article.
I see what you're saying, but if it helps filter out that much cruft from the interview pool then I think its harmless, as long as the stupid questions end there.
However, if someone sought me out on linkedin or Stack overflow careers and put this in front of me ... I'd facepalm them
"This should be a regular consulting gig with an hourly rate, and a clearly defined project mission statement."
I interviewed with a YC company about a year ago and was asked to do #5 except without pay. It's a long story, but in short I let my guard down because it was a YC company only to find myself having wasted a week. The part that really pissed me off is that they came back _months_ later and pretended that no time had passed at all (obviously their hire didn't work out) and asked me to jump through more interviewing hoops. When I rightfully just told them to put money where mouth is they actually insulted me and the free work I had done (written in an email no less).
The point I'm making is that if you need this many hoops for a candidate to jump through you shouldn't be the one hiring. Find someone else with a business-tuned intuition and ability to judge character. If you act like you're the world's greatest company which people will waste hours and even days trying to impress you're going to miss the best people out there.
Also #2:
"Show me a Stack Overflow profile where I can see what kind of communicator and problem solver you are. Link me to an open-source code repository of your stuff."
Kinda cheesy for Jeff to say the first part. I absolutely agree with supporting both info sharing sites and open source. But as a self-taught programmer that ships products and does very well financially, I can tell you that there are effective people out there that don't rely on SO to get things done and therefore probably don't contribute to it.
I do absolutely support open source software but I also don't have a lot of free time. I've written a raster to vector conversion library that I'd love to clean up a bit and open source. But I literally don't have time to do so. Does that make me a second class programmer? Having hired many people over the years I know that I would look very seriously at someone who has shipped a lot of products but doesn't follow programmer community trends like SO. I guess it could mean they're truly horrible programmers with no knowledge to share. It could also mean they're too busy meeting their own (or some company's) targets. It could also mean they like to unplug once in a while and lead a normal life.
I understand why companies like Google take a hardline approach to hiring. They just deal with too many people. But if you're reading Jeff's essay on hiring you're not running Google. If you're a small company you need to look at the outliers first since that where you're likely to find overlooked talent. That's where you're going to find the people who don't jump through hoops but who actually get shit done on a daily basis.
#5 is still a bad rule. It eliminates anyone who currently has a job. That leaves you with people who don't have a job (for whatever reason) or are working as a freelancer.
Freelancers really don't want to work at a company, and in my experience, it doesn't work out. I'm thinking of one person in particular who was an amazing coder, got along great with everyone, and all that... But he wasn't happy there. He quit before his 3 months review came.
Last time I interviewed, 1 of the companies asked me to write a sample app that was useless, but would show I knew what I was doing. I had 48 hours to do it. So yeah, I lost a weekend (well, 16 hours of it) but they were able to know I was capable without a lot of craziness. I actually ended up taking a job somewhere else before my interview with them, though. I had found a better paying job at another company that didn't do any of that stuff. It was just a 1-on-1 interview and discussion of my past work. I'm still here a year later.
I've mentioned this before on HN, I've been given 'homework' after the whiteboard-interview. If they aren't following this:
This should be a regular consulting gig with an hourly rate, and a clearly defined project mission statement.
, then run run away. I ended up doing my not-so-little assignment but I realized he just wanted the unit of work, not an actual employee.
If anyone is in a similar situation and you have the time to do the work, you could do it. Then come in with your laptop with your code and show it to them in the next interview. Don't let someone take advantage of you.
Frankly speaking I interviewed for a bigMegaCorp last year which is pretty famous. Can't give out the name of the company for obvious reasons.
They first had a telephonic round and checked around my skills. They talked of my last projects and make me write some code. It went well. Then they invited me to have some onsite interviews. It was the most pathetic interview experience any body could ever have.
They gave me a laptop and made me code. Which I did. The first day they had three 1.5 hours session coding session based interviews. Of which I missed only on one question. They went pretty fine. They told two days later I need to go through more interviews, It was a another two coding sessions plus a database and common design pattern based interview which I also did well.
But they called in a week later and said I needed to come again. Frankly speaking my patience had ran out. But nevertheless I still went. Only to find I had to go through another three rounds. This time the experience was pathetic. Its was full of puzzles and memorizing arcane facts of various bits and pieces of software which I bet no productive programmer will ever have time for. It was puzzles and fact quizzing galore like I had never seen before. Coupled by one Algorithm interview and I bet they asked mathematics I had never heard of before.
Another week later they told me I was rejected. Now this is what I have a problem with. You want somebody who can build quality software, who knows his trade. Programming languages, Databases, Tools, design patterns, Quality and other daily stuff.
How the hell does it matter the guy doesn't know facts and puzzles? Frankly speaking I can learn that too! If I spend 30 mins a day reading the career cup ebook and other internet puzzle forums I can very well game the algorithm and puzzle rounds. But what will this every say about my skills as a programmer?
All this 'Github as a resume' , and 'Stackoverflow profile as a resume' work fine only in forums like these. Otherwise every interview has a puzzle and algorithm quiz round which has absolutely no relevance to the daily work of a programmer.
Honestly all big web companies claiming to hire the best are definitely hiring the most knowledgeable person. But none of them are hiring outliers, passionate and productive people who can do miracles. This also perfectly explains, why most of their innovation and growth happens through acquisitions and not in house innovations.
Because most companies have people who know a lot, but not necessarily who can do a lot.
These seem like mostly good ideas. I do see some problems with things like GitHub and SO rep becoming so important in the filtering/hiring process - it gives a huge advantage to those who work at companies with good open source policies. Open source contributions and contributions to community resources like SO can be useful, but I hope the lack of them doesn't cost too many people shots at interviews - there are a lot of good developers with no presence on SO or GitHub.
I do, however, have a bigger problem with #5. It seems fine at first glance, provided the company is being honest about it, but from the perspective of the potential employee, it is hard to know if performing well on the 'audition' is really all that is standing between you and the job.
About 8 years or so ago, I was applying for a new job. The interviews all went pretty well, and I was given an audition project that was aligned with this companies product very tightly. I spent several evenings (probably ~8 hours total) working on it and submitted what I thought was a pretty good project. After I followed up with them a while later, I was told that while my submission was the best one they'd seen, they weren't going to be extending me an offer.
I sort of wondered if they were using applicants to build their product, but it seems like it was a lot of overhead on their part. Either way, I wasn't pleased that I'd been given a project to work on even though I apparently wasn't going to get hired anyway.
One thing I've really struggled with since coming to San Francisco is that every company seems happy to take extensive time out of a programmer for an interview, including an audition project.
Yet if you ask this of a designer, you get shot down immediately for soliciting spec work.
I think part of this may be that since programmers typically make more money and tend to be more valued, the upside for them of doing spec work is much higher.
Even if salaries for developers and designers are comparable in startups, this is in-general not true from what I've seen, so designers as a community have banded together to stigmatize doing spec work.
I'm explaining this situation, and how I think we might have gotten into it. To paraphrase a W3C spec: this comment is not normative.
Jeff's suggestion - perhaps not followed by everyone who does an "audition project" - is that the project be paid. (Spec work, of course, isn't guaranteed to be).
Programmers seem to work for free on Linux and other projects, while designers seem to be more eager to get paid. Given these common memes, people may think that programmers are eager to do any kind of work for fun.
The audition project is pretty unheard of. It's time consuming on both ends especially if it's a week or more. What's more common is the 4 hour assignment, which annoys me to no end anyways.
In any technical domain, it's hard to identify the right people, even you base your hiring process on recommondation letters as it is done over here in Germany. These letters, and CVs and degrees as far as that is concerned, only get you so far.
After thinking about it (and deleting my first comment), I see it as some good basic advice for first time employers. What I don't really get is the point with the trial project... Do start-ups and highly qualified experts (supposed you actually want experts) really have that much time to waste? I don't think so.
One important point mentioned in the comments was culture, and yes it is paramount. Especially in disruptive, small companies where you cannot afford to have the average well-adopted worker drone around you usually get from HR in bigger companies, ideally the founders are around to communicate the culture themselves. Ouliers actually are a good place to start looking if don't already know your candidates.
Personally, I like interviews whith people who actually have a say in the hiring, only HR usually sucks. Added benfit: people knowing their job usually can tell after professional discussions (and some tests that can be done in a couple of minutes to identify the liers) if the prospective employee knows his job or not.
Disclaimer: Non-american non-programmer commenting, so please be kind :-)
p.s.: Treat them with respect, because if you don't you'll only get the desperate and they tend to be desperate for a reason.
I don't really agree with this, as usual. If you follow these directions, you might end up ok but you can definitely miss a lot of good people, and yes I'm aware of the meme about being happy to miss Knuth in order to avoid any blubs.
I also think it's not practical for a lot of companies. As well, I know around these parts it is almost sacrosanct that one should have a Github, possibly blog, etc but there are many great programmers that do not have such a portfolio. I mean, many of these programmers are very good, deeply technical, and work on quite serious things.
As far as #5, you will definitely miss out on people there. Many people with real talent actually have bills to pay and companies willing to pay them without some trial project. I understand the reasoning here, I mean, wouldn't it be great if we could go a step further and get someone to work for free for a year before we start paying them, just to make sure they are in it for the "long haul."
#6, I have no problems with; it's quite reasonable.
Honestly, I have followed Atwood for years, how many of these things would he actually pass, besides the thing about being very public? - no ad hominem, purely tangential.
I'd probably be at most risk of not passing the phone screen, depending on how anal the interviewer is (read: whether they are a hard-core C programmer type). E.g. particularly if they asked coding questions about pointers and so forth. I've always hated C, ever since I tried to learn it in high school back in 1986 or so. The big picture stuff about bits and bytes and data structures I think I'd do OK on.
Okay, to get this out of the way first: I completely agree with Jeff that hiring programmers is hard and that the way a lot of companies go about doing it is about as effective as Russian Roulette. I've seen it and felt some of the dismal consequences. As a matter of fact, I still have to work with lots of those people that should have been filtered out in step 1. So yeah, I agree with the idea.
Having said that, his process, taken together, is ridiculously one-sided. The message it sends comes across as "Of course you want to work for us, regardless of all the hassle we're putting you through, who wouldn't? Only someone out of their mind, that's who! And we don't want people like that here."
Think about it, Jeff. You are asking for someone who:
1) is highly skilled
2) has an attractive portfolio
3) is mature enough to communicate well
A lot of people like that already have jobs. If you want them to do #5 -- which might be something they're forbidden by their current employment contract, by the way -- and you also want them to prepare a presentation for #6, you must be the sexiest employer on the Earth.
In short, what do you propose to balance things out?
This is simply unreasonable. Those days off work come from somewhere, usually paid or unpaid vacation time. In other words, you're either cutting into the time reserved for your family/social life or you're paying out of your pocket.
>Why do an audition project when you can fire candidates who do not work out within the first three months?
Because then good people won't come work for you.
I would not interview at a company that had a reputation of firing a significant portion of its employees within 3 months. Not just because of the obvious career and financial risk. But because a company that treats its employees as disposable is likely not to be a good place to work in general. And I doubt I'm the only one who feels this way.
Well it shouldn't be a high occurrence affair. I mean sure if they keep getting rid of people after 3 months I would avoid that company too. But keeping the option and testing the person over 3 months is pretty standard in every job I've taken. I've only seen it used once and everyone on the team agreed with the decision.
Well, if you end up firing a lot of people, give recruiting to someone who's a better judge of character.
Firing when things do not work out is an honest act. What does it have to do with disposability? Both parties will be better off moving in separate directions.
Also, I don't see how an audition will improve anything.
Life changes are the biggest risk to employees performance. No matter what you do, you are not going to predict life changes during the interview.
>Well, if you end up firing a lot of people, give recruiting to someone who's a better judge of character.
It's not just about judging character. It's about the amount of effort an employer puts into the interviewing and hiring process. Interviewing people well takes effort. But if you're willing to fire someone easily, well, why put in the effort?
In my experience, this feeds back on itself. The more comfortable the employer gets with firing, the sloppier the hiring process gets. Soon good people start avoiding the company, which means mediocre people get interviewed. So the interview standards need to drop further, otherwise no one would get hired at all (and besides, you can just fire them if it doesn't work out, so what's the harm?). Pretty soon you're one of those companies with constant churn, where no one seems to last longer than a year.
>Firing when things do not work out is an honest act. What does it have to do with disposability?
It's not about honesty, it's about empathy. A good employer empathizes enough with the employee enough so that they find firing to be painful, even if the firing is justified. But an employer who cannot, or will not, empathize with their employee basically treats their employee as a thing. And if they treat employees as things when it comes to firing, they'll treat them as things all the time. Which is an excellent way to create a terrible work environment.
But how an audition will improve anything? The marginal improvement in the new hire quality that you get from doing an audition is not worth the time and money spent on auditions plus the cost of lost candidates who took offers that did not require auditions.
Actually, I'm not advocating auditions. I'm also really not a fan of the StackOverflow/GitHub/you-shouldn't-have-a-life-outside-of-coding approach either. But just because that approach is overkill doesn't mean that interviews should be be treated lightly (and I think treating interviews lightly is unavoidable if an employer considers firing to be a routine tool).
This is often times the case. However, you loose several weeks doing the audition and you loose a few good candidates who took the offers that did not require the audition.
General Counsel gets a fat paycheck to take care of that.
Avoiding litigations is simple too - do not hire dicks; ever.
If you hired a decent person in the first place, and after two months things are clearly not working out, you can part ways in a civil fashion.
It's not always that simple. What if that person quit their last job to come work here? Quit and moved across the country? Gave up other promising opportunities?
I'm not saying you can't fire people, but unless a trial period is explicitly agreed upon beforehand, there's an expectation that a job will last more than a couple of months if the new employee is putting in real effort. I don't know where to draw the line, but even an employee who isn't a dick might sue if they rearranged their life only to be let go two months later; I'd argue that they also deserve to win.
I was first employee in a start up before I even finished high school. A real start up, with backing from a top VC and an amazing founding team.
On the interview I looked eager to start getting shit done. I was super enthusiastic, asked hard questions, trash talked technologies - I looked like I actually knew what I was doing!
Then I went home and started to learn all those three letter acronyms they talked about.
You have to check that people can actually code. I'm not talking about algorithms, I'm talking about string truncation, url shortening service, time_ago_in_words - some programmers do know how to oversell themselves.
Though I'm not saying there aren't good points here, there are many, I think it's interesting to consider the usual impact of discussions on this subject - typically a great deal of insecurity/defensiveness (I feel it too), which I don't mean to say in a negative light, in fact I think it's quite natural.
It's worth quoting Steve Yegge's fantastic 'get that job at google' [1]:-
'Me: blah blah blah, I like asking question X in interviews, blah blah blah...
You: Question X? Oh man, I haven't heard about X since college! I've never needed it for my job! He asks that in interviews? But that means someone out there thinks it's important to know, and, and... I don't know it! If they detect my ignorance, not only will I be summarily fired for incompetence without so much as a thank-you, I will also be unemployable by people who ask question X! If people listen to Stevey, that will be everyone! I will become homeless and destitute! For not knowing something I've never needed before! This is horrible! I would attack X itself, except that I do not want to pick up a book and figure enough out about it to discredit it. Clearly I must yell a lot about how stupid Stevey is so that nobody will listen to him!'
The problem with stack overflow is that I ask for more questions than I answer.
There's a few reasons for this.
1) Finding time, although I can easily justify asking questions on SO at work it doesn't seem quite right using my employers time to answer somebodies (probably coursework) question and it's something I'd have a hard time justifying to the MBA types when there is other work that I could be doing. So my only option would be to go back home and specifically go on to answer questions.
2) Finding good questions to answer can be difficult since I only use a relatively small toolbox of software that is not particularly popular and I usually have more luck discussing it on google groups than on SO.
So that only leaves me with the easy lowest common denominator questions that have either already been answered or don't look terribly impressive to answer just as a way to earn points.
I would rather write a case study of developing something and problems that I solved along the way and publish this on a blog, of course I would have to be careful doing this about work things as well.
1) Coding. The candidate has to write some simple code, with correct syntax, in C, C++, or Java.
2) OO design. The candidate has to define basic OO concepts, and come up with classes to model a simple problem.
3) Scripting and regexes. The candidate has to describe how to find the phone numbers in 50,000 HTML pages.
4) Data structures. The candidate has to demonstrate basic knowledge of the most common data structures.
5) Bits and bytes. The candidate has to answer simple questions about bits, bytes, and binary numbers.
They even use the same regex example. He attributes other things to Steve, so maybe this is just an oversight, but I'd like to see another link.
As someone who has not yet worked professionally as a programmer (nor hired one), I might not be the best person to comment on this article (so take what I write with a grain of salt). But as someone aspiring to do so, I find this article and others like it on HN frustrating.
For one thing, as mbenjaminsmith noted, for small (and I'd argue medium-sized) companies, setting up so many hoops for a programmer to jump through doesn't strike me as a very smart strategy -- you'll likely end up with very few candidates. But more importantly, the "hoops" listed in the article mostly filter for a particular type of candidate. In fact, the whole "jump through hoops" approach to hiring itself frustrates me -- not because I don't think I could jump through them, but because of the place I would end up in if I did.
I come to programming with (what I consider to be) a fairly diverse background. One of the things that I immediately sense in recently having crossed over (in my free time, currently) into coding and into the profession of coding is how homogeneous the community tends to be: on the surface (for example) almost exclusively male and dominated by a relatively narrow demographic, but deeper down also lacking in a diversity of life experiences.
I would argue that the process described in this article is partly responsible for maintaining this lack of diversity, by the simple fact that it encourages the "hoops" mentality.
Why do programmers think asking if Oct 31 and Dec 25 are the same day is funny? I have no idea. Can I find the largest int in an array? Sure, but why in the world are you asking me this?
And FizzBuzz (linked in #1): yes I get it, it's easy and it filters out 199 out of 200 candidates, but what about those 199 other candidates? Is it not possible that among them, you might have missed other candidates whose broader set of experiences would have taught you something that you didn't know, something more important than the fact they failed FizzBuzz, for whatever inconsequential reasons?
People have such a diversity of skills and experiences. Sure, a portfolio can express some of this. Selecting from your community (#3) also strikes me as very sensible. But to find the outlier candidates who have broader potential and knowledge of problems outside the "canon" of standard programming practice, I think you have to take a more open-ended approach.
In the end, I guess maybe the problem for me is that the whole "How to Hire a Programmer" idea itself strikes me as somewhat misguided. A guide to "hiring a <fill_in_the_blank>" is only ever going to filter for cookie-cutter representations of <fill_in_the_blank>, whatever that profession may be.
> Why do programmers think asking if Oct 31 and Dec 25 are the same day is funny? I have no idea. Can I find the largest int in an array? Sure, but why in the world are you asking me this?
The first question is silly. It's probably intended as a shibboleth, but it's a pretty useless one: people who can answer it probably understand what is octal/decimal/hex, but lots of people who do know octal/decimal/hex have never heard of that silly joke.
The second question is very easy and it's good as a first filter: if you can't answer it then it's very very likely you cannot code. They're asking it because there are indeed people who interview for that position yet cannot answer it.
> And FizzBuzz (linked in #1): yes I get it, it's easy and it filters out 199 out of 200 candidates, but what about those 199 other candidates? Is it not possible that among them, you might have missed other candidates whose broader set of experiences would have taught you something that you didn't know, something more important than the fact they failed FizzBuzz, for whatever inconsequential reasons?
I think the 1/200 ratio is rather hyperbolic and not based on any actual data. In my experience interviewing candidates for a not very well-known company (so we didn't have 1000s of candidates for every position, more like dozens), more than 2/3 of the candidates would pass the really easy questions.
But yes, if the candidate fails Fizzbuzz then it's probably pointless to continue the interview. Of course the candidate might have other skills, but we're talking about interviewing for a programmer position.
> But yes, if the candidate fails Fizzbuzz then it's probably pointless to continue the interview.
I didn't really mean to imply that basic programming skills are not a prerequisite for a programming job, which I guess is what the second quote of mine above seems to imply. What I'm saying is that testing itself shapes the type of candidates you'll get, not just in the way that testing is meant to (i.e. screening out the ones that are not qualified) but also in other, unintended ways.
If the first thing you do in your interview is check if I can pass FizzBuzz, what does that say to me about your company? Tests like fizzbuzz are so uninspiring and uncreative. Aside from the fact that some strong candidates may just blank on them even though they can program fine, the deeper issue is that testing in this way says to me as an interviewee that you see candidates in terms of black-and-white "testable" skills.
> In the end, I guess maybe the problem for me is that the whole "How to Hire a Programmer" idea itself strikes me as somewhat misguided. A guide to "hiring a <fill_in_the_blank>" is only ever going to filter for cookie-cutter representations of <fill_in_the_blank>, whatever that profession may be.
If your team needs a programmer, and you're looking to hire a programmer, then it doesn't matter how knowledgeable, well-rounded, or otherwise interesting someone is, if they're not actually capable of programming.
I don't think anyone would consider it a failing of the interview process if they were looking for an accountant and their hiring process weeded out the people who couldn't actually do accounting.
> I don't think anyone would consider it a failing of the interview process if they were looking for an accountant and their hiring process weeded out the people who couldn't actually do accounting.
Not to knock the accountants, but I think programming involves a lot more creativity and open-ended thinking, and I'm not at all convinced that a focus on "hoop"-like testing brings out those aspects of a potential candidate.
(As noted in my other replies, I didn't mean to imply that programming skills are not important for a programmer to have.)
> And FizzBuzz (linked in #1): yes I get it, it's easy and it filters out 199 out of 200 candidates, but what about those 199 other candidates? Is it not possible that among them, you might have missed other candidates whose broader set of experiences would have taught you something that you didn't know, something more important than the fact they failed FizzBuzz, for whatever inconsequential reasons?
No, absolutely not. If you cannot code fizzbuzz or find the largest int in an array, you will not succeed in a job where you have to program computers (regardless of whether "programmer" is the job title).
It should be mentioned that fizzbuzz is not about your ability to write perfect code down to every last semi-colon in the first pass on the whiteboard or your ability to navigate a maze of secret trap-doors, it's about checking that you know and understand "for" and "if". That's all. Really. I'd be happy to explain modulo to you if that's the problem. If you have a bug in your code, I'll give you hints, not do my evil cackle and fail you.
In hindsight I think I ended up emphasizing the wrong thing in the quote above. Of course being able to handle "for" and "if" is going to be a prerequisite to any programming job, I'm not disputing that. Maybe FizzBuzz is a good test for that, assuming the candidate doesn't go blank under pressure and that the tester is someone as understanding as yourself, willing to give hints to fix bugs, etc.
But the test itself is not neutral: it sends a message about the employer's priorities, conveys an image of what the ideal candidate should be. It really doesn't matter if the employer thinks of it as merely a screening test. Once you're testing something, you're making a statement.
And as a statement, "tests" like FizzBuzz are pretty off-putting and uninspiring, to me at least and (I think) to a lot of potentially strong candidates. I can see that in a large company with thousands of applicants, this kind of thing is handy as a way to cut down the numbers, but I don't get the sense (in general) that the costs of doing things this way are considered or debated enough.
So, we administer an online coding test that's similar in complexity to fizzbuzz, but a different problem. If you come to us through a personal reference, we skip the test. If we come to you, because we've learned about you, we skip the test.
If you come to us, and we don't know each other, you'll expect to be screened. Why would you not prefer writing a tiny bit of code, rather than do the "guess what we want to read on your CV" game? Especially when we tell you it's actually a good filter?
> If you come to us through a personal reference, we skip the test. If we come to you, because we've learned about you, we skip the test.
Sure, and how about add to that: if we can (reliably) evaluate your skills based on work you've done, we skip the test. That together would already significantly cut down the number of people who would have to take the test, no? And I can imagine other ways to cut it down further.
> Why would you not prefer writing a tiny bit of code, rather than do the "guess what we want to read on your CV" game? Especially when we tell you it's actually a good filter?
It's the aggregate result of testing like this that bothers me. I'm just not convinced that you can write a "good filter" in the format of fizzbuzz, by which I mean a filter which does not result (as a side-effect) in attracting (and favouring) a certain type of candidate, with a certain way of thinking and world view -- beyond what you are actually explicitly testing for.
Maybe that's a worthwhile trade-off for a large organization which has to deal with huge numbers of applicants, I understand that. But for smaller companies and startups, where creative thinking and original ideas, problem awareness etc. are important, I just can't see how fizzbuzz-like tests should be a priority.
What's the aggregate result of testing for fizzbuzz? If you pass you go to the next stage, if not, you're out? The point of fizzbuzz is that it is so simple that if you are capable of any programming at all, you cannot not pass it.
When I say "filter", I'm talking about a negative filter: It will sort out the ones that can't program at all, it won't say anything at all about the ones that can.
Perhaps 1 out of 200 is a little high, someone else mentioned 2 out of 20, which is probably closer to home for us. In order to give as many as possible of the 20 a fair chance, we need the 18 rejections to be as effort-efficient as possible. Evaluating a strangers Github portfolio is significantly more work than evaluating his answer to a problem the evaluators are familiar with.
Your assertions about start-ups, large companies and creativity confuse me. We are slightly larger than a large start-up, but we very much want and screen for creativity, ideas and problem awareness. But if you can't code a solution to the very simplest imaginable of problems, we do not want you, and I don't see how it's not suicide for a start-up to think differently.
The biggest difference in this regard between us and a start-up is that we have had some time and resources to develop, tune and make consistent our recruitment process. Do we miss good ones? It would be weird if we didn't. But a bad hire is orders of magnitude more damaging than missing a good hire. And that's twice as true for a start-up with smaller resource buffers to absorb the damage.
> What's the aggregate result of testing for fizzbuzz? If you pass you go to the next stage, if not, you're out? The point of fizzbuzz is that it is so simple that if you are capable of any programming at all, you cannot not pass it.
My assertion is that an aggregate result of fizzbuzz and co. is to narrow the field in areas other than what you're explicitly testing for.
"It will sort out the ones that can't program at all, it won't say anything at all about the ones that can." <- yes, and it will also say a lot to all candidates (for good or bad) about your priorities in hiring. Maybe that's fine, but for me at least (and judging from other comments in this thread, others as well) it would be a turn off. So you would potentially lose me and others as candidates, even though we would pass the test. That's my point.
> But if you can't code a solution to the very simplest imaginable of problems, we do not want you, and I don't see how it's not suicide for a start-up to think differently.
I'm sure you know much more about hiring for startups than I do. But in a very general sense, I don't see screening tests as encouraging creativity, if anything they do the opposite. Others have made similar points in this thread. Maybe they're necessary, but they have costs.
I see your point though about a startup not having the same resources to absorb the damage from a bad hire.
> Is it not possible that among them, you might have missed other candidates whose broader set of experiences would have taught you something that you didn't know, something more important than the fact they failed FizzBuzz, for whatever inconsequential reasons?
Sure, it's possible, but that's the best case scenario. What makes you think it's more probable than the alternative? Hiring someone who cannot program for a programmer role seems to have many potential drawbacks, have you considered them? When your money is on the line, you either learn to make decisions based on probabilities and expected values rather than mere possibilities, or you lose your money with high probability.
> Hiring someone who cannot program for a programmer role seems to have many potential drawbacks, have you considered them?
My problem is with the focus on testing, I didn't mean to imply that programmers don't need to have basic programming skills. The quote above was misleading -- sorry about that -- see my clarifications in other replies.
Worth thinking about it, the thing with the homogeneous nature. Also important during imterviews for programmers or engineers is if they know the market and / or product. It's quite a difference if you developing an iOS app game or some fancy cloud based ERP-sofware. This example works equaly well with ICE drive trains for cars or helicopter rotor blades. Programmer ain't euqal programmer. But that would mean that a trial project actually could help... I should think about it...
I see posts like this regularly on HN (lists of stuff you should do to hire that top 1% of talent). What I don't recall seeing, though, is a post more along the lines of employment branding. I understand how for very small start-ups that may just be a waste of time but when you get to a certain point it seems like you're going to want to do something. That might just be having a labs.yourCompany.com blog, or releasing an open-source project once in a while. I think that great devs want to work places where they know the work is interesting/fun/whatever. It's a lot easier for you to get the word out via say a blog, rather than have a 1-1 conversation with someone over the phone to communicate the cool stuff you're doing. Just my 2 cents.
Totally agree with judging people based on their portfolio, however a lack of StackOverflow achievement currency or github commits doesn't signal much.
I can see all sorts of ways that this will give you false negatives.
This is what I would recommend:
1. Via an online test: check if they can code something very very simple. (This is your threshold that simply exists to not waste so much time.)
2. In their CV: Look for public-facing indicators of skill: github account, stackoverflow account, blog posts, etc. (Bias towards candidates with these indicators, but do not discriminate against those without. These are indicators of true positives and not true negatives.)
3. In a phone interview: ask about and then assess their breadth of knowledge. Find out their passion.
4. In a phone interview: test the depth of their knowledge in a relevant category of knowledge. If they are struggling then attempt a question in something they're passionate about. (This is the point where you detect true negatives.)
5. In the interview room: give them a choice of hard problems -- the sort you would want them to work on if they joined. (This is where you detect true positives.)
6. In the interview room: find out whether they're the right cultural fit for your company. (This is where you filter out any true negatives that remain.)
> Instead, I have my own theory about how we should interview programmers: have the candidate give a 15 minute presentation on their area of expertise. ... The one thing every programmer should know, per Steve Yegge, is how to market yourself, your code, and your project. I wholeheartedly agree. Now pitch me!
Yikes!
This really looks like an anti-test. I'd rather work with people who don't know how to market themselves -- that marketing skill would distort every technical discussion I had with them.
Even if that is a skill you want, a programmer who's really bad will almost certainly know how to bluff for 15 minutes -- they've been practicing their entire career! -- whereas a good programmer hit with that will probably wobble a bit looking for a topic.
Even if you have a extraordinary bullshit detector, it's still a poor signal. It won't tell you what the candidate is weak at -- and that's often the most important thing to determine.
Actually this is entirely how I would expect to go about it.
I recently looked into the way my alma mater hires IT folks and they give you a set of around 6 essay questions. Things like "Please tell me about two projects which best define your ability in terms of X." X might be network engineer, web developer, etc. For grins and giggles (and to help me figure out my hiring process) I went ahead and filled it out. The result was essentially the sort of portfolio that they were asking about.
I do agree that you want to see someone's work but I don't think it needs to be an audition project. Even a properly done portfolio takes a bit of time to put together.
It's funny to see a similar blog do exactly the kind of SEO targeting I've been doing: http://gun.io/blog/how-to-hire-a-programmer/ Same title and everything. Hooray Google Keyword Tool! (The content is different, this is about full time employees and not freelancers.)
I'm glad to see that he recommends _actually paying_ the programmer doing the trial period! I see this kind of article a lot, the advice is always the same, but often they suggest the trial be unpaid.
Interview is a two-way process. The company must know who are their good interviewers and who are not. A not-so-good interviewer may not only give false-positive feedbacks but also contributes to pass really good candidates. To be worse, a lousy interviewer may produce a negative image to the company and actually drive a good candidate away.
I think many people may have the experience that an interviewer was so rude and arrogant that no way would you join the company and work with the interviewer.
I have issues with both #2 and #5, having just gone through the getting hired process myself. I think a portfolio is good, but I can't stand this "let's look at Github and Stack Overflow" attitude. To my interviews I brought four projects that I plan to sell one day and one project that I worked on but wasn't mine, so these projects are not open source and I don't like this trend of employers expecting all this open source code. I have real working programs in my portfolio, that should be good enough, if I show you the code you would just nitpick on it anyways.
Give them an audition project: Just no. The suggestions in the article definitely help, but audition projects as I've seen them are horrible, and even with the suggestions, you still need tons of time that you probably don't have unless you are unemployed. Maybe if it's an hour test-like thing, I had a good test project with a company that was an hour long, it tested me on a programming language I had never used, I guess it was more similar to a hello world programming example than anything else.
However my other experience with audition projects was the week type. It was two questions, given BEFORE the first interview as a pre-screen. The first was a 2 page single spaced paper on an economic question. This was about 10% the length of the major research paper I had to do around the same time, all I could think of was how much better my time would be spent doing that.
The coding question was far worse, something about triangulating a trapped miner using his cell phone records. There were so many things not explained that I would need, like the API of the cell phone records or how the data was stored that I felt like either I would have to ask the guy for more information than he would give or that it would be totally fake. Oh, and on top of all that, they gave me a deadline of the work week, not even a weekend. I was in school at that time and only had a few hours on weekday nights.
From my personal experience (which very well may be colored by the one example above), I would have to say that projects as part of a hiring process are for desperate candidates. It will screen out the dumb or lazy ones, but it will also screen out the skilled ones because they will figure it's not worth their time trying to do tons of work for a company that very well may not give them a job, and that the time would be better spent doing something REAL (and not unpaid work for the company either), maybe even a project to put in a portfolio and show off to all companies they are interviewing for.
If you don't already have a job, there is little risk. However, if you have to quit your job to "try out" another job, you most likely want a little more assurance that you'll still have a job in a few months.
That strategy works great in a down economy when jobs are hard to find. But most good programmers have jobs, and are not going to leave a full-time job for a short-term contract.
I always see these kind of hiring practices put into place and enforced AFTER I'm already hired and working. (no, I don't conjure up hiring practices like these.) Simply put - I wouldn't have gotten my own job if I had to conform to these standards.
I've had a few interview process that have included some or all of these 6 steps, and I've turned down every one of their offers at the end. Not because I don't have the time to work on a sample project,or because I can't cobble together and isolate some sample code for them to take a look at, and not because I don't know how to play the interview game.
It is because your interview process takes too damn long. One or more phone screens, one or more in person interviews, then a request for a sample project or source code, then a request for references, then a meeting with team, etc. That takes months, with weeks of down time in between.
If you can't figure out if a person is a good fit socially and is a competent programmer in less time that, you suck at hiring and shouldn't be doing it.