I've been using whiteboards for 2 decades during the interview process and they work great, if you ask the right questions. I've built great engineering teams at companies like Orbtiz and whiteboards were one of our best tools.
The reason they are important is that they are part of every engineers day-to-day work. We use them to collaborate and design and understand.
Therefore, if a candidate can't whiteboard a design, solution, architecture, etc. in a manner that other engineers can understand it, they are going to struggle.
That being said, the way Google and other companies use whiteboards is an epic fail. Asking someone to code on a whiteboard is stupid. Asking them to solve algorithm puzzlers on a whiteboard is lame. This isn't how we use whiteboards every day so why should a candidate be expected to use them this way during an interview.
An important thing to remember is that broken/not-broken is completely different for each side of that desk: candidate person & hiring person. This is always the case for selection steps.
The employer (or school, etc) wants to make successful hires and to avoid bad ones. False positives are always a big problem. False negatives are usually much smaller one. A process is good if it produces good hires, and no bad ones.
In principle, a process that hires 3 good candidates from a pool containing 5 good candidates and lots of bad ones is good. Avoiding bad hires is paramount. Missing 2 good hires is not as bad as making one bad hire, not near as bad usually.
Most employers don't care about false negatives at all. If they do, they probably care most about missing really good outlier hires, not mistakenly selecting the 5th best candidate instead of the 4th. The only reason to care is fairness. People instintively care about fairness so this affects practices but it's not nearly like false positives.
Employers that need huge volume (eg, the army) are the only ones that care about false positives as much as candidates do.
The candidate (student etc.) generally feels the process is "broken" when it is unfair to them. Unfair means that a person could have done the job well, but cannot get hired. Ie, if the candidate is a "false negative" then the process is broken.
Anyway... When things become systemic, the "brokenness" becomes acute to a candidate. If whiteboards (your architecture version, Google's CS version or any other version) are prevelant and you suck at them... you can't get hired. There are people who suck at your test (whatever it is) but would be good at the job. Your proxy is a biased against some people, every proxy is.
Like I said, a lot of things are like this. One side is working the numbers game, the other side isn't. Insurance markets, college entrance, executive appointments, PayPal's fraudster detection (bastards!), credit scores, investment (even startup investment)...
I agree that most companies don't care about false negatives.
The problem is that very few companies know how much false negatives are costing them, so it is very difficult to evaluate whether the false negatives are—cough—a negligible tradeoff against weeding out the false positives.
False positives are right in your face. But it is nearly impossible to systematically keep track of the candidates that aren't hired to see if they are successful elsewhere.
So people lean towards eliminating false positives at all costs. It's another variation on "Looking for the watch where the light is better."
It's another variation on "Looking for the watch where the light is better."
This really doesn't fit. If there are dozens of watches scattered around, of course you first look where the light is good.
These situations aren't at all symmetric. A false positive always costs you (the company). A false negative is only a cost to you if you fail to hire well for that position.
Also the fact that a candidate you passed on does well elsewhere doesn't mean you were wrong to pass on them (it could, but it's not clear).
I was not equating this to a scenario where you are only looking for one watch. The expression (and anecdote behind it) is also used to describe managing according to metrics that are easy to collect, often at the expense of outcomes.
That applies in the case of caring more about false positives than false negatives, since nobody (or close enough) even knows how many false negatives they have.
Your statement that someone being successful elsewhere doesn't prove we have a false negative underscores the uncertainty involved. It doesn't invalidate what I'm saying in the least.
But if we want to be rigorous, we could do some direct testing. Let's say we give a telephone screening and four interviews to each candidate. Simpkifying somewhat, let's say that our policy is that a fail on any one eliminates the candidate.
If we hire lots of people, we can collect false positive and false negative stats by randomly passing someone who fails exactly one test.
If a lot of people who fail a particular test end up succeeding, we can infer that the test provides false negatives.
On the other hand, if everyone who gets the random "free pass" on one particular test ends up failing, we can presume it has low false negatives, and maybe abandon the random passes for that test.
I understood the analogy you were trying to draw, I just disagree with it.
The original joke/anectdote works because it leads to clearly suboptimal search - the guy knows his watch isn't near the streetlight, but he's looking there because he can see.
This is a very different situation. To abuse the analogy further, the guys best bet to find a watch seems to be just looking near the streetlight. After all, not only can you see better there, but most of the watches are near there too.
What your opponent is saying is that the cost of false negatives is unknown because it's hard to calculate. So people only calculate the cost of of false positive because it's easy to calculate then compare it to the unknown value and conclude that it's greater, not dissimilar to the gentleman who had been looking for the watch under a streetlight.
Your saying "false negative is only a cost to you if you fail to hire well for that position" could be easily disproved with examples of people, who were rejected and started their own competitive business (e.g. [1]). IMHO, one of the goals of having 10Ks of engineers on staff is to keep them from the competition. But since nobody is tracking this and the extreme cases like WhatsApp can be hand-waved away it's easy to pretend that the false negative's cost is miniscule.
If I’d seen any evidence that this cost wasn’t minuscule int the typical case, I’d be much more interested in expending the effort to try and measure it better and act on that - but that has an opportunity cost also. So you are suggesting incur a real cost just in case? I don’t buy it.
The guy with the watch analogy only works if there is good reason to believe he’d be better off searching elsewhere. But we don’t appear to have that here....
I am not sure you checked out the link I've given. It's quite a substantial evidence that not hiring Koum and Acton might have costed FB somewhere around $19B if one were to believe WhatsApp was acquired to neutralize competition and not for its amazing revenue.
Let's be generous and say the WhatsApp was a 10x unicorn so FB overpayed only 9B for it. Which makes 4.5B loss per false negative on each founder. If a false positive was, on average, 1M loss (which is an extremely generous in an organization such as FB in my recollection), it means you could have had 4000 false positives for each of these negatives and still come ahead.
And, it brings us back to the Type I and II errors - it's always cheaper to make one, which has a limited downside than the unknown (potentially very high one). Typical example given is that while you are on the way to an expensive cruise you realize that you might have forgotten to turn off the stove. If you go back you might miss your 10K cruise. If you cary on, you might lose you 1M house. On average, people who miss cruise in such circumstances fare much better than ones, who risk their houses.
I know that story - but think it is enough of an outlier (regardless of how it went down) to basically be irrelevant to almost all hires. The idea that people are routinely filtering out great hires in their thousands in ways that come back to haunt them ... doesn’t seem plausible.
I get what you guys are saying, there just doesn’t seem to be real evidence that is an issue nearly all cases. Whatsapp notwithstanding
No, the idea is that you don't know what you are filtering out. As I said - it's extremely easy to just hand-wave. And if you want to believe that you know the unknown downside I am not the one to persuade you otherwise. But acknowledging that it's just a personal belief not only unsubstantiated but contradicted by data could be a good first step :)
I think we are talking past each other. I have never said I know the unknown, that would be silly. But funds,entsllu can’t avoid false negatives without degrading other performance , assuming some reasonable things. Even without those assumption, there is a very real cost to what you are suggesting. You seem to be waving off this cost.
So in order for it to make sense to spend that money, you want some evidence there will be a positive return. Far from contradicting my position, what limited data I have seen does not support that effort, at least in the general case. Certainly whatsapp and the like don’t speak to it - they are very much outliers.
By all means try it. If your false positive rate explodes it will be an expensive lesson, but that’s always a risk.
One last point, the idea that nobody thinks about this and try’s thinks is laughable , which is worth noting because some comments in this thread almost suggest that it’s true,.
With my example of sacrificing a 10K cruise for a chance to not lose a 1M home, I figure?
> Far from contradicting my position, what limited data I have seen does not support that effort, at least in the general case.
Indeed, the data does not contradict a statement "on average, false positives are 4.5B each", which nobody has argued. However, it does contradict the statement "false negatives have any cost only when you are unable to fill the position as the result", which you have made in this thread.
>One last point, the idea that nobody thinks about this and try’s thinks is laughable , which is worth noting because some comments in this thread almost suggest that it’s true,.
Again, it would be silly to say that nobody thinks about it. Evidently, a number of people think the same way you do.
If there are dozens of watches scattered around, of course you first look where the light is good.
At first, yes. But when one has had enough experience looking for scarce commodities (of whatever kind: genuinely attractive apartments, interesting music / restaurants, romantic partners, etc) one learns that the "where the light is" search technique only gets you so far.
But it is nearly impossible to systematically keep track of the candidates that aren't hired to see if they are successful elsewhere.
LinkedIn could offer this as a service, if there was sufficient demand that they could monetise it. Every company will routinely check out every candidate’s profile anyway. They just need an automated way to check again in 6 months and see if everyone they reject is going to a competitor who is beating them in the market.
It may be understandable, but the prevailing attitude towards type I vs type II errors in tech industry hiring certainly makes me less sympathetic to claims that “we can’t find anyone to hire” and therefore governments should enact policies X,Y,&Z. If they are choosers they ought not to go begging.
>False positives are always a big problem. False negatives are usually much smaller one.
I would agree with you if the supply and demand weren't so skewed towards top developers. Do you think you are choosing between A and D people, or are you really choosing between C and D people and A and B people never walk through the door?
In my experience, there was considerable pressure on hiring managers to hire someone, because people are honestly needed. So, the risk of hiring one bad apple is offset by risk of being seen as someone who will never hire anyone (and thus will be excluded from hiring).
Then again, probably one of the worst bad apple I have seen would ace whiteboard as he was quite good at trics. He just did not like the actual work involved in software development and tended not to do work he don't like except bare minimum.
Once you hire someone you have to give them a chance to be successful. As the threshold for firing someone is (rightly) much higher than simply not hiring them, fixing a false positive is much more expensive than missing out on a good candidate.
Also you can fix some false negatives - allow people to interview again for other positions at the company. I've seen multiple people get hired on the 2nd or 3rd time around and ultimately be successful.
You have therefore eliminated everyone not interested in that from your pool. Good candidates know they are good and will avoid too much contrived hoop-jumping.
True. But I love contract to perm. As an architect either by name or responsibility, it's usually the first six months that are the hardest and take the most hours. I get paid for every single hour I work and can make good money until they realize the error of thier ways and make me permanent.
When we first got married my wife was really hestitant about me working on contract. Over the years, she's seen how much larger my checks are when I do.
I know companies do this, but I don't understand how it works. Given the competitive nature of hiring good engineers what type of candidate would accept a contract position when they probably have five other offers?
As I said below - I will at the proper hourly rate. With the right hourly rate, I can make more than enough to make up for any gap in employment and I'm in a market where jobs are a dime a dozen.
Besides, there are some opportunities where companies just want someone to come in, fix a problem and then after that it's just boring maintenance work. If it's a contract, I can come in, fix the problem, and look for a more interesting full time job without being labeled a "job hopper". I can just put on my resume that the 6 month job was a contract.
I prefer contracting but I found that more companies are willing to pay me my minimum salary than the commensurate hourly rate I ask for to make up for the possible gap in employment, unpaid time off, and self employment taxes.
But, my wife has a stable government job with decent benefits that allows me to go back and forth between contracting and permanent and allows me to take risks working at companies that pique my interest without concern about the long term viability of my job.
While I agree with you, it's also noticeable that many companies do have a broken hiring process, but they don't realize it, because things aren't bad enough yet. However, whiteboarding is not the culprit.
People complain whiteboarding because they've had bad multiple bad experiences with hiring processes that involve whiteboarding and they came to a conclusion that whiteboarding itself is the problem.
Unfortunately, you can't fix a bad hiring process by simply changing the technique. Each technique has its own pros and cons, but no technique will guarantee success.
These are the most common problems I've seen with hiring processes:
1) The people doing the hiring don't really know what they're looking for. Instead, they go by a vague, handwavy notion of whether the candidate is "a fit" or not. You have to define what "a fit" means in such terms that all of the people involve understand it, agree on it and know how to test for it.
2) The people doing the hiring don't know how to test for their criteria. Whether you're using whiteboard or a take-home project or some other testing technique, you have to design the test so that it helps you observe the traits you're looking for. [1] And you have to actually train your people to interview candidates.
3) Lack of transparency. This is a huge problem and it infests all aspects of hiring. Some recruiters won't even tell you which company they're hiring for. A lot of companies misrepresent what you would be doing for them, sometimes intentionally, but usually not. Some companies won't give you perfectly reasonable information about the job. [2] And, thanks to the litigious nature of our society, no company -- anywhere, ever -- will give the candidate honest, detailed feedback of why they were rejected.
4) The company is unreasonably demanding when it comes to candidate's personal time. The typical two-stage process -- a phone screen followed by on-site interview round -- is demanding enough, because it involves more than just the interviews. The candidate has to spend time and effort on other aspects, too: multiple phone calls with the recruiter, preparation for the interviews, travel, and juggling all of this with their personal and professional lives. Some companies don't understand this, so they think there's nothing wrong with a process that involves a take-home project, a debriefing phone call, a face-to-face conversation to test for cultural fit and a full-day on-site interview round.
Most companies don't even know that their hiring process sucks. Of those who eventually realize something is wrong, many end up cargo-culting whatever they read or heard about "more successful" cases. As things are right now, the whole situation is eerily reminiscent of the fuss and hype the industry went through with software development processes. I wouldn't be surprised to see yet another wave of parasitic consultants pop up to prey on companies by flogging their hiring process improvements and best practices.
----------
[1] This is where a lot of people's complaints go wrong. The most frequent grumble is: why does this company's interview process focus on algorithms and data structures, if you'll just end up coding some services in Java? If the company is doing things right, they're using these topics as a tool to observe and measure whatever it is they're looking for; most often, it's the candidate's understanding of computational complexity. Just because you won't be writing your own implementations of common data structures and algorithms, it doesn't mean you shouldn't know how to pick the one that won't blow up your memory or execution time.
[2] I had a case where I was interviewing with a company that gave me a print-out overview of their benefits, which included the price of each health plan, but not what it covered; when I inquired about it, they replied they wouldn't be able to give me that information until I signed the New Hire NDA.
This is exactly the point. A risk averse strategy is optimal for areas of a business that you do not have expertise in. If hiring isn't the thing you do best (supposedly your strengths are writing and selling software) then it makes sense to pick a hiring strategy that doesn't cause disruptions.
> Asking someone to code on a whiteboard is stupid.
I've started to turn this around and so far our senior candidates have loved the experience. (I only wish I had come up with the idea.)
Instead of asking to write code on a whiteboard and solve an arbitrary problem, couple of our engineers came up with an inverted problem. And ever since seeing it in practice, I've been trying to introduce it to every interview I can think of.
We write a piece of code on the whiteboard, and then ask the candidate to review it, as well as modify it to better suit the problem and context. This way the question turns into a discussion on design, constraint exploration, service expectations and externalities. Hell, sometimes it even goes into deep language design issues. Sure enough, it still measures a level coding skills but puts a much higher emphasis on understanding what the problem and its scope really is.
> This isn't how we use whiteboards every day so why should a candidate be expected to use them this way during an interview.
I work at Google, opinions are my own.
I think your assumption that Google interviews based on what you will do on the job is wrong. The reality is that for most SWE positions, we can't assume you will know the things you'd be using for the job because of all the internal tools. Thus, algorithm interviews are more or less a proxy for some combination of general intelligence and coding questions.
I'm not saying it's perfect and there are many things I don't like about it, but I think it's important to know what the goals are when making a critique of whether the process meets those goals.
> more or less a proxy for some combination of general intelligence and coding questions.
It's a proxy for one particular style of coding questions: leetcode. It's nothing more, nothing less.
Calling it a proxy for general intelligence or coding in general is a joke. That's propaganda passed around the company to convince people that the practice should be continued. It strokes the employees own egos to think they passed some magic intelligence test so it motivates them to keep repeating that it measures intelligence/skill.
Its a proxy for having practiced CS questions, not being able to handle the complexities of a modern software project. They basically want to know whether you went to a reputable college, or have a similar skillset to those who do.
Well, maybe I should have said it was an attempt at a proxy for general intelligence? For what it's worth, I agree with you. When people start studying for interview questions, then something is probably wrong..
I have a few friends from large universities (waterloo, stanford) that have left their current job to specifically study for a new job.
My assumption is that there are a lot more people who are studying for these interviews than you think, which is why the proxy is failing (or producing homogeneous results)
On the other hand, it's okay to target sufficient but not necessary. False negatives can be fine, so long as you are finding enough true positives.
Even if you are systemically missing out on a sub-pool who are good, it's not necessarily a bad thing - particularly if you are realistic about its limitations. To my mind the real problem isn't that you might miss good people (you will) but that if your filter is too constrained the good people you find might be too homogeneous.
There are no quality IQ tests that go up to 190. If a test purports to be valid up to that level it is very strong evidence that its authors are frauds or don’t understand statistics or both.
I've legitimately used whiteboards to pseudocode while discussing architecture with teammates plenty of times before. Does it compile? No. But the point is that I can more easily demonstrate the benefits of an approach with some concrete pseudocode to help grok the proposed algorithm.
As an aside, if you've never done systems design or pseudocode on a Smartboard, you're really missing out. Copy/paste, duplicating a slide, recalling earlier slides, etc. Man, that's the way to design a system with quick iteration but still no worry of losing your work.
Yes. That's essentially what I was getting at. Asking someone to code accurately on a whiteboard is nonsense.
When I interviewed at Google, they asked me if my whiteboard code would compile. I said, "give me a laptop and I'll tell you". I don't think they liked that answer. ;)
When I ask candidates if their whiteboard code compiles, it's a gentle hint that there are major structural issues, or an opportunity for them to point out the parts that are psuedocode-y. In general I don't include whether the code compiles (mostly JS, so "runs") in feedback unless it is amazingly perfect or really far off. This seems standard.
This. When interviewers feel that the question was not hard enough or that they didn't like the answer, they often fall back on "you're missing a semicolon".
Whiteboards are okay for ad-hoc lists and diagrams, but they are a poor medium for coding.
Some of the biggest productivity gains I've had over the years were related switching from "writing code down" to "thinking in code" mentality and abandoning linear, paper-like ways of thinking, which hindered me when I was in college.
For anyone who believes that paper and whiteboards are "good enough" for coding, I highly recommend watching some of Victor Bret's or Alan Kay's talks. Pretty much any of them. They consistently touch on the subject of dynamic media as a way to empower people to... well, think better for the lack of better terms.
I think you have got the wrong lesson off those guys. They are about engaging the spatio-visual systems. The whiteboard is the first step away from text towards that. Obviously they want to go a lot further than a whiteboard, but the fact that you can draw things, circle things, wipe things off makes a whiteboard a much more dynamic media than an ASCII interface. I think you would find them pro-whiteboard for thinking design.
>I think you have got the wrong lesson off those guys. They are about engaging the spatio-visual systems. The whiteboard is the first step away from text towards that.
The reason they are important is that they are part of every engineers day-to-day work. We use them to collaborate and design and understand.
Hey if they work for you, that's great.
But the simple fact is that they don't fit everyone's brain.
And that there are other tools available to accomplish precisely the same ends -- some of which have many advantages over whiteboards, in fact. Like these things known as "pen" and "paper", for example.
So this idea that they're part of "every engineer's" day-to-day work is, quite simply, disconnected from reality.
I do agree with you, however, that the problem lies not in the whiteboards themselves but in the way they've used. Which has truly been an epic fail.
> So this idea that they're part of "every engineer's" day-to-day work is, quite simply, disconnected from reality.
> Like these things known as "pen" and "paper", for example.
When you need to share this pen and paper, you use a whiteboard. I don't understand this hate for whiteboards, which are a cross-industry cross-profession standard (including medical) when collaborating.
One of the more interesting things I have noticed is that happy engineers have a little free time each day. They aren't under crushing pressure at all times. To this end, it's very common for whiteboards to accumulate drawings and doodles. This is often turned into "the elaboration game" where one doodle is built upon (by a little or a lot) every now and then. This is probably because coding is problem solving, but also because it's a creative endeavor. Engineers who are happy tend to work around other happy engineers and those engineers are all creative with free time.
* unless both the board and markers are of very high quality, the boards tend to stain over time
* as do your fingers
* and the markers dry up, and the erasers get less useful as well (if they're available at all)
* you have to erase other people's crap (or wonder / ask if it's worth saving) first
* the dizziness that results having to turn back and forth between the board and your audience (that is if you don't end up simply talking into the whiteboard most of the time)
* and from stooping to pick up the pens / erasers / pen caps all the time
* and all of this, mixed with all those indelible memories of crappy interview experiences
I'll take pen and paper any day
which are a cross-industry cross-profession standard
Along with horrible interview experiences unfortunately.
> unless both the board and markers are of very high quality, the boards tend to stain over time
Solved by getting a good office manager.
I'm not even joking. We have very large whiteboards in every room and in almost every space where people can wheel their desks in. Cleaners are happy to clean them up and make sure they are in good condition. One team recently complained that they may be too eager... (Our whiteboards are heavy glass, bolted to the wall. It probably helps.)
Office manager(s) on the other hand always keep their ears open and make sure the office has enough pristine marker packs and erasers available. Having a supply cabinet properly stocked is a cheap investment.
One of the most enjoyable things I can hear at the office is: "Let's hijack a whiteboard".
But -- getting back to the original subject -- for some reason during interviews the odds that you'll get an inadequately stocked and clean whiteboard seems to be almost laughably high.
Consistent with the observation that in many places, the seem to almost make a point of the importance of positive candidate experience in so many other ways (from poorly prepared interviewers to inappropriate / irrelevant questions to stalling and ghosting, and on and on).
Therefore, if a candidate can't whiteboard a design, solution, architecture, etc. in a manner that other engineers can understand it, they are going to struggle.
If that's the kind of whiteboarding you do, no problem. But, don't ask me to write a merge sort in C# (been there done that). If you ask me to sort a list in C#, I'm going to write:
I’ve been in too many NIH situations and when someone asks me one of these questions, often my mind isn’t on the solution. Im thinking about if there’s a reason they asked the question, and maybe this was a mistake.
I get so hung up on this it’s changed how I ask the question. The last time I asked a sort question, all I wanted was a compound comparator. And even then I couched it as “we want the rest endpoint to support sort criteria so we can do a top five list, but to sell the idea, we are going to sort in the UI for now. Give me some code to do that.”
> That being said, the way Google and other companies use whiteboards is an epic fail. Asking someone to code on a whiteboard is stupid. Asking them to solve algorithm puzzlers on a whiteboard is lame. This isn't how we use whiteboards every day so why should a candidate be expected to use them this way during an interview.
Every candidate I interview has the choice between coding on a whiteboarding, and coding in a text editor with rudimentary code formatting.
I really like it when people code in the text editor, because that's one less thing I have to transcribe in my interview evaluation.
Regardless of whether they will choose the whiteboard or the laptop, I have never, and will never mark people down for typos, confusing .length with .size, a missed semicolon, or any other petty bullshit that isn't pertinent to the problem. I'm not hiring a compiler.
Can you point to examples of how you do use whiteboards? I am a self-taught programmer with my own way of noting design strategies and having a tough time finding examples of this.
Not OP, but I use whiteboards to sketch system designs and things like that. For instance, I'd sketch out the path that a request might take through a complicated system (intake via a REST service, flows through a queue, consumed by another service, persisted to a database...). That's the sort of thing that I would be comfortable doing as part of an interview—though I wouldn't expect a junior developer to be able to do it.
AFAICT, the backlash is around interviews where candidates are asked to write code (or pseudocode) on a whiteboard. Being able to implement bubble sort on a whiteboard does not correlate very well with being able to deliver business value, in my experience.
> Being able to implement bubble sort on a whiteboard does not correlate very well with being able to deliver business value, in my experience.
To add to this, it's being able to write code on a whiteboard in a high pressure situation while talking about every step you're taking. Over time I've gotten more comfortable with this, but mainly by doing more interviews, not by getting better at my job.
If you ever find yourself in a situation that Broadcast television would have us believe, you'll be excellent at coding that hotfix into a real time system as the president of the United States looks over your shoulder.
When I was younger, this would probably make me feel insufficient but I have done enough ‘learning stuff’ by now that I expect this kind of comfort to only come with experience.
To all the responders to my query, your answers are super helpful.
Being self-taught, there are just these few holes the internet doesn’t quite cover. But a little reaching out on HN goes a long way in filling the gaps.
I think a lot of the other posters are correct. Diagrams of classes, interactions, event flows, dependencies, network architectures, and yes, even pseudo-code.
Basically, if it is an idea, concept or design, draw it out. If it is functionally correct code that compiles, screen-cast and type it out.
Last time I used a whiteboard was when I was re-architecting some classes and I mapped out the class and function names and how they interacted with each other. This enabled me to see what changes I would need to make/add to get the desired effect.
The time before that was for a database migration where I mapped out each step that had to be done and in which order.
The point is not to have or not have the whiteboard during interview but how and what it is being used for.
The best way I found to interview candidates is to have few different discussions. One of the discussions I like to have is to try and solve some kind of problem that you could potentially have in real life. I am solving problems with people this way on a daily basis so this lets me observe candidate in a setting that is already familiar to me and hopefully to the candidate. The whiteboard is then used exactly as in real life -- as an tool to visualize and communicate ideas and hold it as a kind of cache and reminder for the duration of the discussion. The whiteboard is there during interview but I will not require candidate to use it. If I have trouble understanding the idea I may ask "ok, I can't visualize this, could you maybe draw it for me?"
Nobody programs on whiteboard in real life. If I need to program for presentation, for example to train my team on some topic, I will bring laptop.
Some engineers think that whiteboarding is not clear enough ... that code snipts are better.
The key about whiteboarding is it take away all the specific syntax and gets to the heart of the understanding the logic.
Thats what collaberation is for, i think many young engineers that only code with an ide are nervious whrn they dont have autocomplete available and they lose focus on the core logic.
Programmerd used to write on paper ... this entire process (coding without the ide/search engine) is lost.
But this is the part where the programmer shoes their strength in logic ... not syntax memory.
As an aside, any resources for learning whiteboard techniques? Not specifically solving algorithms on whiteboards, but designing and conveying information during architecture planning/etc?
I feel like there's two angles to this; 1. understanding complex algorithms commonly used as tests. 2. understanding how to convey information on whiteboards efficiently, common paradigms, etc.
#2 is what I think I'd actually enjoy improving. #1 is just SAT style stuff that I infrequently use in my day job and serves as an annoyance more than anything.
Whiteboards should be used for what they're good at: Visually explaining ideas/solutions on an abstract level.
They are terrible at compiling and syntax checking.
Theoretically you should already know whether someone can do the physical code part. That problem is largely solved for and don't require in-house interview.
In-house interview should be used for c̶o̶n̶f̶i̶r̶m̶a̶t̶i̶o̶n̶ b̶i̶a̶s̶ ̶f̶o̶r̶ a̶g̶e̶/̶r̶a̶c̶e̶/̶g̶e̶n̶d̶e̶r̶ determining if the candidate can think/design/solve for the domain space they're being hired into. A whiteboard is a really good tool for assisting in that.
When I worked at http://zipcodedownload.com we constantly used whiteboards to code. That, of course, is when we had a parent company. Now that we are independent its gone by the wayside. Don't like it as well.
They seem to be starting to get that. I recently interviewed with a few companies including google and all of them provided a computer for me to type out my code in the interviews.
> the way Google and other companies use whiteboards is an epic fail.
They are optimizing completely different metrics than you do. They want crème de la crème developers that are perfectly trained not to think about certain classes of problems and waste time there but instantly come with acceptable answers, immediately recognizing the structure of the problem and having the ability to faultlessly transform it into a piece of code without a debugger. It allows the training from the best universities to shine.
The problem is when everybody starts doing the same, but without the prestige, salary, equity and lifestyle that Google provides.
Couldn’t disagree more. I hate whiteboarding and I never do it. Verbal explanations and pair programming work just fine.
I feel slowed down having to write out on a whiteboard when I could instead write some dummy code to get a point across. It just seems crazy to assume all developers everywhere must be good at white boarding.
I tend to move my hand faster than I can hand write, which is why I like typing for all forms of explanation.
Especially since I’m on a full remote team, whiteboarding would be really weird.
I can't recall ever having written actual code on a whiteboard in the course of doing actual work. I have, many times, sketched out a system's architecture or worked out some weird process using boxes and arrows.
I'm sure a shared digital sketchpad would accomplish the same thing for remote workers.
We don't whiteboard at my company, as we favor a take home technical exercise and debrief/walkthrough format for interviewing our engineers. That being said, I'm super turned off by the incessant whining from the community about various interviewing practices. Here's the tough truth I think: Some companies simply do not understand how to properly engage in employee selection. They get some quick guidance from a google search and have too much autonomy in the process to make their own decisions because the company hasn't really empowered their talent processes to be well thought out and planned. This happens frequently. The real shame in it all it that it focuses the conversation on these high level exercises - be it coding on a white board or rallying against take home exercises or demanding to be paid for interview work, and ultimately to the employer you're put in a spot where you can't help but feel a little reactionary as the transactionality of the relationship is constant: I need to know you can do the work, prove to me you can, I will then hire you and pay you to work.
What a cluster it's all becoming on the software engineering side.
And I need to know what the work exactly entails, and what is expected of me. Many interview questions are disjoint with what the job actually requires.
>Prove to me you can
I can show you my past experience, and I can do technical exercises. What I can't do is devote a ridiculous amount of time and effort to poor questions.
As someone who has hired a lot of devs over the years (almost exclusively for our offshore team in India, but sometimes in the EU), I can tell you that the number of candidates who embellish and down right lie about their capabilities and past experience is shocking.
Truth is, I hate both sides of the interview divide, as interviewer and interviewee - it's a horrible, inefficient, flawed process regardless which side you're on, and nobody has the answers.
If Indian offshoring teams constantly lie about their credentials way more than local, I would think having different interview processes for different sources of candidates would be a better option than a process for the lowest common denominator.
People outside of India often embellish or exaggerate too though, even if not close to the degree that I've experienced in India (disclaimer: this is my own opinion based on my own experiences over 10 years or so. Obviously India has a lot of great, honest Engineers, our organisation never sees them though, largely because we don't pay well enough).
A good reputation with solid references that are checked from real engineers and managers that you actually delivered a good product seems like it would suffice to override the straight up lying, to me. Is that not the case?
References are an excellent way to confirm someone that someone did actually work at a particular place in a particular role and that they are not a total asshole. Beyond that, unless you know the reference well, I think they can be unreliably.
The reference knows the candidate and likely feels a much closer connection with them than they do with you. Assuming the candidate isn't an asshole, they will want to help them out by giving them a good recommendation. If the candidate was let go due to downsizing or such, the reference may even feel pressure to help the candidate find a new job.
So it's a great data point to have and as you say, it can help weed out the straight up liars.
But any 2 given candidates with 5 years at Acme Inc are not interchangeable. Some candidates are just better at programming (or some subset of it) than others, and I think employers try to find ways to predict that at interviews.
One technique I have found works is to do incorporate your own open source code into your work. Then when they want to see samples you have commercial quality code that isn't company property.
I have an interview coming up and the process as described to me is take home exercise with an on site debrief. I am thrilled. I've done whiteboard interviews and some I've aced and some I've crashed an burned. I didn't crash and burn because I didn't know the material. I just froze on a tree traversal question one time. And then the nerves and empty conversation just made the situation snowball as I thought to myself, "OMG I know this why am I taking so long, what is this guy thinking right now." I relish the opportunity to provide useful information for the employer to update their prior.
>I'm super turned off by the incessant whining from the community about various interviewing practices.
Wow, what an incredibly tone deaf response.
>Some companies simply do not understand how to properly engage in employee selection.
Maybe changing this prospective is the point of the "incessant whining."
>I need to know you can do the work, prove to me you can, I will then hire you and pay you to work.
I'd have to know how you are testing, but I'd bet dollars to doughnuts your method has no way of predicting output in any meaningful way.
>What a cluster it's all becoming on the software engineering side.
Absolutely. I would say it's endangering the overall quality of the industry. The saying goes, if you are a bad manager or a bad company the first people to leave are the A and B players. The people that will never leave are the C and D players. I suspect people who do all this trivia don't even know what an A or B player looks like.
It's really not though. Here's the bottom line: You want money from a company. The company doesn't owe you jack. shit. If they fucking whiteboard, and you want money from them, then do the whiteboard. Later, when you are employed by them, try to suggest they improve their process. If you have better options, take them.
> Maybe changing this prospective is the point of the "incessant whining."
Sure. But is that the companies #1 priority, or do they have other things to do, like make money and stay in business that take precedent? Anyways, the way you do this is from within or providing feedback and having conversations with recruiters -- you know, human interaction. This job board is the ultimate culmination of millennial passive-aggressive behavior.
I will admit, maybe I am reading too much into the convictions or intentions of the job boards -- but we all are human, so I can only assume this was created by someone who was tired of having to do whiteboards, and rather than confront the problem head on and potentially drive industry-wide change (channeling "one person can change the world"), goes behind the scenes. The job board further fragments and confuses things for everyone. The better option, of course in my opinion, is make things better for everyone by fighting the battle where the companies are.
I am frustrated at times just as much as any other engineer by playing "monkey-see-monkey-do" in front of an audience in order to receive employment and I have indeed passed up job offers because I was turned off by the interview process (but again, that was my choice) but if you take a step back, look at the overall job market, and what other workers have to go through to get a job, we really do have it made, and this seems like a minor nuisance at best.
> rather than confront the problem head on and potentially drive industry-wide change (channeling "one person can change the world"), goes behind the scenes.
No, this is how change is driven. How do you think change happens? Getting a big committee together and getting buy in from all interested parties? That's not how it works. But maybe this is how it works, maybe this job board attracts a bunch of candidates who are tired of the status quo, and other candidates start telling the companies on other job boards that, sorry, but they're more interested in the companies that list on this particular job board, and those companies then see that the talent pool they're drawing from is smaller than it needs to be, and they decide they want to draw from the larger pool, so they change their practices, and then that snowballs, well maybe now you have industry-wide change.
I don't personally see this having much of an impact, but I think the people who started this are doing exactly what you're saying they should be doing, except that you're pillorying them for it because they aren't doing it through some impossible broad consensus approach.
> what other workers have to go through to get a job, we really do have it made, and this seems like a minor nuisance at best
It is true that we have it made, but I do not think it is true that we have it made in this sense. We have it made because our skills are very highly in demand and because of that, we are treated well and paid handsomely once we get a job. It is true that we should never ignore, down-play, or fail to appreciate this privilege, but that doesn't imply that we should not discuss or advocate for improvements where needed.
Where we don't have it made is in the predictability of employment once competence has been demonstrated. Other professions and trades usually have long-and-tedious up-front competence proving phases, but then they have a credential that will be respected by future employers when they have open positions. We are fairly unique in forcing people with long and distinguished careers to continually re-prove themselves. Our employers have open positions, but it takes a lot of pointless and duplicated busy-work for a competent person to slot into them.
I think what it comes down to is that our process is relatively mediocre, good, or even great for inexperienced unproven new entrants to the job market, but rather sub-par for experienced and proven folks. And that's why you see people looking for a better way.
I think there's two different complaints at play in this thread. Surely you're not suggesting that candidates be exempt from proving themselves. The number of developers who become incompetent over time is high. I think the question is whether they should prove themselves through algorithmic questions. I'll never higher anyone just because they have credentials. A Computer Science degree is a credential, yet so many with that credential cannot write the simplest of programs.
It's probably too late to come back to this thread, but no, I'm not talking about the narrower but more common complaint about algorithmic questions. I think the entire approach of constant proving is misguided. There are lots of other ways to do this with lots of prior art in other industries. I believe we have landed on a poor but self-perpetuating solution. What we do to hire is akin to what they do in the performing arts, but our work is entirely dissimilar.
Do mechanical engineers draw up a design for a valve or something at every job they apply for?
This might sound silly -- especially to the HN audience -- but it's akin to the baptist church problem: one group doesn't like their senior pastor, so they take about half the congregation, go down the street, and say "we will do it right this time!" and now you end up with two split churches. This is basically why there are thousands of baptist churches in the south.
Another example it's related to is the XKCD standards comic. Rather than improve existing standards, the committee breaks off with dissenters, forms a new committee, and now you get "Situation: There are now 16 competing standards"
edit: Better Solution? Talk to recruiters? Let them know you won't consider places that force you to play "dance monkey". The letting them know part is key. More broadly: maybe unionize?
It's really not though. Here's the bottom line: You want money from a company
And the company wants to make money from me. In my market, there are a lot more open positions for senior devs than senior devs that are actively looking for jobs. I
And if you have all the time in the world to play that out --great. If you have limited savings like the vast majority of Americans -- time is your enemy, and eventually rent will come due. The company can survive for months, years without your labor and if they have to, they will simply purchase it or outsource it if need be.
You need that money more than they need you. There really are few exceptions to this rule -- AI being one of them.
Again I wish it wasn't like this, but it's the reality for the overwhelmingly majority of software developers in this country.
1. That you are in an area of the country with a lot of opportunities
2. That you kept your eye on the market and made sure that your skill set in demand
3. That you kept your resume updated
4. That you kept your network of recruiters, former coworkers, and references fresh.
If you have limited savings like the vast majority of Americans -- time is your enemy
With the unemployment rate of software developers being extremely low, the chances are that someone looking for a job that you are interviewing, already has a job. When I said my turn around from looking for a job to getting a job is about three weeks, I didn't mean to imply that I was a special snowflake.
A company I was working for went through a few rounds of layoffs before everyone was let go -- this was in 2011. Without fail, every developer and even L2 support person had another comparable job within a month.
If a company makes the hiring process too long, more than likely if you are in the right market and you are just a regular old "Full Stack Developer", by the time you go through the process, you've already received a couple of offers
Besides, if I'm looking for a job and I am unemployed, the time I am taking to do homework, I can be meeting with recruiters, preparing for interviews, brushing up on my skillset, etc...
Here's the other bottom line: A company wants to use your industry and creativity to make more money than it will give to you. You don't owe the company jack. shit.
I agree with your bottom line as well. Both are true! It's a negotiation, and it isn't zero-sum. Working is good for both the employee and the company. Neither is in a universally stronger negotiating position.
I'm not annoyed by bad interview processes and strong preference for false-negatives, I'm annoyed by that in conjunction with the claim that there is a shortage of competent workers. There is some lower-hanging fruit to take care of before that claim makes sense, and improving recruiting processes is among the lowest hanging (along with raising wages, obviously).
> Here's the other bottom line: A company wants to use your industry and creativity to make more money than it will give to you. You don't owe the company jack. shit.
Unfortunately, unless the sides are favored to the labor market (in our cases it is not, and even if it was, there is always outsourcing), this equation is not equal.
Both are true, but it is an extremely unbalanced equation. At the end of the day, the company has the money and you have the skills. Unfortunately only one of those helps you pay the bills -- unless you can figure out a way to pay your rent in exchange for your skills.
This is not true for the industry we're talking about (or at least not as true as you seem to think). It really is in companies' best interest to be attractive to skilled workers in this industry and there really is a chance they will find themselves struggling if they get the wrong reputation. It seems (to me) like this is a major undercurrent in IBM's struggles, for instance (and outsourcing has not been a silver bullet). Thinking the balance is more skewed than it actually is is one of the things that skews the balance.
This whole thing misses the point. We need WhiteboardFreeAndAlsoTakeHomeFree. In other industries, my resume and a quick discussion verifying that I actually did the work I said I did is suffficient
So say you are at a job. fairly happy. Maybe not your dream job, but you decide to look around and interview with me.
I could present you two options:
1) I just hire you after a 30 minute chat. 50% you don't work out, and we fire you after 30 days.
2) We have a longer interview. We whiteboard. We take home test. We whiteboard some more. 90% chance that you don't get fired after 30 days.
Which would you prefer? As a business owner, I am happy to just try people I don't know well - IFF they are willing to accept that there is a very good chance they are fired within the first 30 days.
There is such a huge range in developers. The haves vs the have-nots can be a 100x different productivity difference. I really want to try to tease that difference out. It is really really hard.
Actually I would greatly prefer option 1. I am very confident that given nearly any situation I will prove my worth and have never had an employer disappointed with my work. I am not as confident I would complete a contrived coding skill challenge on the spot on a whiteboard because I never work like that in reality. My job is to sit here and solve unique problems by any means available to me in a maintainable and scalable way. I do that very well and that is hard to test via whiteboard implementations of contrived examples.
That is how it works in other fields. One of our managers asked his friend that owns a body shop how he finds good employees. He basically has a chat with them, feels them out, then hires them. If they don't work out he hires someone else. Perhaps it shouldn't be as stigmatic as "firing" after 30 days but you have instead hired someone on a 30 day contract with option to hire permanently. If they don't work out, that option is not exercised and that person is free to go try somewhere else. I can tell pretty quickly from chatting with a developer if they are a total fraud. I can't tell really quickly who will become a great dev, I just know when someone has the chops to have potential.
#1. I'm totally fine with firing after 30 days if I don't work out. I'm positive it won't, so I'd prefer to prove myself doing real work.
That's the way it used to be done and I think we should go back to doing that. Read the resume, ask some questions about the resume, 30 minutes to an hour full interview, done.
You get hired on, put on a probationary period, if they don't work out, fire them.
Same here. I feel I have vastly more control of my work performance over the course of 30 days than I have control of the assessment of 6 interviewers after seeing me for 30 minutes.
In other words, I think there is far more than 50% chance that things work out after 30 days and far less than 50% chance that I impress 6 interviewers.
Right, then maybe it would self select out those that can't do the job.
Interesting idea. If you fire fast enough it may not actually cost that much more money to a business to hire a bit more as long as you fire the bad ones fast.
> 2) We have a longer interview. We whiteboard. We take home test. We whiteboard some more. 90% chance that you don't get fired after 30 days.
But a 400% increase in that you find some sort of excuse not to hire me and I won't even get hired in the first place. Your percentages only make #2 more attractive if there's a more-or-less equal chance of being hired both ways.
Well, #2 makes more sense if you value stability and not suprises.
I suspect many people would rather do 10 interviews and then get 1 job they keep for a year+ compared to 2 interviews and a 50% chance the job that hire them, fires them in 30 days. But I could be wrong, mostly basing on what I would want.
I've done it many times. It's attractive if the offer is attractive. Last one I took was a $20k raise over the prior job. The one before that was a $30k raise. It's only risky if you're not confident in your skills.
No way! The lamers you end up firing (after 30 days' pay) will easily make a living out of it.
They could even be PARALLEL lamers, pulling 30 days pay off multiple sucker companies.
At my current company, we fired a guy after 3 days because he misrepresented his skills. I didn't interview or hire him, but I suspect his buddy did the interview.
It seems like they could easily drop half of those jobs off of their resume. That's usually the advice given for people who leave jobs after a short period.
I really don't buy that a resume alone can sufficiently put across your actual level of ability as a Software Engineer. And there are plenty of people who are better talkers than doers. One way or another I think it's entirely reasonable to want to see actual code.
If you want to do that by way of an open source portfolio, sure. But not everyone has open source contributions, so then I would need some other process for acquiring a work sample.
I have seen many people here say you should hire people on a contract basis for a few days/a week. I don't know what it's like doing that in the US, but for the UK market that sounds like a ton of administrative overhead for both parties to take on in order to establish if they are even a good fit.
This style of interview works in plenty of other professions that are much more rigorous than software development. You'll be hard pressed to find an aerospace engineering company that is going to ask candidates to evaluate integrals on the whiteboard that they'd normally look up in the CRC or implement a numerical solution for (probably using a prepackaged library), for example. Yet you're proposing an analog of this very thing here.
That's (among other things) what the walkthrough / debrief is for.
I like the format, but it obviously has other downsides, such as a higher up-front investment from the candidate.
That being said I found the format enjoyable last time I went through it. It was less stressful during the interview itself, and I felt more confident walking in. I would recommend it to companies who are considering it.
That would show you they have a strong network of good coders and can communicate specs and get quality code done in a timely manner. You should hire them for management!
Might be worth clarifying whether this is “don’t use a whiteboard during interviews” or “don’t ask candidates to code obscure algorithms” on a whiteboard.
We use a whiteboard in our interviews for sketching UI, and sometimes drawing a database table. Generally candidates find this much easier than talking through their design, and we think that interview very closely represents what we do on a day to day basis.
There was a good article that popped up on here a couple months ago about this: "Organizational Skills Beat Algorithmic Wizardry". [1]
If you're asking candidates to balance a binary search tree when no one in the history of your company has ever had to do that, you're asking the wrong questions.
We do a mock technical meeting with multiple members of our tech team and the candidate presenting options, talking through pros and cons, etc. Much of the interview is about assessing communication rather than technical skills.
The interview is basically a standard tech meeting that we hold regularly in the team to discuss designs of new projects.
Agreed, whiteboard are way too useful to leave out. That's why we have them in every room in the office, not to scare candidates when we happen to conduct interviews in them.
In my opinion it is all about aligning your questions with the day to day task you expect from people. If the work is very algorithm heavy, by all means probe for that in the interview.
I base my interviews around one main question: assume the tech challenge you written beforehand is a prototype for a complete product/service. what are the steps necessary to get us there and beyond?
I do not care about the candidates tech preferences or the exact solution as long as the argument for it is coherent and not just based on heresay. Then I probe for a couple of things within their solution where I see potential weaknesses based on their argument, experience or code in relation what we look for. It is more about problem solving, decision making and communication than computer science.
Candidates are free to just talk trought it or use whiteboard/paper to draw it out. But if you choose the former you better not forgett what you said five minutes ago.
I feel like it’s ok to just say “you dont have to do programming on a analog device”; if they want to write on paper/whiteboard/with a tattoo gun I don't give a damn, but they wont be asked to write actual code. That’s what it means to me.
Yeah, that's key. "Explain how your piece fit into the bigger picture— what other things does it talk to, how tightly are they coupled, how are errors propagated, etc."
I was just speaking with a co-worker about how I consider this interview culture to be a race to the bottom in how much unpaid work companies are getting from prospective employees. When I interviewed with Facebook they do back to back whiteboard interviews for about 6+ hours then sometimes they give "homework" problems, etc. Even skilled candidates can't refuse because a less skilled candidate will come along willing to make the sacrifice.
I don't have any solutions only complaints and bitterness, sorry.
Do you think Facebook is taking pictures of whiteboards after interviews and putting the code into production? I agree day-long interviews are a problem, and homework problems are offensive (plenty of folks refuse them btw), but it's a stretch to say companies are using interviews as a way to get unpaid work.
The premise is not that they're getting unpaid work in the form of programming, just unpaid work in general. If they replace 8 hours of reviewing a candidate's open source contributions and/or side projects with 4 hours of random walks through an undergrad CS textbook, then they're using the candidate's unpaid work to cut down on their HR spending. "How many round piano tuners are there in Fizzbuzz" isn't programming, but it is HR.
Do most programmers have significant amounts of open source contributions? I only had a couple of years of life when I had the luxury of enough time for that sort of thing.
When I'm looking for a job, I usually have 3 or 4 strong prospects and looking back on my spreadsheets that I keep when I am looking, it usually takes about 2 weeks from the time I reach out to my network of recruiters and I have an offer (the shortest was 4 days from looking to having an offer from what was then a top 10 Fortune 500 company).
Why would I go through that trouble? I have 20+ years of experience that I can explain how I architected and developed from the website, to the middle tier, to the database to multiple VPCs on AWS and dev ops. I'm not going to do homework.
When I was first starting out I might, but living in a major metro area that's not on the west coast, there are more openings for senior devs/architects than people to fill them.
If a company wants to try me out, I'm more than willing to do a contract to perm, but my hourly rate is going to be enough to cover self employment taxes, make up for not getting paid for time off and at least two or three weeks between employment.
We have a take home test. It is an obviously fake problem, but we review it to the grade we would for our production code.
We don't find that candidates turn down taking the test - it's capped at 3 hours so that it can't take up lots of time. We do find that we have a low pass rate, and even among candidates with 10+ years experience, we still find candidates who write code that is just not good, and who haven't read the spec.
I'm not at all suggesting that you might be in this category, and if you can find jobs you want that don't require tests that's great.
However, there are people with plenty of experience, who can architect things brilliantly, and who can't write a ~300 line program in Python in a way that satisfies a spec or in a way that anyone would be able to maintain. That's important for many companies, and so I think tests can be one useful part of an interviewing process.
> If a company wants to try me out, I'm more than willing to do a contract to perm, but my hourly rate is going to be enough to cover self employment taxes, make up for not getting paid for time off and at least two or three weeks between employment.
Take a reasonable salary for my market as $140,000, that is about $152000/1720 = $88.37/hour minimum.
Adjust the number based on market, skill set, and the amount of time you estimate between jobs. I'm also not taking into account health benefits that you would need to pay yourself because I'm covered under my wife, or 401K matching. I would add 4% to the hourly rate to cover 401K matching.
A good rule of thumb is 3x the rate you would get as a full time employee this might not be enough in the US given the high cost of health cover for a self employed person.
I'd recommend dropping the full day onsite. I mean what the hell do you talk about for a full day? The small talk must be unbearable after the first 30 minutes.
So you want a company to hire somebody after talking them for 30 minutes? Based off of what? You honestly can't imagine talking about technical software problems for more than 30 minutes? That's a pretty big red flag to me.
I'm assuming you are technical yourself, otherwise please disregard. What you are unintentionally implying is that you can't spot a fraud by talking to them about your chosen profession within 30 minutes. If you can't, you shouldn't be doing the interviewing. If you can, why waste 5.5 hours of the company's time per candidate?
Its not causation, but I'd say it correlates to being a thoughtful person. Drilling down on the architecture blabble to see how deep their understand goes will absolutely tell you whether they could code a solution.
Is there a person who claims to be a coder that has configured and launched a multi-environment microservice with several attached artifacts like DBs and queues, that knows in depth what each is doing, that can't code? At that point you're just testing whether someone has memorized syntax details that they might usually get from an IDE or a 3 second google search in regular development. I have been writing production JS for years but still have to occasionally look up Array.slice vs Array.splice parameters, especially if I haven't done a lot of data munging in a while. That may indicate to a person giving an interview that somehow I'm "faking" it, when in reality its just a momentary stall-out in my brain's lookup tables.
> Is there a person who claims to be a coder that has configured and launched a multi-environment microservice with several attached artifacts like DBs and queues, that knows in depth what each is doing, that can't code?
Funny that you mention it. I know a person who can't shut up about microservices, Kubernetes and CI and can't code their way out of paper bag. They had no problem passing one of those interviews.
And sure rote API memorization isn't the point. But if a candidate can't code say bubblesort (or any kind of sort) in 30 minutes they are not very good at coding - and it doesn't matter if they can quote their architecture patterns by heart.
> But if a candidate can't code say bubblesort (or any kind of sort) in 30 minutes they are not very good at coding
Thats bullshit. Sorry. Most developers will NEVER implement a generic sorting algorithm except in a code interview. You say you don't want rote API memorization, but you do want rote sort method implementation memorization. Yes you can be a system or tools developer and maybe have a higher chance of doing it, but its 99% likely even then that you just use a library for it for the language you're using, or copypastaing it from someone who's studied the specific implementation for your lang.
When you start probing enough it quickly becomes practical/coding interview we all come to detest. Sure enough, that way you might not even need 30 minutes. But shop talk alone doesn't cut it.
Maybe your “shop talk” is different from mine, but I can suss out fakes from the first sentence usually. You’re correct in that they go straight for the broad architecture strokes, and they speak in vague terms. Trying to get a solid grounded answer is like trying to nail jelly to a tree. That’s when you know they just read some wikipedia articles and don’t know the practical applications of the words they’re using.
There's a number of people with CS or SW engineering degrees who can't actually code. Those can often quote you Gang of Four patterns, discuss broad slices of tech and draw architecture diagrams on the board. But still can't code.
I know, I work with several. I’m pretty good at spotting them because I studied them while working with them. Fun fact: if you poke in the right places, you’ll see that they don’t even understand GoF and architecture, they just regurgitate the what they memorized.
Interesting and the first I’ve heard of this and the first confirmation I’ve heard that cost-wise it’s not egregious to do so.
It still seems strange to me to pay for throwaway work but I can also see how that’s really just my personal sentiment / bias and this is probably a pretty good and balanced approach.
Except some of us already have full-time jobs and don't want to spend the weekend working on quizzes, whether it's paid or not. And unless you're very generous with timing taking a day off of work can be a logistical issue, not just a pay issue.
This type of interview self-selects for people with fewer family/non-work obligations, who are willing and able to spend their free time still programming.
You're ignoring the context of the comment you're replying to.
That guy already has a good job. Another company is trying to poach him. Why would he, or any other qualified and gainfully employed candidate go through that unnecessary homework bullshit for a company that's trying to recruit them? The company has its priorities misplaced. They should be trying to convince the candidate to join them, not putting up barriers to their most qualified candidates.
I would totally do homework bullshit for a company/opportunity/compensation that I saw as “better” than what I have currently, but would not do so for anything else.
Insane. Let’s say it’s half that and you make $100k a year. You wouldn’t put in 2 hours for an extra 25k/yr?
I mean this isn’t a lottery, and hopefully you’ve reasoned there will be a high probability of getting hired; say 50%. So then the expected value of your time is $6.25k/hr.
Go, take the moral high ground, and don’t do the homework. That helps rational minded folk achieve an even higher hit rate.
I'm already at top dollar in my area and have been for years. Maybe that's why I despise it so, I have nothing to gain from the trouble. I haven't had to do a cold interview since 1999.
There are much better and less humiliating ways to get top dollar. Work your ass off and build reputation is the biggest one.
If you are a company and you have to call a recruiter, you're already behind. Putting people through that stuff just makes it worse.
We aren't paying for the work-product, we're paying for the time and inconvenience. It's more to show respect for the candidate's time than to actually "hire" them to do an interview.
I mean, even "discount" recruiting platforms like Hired and Vettery will cost 5-figures when we make a hire through them, and traditional recruiters often charge 20% or more of the new-hire's first year salary. A few hundred bucks to get some more information about a prospective coworker is cheap.
Reminds me of the old joke about a scam company who breaks up all of their problems into interview questions and never actually hires anyone. Obviously the composition, decomposition work and quality control would be more labor for worse results but it is easy to see how frustrated interviewees got the concept.
I recently interviewed at a company who wanted to be as hot as Google and Spotify and had copied their interview process and their organization schema.
I said no thanks and moved on to other companies. Felt good. I just avoided working in a culture where there will be high pressure to always perform. :)
Yes, companies don't realize people can get incredibly wealthy working for places like Google. That's why people tolerate that as well as the prestige of having a place like that on the resume. They are so self centered, they think Acme Trucking holds the same allure with "market rates" and shitty benefits.
I'll just point out that companies are spending a lot of employee time doing these interviews, and if they're picky, that greatly increases the cost. Every day of interviews of someone they don't hire is money down the drain.
They can afford it, but still, it's far to say that the wasted effort isn't entirely one-sided. Hiring is an expensive and inefficient transaction.
Figuring out how to make this both more fair and more efficient is a really tough problem.
So the standard FB onsite interview loop is 5 * 45 min, with a 45 min break at some point. Those aren't all whiteboard interviews. It might vary from role to role, but I don't think homework problems are common.
I did get the 45 minute lunch break and dined with another FB employee during that time, but I was in the room doing whiteboard problems the entire time otherwise minus them showing me the office space as we went up a few floors when I first arrived. This was in Seattle, if that matters. I think the room was labeled Occam's Razor and was super small, barely enough room for 3 people to stand.
So not only are you dealing with the standard fare interview nervousness, etc..., but you're also feeling claustrophobic the entire time. Brilliant process they've got up there. People look at me sideways when I tell them that I have no interest in working for places like Facebook or Amazon or Google and these sorts of anecdotes are some real high-quality validation after having had to justify that stance for years.
I would never work for one of those large companies. You really only need one household company on your resume during your career, then you can ignore them.
Those companies are the most visible places to get a high-paying job. They pay at the top of the market. Sure, a niche software company may pay its senior engineers higher, but even well-connected developers may never get any exposure to them.
If you want to get that first house downpayment for the Bay Area, I'd say it's almost essential at this point.
I think a lot of these reactions (and this website) are bitterness following a rejection. I'm not sure why people take it so personally when a company doesn't accept them. I've done whiteboard interviews and got a rejection before, and I thought it was fine. Maybe if you're putting all your eggs in the same basket and want THIS company to accept you, that might be annoying. But that's not a very good life strategy.
No, I don't think any of the code is of any value, it's the time and energy up-front investment is increasing significantly. I understand they are trying to protect their investment (2 years at Facebook would have been over $500K in cost to that company all things considered) but when is it too much and how would we stop it before it gets there?
Having gotten contacted by Facebook, I eventually refused when I realized they were going to ask about algorithms for a technical role that is in no way relevant to that on the job. Skilled candidates can refuse just fine and wait for something better and more realistic that comes along.
Whiteboards are not the problem- rather the questions that candidates are asked are bad.
Asking someone to solve an obscure mathematical problem (the Two Egg problem seems to be in fashion) doesn't do much except test whether the candidate has memorized the answer.
When not in an an interview I like playing with these kinds of problems, but if I look it up I'll probably find interview prep where it shows the answer as well.
"You are given two eggs, and access to a 100-storey building. Both eggs are identical. The aim is to find out the highest floor from which an egg will not break when dropped out of a window from that floor. If an egg is dropped and does not break, it is undamaged and can be dropped again. However, once an egg is broken, that’s it for that egg.
If an egg breaks when dropped from floor n, then it would also have broken from any floor above that. If an egg survives a fall, then it will survive any fall shorter than that.
The question is: What strategy should you adopt to minimize the number egg drops it takes to find the solution?. (And what is the worst case for the number of drops it will take?)"
Uh, if you drop an egg out of the first floor window onto cement, it will break.
The answer should be a form of binary search, but since you only have two eggs, you'd have to do a sequential search from the bottom floor. You have one chance at a mistake, so I would almost say start with the 2nd floor, but I know it will break on the first floor.
I don't see how that in any way gauges programming what so ever.
>The answer should be a form of binary search, but since you only have two eggs, you'd have to do a sequential search from the bottom floor. You have one chance at a mistake, so I would almost say start with the 2nd floor, but I know it will break on the first floor.
yeah, sorta. essentially you partition the 100 floors into some number of groups (i forget what partition length is optimal) and drop the first egg at each segment until it breaks, then go linearly from the previous one
A quick naive not completely optimal but still relevant solution:
Since you have 2 eggs you can start on the 33rd floor. It breaks you can use the 2nd egg from 1st to 32nd floor.
If it doesn't break, use the first egg on the 66th floor. If it does break, the answer is between 33 and 65, if it doesn't the answer is between 66 and 100. That cuts the maximal testing time time from splitting at 1/2 (1st egg at flr 50), to 1/3rd.
To optimize further, since in the bad case we'd have to search up to 34/35 floors, you can mathmatically find a number where you split the drops at an interval (IIRC the "real" optimal answer is 14, then 13, then 12, etc), which means you're doing less explicit checking.
At 14 - i, you have 10 drops to find a range of 13 - i floors in which the optimal path is found. So instead of a worst case of 35, you have a worst case of around 17.
I find the best way to approach this sort of problem is to first find one optimization, and then use the same process to find if there are further optimizations.
It's not the worst type of puzzle except if you do not know of a solution it might take a while to figure it out.
It does relate to programmer's ability to amortize worst case running time.
Once you realize that 1 egg requires linear search you just have to realize that with 2 eggs you have to decrease the level increase by 1 on each success.
Of course those must be some crazy genetically modified eggs.
>It's not the worst type of puzzle except if you do not know of a solution it might take a while to figure it out.
I would guess the interviewer probably rates people who have memorized the solution far higher than people who have never encountered it and solved it with their brains.
Hmmm, that's an interesting question. It's the Price is Right closest without going over solver...
Practical answer, drop from the first floor and then the second. You're probably not going to make it to the third.
I don't want to put my actual answer in case others want to solve it. But I will say there's an obvious solution, but if you apply one of the hard problems of computer science you might eke out a bit of an edge.
EDIT: Actually I don't think that's right. I think the answer varies based on your expectations, and should be tuned after each run if you run the same test multiple times. I do have a good starting point in mind though.
After thinking about it on the ride home today, I was not quite on the right track before. Again don't want to post my solution because the entire point in asking was so that people could see the question and work it through themselves.
Because eggs that don’t break when dropped even 3 ft (never mind multiple floors) are a thing? Perhaps I am just projecting my Earth based experience...
Maybe it's a shittest if you actually try to apply some theory rather than using your common sense and say that "Based on the last time I was in New York, it will definitely break dropped from the first floor"
Whiteboard problems are used to show that you can work through a problem, while communicating well about your thought process. They're also good general intelligence tests, if administered correctly.
I think an obvious reason for this type of interview problems is because developers at work usually talk loudly while solving problems and write code without the help of any external resource while the clock is ticking in the background. Oh, wait...
What both you and the parent post you're deriding are missing is white-boarding (when done correctly) isn't about the problem at all. It's just an easy way to get a candidate in the room and construct a technical conversation.
As an employer I don't care how great your skills are, if you can't walk me through your thought process for 45 minutes you're not going to be overly successful. If you can't incorporate feed back or engage when pressed to change your design you're probably not going to be successful.
Do your recruiting right and you shouldn't need the white board to simulate whether the candidate can code at all (I think white-boarding with the intent of looking for named algorithms is a waste of everyone's time), instead you want to see if they can synthesize information, engage with other people, and otherwise display the soft skills that tend to be much more useful metrics for employee success than coding aptitude.
Granted, my industry is known for not having deep coding problems to solve, this strategy might not be as useful in verticals that require more technology chops.
>As an employer I don't care how great your skills are, if you can't walk me through your thought process for 45 minutes you're not going to be overly successful. If you can't incorporate feed back or engage when pressed to change your design you're probably not going to be successful.
The post you're responding to makes a great point that this isn't the normal working environment for 99% of actually-writing-code developers. I discuss and iterate designs with my coworkers/leads/managers/architects just fine, but a surprise algorithmic problem you have to simultaneously solve, draw on a whiteboard, and constantly present to an audience while being timed isn't something I have been great at. I bet many others aren't either. I got good at it after a few interviews last time I was looking for a job; but if I hit the interview trail again today, I'm confident that I would flounder in that setting for a while.
Imagine if, instead, the candidate worked out a project -- maybe at home, or maybe in an interview room for some time -- then you take time to review and understand their work and then go through this back-and-forth process of discussing their work. This would also benefit the candidate's understanding of how you collaborate to problem solve in your work environment.
The problem with this approach is that it requires the interviewer to actually invest time to understand the candidate's work.
Anyway, I hope you see why many see it as a problem that the process for a candidate to succeed at a job interview is to practice interviewing skills rather than demonstrating their engineer skills.
> As an employer I don't care how great your skills are, if you can't walk me through your thought process for 45 minutes you're not going to be overly successful.
As an investor, I don't care how great your skills are, if you can't come up with your new business idea on the spot and walk me through your thought process as you're doing it, you probably suck.. right?
Nah.
Unless you already have the answer, the first step is to come up with the solution. This could involve time spent alone just thinking, or brainstorming with coworkers, or researching the problem on the net, etcetra. What the right steps are depend on the problem, available resources, your background, even your preferred way of working.
The second step is to present and dissect the solution. I dare say this is the point at which the majority of engineers have no problem whatsoever talking about their thought process.
Do you work in an industry where engineers genuinely have to walk somebody through their thought process while they're trying to think up a solution?
If a problem comes up at a meeting and you don't have a solution on the spot, you take note and discuss it later or add it on the agenda for the next meeting. I think that's how people work.
I don't care about where, when, and how you came up with the idea. I care that you can present and discuss it once it's ready for that.
I think this is why many prefer the take-home test. They can focus on the solution. Then, at the interview, they can focus on the talk about the solution. It's also a great equalizer as e.g. people coming to work in a domain they have less experience with can take more time to research the solution space.
The complaints about homework taking up too much time may be valid, but I'd be most happy to trade 2 hours of interviews for about 75 minutes of homework and 45 minutes of interview.
"When done correctly" is the key here. Alas, in my experience I was usually heavily penalized if I kept my mouth shut for as little as 15-20 seconds because interviewer "couldn't see my thought process". At the same time expecting from me to write syntactically and semantically correct, industrial strength bug-free code with optimum space and time performance for a problem I never ever encountered with the interviewer "helpfully" distracting me 4-5 times a minute.
Divining rods are trusted by many, and have been in use for centuries. If they weren't a good way of locating water, why would so many people use them?
> They're also good general intelligence tests, if administered correctly.
And communism probably redistributes wealth if it is administered correctly.
If companies want to hire based on general intelligence (not personality, team work, communication and so on) then they should administer IQ tests, which are based on a century of rigorous research. That would at least give you a fighting chance of measuring G.
Why do you think white boarding can measure general intelligence?
Comment: I think whiteboards get unnecessary flak when the real culprits are games and riddles. I would argue that using a whiteboard to convey ideas is a fundamental skill if one has to work in a creative team.
Question: How do you validate that the posted company or job position doesn't need whiteboard? Is it after the fact, as in a disgruntled candidate complaining? I couldn't find any information about this on the site.
> I would argue that using a whiteboard to convey ideas is a fundamental skill if one has to work in a creative team.
Yet a lot of people are never really taught to do it. Perhaps they pick up this fundamental skill just fine after a couple whiteboarding sessions in the company.
If a company is rejecting candidates for not having a skill that is really easy and quick to obtain, they'd better not be one of these companies that keep complaining about shortage of talent.
FYI All your subscriber's emails are accessible at https://whiteboardfree.com/email_subscribers.json since you didn't put any authentication on those endpoints. It's a basic Rails app so I'm guessing other users may got a hold of the list already.
So you're telling me that someone who created a site to post job adverts for jobs that don't test programming skills, has managed to write a security vulnerability into the site? Never!
I don't agree with making candidates write algorithms they don't need to know in a format that they don't need to be able to write them, but I am in favour of testing candidates thoroughly.
StackOverflow's job board has the "Joel Test" part where employers posting jobs can check off the points from https://www.joelonsoftware.com/2000/08/09/the-joel-test-12-s... that they fulfill. One of them is "quiet working conditions", which comes close. Before looking it up just now, I could have sworn the point was something like "every developer has a door they can close", but oh well.
Anyway, I hoped you could search only for jobs where this box was checked by the company, but apparently not...
EDIT: It does allow searching for minimal salary and remote jobs, among the wishes listed in a sibling comment.
> Before looking it up just now, I could have sworn the point was something like "every developer has a door they can close", but oh well.
EDIT: Ok, my mistake, my memory is failing me. "Doors that close" is used in a whole bunch of Joel blog posts, and "walls and doors" is used in the original Joel Test description.
But WaybackMachine seems to confirm that the official label from the "Joel Test" has always been "quiet working conditions" and does not appear to have been changed.
Sounds plausible, but the blog post from 2000 also has it as "Do programmers have quiet working conditions?"
So unless Joel went back and rewrote the past, it might be our memories. I am fairly sure that he did have another blog post that stated explicitly that every developer at FogCreek had a door.
Wow that would really turn the tables; a job board where companies must not only disclose their interviewing practices and durations but their office environment as well. It would be developer centric so developers can pick and choose the companies they want to work for based on often hidden parameters. Just the transparency requirements would automatically filter out most the chaff companies. I might have to build that out of spite. :)
Even better, I would love a site that classifies job offers/companies by:
- Has an onboarding process for new hires
- Has quiet working conditions / open office
- Has clear requirements
- Salary range
- Allows remote working
- Type of interview and time to get a response
Check out https://www.jobiki.com. We built it in collaboration with people like yourself to search for companies based on information like that. Whether you want companies with pod or cubed desks, or open office. (Among MANY other things)
Unfortunately we haven't yet expanded much out of the Midwest, so it MAY not be of use to you yet. But check it if your interested.
I went there, typed in "developer" in the search box, and got dropped on a map zoomed in on the Kansas/Oklahoma border and a message saying my search didn't match anything. Took me a while to notice the "zoom out on the map" part. Not sure why I landed in the middle of nowhere in the US, I'm located in Europe. In any case, super unintuitive.
Also, this might be an American thing I'm missing, but which (if any) option of "Pod Office Layout" or "Closed (Cubicle) Office Layout" corresponds to "proper walls and a proper door"?
I'd also suggest adding a filter for minimum salary.
Thanks for checking it out and your feedback! I can make some changes to hopefully clear up some confusion.
On Jobiki, we don't have job postings. Just companies and data about them. So the "minimum salary" and "developer" searches are more job related, while we are more company related. Either way, helpful feedback.
-----------------
As for the "proper walls and a proper door" aspect, are there developer/traditional employee jobs in Europe that can have a real office? In the US, unless you are a manager, executive, very experienced employee, you don't usually get an actual office. Maybe its more common in Europe?
At least in Scandinavia I'd say it's fairly common to have a real office. It seems the cubicle thing never really caught on here, but now there's a trend toward open plan offices, unfortunately. Software companies seem to be among the worst in this regard.
I have a real, quiet office with lots of daylight. That's actually one of the reasons I'm staying at this job. That, and the fact that I'm not subjected to low-status micromanagement techniques like Scrum. In short, I have working conditions befitting a qualified adult.
Individual offices for developers are unusual in the US these days, but that's exactly why someone would want that information. There are also shared offices and team rooms.
"Pod" seems to mean completely different things to different people.
"Pet friendly" workplaces tend to be noisy and discriminate against people with allergies, so some users will want to filter them out.
Even if you don't want to track current openings, what kind of jobs a company has is important information.
Thanks for the response. That's very true. We've also been toying with different verbiage for "Pods", as I would even agree, could be many different things.
And good note on the "pet friendly" environment.
We in the processes of adding that information to outline what positions a company hires for. (Whether or not they are currently open or not) Well, more what departments that a company hires for. Going all the way down to job title seems a bit too granular, maybe I am wrong? But by department seems a happy medium?
> are there developer/traditional employee jobs in Europe that can have a real office?
Yes. Maybe not every single developer; there might be two or three or four to an office, but it's still an office with walls and a door. There are companies with more open spaces and twenty developers to a room, but the (much) smaller ones are the usual case from what I've seen. ("Europe" is a big place, of course.)
That's not a bad idea. I think I'll make something like this, a site that let's you search for dev jobs with filters that devs actually care about. Any other suggestions for filters?
Wow, there are so many new ways that people are coming up with to recruit software developers. Clearly there is a problem here that needs to be solved.
I understand that the company needs a way to vet people to make sure they are technically proficient for the job. However, there must be a better way of filtering people. I just bailed on a company after they told me what the interview process would be:
- Take home coding quiz where you need to sign up and use their algorithm platform. As you would expect, there was asynchronous programming and a recursion algorithm involved (two for one!)
- 3 more technical interviews. 2 of which were screen share and/or white boarding sessions. 1 is a "riddle part", which I assume is some kind of brain teaser involving a recursive algorithm.
After I saw the coding quiz and all the follow on stuff I just told them I wasn't interested. I was kind of offended, and it didn't put me in a positive mindset towards the company and engineers that worked there.
I still whiteboard my programming question when giving interviews, can someone summarize the case against whiteboards?
I'm just looking for the candidate's thought process and some pseudocode. I could do the same thing in an empty text file but the substance of the interview wouldn't change.
My feeling right now is that being anti whiteboard is focusing on trivialities like complaining that the office style guide uses Allman vs K&R bracketing or something. But I'm open to having my mind changed if there's something more to it.
Being anti-whiteboard is a proxy for being against the "regurgitate this CS trivia on a board in front of me" style of interview, not against the use of a whiteboard.
Most problems in practice that require non-trivial application of algorithms, dynamic programming and the like have two characteristics that make them differ considerably from whiteboard style quizzes. For one, they're going to be much less contrived (pick a random leetcode question for an example of this, odds are it's contrived). Secondly, they're going to be resolved when developers talk to each other about them and there is recognition of the solution, which most of the time is an application of algorithms and data structures with library implementations.
The ability to whiteboard things like "given an array of integers and a target find the indices of the pair of integers in the array that sum to the target" and "find the first missing positive integer in an unsorted array in O(n) time and O(1) space" provides almost no useful information about a candidate's ability to solve an actual problem the company faces in most instances. There are exceptions, but interviewing everyone for the standard of that one exception is asinine.
I like to use whiteboards for non-coding design questions, like talking through how you would architect an app, drawing DB schema diagrams, etc. It's a tool that we use for collaborating on things like this in the office, so demonstrating an ability to do it is meaningful.
Actually hand-writing code with a pen has always seemed weird to me. As a programmer, I compose code with a keyboard. So I give candidates a laptop and use an interactive screensharing tool (I like https://codepad.remoteinterview.io) where they can run code in the language of their choice in a browser, and I can watch and talk through the process.
Whiteboarding is fine when done well, horrible when done poorly. The variance is on the side of the interviewers.
I've done a lot of interviews over the years, and I've encountered a couple stereotypes:
1. The gotcha interviewers. He pulled out his old algorithm text book and found the hardest problem.
2. The no bullshit interviewer. Asks you to code a somewhat challenging, but solvable problem. No tricks. Basically non-trival fizz buzz problems. The problem with these is there are only so many practical problems you can give before it gets put up on leetcode, and every interviewee has memorized the code.
3. The clueless interviewer. They try to give you a weird mix of an IQ test and a personality test, and maybe throw in a little bit of API trivia while they are at it.
Is whiteboard interviewing so bad that people would go out of their way to avoid it? I mean, I've encountered / can imagine horror stories for all the usual alternatives: a) take this 3-day long mini-project, do it using all of our preferred technologies (some of which you'll be using for the first time) and get it back to us - it should only take you 4 hours! Or b) do this on site, using our machines and environments which will be unfamiliar to you. Or c) speak to some random invigilator over Skype. Or d) do a written test, or e) take us through your GitHub collection, which may or may not exist, etc.
My preferred way of testing people was usually to try to get them to show me that they have thought about why they develop software in the fashion that they do. I would prefer to do this in person, at a very high level, but it often ended up being a written test (where necessary, the language was your choice, or pseudocode). Is this wrong?
> Is whiteboard interviewing so bad that people would go out of their way to avoid it? I mean, I've encountered / can imagine horror stories for all the usual alternatives: a) take this 3-day long mini-project, do it using all of our preferred technologies (some of which you'll be using for the first time) and get it back to us - it should only take you 4 hours! Or b) do this on site, using our machines and environments which will be unfamiliar to you. Or c) speak to some random invigilator over Skype. Or d) do a written test, or e) take us through your GitHub collection, which may or may not exist, etc.
This is the problem. There was recently a large HN thread about the take-home projects and how so many people didn't like them.
None of the options are popular: if you use whiteboarding, people will complain it's a bunch of brain teasers unrelated to real programming work. If you use take-home projects, people will complain about doing unpaid work.
Would professional certifications help? I doubt it -- they would need to be based on standardized tests, which would have the same drawbacks as the whiteboarding questions nobody likes.
To give an alternative approach - I found that in general, I could get a pretty good idea about a candidate using either the verbal interaction or written test.
The idea was to give questions which are almost a polar opposite of the strange trivia style questions you see in certification exams. Instead of the candidate having to remember which overload of an API returns a byte[], we'd instead talk about things like how they prefer to structure a project, and how this feeds into maintainability, etc. If the person was into object orientation, we'd talk about that, if they were into a more functional style, same angle, but I want them to have thought about it from a software engineering side. It sounds wishy-washy, but with just a few pointed questions, you can detect both BS answers, as well as someone who hasn't really thought about it. I tend to find that this quality in an individual is a very important predictor of performance, as it indicates deep care and interest about their craft. With good candidates, you can get really deep into a discussion and know within 5 minutes that they are quality.
Along the way, you'd also get a pretty good picture of the persons strengths and stylistic preferences, which you can use to determine how and where they would fit in with the team(s) you had in mind.
The only downsides are that I wasn't always available for these interviews, and the written equivalent intimidating and could take long. Also, various people swore that they either can't perform well in written or verbal conditions, so perhaps letting the candidate choose between the two might be beneficial.
> Is whiteboard interviewing so bad that people would go out of their way to avoid it?
Yes. If you have problems with anxiety or imposter syndrome its a special kind of hell - especially when you go in absolutely able to do the work and being qualified, and then getting a hard no on the basis of "you're not a good enough programmer". Even for someone without an ego to be harmed, it feels like a cruel and arbitrary way to determine your future financial and societal well being.
I recently had an onsite tech screen at a FANG company that requires whiteboard programming. The problem itself wasn't hard but I missed a requirement that was actually a hint. I shutdown and couldn't move on or work through it.
That said, I don't think whiteboard programming is always bad but it is a skill that I don't exercise and it showed.
My advice, talk through the problem with the interviewer and don't start writing code on the whiteboard until the interviewer has ok'd your proposed solution.
I usually need like good 5-10 minutes of quite time to think about the problem in peace. I just can't seem to do with interviewer staring at me and hounding me to 'think out loud', sorry I can't talk and think at the same time. I have no practise at 'thinking out loud' , i never do it in real life.
200 technical interviews?! What the hell? You shouldn't have to do 200 in your lifetime. This kinda reminds me of those MCSE certification bootcamps they used to have. They didn't teach you anything useful, they just taught you how to hack the test.
At plated we do two coding interviews. The first is concrete, we have a library that is poorly written and their job is to implement a new feature and refactor it. This is a great interview to understand core skills. The second interview is a modeling exercise where we build an app from scratch with escalating amount of complexity but only asking for what the database and api endpoints would look like.
My personal experience in hiring is that people should be able to explain concepts in a clear why. So usually I ask to explain an pattern that they say they have experience in. I let him/her choose how they want to explain it whiteboard / pen and paper / words only, sitting or standing. I don't care as long as the person is good in explaining and has a firm grasp of the subject. Both are needed to do a proper job. There are lots of people that are unable to explain MVC in a clear let alone correctly even if they used .NET MVC or spring MVC (or whatever)for 10+ years. After that is seldom useful to ask anything else.. this approach had never failed me.
Do people hate whiteboarding in general or the questions they're being asked to draw?
I can fully understand dislike for puzzles and being asked to implement irrelevant algorithms in pseudo code. The interview should reflect the job being interviewed for.
I've found whiteboarding to be mutually helpful in questions about system architecture. Plus, it's something that we actually do on-the-job.
Yes, but you don't write what you expect to be fully functional code on a whiteboard. A whiteboard is good for sharing ideas. Using it to write and assess code is pretty stupid if you step back and think about it.
I mean ask yourself this: do you ever use the whiteboard to write code to the extent you expect some poor kid trying to get his first job to do? Hell no, you screen share an IDE or paste code in an email.
People used to have to write code on paper (I did this a few times in college ages ago) and that comes from the cost of compiling stuff way back when. Computing time is friggin cheap, that's why no one does it. To expect some new guy to be adept at that ancient practice is silly.
I agree. This is a form of the psychological concept of encoding specificity principle[1] which states that recall is most effective when the conditions at the time of encoding match the conditions at the time of retrieval.
UI Design or a Frontend Software Engineer? I'd expect people working on the frontend to just as proficient in time complexity, space complexity, and performance analysis as a backend engineer. I don't particularly gravitate towards companies working in CRUD apps, but single page applications with complex UI flows and data collection can certainly need those abilities. For one, it's not particularly uncommon to have an actual tree in the UI. How would you represent that tree? How are you going to handle it as it grows to thousands of nodes?
I think few frontend developers would consider the work they do to be "UI design".
This all remains me of a quote I have recently seen:
"Most Programmer wouldn't hire themselves"
As long as we keep asking people to do things that aren't like the work that we do, we will continue to get mismatches and people that interview well but work poorly. I don't have a great solution, finding good people is hard.
What would be a good selection strategy without technical interviews? I've seen people with good CV and reference projects being hired, only to turn out they literally couldn't program. It's old news that sneaking to a CS degree without learning anything substantial is possible, and those are easily debunked in even simple task like solving FizzBuzz on the whiteboard.
Both of these have huge disadvantages for interviewing:
- When working in front of a computer, people get worse at communicating. Even if they try to speak out their thougts, it usually doesn't work well, as thinking and operating an IDE (or whatever at hand) already occupies a lot of brain capacity.
- Take-home exercises are hard to control: You don't know how long a candidate worked on it. Even if there's a target like 4 hours, some might work all night on it - either because they couldn't do it in the allocated time, or because they fear some disadvantage. Apart of that, they might pay someone to solve it for them.
I've been programming since BASIC on a VIC-20 sometime around age 6. I can code in like 10 languages and have written everything from crypto and network protocols to distributed systems, database sync engines, and high throughput video compression/decompression codecs.
I always fail whiteboard interviews. I can't code at all on a whiteboard.
Aside: With about 40 job postings, each at 300$/month, they're probably making $144k with little to no operational or infrastructure costs. They should be on https://www.indiehackers.com/products.
If you're interested in someones coding prowess and process, review something they have written with them. It's far more effective than commanding some nervous candidate to get in front of a group of people and start writing code on a wall.
There is a Netflix and Lyft post, but going by past experiences there are whiteboard coding problems. Are these companies spamming it? How do they police these possibly fake posts? After all, the thing recruiters care most is how many people apply.
Some proportion of perfectly good engineers couldn't pass a whiteboard interview to save their lives, for much the same reason I couldn't pass a job interview that measured communication skills by karaoke performance and emoji literacy.
The company needs to evaluate your problem-solving skills though. I mean, I'll admit that writing code on a literal whiteboard is more physically arduous than on a keyboard or even a piece of paper, but if the root issue is live coding challenges themselves, I don't see a way around it.
Some companies look at your GitHub profile instead, which imo is much more unfair, because it presumes you've had the free time and the desire to work on side projects, which shouldn't be a requirement for a job.
Maybe companies should adopt an either-or policy? Give the interviewer the option to demonstrate their skills in the form that works best for them.
I don't mind whiteboarding. I do mind questions that are puzzles or algorithmic in a way that does not directly reflect the company's current software.
I usually (ok, almost always) flesh out designs and pseudocode with pencil and paper anyway, so changing to use a whiteboard isn't an issue.
Standing in front of a handful of strangers glaring at me while I try and figure out some stupid algorithm they had to Google themselves is hazing and should be stopped.
Edit: Also, dinging me for small syntax issues on a whiteboard when I'm used to using an IDE is not really cool.
You know, when I'm interviewing someone my VASTLY preferred approach is to have them walk me through something they've coded that they're proud of. That's not aways possible, but often it is. Give me a good slow walk through your github repo and tell me where and why you made the choices you did, what the tradeoffs were, etc. . . Better than making some poor bastard jump through hoops on problems that only vaguely resemble the ones they'll be solving at work.
Yes this is the best method I have found. Let them explain something they have worked on that provides them with experience they believe qualifies them for the position. Here I can surely expect them to have depth, assess their communication skills, be able to explain further on questions from me etc.
I'd love to hear alternatives, rather than a list of who doesn't.
I don't use whiteboards for coding tests, I use whiteboards as one (of several) ways to evaluate how someone thinks on their feet without having Stack Overflow & Google at their fingertips. I don't only want to know if someone can code, I want to know about their process, how they think about things, what assumptions they start with, and what kinds of things they like to explore. I want to see some personality. Take home problems don't measure that very well, nor do coding tests with online resources available.
I've never met a perfect interview, but what are some suggestions for more acceptable and/or better ways to evaluate how someone thinks on their feet?
I remember hearing that Google's done quite a bit of research on hiring practices vs outcomes, and that one of the leading predictors is having external groups do the interviewing, i.e., making sure the people doing the interviewing have no stake in the outcome.
My hypothesis (of what people want in an interview):
Interviewer to engage in conversation with applicant about applicant's prior work in detail. If the applicant can give sufficient explanations of systems/projects and the interviewer can ask appropriate probing questions and get sufficient responses to those questions from the applicant.. then the interview can be concluded positively.
The crux of it, from the applicant point of view, is that they want to be believed and treated "professionally". Further it's "insulting" to be questioned about anything other than specific experiences. That's why you see objections to every sort of "objective" line of interview:
sample projects - too much time, why am I coding for you for free?
github/open source - I have hobbies outside of programming
white boarding/google doc coding - not my environment for writing code. I need emacs with xyz keybindings or we're not going to make progress.
algorithm problems solving - the job doesn't entail re-implementing hash tables, or sorting algorithms... The question(s) are not germane to the position
> they want to be believed and treated "professionally". Further it's "insulting" to be questioned about anything other than specific experiences.
Yeah, I agree; I have the same hypothesis that this is what people want. And it's a problem to assume and/or act like talking about anything other than specific coding experience is in any way insulting. That's not exactly a realistic expectation, nor is it a great attitude for getting hired.
Personality, behavior, poise, attitude and communication skills are absolutely relevant to someone's ability to do their job on a team. If you can't think on your feet or answer a question without a computer in front of you, and you start getting angry, how will you behave in meetings? That behavior is precisely one of things I look for when I ask candidates to think on their feet.
If someone assumes that being asked certain questions is "unprofessional", then that's a red flag and I'll be more careful before hiring someone like that. Now that I think about it, there's a bit of irony that if someone performs poorly on these kinds of questions, I'm likely to spend more time investigating that and asking more questions along those lines.
Did you mean to reply to someone else? I appreciate the reply, but I’m confused about what you're responding to and what point you’re trying to make. I haven’t suggested or advocated anything about doing work for free.
That said, this example doesn’t convince me of much. Doctors have to go though a full year (or more) of underpaid work (called residency), and worse, extremely long and on-call hours. They think programmers are pretty spoiled, at least the ones I know personally do. When I've complained to them about pulling 80 hour weeks in films & games (sometimes without overtime pay), I've gotten words back like "lazy" and "whiny". :P
Doctors also have to do unpaid diagnoses when they interview for jobs, in order to prove they’re real. And they have to do unpaid diagnoses for their family and friends, just like we have to do IT for family & friends.
If you want the job, you do need to demonstrate you’re capable of performing the job, no? What alternative are you suggesting?
I had to write a JavaScript program on graph paper (with no computer on hand) as a college test. Then the professor would type them out afterwards with you and try to run it. Whenever I hear about this whiteboarding issue I think of that. It was actually kind of fun in a weird way though. I drew nice curly braces.
Heh, I code pretty often on paper when I'm working through a design and my style has evolved to just using < and > as proxies for curly braces which saves me time, since trying to draw a nice curly brace takes me so long.
I do not mind whiteboarding at all. I have three whiteboards at home for brainstorming. I draw the line at doing take home tests.
I cannot interview a company if I am doing a take home assignment. With whiteboarding, I get to also assess my potential co-workers abilities by seeing if they are asking interesting questions.
This is great. I suspect the "coding exercises" in interviews are really dominance hierarchy games. It goes against politeness theory and is a threat to an individual's 'negative face' or need for autonomy.
I interviewed once at a place where a friend worked as an engineer. She assured me that they didn't do whiteboard questions. Guess what I had to contend with during two different interviews...
At my work one part of the interview is a System Design test, where the candidate is presented with a problem and we ask them to come up with a system (i.e. by gathering requirements etc)
Usually this means drawing stuff on the whiteboard with boxes and arrows. Not looking for the perfect system or 'correct' solution, just really having a look at what the candidate is experienced in and how they think.
Whiteboards are fine for that use case, and can be very expressive.
Doing binary search trees and stuff though doesn't seem as useful really.
This is a good idea, although the focus on 'anti-whiteboard' is a bit limiting. It's a shame it's its own site instead of a feature on other sites.
There's no interview process that works best for every candidate; some prefer whiteboards or homework or practical debugging on their on computer or hand-wavey design or whatever. Candidates (and companies!) could save a lot of time by letting applicants filter their searches to companies using methods that will let them put their best foot forward.
I'd much rather code on a whiteboard than do a take-home assignment. I may be in the minority but I rather suspect I'm not the only one.
I can understand that some people find whiteboard coding really stressful. I never have, and I have found take-home assignments to be a huge time-sink which typically results in minimal feedback (and sometimes none at all).
Maybe the right thing to do is to just give interviewees a choice.
Some companies effectivdely use whiteboarding, others don't.
This is a true snapshot of the absurd sense of entitlement among some tech workers these days. When the bubble eventually pops, people will look back at this website promotion with awe.
It's like challenging the generally accepted standard that interviewees dress up a bit more formally than a typical day at the office (i.e. button down versus old T-shirt).
So is this supposed to be a good or a bad thing? I've used whiteboards in a lot of interviews to great effect. I wouldn't confuse whiteboards with riddles. I also hate riddles and puzzles because they check for the riddle-solving skill, not the actual skill which will be used at work. Whiteboards are a completely orthogonal topic to riddles.
I really like the idea of this, but how many jobs posted here are for senior or high experience positions? The whiteboard filter can still happen at any of these companies in junior developer ranks.
This just creates elitism where the "senior" developers get nice hiring processes, and the junior developers get stuck with traditional torture.
I think its still valid if someone doesn't have a good-size body of work, however the more CS heavy it is and unrelated to the actual work the more you're optimizing for "did this person recently graduate with a CS degree".
What i want the most is being able to put these companies to test, i have 0 problems with whiteboard problems but like evaluating them, are they all compotent or is it the company a one trick pony with only a few decent engineers.
It would be very useful to be able to filter just the remote positions. Searching for "remote" as a location returns no results, and I can see in the list that some remote positions are being offered.
I think whiteboards are much better than the remote code-as-we-watch stuff, which is even more common and not a good way to measure actual work skills.
That's the first thing I noticed about the list. I interviewed with Lyft last year and they did whiteboard style interviews. Since when did this change?
too many programmers cant program and hurt their fee-fees so a company was made to accept fucking 300$ per ad. Anyone who cant whiteboard or pseudo is fucking retarded.
Whiteboard problems work great as a general intelligence test if administered correctly. A well designed whiteboard problem will filter out people of lesser intelligence. This means the people you would be working with would generally be more intelligent. This is not a bad thing.
Furthermore, whiteboards are great communication tools. I spend some time on VRChat's whiteboard room teaching people various algorithms, and other abstract things to help me learn. Someone who knows how to use a whiteboard and communicate correctly would be a better hire than someone who doesn't.
If the company you work at uses whiteboard problems to vet interviewees, just drill them. With a little practice, you'll have a better chance.
This is so wrong that it actually hurts my brain. Albert Einstein, a certifiable genius probably could not tell you how to invert a binary tree, this has nothing to do with intelligence and everything to do with how many HackerRank problems you have memorized.
True, but there are also brain-teaser type problems that don't require memorization of obscure algorithms, and those could actually be disguised intelligence tests. (But I suppose if you really wanted to you could memorize most of the brain-teaser problems in existence.)
Obviously, but I was specifically referring to the typical algorithm oriented whiteboard questions found at major tech company interviews. Working a real world problem on a whiteboard I see no issue with.
I've been using whiteboards for 2 decades during the interview process and they work great, if you ask the right questions. I've built great engineering teams at companies like Orbtiz and whiteboards were one of our best tools.
The reason they are important is that they are part of every engineers day-to-day work. We use them to collaborate and design and understand.
Therefore, if a candidate can't whiteboard a design, solution, architecture, etc. in a manner that other engineers can understand it, they are going to struggle.
That being said, the way Google and other companies use whiteboards is an epic fail. Asking someone to code on a whiteboard is stupid. Asking them to solve algorithm puzzlers on a whiteboard is lame. This isn't how we use whiteboards every day so why should a candidate be expected to use them this way during an interview.
An important thing to remember is that broken/not-broken is completely different for each side of that desk: candidate person & hiring person. This is always the case for selection steps.
The employer (or school, etc) wants to make successful hires and to avoid bad ones. False positives are always a big problem. False negatives are usually much smaller one. A process is good if it produces good hires, and no bad ones.
In principle, a process that hires 3 good candidates from a pool containing 5 good candidates and lots of bad ones is good. Avoiding bad hires is paramount. Missing 2 good hires is not as bad as making one bad hire, not near as bad usually.
Most employers don't care about false negatives at all. If they do, they probably care most about missing really good outlier hires, not mistakenly selecting the 5th best candidate instead of the 4th. The only reason to care is fairness. People instintively care about fairness so this affects practices but it's not nearly like false positives.
Employers that need huge volume (eg, the army) are the only ones that care about false positives as much as candidates do.
The candidate (student etc.) generally feels the process is "broken" when it is unfair to them. Unfair means that a person could have done the job well, but cannot get hired. Ie, if the candidate is a "false negative" then the process is broken.
Anyway... When things become systemic, the "brokenness" becomes acute to a candidate. If whiteboards (your architecture version, Google's CS version or any other version) are prevelant and you suck at them... you can't get hired. There are people who suck at your test (whatever it is) but would be good at the job. Your proxy is a biased against some people, every proxy is.
Like I said, a lot of things are like this. One side is working the numbers game, the other side isn't. Insurance markets, college entrance, executive appointments, PayPal's fraudster detection (bastards!), credit scores, investment (even startup investment)...
I agree that most companies don't care about false negatives.
The problem is that very few companies know how much false negatives are costing them, so it is very difficult to evaluate whether the false negatives are—cough—a negligible tradeoff against weeding out the false positives.
False positives are right in your face. But it is nearly impossible to systematically keep track of the candidates that aren't hired to see if they are successful elsewhere.
So people lean towards eliminating false positives at all costs. It's another variation on "Looking for the watch where the light is better."
These situations aren't at all symmetric. A false positive always costs you (the company). A false negative is only a cost to you if you fail to hire well for that position.
Also the fact that a candidate you passed on does well elsewhere doesn't mean you were wrong to pass on them (it could, but it's not clear).
I was not equating this to a scenario where you are only looking for one watch. The expression (and anecdote behind it) is also used to describe managing according to metrics that are easy to collect, often at the expense of outcomes.
That applies in the case of caring more about false positives than false negatives, since nobody (or close enough) even knows how many false negatives they have.
Your statement that someone being successful elsewhere doesn't prove we have a false negative underscores the uncertainty involved. It doesn't invalidate what I'm saying in the least.
But if we want to be rigorous, we could do some direct testing. Let's say we give a telephone screening and four interviews to each candidate. Simpkifying somewhat, let's say that our policy is that a fail on any one eliminates the candidate.
If we hire lots of people, we can collect false positive and false negative stats by randomly passing someone who fails exactly one test.
If a lot of people who fail a particular test end up succeeding, we can infer that the test provides false negatives.
On the other hand, if everyone who gets the random "free pass" on one particular test ends up failing, we can presume it has low false negatives, and maybe abandon the random passes for that test.
I understood the analogy you were trying to draw, I just disagree with it.
The original joke/anectdote works because it leads to clearly suboptimal search - the guy knows his watch isn't near the streetlight, but he's looking there because he can see.
This is a very different situation. To abuse the analogy further, the guys best bet to find a watch seems to be just looking near the streetlight. After all, not only can you see better there, but most of the watches are near there too.
What your opponent is saying is that the cost of false negatives is unknown because it's hard to calculate. So people only calculate the cost of of false positive because it's easy to calculate then compare it to the unknown value and conclude that it's greater, not dissimilar to the gentleman who had been looking for the watch under a streetlight.
Your saying "false negative is only a cost to you if you fail to hire well for that position" could be easily disproved with examples of people, who were rejected and started their own competitive business (e.g. [1]). IMHO, one of the goals of having 10Ks of engineers on staff is to keep them from the competition. But since nobody is tracking this and the extreme cases like WhatsApp can be hand-waved away it's easy to pretend that the false negative's cost is miniscule.
1. https://en.wikipedia.org/wiki/WhatsApp#History
If I’d seen any evidence that this cost wasn’t minuscule int the typical case, I’d be much more interested in expending the effort to try and measure it better and act on that - but that has an opportunity cost also. So you are suggesting incur a real cost just in case? I don’t buy it.
The guy with the watch analogy only works if there is good reason to believe he’d be better off searching elsewhere. But we don’t appear to have that here....
I am not sure you checked out the link I've given. It's quite a substantial evidence that not hiring Koum and Acton might have costed FB somewhere around $19B if one were to believe WhatsApp was acquired to neutralize competition and not for its amazing revenue.
Let's be generous and say the WhatsApp was a 10x unicorn so FB overpayed only 9B for it. Which makes 4.5B loss per false negative on each founder. If a false positive was, on average, 1M loss (which is an extremely generous in an organization such as FB in my recollection), it means you could have had 4000 false positives for each of these negatives and still come ahead.
And, it brings us back to the Type I and II errors - it's always cheaper to make one, which has a limited downside than the unknown (potentially very high one). Typical example given is that while you are on the way to an expensive cruise you realize that you might have forgotten to turn off the stove. If you go back you might miss your 10K cruise. If you cary on, you might lose you 1M house. On average, people who miss cruise in such circumstances fare much better than ones, who risk their houses.
I know that story - but think it is enough of an outlier (regardless of how it went down) to basically be irrelevant to almost all hires. The idea that people are routinely filtering out great hires in their thousands in ways that come back to haunt them ... doesn’t seem plausible.
I get what you guys are saying, there just doesn’t seem to be real evidence that is an issue nearly all cases. Whatsapp notwithstanding
No, the idea is that you don't know what you are filtering out. As I said - it's extremely easy to just hand-wave. And if you want to believe that you know the unknown downside I am not the one to persuade you otherwise. But acknowledging that it's just a personal belief not only unsubstantiated but contradicted by data could be a good first step :)
I think we are talking past each other. I have never said I know the unknown, that would be silly. But funds,entsllu can’t avoid false negatives without degrading other performance , assuming some reasonable things. Even without those assumption, there is a very real cost to what you are suggesting. You seem to be waving off this cost.
So in order for it to make sense to spend that money, you want some evidence there will be a positive return. Far from contradicting my position, what limited data I have seen does not support that effort, at least in the general case. Certainly whatsapp and the like don’t speak to it - they are very much outliers.
By all means try it. If your false positive rate explodes it will be an expensive lesson, but that’s always a risk.
One last point, the idea that nobody thinks about this and try’s thinks is laughable , which is worth noting because some comments in this thread almost suggest that it’s true,.
> You seem to be waving off this cost.
With my example of sacrificing a 10K cruise for a chance to not lose a 1M home, I figure?
> Far from contradicting my position, what limited data I have seen does not support that effort, at least in the general case.
Indeed, the data does not contradict a statement "on average, false positives are 4.5B each", which nobody has argued. However, it does contradict the statement "false negatives have any cost only when you are unable to fill the position as the result", which you have made in this thread.
>One last point, the idea that nobody thinks about this and try’s thinks is laughable , which is worth noting because some comments in this thread almost suggest that it’s true,.
Again, it would be silly to say that nobody thinks about it. Evidently, a number of people think the same way you do.
Perhaps they do that because it has proven out over time. At any rate, we’re well past diminishing returns here, so I’m out.
If there are dozens of watches scattered around, of course you first look where the light is good.
At first, yes. But when one has had enough experience looking for scarce commodities (of whatever kind: genuinely attractive apartments, interesting music / restaurants, romantic partners, etc) one learns that the "where the light is" search technique only gets you so far.
Point is, it very often gets you far enough. So you try it first, and only adjust when you have real evidence that this is not working for you.
Certainly in this scenario - the false positive rate is only important if the true positive rate is too low. It's not at all clear that is a factor.
So you try it first, and only adjust when you have real evidence that this is not working for you.
That's the whole point -- in order to find evidence that it isn't working, you have to start looking at the false negatives --
or that is to say, where the "light is not".
You have just equated hiring a coder for a job to finding a spouse.
Actually no, I didn't.
But it is nearly impossible to systematically keep track of the candidates that aren't hired to see if they are successful elsewhere.
LinkedIn could offer this as a service, if there was sufficient demand that they could monetise it. Every company will routinely check out every candidate’s profile anyway. They just need an automated way to check again in 6 months and see if everyone they reject is going to a competitor who is beating them in the market.
It may be understandable, but the prevailing attitude towards type I vs type II errors in tech industry hiring certainly makes me less sympathetic to claims that “we can’t find anyone to hire” and therefore governments should enact policies X,Y,&Z. If they are choosers they ought not to go begging.
>False positives are always a big problem. False negatives are usually much smaller one.
I would agree with you if the supply and demand weren't so skewed towards top developers. Do you think you are choosing between A and D people, or are you really choosing between C and D people and A and B people never walk through the door?
It's still the case! At the end of the day, the person making the call is a hiring manager.
It's pretty hard for a hiring manager to be called out as too selective. However, if you let in a few bad apples then you look really bad.
The incentives are for hiring managers to be too careful.
It is, oddly enough, the case that if you are not hiring any bad engineers that you probably are being too selective.
In my experience, there was considerable pressure on hiring managers to hire someone, because people are honestly needed. So, the risk of hiring one bad apple is offset by risk of being seen as someone who will never hire anyone (and thus will be excluded from hiring).
Then again, probably one of the worst bad apple I have seen would ace whiteboard as he was quite good at trics. He just did not like the actual work involved in software development and tended not to do work he don't like except bare minimum.
It's easy to fix a false positive, it's impossible to fix a false negative. You figure FOMO would kick in at some point.
Once you hire someone you have to give them a chance to be successful. As the threshold for firing someone is (rightly) much higher than simply not hiring them, fixing a false positive is much more expensive than missing out on a good candidate.
Also you can fix some false negatives - allow people to interview again for other positions at the company. I've seen multiple people get hired on the 2nd or 3rd time around and ultimately be successful.
Simple - contract to perm.
You have therefore eliminated everyone not interested in that from your pool. Good candidates know they are good and will avoid too much contrived hoop-jumping.
True. But I love contract to perm. As an architect either by name or responsibility, it's usually the first six months that are the hardest and take the most hours. I get paid for every single hour I work and can make good money until they realize the error of thier ways and make me permanent.
When we first got married my wife was really hestitant about me working on contract. Over the years, she's seen how much larger my checks are when I do.
I know companies do this, but I don't understand how it works. Given the competitive nature of hiring good engineers what type of candidate would accept a contract position when they probably have five other offers?
As I said below - I will at the proper hourly rate. With the right hourly rate, I can make more than enough to make up for any gap in employment and I'm in a market where jobs are a dime a dozen.
Besides, there are some opportunities where companies just want someone to come in, fix a problem and then after that it's just boring maintenance work. If it's a contract, I can come in, fix the problem, and look for a more interesting full time job without being labeled a "job hopper". I can just put on my resume that the 6 month job was a contract.
I prefer contracting but I found that more companies are willing to pay me my minimum salary than the commensurate hourly rate I ask for to make up for the possible gap in employment, unpaid time off, and self employment taxes.
But, my wife has a stable government job with decent benefits that allows me to go back and forth between contracting and permanent and allows me to take risks working at companies that pique my interest without concern about the long term viability of my job.
While I agree with you, it's also noticeable that many companies do have a broken hiring process, but they don't realize it, because things aren't bad enough yet. However, whiteboarding is not the culprit.
People complain whiteboarding because they've had bad multiple bad experiences with hiring processes that involve whiteboarding and they came to a conclusion that whiteboarding itself is the problem.
Unfortunately, you can't fix a bad hiring process by simply changing the technique. Each technique has its own pros and cons, but no technique will guarantee success.
These are the most common problems I've seen with hiring processes:
1) The people doing the hiring don't really know what they're looking for. Instead, they go by a vague, handwavy notion of whether the candidate is "a fit" or not. You have to define what "a fit" means in such terms that all of the people involve understand it, agree on it and know how to test for it.
2) The people doing the hiring don't know how to test for their criteria. Whether you're using whiteboard or a take-home project or some other testing technique, you have to design the test so that it helps you observe the traits you're looking for. [1] And you have to actually train your people to interview candidates.
3) Lack of transparency. This is a huge problem and it infests all aspects of hiring. Some recruiters won't even tell you which company they're hiring for. A lot of companies misrepresent what you would be doing for them, sometimes intentionally, but usually not. Some companies won't give you perfectly reasonable information about the job. [2] And, thanks to the litigious nature of our society, no company -- anywhere, ever -- will give the candidate honest, detailed feedback of why they were rejected.
4) The company is unreasonably demanding when it comes to candidate's personal time. The typical two-stage process -- a phone screen followed by on-site interview round -- is demanding enough, because it involves more than just the interviews. The candidate has to spend time and effort on other aspects, too: multiple phone calls with the recruiter, preparation for the interviews, travel, and juggling all of this with their personal and professional lives. Some companies don't understand this, so they think there's nothing wrong with a process that involves a take-home project, a debriefing phone call, a face-to-face conversation to test for cultural fit and a full-day on-site interview round.
Most companies don't even know that their hiring process sucks. Of those who eventually realize something is wrong, many end up cargo-culting whatever they read or heard about "more successful" cases. As things are right now, the whole situation is eerily reminiscent of the fuss and hype the industry went through with software development processes. I wouldn't be surprised to see yet another wave of parasitic consultants pop up to prey on companies by flogging their hiring process improvements and best practices.
----------
[1] This is where a lot of people's complaints go wrong. The most frequent grumble is: why does this company's interview process focus on algorithms and data structures, if you'll just end up coding some services in Java? If the company is doing things right, they're using these topics as a tool to observe and measure whatever it is they're looking for; most often, it's the candidate's understanding of computational complexity. Just because you won't be writing your own implementations of common data structures and algorithms, it doesn't mean you shouldn't know how to pick the one that won't blow up your memory or execution time.
[2] I had a case where I was interviewing with a company that gave me a print-out overview of their benefits, which included the price of each health plan, but not what it covered; when I inquired about it, they replied they wouldn't be able to give me that information until I signed the New Hire NDA.
False-negatives are only a small problem for employers because they don't experience them.
This is exactly the point. A risk averse strategy is optimal for areas of a business that you do not have expertise in. If hiring isn't the thing you do best (supposedly your strengths are writing and selling software) then it makes sense to pick a hiring strategy that doesn't cause disruptions.
> Asking someone to code on a whiteboard is stupid.
I've started to turn this around and so far our senior candidates have loved the experience. (I only wish I had come up with the idea.)
Instead of asking to write code on a whiteboard and solve an arbitrary problem, couple of our engineers came up with an inverted problem. And ever since seeing it in practice, I've been trying to introduce it to every interview I can think of.
We write a piece of code on the whiteboard, and then ask the candidate to review it, as well as modify it to better suit the problem and context. This way the question turns into a discussion on design, constraint exploration, service expectations and externalities. Hell, sometimes it even goes into deep language design issues. Sure enough, it still measures a level coding skills but puts a much higher emphasis on understanding what the problem and its scope really is.
It's also a lot of fun.
I really like that idea and plan to steal it next time it seems useful. Thanks!
Go ahead!
> This isn't how we use whiteboards every day so why should a candidate be expected to use them this way during an interview.
I work at Google, opinions are my own.
I think your assumption that Google interviews based on what you will do on the job is wrong. The reality is that for most SWE positions, we can't assume you will know the things you'd be using for the job because of all the internal tools. Thus, algorithm interviews are more or less a proxy for some combination of general intelligence and coding questions.
I'm not saying it's perfect and there are many things I don't like about it, but I think it's important to know what the goals are when making a critique of whether the process meets those goals.
> more or less a proxy for some combination of general intelligence and coding questions.
It's a proxy for one particular style of coding questions: leetcode. It's nothing more, nothing less.
Calling it a proxy for general intelligence or coding in general is a joke. That's propaganda passed around the company to convince people that the practice should be continued. It strokes the employees own egos to think they passed some magic intelligence test so it motivates them to keep repeating that it measures intelligence/skill.
Its a proxy for having practiced CS questions, not being able to handle the complexities of a modern software project. They basically want to know whether you went to a reputable college, or have a similar skillset to those who do.
Well, maybe I should have said it was an attempt at a proxy for general intelligence? For what it's worth, I agree with you. When people start studying for interview questions, then something is probably wrong..
I have a few friends from large universities (waterloo, stanford) that have left their current job to specifically study for a new job.
My assumption is that there are a lot more people who are studying for these interviews than you think, which is why the proxy is failing (or producing homogeneous results)
That's true.
On the other hand, it's okay to target sufficient but not necessary. False negatives can be fine, so long as you are finding enough true positives.
Even if you are systemically missing out on a sub-pool who are good, it's not necessarily a bad thing - particularly if you are realistic about its limitations. To my mind the real problem isn't that you might miss good people (you will) but that if your filter is too constrained the good people you find might be too homogeneous.
You can also cram for IQ tests the only ones with validly are those taken cold
How much cramming would it take an average person to score 190 on a quality IQ test?
There are no quality IQ tests that go up to 190. If a test purports to be valid up to that level it is very strong evidence that its authors are frauds or don’t understand statistics or both.
Depends on how far from 190 you start.
Assume 100 for "average"
Google also has a very big pool of applicants.
IMO the biggest problem is that small startups copy the Google's approach while their hiring environment is completely different.
I've legitimately used whiteboards to pseudocode while discussing architecture with teammates plenty of times before. Does it compile? No. But the point is that I can more easily demonstrate the benefits of an approach with some concrete pseudocode to help grok the proposed algorithm.
As an aside, if you've never done systems design or pseudocode on a Smartboard, you're really missing out. Copy/paste, duplicating a slide, recalling earlier slides, etc. Man, that's the way to design a system with quick iteration but still no worry of losing your work.
Yes but I think the point is a lot of interviewers do not go for pseudocode. They want syntax perfect solutions on a whiteboard.
Yes. That's essentially what I was getting at. Asking someone to code accurately on a whiteboard is nonsense.
When I interviewed at Google, they asked me if my whiteboard code would compile. I said, "give me a laptop and I'll tell you". I don't think they liked that answer. ;)
When I ask candidates if their whiteboard code compiles, it's a gentle hint that there are major structural issues, or an opportunity for them to point out the parts that are psuedocode-y. In general I don't include whether the code compiles (mostly JS, so "runs") in feedback unless it is amazingly perfect or really far off. This seems standard.
This. When interviewers feel that the question was not hard enough or that they didn't like the answer, they often fall back on "you're missing a semicolon".
Whiteboards are okay for ad-hoc lists and diagrams, but they are a poor medium for coding.
Some of the biggest productivity gains I've had over the years were related switching from "writing code down" to "thinking in code" mentality and abandoning linear, paper-like ways of thinking, which hindered me when I was in college.
For anyone who believes that paper and whiteboards are "good enough" for coding, I highly recommend watching some of Victor Bret's or Alan Kay's talks. Pretty much any of them. They consistently touch on the subject of dynamic media as a way to empower people to... well, think better for the lack of better terms.
I think you have got the wrong lesson off those guys. They are about engaging the spatio-visual systems. The whiteboard is the first step away from text towards that. Obviously they want to go a lot further than a whiteboard, but the fact that you can draw things, circle things, wipe things off makes a whiteboard a much more dynamic media than an ASCII interface. I think you would find them pro-whiteboard for thinking design.
>I think you have got the wrong lesson off those guys. They are about engaging the spatio-visual systems. The whiteboard is the first step away from text towards that.
Not when you are literally writing code on it.
I guess I'll just point out the distinction.. there's an obvious gap between writing code and writing pseduo code / logic structures.
And there's nothing wrong with whiteboarding pseudo-code.
bret's apparatus in dynamicland is literal pieces of paper embodying code, which, quote, "also are fully functional pieces of paper".
The reason they are important is that they are part of every engineers day-to-day work. We use them to collaborate and design and understand.
Hey if they work for you, that's great.
But the simple fact is that they don't fit everyone's brain. And that there are other tools available to accomplish precisely the same ends -- some of which have many advantages over whiteboards, in fact. Like these things known as "pen" and "paper", for example.
So this idea that they're part of "every engineer's" day-to-day work is, quite simply, disconnected from reality.
I do agree with you, however, that the problem lies not in the whiteboards themselves but in the way they've used. Which has truly been an epic fail.
> So this idea that they're part of "every engineer's" day-to-day work is, quite simply, disconnected from reality.
> Like these things known as "pen" and "paper", for example.
When you need to share this pen and paper, you use a whiteboard. I don't understand this hate for whiteboards, which are a cross-industry cross-profession standard (including medical) when collaborating.
One of the more interesting things I have noticed is that happy engineers have a little free time each day. They aren't under crushing pressure at all times. To this end, it's very common for whiteboards to accumulate drawings and doodles. This is often turned into "the elaboration game" where one doodle is built upon (by a little or a lot) every now and then. This is probably because coding is problem solving, but also because it's a creative endeavor. Engineers who are happy tend to work around other happy engineers and those engineers are all creative with free time.
I don't understand this hate for whiteboards,
What's not to hate?
* unless both the board and markers are of very high quality, the boards tend to stain over time
* as do your fingers
* and the markers dry up, and the erasers get less useful as well (if they're available at all)
* you have to erase other people's crap (or wonder / ask if it's worth saving) first
* the dizziness that results having to turn back and forth between the board and your audience (that is if you don't end up simply talking into the whiteboard most of the time)
* and from stooping to pick up the pens / erasers / pen caps all the time
* and all of this, mixed with all those indelible memories of crappy interview experiences
I'll take pen and paper any day
which are a cross-industry cross-profession standard
Along with horrible interview experiences unfortunately.
> unless both the board and markers are of very high quality, the boards tend to stain over time
Solved by getting a good office manager.
I'm not even joking. We have very large whiteboards in every room and in almost every space where people can wheel their desks in. Cleaners are happy to clean them up and make sure they are in good condition. One team recently complained that they may be too eager... (Our whiteboards are heavy glass, bolted to the wall. It probably helps.)
Office manager(s) on the other hand always keep their ears open and make sure the office has enough pristine marker packs and erasers available. Having a supply cabinet properly stocked is a cheap investment.
One of the most enjoyable things I can hear at the office is: "Let's hijack a whiteboard".
Solved by getting a good office manager.
In theory, I guess.
But -- getting back to the original subject -- for some reason during interviews the odds that you'll get an inadequately stocked and clean whiteboard seems to be almost laughably high.
Consistent with the observation that in many places, the seem to almost make a point of the importance of positive candidate experience in so many other ways (from poorly prepared interviewers to inappropriate / irrelevant questions to stalling and ghosting, and on and on).
Therefore, if a candidate can't whiteboard a design, solution, architecture, etc. in a manner that other engineers can understand it, they are going to struggle.
If that's the kind of whiteboarding you do, no problem. But, don't ask me to write a merge sort in C# (been there done that). If you ask me to sort a list in C#, I'm going to write:
sortedList = employeeList.OrderBy(e => e.LastName + " " + e.FirstName).ToList();
I’ve been in too many NIH situations and when someone asks me one of these questions, often my mind isn’t on the solution. Im thinking about if there’s a reason they asked the question, and maybe this was a mistake.
I get so hung up on this it’s changed how I ask the question. The last time I asked a sort question, all I wanted was a compound comparator. And even then I couched it as “we want the rest endpoint to support sort criteria so we can do a top five list, but to sell the idea, we are going to sort in the UI for now. Give me some code to do that.”
I wouldn't even bother remembering that pattern.
With R#
Generate Code -> create Relationsl Comparison and click on the properties that I want to use for comparison.
> That being said, the way Google and other companies use whiteboards is an epic fail. Asking someone to code on a whiteboard is stupid. Asking them to solve algorithm puzzlers on a whiteboard is lame. This isn't how we use whiteboards every day so why should a candidate be expected to use them this way during an interview.
Every candidate I interview has the choice between coding on a whiteboarding, and coding in a text editor with rudimentary code formatting.
I really like it when people code in the text editor, because that's one less thing I have to transcribe in my interview evaluation.
Regardless of whether they will choose the whiteboard or the laptop, I have never, and will never mark people down for typos, confusing .length with .size, a missed semicolon, or any other petty bullshit that isn't pertinent to the problem. I'm not hiring a compiler.
not that i'd feel like my success in interviews hinges on this, i just find this very refreshing. where do you work, and are you hiring?
Google, and yes, and unfortunately every interviewer is different, as all the bad experiences people post about can attest to.
Can you point to examples of how you do use whiteboards? I am a self-taught programmer with my own way of noting design strategies and having a tough time finding examples of this.
Not OP, but I use whiteboards to sketch system designs and things like that. For instance, I'd sketch out the path that a request might take through a complicated system (intake via a REST service, flows through a queue, consumed by another service, persisted to a database...). That's the sort of thing that I would be comfortable doing as part of an interview—though I wouldn't expect a junior developer to be able to do it.
AFAICT, the backlash is around interviews where candidates are asked to write code (or pseudocode) on a whiteboard. Being able to implement bubble sort on a whiteboard does not correlate very well with being able to deliver business value, in my experience.
> Being able to implement bubble sort on a whiteboard does not correlate very well with being able to deliver business value, in my experience.
To add to this, it's being able to write code on a whiteboard in a high pressure situation while talking about every step you're taking. Over time I've gotten more comfortable with this, but mainly by doing more interviews, not by getting better at my job.
If you ever find yourself in a situation that Broadcast television would have us believe, you'll be excellent at coding that hotfix into a real time system as the president of the United States looks over your shoulder.
When I was younger, this would probably make me feel insufficient but I have done enough ‘learning stuff’ by now that I expect this kind of comfort to only come with experience.
Here is a nice link I randomly found on google: https://leanpub.com/visualising-software-architecture/read#l...
I spent about 30 seconds looking at it, but at least the first chapter 1 seems to be about proper whiteboard techniques
Nice. Thanks.
To all the responders to my query, your answers are super helpful.
Being self-taught, there are just these few holes the internet doesn’t quite cover. But a little reaching out on HN goes a long way in filling the gaps.
I think a lot of the other posters are correct. Diagrams of classes, interactions, event flows, dependencies, network architectures, and yes, even pseudo-code.
Basically, if it is an idea, concept or design, draw it out. If it is functionally correct code that compiles, screen-cast and type it out.
Last time I used a whiteboard was when I was re-architecting some classes and I mapped out the class and function names and how they interacted with each other. This enabled me to see what changes I would need to make/add to get the desired effect.
The time before that was for a database migration where I mapped out each step that had to be done and in which order.
Agreed, i would love to see a Youtube video of the right way to use Whiteboards like OP is talking about.
I agree.
The point is not to have or not have the whiteboard during interview but how and what it is being used for.
The best way I found to interview candidates is to have few different discussions. One of the discussions I like to have is to try and solve some kind of problem that you could potentially have in real life. I am solving problems with people this way on a daily basis so this lets me observe candidate in a setting that is already familiar to me and hopefully to the candidate. The whiteboard is then used exactly as in real life -- as an tool to visualize and communicate ideas and hold it as a kind of cache and reminder for the duration of the discussion. The whiteboard is there during interview but I will not require candidate to use it. If I have trouble understanding the idea I may ask "ok, I can't visualize this, could you maybe draw it for me?"
Nobody programs on whiteboard in real life. If I need to program for presentation, for example to train my team on some topic, I will bring laptop.
Some engineers think that whiteboarding is not clear enough ... that code snipts are better.
The key about whiteboarding is it take away all the specific syntax and gets to the heart of the understanding the logic.
Thats what collaberation is for, i think many young engineers that only code with an ide are nervious whrn they dont have autocomplete available and they lose focus on the core logic.
Programmerd used to write on paper ... this entire process (coding without the ide/search engine) is lost.
But this is the part where the programmer shoes their strength in logic ... not syntax memory.
As an aside, any resources for learning whiteboard techniques? Not specifically solving algorithms on whiteboards, but designing and conveying information during architecture planning/etc?
I feel like there's two angles to this; 1. understanding complex algorithms commonly used as tests. 2. understanding how to convey information on whiteboards efficiently, common paradigms, etc.
#2 is what I think I'd actually enjoy improving. #1 is just SAT style stuff that I infrequently use in my day job and serves as an annoyance more than anything.
> Asking someone to code on a whiteboard is stupid
I think this is largely the point.
Whiteboards should be used for what they're good at: Visually explaining ideas/solutions on an abstract level.
They are terrible at compiling and syntax checking.
Theoretically you should already know whether someone can do the physical code part. That problem is largely solved for and don't require in-house interview.
In-house interview should be used for c̶o̶n̶f̶i̶r̶m̶a̶t̶i̶o̶n̶ b̶i̶a̶s̶ ̶f̶o̶r̶ a̶g̶e̶/̶r̶a̶c̶e̶/̶g̶e̶n̶d̶e̶r̶ determining if the candidate can think/design/solve for the domain space they're being hired into. A whiteboard is a really good tool for assisting in that.
When I worked at http://zipcodedownload.com we constantly used whiteboards to code. That, of course, is when we had a parent company. Now that we are independent its gone by the wayside. Don't like it as well.
They seem to be starting to get that. I recently interviewed with a few companies including google and all of them provided a computer for me to type out my code in the interviews.
> the way Google and other companies use whiteboards is an epic fail.
They are optimizing completely different metrics than you do. They want crème de la crème developers that are perfectly trained not to think about certain classes of problems and waste time there but instantly come with acceptable answers, immediately recognizing the structure of the problem and having the ability to faultlessly transform it into a piece of code without a debugger. It allows the training from the best universities to shine.
The problem is when everybody starts doing the same, but without the prestige, salary, equity and lifestyle that Google provides.
The problem is when everybody starts doing the same, but without the prestige, salary, equity and lifestyle that Google provides.
Or without actually needing to problems that even remotely resemble the kinds of problems that Google (sometimes) solves.
Couldn’t disagree more. I hate whiteboarding and I never do it. Verbal explanations and pair programming work just fine.
I feel slowed down having to write out on a whiteboard when I could instead write some dummy code to get a point across. It just seems crazy to assume all developers everywhere must be good at white boarding.
I tend to move my hand faster than I can hand write, which is why I like typing for all forms of explanation.
Especially since I’m on a full remote team, whiteboarding would be really weird.
I can't recall ever having written actual code on a whiteboard in the course of doing actual work. I have, many times, sketched out a system's architecture or worked out some weird process using boxes and arrows.
I'm sure a shared digital sketchpad would accomplish the same thing for remote workers.
This guy/gal gets it.
We don't whiteboard at my company, as we favor a take home technical exercise and debrief/walkthrough format for interviewing our engineers. That being said, I'm super turned off by the incessant whining from the community about various interviewing practices. Here's the tough truth I think: Some companies simply do not understand how to properly engage in employee selection. They get some quick guidance from a google search and have too much autonomy in the process to make their own decisions because the company hasn't really empowered their talent processes to be well thought out and planned. This happens frequently. The real shame in it all it that it focuses the conversation on these high level exercises - be it coding on a white board or rallying against take home exercises or demanding to be paid for interview work, and ultimately to the employer you're put in a spot where you can't help but feel a little reactionary as the transactionality of the relationship is constant: I need to know you can do the work, prove to me you can, I will then hire you and pay you to work.
What a cluster it's all becoming on the software engineering side.
>I need to know you can do the work
And I need to know what the work exactly entails, and what is expected of me. Many interview questions are disjoint with what the job actually requires.
>Prove to me you can
I can show you my past experience, and I can do technical exercises. What I can't do is devote a ridiculous amount of time and effort to poor questions.
I wish it was that simple :(
As someone who has hired a lot of devs over the years (almost exclusively for our offshore team in India, but sometimes in the EU), I can tell you that the number of candidates who embellish and down right lie about their capabilities and past experience is shocking.
Truth is, I hate both sides of the interview divide, as interviewer and interviewee - it's a horrible, inefficient, flawed process regardless which side you're on, and nobody has the answers.
If Indian offshoring teams constantly lie about their credentials way more than local, I would think having different interview processes for different sources of candidates would be a better option than a process for the lowest common denominator.
Yes, I do agree to a point.
People outside of India often embellish or exaggerate too though, even if not close to the degree that I've experienced in India (disclaimer: this is my own opinion based on my own experiences over 10 years or so. Obviously India has a lot of great, honest Engineers, our organisation never sees them though, largely because we don't pay well enough).
A good reputation with solid references that are checked from real engineers and managers that you actually delivered a good product seems like it would suffice to override the straight up lying, to me. Is that not the case?
References are an excellent way to confirm someone that someone did actually work at a particular place in a particular role and that they are not a total asshole. Beyond that, unless you know the reference well, I think they can be unreliably.
The reference knows the candidate and likely feels a much closer connection with them than they do with you. Assuming the candidate isn't an asshole, they will want to help them out by giving them a good recommendation. If the candidate was let go due to downsizing or such, the reference may even feel pressure to help the candidate find a new job.
So it's a great data point to have and as you say, it can help weed out the straight up liars.
But any 2 given candidates with 5 years at Acme Inc are not interchangeable. Some candidates are just better at programming (or some subset of it) than others, and I think employers try to find ways to predict that at interviews.
One technique I have found works is to do incorporate your own open source code into your work. Then when they want to see samples you have commercial quality code that isn't company property.
If you had to do 6 consecutive coding interviews only to be rejected on the last one, you'd whine too.
What really gets me is when you pass all 6, only to be told that you did great but there are no suitable positions open.
I have an interview coming up and the process as described to me is take home exercise with an on site debrief. I am thrilled. I've done whiteboard interviews and some I've aced and some I've crashed an burned. I didn't crash and burn because I didn't know the material. I just froze on a tree traversal question one time. And then the nerves and empty conversation just made the situation snowball as I thought to myself, "OMG I know this why am I taking so long, what is this guy thinking right now." I relish the opportunity to provide useful information for the employer to update their prior.
>I'm super turned off by the incessant whining from the community about various interviewing practices.
Wow, what an incredibly tone deaf response.
>Some companies simply do not understand how to properly engage in employee selection.
Maybe changing this prospective is the point of the "incessant whining."
>I need to know you can do the work, prove to me you can, I will then hire you and pay you to work.
I'd have to know how you are testing, but I'd bet dollars to doughnuts your method has no way of predicting output in any meaningful way.
>What a cluster it's all becoming on the software engineering side.
Absolutely. I would say it's endangering the overall quality of the industry. The saying goes, if you are a bad manager or a bad company the first people to leave are the A and B players. The people that will never leave are the C and D players. I suspect people who do all this trivia don't even know what an A or B player looks like.
> Wow, what an incredibly tone deaf response.
It's really not though. Here's the bottom line: You want money from a company. The company doesn't owe you jack. shit. If they fucking whiteboard, and you want money from them, then do the whiteboard. Later, when you are employed by them, try to suggest they improve their process. If you have better options, take them.
> Maybe changing this prospective is the point of the "incessant whining."
Sure. But is that the companies #1 priority, or do they have other things to do, like make money and stay in business that take precedent? Anyways, the way you do this is from within or providing feedback and having conversations with recruiters -- you know, human interaction. This job board is the ultimate culmination of millennial passive-aggressive behavior.
> If you have better options, take them.
> This job board is the ultimate culmination of millennial passive-aggressive behavior.
This job board just enables more people to pursue a course of action that you're already suggesting they do!
I will admit, maybe I am reading too much into the convictions or intentions of the job boards -- but we all are human, so I can only assume this was created by someone who was tired of having to do whiteboards, and rather than confront the problem head on and potentially drive industry-wide change (channeling "one person can change the world"), goes behind the scenes. The job board further fragments and confuses things for everyone. The better option, of course in my opinion, is make things better for everyone by fighting the battle where the companies are.
I am frustrated at times just as much as any other engineer by playing "monkey-see-monkey-do" in front of an audience in order to receive employment and I have indeed passed up job offers because I was turned off by the interview process (but again, that was my choice) but if you take a step back, look at the overall job market, and what other workers have to go through to get a job, we really do have it made, and this seems like a minor nuisance at best.
> rather than confront the problem head on and potentially drive industry-wide change (channeling "one person can change the world"), goes behind the scenes.
No, this is how change is driven. How do you think change happens? Getting a big committee together and getting buy in from all interested parties? That's not how it works. But maybe this is how it works, maybe this job board attracts a bunch of candidates who are tired of the status quo, and other candidates start telling the companies on other job boards that, sorry, but they're more interested in the companies that list on this particular job board, and those companies then see that the talent pool they're drawing from is smaller than it needs to be, and they decide they want to draw from the larger pool, so they change their practices, and then that snowballs, well maybe now you have industry-wide change.
I don't personally see this having much of an impact, but I think the people who started this are doing exactly what you're saying they should be doing, except that you're pillorying them for it because they aren't doing it through some impossible broad consensus approach.
> what other workers have to go through to get a job, we really do have it made, and this seems like a minor nuisance at best
It is true that we have it made, but I do not think it is true that we have it made in this sense. We have it made because our skills are very highly in demand and because of that, we are treated well and paid handsomely once we get a job. It is true that we should never ignore, down-play, or fail to appreciate this privilege, but that doesn't imply that we should not discuss or advocate for improvements where needed.
Where we don't have it made is in the predictability of employment once competence has been demonstrated. Other professions and trades usually have long-and-tedious up-front competence proving phases, but then they have a credential that will be respected by future employers when they have open positions. We are fairly unique in forcing people with long and distinguished careers to continually re-prove themselves. Our employers have open positions, but it takes a lot of pointless and duplicated busy-work for a competent person to slot into them.
I think what it comes down to is that our process is relatively mediocre, good, or even great for inexperienced unproven new entrants to the job market, but rather sub-par for experienced and proven folks. And that's why you see people looking for a better way.
I think there's two different complaints at play in this thread. Surely you're not suggesting that candidates be exempt from proving themselves. The number of developers who become incompetent over time is high. I think the question is whether they should prove themselves through algorithmic questions. I'll never higher anyone just because they have credentials. A Computer Science degree is a credential, yet so many with that credential cannot write the simplest of programs.
It's probably too late to come back to this thread, but no, I'm not talking about the narrower but more common complaint about algorithmic questions. I think the entire approach of constant proving is misguided. There are lots of other ways to do this with lots of prior art in other industries. I believe we have landed on a poor but self-perpetuating solution. What we do to hire is akin to what they do in the performing arts, but our work is entirely dissimilar.
Do mechanical engineers draw up a design for a valve or something at every job they apply for?
I don't understand, what does confronting the problem head on look like to you? Or how is creating a job board like this not such a solution?
This might sound silly -- especially to the HN audience -- but it's akin to the baptist church problem: one group doesn't like their senior pastor, so they take about half the congregation, go down the street, and say "we will do it right this time!" and now you end up with two split churches. This is basically why there are thousands of baptist churches in the south.
Another example it's related to is the XKCD standards comic. Rather than improve existing standards, the committee breaks off with dissenters, forms a new committee, and now you get "Situation: There are now 16 competing standards"
edit: Better Solution? Talk to recruiters? Let them know you won't consider places that force you to play "dance monkey". The letting them know part is key. More broadly: maybe unionize?
OK, that makes sense... but what is a better solution?
It's really not though. Here's the bottom line: You want money from a company
And the company wants to make money from me. In my market, there are a lot more open positions for senior devs than senior devs that are actively looking for jobs. I
And if you have all the time in the world to play that out --great. If you have limited savings like the vast majority of Americans -- time is your enemy, and eventually rent will come due. The company can survive for months, years without your labor and if they have to, they will simply purchase it or outsource it if need be.
You need that money more than they need you. There really are few exceptions to this rule -- AI being one of them.
Again I wish it wasn't like this, but it's the reality for the overwhelmingly majority of software developers in this country.
Yes, my comment has a few assumptions
1. That you are in an area of the country with a lot of opportunities
2. That you kept your eye on the market and made sure that your skill set in demand
3. That you kept your resume updated
4. That you kept your network of recruiters, former coworkers, and references fresh.
If you have limited savings like the vast majority of Americans -- time is your enemy
With the unemployment rate of software developers being extremely low, the chances are that someone looking for a job that you are interviewing, already has a job. When I said my turn around from looking for a job to getting a job is about three weeks, I didn't mean to imply that I was a special snowflake.
A company I was working for went through a few rounds of layoffs before everyone was let go -- this was in 2011. Without fail, every developer and even L2 support person had another comparable job within a month.
If a company makes the hiring process too long, more than likely if you are in the right market and you are just a regular old "Full Stack Developer", by the time you go through the process, you've already received a couple of offers
Besides, if I'm looking for a job and I am unemployed, the time I am taking to do homework, I can be meeting with recruiters, preparing for interviews, brushing up on my skillset, etc...
Here's the other bottom line: A company wants to use your industry and creativity to make more money than it will give to you. You don't owe the company jack. shit.
I agree with your bottom line as well. Both are true! It's a negotiation, and it isn't zero-sum. Working is good for both the employee and the company. Neither is in a universally stronger negotiating position.
I'm not annoyed by bad interview processes and strong preference for false-negatives, I'm annoyed by that in conjunction with the claim that there is a shortage of competent workers. There is some lower-hanging fruit to take care of before that claim makes sense, and improving recruiting processes is among the lowest hanging (along with raising wages, obviously).
> Here's the other bottom line: A company wants to use your industry and creativity to make more money than it will give to you. You don't owe the company jack. shit.
Unfortunately, unless the sides are favored to the labor market (in our cases it is not, and even if it was, there is always outsourcing), this equation is not equal.
Both are true, but it is an extremely unbalanced equation. At the end of the day, the company has the money and you have the skills. Unfortunately only one of those helps you pay the bills -- unless you can figure out a way to pay your rent in exchange for your skills.
This is not true for the industry we're talking about (or at least not as true as you seem to think). It really is in companies' best interest to be attractive to skilled workers in this industry and there really is a chance they will find themselves struggling if they get the wrong reputation. It seems (to me) like this is a major undercurrent in IBM's struggles, for instance (and outsourcing has not been a silver bullet). Thinking the balance is more skewed than it actually is is one of the things that skews the balance.
> a take home technical exercise
> prove to me you can (do the work)
A take home exercise only proves they can get the work done elsewhere. It doesn't prove that they actually did it.
This whole thing misses the point. We need WhiteboardFreeAndAlsoTakeHomeFree. In other industries, my resume and a quick discussion verifying that I actually did the work I said I did is suffficient
So say you are at a job. fairly happy. Maybe not your dream job, but you decide to look around and interview with me.
I could present you two options:
1) I just hire you after a 30 minute chat. 50% you don't work out, and we fire you after 30 days.
2) We have a longer interview. We whiteboard. We take home test. We whiteboard some more. 90% chance that you don't get fired after 30 days.
Which would you prefer? As a business owner, I am happy to just try people I don't know well - IFF they are willing to accept that there is a very good chance they are fired within the first 30 days.
There is such a huge range in developers. The haves vs the have-nots can be a 100x different productivity difference. I really want to try to tease that difference out. It is really really hard.
Actually I would greatly prefer option 1. I am very confident that given nearly any situation I will prove my worth and have never had an employer disappointed with my work. I am not as confident I would complete a contrived coding skill challenge on the spot on a whiteboard because I never work like that in reality. My job is to sit here and solve unique problems by any means available to me in a maintainable and scalable way. I do that very well and that is hard to test via whiteboard implementations of contrived examples.
That is how it works in other fields. One of our managers asked his friend that owns a body shop how he finds good employees. He basically has a chat with them, feels them out, then hires them. If they don't work out he hires someone else. Perhaps it shouldn't be as stigmatic as "firing" after 30 days but you have instead hired someone on a 30 day contract with option to hire permanently. If they don't work out, that option is not exercised and that person is free to go try somewhere else. I can tell pretty quickly from chatting with a developer if they are a total fraud. I can't tell really quickly who will become a great dev, I just know when someone has the chops to have potential.
#1. I'm totally fine with firing after 30 days if I don't work out. I'm positive it won't, so I'd prefer to prove myself doing real work.
That's the way it used to be done and I think we should go back to doing that. Read the resume, ask some questions about the resume, 30 minutes to an hour full interview, done.
You get hired on, put on a probationary period, if they don't work out, fire them.
> 1) I just hire you after a 30 minute chat. 50% you don't work out, and we fire you after 30 days.
Personally, that's my preference. If I don't think I can do the job I'll know after a 30 minute chat.
Same here. I feel I have vastly more control of my work performance over the course of 30 days than I have control of the assessment of 6 interviewers after seeing me for 30 minutes.
In other words, I think there is far more than 50% chance that things work out after 30 days and far less than 50% chance that I impress 6 interviewers.
Right, then maybe it would self select out those that can't do the job.
Interesting idea. If you fire fast enough it may not actually cost that much more money to a business to hire a bit more as long as you fire the bad ones fast.
> 2) We have a longer interview. We whiteboard. We take home test. We whiteboard some more. 90% chance that you don't get fired after 30 days.
But a 400% increase in that you find some sort of excuse not to hire me and I won't even get hired in the first place. Your percentages only make #2 more attractive if there's a more-or-less equal chance of being hired both ways.
Well, #2 makes more sense if you value stability and not suprises.
I suspect many people would rather do 10 interviews and then get 1 job they keep for a year+ compared to 2 interviews and a 50% chance the job that hire them, fires them in 30 days. But I could be wrong, mostly basing on what I would want.
There's such a thing as contract-to-hire for this exact scenario.
Why did the industry seem to forget this?
Most good developers are already in a stable job.
I would not leave a w2 job for a contract job. Nor would any good developer I know.
This is why the industry "forgets" it. It's not that attractive to a good candidate.
I've done it many times. It's attractive if the offer is attractive. Last one I took was a $20k raise over the prior job. The one before that was a $30k raise. It's only risky if you're not confident in your skills.
No way! The lamers you end up firing (after 30 days' pay) will easily make a living out of it. They could even be PARALLEL lamers, pulling 30 days pay off multiple sucker companies.
At my current company, we fired a guy after 3 days because he misrepresented his skills. I didn't interview or hire him, but I suspect his buddy did the interview.
Hah, you should be able to see the pattern pretty quick?
Wow, 8 jobs last year? Busy life?
It seems like they could easily drop half of those jobs off of their resume. That's usually the advice given for people who leave jobs after a short period.
I really don't buy that a resume alone can sufficiently put across your actual level of ability as a Software Engineer. And there are plenty of people who are better talkers than doers. One way or another I think it's entirely reasonable to want to see actual code.
If you want to do that by way of an open source portfolio, sure. But not everyone has open source contributions, so then I would need some other process for acquiring a work sample.
I have seen many people here say you should hire people on a contract basis for a few days/a week. I don't know what it's like doing that in the US, but for the UK market that sounds like a ton of administrative overhead for both parties to take on in order to establish if they are even a good fit.
This style of interview works in plenty of other professions that are much more rigorous than software development. You'll be hard pressed to find an aerospace engineering company that is going to ask candidates to evaluate integrals on the whiteboard that they'd normally look up in the CRC or implement a numerical solution for (probably using a prepackaged library), for example. Yet you're proposing an analog of this very thing here.
> But not everyone has open source contributions
In my experience, anyone we've hired WITH open source contributions has been 10x better than those without.
That's (among other things) what the walkthrough / debrief is for.
I like the format, but it obviously has other downsides, such as a higher up-front investment from the candidate.
That being said I found the format enjoyable last time I went through it. It was less stressful during the interview itself, and I felt more confident walking in. I would recommend it to companies who are considering it.
That would show you they have a strong network of good coders and can communicate specs and get quality code done in a timely manner. You should hire them for management!
Might be worth clarifying whether this is “don’t use a whiteboard during interviews” or “don’t ask candidates to code obscure algorithms” on a whiteboard.
We use a whiteboard in our interviews for sketching UI, and sometimes drawing a database table. Generally candidates find this much easier than talking through their design, and we think that interview very closely represents what we do on a day to day basis.
There was a good article that popped up on here a couple months ago about this: "Organizational Skills Beat Algorithmic Wizardry". [1]
If you're asking candidates to balance a binary search tree when no one in the history of your company has ever had to do that, you're asking the wrong questions.
[1] http://prog21.dadgum.com/177.html
Yep. Which is why we don’t.
We do a mock technical meeting with multiple members of our tech team and the candidate presenting options, talking through pros and cons, etc. Much of the interview is about assessing communication rather than technical skills.
The interview is basically a standard tech meeting that we hold regularly in the team to discuss designs of new projects.
Sorry — that comment was meant in support of yours, with the "you" being generic :)
Agreed, whiteboard are way too useful to leave out. That's why we have them in every room in the office, not to scare candidates when we happen to conduct interviews in them.
In my opinion it is all about aligning your questions with the day to day task you expect from people. If the work is very algorithm heavy, by all means probe for that in the interview.
I base my interviews around one main question: assume the tech challenge you written beforehand is a prototype for a complete product/service. what are the steps necessary to get us there and beyond?
I do not care about the candidates tech preferences or the exact solution as long as the argument for it is coherent and not just based on heresay. Then I probe for a couple of things within their solution where I see potential weaknesses based on their argument, experience or code in relation what we look for. It is more about problem solving, decision making and communication than computer science.
Candidates are free to just talk trought it or use whiteboard/paper to draw it out. But if you choose the former you better not forgett what you said five minutes ago.
I thought it was "you won't have to use a whiteboard at this workplace" lol.
I agree.
Obscure “I had to google it before asking you” problems are the problem not the whiteboard. I think whiteboards are good visual aides.
Sometimes I just ask people to draw a frog though.
We often draw hats and shirts, but we do that but not the candidate. That happens to be related to our domain.
I feel like it’s ok to just say “you dont have to do programming on a analog device”; if they want to write on paper/whiteboard/with a tattoo gun I don't give a damn, but they wont be asked to write actual code. That’s what it means to me.
I really like them for Infrastructure or systems diagrams, too!
Yeah, that's key. "Explain how your piece fit into the bigger picture— what other things does it talk to, how tightly are they coupled, how are errors propagated, etc."
The trend of whiteboard interviews is quite catching up.
I was just speaking with a co-worker about how I consider this interview culture to be a race to the bottom in how much unpaid work companies are getting from prospective employees. When I interviewed with Facebook they do back to back whiteboard interviews for about 6+ hours then sometimes they give "homework" problems, etc. Even skilled candidates can't refuse because a less skilled candidate will come along willing to make the sacrifice.
I don't have any solutions only complaints and bitterness, sorry.
Do you think Facebook is taking pictures of whiteboards after interviews and putting the code into production? I agree day-long interviews are a problem, and homework problems are offensive (plenty of folks refuse them btw), but it's a stretch to say companies are using interviews as a way to get unpaid work.
Man, it would serve them right to have tried using my whiteboarded regular expression parser. /still bitter
The premise is not that they're getting unpaid work in the form of programming, just unpaid work in general. If they replace 8 hours of reviewing a candidate's open source contributions and/or side projects with 4 hours of random walks through an undergrad CS textbook, then they're using the candidate's unpaid work to cut down on their HR spending. "How many round piano tuners are there in Fizzbuzz" isn't programming, but it is HR.
Do most programmers have significant amounts of open source contributions? I only had a couple of years of life when I had the luxury of enough time for that sort of thing.
At my startup we pay interviewees a decent hourly consulting rate for take-home problems or full-day followup onsites (after the first interview).
We only do them when we are pretty excited about a candidate but need more information to make a hiring decision.
It's cheap, really, considering the cost of sourcing candidates and the cost of making a bad hiring decision.
When I'm looking for a job, I usually have 3 or 4 strong prospects and looking back on my spreadsheets that I keep when I am looking, it usually takes about 2 weeks from the time I reach out to my network of recruiters and I have an offer (the shortest was 4 days from looking to having an offer from what was then a top 10 Fortune 500 company).
Why would I go through that trouble? I have 20+ years of experience that I can explain how I architected and developed from the website, to the middle tier, to the database to multiple VPCs on AWS and dev ops. I'm not going to do homework.
When I was first starting out I might, but living in a major metro area that's not on the west coast, there are more openings for senior devs/architects than people to fill them.
If a company wants to try me out, I'm more than willing to do a contract to perm, but my hourly rate is going to be enough to cover self employment taxes, make up for not getting paid for time off and at least two or three weeks between employment.
Just to provide another perspective...
We have a take home test. It is an obviously fake problem, but we review it to the grade we would for our production code.
We don't find that candidates turn down taking the test - it's capped at 3 hours so that it can't take up lots of time. We do find that we have a low pass rate, and even among candidates with 10+ years experience, we still find candidates who write code that is just not good, and who haven't read the spec.
I'm not at all suggesting that you might be in this category, and if you can find jobs you want that don't require tests that's great.
However, there are people with plenty of experience, who can architect things brilliantly, and who can't write a ~300 line program in Python in a way that satisfies a spec or in a way that anyone would be able to maintain. That's important for many companies, and so I think tests can be one useful part of an interviewing process.
> If a company wants to try me out, I'm more than willing to do a contract to perm, but my hourly rate is going to be enough to cover self employment taxes, make up for not getting paid for time off and at least two or three weeks between employment.
What does that number look like? 10k a week?
For a 1099 contract where I'm responsible for my own self employment taxes....
2080 - hours in a work year
-80 Paid Holidays
-120 Paid Time off
-120 Gap between employment (based on my history)
= 1760 work hours per year
FTE Salary * .028 (Employer Medicare Contribution) + $7967 (maximum social security amount - 128500*.062) = minimum salary
minimum salary/1720 hours = pay rate.
Take a reasonable salary for my market as $140,000, that is about $152000/1720 = $88.37/hour minimum.
Adjust the number based on market, skill set, and the amount of time you estimate between jobs. I'm also not taking into account health benefits that you would need to pay yourself because I'm covered under my wife, or 401K matching. I would add 4% to the hourly rate to cover 401K matching.
A good rule of thumb is 3x the rate you would get as a full time employee this might not be enough in the US given the high cost of health cover for a self employed person.
I'd recommend dropping the full day onsite. I mean what the hell do you talk about for a full day? The small talk must be unbearable after the first 30 minutes.
So you want a company to hire somebody after talking them for 30 minutes? Based off of what? You honestly can't imagine talking about technical software problems for more than 30 minutes? That's a pretty big red flag to me.
I'm assuming you are technical yourself, otherwise please disregard. What you are unintentionally implying is that you can't spot a fraud by talking to them about your chosen profession within 30 minutes. If you can't, you shouldn't be doing the interviewing. If you can, why waste 5.5 hours of the company's time per candidate?
Plenty of people are great talkers about profession, big picture, architecture blabble and so on. It does not correlate with coding ability.
Its not causation, but I'd say it correlates to being a thoughtful person. Drilling down on the architecture blabble to see how deep their understand goes will absolutely tell you whether they could code a solution.
Is there a person who claims to be a coder that has configured and launched a multi-environment microservice with several attached artifacts like DBs and queues, that knows in depth what each is doing, that can't code? At that point you're just testing whether someone has memorized syntax details that they might usually get from an IDE or a 3 second google search in regular development. I have been writing production JS for years but still have to occasionally look up Array.slice vs Array.splice parameters, especially if I haven't done a lot of data munging in a while. That may indicate to a person giving an interview that somehow I'm "faking" it, when in reality its just a momentary stall-out in my brain's lookup tables.
> Is there a person who claims to be a coder that has configured and launched a multi-environment microservice with several attached artifacts like DBs and queues, that knows in depth what each is doing, that can't code?
Funny that you mention it. I know a person who can't shut up about microservices, Kubernetes and CI and can't code their way out of paper bag. They had no problem passing one of those interviews.
And sure rote API memorization isn't the point. But if a candidate can't code say bubblesort (or any kind of sort) in 30 minutes they are not very good at coding - and it doesn't matter if they can quote their architecture patterns by heart.
> But if a candidate can't code say bubblesort (or any kind of sort) in 30 minutes they are not very good at coding
Thats bullshit. Sorry. Most developers will NEVER implement a generic sorting algorithm except in a code interview. You say you don't want rote API memorization, but you do want rote sort method implementation memorization. Yes you can be a system or tools developer and maybe have a higher chance of doing it, but its 99% likely even then that you just use a library for it for the language you're using, or copypastaing it from someone who's studied the specific implementation for your lang.
I disagree. You aren't asking the right questions or probing enough.
When you start probing enough it quickly becomes practical/coding interview we all come to detest. Sure enough, that way you might not even need 30 minutes. But shop talk alone doesn't cut it.
Maybe your “shop talk” is different from mine, but I can suss out fakes from the first sentence usually. You’re correct in that they go straight for the broad architecture strokes, and they speak in vague terms. Trying to get a solid grounded answer is like trying to nail jelly to a tree. That’s when you know they just read some wikipedia articles and don’t know the practical applications of the words they’re using.
Those have to be really clumsy then.
There's a number of people with CS or SW engineering degrees who can't actually code. Those can often quote you Gang of Four patterns, discuss broad slices of tech and draw architecture diagrams on the board. But still can't code.
I know, I work with several. I’m pretty good at spotting them because I studied them while working with them. Fun fact: if you poke in the right places, you’ll see that they don’t even understand GoF and architecture, they just regurgitate the what they memorized.
I find that highly dubious. CS degrees from where?
Interesting and the first I’ve heard of this and the first confirmation I’ve heard that cost-wise it’s not egregious to do so.
It still seems strange to me to pay for throwaway work but I can also see how that’s really just my personal sentiment / bias and this is probably a pretty good and balanced approach.
Except some of us already have full-time jobs and don't want to spend the weekend working on quizzes, whether it's paid or not. And unless you're very generous with timing taking a day off of work can be a logistical issue, not just a pay issue.
This type of interview self-selects for people with fewer family/non-work obligations, who are willing and able to spend their free time still programming.
Sometimes you might have to inconvenience yourself to improve your career.
Ya, if you paid double market rates, I'd do it. I suspect most people who do this sort of thing probably pay market or worse.
Cost of doing business. Not willing to put in your free time? No need to worry about getting a new job.
Are you also job hunting at work, doing interviews on your employers time, etc?
You're ignoring the context of the comment you're replying to.
That guy already has a good job. Another company is trying to poach him. Why would he, or any other qualified and gainfully employed candidate go through that unnecessary homework bullshit for a company that's trying to recruit them? The company has its priorities misplaced. They should be trying to convince the candidate to join them, not putting up barriers to their most qualified candidates.
I would totally do homework bullshit for a company/opportunity/compensation that I saw as “better” than what I have currently, but would not do so for anything else.
So your salary + $1? How much better? I'm curious. I'd probably go through this for maybe 50% above market rates.
Insane. Let’s say it’s half that and you make $100k a year. You wouldn’t put in 2 hours for an extra 25k/yr?
I mean this isn’t a lottery, and hopefully you’ve reasoned there will be a high probability of getting hired; say 50%. So then the expected value of your time is $6.25k/hr.
Go, take the moral high ground, and don’t do the homework. That helps rational minded folk achieve an even higher hit rate.
I'm already at top dollar in my area and have been for years. Maybe that's why I despise it so, I have nothing to gain from the trouble. I haven't had to do a cold interview since 1999.
There are much better and less humiliating ways to get top dollar. Work your ass off and build reputation is the biggest one.
If you are a company and you have to call a recruiter, you're already behind. Putting people through that stuff just makes it worse.
Not sure! I guess it depends on how much homework.
We aren't paying for the work-product, we're paying for the time and inconvenience. It's more to show respect for the candidate's time than to actually "hire" them to do an interview.
I mean, even "discount" recruiting platforms like Hired and Vettery will cost 5-figures when we make a hire through them, and traditional recruiters often charge 20% or more of the new-hire's first year salary. A few hundred bucks to get some more information about a prospective coworker is cheap.
Reminds me of the old joke about a scam company who breaks up all of their problems into interview questions and never actually hires anyone. Obviously the composition, decomposition work and quality control would be more labor for worse results but it is easy to see how frustrated interviewees got the concept.
I've seen companies that maintain open source projects have interviewees work on that actual open source project for 4hrs as an interview.
I recently interviewed at a company who wanted to be as hot as Google and Spotify and had copied their interview process and their organization schema.
I said no thanks and moved on to other companies. Felt good. I just avoided working in a culture where there will be high pressure to always perform. :)
Yes, companies don't realize people can get incredibly wealthy working for places like Google. That's why people tolerate that as well as the prestige of having a place like that on the resume. They are so self centered, they think Acme Trucking holds the same allure with "market rates" and shitty benefits.
I'll just point out that companies are spending a lot of employee time doing these interviews, and if they're picky, that greatly increases the cost. Every day of interviews of someone they don't hire is money down the drain.
They can afford it, but still, it's far to say that the wasted effort isn't entirely one-sided. Hiring is an expensive and inefficient transaction.
Figuring out how to make this both more fair and more efficient is a really tough problem.
So the standard FB onsite interview loop is 5 * 45 min, with a 45 min break at some point. Those aren't all whiteboard interviews. It might vary from role to role, but I don't think homework problems are common.
I did get the 45 minute lunch break and dined with another FB employee during that time, but I was in the room doing whiteboard problems the entire time otherwise minus them showing me the office space as we went up a few floors when I first arrived. This was in Seattle, if that matters. I think the room was labeled Occam's Razor and was super small, barely enough room for 3 people to stand.
So not only are you dealing with the standard fare interview nervousness, etc..., but you're also feeling claustrophobic the entire time. Brilliant process they've got up there. People look at me sideways when I tell them that I have no interest in working for places like Facebook or Amazon or Google and these sorts of anecdotes are some real high-quality validation after having had to justify that stance for years.
I would never work for one of those large companies. You really only need one household company on your resume during your career, then you can ignore them.
Those companies are the most visible places to get a high-paying job. They pay at the top of the market. Sure, a niche software company may pay its senior engineers higher, but even well-connected developers may never get any exposure to them.
If you want to get that first house downpayment for the Bay Area, I'd say it's almost essential at this point.
That is exactly why I applied. I run a startup and we wanted to get the clout I was going to spend a year at FB as "time served" for the startup.
I think a lot of these reactions (and this website) are bitterness following a rejection. I'm not sure why people take it so personally when a company doesn't accept them. I've done whiteboard interviews and got a rejection before, and I thought it was fine. Maybe if you're putting all your eggs in the same basket and want THIS company to accept you, that might be annoying. But that's not a very good life strategy.
Is the implication that they're using your code in production after an interview?
No, I don't think any of the code is of any value, it's the time and energy up-front investment is increasing significantly. I understand they are trying to protect their investment (2 years at Facebook would have been over $500K in cost to that company all things considered) but when is it too much and how would we stop it before it gets there?
Having gotten contacted by Facebook, I eventually refused when I realized they were going to ask about algorithms for a technical role that is in no way relevant to that on the job. Skilled candidates can refuse just fine and wait for something better and more realistic that comes along.
Whiteboards are not the problem- rather the questions that candidates are asked are bad.
Asking someone to solve an obscure mathematical problem (the Two Egg problem seems to be in fashion) doesn't do much except test whether the candidate has memorized the answer.
> the Two Egg problem seems to be in fashion
When not in an an interview I like playing with these kinds of problems, but if I look it up I'll probably find interview prep where it shows the answer as well.
Could you summarize just the problem please?
"You are given two eggs, and access to a 100-storey building. Both eggs are identical. The aim is to find out the highest floor from which an egg will not break when dropped out of a window from that floor. If an egg is dropped and does not break, it is undamaged and can be dropped again. However, once an egg is broken, that’s it for that egg.
If an egg breaks when dropped from floor n, then it would also have broken from any floor above that. If an egg survives a fall, then it will survive any fall shorter than that.
The question is: What strategy should you adopt to minimize the number egg drops it takes to find the solution?. (And what is the worst case for the number of drops it will take?)"
http://datagenetics.com/blog/july22012/index.html
Uh, if you drop an egg out of the first floor window onto cement, it will break.
The answer should be a form of binary search, but since you only have two eggs, you'd have to do a sequential search from the bottom floor. You have one chance at a mistake, so I would almost say start with the 2nd floor, but I know it will break on the first floor.
I don't see how that in any way gauges programming what so ever.
>The answer should be a form of binary search, but since you only have two eggs, you'd have to do a sequential search from the bottom floor. You have one chance at a mistake, so I would almost say start with the 2nd floor, but I know it will break on the first floor.
yeah, sorta. essentially you partition the 100 floors into some number of groups (i forget what partition length is optimal) and drop the first egg at each segment until it breaks, then go linearly from the previous one
A quick naive not completely optimal but still relevant solution:
Since you have 2 eggs you can start on the 33rd floor. It breaks you can use the 2nd egg from 1st to 32nd floor.
If it doesn't break, use the first egg on the 66th floor. If it does break, the answer is between 33 and 65, if it doesn't the answer is between 66 and 100. That cuts the maximal testing time time from splitting at 1/2 (1st egg at flr 50), to 1/3rd.
To optimize further, since in the bad case we'd have to search up to 34/35 floors, you can mathmatically find a number where you split the drops at an interval (IIRC the "real" optimal answer is 14, then 13, then 12, etc), which means you're doing less explicit checking.
At 14 - i, you have 10 drops to find a range of 13 - i floors in which the optimal path is found. So instead of a worst case of 35, you have a worst case of around 17.
I find the best way to approach this sort of problem is to first find one optimization, and then use the same process to find if there are further optimizations.
It's not the worst type of puzzle except if you do not know of a solution it might take a while to figure it out.
It does relate to programmer's ability to amortize worst case running time.
Once you realize that 1 egg requires linear search you just have to realize that with 2 eggs you have to decrease the level increase by 1 on each success.
Of course those must be some crazy genetically modified eggs.
>It's not the worst type of puzzle except if you do not know of a solution it might take a while to figure it out.
I would guess the interviewer probably rates people who have memorized the solution far higher than people who have never encountered it and solved it with their brains.
Much appreciated.
Hmmm, that's an interesting question. It's the Price is Right closest without going over solver...
Practical answer, drop from the first floor and then the second. You're probably not going to make it to the third.
I don't want to put my actual answer in case others want to solve it. But I will say there's an obvious solution, but if you apply one of the hard problems of computer science you might eke out a bit of an edge.
EDIT: Actually I don't think that's right. I think the answer varies based on your expectations, and should be tuned after each run if you run the same test multiple times. I do have a good starting point in mind though.
Final edit but too late to edit my post:
After thinking about it on the ride home today, I was not quite on the right track before. Again don't want to post my solution because the entire point in asking was so that people could see the question and work it through themselves.
> closest without going over
That's The Price Is Right, not Wheel of Fortune ;-)
My bad, I haven't watched either in at least a decade.
I was definitely picturing The Price is Right at least.
Because eggs that don’t break when dropped even 3 ft (never mind multiple floors) are a thing? Perhaps I am just projecting my Earth based experience...
Maybe it's a shittest if you actually try to apply some theory rather than using your common sense and say that "Based on the last time I was in New York, it will definitely break dropped from the first floor"
Whiteboard problems are used to show that you can work through a problem, while communicating well about your thought process. They're also good general intelligence tests, if administered correctly.
I think an obvious reason for this type of interview problems is because developers at work usually talk loudly while solving problems and write code without the help of any external resource while the clock is ticking in the background. Oh, wait...
What both you and the parent post you're deriding are missing is white-boarding (when done correctly) isn't about the problem at all. It's just an easy way to get a candidate in the room and construct a technical conversation.
As an employer I don't care how great your skills are, if you can't walk me through your thought process for 45 minutes you're not going to be overly successful. If you can't incorporate feed back or engage when pressed to change your design you're probably not going to be successful.
Do your recruiting right and you shouldn't need the white board to simulate whether the candidate can code at all (I think white-boarding with the intent of looking for named algorithms is a waste of everyone's time), instead you want to see if they can synthesize information, engage with other people, and otherwise display the soft skills that tend to be much more useful metrics for employee success than coding aptitude.
Granted, my industry is known for not having deep coding problems to solve, this strategy might not be as useful in verticals that require more technology chops.
>As an employer I don't care how great your skills are, if you can't walk me through your thought process for 45 minutes you're not going to be overly successful. If you can't incorporate feed back or engage when pressed to change your design you're probably not going to be successful.
The post you're responding to makes a great point that this isn't the normal working environment for 99% of actually-writing-code developers. I discuss and iterate designs with my coworkers/leads/managers/architects just fine, but a surprise algorithmic problem you have to simultaneously solve, draw on a whiteboard, and constantly present to an audience while being timed isn't something I have been great at. I bet many others aren't either. I got good at it after a few interviews last time I was looking for a job; but if I hit the interview trail again today, I'm confident that I would flounder in that setting for a while.
Imagine if, instead, the candidate worked out a project -- maybe at home, or maybe in an interview room for some time -- then you take time to review and understand their work and then go through this back-and-forth process of discussing their work. This would also benefit the candidate's understanding of how you collaborate to problem solve in your work environment.
The problem with this approach is that it requires the interviewer to actually invest time to understand the candidate's work.
Anyway, I hope you see why many see it as a problem that the process for a candidate to succeed at a job interview is to practice interviewing skills rather than demonstrating their engineer skills.
> As an employer I don't care how great your skills are, if you can't walk me through your thought process for 45 minutes you're not going to be overly successful.
As an investor, I don't care how great your skills are, if you can't come up with your new business idea on the spot and walk me through your thought process as you're doing it, you probably suck.. right?
Nah.
Unless you already have the answer, the first step is to come up with the solution. This could involve time spent alone just thinking, or brainstorming with coworkers, or researching the problem on the net, etcetra. What the right steps are depend on the problem, available resources, your background, even your preferred way of working.
The second step is to present and dissect the solution. I dare say this is the point at which the majority of engineers have no problem whatsoever talking about their thought process.
Do you work in an industry where engineers genuinely have to walk somebody through their thought process while they're trying to think up a solution?
If a problem comes up at a meeting and you don't have a solution on the spot, you take note and discuss it later or add it on the agenda for the next meeting. I think that's how people work.
I don't care about where, when, and how you came up with the idea. I care that you can present and discuss it once it's ready for that.
I think this is why many prefer the take-home test. They can focus on the solution. Then, at the interview, they can focus on the talk about the solution. It's also a great equalizer as e.g. people coming to work in a domain they have less experience with can take more time to research the solution space.
The complaints about homework taking up too much time may be valid, but I'd be most happy to trade 2 hours of interviews for about 75 minutes of homework and 45 minutes of interview.
"When done correctly" is the key here. Alas, in my experience I was usually heavily penalized if I kept my mouth shut for as little as 15-20 seconds because interviewer "couldn't see my thought process". At the same time expecting from me to write syntactically and semantically correct, industrial strength bug-free code with optimum space and time performance for a problem I never ever encountered with the interviewer "helpfully" distracting me 4-5 times a minute.
That they're used for those things doesn't mean they work. They're especially not good general intelligence tests.
I'd wager that the fact that many high power companies use whiteboards for interviews point to the fact that they're useful.
This is appealing to authority/popularity.
But since we’ve gone there, many high power companies also have a lot of chuckleheads running around.
Divining rods are trusted by many, and have been in use for centuries. If they weren't a good way of locating water, why would so many people use them?
The fact that there are so many successful fast food chains points to the fact that the food is good and healthy.
Maybe they're only useful when you're a high power company with $200k+ total comp and more applicants than you know what to do with.
> They're also good general intelligence tests, if administered correctly.
Is there any scientific evidence to support that, or is that just industry hokum?
> They're also good general intelligence tests, if administered correctly.
And communism probably redistributes wealth if it is administered correctly.
If companies want to hire based on general intelligence (not personality, team work, communication and so on) then they should administer IQ tests, which are based on a century of rigorous research. That would at least give you a fighting chance of measuring G.
Why do you think white boarding can measure general intelligence?
Well said!
Comment: I think whiteboards get unnecessary flak when the real culprits are games and riddles. I would argue that using a whiteboard to convey ideas is a fundamental skill if one has to work in a creative team.
Question: How do you validate that the posted company or job position doesn't need whiteboard? Is it after the fact, as in a disgruntled candidate complaining? I couldn't find any information about this on the site.
> I would argue that using a whiteboard to convey ideas is a fundamental skill if one has to work in a creative team.
Yet a lot of people are never really taught to do it. Perhaps they pick up this fundamental skill just fine after a couple whiteboarding sessions in the company.
If a company is rejecting candidates for not having a skill that is really easy and quick to obtain, they'd better not be one of these companies that keep complaining about shortage of talent.
FYI All your subscriber's emails are accessible at https://whiteboardfree.com/email_subscribers.json since you didn't put any authentication on those endpoints. It's a basic Rails app so I'm guessing other users may got a hold of the list already.
I guess their Privacy Policy checks out https://whiteboardfree.com/privacy
belly laughs
This is a pretty crummy wait to disclose a security vulnerability.
Yeah, this one is pretty bad. But let’s try to do better, not worse.
So you're telling me that someone who created a site to post job adverts for jobs that don't test programming skills, has managed to write a security vulnerability into the site? Never!
I don't agree with making candidates write algorithms they don't need to know in a format that they don't need to be able to write them, but I am in favour of testing candidates thoroughly.
I never realized whiteboarding was an issue. I wish someone would would make one of these sites for open office free jobs.(not the softare)
StackOverflow's job board has the "Joel Test" part where employers posting jobs can check off the points from https://www.joelonsoftware.com/2000/08/09/the-joel-test-12-s... that they fulfill. One of them is "quiet working conditions", which comes close. Before looking it up just now, I could have sworn the point was something like "every developer has a door they can close", but oh well.
Anyway, I hoped you could search only for jobs where this box was checked by the company, but apparently not...
EDIT: It does allow searching for minimal salary and remote jobs, among the wishes listed in a sibling comment.
> Before looking it up just now, I could have sworn the point was something like "every developer has a door they can close", but oh well.
EDIT: Ok, my mistake, my memory is failing me. "Doors that close" is used in a whole bunch of Joel blog posts, and "walls and doors" is used in the original Joel Test description.
But WaybackMachine seems to confirm that the official label from the "Joel Test" has always been "quiet working conditions" and does not appear to have been changed.
Sounds plausible, but the blog post from 2000 also has it as "Do programmers have quiet working conditions?"
So unless Joel went back and rewrote the past, it might be our memories. I am fairly sure that he did have another blog post that stated explicitly that every developer at FogCreek had a door.
I remember the "all developers should have offices" from Joel. Did he change his mind recently?
Wow that would really turn the tables; a job board where companies must not only disclose their interviewing practices and durations but their office environment as well. It would be developer centric so developers can pick and choose the companies they want to work for based on often hidden parameters. Just the transparency requirements would automatically filter out most the chaff companies. I might have to build that out of spite. :)
+1 to this.
Even better, I would love a site that classifies job offers/companies by:
Check out https://www.jobiki.com. We built it in collaboration with people like yourself to search for companies based on information like that. Whether you want companies with pod or cubed desks, or open office. (Among MANY other things)
Unfortunately we haven't yet expanded much out of the Midwest, so it MAY not be of use to you yet. But check it if your interested.
> Check out https://www.jobiki.com.
I went there, typed in "developer" in the search box, and got dropped on a map zoomed in on the Kansas/Oklahoma border and a message saying my search didn't match anything. Took me a while to notice the "zoom out on the map" part. Not sure why I landed in the middle of nowhere in the US, I'm located in Europe. In any case, super unintuitive.
Also, this might be an American thing I'm missing, but which (if any) option of "Pod Office Layout" or "Closed (Cubicle) Office Layout" corresponds to "proper walls and a proper door"?
I'd also suggest adding a filter for minimum salary.
Hope this helps.
Thanks for checking it out and your feedback! I can make some changes to hopefully clear up some confusion.
On Jobiki, we don't have job postings. Just companies and data about them. So the "minimum salary" and "developer" searches are more job related, while we are more company related. Either way, helpful feedback.
-----------------
As for the "proper walls and a proper door" aspect, are there developer/traditional employee jobs in Europe that can have a real office? In the US, unless you are a manager, executive, very experienced employee, you don't usually get an actual office. Maybe its more common in Europe?
At least in Scandinavia I'd say it's fairly common to have a real office. It seems the cubicle thing never really caught on here, but now there's a trend toward open plan offices, unfortunately. Software companies seem to be among the worst in this regard.
I have a real, quiet office with lots of daylight. That's actually one of the reasons I'm staying at this job. That, and the fact that I'm not subjected to low-status micromanagement techniques like Scrum. In short, I have working conditions befitting a qualified adult.
Individual offices for developers are unusual in the US these days, but that's exactly why someone would want that information. There are also shared offices and team rooms.
"Pod" seems to mean completely different things to different people.
"Pet friendly" workplaces tend to be noisy and discriminate against people with allergies, so some users will want to filter them out.
Even if you don't want to track current openings, what kind of jobs a company has is important information.
Thanks for the response. That's very true. We've also been toying with different verbiage for "Pods", as I would even agree, could be many different things.
And good note on the "pet friendly" environment.
We in the processes of adding that information to outline what positions a company hires for. (Whether or not they are currently open or not) Well, more what departments that a company hires for. Going all the way down to job title seems a bit too granular, maybe I am wrong? But by department seems a happy medium?
> are there developer/traditional employee jobs in Europe that can have a real office?
Yes. Maybe not every single developer; there might be two or three or four to an office, but it's still an office with walls and a door. There are companies with more open spaces and twenty developers to a room, but the (much) smaller ones are the usual case from what I've seen. ("Europe" is a big place, of course.)
Very interesting. Sounds like something we need to look into. Thanks for the feedback.
That's not a bad idea. I think I'll make something like this, a site that let's you search for dev jobs with filters that devs actually care about. Any other suggestions for filters?
Wow, there are so many new ways that people are coming up with to recruit software developers. Clearly there is a problem here that needs to be solved.
I understand that the company needs a way to vet people to make sure they are technically proficient for the job. However, there must be a better way of filtering people. I just bailed on a company after they told me what the interview process would be:
- Take home coding quiz where you need to sign up and use their algorithm platform. As you would expect, there was asynchronous programming and a recursion algorithm involved (two for one!)
- 3 more technical interviews. 2 of which were screen share and/or white boarding sessions. 1 is a "riddle part", which I assume is some kind of brain teaser involving a recursive algorithm.
After I saw the coding quiz and all the follow on stuff I just told them I wasn't interested. I was kind of offended, and it didn't put me in a positive mindset towards the company and engineers that worked there.
It all amounts to little more than our industries hazing rituals.
This is what I tell people all the time. That one word sums up nearly every dysfunctional interviewing and filtering process.
Hazing.
It's just monkeys making sure they only hire other similar-looking and similarly-acting monkeys.
I still whiteboard my programming question when giving interviews, can someone summarize the case against whiteboards?
I'm just looking for the candidate's thought process and some pseudocode. I could do the same thing in an empty text file but the substance of the interview wouldn't change.
My feeling right now is that being anti whiteboard is focusing on trivialities like complaining that the office style guide uses Allman vs K&R bracketing or something. But I'm open to having my mind changed if there's something more to it.
Being anti-whiteboard is a proxy for being against the "regurgitate this CS trivia on a board in front of me" style of interview, not against the use of a whiteboard.
Most problems in practice that require non-trivial application of algorithms, dynamic programming and the like have two characteristics that make them differ considerably from whiteboard style quizzes. For one, they're going to be much less contrived (pick a random leetcode question for an example of this, odds are it's contrived). Secondly, they're going to be resolved when developers talk to each other about them and there is recognition of the solution, which most of the time is an application of algorithms and data structures with library implementations.
The ability to whiteboard things like "given an array of integers and a target find the indices of the pair of integers in the array that sum to the target" and "find the first missing positive integer in an unsorted array in O(n) time and O(1) space" provides almost no useful information about a candidate's ability to solve an actual problem the company faces in most instances. There are exceptions, but interviewing everyone for the standard of that one exception is asinine.
I like to use whiteboards for non-coding design questions, like talking through how you would architect an app, drawing DB schema diagrams, etc. It's a tool that we use for collaborating on things like this in the office, so demonstrating an ability to do it is meaningful.
Actually hand-writing code with a pen has always seemed weird to me. As a programmer, I compose code with a keyboard. So I give candidates a laptop and use an interactive screensharing tool (I like https://codepad.remoteinterview.io) where they can run code in the language of their choice in a browser, and I can watch and talk through the process.
Whiteboarding is fine when done well, horrible when done poorly. The variance is on the side of the interviewers.
I've done a lot of interviews over the years, and I've encountered a couple stereotypes:
1. The gotcha interviewers. He pulled out his old algorithm text book and found the hardest problem.
2. The no bullshit interviewer. Asks you to code a somewhat challenging, but solvable problem. No tricks. Basically non-trival fizz buzz problems. The problem with these is there are only so many practical problems you can give before it gets put up on leetcode, and every interviewee has memorized the code.
3. The clueless interviewer. They try to give you a weird mix of an IQ test and a personality test, and maybe throw in a little bit of API trivia while they are at it.
Is whiteboard interviewing so bad that people would go out of their way to avoid it? I mean, I've encountered / can imagine horror stories for all the usual alternatives: a) take this 3-day long mini-project, do it using all of our preferred technologies (some of which you'll be using for the first time) and get it back to us - it should only take you 4 hours! Or b) do this on site, using our machines and environments which will be unfamiliar to you. Or c) speak to some random invigilator over Skype. Or d) do a written test, or e) take us through your GitHub collection, which may or may not exist, etc.
My preferred way of testing people was usually to try to get them to show me that they have thought about why they develop software in the fashion that they do. I would prefer to do this in person, at a very high level, but it often ended up being a written test (where necessary, the language was your choice, or pseudocode). Is this wrong?
> Is whiteboard interviewing so bad that people would go out of their way to avoid it? I mean, I've encountered / can imagine horror stories for all the usual alternatives: a) take this 3-day long mini-project, do it using all of our preferred technologies (some of which you'll be using for the first time) and get it back to us - it should only take you 4 hours! Or b) do this on site, using our machines and environments which will be unfamiliar to you. Or c) speak to some random invigilator over Skype. Or d) do a written test, or e) take us through your GitHub collection, which may or may not exist, etc.
This is the problem. There was recently a large HN thread about the take-home projects and how so many people didn't like them.
None of the options are popular: if you use whiteboarding, people will complain it's a bunch of brain teasers unrelated to real programming work. If you use take-home projects, people will complain about doing unpaid work.
Would professional certifications help? I doubt it -- they would need to be based on standardized tests, which would have the same drawbacks as the whiteboarding questions nobody likes.
To give an alternative approach - I found that in general, I could get a pretty good idea about a candidate using either the verbal interaction or written test.
The idea was to give questions which are almost a polar opposite of the strange trivia style questions you see in certification exams. Instead of the candidate having to remember which overload of an API returns a byte[], we'd instead talk about things like how they prefer to structure a project, and how this feeds into maintainability, etc. If the person was into object orientation, we'd talk about that, if they were into a more functional style, same angle, but I want them to have thought about it from a software engineering side. It sounds wishy-washy, but with just a few pointed questions, you can detect both BS answers, as well as someone who hasn't really thought about it. I tend to find that this quality in an individual is a very important predictor of performance, as it indicates deep care and interest about their craft. With good candidates, you can get really deep into a discussion and know within 5 minutes that they are quality.
Along the way, you'd also get a pretty good picture of the persons strengths and stylistic preferences, which you can use to determine how and where they would fit in with the team(s) you had in mind.
The only downsides are that I wasn't always available for these interviews, and the written equivalent intimidating and could take long. Also, various people swore that they either can't perform well in written or verbal conditions, so perhaps letting the candidate choose between the two might be beneficial.
> Is whiteboard interviewing so bad that people would go out of their way to avoid it?
Yes. If you have problems with anxiety or imposter syndrome its a special kind of hell - especially when you go in absolutely able to do the work and being qualified, and then getting a hard no on the basis of "you're not a good enough programmer". Even for someone without an ego to be harmed, it feels like a cruel and arbitrary way to determine your future financial and societal well being.
I recently had an onsite tech screen at a FANG company that requires whiteboard programming. The problem itself wasn't hard but I missed a requirement that was actually a hint. I shutdown and couldn't move on or work through it.
That said, I don't think whiteboard programming is always bad but it is a skill that I don't exercise and it showed.
My advice, talk through the problem with the interviewer and don't start writing code on the whiteboard until the interviewer has ok'd your proposed solution.
I usually need like good 5-10 minutes of quite time to think about the problem in peace. I just can't seem to do with interviewer staring at me and hounding me to 'think out loud', sorry I can't talk and think at the same time. I have no practise at 'thinking out loud' , i never do it in real life.
And that's why you have to practice.
I was the same but I do a lot better now after having done some 200+ technical interviews and a lot of practice on my own.
200 technical interviews?! What the hell? You shouldn't have to do 200 in your lifetime. This kinda reminds me of those MCSE certification bootcamps they used to have. They didn't teach you anything useful, they just taught you how to hack the test.
Yeah this would maybe only work for simple problems, that one can solve without thinking too far.
At plated we do two coding interviews. The first is concrete, we have a library that is poorly written and their job is to implement a new feature and refactor it. This is a great interview to understand core skills. The second interview is a modeling exercise where we build an app from scratch with escalating amount of complexity but only asking for what the database and api endpoints would look like.
This seems pretty reasonable to me.
My personal experience in hiring is that people should be able to explain concepts in a clear why. So usually I ask to explain an pattern that they say they have experience in. I let him/her choose how they want to explain it whiteboard / pen and paper / words only, sitting or standing. I don't care as long as the person is good in explaining and has a firm grasp of the subject. Both are needed to do a proper job. There are lots of people that are unable to explain MVC in a clear let alone correctly even if they used .NET MVC or spring MVC (or whatever)for 10+ years. After that is seldom useful to ask anything else.. this approach had never failed me.
Do people hate whiteboarding in general or the questions they're being asked to draw?
I can fully understand dislike for puzzles and being asked to implement irrelevant algorithms in pseudo code. The interview should reflect the job being interviewed for.
I've found whiteboarding to be mutually helpful in questions about system architecture. Plus, it's something that we actually do on-the-job.
Yes, but you don't write what you expect to be fully functional code on a whiteboard. A whiteboard is good for sharing ideas. Using it to write and assess code is pretty stupid if you step back and think about it.
I mean ask yourself this: do you ever use the whiteboard to write code to the extent you expect some poor kid trying to get his first job to do? Hell no, you screen share an IDE or paste code in an email.
People used to have to write code on paper (I did this a few times in college ages ago) and that comes from the cost of compiling stuff way back when. Computing time is friggin cheap, that's why no one does it. To expect some new guy to be adept at that ancient practice is silly.
I agree. This is a form of the psychological concept of encoding specificity principle[1] which states that recall is most effective when the conditions at the time of encoding match the conditions at the time of retrieval.
[1]: https://en.wikipedia.org/wiki/Encoding_specificity_principle
For example, what does the ability to reverse an AVL tree on the whiteboard have to do with doing UI design in XAML?
UI Design or a Frontend Software Engineer? I'd expect people working on the frontend to just as proficient in time complexity, space complexity, and performance analysis as a backend engineer. I don't particularly gravitate towards companies working in CRUD apps, but single page applications with complex UI flows and data collection can certainly need those abilities. For one, it's not particularly uncommon to have an actual tree in the UI. How would you represent that tree? How are you going to handle it as it grows to thousands of nodes?
I think few frontend developers would consider the work they do to be "UI design".
Full stack development, desktop, mobile and Web.
A UI tree component is not an AVL tree, plus most shops just stick with whatever comes with the framework, or buy some stuff like DevExpress toolbox.
Also UI knowledge requires design skills and knowing what the stack is capable of, not CS bingo about stuff I haven't used in 20 years.
I can dust off my data structures and algorithms CS book if I ever need to implement one.
This all remains me of a quote I have recently seen: "Most Programmer wouldn't hire themselves"
As long as we keep asking people to do things that aren't like the work that we do, we will continue to get mismatches and people that interview well but work poorly. I don't have a great solution, finding good people is hard.
Don’t whiteboard during interviews? Or don’t whiteboard at all? Dropping the former is good, the latter is bad...
What would be a good selection strategy without technical interviews? I've seen people with good CV and reference projects being hired, only to turn out they literally couldn't program. It's old news that sneaking to a CS degree without learning anything substantial is possible, and those are easily debunked in even simple task like solving FizzBuzz on the whiteboard.
No whiteboard doesn't mean no technical interview. You can pair up on a real computer, you can have a take home, things like that.
Both of these have huge disadvantages for interviewing:
- When working in front of a computer, people get worse at communicating. Even if they try to speak out their thougts, it usually doesn't work well, as thinking and operating an IDE (or whatever at hand) already occupies a lot of brain capacity.
- Take-home exercises are hard to control: You don't know how long a candidate worked on it. Even if there's a target like 4 hours, some might work all night on it - either because they couldn't do it in the allocated time, or because they fear some disadvantage. Apart of that, they might pay someone to solve it for them.
> Apart of that, they might pay someone to solve it for them.
I think that can be solved by having the candidate talk about their solution and how they made certain choices.
I love FizzBuzz and am constantly astonished on the amount of candidates it weeds out in the first in-person interview.
What about CS-riddles/competitive programming free companies? Who only ask relevant to the job questions during interviews.
I've been programming since BASIC on a VIC-20 sometime around age 6. I can code in like 10 languages and have written everything from crypto and network protocols to distributed systems, database sync engines, and high throughput video compression/decompression codecs.
I always fail whiteboard interviews. I can't code at all on a whiteboard.
You're the false negative that nearly every software company seems intent on avoiding at all costs...
...and probably a better software developer than 99% of the people they end up hiring.
I ended up starting my own so meh.
https://zerotier.com/
Aside: With about 40 job postings, each at 300$/month, they're probably making $144k with little to no operational or infrastructure costs. They should be on https://www.indiehackers.com/products.
If you're interested in someones coding prowess and process, review something they have written with them. It's far more effective than commanding some nervous candidate to get in front of a group of people and start writing code on a wall.
There is a Netflix and Lyft post, but going by past experiences there are whiteboard coding problems. Are these companies spamming it? How do they police these possibly fake posts? After all, the thing recruiters care most is how many people apply.
...do people really hate whiteboard interviews enough for a website like this to make money?
Some proportion of perfectly good engineers couldn't pass a whiteboard interview to save their lives, for much the same reason I couldn't pass a job interview that measured communication skills by karaoke performance and emoji literacy.
The company needs to evaluate your problem-solving skills though. I mean, I'll admit that writing code on a literal whiteboard is more physically arduous than on a keyboard or even a piece of paper, but if the root issue is live coding challenges themselves, I don't see a way around it.
Some companies look at your GitHub profile instead, which imo is much more unfair, because it presumes you've had the free time and the desire to work on side projects, which shouldn't be a requirement for a job.
Maybe companies should adopt an either-or policy? Give the interviewer the option to demonstrate their skills in the form that works best for them.
Up next: elevator music free jobs. Dev jobs at Companies without that predictable elevator music.
Maybe I’m missing the point though.
The point is "post a job for $xxx". This shit has gotten popular after the success of weworkremotely
I don't mind whiteboarding. I do mind questions that are puzzles or algorithmic in a way that does not directly reflect the company's current software.
I usually (ok, almost always) flesh out designs and pseudocode with pencil and paper anyway, so changing to use a whiteboard isn't an issue.
Standing in front of a handful of strangers glaring at me while I try and figure out some stupid algorithm they had to Google themselves is hazing and should be stopped.
Edit: Also, dinging me for small syntax issues on a whiteboard when I'm used to using an IDE is not really cool.
Maybe Lyft varies it's interviews, but:
1) https://whiteboardfree.com/job_posts/52 - Lyft - Data Engineering
2) https://eng.lyft.com/interviewing-tips-for-android-engineers... - "Expect a few coding questions either on a whiteboard " ( Dec 21, 2017 )
Pretty sure Netflix (also has a job listed) does whiteboard based evaluation too.
You know, when I'm interviewing someone my VASTLY preferred approach is to have them walk me through something they've coded that they're proud of. That's not aways possible, but often it is. Give me a good slow walk through your github repo and tell me where and why you made the choices you did, what the tradeoffs were, etc. . . Better than making some poor bastard jump through hoops on problems that only vaguely resemble the ones they'll be solving at work.
Yes this is the best method I have found. Let them explain something they have worked on that provides them with experience they believe qualifies them for the position. Here I can surely expect them to have depth, assess their communication skills, be able to explain further on questions from me etc.
I'd love to hear alternatives, rather than a list of who doesn't.
I don't use whiteboards for coding tests, I use whiteboards as one (of several) ways to evaluate how someone thinks on their feet without having Stack Overflow & Google at their fingertips. I don't only want to know if someone can code, I want to know about their process, how they think about things, what assumptions they start with, and what kinds of things they like to explore. I want to see some personality. Take home problems don't measure that very well, nor do coding tests with online resources available.
I've never met a perfect interview, but what are some suggestions for more acceptable and/or better ways to evaluate how someone thinks on their feet?
I remember hearing that Google's done quite a bit of research on hiring practices vs outcomes, and that one of the leading predictors is having external groups do the interviewing, i.e., making sure the people doing the interviewing have no stake in the outcome.
My hypothesis (of what people want in an interview):
Interviewer to engage in conversation with applicant about applicant's prior work in detail. If the applicant can give sufficient explanations of systems/projects and the interviewer can ask appropriate probing questions and get sufficient responses to those questions from the applicant.. then the interview can be concluded positively.
The crux of it, from the applicant point of view, is that they want to be believed and treated "professionally". Further it's "insulting" to be questioned about anything other than specific experiences. That's why you see objections to every sort of "objective" line of interview:
sample projects - too much time, why am I coding for you for free?
github/open source - I have hobbies outside of programming
white boarding/google doc coding - not my environment for writing code. I need emacs with xyz keybindings or we're not going to make progress.
algorithm problems solving - the job doesn't entail re-implementing hash tables, or sorting algorithms... The question(s) are not germane to the position
etc etc etc
> they want to be believed and treated "professionally". Further it's "insulting" to be questioned about anything other than specific experiences.
Yeah, I agree; I have the same hypothesis that this is what people want. And it's a problem to assume and/or act like talking about anything other than specific coding experience is in any way insulting. That's not exactly a realistic expectation, nor is it a great attitude for getting hired.
Personality, behavior, poise, attitude and communication skills are absolutely relevant to someone's ability to do their job on a team. If you can't think on your feet or answer a question without a computer in front of you, and you start getting angry, how will you behave in meetings? That behavior is precisely one of things I look for when I ask candidates to think on their feet.
If someone assumes that being asked certain questions is "unprofessional", then that's a red flag and I'll be more careful before hiring someone like that. Now that I think about it, there's a bit of irony that if someone performs poorly on these kinds of questions, I'm likely to spend more time investigating that and asking more questions along those lines.
Ask your doctor to do a free diagnosis so you know he's for real. See where he instructs you to go.
Did you mean to reply to someone else? I appreciate the reply, but I’m confused about what you're responding to and what point you’re trying to make. I haven’t suggested or advocated anything about doing work for free.
That said, this example doesn’t convince me of much. Doctors have to go though a full year (or more) of underpaid work (called residency), and worse, extremely long and on-call hours. They think programmers are pretty spoiled, at least the ones I know personally do. When I've complained to them about pulling 80 hour weeks in films & games (sometimes without overtime pay), I've gotten words back like "lazy" and "whiny". :P
Doctors also have to do unpaid diagnoses when they interview for jobs, in order to prove they’re real. And they have to do unpaid diagnoses for their family and friends, just like we have to do IT for family & friends.
If you want the job, you do need to demonstrate you’re capable of performing the job, no? What alternative are you suggesting?
Patients can't tell whether a doctor is competent or not. That's part of why they need verifiable credentials.
I had to write a JavaScript program on graph paper (with no computer on hand) as a college test. Then the professor would type them out afterwards with you and try to run it. Whenever I hear about this whiteboarding issue I think of that. It was actually kind of fun in a weird way though. I drew nice curly braces.
> I drew nice curly braces.
Heh, I code pretty often on paper when I'm working through a design and my style has evolved to just using < and > as proxies for curly braces which saves me time, since trying to draw a nice curly brace takes me so long.
I do not mind whiteboarding at all. I have three whiteboards at home for brainstorming. I draw the line at doing take home tests.
I cannot interview a company if I am doing a take home assignment. With whiteboarding, I get to also assess my potential co-workers abilities by seeing if they are asking interesting questions.
This is great. I suspect the "coding exercises" in interviews are really dominance hierarchy games. It goes against politeness theory and is a threat to an individual's 'negative face' or need for autonomy.
I interviewed once at a place where a friend worked as an engineer. She assured me that they didn't do whiteboard questions. Guess what I had to contend with during two different interviews...
At my work one part of the interview is a System Design test, where the candidate is presented with a problem and we ask them to come up with a system (i.e. by gathering requirements etc)
Usually this means drawing stuff on the whiteboard with boxes and arrows. Not looking for the perfect system or 'correct' solution, just really having a look at what the candidate is experienced in and how they think.
Whiteboards are fine for that use case, and can be very expressive.
Doing binary search trees and stuff though doesn't seem as useful really.
This is a good idea, although the focus on 'anti-whiteboard' is a bit limiting. It's a shame it's its own site instead of a feature on other sites.
There's no interview process that works best for every candidate; some prefer whiteboards or homework or practical debugging on their on computer or hand-wavey design or whatever. Candidates (and companies!) could save a lot of time by letting applicants filter their searches to companies using methods that will let them put their best foot forward.
I'd much rather code on a whiteboard than do a take-home assignment. I may be in the minority but I rather suspect I'm not the only one.
I can understand that some people find whiteboard coding really stressful. I never have, and I have found take-home assignments to be a huge time-sink which typically results in minimal feedback (and sometimes none at all).
Maybe the right thing to do is to just give interviewees a choice.
Some companies effectivdely use whiteboarding, others don't.
This is a true snapshot of the absurd sense of entitlement among some tech workers these days. When the bubble eventually pops, people will look back at this website promotion with awe.
It's like challenging the generally accepted standard that interviewees dress up a bit more formally than a typical day at the office (i.e. button down versus old T-shirt).
Using a whiteboard to give a talk about your project is desirable in my opinion , you might need to be more specific with what is whiteboard free.
I was “surprise whiteboarded” recently. It was my first whiteboard interview and I actually thought it was an interesting experience.
So is this supposed to be a good or a bad thing? I've used whiteboards in a lot of interviews to great effect. I wouldn't confuse whiteboards with riddles. I also hate riddles and puzzles because they check for the riddle-solving skill, not the actual skill which will be used at work. Whiteboards are a completely orthogonal topic to riddles.
I really like the idea of this, but how many jobs posted here are for senior or high experience positions? The whiteboard filter can still happen at any of these companies in junior developer ranks.
This just creates elitism where the "senior" developers get nice hiring processes, and the junior developers get stuck with traditional torture.
I think its still valid if someone doesn't have a good-size body of work, however the more CS heavy it is and unrelated to the actual work the more you're optimizing for "did this person recently graduate with a CS degree".
So all of these ask for a 2-week home project?
How about "night-mode.com": developer jobs at old-school companies that interview with a blackboard and chalk.
What i want the most is being able to put these companies to test, i have 0 problems with whiteboard problems but like evaluating them, are they all compotent or is it the company a one trick pony with only a few decent engineers.
It would be very useful to be able to filter just the remote positions. Searching for "remote" as a location returns no results, and I can see in the list that some remote positions are being offered.
Free alternative: https://github.com/poteto/hiring-without-whiteboards
Cool. A website full of companies to avoid. Thanks for posting!
A lot of the important tech decision take place around a whiteboard where people communicate their ideas.
Whiteboard questions are okay. Only giving tasks is not how a real workplace work
The Terms of Service page is blank (https://whiteboardfree.com/terms)
I think whiteboards are much better than the remote code-as-we-watch stuff, which is even more common and not a good way to measure actual work skills.
Lyft doesn't do whiteboard when hiring engineers?
That's the first thing I noticed about the list. I interviewed with Lyft last year and they did whiteboard style interviews. Since when did this change?
It didnt. I'd guess they are just pulling in a ton of jobs so they look popular enough to justify the fee to post a job there
Am I the only one who thought this post title meant jobs at companies that simply don't use whiteboards in the day-to-day aspect of working?
I've been coding for 25 years and I wrote some complicated piece of software, but I would be incapable to write code on a whiteboard.
Best job board marketing I've seen in a while.
Call me racist, but I also don't like blackboards.
too many programmers cant program and hurt their fee-fees so a company was made to accept fucking 300$ per ad. Anyone who cant whiteboard or pseudo is fucking retarded.
Whiteboard rhymes with waterboard.
ish
Makes me wonder where he's from. Maybe with a South African accent or similar. They sound very different with my Ohio accent though.
Whiteboard problems work great as a general intelligence test if administered correctly. A well designed whiteboard problem will filter out people of lesser intelligence. This means the people you would be working with would generally be more intelligent. This is not a bad thing.
Furthermore, whiteboards are great communication tools. I spend some time on VRChat's whiteboard room teaching people various algorithms, and other abstract things to help me learn. Someone who knows how to use a whiteboard and communicate correctly would be a better hire than someone who doesn't.
If the company you work at uses whiteboard problems to vet interviewees, just drill them. With a little practice, you'll have a better chance.
This is so wrong that it actually hurts my brain. Albert Einstein, a certifiable genius probably could not tell you how to invert a binary tree, this has nothing to do with intelligence and everything to do with how many HackerRank problems you have memorized.
What's so hard about inverting a binary tree anyway? If Einstein lived today and was into computer science, he'd be able do this in his sleep.
True, but there are also brain-teaser type problems that don't require memorization of obscure algorithms, and those could actually be disguised intelligence tests. (But I suppose if you really wanted to you could memorize most of the brain-teaser problems in existence.)
Google has already shown that those types of questions have 0 correlation to developer success.
There are many things you can do with a whiteboard other than inverting binary trees. You're thinking too narrowly.
Obviously, but I was specifically referring to the typical algorithm oriented whiteboard questions found at major tech company interviews. Working a real world problem on a whiteboard I see no issue with.
This whole post is about companies that "don't whiteboard". Not companies that "only whiteboard for real world problems".