Wednesday, October 22, 2008

Interviewing for fun and profit

I've been interviewing a lot lately - starting the Windows Home Server project, helping build Servers and Tools division in China, serving on Seattle hiring committee at Google, and now building my new team. All this meant spending anywhere from 2-3 hours a week at Google to 7-8 hours a day in China on recruiting activities.

From Seattle to Lahore, from Google office in Fremont to Microsoft campus in Shanghai, the candidates that end up getting offers look very similar. This post is about how to be that kind of a candidate.

Getting a job at a good software development company is like being a genius - it is 1% inspiration, and 99% perspiration. This means a lot of work, vast majority of which must happen before the event. The good news is that this work is portable - once done, it opens a lot of doors, and puts you in a position where you choose where you want to work, not the other way around.

So without much ado, let's delve into the gory details.

Prework - long term

All interviewers look for basically the same thing - candidates that are "smart and get things done". There is no good way to determine one's intelligence, so we use an approximation. We ask the interviewees to solve complex algorithmic problems on the whiteboard. Luckily, unlike intelligence, this ability can be trained.

Unfortunately, most software development jobs, with very rare exceptions, do not provide on-the-job training for algorithmic skills. A colleague of mine says that there are two types of developers - people who can call APIs, and people who can build APIs. Vast majority of the jobs in this industry are about calling APIs. The most highly paid jobs, however, are about building them. Most teams at Microsoft, Google, and the like are looking for people of the second kind.

If you are not already working in the team that builds APIs, you need to learn to love computer science as science. Get interested in algorithms design. Learn about computer architecture. Start paying attention to how the functions you're calling are implemented.

Do you know what's actually inside the HashMap class you've been using all the time? What algorithm does STL use for sorting? Why? When is heap sort faster than the merge sort? What does a modern computer really do when reading a byte out of memory?

If you live near a good university, take a class every few years to keep connection with the scientific side of the software development.

In an earlier article ( I claimed that reading and doing the exercises in three books will get you a job in any good company. I still believe this to be the case - but you do need to actually read these books, and do the exercises. Just buying and putting them on the shelf will not work!

The "getting things done" part during the interview is usually proxied by having the candidate to write a lot of code on the white board. The better the code looks - syntactically correct, concise, professional - the more likely that the candidate has written a lot of code in his or her career, and therefore, got more things done.

One can see the difference when doing something as simple as copying a string:

while(*d++ = *s++) // Good!
 // Generates the same binary code,
// but too much text!
int i;
for(i = 0; s[i] != '\0'; ++i)
d[i] = s[i];
d[i] = s[i];
(This is where STL with its "string s1 = s2;" really gets you :-).)

The only way to good code is to practice writing - a lot! Jump into the parts of your project that generates as much code as possible. If you're a developer, and you're not writing at least 20Kloc/year, your coding muscle does not get enough exercise. Get a coding project at home - write a piece of automation software for your light switches!

Prework - short term

The above suggestions are designed to get and keep you in shape as a software developer. They maintain the set of skills (AKA coding) that makes you employable. However, even great software engineers often do not get the job they want because there are other things that are required for a match.

I've been asking every person who I interviewed for various management positions at Microsoft what they are looking for when interviewing people. Every time the passion for technology was coming out very close to the top. What this means is that ideal candidate (1) wants to work in this general area, and (2) wants this job.

As an interviewee, you must have have plausible answers for both.

The best (and the only) proof that you want to work in this general area is to demonstrate that you invested time to research deeply the technologies/customers/competitive landscape/standards/other subcurrents that shape this part of the industry. Buzzwords are not enough - deep knowledge to an extent where one knows enough to form an educated opinion is required. I've seen many a qualified candidate being rejected because they gave standard, cookie-cutter answers to questions which an interviewer had considered too shallow.

SOAP is a great technology? Why? Because it uses XML? And why is that a plus?

The best (and the only) proof that you want this job is that you've spent time to learn about it. A lot of information is available publically. For example, if you're interviwing for a position of a program manager at Microsoft, it helps to know what this job really is.

Whenever I go on college recruiting trips for Microsoft, the most amusing thing is interviewing candidates for program management positions. A program manager's job does not actually involve management, nor it is any more senior compared to test and dev disciplines. Both of these aspects are described in detail at both the Microsoft recruiting web sites, as well as on the application for the college interviews. Yet 90% of the college candidates that I interviewed on campus assumed that (1) they will be managing people, and (2) right out of school they were entitled to do this.

If it's an internal transfer within the same company, more often than not most of the information about the project is available. I once had a case where a candidate knew more about the history of the project, the market, and the competition - without ever having worked on it - than I did. Needless to say, she got the job :-).

More often than not, recruiters and interviewers will go out of the way telling you directly what will be on the interview. An email that I've got from Google when I interviewed there listed all the areas where the interviews focused. Microsoft recruiters give out no less details. I am telling every candidate, before meeting them, about the areas where the interview will go. You have no idea how many people completely ignore the advice!

If, for example, it's written on the web site, in the recruiter email, and in the interview packet that THE INTERVIEW WILL INVOLVE WRITING CODE ON THE WHITE BOARD, and when you show up at the company and are asked to write code on the white board you say that your coding is "rusty", what outcome do you expect? It's a double-whammy: not only do you not match an explicit qualification for the position, you have wasted company's time and money by coming to the interview, because you apparently have not even bothered to read the description of the job!

Do prepare good, deep questions about things that you should want to know, but cannot. Avoid cliches - believe me, "Describe your typical day as a developer at X" does you no good. This question has been asked many times. Too many. The only reaction the interviewer has to a question like this - "Oh, here we go again..."

The day of the interview

Don't be late. It's unprofessional. Plus it cuts into the time that is available for you to solve the problem :-).

Don't use pseudocode, unless the problem requires it (many graph algorithms do). If you're a great coder, you should be writing in a real computer language with ease. Plus we warned you that (1) we're looking for great coders, and (2) you will be coding on a white board.

Avoid syntax errors when coding. Well-written, well-formatted code is classy and shows experience.

Do be VERY detail-oriented. Check and recheck what you're writing. Do not shrug off the "off-by-one" errors. If you're "always messing up some mundane detail" ((c) Office Space), you should not be in this trade. Messing up mundane details in the past had caused spaceships to miss their destination, to say nothing about expensive product recalls.

Do listen to what the interviewer says. This cannot be overemphasized! They are here to give you cues, to help you. Most of the time they genuinely want you to succeed. They know their problems. If an interviewer makes a suggestion, TAKE IT. (S)he knows!

Do test your code. After the code has been written, think about small, but exhaustive set of test inputs that would exercise all the code paths in your function, and elicit all the differences in behavior. For example if you're writing a binary search, test it on inputs with odd and even number of elements. An algorithm that takes an array should work well on inputs of size 0, 1, 2 and 3. Test the inputs that might cause overflows and out of bounds conditions.

Do step through the code as written, not as you think it should work. This is the most frequent error that I have been observing - a person would look at the data input and then reason through it with the algorithm (s)he had in mind - but not the one on the white board. Pretend you're a CPU instead. Write a sample on the board, create a column for every variable, and start iterating - mechanically.

Be humble. If you're saying "I am really good at X", this is a challenge to the interviewer. They will give you harder problems, give you less help, and treat the failure harsher because you behaved as an arrogant braggart. It's much better to underpromise and overdeliver than the other way around.

And last but not least, be enthusiastic. Ask questions. Offer suggestions. Demonstrate that YOU WANT THIS JOB.

Speaking of wanting the job.

It helps to set up multiple interviews when you're thinking about changing the job, and sequence them such that the more desireable jobs go last. Interviewing is a frame of mind, and it helps to put onself into the right state. But on the day you go to the interview - even if you aren't 100% sure this is the job you'll take in the end - on this day you have only one goal - to get this job. WANT IT!

Happy interviewing!

By the way - we are hiring!


BadTux said...

And now that you've heard about how it's done at Microsoft/Google, here's another data point from the bleeding-edge startup perspective:

* Do not lie on your resume. Do not put that you were a "Senior Software Engineer" (even if it said that on your business card) if you were not, in fact, a team lead with significant design and implementation decisions under your belt. We will find out when we interview, and then we'll kick your rear to the street even if we have an individual contributor position suited for your skillset.

* I look for flexibility on your resume. If every job on your resume is exactly like every job, you aren't the person for us, because we're doing things that have never been done before and we need someone who can wrap his mind around different things. So when making career decisions try to use your current position to leverage to something that's got enough similarity that your current experience will work, but that is different enough that you can show you're not just a Java geek or a COBOL nerd.

* I mostly interview higher-level candidates for lead positions, people who generally have at least five and probably ten years of experience under their belt, so I don't bother asking you to write code in any particular programming language. The reality is that someone who has a varied resume has programmed in a variety of programming languages and can quickly pick up whatever language our project happens to be using. So unlike Sergey, who apparently is hiring for lower-level positions, I haven't asked anybody to write code on the whiteboard for years. I have people for that.

* On the other hand, I very definitely expect you to be able to describe, diagram, or pseudo-code a solution to a problem that I present to you. For example, one candidate this week came in with a resume that was heavy on embedded programming. So I started talking to him about an experience I had programming a PIC processor for the front panel of a network filer, which was responsible for updating the 3x32 LCD display on the front panel from data sent via RS232 from the host and inputting data from four simple keyswitches in order to send them via RS232 to the main host. He didn't know what a PIC was or what company made it. Bzzzt. Wrong answer. Anybody who's truly followed the field of embedded programming knows what a PIC is, the nasty little buggers are in everything from microwave ovens to washing machines nowadays. But not a fatally wrong answer, so I described this particular PIC to him: No RS232 port, just 24 GPIO pins. 9 pins used to strobe data to the LCD. 4 pins used to sample the keyswitches. 2 pins used for TX/RX for the RS232 which runs at 9600 baud. 1mhz. 1 clock per instruction, except branches that are 2 clocks. No timer. No interrupts. 1K RAM, 4K 12-bit word EEPROM program memory. And then I asked him to tell me what the program would look like that could handle receiving data from the host while polling the keyswitches and sending data to the host. I wasn't looking for code. I was looking for basic concepts, like whether he knew cycle-counting, whether he knew what oversampling was for receive data, whether he knew about key debouncing, whether he could divide the available oversample time slice into the four different cooperative tasks involved and use a state machine in each to handle the task. He massively failed everywhere no matter how much handholding I gave him. That was when I concluded he'd lied on his resume when he said he was a senior engineer. He didn't even know where to start when given a problem to solve.

* Which brings up another point. Listen to the interviewer as the interviewer describes things he's done. Likely the interviewer is telling you this for a reason. Such as, for example, that these are things that you're going to be hired to do when hired because the interviewer has no desire to do them again. If you look bored while the interviewer is doing this, the interviewer is going to conclude that you're not interested in the position.

* WORK ON YOUR COMMUNICATION SKILLS. Individual contributors may be able to get along with being as communicative as a rock. But unless you want to remain an individual contributor forever, you're going to have to learn how to communicate. This is mostly practice, practice, practice. When assigned a task, practice documenting how you intend to solve that task, including the task definition and API, and giving said document to your project lead for feedback. If your project has a Wiki, practice updating it with the current state of the project as it currently exists in the source control system, including the current state of whatever API's it has, an overview of the project's architecture, and so forth. Most software geeks hate to write. But it's a skill you need to have if you're going to move beyond individual contributor, because if you cannot break down a project into easily-described chunks when you have the project right in front of you, there's no way you will be able to do so when given a blank slate and a pocketful of marketing requirements and are told, "give me a product that does X, Y, and Z." Especially for senior engineers, which is what I am interviewing. As Sergey says, API design is what we're looking for. But if you can't communicate that API design to your implementation team, you're no use to us. It may have been that my guy who failed with PIC actually did have skills, but simply lacked communication skills. Either way, he would not be useful for leading a team of kernel engineers, which is what we were looking for.

* We know it's your lunch time, unfortunately you said you needed to be out of here by 1pm so we can't break right now for anything other than a snack, but don't pick out a big bowl of noodles to slurp during your next interview when we take you to the break room to grab a snack! That's just rude. Grab a cookie and a soda.

In general, what we're looking for at bleeding-edge startups are creative, imaginative people who can come up with solutions to things that nobody has ever done before and then communicate those solutions to an implementation team that likely is offshore someplace. You have to understand good architectural principles, maintainability, and ease of use, and have the ability to understand where we're going with the product in the future so you can leave your objects and object hierarchy open-ended enough to grow. We're not looking for language geeks but we are looking for someone who has a broad perspective on the problem set we're hiring you to solve. And as Sergey points out, someone with architectural chops, someone who can design an API. So if you're someone who has worked at a typical large corporation for ten years doing typical individual contributor work under the guidance of someone else, probably we're not interested in you -- *unless* you do something else to make us interested in you. Such as create an interesting Open Source project, participate meaningfully in industry groups discussing fundamental problems of the field we're working in, etc. We can hire all the code geeks we would ever want in India or China for peanuts. But guys with real design chops and a real high-level understanding of fundamental problems of our field are much rarer and we snap them up whenever we can.

And there's the perspective from startup-land. And if you're a Linux kernel geek who loves reading network stack code, knows the difference between git and netdev, modifies network drivers for dinner and has an interest in virtualization and paravirtualization topics especially when dealing with network drivers, do a whois on me and drop me some EMAIL :-).

Sergey Solyanik said...

Just to clarify writing the code part...

It is an official position of both Google and Microsoft that developers must be able to write code, irrespective of the level.

Therefore, the coding questions are asked of every individual contributor (and most dev management positions at Microsoft, up to the director of development).

I remember in Seattle hiring committee at Google an eng director tried to push through a candidate that was very senior, but did not do so well on the coding questions - there was nothing short of rebellion among the rest of the committee members, and this incident was talked over and over again afterwards. The candidate did not get hired.

BadTux said...

And from the startup side of things, if you don't have coding in your recent history, we won't even look at your resume because nobody here has the luxury of being a pure manager type. Everybody in our engineering department up to our CTO does at least some coding. One of the ways we get our junior teams started on projects is to whip up a quick scaffold and pass it down to them so they can fill it in. You *will* get some questions about languages you've listed on your resume and if you can't answer them you are out of luck. And you *will* be required to display algorithmic knowledge as well as development process knowledge even if you're being hired for a senior position in the engineering department, even if we don't ask you to actually write code at the interview because your particular language set doesn't have direct relevance to what we're doing. The guy earlier this week who didn't know PIC programming? I didn't expect him to program PIC on my whiteboard. Assembly languages are a dime a dozen and easy to pick up for someone who has done assembly language programming for any similar processor. He supposedly knew 8035, I don't know 8035, so it wasn't any use asking him to do 8035 for me. I *did* expect him to at least be able to pseudo-code what that PIC program must do in order to operate, given the constraints that I gave him. Algorithmic skill is an absolute must.

That said, the most important skill is what Sergey called "get'r'done". I will ask more questions to find out your approach to solving practical problems and about your general knowledge of the industry and the area you're being hired to handle than specific programming questions. I want to know if, when I present a general problem to you, you can ask me the questions necessary to define the problem, then formulate a solution. In startup land we don't have the luxury of sitting on our rears waiting for someone else to tell us what to do, we have to be proactive, because the clock is ticking, the burn rate is burning, and if we don't get product out the door in time and with specifications compelling enough to shout "buy me!", we're going belly up. We can hire programmers in China for ridiculously cheap who can follow directions and churn out code. People who can actually create those directions for those programmers, who have a wide enough and varied enough background to understand the big picture of how you get a product out the door but have enough low-level coding skills to know how to get those guys in China or India coding away... ah yes, you guys are the ones we slobber over.

Anonymous said...

while(*d++ = *s++) // Good! ;

I always wondered why Windows has a new stack overflow bug every day. Now I know. You're selecting for incompetence.