Google's “Director of Engineering” Hiring Test

It's a bit strange to have someone non-technical interviewing a techie. You end up with stupid discussions like the one about Quicksort. If you point out qs is one of several things with the same big-O, you'll probably also get it "wrong". But the real problem is that a guy who is just reading off a sheet can't give any form of nuanced feedback. Was the guy blagging the sort algo question? Did he know if in detail? Does he know what the current state of research on that area is? There's no way to know that if your guy is just a recruiter, but I'm sure even a relatively junior coder would be able to tell if someone was just doing technical word salad.

I wonder what would happen if ordinary people recruited for medical doctor jobs? Would you be comfortable rejecting a guy who'd been in medical school for 10 years based on his not knowing what the "funny bone" is? Wouldn't you tell your boss that you felt a bit out of that league? It's amazing you can get someone to do this without them going red in the face.

One example of how this can be handy is when doing low-level NIC bring up, I've read PCIe TLPs from a logic analyzer to correlate them to network traffic.

There is absolutely no purpose to knowing off the top of your head how long an ethernet address is, or even what system call will retrieve an inode (his bickering over stat() "filling in" rather than "returning" was bogus, for what it's worth). The top Google search result for each of these questions has the answer. Knowing these things isn't part of being a practicing programmer; knowing how to find out is.

Applicant Doctor: "They're double membrane-bound organelles found in all eukaryotic organisms, commonly between 0.75 and 3μm in diameter, that generate most of the cell's supply of adenosine triphosphate"

Recruiter: "No. They are the powerhouse of the cell".

This isn't even my domain, but I remember some of this stuff. I had to dig in to this area years ago, and was knee-deep in that level for several months debugging and configuring stuff. Some of it stays with you, even if you're not there any more.

For someone applying for a director of engineering, I'm kind of split as to whether this should be required "off the cuff" knowledge. Would certainly help, but seems it would depend on the culture of the company - how hands-on they expect director-level folks to be - some companies seem to want that, some don't.

In interviews, I've had people give answers I'm not familiar with. The easy thing to do is to ask them to explain how their answer works. You get to vet whether it seems legit, and as a bonus see how good they are at communication in a genuine teaching moment.

Those functions return a error code, you pass in a stat structure and the function populates that structure.

He was saying (correctly), that they don't return (in the classic C sense) the inode. They return an error code.

To me that is a big difference...

int lstat(const char path, struct stat buf);

vs stat* lstat(const char *path);

2 completely different functions.

What's completely wrong with the situation is that the recruiter was saying "wrong" when he was giving detailed answers. The right solution is to have a non-skilled recruiter take careful notes, asking the candidate to repeat if necessary, in the cases where the candidate insists they know the answer.

Then only in cases where the candidate can't answer a question should it be marked as "wrong." In other cases the answers should be run past someone knowledgeable.

I've gone through a phone interview with Google myself, and it was nothing like this: I spoke with a real, skilled engineer, and there was nothing like a "gotcha" question where I had to guess the exact term he meant. Well, except where he asked "I bet you know what my next question is" and I didn't guess "How can the algorithm be faster?" But he didn't count that against me. :)

Btw, that is the only thing I really disagreed with OP on, the rest seemed just ridiculous.

"Well, it depends what color model you are using, which differentiates between additive and subtractive color mixing, the medium used (print, screen, etc.)...

No. It's Red, Green, and Blue.

Not surprised by the questions. Most are [somewhat] common and not particular challenging.

I am surprised by the stupid counter-answers from the recruiter thought. This guy should not be giving phone interview.

But, obvs, the answer is that you Bing it.

The medical recruiter asks to see the diploma. The doctor shows his degree and gets the job.

Are you talking about the recruiter?

https://en.wikipedia.org/wiki/Comparison_of_MD_and_DO_in_the...

lstat has the same signature as stat: it passes the stat structure back through an output pointer.

Uh, my main languages are PHP and Python (high level stuff) and I'm a student (not someone with 10 years of experience) but I knew that. 3 bytes for the vendor block, 3 bytes for the device.

> Or what the actual SYN/ACK etc tcp flags are?

Yeah the actual bytes, who ever uses that? A MAC address I've seen plenty of times in my life as hex, and I've seen the TCP setup flags being exchanged plenty of times in Wireshark and looked up the hex once when I was implementing TCP from scratch, but I still wouldn't expect anyone to know that.

> You just need to know what they're used for

Agreed on that. I wouldn't blame anyone for not knowing the size of a MAC address, I just didn't think that one is that obscure.

I think you're onto something greater. Maybe Google should hire more smart people who were developing in the 1990's as they're a nice, middle ground between ambitious, young folks and the been-there-done-that greybeards. Them realizing such people already know all the shit they're trying to teach their developers might be way to sneak older people into these tech firms. Haha.

We will stop here because it's obvious that you don't have the necessary skills to write or review network applications. You should learn the Linux function calls, how the TCP/IP stack works, and what big-O means to eventually qualify if you are interviewed at a later time.

Come on Hacker News, I know you hate recruiters, but do you really think that happened?

I was more impressed by how simply he explained many of them in real-world terms. Especially countering Big O. That plus what he's already built indicates he's exactly the kind of person to direct the sort of projects they're building. That they filtered him with this garbage Q&A speaks volumes about Google's ineffective hiring. Plus, indicates what kind of people might have made it into the organization.

/colorblind

She was more artsy, I was insistent that the primary colors were RGB, she was insistent that they were RYB. We googled. We were both right in some senses.

Why he's being interviewed for that position is a different question entirely, and I can imagine Google being totally right or totally wrong.

It's a technical interview, the question should have been technically correct. "What function passes by-reference copies of inodes?"

If you're interviewing for the rare 1% case where the job will actually require you to tinker with a TCP stack, then by all means ask about those details. For 99% of programming jobs today, it's irrelevant trivia used as nothing more than an ego measuring device.

That said I can't recall any of that stuff mattering in the stuff I do for work for quite some time and most of it is a google search away.

That interview was just weird, it was like asking the boiler pressure for a steam train to someone who was a master engineer working on electric trains.

That is why you know the answer. Come back in 10 years and let us know if you still know it. What you think might be mainstream in a computer science class are rarely used in application. And if they are they can be easily looked up.

I used to know the exact effective distance of a CATV cable when I was a student. Useful? Sure. Something I need to remember for the rest of my life? Definitely not.

As another example, in 17+ years writing ISO level 7 programs, I have never once needed to use the Mac address.

2) We are getting description from him. When I interviewed at Google, they asked me to prove that P is equal to NP, and I did, but they said that it took me longer than allotted 45 minutes and I didn't get an offer.

I've been looking at them on NICs, adding them to DHCP databases and using them for simple authorizations in iptables (and prior equivalents) since, oh, 1992 or thereabouts. They weren't new then.

It would take me a moment to count them in my head, that's all. "aa, bb, cc, dd, ee, ff -- that's six bytes".

Thanks for the refresher! I knew that too when I was in college. Good luck remembering that 5 years from now :)

In a recruiting situation, if the recruiter is going all "right and wrong" on the interviewee, they should know these dirty details.

I also disagree on the ethernet address length. You know how long IPv4 addresses are, you know how long IPv6 addresses are, why would it be so extraordinary to also know how long MAC adresses are?

I think it could be useful to find out in an interview whether the applicant knows stuff because they have actually implemented low-level code and gained an intricate understanding, or whether they just did used some high level APIs and were never interested in more details than "I have a handle right here, it does all I need". For some positions that would be an important distinction.

Personally, if I were interviewing people, I'd hire the guy that explains something to me that I did not know, but that I find interesting and would have attempted to understand, too.

While I agree, I have seen interviewers where if he answered "fstat" they would have come back with "Wrong! fstat passes the structure back by reference, it does not return it!"

With this style of interview I can't blame him for thinking it might be a trick question and then trying to explain why he didn't say those other functions as an answer.

I did not see any argument here from the statement in the article. The recruiter clearly had little clue about what is right and wrong. And the way the recruiter assess the answer by throwing right/wrong seems more rude to me compared to the author "wanting to be right".

Please do not speculating based on something that is not present in the article.

I had done similar interviews before, the recruiters I worked with did not show the same level incompetence as this one. When I want to be more specific on details, they would suggest that they think it's enough and move on. Not like this recruiter who just throw a 'wrong'.

The rejoinder, of course, is that it's probably misguided to structure your recruiting around a spam filter.

I've never taken a CS class, and I still know it. I've just stared at enough packet dumps and debugged enough issues that it happens to stick.

I think it's a stupid question to ask as a screener, as it tests familiarity with trivia. It's the kind of thing I'd be happy to see that somebody knew, but it's never something I'd downcheck them for not knowing. It could just mean, as in your case, that there deep knowledge is somewhere else.

Doctors know a ton of random crap tangentially related to their jobs that they were forced to memorize.

The pain of being tossed out onto the pavement would totally be worth it.

Take, for example, the sorting question. "Why is QuickSort the best sorting algorithm?" The answer being looked for was, "It has the best Big O."

And this is wrong. Its average case is O(n log(n)). Its worst case is O(n^2). Which do you call its big-O? Moving on, the average case of O(n log(n)) is matched by a wide variety of sorting algorithms. How do you choose one?

Here is a better answer.

QuickSort is a very simple to implement algorithm which achieves the lowest average number of operations on a randomly sorted list. Which is why it is so widely adopted despite sometimes being very slow.

However Timsort appears to be the fastest general purpose sorting algorithm for the mix of random and partially sorted lists seen in practice.

When I tend to notice that sorting is slow, generally that's a larger workload where some type of merge sort would be appropriate.

I feel like I've seen 2-3 articles alone in the last month that has rediscovered head-of-line blocking and UDP. I'd say 1/10 engineers I talk to even know what cache aware datastructures are.

The thing that separates someone who can just string together whatever they find on npm to people who build real systems is this deep understanding. You're not going to need this for your standard LoB apps. However if you're in the business of building software you're definitely going to want people like this.

We are all a product of our environment. And there is certainly a chance in 10 years the OP will still know how many bytes are in a MAC address. I only mentioned computer science based on the "student" part of the quote.

Personally I've been spoiled by working on higher level stuff so if I do look at a packet dump I usually filter the packet headers out. But again, we're a product of our environment and asking questions that a reasonable skilled engineer might go their entire career without knowing the answer to except in that interview is a bit suspect and will likely disqualify people that would be otherwise great for the job.

"Look I get the gist of what this screening is trying to do but please write down my answers and feed them to the technical staff who authored this Q&A and have them verify their misunderstandings and have them get back to me please. I am very confident you all have subtle details mixed up and could benefit by a strong candidate who understands such details with a higher level of fidelity."

Guess its another loss given to the recruitment focus on avoiding false positives over false negatives.

Depends on what you actually do in your job, I suppose. If you use that information regularly or troubleshoot networks regularly you would soon enough memorize it.

> and if you need the specifics, you'll find out with a single search.

Someone doing network programming _does_ need the specifics.

Linux:

    int stat(const char *path, struct stat *buf);

    stat() stats the file pointed to by path and fills in buf.

    int stat(const char *restrict path, struct stat *restrict buf);

    The stat() function obtains information about the file pointed to by path.  
    Read, write or execute permission of the named file is not
    required, but all directories listed in the path name leading to the file must be searchable.

When I was an interviewer at Google it felt like 90%+ of interviews were with candidates who had less than four years of experience. Probably half were fresh out of college. After the fifth candidate in a row who can't do simple recursion or algorithmic analysis (and I mean simple) you get pretty discouraged. In one phone interview I got to interview an experienced engineer with over twenty years of experience in C. He completed the question I usually have to spend 45 minutes on with a new-grad in <10 minutes. It was probably my favorite interview of all time because I actually got to discuss the subtleties and he reaffirmed that I could maintain high standards.

For instance, perhaps the interviewer asked "What makes quicksort a good sorting method," instead of "What makes quicksort the best sorting method"—a very small difference in phrasing. In that case, the answer of "It's not always the best, or even suitable" is still technically true, but much more wrong. (And an answer like the one you started with, "Its average case is O(n log (n)), its worst case is O(n^2)," would have been enough to pass... but sitting on the phone and arguing about storage topology is itself a failure.)

As I mentioned in another comment https://news.ycombinator.com/item?id=12702130 , my (five-year-old, faulty) memory of Google's SRE phone interview is that they asked another question here with a very small but important phrasing difference: "What is the signal sent by the kill command" instead of "What is the kill signal". If you make that change, the interviewee's answer of "SIGKILL" becomes wrong, and the interviewer is right to insist on SIGTERM (which would otherwise make no sense). It is a quite literal game of telephone.

(Again, I can also imagine Google being totally wrong and the interviewer mangling the questions.)

Trying to find the TCP stack engineer to build your app is like hiring a petrochemical engineer to do oil changes. If your pockets are as deep as Google's, yes, you can do that, but it's by no means necessary.

I'm fine working with non-technical people (or who don't know any given field), but I wouldn't be fine working with those people if they were insistent that they did know about these things about which they actually had no clue, or if they were in a position where they really need to know this stuff.

I have coworkers who don't know how to use the command line, but they aren't engineers and they don't try to tell me what commands to run when I pull up a console, so it's fine. If they keep insisting that I should use "dir" and that "ls" is wrong, that would be a problem. If they were the CTO, that would be a problem.

When we hire engineers for customer support, the non-technical operations guy interviews them, but he always has at least one engineer do a portion of the interview because he knows he's not fully qualified to judge someone's technical chops.

The OP isn't being condescending just because someone didn't know stuff. It's because someone didn't know stuff, but because they acted like they did. You want to be insufferable, insist you know better than the experts in a given field.

This interview must be some kind of a mistake. It's like testing Richard Feynman's knowledge with questions about multiplication table (where you have wrong answers on your table).

It's so stupid that it's almost not possible to laugh.

"lsof just prints open files... you would use netstat, not lsof."

I tried correcting him, but he wouldn't listen and ended the call soon after.

I never received a call back.

I personally wouldn't want to work with someone who wasn't able to understand what I mean (in such an obvious case, at least) and wasn't able to answer to that meaning.

(Of course, I'd also personally prefer someone who would point out that this is inaccurate, but do so in a charming and off-hand way, to make me feel comfortable. A high bar, maybe, but for a director that's definitely a bar they should clear).

Maybe 10 years from now I'll have forgotten everything about the hex/binary/octal representations of numbers, but I certainly hope not!

Uh actually my classes didn't teach me that. Side projects did.

And that, ladies and gentlemen, is how we ended up with npm.

Whatever this thing... this... stringing together a 100 random github projects to get a webpage that kinda of works about 80% of the time is called -- please let's agree not to call it "software engineering".

   Q: What does lstat return?
   A: A struct
   >> Wrong, it returns an error code

Which is really bad, when they're trying to hire people for a job where thinking is the fundamental skill...

As to high standards, you are testing for things that have very little to do with someone being good at the job. High arbitrary standards often remove the most talented people who generally don't have the same background as you.

EX: Suppose you where looking for a CEO, well having a collage degree seems like a reasonable requirement. However, a surprising number the best CEOs don't.

Maybe I'm a bad programmer, but I don't feel like I would be a better one with that specific fact committed to memory. I dunno.

The fact that several Google people posting here literally can't believe this is true shows how fucked up their process is.

(Assuming they get no false positives, which, thinking about it, is a big assumption)

I know this is irrelevant to the larger point of your post, and I'm sure that you know this already, but the worst case is the big-O. This is just another reason why "big-O" is not the most helpful thing to discuss in practice.

It reminds me of a classic Dilbert. https://goo.gl/images/7DhC9f

    These functions return information about a file.

'a'

No. The right answer is 'A'. We will stop here because it's obvious that you don't have the necessary skills.

For question 5, you asked about an inode, and were told about an inumber, and got back an answer insisting that the inode was an index.

For question 6, maybe change "inode" to "information in the inode". The interviewee still has not figured out the distinction between an inode and an inumber.

For questions 7 and 8, apply the changes I suggested.

At what point do you decide that the interviewee is hopelessly arrogant and not worth your further time? And how do you get them off the phone gracefully?

Maybe around question 10, when they're quoting bits to show off and not saying the words that would actually let them communicate with other engineers like "SYN" and "ACK"?

No ill will is required on the interviewee's side, unless you consider refusing to waste time on bad candidates "ill will".

I did not learn this because some course demanded I learn a text book by heart. I'm not doing that kind of theoretical university. I knew that particular thing because I look at a lot of packet captures.

Big-O is a way of categorizing the growth of mathematical functions. Those functions can represent anything. It is wrong to talk about the big-O of an algorithm without specifying what you are measuring. Be it average operations, worst case operations, average memory, worst case memory, amortized average time, amortized average memory, and so on.

It happens to be that when we talk informally, we're usually talking about the worst case we are willing to think about. Quicksort's worst case is a sorted set, so we think of that as O(n^2). But then we turn around and cheerfully call a hash lookup O(1) because its O(n) worst case is incredibly rare in practice.

The conversation that I'd have to reconstruct has a very combative interviewee. Which would also fit said interviewee deciding to write up the article that I read. Which would mean that Google dodged a bullet.

I find it interesting to note that his site is down. There are a lot of possible causes, but it isn't good advertising for his webserver software.

The point still stands though. First, it is much easier to have side projects when you are in school so the causal relationship might still be there just one step removed. But the more important point I was making was that school or a side project or even a project at work, if people go 10 years without using it, they forget things.

As an aside, you may want to drop the "Uh" in-front of your sentences. It conveys a certain tone that I'm not sure you are doing intentionally.

I do plenty of interviews, and so I do have some sympathy for the idea that some overwhelming majority of applicants simply cannot perform even the most basic coding tasks but are somehow trying to sneak in anyway, but at the same time I can't escape the suspicion that a lot of the stories are actually from badly-designed or badly-calibrated interviews gone off the rails.

I absolutely believe that the article we're discussing here gives a fair view of a Google phone screen, since I've been through it too and even got asked some of the same questions. The only reason I "passed" was that I recalibrated way, way down to meet the expectations and the technical understanding of the recruiter I was talking to (who did very clearly seem to be reading off a prepared script). This was surprising since Google had reached out to me to ask me to apply, which one would think indicates a confidence in basic technical skills, but if I hadn't caught that and adjusted how I was interpreting and answering the random trivia they threw at me, I likely wouldn't have passed the screen and would have been labeled just another impostor trying to sneak into a job I'm unqualified for.

So when I hear someone else talking about all the "unqualified" applicants they get, I can't help wondering how many really were qualified applicant talking to unqualified interviewer using unqualified interview process.

(disclosure: I don't work for Google, don't ever intend to work for Google, and in fact hung up on a later screening call out of frustration with the way they ran their process, which at least finally got their recruiters to stop spamming me)

IIRC, you can return structs from functions in C. You have to access the values in the returned struct with dot notation, of course, like point.x .

Pre-ANSI C may not have had this, but later, it did. Remember reading it in the 2nd (ANSI) edition of the Kernighan & Ritchie C book, and also used it myself in some programs.

https://www.google.co.in/?q=can+you+return+a+structure+in+c

Edit:

Might want to consider the cost of copying, depending on the perf requirements, size of the struct, whether the function call is in a tight loop, etc.

ERR_CONNECTION_REFUSED

Still, it shows a massive company trying to streamline some things and failing terribly. I personally wouldn't want to work at Google today. What might have been cool once is now nothing but a standard large company like IBM or Amazon. There's a great Quora post by some former Google people that say as much.

OS X: (OK, so I lied about the "fine" part)

    The stat() function obtains information about the file pointed to by path

    Get status information on a file

I haven't brought it up yet because we once had a fight over whether that thing you put outside your shower is called a bath mat or a bath rug (Both are correct in different circumstances)

I may have gotten a technical interviewer because I can list a half dozen people who know me who work at Google, most of whom can directly vouch for me.

Not sure why the OP got an unskilled reviewer, but if they didn't list references inside the company, they may have been thrown in the "random unknown applicant" bucket.

This rubbish flies in our industry since we are not professional engineers.

[1]: https://en.wikipedia.org/wiki/Principles_and_Practice_of_Eng...

My friends and I used to spend bar nights drinking over torturous interview questions (yes, I have always been this nerdy). For instance: we had a gruesome sequence of questions on how to implement the fastest possible traceroute that you could only clear if you knew about a trick using the IP timestamp option.

Later, I got (what I thought was) smarter about interviewing, and moved to more surgical questions. I'd ask candidates to debug a C program that segfaulted in malloc, or ask them to describe the utility functions they carried with them from project to project.

After taking over recruiting for a company that really needed to hire at a specific clip in order to balance sales and delivery, I'm embarrassed that I thought I was interviewing effectively with stuff like this.

You can't learn about someone's capabilities by putting them on the spot with trivia questions.

But... the kill command is the command to send arbitrary signals. It sends them all.

None of that was a reasonable predictor of my future performance at Google. You would have filtered out a perfectly good candidate (and this is, IMHO, the biggest issue is that Google rejects a number of people who would be great employees with its early filters). I can't say I have a better system.

The only interviews that made any sense for the time I got hired were the ones with the specific team members I was going to join. Once we got to chatting it was pretty clear I was a good technical fit for the team.

I still want to emphasize I don't have a solution to the high false negative rate in the pre-screening procecss.

Or to put it in terms you'll understand, clicking around on your comment history a bit, we get a pretty solid impression of what kind of guy you are.

> Nonsense.

s/\./:

  ~ $ lsof -a -n -c chrome -iTCP:443 | head -2 #sanitized output
  COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
  chrome   1234 me   12u  IPv4 12345      0t0  TCP 127.0.0.1:12345->127.0.0.2:https (ESTABLISHED)

It's what happens when you get old. There were commonly C compilers on 8, 16, 20, 24 and 32 bit architectures, some big-endian, some not, and you couldn't google things in 20 seconds back in those bad old days when old farts were building the networks that would be used to connect to us all to google.

(And he has apparently been in networking since)

Like this one: https://www.youtube.com/watch?v=yu44JRTIxSQ

That last sentence is important.

Remember, you have to walk through the interview thru OP's mindset and how he took into consideration the interviewer's analness.

But that's not the point of these questions. These questions are a 5-10min phone pre-screen before getting to the actual interviews. They test if the candidate has experience in a given field, not if they can search for information or what are the precise bounds of their capabilities.

It's trivia, but it's trivia that is a reasonably high confidence proxy for experience in e.g. network programming/design (resp. other fields). If someone claims they have networking experience and don't know SYN/SYN-ACK/ACK it's in my opinion a large red flag.

Thanks for correcting me!

I thought you were right at first but then realized what was going on. This is a pretty subtle point and mostly uninteresting for well-understood algorithms like quicksort. But one slightly less subtle point is that big-theta isn't average case, it is the combination of big-O and big-omega, i.e., bounded from above and below (possibly with different constant factors) by the same asymptotic behavior.

Hmm, let me think. Somebody working with routers for 10+ years.

> ..what the actual SYN/ACK etc tcp flags are?

Same.

I mean, these things are just as familiar to me as grade school multiplication, by now..I bet there's things familiar to you which I couldn't even be bothered to memorise and instead had to turn to a search engine for.

The ostensible reason they get deployed (I say "ostensible" because we all know that in reality the on-site interview consists of the same stupid kinds of questions) is to keep the employer from wasting time conducting more sophisticated interviews for candidates who have no hope of passing. But that's dumb for at least two big reasons:

* The filters obviously reject candidates who would do well on more serious challenges --- worse, they do it insidiously, because you can't tell that they're rejecting good people, only that you're seeing fewer bozos, which makes them look like they work. In reality, a new norm has arisen where the most qualified candidates get to skip these processes entirely, because we all know they're a crap-shoot and don't want to lose good people.

* Properly administered work sample challenges actually take less employer time than these stupid trivia screens do. That means there's literally no purpose to the trivia screens whatsoever; they do nothing but harm.

In my last comment, I just wrote off the top of my head a sketch for a work sample test that addresses the same concern as the dumb TCP/IP trivia quiz from the original post. It took me I think something like 30 seconds to come up with it. Think about how you'd score that (remember: the bar here is "must be more predictive than that dumb trivia question"). I'm thinking something along the lines of "run the code and see if it opens a new TCP connection".

Assuming your team has enough sophistication to build work sample challenges like this, try to justify the trivia interviews. I think you can't.

SRE's are hired to fix outages and other problems asap, knowing trivia is very important then since at that point you might be losing a million dollars per second.