Google's “Director of Engineering” Hiring Test

I knew all these answers too, because I was a developer in the 1990s.

There is absolutely no purpose to knowing off the top of your head how long an ethernet address is, or even what system call will retrieve an inode (his bickering over stat() "filling in" rather than "returning" was bogus, for what it's worth). The top Google search result for each of these questions has the answer. Knowing these things isn't part of being a practicing programmer; knowing how to find out is.

The question was what function "returns" an inode.

Those functions return a error code, you pass in a stat structure and the function populates that structure.

He was saying (correctly), that they don't return (in the classic C sense) the inode. They return an error code.

To me that is a big difference...

int lstat(const char path, struct stat buf);

vs stat* lstat(const char *path);

2 completely different functions.

But what if Google is down and you are tasked with diagnosing it?

This strikes me as an entirely trivial point, the meaning of the question was pretty clear. It wasn't "what is the literal return value". Many APIs will return error codes, and people still talk about them as "returning" certain values colloquially. Of course, if the OP's answer would've been "it returns an error code :) but I assume you're talking about..." I would think that's fine.

Btw, that is the only thing I really disagreed with OP on, the rest seemed just ridiculous.

Ask Jeeves? Being able to find it in a book might be useful, but when Google is down I bet they would like to get things up and running as soon as possible.

I lost it at this one. As if there is some single point of failure that's going to bring all of Google down, and some intrepid director of engineering has to inspect some TCP packets by hand to fix it.

Then you do what people did before Google when they didn't know some trivia: Use a book.

If they are trying to hire for a position that will somehow be on the line for what happens when all of Google is down, I for real hope that they get some higher-power questions into that interview.

But, obvs, the answer is that you Bing it.

Is that a trick question? Open APUE, ``apropos inode``, ...

Yes, but the recruiter should have said "oh, you're technically right (the best futurama), next question".

This is where I threw up my hands too. The Director of Engineering does not need to know the difference between SIGTERM and SIGKILL, or how many bytes are in a MAC address. I guess it's a nice bonus if he does, but he'll spending 10 hours per day in meetings talking about roadmaps, shielding his team from the execs, and removing productivity roadblocks. "Third engineer from the left" is doing the packet inspection--ask HIM about SYN and ACK.

There's a precise distinction, and thinking about it as "returning" anything other than an error code is chummy human thinking rather than the sort of precise knowledge the test was (supposedly) looking for. The question was actually worse than asking "I'm thinking of a number between 1 and 10, what is it?" because rather than applying a random filter to candidates, it punished the candidate for having precise knowledge.

Not agreeing with the recruiters attitude but "if Google is down you will need to know this to diagnose what the problem is." was his comment. In that case the developer should have cheat-sheets printed on their desks.

if Google is down you'll have other problems (SYN, ACK, SYNACK won't help much with the DDoS that's killing it)

In C, what stat does is how you return multiple values from a function.

lstat has the same signature as stat: it passes the stat structure back through an output pointer.

But not if you're a literal lunatic who thinks that because you literally used the word return with an error code that's what it returns. And. Only. That.

I bet Google has their own internal "backup Google", for just such an occasion.

He wouldn't need to know it for the job, but as others have said he probably would know it if he really had the experience on his resume. That's the value for me sometimes in questions like that, especially in an initial phone screen, to decipher if you're bullshitting or exaggerating your credentials.

"I knew all these answers too, because I was a developer in the 1990s."

I think you're onto something greater. Maybe Google should hire more smart people who were developing in the 1990's as they're a nice, middle ground between ambitious, young folks and the been-there-done-that greybeards. Them realizing such people already know all the shit they're trying to teach their developers might be way to sneak older people into these tech firms. Haha.

Reading more closely, it sounds like they are not interviewing him for a director of engineering position; it just sounds like he thinks his current role, CEO-who-writes-code of a very small software company (http://www.gwan.com/about), qualifies him for a director-of-engineering-level position. He's probably being interviewed for an SRE team lead or thereabouts.

Why he's being interviewed for that position is a different question entirely, and I can imagine Google being totally right or totally wrong.

The difference is memory management, completely different. Don't ask me for a function that "Returns" something in a technical interview and when I call you out on it say I'm wrong, when I'm not.

It's a technical interview, the question should have been technically correct. "What function passes by-reference copies of inodes?"

Technically stat neither returns nor fills in the inode, because the actual structure of the inode can vary by filesystem, and the inode will contain more fields than struct stat.

In a recruiting situation, if the recruiter is going all "right and wrong" on the interviewee, they should know these dirty details.

I also disagree on the ethernet address length. You know how long IPv4 addresses are, you know how long IPv6 addresses are, why would it be so extraordinary to also know how long MAC adresses are?

I think it could be useful to find out in an interview whether the applicant knows stuff because they have actually implemented low-level code and gained an intricate understanding, or whether they just did used some high level APIs and were never interested in more details than "I have a handle right here, it does all I need". For some positions that would be an important distinction.

Personally, if I were interviewing people, I'd hire the guy that explains something to me that I did not know, but that I find interesting and would have attempted to understand, too.

> his bickering over stat() "filling in" rather than "returning" was bogus, for what it's worth

While I agree, I have seen interviewers where if he answered "fstat" they would have come back with "Wrong! fstat passes the structure back by reference, it does not return it!"

With this style of interview I can't blame him for thinking it might be a trick question and then trying to explain why he didn't say those other functions as an answer.

I think the problem was that he didn't update his knowledge about what an inode is after the interviewer told him. He still thought it was an integer value, as opposed to a metadata structure. If inode meant inode number, it works be reasonable to assume there was a function that would return it.

I don't think these questions are unreasonable as a spam filter. Yeah, they're trivia, but if you had actually worked in that space, I'd be surprised if you didn't know a lot of that trivia just by virtue of exposure.

The rejoinder, of course, is that it's probably misguided to structure your recruiting around a spam filter.

The length of an ethernet address is a trivia question. It's a good way to score a board game. Filtering out candidates based on it is lunacy.

How can you imagine Google being totally right here? The disconnect between the questions being asked and the interviewer's lack of knowledge made the interview a waste of time no matter WHAT role they are interviewing for.

Take, for example, the sorting question. "Why is QuickSort the best sorting algorithm?" The answer being looked for was, "It has the best Big O."

And this is wrong. Its average case is O(n log(n)). Its worst case is O(n^2). Which do you call its big-O? Moving on, the average case of O(n log(n)) is matched by a wide variety of sorting algorithms. How do you choose one?

Here is a better answer.

QuickSort is a very simple to implement algorithm which achieves the lowest average number of operations on a randomly sorted list. Which is why it is so widely adopted despite sometimes being very slow.

However Timsort appears to be the fastest general purpose sorting algorithm for the mix of random and partially sorted lists seen in practice.

When I tend to notice that sorting is slow, generally that's a larger workload where some type of merge sort would be appropriate.

I agree with the first and second sentence. As to the third: what would be your thought process as to someone who claimed to be a network programmer on the phone but couldn't answer most of those questions?

What's bogus about his comment on stat() exactly. It literally returns an int and fills in the struct that it takes as a parameter. If he had no idea what it did he could check Google or the man pages which are going to give him some variety of the same answer such as the two examples below.

Linux:

    int stat(const char *path, struct stat *buf);

    stat() stats the file pointed to by path and fills in buf.

Mac OS X 10.11:

    int stat(const char *restrict path, struct stat *restrict buf);

    The stat() function obtains information about the file pointed to by path.  
    Read, write or execute permission of the named file is not
    required, but all directories listed in the path name leading to the file must be searchable.

It may be pedantic but stat definitely doesn't "return" an inode. One of the problems with technical interviews like this is you often have no idea what the interviewer is really looking for. Some might just want to know what you would use to get the information about an inode and another might be seeing how you describe it as a test of your knowledge of pointers or something. Often it's impossible to know and that could have been easily a trick question where the right answer is "there isn't one but you can use stat to get this information as it takes a struct pointer and will place the information into that struct". Of course you know that better than most or you wouldn't have made stockfigher.

I can imagine Google being totally right because I can imagine the interviewee not accurately remembering the conversation here. (I expect, for instance, that he did not write down the interview as it was happening.) In fact, conditioned on the assumption that Google is right that this guy's experience is better suited for SRE than director-level, it is pretty likely that he did not understand the questions being asked / thought the questions were beneath him / etc. and therefore wrote them down inaccurately.

For instance, perhaps the interviewer asked "What makes quicksort a good sorting method," instead of "What makes quicksort the best sorting method"—a very small difference in phrasing. In that case, the answer of "It's not always the best, or even suitable" is still technically true, but much more wrong. (And an answer like the one you started with, "Its average case is O(n log (n)), its worst case is O(n^2)," would have been enough to pass... but sitting on the phone and arguing about storage topology is itself a failure.)

As I mentioned in another comment https://news.ycombinator.com/item?id=12702130 , my (five-year-old, faulty) memory of Google's SRE phone interview is that they asked another question here with a very small but important phrasing difference: "What is the signal sent by the kill command" instead of "What is the kill signal". If you make that change, the interviewee's answer of "SIGKILL" becomes wrong, and the interviewer is right to insist on SIGTERM (which would otherwise make no sense). It is a quite literal game of telephone.

(Again, I can also imagine Google being totally wrong and the interviewer mangling the questions.)

My worst offense of this was for a linux admin position when asked from another abmin about how to list connections on a machine. My answer was "lsof -i, since I've found it's easiest to pull granular information."

"lsof just prints open files... you would use netstat, not lsof."

I tried correcting him, but he wouldn't listen and ended the call soon after.

I never received a call back.

Yes, but we are all chummy humans with our chummy human thinking :) We are generally gifted with the ability of shorthand, context, and understanding that surpasses a purely technical understanding of words, and this is exactly the kind of situation where that helps.

I personally wouldn't want to work with someone who wasn't able to understand what I mean (in such an obvious case, at least) and wasn't able to answer to that meaning.

(Of course, I'd also personally prefer someone who would point out that this is inaccurate, but do so in a charming and off-hand way, to make me feel comfortable. A high bar, maybe, but for a director that's definitely a bar they should clear).

To be fair, the rest of the questions were demonstrably pedantic, and I would completely expect a question like this to be on the bill of fare:

   Q: What does lstat return?
   A: A struct
   >> Wrong, it returns an error code

That type of technical specificity about what well-known functions return is absolutely something I've heard people use as an interview question. Knowing it returns an error code (rather than the value you want) seems like a good indicator of "has actually coded in C" (whereas many other languages return the value you want and raise an exception if it had an error).

I'm a network programmer and I forgot exactly how long an Ethernet address is. (I would have guessed 6 based on memory, but I wouldn't be 100% sure.) It's an opaque structure to me; if I need the size I'll sizeof it, which is clearer anyway.

Maybe I'm a bad programmer, but I don't feel like I would be a better one with that specific fact committed to memory. I dunno.

> Its average case is O(n log(n)). Its worst case is O(n^2). Which do you call its big-O?

I know this is irrelevant to the larger point of your post, and I'm sure that you know this already, but the worst case is the big-O. This is just another reason why "big-O" is not the most helpful thing to discuss in practice.

You are right that several of these questions could be due to his misunderstanding the questions asked at the time, answering the wrong one, and then remembering what he thought he was asked. But it is beyond my imagination to reconstruct a plausible conversation that could result in the one recorded without there being considerable ill will on both sides.

Here's the first sentence of the DESCRIPTION section of the stat(2) man page on my Linux systems:

    These functions return information about a file.

Here's a thought experiment: read the article, replacing the interviewer-side questions with ones that make them sound more plausible. (This is the side that we should believe to be less accurate, if only because the interviewer isn't reporting their questions.) Pretend you're the interviewer, and ignore the internal monologue.

For question 5, you asked about an inode, and were told about an inumber, and got back an answer insisting that the inode was an index.

For question 6, maybe change "inode" to "information in the inode". The interviewee still has not figured out the distinction between an inode and an inumber.

For questions 7 and 8, apply the changes I suggested.

At what point do you decide that the interviewee is hopelessly arrogant and not worth your further time? And how do you get them off the phone gracefully?

Maybe around question 10, when they're quoting bits to show off and not saying the words that would actually let them communicate with other engineers like "SYN" and "ACK"?

No ill will is required on the interviewee's side, unless you consider refusing to waste time on bad candidates "ill will".

Technically, no.

Big-O is a way of categorizing the growth of mathematical functions. Those functions can represent anything. It is wrong to talk about the big-O of an algorithm without specifying what you are measuring. Be it average operations, worst case operations, average memory, worst case memory, amortized average time, amortized average memory, and so on.

It happens to be that when we talk informally, we're usually talking about the worst case we are willing to think about. Quicksort's worst case is a sorted set, so we think of that as O(n^2). But then we turn around and cheerfully call a hash lookup O(1) because its O(n) worst case is incredibly rare in practice.

Your imagination is better than mine. :-)

The conversation that I'd have to reconstruct has a very combative interviewee. Which would also fit said interviewee deciding to write up the article that I read. Which would mean that Google dodged a bullet.

I find it interesting to note that his site is down. There are a lot of possible causes, but it isn't good advertising for his webserver software.

And here it is.

I do plenty of interviews, and so I do have some sympathy for the idea that some overwhelming majority of applicants simply cannot perform even the most basic coding tasks but are somehow trying to sneak in anyway, but at the same time I can't escape the suspicion that a lot of the stories are actually from badly-designed or badly-calibrated interviews gone off the rails.

I absolutely believe that the article we're discussing here gives a fair view of a Google phone screen, since I've been through it too and even got asked some of the same questions. The only reason I "passed" was that I recalibrated way, way down to meet the expectations and the technical understanding of the recruiter I was talking to (who did very clearly seem to be reading off a prepared script). This was surprising since Google had reached out to me to ask me to apply, which one would think indicates a confidence in basic technical skills, but if I hadn't caught that and adjusted how I was interpreting and answering the random trivia they threw at me, I likely wouldn't have passed the screen and would have been labeled just another impostor trying to sneak into a job I'm unqualified for.

So when I hear someone else talking about all the "unqualified" applicants they get, I can't help wondering how many really were qualified applicant talking to unqualified interviewer using unqualified interview process.

(disclosure: I don't work for Google, don't ever intend to work for Google, and in fact hung up on a later screening call out of frustration with the way they ran their process, which at least finally got their recruiters to stop spamming me)

Most people don't know that lsof works into two modes- the typical one is to run it against a process ID. The rarer one is no args, which runs it against everything on the system. however, I observe that lsof needs to run as root to print the same information that netstat -tanp returns for a reglar user.

>In C, what stat does is how you return multiple values from a function.

IIRC, you can return structs from functions in C. You have to access the values in the returned struct with dot notation, of course, like point.x .

Pre-ANSI C may not have had this, but later, it did. Remember reading it in the 2nd (ANSI) edition of the Kernighan & Ritchie C book, and also used it myself in some programs.

https://www.google.co.in/?q=can+you+return+a+structure+in+c

Edit:

Might want to consider the cost of copying, depending on the perf requirements, size of the struct, whether the function call is in a tight loop, etc.

This is the funniest thread I've ever seen on HN

Somebody sack the writer ;) - so, OK, I doubt I'd argue the toss if it were an interview situation. But when writing a comment or documentation I think I'd take inspiration from the documentation for either of these two fine systems.

OS X: (OK, so I lied about the "fine" part)

    The stat() function obtains information about the file pointed to by path

Windows, here the VC++ CRT - fantastically poorly described, if you ask me, though of course you shouldn't be using any of this POSIX shit on Windows, so if it confuses anybody enough to make them go and find FindFirstFile then it can only be a good thing:

    Get status information on a file

I think you can probably predict it: I would generate a work-sample test for it. For a network programmer, I might have them implement a 3WH coded directly to pcap_write() (which requires you to populate the Ethernet frame header). Like the best work-sample challenges, doing a raw 3WH is kind of fun if you haven't done it before.

My friends and I used to spend bar nights drinking over torturous interview questions (yes, I have always been this nerdy). For instance: we had a gruesome sequence of questions on how to implement the fastest possible traceroute that you could only clear if you knew about a trick using the IP timestamp option.

Later, I got (what I thought was) smarter about interviewing, and moved to more surgical questions. I'd ask candidates to debug a C program that segfaulted in malloc, or ask them to describe the utility functions they carried with them from project to project.

After taking over recruiting for a company that really needed to hire at a specific clip in order to balance sales and delivery, I'm embarrassed that I thought I was interviewing effectively with stuff like this.

You can't learn about someone's capabilities by putting them on the spot with trivia questions.

> they asked another question here with a very small but important phrasing difference: "What is the signal sent by the kill command" instead of "What is the kill signal". If you make that change, the interviewee's answer of "SIGKILL" becomes wrong, and the interviewer is right to insist on SIGTERM (which would otherwise make no sense).

But... the kill command is the command to send arbitrary signals. It sends them all.

What is a 3WH ?

Probably "3 way handshake", see https://en.wikipedia.org/wiki/Transmission_Control_Protocol#...

I can't disagree with you enough. Engineers should always strive to be as clear as possible about what they're doing. The word "return" has a precise meaning in the C language. If the recruiter was unaware of that, that's their bad entirely.

Yeah, you're right about needing root (edit: for processes not owned by the current user). Though, if you have the access, it's a very, very worthwhile tool to learn:

  ~ $ lsof -a -n -c chrome -iTCP:443 | head -2 #sanitized output
  COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
  chrome   1234 me   12u  IPv4 12345      0t0  TCP 127.0.0.1:12345->127.0.0.2:https (ESTABLISHED)

Actually, OP replied exactly as he should because after the first few exchanges, you can tell how anal the interviewer is and as such you have to answer accordingly. Some easy going interviewers might say that its about right, some might say its technically right, etc.

Remember, you have to walk through the interview thru OP's mindset and how he took into consideration the interviewer's analness.

> You can't learn about someone's capabilities by putting them on the spot with trivia questions.

But that's not the point of these questions. These questions are a 5-10min phone pre-screen before getting to the actual interviews. They test if the candidate has experience in a given field, not if they can search for information or what are the precise bounds of their capabilities.

It's trivia, but it's trivia that is a reasonably high confidence proxy for experience in e.g. network programming/design (resp. other fields). If someone claims they have networking experience and don't know SYN/SYN-ACK/ACK it's in my opinion a large red flag.

On the other hand, you are interviewing a programmer. Their daily life revolve around technicalities where the difference between returns and fills matter a great deal. Expecting them, all of a sudden, to throw that out of the window and parse technical questions in a hand wavy way is ridiculous.

returning multiple values was the key part of the sentence

Apologies. Throughout my CS undergrad I had only been given the impression and understanding that Big-O measured worst case (lower bound, no worse than), Big-Theta average case, and Big-Omega best case (upper bound, no better than). Looking into it more now, I see that there are some more subtleties I either missed in class or was never taught.

Thanks for correcting me!

yes it's called "Stevens" https://www.amazon.com/TCP-Illustrated-Vol-Addison-Wesley-Pr... although you might also check TAOCP.

The subtlety here is basically that big-O and big-omega and friends are ways of characterizing functions, and functions map one input to one output. "Running time of a problem of size n" is not a function; it has a range of possible values for a given n. "Maximum running time of a problem of size n" is a function. That function itself, an² + bn + c for some constants a, b, and c, has lower and upper asymptotic bounds.

I thought you were right at first but then realized what was going on. This is a pretty subtle point and mostly uninteresting for well-understood algorithms like quicksort. But one slightly less subtle point is that big-theta isn't average case, it is the combination of big-O and big-omega, i.e., bounded from above and below (possibly with different constant factors) by the same asymptotic behavior.

These screens pass people who can answer trivia but who can't effectively code, and they reject people who have a gift for solving engineering problems with code but who can't answer trivia questions when put on the spot.

The ostensible reason they get deployed (I say "ostensible" because we all know that in reality the on-site interview consists of the same stupid kinds of questions) is to keep the employer from wasting time conducting more sophisticated interviews for candidates who have no hope of passing. But that's dumb for at least two big reasons:

* The filters obviously reject candidates who would do well on more serious challenges --- worse, they do it insidiously, because you can't tell that they're rejecting good people, only that you're seeing fewer bozos, which makes them look like they work. In reality, a new norm has arisen where the most qualified candidates get to skip these processes entirely, because we all know they're a crap-shoot and don't want to lose good people.

* Properly administered work sample challenges actually take less employer time than these stupid trivia screens do. That means there's literally no purpose to the trivia screens whatsoever; they do nothing but harm.

In my last comment, I just wrote off the top of my head a sketch for a work sample test that addresses the same concern as the dumb TCP/IP trivia quiz from the original post. It took me I think something like 30 seconds to come up with it. Think about how you'd score that (remember: the bar here is "must be more predictive than that dumb trivia question"). I'm thinking something along the lines of "run the code and see if it opens a new TCP connection".

Assuming your team has enough sophistication to build work sample challenges like this, try to justify the trivia interviews. I think you can't.

Thanks for the detailed response. Frankly, I was awful at recruiting: grilling candidates who claimed to know C++ on the intricacies of templates or whether they knew what Koenig lookup was. Caught a few fibbers but I don't understand in retrospect what the point was.

> These screens pass people who can answer trivia but who can't effectively code, and they reject people who have a gift for solving engineering problems with code but who can't answer trivia questions when put on the spot.

SRE's are hired to fix outages and other problems asap, knowing trivia is very important then since at that point you might be losing a million dollars per second.