Google's “Director of Engineering” Hiring Test

First, it is definitely standard process to tell him (if they didn't, that's a definite failure). Again, remember you only have one side of the story here.

I like to try to gather facts before assuming things. IE Ready, aim, fire, not fire, ready, aim.

Admittedly more difficult in this case (and certainly, i have no access to it)

Second i'm going to point out a few things:

Experience may translate into wisdom, it may not. Plenty of companies promote people just because they last long enough. So 20 years experience managing may translate into a high level manager, it may not!

I hold a bunch of patents too on compilers and other things, it's not indicative of much in terms of skill, because almost anything is patentable.

Lastly, SRE is not an ordinary site maintenance position by any means. I"m not even sure where to begin to correct that. I guess i'd start here: https://landing.google.com/sre/interview/ben-treynor.html

Does this mean this person is under/overqualified/exactly right? I literally have no idea. I just don't think it's as obvious one way or the other.

"Well, that sounds like a dumb recruitment process."

Judging an entire recruitment process based on one side of a story from a person who's clearly upset about an interview, and even 3 sentences i wrote on hacker news, seems ... silly.

If you want to do it, okay.

But everyone in this entire thread seems to be making snap judgements without a lot of critical thinking. That makes me believe a lot of people here have a ton of pre-existing biases they are projecting onto this in one direction or the other (and you are, of course, welcome to claim i fall into this category too!)

I almost didn't jump into this discussion because it seems so polarized and rash compared to a lot of others

I think i'm just going to leave it alone because it's not clear to me the discussion is going to get any more reasonable.

Google has never made it that clear what position I was interviewing for (and definitely not what team/role) when I interviewed with them. This was sort of pitched as a selling point, since after being hired you'd float around and find the niche eventually?

>Judging an entire recruitment process based on one side of a story from a person who's clearly upset about an interview,

It's not just this guy. There have been others: https://twitter.com/mxcl/status/608682016205344768

There's another measure I use to measure the quality of their hiring process. The output. Namely the track record of products Google has developed in house in the last 10 years.

I've also heard a few stories about friends applying for a position and being shunted by the hiring process into the hiring funnel for other (plainly unsuitable) positions. When I hear a very specific criticism from two separate places it's hard to stay skeptical.

> SRE is not an ordinary site maintenance position by any means

Then why ask about the nitty gritty details required by maintenance personnel as part of the screening process - things I would rather have my high level employees looking up rather than relying on a possibly faulty memory.

> Judging an entire recruitment process based on one side of a story from a person who's clearly upset about an interview, and 3 sentences i wrote on hacker news, seems ... silly.

This kind of opinion is not formed in a vacuum. It's formed of the dozens of posts that appear every year about how someone who seems qualified is turned down for spurious reasons like "being unable to reverse a binary tree on a whiteboard". It's what makes this particular post so believable - it fits the stereotype. Even your own developers who post here say "yeah, that's more accurate than inaccurate." Perhaps it wouldn't hurt to "undercover boss" your way through the interview process...

Speaking for myself, and only myself... I turn down all Google recruiters because I know I would not pass Google's interview process. Not because I don't have the skills, but because I don't have a college degree. Because I don't see the return on investment for studying for the next 6 weeks just to pass the interview process, especially when I won't even know if I'm getting a job I'll enjoy.

> I think i'm just going to leave it alone because it's not clear to me the discussion is going to get any more reasonable.

How about the responses from your own employees which are pointing out that they see the problem too. Are they being unreasonable?

When was this? This was the case when i started (~2006), but it definitely changed and is not the case anymore.

> Judging an entire recruitment process based on one side of a story from a person who's clearly upset about an interview, and even 3 sentences i wrote on hacker news, seems ... silly.

How about the dozens of other seemingly qualified people who have complained about the google process?

> There's another measure I use to measure the quality of their hiring process. The output. Namely the track record of products Google has developed in house in the last 10 years.

That's a poor metric to evaluate the rampant complaints about a high false negative rate. I don't think that many people are disputing that the people who do get hired are qualified most of the time.

Probably late 2000s when I was last on site. Google bugs me every year (most recently a week or two ago), but I don't usually push on the process.

"Then why ask about the nitty gritty details required by maintenance personnel as part of the screening process - things I would rather have my high level employees looking up rather than relying on a possibly faulty memory. "

This is one reason why i find it super-strange. It's not a set of "high level employee" questions. It's a standard SRE pre-screening.

"How about the responses from your own employees which are pointing out that they see the problem too. Are they being unreasonable?"

My view of unreasonable is not about whether there is a problem or not. It's not about the consensus. I don't actually have an opinion myself on the hiring process. If people i work on recruiting raise problems, i try to solve them. I have not had trouble trying to recruit in general. So i haven't formed a strong opinion, even after 11 years. If folks want to decide the process is horrible, okay. If folks want to decide it's great, that's also okay.

But it's unreasonable because it's both super-quick reaction without time to settle and think, and not aimed at anything other than trying to reinforce one view or the other.

Nobody is actually listening to each other, they are just trying to force whatever their view is, good or bad, on others.

So to answer you directly, i don't think pointing out a problem is unreasonable, but that's not my complaint. My complaint is that the actual discussion is not a discussion, but mostly people just arguing on the internet. IE You shouldn't take me saying "unreasonable" as a proxy for "me saying i think their viewpoint is wrong". I just think the mechanism of discussion here is unlikely to yield fruitful results.

"How about the dozens of other seemingly qualified people who have complained about the google process?"

And what's the other side of that? IE the literally tens of thousands to hundreds of thousands who haven't?

Again, i'm not saying there is no problem, i'm just saying this is probably not a great mechanism to evaluate whether there is a problem or not.

If you want actual usable data, this wouldn't be the way to get it, good or bad.

Interesting. I could look up the date it changed, but it definitely changed because folks didn't like the old way :P.

Now, instead, they generally don't recruit (google is too large to not have exceptions) without some specific hiring managers and headcount in mind.

They will tell you what those groups are and what they do. So for example, the person i interviewed last week was targeted at two teams. I actually specifically asked if he knew what he was being interviewed for, because i like to get some idea what the candidate thinks whatever job they are interviewing for means, and he was able to tell me the two groups and knew what they did.

> It's a standard SRE pre-screening.

To clarify, I was speaking of your standard SRE hires, whose position you referred to as "not maintenance drones".

When the in house engineers come out with products like Wave and Glass while things like Maps and Android are purchased you have to wonder.

I suspect for someone who has failed - rightfully or not - a recruitment exam in this manner, it may in fact be the only cathartic mechanism.

>First, it is definitely standard process to tell him (if they didn't, that's a definite failure). Again, remember you only have one side of the story here.

"Standard process" is what actually happens in the real world. Alas, standard process is to not tell him.

>But everyone in this entire thread seems to be making snap judgements without a lot of critical thinking. That makes me believe a lot of people here have a ton of pre-existing biases they are projecting onto this in one direction or the other (and you are, of course, welcome to claim i fall into this category too!)

Your story is also just one side of the story - actually, you weren't even involved so it's neither side. Still, you spend all your effort on saying why for example this guy's patents mean nothing and he's likely incompetent. I'd call that snap judgement, lack of critical thinking, and biased conjecture,

So, as someone who went through the process and got through it (so is less inclined to hold a grudge):

> Then why ask about the nitty gritty details required by maintenance personnel as part of the screening process - things I would rather have my high level employees looking up rather than relying on a possibly faulty memory.

AIUI you can get easily 5 or more of the pre-screen questions wrong and still proceed to the next stage, depending on your experience and how wrong you are. The point here is not that you know each and every one of those things, but to show that you are, in general, knowledgeable enough to spend Engineer hours on.

And your judgement of these questions is seriously impaired by the fact that they are written down wrong. I assume, that the author of this post has written down a rough transcript from memory and as such it's colored by their own (mis)understanding of the question and whatever got leaked from memory in the meantime. The questions he wrote down are, at the very least, not verbatim the ones from the checklist given to recruiters (and there is a strong emphasis on reading them out verbatim there, so I consider it relatively unlikely that the recruiter didn't do that).

> It's formed of the dozens of posts that appear every year about how someone who seems qualified is turned down for spurious reasons like "being unable to reverse a binary tree on a whiteboard". It's what makes this particular post so believable - it fits the stereotype.

Exactly. You are reading "dozens of posts every year" from disgruntled interviewees who got rejected and are pissed. On the flip side, a quick internet search will tell you that Google gets on the order of millions of applications each year, meaning you don't hear from >99.99% of applicants.

There is also the widely advertised fact, that the Google hiring process accepts a high false-negative rate, if that also means a very low false-positive rate, so it is to be expected that a good percentage of qualified applicants still get rejected. It is thus also to be expected, that you hear from some of them. Meanwhile, again, you are not hearing from the thousands of qualified applicants that do get accepted each year. Because an "I interviewed at Google. It was pleasant, everyone was really nice and they got me a good offer" blog post won't draw a crowd on hacker news, even if it was written.

> How about the responses from your own employees which are pointing out that they see the problem too. Are they being unreasonable?

Let's not ignore the responses from Employees that don't think there is a problem.

From reading this post, I'd say a likely reason for the rejection is, that this person wasn't being particularly pleasant. Frankly, he comes of as kind of an arrogant prick. And, as a general rule, engineers at Google, just like everyone else, don't particularly like having unpleasant people on their team. And I also believe this post has gotten enough upvotes, that someone will look into the situation to see what went actually wrong here.

> "Standard process" is what actually happens in the real world. Alas, standard process is to not tell him.

Inferring what's standard from a sample size of 1 (which is ~0.0001%) is very questionable.

> Still, you spend all your effort on saying why for example this guy's patents mean nothing and he's likely incompetent.

That is not at all what they where saying. They where saying that patents aren't conclusive evidence of competency.

FWIW, I got told what I was going to work on on my first day, by my new manager, when they picked me up for lunch. Before that, I didn't even know the PA. From what I can tell, that is standard practice for SREs, as SRE is very understaffed, so there is a lot of arguments and back-and-forth around where people are most needed.

Psst: the Rasmussen brothers were behind both Maps and Wave.

https://en.wikipedia.org/wiki/Lars_Rasmussen_(software_devel...

No, the policy/process DannyBee references is fiction. What's standard is what happens in reality. I'm clearly not talking about statistics.

For your second point, DannyBee focuses his efforts on discrediting this seemingly exceptionally qualified candidate, never yielding an inch from his position that Google is exceptional and can make no mistakes.

Not sure what you're saying here. Wave was great technically; the market fit just wasn't there.

> What's standard is what happens in reality.

Infering what is "reality" from a sample size of ~0.0001% is clearly ridiculous. By that logic, it would be "standard" to be born a conjoined twin. Actually, it would be 10x as likely as what "standard" is.

> I'm clearly not talking about statistics.

You might benefit from doing so, though. It might help you realize what nonsense you are saying.

> DannyBee focuses his efforts on discrediting this seemingly exceptionally qualified candidate

No, this is factually incorrect. Repeating something factually incorrect doesn't make it more correct.

> never yielding an inch from his position that Google is exceptional and can make no mistakes.

You either can't or won't read. They very clearly acknowledged the possibility of a mistake several times in each post they made.

Yep, those engineers they took on in the last ten years must suck, they've only managed to develop technologies that grew Google's annual revenue from 10 billion dollars in 2006 to 75 billion in 2015. That's the kind of track record that has to make you question the hiring process, right?

I interviewed at Google in March 2014 and was given an offer. I wasn't interviewing for a specific team. After the in-person interviews my recruiter set me up with 2 different team managers to talk to about potentially joining their team. I wasn't interested in either team, and my recruiter said "That's ok, we'll find a place for you," and a few days later found a new manager for me to chat with. I joined their team.

I did know I was interviewing for a general SWE role, but not anything more than that, and from all appearances the team was completely up in the air until after my interviews.

I don't know how much has changed since 2014. I also didn't get any of these pre-screen testing questions from a non-engineer. Is that normal practice for all interviews now?

>> as SRE is very understaffed in all likelihood due to the flaws in the process. I know quite a few people, who I highly respect, who IMHO are better than the people I know who work at google, who flunked the process.

There are a lot of assumptions being made here. Sometimes companies grow despite poor hiring decisions. I think you need a finer-grained view than just revenue to really tell whether you're doing a good job or not. Lots of terrible decisions have been justified by this "the revenue went up so we must be doing a good job" line of reasoning.

You seem to be confusing "I have a smug twitter-sized sound-bite response" for "I have a worthwhile counter-argument".

It's a common failing these days, but you should probably look into getting it fixed.

That said, yes, Google's hiring process is questionable. The Web is full of horror stories from obviously-qualified people who Google passed on, often very early in the process when no engineer had talked to them, and this suggests Google's success is not sustainable so long as that continues. They'll be able to hire fresh CS grads out of Stanford forever with this process, but the experienced/unconventional people they flunk out on the early screens are not going to come to them, and when their current crop of experienced/unconventional engineers retire or take jobs elsewhere, Google's finally going to have to fix this problem and stop pretending that it's better to pass on a thousand highly-qualified candidates than to give one unqualified candidate an on-site. That, or tumble back down into mediocrity.

(which, to be fair, is already mostly the case; Google is largely a mediocre company, with only a couple of externally-visible brights spots of talent or innovation clustered in a couple of particular teams, and otherwise Google runs on inertia and the hope that the 0.1% of interesting stuff they come up with will keep the 99.9% of mediocrity afloat)

> they are written down wrong.

Please, feel free to correct the record, then, with the correct screening questions. The proverbial cat is out of the bag, and has gone tearing down the street towards everyone trying to make a buck by "training" hopeful young graduates on how to make it through the Google interview process.

> Because an "I interviewed at Google. It was pleasant, everyone was really nice and they got me a good offer" blog post won't draw a crowd on hacker news

No, it won't. Because it's the tech equivalent of a lottery winner saying they think the lottery system is a fair and equitable way to distribute money.

> Let's not ignore the responses from Employees that don't think there is a problem

Same problem. If you're in, you passed the Google employment lottery, so it's much more interesting (and should be more meaningful to management) when insiders agree that the hiring process has problems.

Now then, of course, so long as directors find that they have plenty of applicants to back fill attrition and grow, they have no reason to think the hiring process is broken; so long as Google is happy hiring not necessarily the best people for the job, but the ones lucky enough to dodge more false negative flags than everyone else. Better to be lucky than good.

All that said, yeah, Google's hiring process works for Google. Coming here, to a conversation started by a crappy screening experience, and expecting respect for a process with so many false negatives is a bit optimistic, though.

I think you're neglecting the continuous improvement of successful projects, which take quite a bit of engineering effort.

Was it software quality that killed Wave and Glass, or was it more of the market not wanting either of those things? (To digress, it seems like both of those products came too early. Do you think that wearable computers will _never_ exist? And Slack seems to be the Wave-like thing that the market wanted.)

> Then why ask about the nitty gritty details required by maintenance personnel as part of the screening process

I'm not sure what "nitty gritty details" you're talking about here.

As much as some people here think it's impressive knowledge[1] to be able to give the size of an ethernet MAC address without Googling it, that's something that anyone with experience in computer networking oughts to know. Not at all because it's useful knowledge, but simply because if you actually spend time looking at network traffic dumps or ARP tables or DHCP configuration or SLAAC assignments you'll be seeing MAC addresses so often that it just becomes obvious. Just like knowing that an IPv4 is 4 bytes and an IPv6 16 bytes. Or that a TCP connection starts with a 3-way SYN/SYN-ACK/ACK handshake.

And the same thing applies to the other questions that look like meaningless details: knowing what an inode is and what syscall returns inode data for a path is something that someone with system-level C programming experience should know. stat(2) is far from being something obscure. Knowing what signal is sent by the kill(1) command is maybe slightly more on the trivia side IMO, but it's still a very well known fact.

A candidate is most likely not expected to know the answer to all of these questions. But failing in all of the categories is IMO a fairly strong red flag for someone interviewing for SRE, where in general people are usually expected to be comfortable with at least one of {networking, system administration, Linux internals}. In fact, this domain specific knowledge is the biggest differentiator between "standard" SWE and SRE-SWE, even though the lines get blurrier and blurrier.

This also indirectly answers this:

> things I would rather have my high level employees looking up rather than relying on a possibly faulty memory

You would have to be out of touch with the field for quite a while to forget such basic things. Which is likely something that you want to test for in such interviews. To go with a metaphor: if you claim to be a fluent English speaker on your resume, you can't be excused of "faulty memory" if you forget how to conjugate "to be" in the present tense. It's not something you forget easily, and if you did forget you most likely can't say you're fluent anymore.

Disclaimer: I was an SRE at Google for 2.5 years, but I'm not familiar with the early phases of the recruiting process.

[1] https://news.ycombinator.com/item?id=12701486

I can expand beyond 140 characters if you like. The OP claimed that in the past ten years, as a result of their hiring practices, Google's product output quality has noticeably declined, presumably as compared to the search product on which their name was made, and gmail, which they launched in 2004. And it's easy and fashionable to knock Google because maps is not as good as you remember it used to be, or because they shut down reader, or because plus didn't manage to unseat facebook.

Well, in 2006 Google was a 10 billion dollar search and ad company with a fledgeling email business without a revenue model, who had just bought youtube. In 2008 they shipped a mobile phone operating system. That's now a thirty billion dollar business which has been built up through talent within google. They undermined Microsoft's office monopoly with an online office suite (okay, some acquisitions underpinning that). They have a credible seat at the top table in the cloud market. And they continued to develop their core ad platform to drive more revenue growth.

I've got no particular reason to stand up for Google, they're quite big enough to look after themselves, but the idea that their product flops in the last decade outweigh those product successes, and can be held up as evidence that there is something deeply rotten in their hiring model, seems to be cherrypicking to me. 70% mobile OS share, 70% search share, and 50% of global online ad revenue... that's a pretty good kind of mediocrity.

It's still the case that other than search and ads, most of Google's biggest hits were acquired rather than the result of in-house initiatives (even Google Analytics, which is probably one of their more heavily-relied-on products, was acquired). Google doesn't hire people who will create stuff like Android; they hire people who can pass their interview process, and get new product and service lines mostly through acquiring teams of people who probably can't pass their interviews.

It's also the case that Google is acquiring a reputation for bad interview/hiring processes, and for hiring people who have a Ph.D. in CS and putting them to work on CRUD web apps that any random coding-bootcamp grad could build, since there's just not enough interesting in-house work to keep all those top talents occupied.

Slack is in no way like Wave. Now you're just over reaching with your comparisons. Wave's flaw was showing you what the other person was typing as they were typing it. You try to separate quality from functionality and stick that to market's fault because it doesn't want Wave's functionality. That is not mutually exclusive. Wave's quality was egregious.

> Please, feel free to correct the record, then, with the correct screening questions.

No can do. I actually like my job. And I also like my coworkers and don't want to make their life any harder.

> No, it won't. Because it's the tech equivalent of a lottery winner saying they think the lottery system is a fair and equitable way to distribute money.

The same goes for a "I interviewed at Google. It was pleasant, everyone was really nice but sadly I didn't got accepted" post.

The fact remains, that you don't read from >99.99% of people. My interview process was very pleasant. I had a bunch of nice conversations about programming and computers with friendly and humorous people.

> Same problem. If you're in, you passed the Google employment lottery, so it's much more interesting (and should be more meaningful to management) when insiders agree that the hiring process has problems.

There are a lot of insiders. With a lot of opinions.

> so long as Google is happy hiring not necessarily the best people for the job, but the ones lucky enough to dodge more false negative flags than everyone else.

Well, the thinking here isn't really "we want strictly the best". That would be a hopeless idea from the get-go. The thinking is "there is a hiring bar that we want people to pass and we want to hire exclusively from above that. We don't care about the sampling of that, as long as we get that". What they end up with is a pretty broad sample of that population. Some (like me probably, tbh) just barely pass the bar, some are the very top. Some other top-people got unfortunately rejected, some other barely passing people too.

So yes. There is indeed no ambition to actually get just the top 100K engineers in the world.

> All that said, yeah, Google's hiring process works for Google. Coming here, to a conversation started by a crappy screening experience, and expecting respect for a process with so many false negatives is a bit optimistic, though.

Well, mostly I (and DannyBee) are just pointing out obvious flaws in the discussion here. Like the obvious self-selection bias and selective reporting. And the also obvious fact that this particular post was written while angry and only represents one side of the story; and that not even accurately.

Secondarily, in these long-wound comment threads on reddit/hackernews/twitter, people seem to usually not even be aware of the goals of the hiring process and think "look, here, three prominent false negatives" is an actual argument about the process being flawed.

Why is it a poor metric? Isn't the point of hiring employees to ideally build and launch successful products?

I think Google is pretty good at hiring "qualified" engineers who are very good at maintaining and scaling existing systems, but the process definitely selects against entrepreneurial product-focused engineers. Maybe Google thinks that's fine though: they can always pick them up through an acquisition later, albeit at 100x the price.

>I can expand beyond 140 characters if you like. The OP claimed that in the past ten years, as a result of their hiring practices, Google's product output quality has noticeably declined, presumably as compared to the search product on which their name was made, and gmail, which they launched in 2004. And it's easy and fashionable to knock Google because maps is not as good as you remember it used to be, or because they shut down reader, or because plus didn't manage to unseat facebook.

I don't necessarily blame them for plus (facebook was clearly a marketing success, not a technology success), but maps' decline isn't anybody else's fault. It has declined in quality and that is plainly an engineering failure not a product failure.

>Well, in 2006 Google was a 10 billion dollar search and ad company with a fledgeling email business without a revenue model, who had just bought youtube. In 2008 they shipped a mobile phone operating system. That's now a thirty billion dollar business which has been built up through talent within google. They undermined Microsoft's office monopoly with an online office suite (okay, some acquisitions underpinning that).

Well, yes. Acquisitions underpinned all of that success.

>I've got no particular reason to stand up for Google, they're quite big enough to look after themselves, but the idea that their product flops in the last decade outweigh those product successes, and can be held up as evidence that there is something deeply rotten in their hiring model, seems to be cherrypicking to me. 70% mobile OS share, 70% search share, and 50% of global online ad revenue... that's a pretty good kind of mediocrity.

All predicated upon outside purchases or the original self-reinforcing search monopoly developed before 2004.

What's worse is that they've often used their search monopoly to try to break into other markets (flights, shopping, etc. - plenty of stuff like this got preferential SERPs treatment) and failed because what they released was crap. That is, they failed even with a huge home ground advantage - the kind of monopoly advantage that let Microsoft make IE6 (IE6!) the industry standard for years and got them slapped by the DoJ couldn't even be put to good use by Google.

I'm not denying that they have some good engineers but the idea that they're the creme de la creme of the industry with the best hiring process is way way off base.

And Comcast has some of the best customer service and engineering because they don't seem to be losing any customers.

Right?

Funny you should mention that. I was just using maps and thinking "this is worse than it used to be".

From what I've heard from insiders, the adwords code base is an enormous mess. Not surprising for a product that old perhaps, but this points to their engineering practises being about as mediocre as the industry average.

I don't honestly know why people want slack. It seems to just be in vogue - one of those weird network effect things. It doesn't seem to have anything to do with their feature-set or engineering quality because it's not noticeably better than, say, hipchat.

>To digress, it seems like both of those products came too early. Do you think that wearable computers will _never_ exist?

They already exist.

Google internally-initiated successful products that come to mind: Cloud (2nd or 3rd in market, lots of revenue and growth), Play Store (also lots of revenue and growth), TPU chip, SDN, Photos, Chrome, ChromeOS.

Google (vs Alphabet) often acquires companies that have a seed of a useful product. Android for example was apparently not in a usable state when it was acquired. 99% of the creative work is making the thing actually work, not in having the prototype.

To say Google's own engineers didn't create Android because they didn't commit the very first line of code is doing them a disservice.