Google's “Director of Engineering” Hiring Test

Well, general interviewing (unrelated to tech) contains various amounts of "are you lying on your resume" type questions. If someone walks in with a breakdown of 10 years dev, 5 years management, they should be able to at least comfortably answer system/coding type questions. As in, if you do something every day for 10 years, you don't forget all of it in 5.

I had a candidate in a few months ago that was interviewing for Software Development Manager, so he got an initial phone screen and then a face-to-face with myself and another dev on the team he'd be managing. I was impressed with how little he knew about programming.

"Name some data structures." "What does MVC stand for?" "Name some design patterns" etc. All of which were unanswerable. Generally when it becomes clear someone was dishonest about their skillset, the ability to get hired for any position becomes impossible.

How can you not know what MVC stands for? It's pretty much a buzzword!

I mean, yeah, 99% of candidates should know what that means because it is an extremely common initialism. Although, I could see some engineer who worked on networking drivers for 10 years might not be up to date on the design patterns of frontend engineering.

My kick-out questions:

"Could you write out what an HTTP request and response looks like on the board?"

I'm really surprised at how many people can't do this. If you've spent five years developing web, surely you've had to look at raw requests, either debugging using netcat or with wireshark or just looking at the information in the Chrome/Firefox debugger?

"What's the difference between a GET and a POST request?"

"What is the difference between a statically typed and a dynamically typed language?"

I had one candidate try to tell me Java was dynamically typed and Scala was statically typed. It was for a Scala position. They also said "statistically typed" instead of statically, even after I corrected them.

-_-

Not all programming/engineering circles use the same buzzwords. For five years my mobile development groups used the concept without the acronym.

I believe 90% of my coworkers and former coworkers would be unable to answer the HTTP response question.

And 95% haven't used netcat or wireshark. I wouldn't have either, if it wasn't for some particular work related to messaging.

They're able to develop reasonable line of business websites in spite of that.

I would be extremely worried if they were unable to answer about the difference between GET or POST, or the difference between statically and dynamically typed languages, so I agree with those.

Why it is important to know the difference between statically and dynamically typed languages? If one writes in only one of those (or one set) it is not important to him/her and doesn't specifically make him/her a worse programmer in that particular language.

Knowing a cursory difference between a statically and dynamically typed language in this day in age is not an unreasonable requirement for many developer positions, especially web development where you're often using a mix of languages.

As always, this sort of question is a test of competence by proxy and there are usually outliers, but statistically speaking, I think you'll find a very high correlation between inept programmers and people who don't know the difference.

General domain knowledge. We do Scala and a little bit of Python here and there. You should know the difference to show you're well rounded. Senior devs sound have some experience in both types.

I have a follow up question, "What are the advantages of a dynamically typed language over a statically typed one?"

This one kinda exposes the "Java-zellot" side of programming. If you love Scala and you're applying for Scala position, you don't often think like this. Being able to think critically about the things that are harder in Scala, that would be easier in a language without strict type checking, is a another good way to gauge if people can think critically.

The number of web devs I've encountered who regard my ability to talk HTTP over telnet as black magic makes me sad.

Agreed. I interviewed a few QA candidates at a previous company that used a term completely differently than we did. When I rephrased the question from defining the term to "what kind of test would you run in this situation" I got the kind of answers I would expect. It's far more important that a candidate understands the concepts needed to solve a problem, than that they have memorized a term.

Hell, someone could be able to define MVC and explain how you would use it, but have no idea how to actually implement something using it for a given programming language.

I'd be able to write HTTP request by hand, I've done that quite often, however I would not expect that to be a common skill. Looking at something, even often does not in any capacity mean that you would be able to reproduce that from memory.

Even then it's worth remembering not every MVC is the same. Fat/slim models. Intelligent/simple views. There's lots of approaches to even a well known paradigm.

> "Could you write out what an HTTP request and response looks like on the board?"

Why should anyone remember what an http request or response should look like? Statically typed vs. dynamically typed language?

Fuck.

Are these entry-level positions or for someone with 10 years work-ex? A simple search on Google can tell anyone the answer of these questions, why do you expect people to carry an imprint of it in their memory? If the problem they'll work on mandates knowing these things it'd be pretty easy to solve with just one search. It is exactly questions like these that are worth kicking the host organization back in the butt.

Either your interviewing process is hilariously stupid or you're just spiking it up to boost the ego here.

Scala is statically typed.

I would not want to work for or with him, if he asked me that in an interview, I'd walk out.... if he think thats what good computer scientist should know...

Static versus dynamic typing is so fundamental that I don't see how a programmer could be remotely competent without having been exposed to those concepts enough to have internalized them. It would be like an accountant not knowing what the number 4 is. Yes, you can look it up, but if you need to then how did you ever get this far?

That's exactly what happened to me. I was stuck in embedded systems world right out of college and then one day took interview with Google, they were asking me questions clearly looking to hear "MVC" in my answer but I just didn't know it back then...

It is also a ridiculously dogmatic question. Many people believe into a fallacy that static typing makes safer programs, for example, and expect that somewhere in the answer.

In all fairness, if you can talk normal HTTP 1.1 over telnet with some service, someone configured TLS wrong ;) And if you can talk HTTPS over telnet unassisted... well, I am truly impressed.

Knowing proper terminology is necessary in order to stay up to date with developments in your professional subject area. If concept X is an established concept in your professional area of expertise, and you don't even know that its name is concept X, then you likely have not read much about concept X, and consequently, are likely not up to date with current developments related to concept X. This isn't just semantics, it's about professional literacy.

Could you explain how static typing makes less safe programs?

That's a preposterous attitude. Just imagine if we took a similar approach to hiring for other kinds of jobs:

"OK, so you'd like to work here as a mechanic. What's the difference between automatic and manual transmission?"

"It's not fair to expect me to know that off the top of my head. If I need to know, I'll just do a Google search."

How is it dogmatic? Sure, there's a lot of dogma around which one is better, but simply explaining what each one is and what's objectively different about them isn't remotely dogmatic.

There's always openssl s_client

Right. Everyone and their mum know what M and V stands for, right? Now .. C? C is tricky. Please don't ask any further questions about C, will you?

I'm not sure if this is a joke. Model and View are really clear. Controller I usually find munged in with the View and it's not always profitable or clarifying to separate it.

Static typing is basically a bunch of free type-based unit tests. You can write safer programs in dynamic languages, but you need to write and maintain a lot more tests.

That's a crap comparison.

You're gonna have a hard time drawing a comparison between a line of work where you build things and one where you fix things.

OK, ask me that question about defining the difference and I'll argue with the question, and back up my argument with examples of how type systems are far more of a spectrum of different cases than a stark static/dynamic binary.

And then your non-engineer phone screener who's expecting the answer to match the scripted sheet will conclude that I don't know this "fundamental" thing and thus am unqualified.

You can't compare static + N tests, vs not static with M > N tests.

Compare static with N tests, vs not static with N tests. In what case would the not static be safer?

If the type system is not expressive enough and you have to get around it?

The claim that "dynamically type language" allows code to more closely follows the business logic has merits. And you could follow from that to claim that type system could be causing more bugs (ie less safe).

I basically lived in Wire shark for a couple of years working for a voip company and still use stuff like curl all the time and I don't think I could walk through an http request of the top of my head.

If they didn't know it, I'd want to dig down into whether the understand the specifics of their particular language at least.

I actually just sat down in a meeting with a dozen programmers, some of them with decades of experience, and half of them didn't know what functional programming was.

There is some truth to the saying that some people don't have ten years experience, they have two years of experience five times in a row.

Learning to use wireshark or tcpdump is a power tool that does show whether you got more experience in understanding the lower levels, or stayed at requirements-and-tests. (Not necessarily bad, but a good "fork" to jump off from)

But could you describe the general structure?

Yeah expecting many people to be able write out a complete http request from memory without a reference to look at. But the general structure of a http request is something so basic to web development that asking what the structure of a http request looks like isn't an unreasonable expectation.

Request line (method, uri), header(s), empty, body...

"OK, so you'd like to work here as a mechanic. What's the difference between automatic and manual transmission?"

Nope, more like "can you write out on the board what types of connectors are used in the car cooling system and in what order".

And the half that didn't know where much worse programmers than those that knew?

Your example shows that not every programmer has to know that.

GET / HTTP/1.1\r\n and some kind of sensible response is not too much to expect someone to know. HTTP is super easy and I see the HTTP transaction test as "did you ever get curious as to how exactly a core part of the current Internet actually works". I'm sure that there are app developers out there who can spin crud stuff all day and have no idea about this, just as there are curious people who couldn't stand up todomvc to save their life, but in general, all of the most talented people I've worked with knew their stuff front to back, and had at least a few areas of expertise.

CGI is also cool to learn about the workings of, since it almost seems too simple.

Ok, how about "What is the firing order on a Chevy LS1?" ?

Yep, and when devs watch me key in the s_client pipe to OpenSSL to dump the cert info it's like I've become Neo and entered bullet time. I guess they don't need to know this stuff, but trying to do things like editing a hosts file in OSX, flushing dns, opening an incognito tab, looking at the SSL cert through the GUI, manually comparing it to one in an editor....vs a pretty short one-liner with curl or OpenSSL and a diff....I guess I'm either biased or lazy. I also almost never get asked WTF I just did, just a "wow, thanks" at most.

A previous employer had a sysadmin wiki. We call it Devops now, but I really liked working with the plain-text files of Dokuwiki there. Confluence is good for some things, but as a notebook of shell snippets and when to use them, it's not great.

Which would be true, or rather: over-qualified.

This isn't a question a non-engineer phone screener can ask. Coming up with first pass filters that don't require an engineer to interpret is harder.

I have yet to see a large static typed program that didn't -- somewhere -- run into the limits of static typing and contain a set of workarounds, using void* or linguistic equivalent. That's code a dynamic language doesn't need.

The only code you can be sure isn't buggy is code that doesn't exist.

> I see the HTTP transaction test as "did you ever get curious as to how exactly a core part of the current Internet actually works".

Sure I did.

Then I forgot most of the details because they didn't matter, and I knew I could look them up quickly if I ever needed to write a HTTP client/server for some reason.

if you answer "GET / HTTP/1.1\r\n" I'm going to ask you if you left anything out.

Because you did: after that you have to provide a Host: <hostname>.

This is proposed as a question that an engineer would ask, not some base-level screener.

Exactly. These are memory tests, not ability tests. Beyond a very basic level, memory tests are too random to be useful.

I once aced a geography exam because I happened to read up on the economics of Nigeria just before I took it. By sheer luck, there was a question about Nigeria in the paper.

If I'd read about Zimbabwe instead I'd have been screwed.

Neither possibility provided much insight into my competence as a geographer.

Even if a job spec needs specific knowledge of key facts, you can't generalise from pass/fail memory questions to broad spectrum competence, or lack of it.

If a candidate has no idea what an HTML request is, that's one thing. If they know damn well what a request is but can't list all the elements in a stressful interview while you're staring at them, - because in fact they spent the last year working on database code, and the API stuff was the year before that - that's something else entirely.

I'm sure a lot of people know what MVC stands for. I don't think there's anyone on the planet who can be sure they know what it really means.

> It would be like an accountant not knowing what the number 4 is.

It's a hypothetical no-go! Every person, even the fourth grader knows the number 4. So why ask a question that measures their ability to remember 4, say 4 or show that they know 4.

> I don't see how a programmer could be remotely competent without having been exposed…

Share this link with them:

http://stackoverflow.com/questions/1517582/what-is-the-diffe...

Invest in people and people will invest back in your business. Interview process that I follow at my workplace has just one goal to assess: whether or not it'd be great to work with this person and spend over ~50 hours per week with them.

Are you hiring fun people who know nothing about computers? Or are there actually more criteria than you let on here?

You would've gotten it wrong though!

You need two newlines to finis the request, plus the HTTP 1.1 standard requires clients to send a Host: header for all requests.

Not saying every interviewer would care about that in an early screening process.

Well, following Netflix's mantra - it is a team and not family that you are hiring for. Anyone can Google and find answers, doesn't mean you would hire everyone, would it?

There are a number of basic items that a competent programmer needs to know off the top of his head. If they had to google for every single item, then their productivity goes down the drain and so does the entire team's productivity. You should fix your hiring.

> I also almost never get asked WTF I just did, just a "wow, thanks" at most.

Asking "wtf did you just do" is responsible for probably 1/5 to 1/4 of the professional knowledge I have today. It's sad that many people ignore that people will often teach you their little tricks if you ask.

> hiring fun people…

Absolutely! This is super super important. Fun to work with, not annoying to waste time with.

> know nothing about computers

It's sad that you think this way of people who couldn't answer your questions at the expected level.

> Or are there actually more criteria than you let on here?

Yes! One way to know if they're any good or not suitable is by giving them a problem statement like so:

'Design X, feel free to choose a language that's suitable for this problem', and then may be proceed to hint with: 'You might want to look at advantages of Static versus dynamic typing'… and then let them ask whatever questions they want to ask or read up or search or start implementing whatever.

Observe what they do -- and how fast can they get to the decision of what language and why. And how to make X (break down of steps) or if they can dive and start making X there itself. Note, if they had theoretical knowledge of what you seek during an interview it will work to their advantage naturally. Or sometimes not.

Of course, this process may not work for you as it does for us -- therefore seeking direct answers about static vs dynamic language may not be such a bad question after all (I get it), but expecting people to accurately remember what an http request or its response looks like may not be fruitful at all. It can throw good people off guard and ruin the rest of the interview for them.

void* is usually not a symptom of limits of static typing, but limits of the [type system] design or human brain. You can think of it as "ok, I give up. Anything can be passed here, proceed at your own risk, compiler will not save you here, errors will show up at runtime". Even the memory safe Rust does not do without such unsafe blocks. In dynamically typed languages that is everywhere, though. I have said this before: safety benefits of static typing show up when you are working with at least data structures, not simple variables. Imagine you have an external endpoint or library call that is specified to return a single object and does exactly that. At some time after release you are the maintenance programmer responsible for implementing spec changes:

  * The object returned no longer has member/property x, it is obtained by other means;
  * The endpoint returns list of such objects.

How sure are you that tests in dynamic language cover these cases? My experience shows that tests very rarely get designed to anticipate data changes, because data is driving test design. Which is more likely for a test: a) to test whether object returned contains keys x, y and z; b) to check if the object returned is_list() (see appendix)? Static typing covers such cases. Static typing is not something that magically saves oneself from shooting them in the foot, but is nevertheless a safety tool that CAN be used. It is of course a burden if one does not intend to use it and that is the core of the debate.

Fun thing: in the second case if your code manages to convert input list to a map and assign one returned object to a key that coincides with the removed property and map access looks syntactically the same as property access (a very specific set of assumptions, though), the bug can butterfly quite deep into the code before manifesting :)

What? HTTP 1.1 doesn't require TLS.