The bar has also been raised significantly. I had an interview recently where I solved the algorithm question very quickly, but didn't refactor/clean up my code perfectly and was rejected.
So with that in mind I'll see you all at ReInvent
While it's fine when doing the job the purpose of the interview is to gauge your ability to understand and solve problems, while AI can help you with that you understanding how to do it yourself signals that you'll be able to solve other more complex wider-spanning problems.
Just like with a calculator - it's important for candidates to know _why_ something works and be able to demonstrate that as much as them knowing the solution.
They said the opposite of that. Unless you think it's not possible to figure out problems and you can only do them by rote memorization?
This is literally what AI is, and why they don't want it used in the interview.
It is really good at being a "college professor" that you can bounce ideas off of, though. It is not going to give you the solution (it fundamentally can't), but it can serve to help guide you. Stuff like "A similar problem was solved with <insert research paper>, perhaps there is an adaptation there for you to consider?"
We're long past a world where one can solve problems in a vacuum. You haven't been able to do that for thousands, if not millions, of years. All new problems are solved by standing on the shoulders of problems that were solved previously. One needs resources to understand those older problems and their solutions to pave the way to solving the present problems. So... If you can't use the tools we have for that during the interview, all you can lean on is what you were able to memorize beforehand.
But that doesn't end up measuring problem solving ability, just your ability to memorize and your foresight in memorizing the right thing.
And this is why I never give coding-puzzle interviews. I just have a chat about your past projects (based on resume). We'll go deep into the technical details and it is easy in such a conversation to get a feel for whether you actually contributed significantly to the things the resume says you did.
This doesn't work, because regardless of what the rules say if i think all my competitors are using AI (and you won't be able to reliably detect it) i'll feel pressured to use it as well. This is true of any advantage (spending extra time on 2 hour takehome assignment is the classic version of this)
"Write this code, but don't read the API definition (like a normal developer would do in the course of their work)"
"Whiteboard this CRUD app, but don't verify you did it right using online sources (like a normal developer would do in the course of their work)"
"Type this function out in a text document so that you don't have the benefit of Intellisense (like a normal developer would have in the course of their work)"
"Design this algorithm, but don't pull up the research paper that describes it (like a normal developer would do in the course of their work)"
You're testing a developer under constraints that nobody actually has to actually work under. It's like asking a prospective carpenter to build you a doghouse without using a tape measure.
It is not possible to solve a problem from scratch. You must first invent the universe, as they say. Any solution you come up with for a new problem will build upon solutions others have made for earlier problems.
In the current age, under a real-world scenario, you are going to use AI to help discover those earlier solutions on which to build upon. Before AI you would have consulted a live human instead. But humans, while not what we consider artificial, are what we consider intelligent and therefore presumably fall under the same rule, so that distinction is moot anyway.
Which means that, without access to the necessary tools during the interview, any pre-existing solution you might need to build upon needs to be memorized beforehand. If you fail to remember, or didn't build up memories of the right thing, before going into the interview, then you can't possibly solve the problem, even if you are quite capable of problem solving. Thus, it ends up being a test of memory, not a test of problem solving ability.
And for what? AI fundamentally cannot solve new problems anyway. At best, it can repeat solutions to old problems already solved, but why on earth would you be trying to solve problems already solved in the first place? That is a pointless waste of time, and a severe economic drain for the business. Being able to repeat solutions to problems already solved is not a useful employment skill.
With that said, none of the interviews I've had over the last couple months included questions that could reasonably be done with an LLM. The context is usually wrong, technical challenges were done live on a video call and it would be horribly obvious if a candidate was just prompting an LLM for an answer.
I’ve been out of work for a couple of years due to complicated immigration reasons, and I was most recently a people manager (although with a few direct technical tasks still). I honestly don’t remember many of the deep technical details of things to which I genuinely contributed significantly as an individual contributor or tech lead, despite those being entirely real and despite me still being a capable hands-on technical person. I’ve had so many jobs recently reject me for reasons like this without giving me a chance to actually demonstrate what I can do.
Memory tests are biased toward people who did the work recently, and biased against people with ADHD (who often have worse long-term memory for such details without being worse hires).
The coding interview shouldn’t be just a blind submit and wait for feedback, nor a live rushed and high-pressure puzzle test (you’re quite right in that regard). Ideally it should be the candidate doing what’s expected to be 1-3 hours of work asynchronously at their convenience within a period of a few days, and then discussing (maybe even presenting/demoing) live in a way that shows deep technical understanding and good communication skills. That avoids conflating memory tests with technical tests. Certain live coding tests can also be okay, but I agree it’s easy to make them unnecessarily uncomfortable with a false signal either way.
I've never been in a situation where I could not ask for clarification on something except in interview situations. I asked an interviewer once "is this how people normally work here? they just get a few sentences and plow ahead, without being able to ask for more details, clarifications, or use cases?". "Well, no, but you have to use your best judgement here".