This makes it increasingly difficult to fairly assess a candidate's actual skills and independent thinking ability.
How are interview processes changing at your company — or at places where you're interviewing — to adapt to this new reality?
Are there any new patterns, tools, or formats you're using to ensure a fair evaluation?
I have a standard set of behavioral questions.
Just ask them how photons lose energy due to inflation and where that energy goes.
The complexity in most development is managing business complexity and a large code base. You’re not going to be able to suss that out by coding interview.
System design based on their actual experience and behavioral questions are better anyway.
Besides if your coding interview can be passed by using an LLM and your day to day coding can’t, by definition your interview isn’t an accurate assessment of whether they can do the job.
One suggestion I saw recently in a thread was asking deliberately incorrect questions, such as how to implement a particular solution using an irrelevant technology. LLMs are so 'eager to please' they seem to just BS some nonsense in response. Not sure how I feel about that approach however.
Both system design / verbal discussions and programming rounds are critical to the interview process.
2. Do not ask basic software literacy questions. First of all, this was completely stupid even before LLMs. Secondly, its easy to cheat. If you absolutely have to do this then do it terms of measures. Most people in software are entirely incapable of measuring anything and LLMs cannot fix their personality deficiency.
3. Ask all questions where the expected answer is a not some factoid nonsense but a decision they must make. Evaluate their answer on the grounds of risk, coverage, delivery, and performance. For example if you are interviewing a AI/ML guy ask them about how they overcome bias in the algorithms and how they weigh the consequences of different design outcomes. If they are a QA ask them about how they will take ownership of quality analysis for work already in production or how they will coach developers when communicating steps to reproduce a defect.
4. As an interviewer you should know, by now, how to listen to people. That is so much more than just audible parsing of words. If their words say one thing, but their body language says something different then they are full of shit. Its okay that they aren't experts in everything. Their honesty and humility is far more important. They can get every question wrong, but if their honesty is on and they can make solid decisions then they are at least in the top half of consideration. 5. Finally, after evaluating their decision making ability and risk analysis then ask them for a story where they have encountered such a problem in the past and had to learn from failure.
This question comes up at least once a month so this answer is copy/paste from a prior comment.