Hacker News

The same way we did before. Very very simple code submission (most people using AI use it even though we call out we're going to ask them later to modify later without AI tools writing code for them) then pairing interview where we ask some basic "are you actually at the level you say you are" question, then ask them to extend their program submission with:

- engineers on the call as pairing assistants - google, ai tools, whatever for libraries, syntax, etc. we tell the candidate directly that it's impossible for us to gsther signal on how they think about problems if they ask Claude to just whip them up a solution - the expected output - their own unit test suite

Every candidate that has submitted an ai submission thus far has failed because they have literally no idea where to go. I've interviewed dozens at this point. I'm not saying "they're unfamiliar with the structure", I'm saying "they cannot actually break down the problem even verbally".

It doesn't matter their pedigree or past experience on their resume, if they used AI to generate they don't seem to be able to resurrect the skills that actually matter for the thing, engineering and product work.

Note because I know folks hate code submissions. It's not hard. We give a CSV with 3 columns, 10 lines. Do some basic mapping and some structuring, some basic data modeling. We only expect about 1 actual class or struct. Then unit tests and it should run in the terminal. Max submission length with verbosity has been a java program at something like a hundred lines total if that, most folks complete the submission in an hour or two. Extension is that we modify one of the rules and extend the CSV by 5 lines.

I seem to still be getting good signal from this, since the engineers that I've hired off of this have been fantastic with or without AI tooling immediately in their hands during the day.

We were hiring for senior devs java/spring boot so I coded a tiny "Todo app" rest API, basically a single crud controller, and filled the code with many mistakes. Some obvious, some less obvious. Missing authentication, no tests, logging with System.out etc.. then we ask the candidate to review the code (we provide a laptop with a few IDEs but no AI tools). When they point out errors, I get them to explain why they are errors by playing the junior engineer to see how they explain technical issues to someone that might not have the same understanding I am not looking for them to find all issues but if they can't find at least some it's a red flag. It takes about 20-25 minutes and gives really good insights. The other 30 minutes we asked some questions about past projects and some technical questions

So far it provided good signals

To me AI hasn't had any impact on interviewing, only on filtering the CVs as I immediately flag obviously LLM-generated ones that don't have any substance, while those that are LLM-assisted with tasteful judgement are fine.

On the interview side, I have gone through a phase which required do an architectural design task on a whiteboard. I realized how stressful and not very helpful at evaluating it might be when I myself had to sit on the interviewee's side several times. It's difficult to get into this play-design mode.

Currently, I just make natural conversations, ask some technical questions on the level where I estimate the candidate is, try to understand the level of their knowledge and whether I would like to work with them in one team as a person. Maybe it wouldn't work for highly specialized roles, but for a general backend work I have been happy with the results.

Leetcode doesn’t tell you anything apart from someone memorised set of questions and their solution.

For your question, I assume you want people who can solve problems, can explain their thought process and reasons of decisions. I did interview devs recently, and my questions were related to their experience in their CV. Something like, tell me about project X in Y company, what did you do, what did you use, why, what did you learn from it.

My personal opinion is that LLMs are no difference from developers who over engineer and over complicate things. You can get them to do good work with the right steering and right understanding of the whole system within the context of the company.

Prompts, prompts, prompts.

Give them a time-constrained challenging problem that is wide in scope and see how well a candidate can decompose it before feeding it to AI. You can get pretty high signal within the first 2 prompts. Are they able to effectively steer the model or do they just ride with the flow and accept every AI suggestion?

How well do they know their models and their limitations? Are they able to switch tools/models effectively on the fly depending on the task? Or are they just using cursor auto mode and copy-pasting the task description into their IDE? Do they have a custom harness/workflow? What skills are they using, if any?

Judge them on quality of the output first. And pay attention to their taste.

Measure them on merit, their body of work and how well they will fit into the team culture.

As for the questions you’re asking, instead of asking old-school coding interview questions ask them stuff that is actually relevant to agentic programming.

Ask them to build a harness, have them explain what skills are, have them build a self-made implementation of Claude Code as take-home assignment.

The times have changed, you need to update interviewing questions to meet them. Start with agentic questions and ramp them up into more complex scenarios. This is how you gauge your candidates, see how they think and solve their way out of it.

Try to determine how creative, adaptable, motivated the candidate is. Figure out what there actual skill-sets are that they can bring to you, not whether they know some CS pedantry

So I actually had a take home challenge for an interview a few months ago, which didn't ask for any code at all. Instead they gave us a prompt for something to build (a tool that could break down an image into it's component colours), and we basically built whatever we wanted.

Honestly one of the most fun interviews I've ever done - it involved really understanding the problem, and building an interesting solution. AI was totally allowed and expected, but you had to prove you understood the problem and what you'd made in the interview. I felt it was actually way better than traditional interviews, because it gave a chance to show the actual quality and thought of my work rather than a glorified logic puzzle.

The org I am at we have build an internal harness for coding, and we only hire devs that do work "AI-first". We did start hiring juniors as well as seniors; the process for either of them is the same. We drop them into any of our repos without any prior knowledge of what the project is about; and ask them to interact with the harness to build a larger feature that has been specified in project management. For the task, they can only use AI; there is no room for any manual coding at this particular org. We grade them more or less on their interaction and problem solving patterns, as well as asking us questions; and how they write prompts. The coding interview last 2 hours, and there are only 3 rounds of interviews. Before the coding interview, we do send out a brief of what it will entail.

Maybe this sounds a little strange, but this has worked exceptionally well, as we have now several juniors working with us, that do still get checked by seniors. The harness itself which holds several levels of standards, rules as well as quality gates, allows these juniors to ship out a huge amount of high quality code. We still reject about 80% of the people who interview with us, because even though they know how to code with AI, their AI interaction patterns just aren't on par with what we need. I really started to like this process. Also to clarify, the harness isn't some Coding agent setup, but rather a whole setup that works well with Codex, Claude or Cursor; its well maintained versioned and constantly improved upon, with a huge amount of automations, rules and hooks. The moment someone starts working on anything at all, this is immediately connected and tracked with project management.

Ive been trialing different types of interviews for over 2 years now and here’s what I’ve found.

I tried 2 flavors of a technical interview and this round was 1 hour in total.

The first was a traditional interview problem where we give the candidate a codebase and a docker image to run against. The problem itself involves using the docker image to post and get responses from but the candidate is expected to write a new helper/service in any language they choose. We ask for no agentic workflows for this but any other resource is fair game. This has been the most successful for us to find candidates and generally provides the best experience.

Our full agentic interview involves an existing fullstack typescript codebase that we give to the candidate a week or two before. The candidate is free to use any agentic workflows they want during the interview and we give them a ticket that we went to implement during the interview. I find a ton of variability in this interview and a lot of people will just take the entire ticket and one shot into Claude without prepping. This is lead to the biggest disparity in results of candidates so we stopped doing it.

it feels surreal to see that people still do interviews, when I've been applying for 8 months, not even getting to an interview.

Ask them how many "R"s are in the word "strawberry"

I am very curious about this as well, as an IC. I have not written code by hand for almost two years now and have produced a mountain of Python code over the past year (well architected, using all of my experience in other languages to support good judgement). However, Python is new to me -- I've never hand written a line.

This is all fine and good at my current job, but were I to ever go back on the market. What an uncomfortable conversation!

How does one approach this weird state we're in?

Same way as I did before.

Are they interesting to talk to? Are they passionate, curious, authentic, opinionated? Do they have a product mindset? Do they need to be told what to do? Would you hang out with them after work? Can they explain themselves in a way that’s easy to understand?

In my experience these are much better predictors of success than leetcode.

how many applicants are you getting? this plays a big role.. how many submissions do you have to sift through will dictate what ai agents to use. if you value the job: multiple of interview rounds are needed. Then its up to your hiring crew. Take this time to create your system of hiring, Of course a simpler approach is to ask what code editing software they use and hire the person who says, " I don't use a code editor. I use terminal!" hire those people.

Basically the same as always--an extended interactive Turing test. Can they speak coherently and in-depth about the things they claim to have done on their CV/resume?

i think it would make more sense to let developers whatever coding agent they want to use to build something complex and then have them explain every bits and pieces of what was built, why, alternatives, design, scalability considerations etc.. this would result in much better technical discussion instead of having someone code manually. every developer needs to use a coding assistant today to build what they need to with the best quality in short time. so interviews need to be way to see if the developer can do that in real work.

Recently, I was screen-interviewed for coding speed with a fairly easy project. The company used a 3rd party platform which gave me 90m to complete 4 levels, each one building upon the other.

I think that was a nice test, but the platform is too buggy to recommend.

Conventional technical interview (DSA, whiteboard architecture).

Then 1-2 day paid work trial.

Interestingly…no one has said no to the trial.

It turns out that people want to derisk their decision just as much, if not more.

[flagged]

[dead]

[dead]

[flagged]