20 Interview Questions From Real AI Engineer Interviews

Kunal S | 15 Aug 2026

Most interview prep is theory — how RAG works, what fine-tuning does. That's not what gets asked once you're past a few years in. What gets asked is a broken production situation, and the interviewer watching whether you reach for the obvious fix or the right one.

Here are 20 of those, with what a good answer actually covers. Read the question, then check yourself against the expected answer before you scroll.

Retrieval & RAG

1. Your RAG system has 92% recall@5 over a 40,000-doc wiki. Someone asks "what's our current deployment process" and it confidently returns an answer from three versions ago. Recall looks fine. What are you actually fixing?
What's expected: Not retrieval tuning. High recall just means a relevant chunk came back, not the current one. You need staleness handled as its own signal — rank by recency, catch when two retrieved chunks disagree, and say "these sources conflict" instead of quietly picking one.

2. Same RAG chatbot, six languages. English is 90% accurate. French and German are stuck at 60%. But recall@k is basically the same across all of them. Where do you look first?
What's expected: Not the embedding model — that's the easy guess and usually wrong. If retrieval is equally good but answers aren't, the problem is downstream: chunking that breaks mid-sentence in other languages, a prompt that was only ever tested in English, or the model reasoning worse in non-English even with the right context. Test each stage before you touch the embeddings.

3. A finance RAG bot needs to answer "what was the YoY revenue change for Company X" — the two numbers live in two separate filings, both retrieved with high confidence, both stuffed into one prompt. It gets it wrong sometimes. Why?
What's expected: Because stuffing two retrieved chunks into one prompt and hoping the model connects them isn't reliable at scale. This needs multi-hop handling — break the question into sub-queries, retrieve each number separately, or let the system explicitly reason about what it's still missing before it answers.

4. You're building legal-tech RAG. A wrong answer means a lawyer cites a case that doesn't exist. Product says "just add a disclaimer." What do you actually build?
What's expected: A disclaimer doesn't stop a bad answer, it just documents that one happened. What you need is verification — force the model to cite the exact passage behind every claim, then check programmatically that the citation actually backs the claim, and block anything it can't ground.

Agents & Tool Use

5. Your support agent can refund up to $50. Customer says "that didn't fix it," and the agent refunds another $50 for the same issue, because it treats every turn independently. Will "don't issue duplicate refunds" in the prompt fix this?
What's expected: No — and knowing why is the whole point. Prompt instructions aren't a real control for money moving. You need a system-level guardrail: a refund ledger per customer per issue, and refunds treated as one idempotent action tied to a case ID, not a fresh decision every message.

6. A three-agent pipeline — search, summarize, write — works fine on simple questions. On complex ones, the summarizer starts summarizing its own earlier summary instead of the original source, and quality drops the longer it runs. Is this a "need a bigger model" problem?
What's expected: No. This is a state bug, not a smarts bug. Everything's getting dumped into one growing context with no way to tell source material from the agent's own output. Fix it by tagging provenance explicitly, so agents always know what's ground truth and what's their own prior reasoning.

7. Leadership wants an agent that runs SQL against production data based on plain-English questions. Demo looks great. What do you focus on before this touches real data?
What's expected: Not making the SQL translation more accurate — that's the visible part everyone's excited about, but it's the smaller risk. The real work is a read-only DB role for the agent, query timeouts and row limits, and making sure it physically cannot run a DROP or UPDATE, no matter how good the query generation gets.

8. Should a coding agent be allowed to push straight to main, or only open a PR for review? Your team lead says a good enough agent should be trusted to push directly. How do you make this call?
What's expected: Not by how accurate the agent currently is — that number keeps changing and you don't want to renegotiate the boundary every model upgrade. Set it by blast radius instead: gate anything hard to undo, like a direct push to shared main, and let the agent move freely on anything cheap to reverse, like a draft PR.

Reliability, Cost & Latency

9. Finance wants a 60% cost cut this quarter with no visible quality drop, on a feature that runs every query — even "what are your business hours" — through a large frontier model. What's the move?
What's expected: Not swapping everything to a cheaper model and hoping it holds. You need tiered routing — classify queries by how hard they actually are, send the simple/high-frequency ones somewhere cheap, save the expensive model for the queries that need it, and shadow-test the quality gap before rolling it out for real.

10. P95 latency jumps from 800ms to 3.2 seconds overnight. No deploy happened. Traffic is flat. The only thing that changed is your model provider silently bumped a version behind the same API. How would you have caught this before users complained?
What's expected: By treating your model provider like any other dependency you monitor — because that's what it is now. The version bump is the prime suspect the moment nothing on your side changed. Confirm it against the provider's changelog or your own eval suite, and pin model versions going forward so this can't happen silently again.

11. You're spending $40K/month on API calls. Someone proposes self-hosting an open model to bring marginal cost to near zero. How do you evaluate that?
What's expected: Not just "is the open model as good." That's only half the question. Price in the GPU infra, the engineering time to build and maintain serving — batching, scaling, monitoring, upgrades — and the opportunity cost of that time. Self-hosting usually only wins past a certain volume. Find that breakeven number before you recommend anything.

12. Your RAG pipeline calls three services — retrieval, reranking, generation — each individually at 99.5% uptime. Every dashboard is green. Users say it feels flaky anyway. What's going on?
What's expected: Reliability compounds. Three services at 99.5% each land you around 98.5% end-to-end — worse than any single dashboard shows, especially once retries stack on top. Fix it with tracing across the full request chain, not per-service dashboards, and build a real fallback — a cached or degraded response — instead of a hard failure when one link is slow.

Safety & Compliance

13. A healthcare assistant is only supposed to summarize what a patient reported, never diagnose. It sometimes phrases a summary like a diagnosis anyway. Your teammate adds "do not diagnose" to the system prompt and calls it done. Will that hold up under audit?
What's expected: No. A prompt instruction will work most of the time and fail unpredictably, which isn't good enough in a regulated setting. You need a structural check after generation — a classifier or rule-based scan for diagnostic language before it reaches a clinician, blocking or flagging anything that crosses the line.

14. You're expanding an internal AI assistant to an EU subsidiary. It currently logs full transcripts to a US data warehouse — same as everywhere else. Legal flags it. Is this a paperwork fix?
What's expected: No — it's an architecture decision. EU data may legally need to stay in-region, which touches your logging pipeline, your vector store, anything used for fine-tuning. Find out what residency rules actually apply before you design anything, and treat regional isolation as a first-class part of the system, not something patched on after legal complains.

15. A coding assistant is about to get full read access to your codebase. A senior engineer points out there are hardcoded API keys sitting in old config files nobody's rotated. Is that a separate cleanup task?
What's expected: No — once you give a system access, you own what that access can leak. A credential sitting in context can get quoted back in an answer, logged, or pulled out through prompt injection. Either scope the assistant away from those files until they're cleaned up, or add a secrets-detection pass before content reaches the model. Treat it as blocking the launch, not a parallel task.

Model Selection & Fine-Tuning

16. A ticket-routing task currently uses an expensive frontier model. A teammate wants to fine-tune a small open model on 15,000 labeled tickets to save money. Is this just a cost decision?
What's expected: Not only. Classification into a fixed set of categories, with decent labeled data, is actually a strong fine-tuning candidate on its own merits — this isn't open-ended generation, where fine-tuning tends to disappoint. Confirm the task fits before you chase the savings, then separately think about ongoing cost: label drift as categories change, and who owns retraining when that happens.

17. You want to replace a frontier model with a small one fine-tuned on 500 "ideal" summaries your best reps wrote. Is 500 examples enough?
What's expected: You don't guess — you find out. Build a held-out eval set, fine-tune, and compare against the same rubric you use for the frontier model. And flag going in that 500 examples might nail the style and format but miss edge cases barely represented in that sample — angry customers, weird call topics — so you monitor for that gap after launch instead of assuming the eval set covers it.

18. A voice assistant needs to respond in under 300ms to feel natural. Your model alone takes 600ms. One teammate wants a smaller fine-tuned model, another wants to keep the big model and just stream the response. What do you check before picking a side?
What's expected: Whether 300ms is a hard technical limit or a feel target — because those need different fixes. Streaming can make 600ms feel instant without actually cutting total time. A smaller model actually cuts compute but risks quality. You don't know which one to build until you know what "natural" actually requires.

Evaluation & Quality

19. A new prompt scores higher on your 200-example offline eval. You ship it. Two weeks later, complaints go up instead of down. Is your eval suite broken? What's expected: Probably not — probably it doesn't match real traffic. Your 200 examples might skew toward short, clean inputs while production traffic is messier and longer. Pull actual production failures, add them to your golden set, and treat this as a sign your eval needs to evolve, not a reason to stop trusting evals altogether.

20. You pilot an LLM-as-judge to replace manual review. It agrees with human reviewers 85% of the time. Leadership wants to know: good enough to remove humans entirely? What's expected: You can't answer with just the 85% — you need to know what the 15% disagreement actually looks like. Is the judge too lenient or too harsh? Are the errors clustered in one category, like responses that push back on the user? Known failure modes — self-preference bias, position bias, drifting leniency over time — need active testing. Usually the right answer is hybrid: automate the clear cases, keep a human on the categories the judge gets wrong.

What to actually prepare

If you only have time to study one thing, study this: naming the real category of the problem before you touch the fix. Every question above rewards that same move —

State management, not prompting (agents that repeat actions, agents that lose context)
Verification, not policy (disclaimers and "don't do X" prompts don't hold under pressure)
Distribution, not a broken tool (evals that pass while users complain, retrieval that's "equal" but generation isn't)
Total cost, not sticker price (self-hosting, fine-tuning, model downgrades)
Architecture, not legal sign-off (data residency, compliance, access control)
Blast radius, not current accuracy (how much autonomy a system gets)

Practice saying your answer out loud, not just thinking it. Catch yourself before you fall into the obvious fix — naming the trap is itself a signal to the interviewer. And always end with how you'd check the fix actually worked: a metric, an eval, something you'd watch in production. That last part is what most people skip, and it's usually the difference between a fine answer and a hire.