What is the difference between DPO and RLHF with PPO in GenAI interviews?

RLHF requires a separate reward model and an online reinforcement learning loop (PPO), which is computationally expensive and unstable. DPO (Direct Preference Optimization) mathematically derives an implicit reward from the policy itself, optimizing policy weights directly on preference pairs using a stable, closed-form cross-entropy loss.

Why is vLLM with PagedAttention preferred for LLM serving?

Traditional LLM serving suffers from massive GPU memory fragmentation in the Key-Value (KV) cache, limiting batch size. PagedAttention divides the KV cache into non-contiguous virtual memory blocks (like OS paging), virtually eliminating fragmentation (reducing waste to <4%) and boosting throughput by 2x to 4x.

What is LoRA and how do you choose the rank r and alpha?

LoRA decomposes weight updates into low-rank matrices W = W0 + (alpha/r) * (B x A). Typical values for rank r are 8, 16, or 32 depending on task complexity. Setting alpha to 2x the rank r is standard practice. LoRA reduces trainable parameters by over 99% while matching full fine-tuning performance.

Generative AI & LLM Engineer Interview Guide 2026: RAG, Fine-Tuning & Architecture

Q: How do you mitigate hallucinations in production RAG systems?

Mitigation requires a multi-stage approach: (1) Hybrid search combining dense embeddings with sparse BM25; (2) Cross-Encoder re-ranking to filter irrelevant chunks; (3) Contextual Retrieval where each chunk is prepended with explanatory context; and (4) Strict grounding prompts with automated faithfulness metrics (Ragas/TruLens).

Q: What is LoRA and how do you choose the rank r and alpha?

LoRA decomposes weight updates into low-rank matrices W = W0 + (alpha/r) * (B x A). Typical values for rank r are 8, 16, or 32 depending on task complexity. Setting alpha to 2x the rank r is standard practice. LoRA reduces trainable parameters by over 99% while matching full fine-tuning performance.

1. Transformer Internals: Attention, RoPE & KV Caching

Interviewers at OpenAI, Anthropic, Google, and Meta probe deep mechanical understanding of the transformer decoder architecture.

Scaled Dot-Product Attention Equation:

Attention(Q, K, V) = softmax( (Q * K^T) / sqrt(d_k) ) * V

Why divide by sqrt(d_k)? For large projection dimensions, dot products grow large in magnitude, pushing the softmax function into regions with tiny gradients (gradient vanishing). Scaling by sqrt(d_k) stabilizes variance to 1.0.

What is the Key-Value (KV) Cache?
During autoregressive generation, each token attends to all prior tokens. Recomputing Keys and Values for historical tokens at every generation step scales O(N^2). KV Caching stores previous K and V vectors in GPU VRAM, reducing per-token generation complexity to O(N).

2. Production RAG: Chunking, Hybrid Search & Re-Ranking

Basic vector search fails in production due to embedding loss and semantic drift. Production RAG interviews require discussing the multi-tier retrieval pipeline:

1. Intelligent Document Chunking

Fixed-size chunking with 15% overlap vs Semantic Chunking (split by embedding distance threshold) and Hierarchical Parent-Child chunking (search small chunks, feed full parent context).

2. Hybrid Search (Reciprocal Rank Fusion)

Combines Dense Vector Search (semantic similarity via cosine distance) with Sparse BM25 Keyword Search (exact product IDs, error codes, names) using RRF scoring: Score = sum( 1 / (60 + rank_i) ).

3. Cross-Encoder Re-Ranking

Bi-encoders compute vector embeddings separately. Cross-encoders (like Cohere ReRank or BGE-Reranker) feed [Query, Document] jointly through attention layers for high-precision top-K filtering.

3. Fine-Tuning & Alignment: LoRA, QLoRA, DPO vs RLHF

When should you fine-tune vs RAG? Rule of thumb: Use RAG for external dynamic knowledge retrieval; use Fine-Tuning to teach specific styles, syntax, structure, or tone.

Method	Mechanism	VRAM Requirement	Use Case
Full Fine-Tuning	Updates 100% of base model weights	4x - 6x model size (High)	Foundation domain adaptation
LoRA (PEFT)	Freezes base weights, trains low-rank matrices (A * B)	< 1% of base weights (Low)	Task customization & style tuning
QLoRA	Quantizes base model to NF4 + LoRA adapters	Runs 70B models on single GPU	Consumer GPU fine-tuning
DPO (Direct Preference)	Aligns model directly on chosen/rejected pairs without reward model	Standard fine-tuning VRAM	Safety & helpfulness alignment

4. Inference Optimization: vLLM, PagedAttention & Quantization

In high-throughput AI engineering interviews, serving economics and millisecond latency matter.

PagedAttention (vLLM): Solves GPU memory fragmentation by treating KV cache memory like virtual memory pages in an OS, allowing non-contiguous allocation and boosting serving throughput by 2x–4x.
Continuous (Iteration-Level) Batching: Dynamically adds incoming requests to the batch at each token iteration rather than waiting for entire sequence completions.
Quantization Formats: AWQ (Activation-aware Weight Quantization), GPTQ, and GGUF allow 4-bit weight compression with minimal perplexity degradation.

5. GenAI System Design: Real-World Interview Architectures

Designing enterprise GenAI platforms requires orchestrating guardrails, semantic caching, and LLM routers:

Semantic Caching (GPTCache / Redis)

Avoid re-querying expensive LLMs for semantically identical prompts. Embed incoming queries and check cosine similarity against cached responses (threshold > 0.92) to cut API costs by 40–60%.

Guardrails & Content Moderation Pipelines

Implement asynchronous pre-prompt validation (NeMo Guardrails, Llama-Guard) to catch prompt injection and jailbreak attempts before tokens reach the main inference engine.

Practice Generative AI & ML Mock Interviews

Test your architecture instincts with Sophia AI. Face real-time voice questions on transformers, RAG design, fine-tuning loss functions, and inference latency trade-offs.

GenAI & ML Mock Interview LLM System Design Mock Agentic AI & MCP Mock

6. Frequently Asked GenAI Interview Questions

What is the difference between DPO (Direct Preference Optimization) and RLHF with PPO?

Traditional RLHF requires training a separate Reward Model, then using Proximal Policy Optimization (PPO) reinforcement learning to optimize the policy model against the reward model. DPO mathematically reformulates the objective, showing that policy weights can be optimized directly on chosen vs rejected preference pairs using a closed-form loss function, eliminating the unstable RL training loop and separate reward model.

How do you mitigate hallucinations in production RAG systems?

1) Implement Self-RAG / Corrective RAG (evaluating retrieved document relevance before generation). 2) Enforce strict system prompt grounding with citation requirements. 3) Use Cross-Encoder re-ranking to filter noisy retrieval chunks. 4) Use confidence thresholding and automated evaluation pipelines (Ragas faithfulness metric).

Foundational Research Papers & System References

The technical architectures and mathematical derivations in this guide reflect foundational research papers and production serving frameworks:

•Attention Is All You Need (Vaswani et al., 2017): Scaled dot-product attention and multi-head self-attention foundations (arXiv:1706.03762).
•LoRA: Low-Rank Adaptation of Large Language Models (Hu et al., 2021): Low-rank decomposition for parameter-efficient fine-tuning (arXiv:2106.09685).
•Direct Preference Optimization (Rafailov et al., 2023): Closed-form implicit reward modeling without PPO reinforcement loops (arXiv:2305.18290).
•PagedAttention & vLLM (Kwon et al., 2023): High-throughput LLM serving via virtual memory paging (arXiv:2309.06180).
•Intervio AI Mock Interview Practice Rooms: Practice live voice questions at /practice/generative-ai-ml-mock-interview, /practice/llm-system-design-mock-interview, and /practice/ai-agent-engineer-mock-interview.

Generative AI & LLM Engineer Interview Guide 2026: RAG, Fine-Tuning & System Design

Key Takeaways — GenAI Engineer Interviews 2026