Stephen Cheng
As an AI engineer, we build and deploy AI systems in production environments. Then they turn models like LLMs into reliable applications that deliver business value. In this post, we’ll cover interview questions on LLMs, core AI concepts, system design, deployment and MLOps, coding and implementation, and real-world scenarios for AI engineer interviews.
An AI model is any system that performs tasks that predominantly require human intelligence, including some rule-based or symbolic approaches. A machine learning model, on the other hand, is a subset that learns patterns directly from the fed data. In production terms, AI engineers most often work with ML models, especially LLMs, but must also integrate some non-ML components, such as rule engines or knowledge graphs, when needed for a highly reliable output.
A large language model is, as the name suggests, a large, transformer-based neural network that is trained on massive text corpora to generate coherent text. In production systems, LLMs serve as the reasoning engine behind the chatbots, summarizers, and agents.
Tokenization converts raw text into numerical tokens that the model can process. It directly affects the cost, the context length, and how the model sees text fundamentally. In other words, text segments are assigned numerical identifiers, called tokens, so the text is ready to be fed to the model. Production systems use subword tokenizers like BPE and monitor the token counts to control the API expenses and latency.
A prompt is the input text that is fed to the LLM to produce some output. It is generally an engineered artifact that contains instructions, sometimes examples, and a full context to maximize the adaptability of the LLM.
Prompting changes the behavior at inference time. However, fine-tuning updates the model weights, which is slower but generally leads to more customized large language models. This makes prompting faster and cheaper for most production use-cases.
RAG retrieves the relevant documents from a knowledge base that you assign and adds them to the prompt. The LLM therefore uses that extra verified data to predict more reliable information and reduce hallucinations.
Embeddings are dense vector representations that understand semantic meanings. They help with semantic search and retrieval in vector databases. You can think of an embedding as a table that takes the numerical input and assigns it a high-dimensional vector in a different, more representative space.
Combine automated metrics, like faithfulness and relevance, with human review and business KPIs. In production, we basically require a continuous evaluation pipeline.
Gaps in the training data and the overconfident nature of next-token prediction can force the model to predict tokens without sufficient grounding, which is known as hallucination.
RAG is one of the most reliable and efficient methods, since it delivers grounding against trusted sources you provide. Combining it with structured output formats, self-consistency checks, and fact-checking further reduces hallucinations.
They process in parallel using self-attention instead of a recurrence. Modern large language models are decoder-only and predict the tokens autoregressively.
Attention computes weighted relevance between the tokens to allow for long-range dependencies regardless of their position.
The maximum number of tokens the model can process in one inference call. It limits prompt design and conversation history management.
It controls the randomness in sampling: lower values produce more deterministic output, while higher values increase creativity. In production systems, it’s tuned per use case to balance reliability and variety.
The systematic design, testing, verification, tuning, and iteration of prompts to achieve the best reliable behavior. Prompts are versioned with code in production and depend heavily on the model itself rather than a generic template.
Frontend, conversation memory, prompt orchestration, LLM API call, output validation, logging, rate limiting, and cost tracking.
Apply prompt compression, caching, smaller or distilled models, speculative decoding, and hardware acceleration.
Log requests/responses, run automated quality scores, sample for human review, track business metrics, and set degradation alerts.
Containerize with FastAPI, integrate via SDKs or self-hosted servers like vLLM, orchestrate with Kubernetes or serverless, and use CI/CD for prompts and code.
Docker/Kubernetes, vLLM or Text Generation Inference, vector databases like Pinecone or Weaviate, LangSmith, and cloud platforms like AWS Bedrock, Azure OpenAI, or GCP Vertex AI.
The degradation is caused by some changes in input distribution, the user behavior, or the data sources. It appears as a declining relevance or rising hallucinations.
Apply continuous batching, quantization like 4/8-bit, model parallelism, and engines such as vLLM or TGI.
AI, Interview, Machine Learning — Jul 28, 2026
Made with ❤️ and ☀️ on Earth.