Stephen Cheng
Prompt engineering is the practice of designing and iterating on inputs to language models to get reliable, high-quality outputs. It involves structuring instructions, examples, and context in ways that shape model behavior without touching the underlying model weights. In practice, it spans everything from writing a single clear instruction to designing a full system prompt with persona, constraints, output format requirements, and examples. A good prompt is specific about the task, clear about expected output format, and doesn’t leave the model to fill in assumptions you haven’t made explicit. It includes the right amount of context: enough to ground the response, not so much that it introduces noise.
Temperature and sampling parameters introduce randomness, so output varies across runs even with an identical prompt. Beyond that, long prompts can create attention dilution, where earlier instructions get less weight than later ones. Model updates can silently shift behavior. This one bites teams in production more than they expect. And prompt sensitivity is real: a single word change can meaningfully alter output distribution. If consistency matters, lower the temperature and specify output format explicitly.
The most common: ambiguous instructions the model resolves in an unexpected direction; missing context that forces assumptions; format not specified, so the model defaults to prose when you wanted JSON; conflicting instructions in system and user turns. Not all bad outputs are prompt problems. Sometimes it’s a model limitation, and no amount of rephrasing will fix it.
Long prompts are a context management problem before they’re a prompt engineering problem. Audit what’s actually in there. System prompts accumulate redundant instructions over time and nobody notices. Order content so the highest-priority instructions appear where the model attends most strongly (beginning and end). Use summarization for conversation history rather than appending every prior turn verbatim. And measure: if adding more context is degrading output quality, you’ve likely hit the model’s effective context limit regardless of the technical window size.
Break the task into sub-tasks that can each be prompted independently, with outputs from one feeding the next. This is usually better than a single prompt that tries to do everything. Complex single prompts are harder to debug because you can’t tell which part went wrong. Let failure likelihood drive the decomposition: where are the riskiest steps, and how expensive is it to recover from a mistake there? Parallel decomposition works for tasks without sequential dependencies.
Be explicit about the expected reasoning style: “Before using any tool, state your plan. After each tool call, assess whether the result moves you closer to the goal before proceeding.” This makes the agent’s reasoning legible in the trace, which is essential for debugging. For complex tasks, decompose into explicitly named phases. Vague instructions like “complete the task” leave too much room for the agent to take unexpected paths. And it will.
AI, Prompt Engineering — Apr 2, 2026
Made with ❤️ and ☀️ on Earth.