Demystifying LLM prompts: From crafting inputs to engineering intelligent systems

The advent of Large Language Models (LLMs) has revolutionized how we approach application development, introducing new paradigms for human-computer interaction. Central to leveraging these powerful models are the concepts of Prompt crafting/designing and Prompt engineering. While often used interchangeably in casual discourse, a closer examination of their roles, scopes, and goals, the context of contemporary LLM history shows distinct yet interconnected disciplines. This article aims to delineate these differences and highlight their synergistic relationship in building effective LLM-powered applications.

1. Primary Goal

Prompt Crafting/Designing: The primary goal of prompt crafting or designing is to create a specific, well-structured, and effective textual input (the prompt) that elicits the most accurate, relevant, and desired immediate response from an LLM for a particular interaction. It focuses on the clarity and efficacy of the individual prompt to guide the model's completion. In simplest form, it's about crafting the prompt so that its completion contains the information required to address the problem at hand.

Prompt Engineering: Prompt Engineering has a broader primary goal: to design, build, and optimize entire LLM-based applications or systems that reliably and effectively solve user problems or achieve specific objectives. It aims to unlock the true potential of LLMs by structuring the overall communication between the user, the application, and the LLM. Prompt engineering helps improve discover capabilities, reliability, reduce failure cases, and save on computing costs when building with LLMs. It is a key part of the "essential tech stack" for adapting LLMs to specific use cases and achieving accuracy and reliability.

2. Scope of Activities

Prompt Crafting/Designing: The scope of prompt crafting is centered on the composition of the prompt itself. This includes:

• Choosing the right words and phrasing.

• Structuring the prompt with clear instructions, examples (few-shot learning), and context.

• Applying techniques like chain-of-thought within the prompt to guide the LLM's reasoning for that specific query.

• Formatting the prompt to be easily understood by the LLM (e.g., using Markdown).

• Making the prompt resemble patterns seen in the LLM's training data.

Prompt Engineering: Prompt Engineering encompasses a much wider range of activities, often involving the entire lifecycle of an LLM application. This includes:

• Prompt crafting/designing: This is a core skill within prompt engineering.

• Context retrieval and management: Gathering, triaging, prioritizing, and incorporating dynamic context (e.g., user data, external documents via Retrieval-Augmented Generation - RAG).

• Application logic: Designing the flow of interaction, including state management for multi-turn conversations and programmatic construction of prompts.

• Tool integration: Enabling LLMs to use external tools and APIs to access real-time information or perform actions.

• Workflow design: Breaking down complex tasks into smaller, manageable sub-tasks that can be orchestrated, potentially involving multiple LLM calls or even different models or by implementing BoM (Bucket-of-Models) strategy and LLM Orchestration.

• Evaluation and optimization: Systematically testing and refining prompts, models, and the overall application to improve performance, reliability, and safety.

• Safety and alignment: Implementing measures to ensure outputs are helpful, honest, and harmless, and to defend against misuse like prompt hacking.

3. Primary Output

Prompt Crafting/Designing: The primary output of prompt crafting is a well-formulated prompt (a text string or structured input like ChatML) ready to be sent to an LLM for a single turn of interaction.

Prompt Engineering: The primary output of prompt engineering is a functioning LLM-based application, system, or workflow that can handle a variety of inputs and achieve a defined goal over potentially many interactions.

4. Methodology

Prompt Crafting/Designing: The methodology involves the direct manipulation and refinement of text. It relies on understanding how LLMs process language, the importance of clarity, the use of explicit instructions, and the strategic placement of context and examples. Techniques include:

• Zero-shot prompting (asking directly).

• Few-shot prompting (providing examples).

• Chain-of-thought prompting (instructing the model to "think step-by-step" within the prompt).

• Using specific formatting (e.g., Markdown, XML, YAML) to structure the prompt.

Prompt Engineering: The methodology is systemic and often iterative, involving:

• System design: Architecting the flow of information between the user, application logic, external data sources, tools, and the LLM.

• Programmatic prompt construction: Dynamically assembling prompts based on user input, retrieved context, and application state.

• Context management: Implementing strategies like RAG, managing conversation history, and deciding what information is relevant.

• Tool use and orchestration: Defining tools, enabling the LLM to call them, and integrating their outputs back into the conversation/workflow.

• Multi-level sophistication: This can range from simple prompt wrapping to complex stateful interactions with tools and agency.

• Evaluation frameworks: Employing offline and online evaluation techniques, including A/B testing and LLM-as-judge.

5. Analogy

Prompt crafting/designing: Prompt crafting could be likened to writing a specific, clear, and impactful line of dialogue or a stage direction for an actor (the LLM) in a single scene. It's about the immediate instruction.

Prompt engineering: Playwriting analogy is given by the world's top 1% Researchers. The prompt engineer is the "lead playwright and the showrunner," responsible for the overall structure of the prompt (the script), the characters (user, assistant, system, tool), their interactions, and how external elements (like tool outputs or retrieved documents) are woven into the narrative to achieve the user's goal.

6. Relationship with Each Other

Prompt crafting/designing is a fundamental and essential skill or component within the broader discipline of Prompt Engineering.

• Prompt engineering from its simplest form expands to a much larger picture of prompt engineering that involves moves well beyond a single prompt and discuss the entire LLM-based applications.

• Designing effective prompts is one of the learning outcomes under the umbrella of prompt engineering.

• Good prompting (which aligns with crafting/designing) as a technique to increase LLM performance, alongside RAG and fine-tuning, all under the broader strategy of developing LLM applications.

Outcome: One cannot be an effective Prompt Engineer without being skilled in Prompt Crafting/Designing.

The image was generated using Imagen 3 using eight keywords without any prompt.

LinkedIn