Prompt Engineering

Operaide orchestrates LLMs for tasks like summarizing content, analyzing documents, and creating database entries. Prompt engineering determines the quality of the output.

What is Prompt Engineering

Prompt engineering is the practice of structuring input to an LLM to get a specific output. At its simplest, you tell the LLM what the task is and what form the answer should take. The iterative part is testing different phrasings to balance output quality and length.

The Importance of Prompt Engineering

Imagine using ChatGPT. Given the chat feature, you want to get information on the renown novel A Hitchhiker's Guide to the Galaxy

Prompt 1: Tell me about A Hitchhiker's Guide to the Galaxy.

Prompt 2: Summarize 'A Hitchhiker's Guide to the Galaxy' by Douglas Adams. Tell me about what type of book this is, what the plot is, and what impact it had on pop culture. Keep the summary as short as possible, providing only the most important information.

These are the two distinct output generated by ChatGPT:

For Prompt 1

The Hitchhiker's Guide to the Galaxy is a beloved comedy science fiction series created by Douglas Adams. It began as a radio series in 1978 and has since been adapted into novels, a television series, a movie, and a stage show. The Story: The story follows Arthur Dent, an ordinary Englishman whose life is turned upside down when the Earth is destroyed to make way for a hyperspace bypass. Arthur is saved by his friend Ford Prefect, a researcher for the titular "Hitchhiker's Guide to the Galaxy." Together, they embark on a wild and wacky journey through the galaxy, encountering bizarre alien species, philosophical robots, and a towel that is considered the most useful item in the universe. Why People Love It:

• Humor: The series is known for its witty humor and irreverent tone.
• Philosophy: It explores deep philosophical questions about life, the universe, and everything, often with a humorous twist. ...

For Prompt 2

A Hitchhiker's Guide to the Galaxy is a humorous science fiction comedy series by Douglas Adams. The plot follows Arthur Dent, a hapless Englishman who finds himself on an intergalactic adventure after the Earth is destroyed to make way for a hyperspace bypass. He is joined by his friend Ford Prefect, a researcher for the titular guidebook. Together, they encounter a variety of strange creatures and planets, all while trying to survive the chaotic and often absurd nature of the universe.

This series has had a significant impact on pop culture, influencing countless other works of science fiction and comedy. Its humor, which often relies on absurdity and satire, has resonated with readers of all ages. Iconic lines like "Don't Panic" and the concept of the towel as the most useful item in the universe have become part of popular culture.

As you can see, the second prompt offered a more concise answer, providing only the most valuable information, whereas the first prompt had to be cut in length because it exceeded what anyone would be willing to read for a quick information.

This example applies to customer data as well. When analyzing a document, the user wants only the most necessary information. When summarizing a transcript from a meeting, the LLM has to know what is the desired output of the user.

Prompting Techniques

The following techniques are used in Operaide to define LLM tasks for Agents and Skills.

Zero-Shot Prompting

The Zero-Shot Prompting (ZSP) approach for prompt engineering of LLMs works by performing tasks without any prior training on the task. This is done by providing the LLM with a prompt that describes the task in natural language. Zero-Shot Prompting is considered to have flaws, since the LLM is not optimized for domain-specific knowledge or may have bias, hence needs instruction tuning to produce desired outputs more correctly. Zero-Shot Prompting in itself is a quite powerful technique if the LLM is trained properly, and therefore offers a use case for users who want to use it for general tasks without having to invest the time and resources to train the LLM on a dataset for each task.

WHAT: ZSP is the simplest form of prompting, in which a problem is described in simple language to which the LLM responds.

WHY: ZSP is suitable for simple questions where the topic cannot be obtained through overly complex requirements.

HOW: Simple, natural language.

EXAMPLE: Create a simple, nice response to a customer that you could not find any data for the request.

Few-Shot Prompting

This type of prompt engineering, also called in-context learning, makes use of examples provided to the LLM that will create some context for generating content. By providing a set of examples on what a desired output would look like, the LLM can be pre-trained to learn new tasks. Few-Shot-Prompting offers a significant reduction for task-specific data to be trained on and it is less likely to learn an overly narrow distribution from the dataset. To sum it up, this method learns on a set of examples to rapidly adapt to new tasks.

WHAT: The LLM receives some examples as context for task fulfillment.

WHY: Faster adaptation to new tasks with less training data.

HOW: Provide examples before the actual task.

EXAMPLE: You are given a set of documents. For each of these documents, create a short summary. For example: - Summary: what the document is about. - Data: what important data you could find. ...

Chain-of-Thought Prompting

In this method, chains of thought demonstrations are presented to the LLM as examples for generating a desired output. A chain of thought refers to a set of reasoning steps that result in an output, helping the LLM to understand concepts that will then be applied to generate an answer. These reasoning steps are generally in the form of step-by-step answer examples, resulting in complex decision capabilities. Chain-of-Thought learning helps the LLM to come up with chains of thoughts by putting 'Let us think step by step' before the output that is to be generated. This way the LLM achieves context on how 'Let's think step by step' works and in what way it will apply it to the generated output. Chain-of-Thought prompting improves the ability of an LLM to generate content using reasoning methods.

WHAT: The LLM is provided with examples of step-by-step thinking (chain-of-thought) to solve complex problems.

WHY: Improves the LLM's ability to think logically and solve complex problems.

HOW: Presentation of step-by-step solution paths as examples.

EXAMPLE: Here is a document in markdown format. Create a table of contents for this document. Let's start by looking for the subtitle sections. Next, look for any ...

Summary

Well-structured prompts determine whether an LLM produces a usable answer. Test different techniques against your use case and pick the one that gives the most consistent output.

Another excellent resource is the Prompt Engineering Guide, which provides many more techniques.