Prompt Engineering — Designing Instructions That Reproduce— Structuring, few-shot, chain-of-thought, and using evaluation to keep them from breaking

01What a Prompt Is — Defining "the Instruction" Precisely

A prompt (= the instruction text you hand to an AI) is the input text you give to a large language model (= an AI trained on vast amounts of text). Everything from a single line — "summarize this," "give me three headline options under the following conditions" — to a paragraph of several hundred words specifying role, premises, constraints, and output format, is a prompt.

Here is the first fact to grasp. The AI is not computing "what is correct"; it is computing "how plausible the next word is." That is why a slight difference in instruction moves the output. With the same intent, a vague phrasing returns a vague answer and a concrete phrasing returns a concrete one. Think of prompt engineering as turning this property to your advantage — steering the output toward what you want.

Translated to the pharma workplace, this touches every piece of daily writing work: drafting promotional materials, summarizing case reports, interpreting regulatory documents, drafting internal FAQs. When the way of asking is personal and undocumented, quality wobbles every time staff change. Put the way of asking into words as a design, and anyone on the team can run it and get a consistent level of quality.

02Structuring the Instruction — Four Parts That Cut Ambiguity

A good prompt is not an off-the-cuff sentence; it is a combination of parts. The first thing that works in practice is to write these four explicitly and separately.

Simply writing these four separately raises the stability of the output visibly. In pharma especially, forgetting to hand over the context is the single largest source of accidents. To keep it from writing an unapproved indication, you must hand over the frame — "these are the only indications you may use" — as context, every single time you ask. Withhold the frame, and the AI freely "inflates."

03Using few-shot — Show a Few Examples and Transfer the Pattern

Few-shot prompting (= showing a handful of worked examples before asking the real question) is one of the most cost-effective techniques. Instead of instructing in fine-grained words, you line up two to five "input → desired output" pairs and place the real input last. The AI reads the pattern and returns an answer in the same register.

That this works was shown at scale in the study reporting GPT-3 by Brown et al. (2020). Comparing zero-shot (= no examples at all), one-shot (just one), and few-shot (several examples), accuracy rose on many tasks as the number of examples increased. You can change behavior without retraining, just by putting examples in the prompt — that is the core of few-shot.

In practice, it earns its keep in situations like these.

• When you want to standardize the format of the output — the tone of headings, sentence endings, a feel for length: showing an example conveys these more accurately than explaining them in words.
• When you want to show the criterion by pattern — showing a few "this expression is OK / this one is NG" pairs conveys the boundary more sharply than words do.
• When you want to fix the handling of technical terms — showing in advance the translations or phrasings decided internally reduces variation.

Note, however, that the choice of examples matters. If the examples are biased, it imitates the bias along with everything else. Mix in a slightly exaggerated example and it returns exaggerated output; mix in an outdated expression and it returns outdated output. Because few-shot is a "lean toward what you showed" technique, a person must first confirm that the examples themselves comply with the approved content and the norms.

04Chain-of-Thought — Have It Write the Process to Raise Accuracy

Chain-of-thought prompting (CoT, = having the model write not just the answer but the reasoning along the way) is a technique proposed by Wei et al. (2022). On arithmetic and logic problems, they showed that asking "think step by step, then answer" rather than "give the answer" raises the rate of correct answers. It resembles how a human writing out the intermediate steps makes fewer calculation errors.

Relatedly, Kojima et al. (2022) reported that simply appending the phrase "let's think step by step" can, in some cases, improve reasoning quality without any examples. Understand CoT as a method that, rather than forcing a complex judgment out in one breath, prevents leaps by having the grounds put into words along the way.

In pharma practice, CoT helps in ways like these.

• Leave a record of the grounds for a regulatory check — asking "explain, in order and against the statute, whether this expression amounts to exaggeration, then give your conclusion" records the reasoning process and makes it easier for a person to verify.
• Reduce omissions in a summary — when summarizing a long document, splitting it into stages ("first list the points, then judge their importance, then summarize") reduces the chance of dropping key points.

05Evaluating the Output — Turn "Seems Good" Into a Measurable Standard

Refine the prompt and the output improves. But "it feels better" is not usable for work. The second half of prompt engineering is building a mechanism to evaluate output objectively. Measure by a yardstick decided in advance, not by feel.

You can organize the axes of evaluation by splitting them into at least these three.

What matters is separating the person who builds the evaluation from the person who writes the prompt. When the author judges their own work "good," the standard tends to go easy on itself. Have a different person, or a different vantage point, do the evaluation — this separation prevents the padding of quality. In addition to human evaluation, it also helps to prepare "expected answers" for a small set of representative cases and check against them every time you change the prompt.

06Regression Testing — Keep the Prompt in a State Where "You Notice When It Breaks"

Even once a prompt is written well, it is not safe forever. When you tweak the wording a little, or when the version of the AI you use goes up, output that used to come out correctly can quietly break. Checking all of it by hand every time is not realistic. So you set up regression testing (= a mechanism that repeatedly confirms whether behavior that previously passed still passes).

The skeleton of the method is simple.

• Assemble a few dozen pairs of representative inputs and expected outputs — prioritize cases where a problem arose in the past and cases where the boundary is delicate.
• Every time you change the prompt or the AI, run those pairs in a batch — automatically check whether they match expectations and whether they have deviated.
• If it drifts, stop — if even one output touches the norms, roll that change back before publishing.

In pharma especially, deliberately assembling test cases that confirm "does it avoid producing an exaggerated expression" and "does it avoid writing an unapproved indication" is what pays off. Throw dangerous inputs at it on purpose and confirm that the AI holds the line. This is the same idea as "test the fragile spots most heavily" in the world of code. It connects straight to the topic of test generation we take up next time (Vol. 4).

07Common Pitfalls — Failures That Recur in the Field

The places people stumble in prompt design are mostly the same. Knowing them in advance lets you avoid them.

• Getting greedy all at once — cram in "summarize it, also write headlines, and while you're at it proofread it" and each comes out half-baked. Split the tasks into separate calls.
• Forgetting to hand over the premises — ask without writing the approved indications or the target audience, and the AI fills the blanks on its own. State the context explicitly every time.
• Transferring the bias of the examples as is — if the few-shot examples are exaggerated or outdated, so is the output. Have a person inspect the examples first.
• Mistaking the process explanation for proof — the CoT "reasoning process" is an explanation, not a proof. Verify the conclusion independently.
• Never looking again once it works — output breaks with AI updates and small edits. Keep watch continuously with regression tests.
• Writing confidential or personal information directly into the prompt — follow internal rules and contractual terms for handling case data and information restricted to internal use. Don't paste it casually.

Every one of these is less a matter of technique than of design discipline. Put the way of asking into words, state the premises, separate the evaluation, protect it with tests. More than flashy tricks, this plain discipline is what holds up quality.

08Connections to Other Chapters on This Site

Prompt design connects to the other chapters of this site as follows. Read them together and the dots become a line.

• AI Programming Vol. 4 — Writing Tests With AI — extends this installment's "regression testing" into the context of code test generation and TDD. A continuation of the idea of protecting the prompt.
• AI Marketing Vol. 5 — Generated Content and Review — how the drafts made with few-shot and CoT are received at the "exit" of material review. The junction of design and review.
• Ad Regulations 01 — Pharmaceutical Act §§66–68 — the foundation of the "norm compliance" axis of evaluation. The original text on the line between exaggeration, unapproved promotion, and information provision.
• Material Review series — the practical work of the review that finally receives the AI-made drafts. It covers the craft of the human final judgment.

Prompt engineering is not a hunt for a magic incantation. Split the instruction into four parts, show examples when needed, and have complex judgments write out the process — each is a plain effort to "cut ambiguity and prevent leaps." And then evaluate the crafted output by a decided yardstick rather than by feel, and keep protecting it with regression tests. Only when you go this far does a prompt turn from "a lucky one-off" into "a mechanism anyone can run and reproduce."

In the pharma workplace, this reproducibility translates directly into safety. The Pharmaceutical and Medical Devices Act does not slacken just because the AI can write fast — exaggeration is §66, unapproved promotion is §68, information provision is §68-2. What gets judged is not "who wrote it" but "what is written." Put the prompt into words as a design, and fence it in with evaluation and tests. That is the surest footing for handling generative AI inside the boundaries of regulation. Next time, we take this idea of "protecting with tests" forward into writing tests together with AI, and TDD.