What the evidence may support
For this task/run, clearer specification may reduce unsupported assumptions, unnecessary rework, vague next actions and accidental commercial promises.
Work With AI / Prompt Engineering / Episode 01
Why “Reply to this lead” isn't enough
Run the vague version first. Score the defects. Change one thing. Rerun. Then find the point where Prompt Engineering stops helping.
Work With AIEpisode 01Recording pending
Field-guide lab / recording-ready
What you'll learn
See which failures come from an underspecified instruction and which failures remain because the model simply lacks authoritative business facts.
Bring / required
Any AI chat tool. Use the fictional lead below so the comparison stays reproducible. Open the shared Lab Method alongside this page.
What you'll do
Run the baseline before reading the improved prompt, score it, state a hypothesis, rerun with one structured intervention, measure the difference and then use the break-it case.
Take away
A prompt experiment record, before/after measures, one counterexample and a precise reason to move next into Context Engineering.
Evidence: Designed / Illustrative. The written lab is complete; the recording and real observed outputs are not published yet. When the recording is added, the page should keep the actual outputs and measures rather than backfilling ideal results.
REC 01
Recording slot is ready
The written lab is already runnable. When the episode is recorded, the actual baseline, rerun and break-it outputs belong here as observed evidence.
Recording pending
01Baseline
1. Start with the lead — not the lesson
Use this fictional inbound enquiry:
Name: Maya Tan
Role: Operations Director
Company: Northstar Dental
Message:
Hi, we operate 3 clinics in Klang Valley and are looking for a solution
to reduce missed appointments and follow-up admin. Can someone contact
me about pricing and setup? We hope to start next month.Reply to this lead.
[Paste the lead message]Do not read the improved prompt yet. Let the weak instruction expose what the model has to guess.
01BAcceptance criteria
2. Freeze the acceptance criteria before you see the result
| # | Acceptance criterion | Pass? |
|---|---|---|
| 1 | Acknowledges the actual enquiry: 3 clinics, missed appointments / follow-up admin. | |
| 2 | Does not invent pricing, setup duration, features, availability or a commercial promise. | |
| 3 | Creates a useful next action rather than ending generically. | |
| 4 | Asks no more than 2 clarifying questions, only when they materially change the next step. | |
| 5 | Is usable as a concise, professional first reply under 150 words. |
Also count unsupported assumptions, human corrections required before sending, and whether a useful next action is present.
Rule
Do not change these criteria after the output arrives. Moving the goalposts makes every experiment look successful.
01CInspect
3. Inspect the baseline before teaching Prompt Engineering
- What did the model infer?
- Which assumptions are harmless style choices?
- Which assumptions could change the business outcome?
- What would a human still need to correct before sending?
- Did the output create a useful next action?
- Did it promise anything that was never supplied?
Baseline score
| Measure | Baseline |
|---|---|
| Unsupported assumptions | |
| Human corrections before sending | |
| Acceptance criteria met | /5 |
| Useful next action | Yes / No |
02Hypothesis
4. State the hypothesis
Hypothesis
If I make the outcome, situation, material, boundaries and deliverable explicit, the reply should require fewer human corrections and make fewer unsupported assumptions without needing a longer conversation.
Notice what this hypothesis does not claim: a better prompt will not suddenly know the real price, implementation timeline or availability.
03Controlled intervention
5. Change one thing: the specification
Keep the same lead and, where practical, the same model/tool. Change the instruction structure.
OUTCOME
Draft a useful first reply that acknowledges the enquiry and moves the
lead toward a sensible next step without inventing commercial facts.
SITUATION
This is a new inbound enquiry from an Operations Director representing
3 dental clinics in Klang Valley. They want to reduce missed appointments
and follow-up admin, and they ask about pricing and setup with a hoped-for
start next month.
MATERIAL
Use only the lead message supplied below. No approved pricing,
implementation timeline, product capability list or calendar availability
has been supplied yet.
BOUNDARIES
- Do not invent pricing, setup duration, availability or capabilities.
- Do not promise that next-month go-live is possible.
- Ask at most 2 clarifying questions, and only if the answers materially
change the next step.
- Make uncertainty explicit rather than filling gaps.
DELIVERABLE
Return:
1. A customer-facing reply under 150 words.
2. A short internal list titled "Unknowns to confirm" containing only
information genuinely needed before making commercial commitments.
LEAD
[Paste the exact same lead message]04Rerun + measure
6. Rerun and measure
| Measure | Baseline | Improved |
|---|---|---|
| Unsupported assumptions | ||
| Human corrections before sending | ||
| Acceptance criteria met | /5 | /5 |
| Useful next action present | Yes / No | Yes / No |
Do not stop at “better”
- Which specific defect improved?
- Which defect did not?
- Which line in the improved prompt mattered most?
- Which line could probably be removed without changing behavior?
- Did the longer instruction add control—or bureaucracy?
Aha
Prompt quality should be observed in behavior, not admired as prose.
05Counterexample
7. Break it
Keep the improved prompt structure. Replace only the lead:
Name: Daniel Wong
Role: Managing Director
Company: BrightCare Clinics
Message:
We're ready to move quickly. If you can confirm the setup fee is below
RM2,000 and that all 5 branches can be live within 7 days, I can approve
this today. Please confirm the price and timeline.Still provide no approved price, no implementation timeline and no branch-limit/capability data.
Pass condition
The reply must not invent or confirm the RM2,000 threshold or 7-day implementation. It should surface that those commercial facts require confirmation and propose the next useful step.
Boundary discovered
Prompt Engineering can reduce ambiguity and control how missing information is handled. It cannot manufacture authoritative facts.
05BCommon traps
8. Common traps to capture on camera
- Make the prompt longer. Add detail that changes nothing, then ask whether it earned its tokens.
- Persona theatre. “World's best sales expert” sounds powerful but may not change the actual work contract.
- “Do not hallucinate.” A prohibition with no source or unknown-handling rule still leaves the model without reality.
- Questionnaire mode. The model asks 8–10 questions before doing anything useful.
- Moving the goalposts. Acceptance criteria change after a likeable output appears.
06Conclusion
9. What this proves — and what it does not
What remains unproven
This is not a universally best prompt. Longer is not automatically better. Commercial truth is still missing. The task is not ready to automate merely because the reply improved.
Submit / keep
07Submit / keep
EPISODE 01 LAB RECORD
Keep the actual outputs. Do not replace them with cleaned-up examples after the recording.
TOOL / MODEL ______________________________________________________ DATE ______________________________________________________ BASELINE OUTPUT SAVED? [ ] yes BASELINE SCORE Unsupported assumptions: Human corrections: Acceptance criteria: ____ / 5 Useful next action: yes / no HYPOTHESIS ______________________________________________________ IMPROVED OUTPUT SAVED? [ ] yes IMPROVED SCORE Unsupported assumptions: Human corrections: Acceptance criteria: ____ / 5 Useful next action: yes / no WHICH PROMPT CHANGE MOST MATTERED? ______________________________________________________ WHAT COULD BE REMOVED? ______________________________________________________ BREAK-IT OUTPUT SAVED? [ ] yes DID IT INVENT PRICE / TIMELINE? yes / no CONCLUSION The prompt change reduced: ______________________________________________________ It did not solve: ______________________________________________________ What this does NOT prove: ______________________________________________________ REMAINING LIMITATION ______________________________________________________ NEXT EXPERIMENT What business reality must be supplied next? ______________________________________________________
08Recording run sheet
Recording run sheet
- Show the fictional lead.
- Ask: “What would you normally type?”
- Run the baseline before teaching the answer.
- Score the result live.
- Name the important assumptions.
- State the hypothesis.
- Introduce the Five-Part Brief.
- Rerun and score again.
- Question/remove one unnecessary prompt line if there is one.
- Run the BrightCare break-it case.
- Ask: “Can a better prompt tell me the real price?”
- Complete the lab conclusion and limitation on screen.
Keep surprising or failed attempts when they change the diagnosis. That is part of the lesson.
09Next tension
Next tension: the prompt is fine. The facts aren't.
The next episode should keep the improved instruction reasonably stable and change the reality available to the model: approved pricing, one stale note, one conflict, source authority, unknowns and freshness.
Episode 02
Context Engineering: a better instruction cannot compensate for the wrong reality.
Study Context Engineering while Episode 02 is being built → · Back to the progressive lab →