4. Building Problem Solving and Critical Thinking through Prompt Engineering
ZERO-SHOT PROMPTING
CHAIN OF THOUGHT (COT) PROMPTING
FEW-SHOT APPROACH
COT WITH SELF CONSISTENCY
Just concentrate on these two first
4A. Zero Shot
just add "A:Let's think step by step" after your question prompt
Large Language Models are Zero-Shot Reasoners https://arxiv.org/pdf/2205.11916.pdf
�4A: Example of "Zero-Shot" Prompt Engineering in Calculus�(Implicit Differentiation and finding equation of tangent lines)
Example: Find the equation of the tangent line at the given point:
y2e2x = 3y + x2 at (0,3)
Answer:
4A. "Zero Shot" Prompt Engineering
Example:
SAGE GPT-3.5 (Assistant)
Answer is WRONG!
y=3
See link: https://poe.com/s/Gx7T9oAZNSWcf6vcjSEp
4A. "Zero Shot" Prompt Engineering
Using the prompt engineering "zero shot" template:
Q. (Your original question)
A: "Let's think step by step"
SAGE GPT3.5 (Assistant) gets it correct! See
4A: Example from F1 Biology�Genetics with Punnett Squares �(or S3 Probability)
Original Question (O-Q): In pea plants, the trait for flower color is determined by a single gene with two alleles: purple (P) and white (p). If a heterozygous purple-flowered pea plant (Pp) is crossed with a homozygous white-flowered pea plant (pp), what is the probability of the offspring having purple flowers?
True (Teacher Generated) Answer:
In this case, we have a cross between a heterozygous purple-flowered pea plant (Pp) and a homozygous white-flowered pea plant (pp). We can use a Punnett square to determine the possible genotypes of the offspring and their probabilities.
P p
p Pp pp
p Pp pp
From the Punnett square, we can see that there are two possible genotypes for the offspring: Pp and pp. Since the purple flower color is determined by the presence of at least one dominant P allele, plants with the Pp genotype will have purple flowers.
There are a total of 4 boxes in the Punnett square, and 2 of them have the Pp genotype. Therefore, the probability of the offspring having purple flowers is 2 out of 4 or 1/2 (50%).
4A. "Zero Shot" Prompt Engineering
4A. "Zero Shot" Prompt Engineering
Using the prompt engineering "zero shot" template:
Q. (Your original question)
A: "Let's think step by step"
ChatGPT gets it correct! See
4B. What is "Chain of Thought" (CoT) Prompt Engineering?
Chain of Thought (CoT) is a technique we can use to improve the LLMs' performance on certain tasks.
Instead of asking the LLM a question directly, we break down the problem into smaller parts, and guide the model through solving each part step by step.
This method can help the model generate more accurate answers, and it allows us to understand the problem-solving process more clearly.
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models https://arxiv.org/pdf/2201.11903.pdf - "enables" reasoning capabilities of the LLM through intermediate steps of reasoning
�4B. Example of how CoT Prompt Engineering works (S3 Math: Linear Inequalities)
Example 1: If x<0, determine if the inequality x/6 < -x/5 is true.
Answer: x/6 > -x/5 <=> (30)* x/6 < (30)*(-x/5)
<=> 5x < -6
<=> 11x < 0
<=> x < 0
Therefore, the inequality x/6 < -x/5 is TRUE if x<0!
Therefore, TRUE!
Let's see if ChatGPT gets it correct?
Putting O-G into ChatGPT, get
FALSE
which is wrong!
ChatGPT shareable link:
https://poe.com/s/vF9cOrHZxedfW3SPy74P
Where did it go wrong?
Example:
Consider teaching the LLM how to solve a "similar problem":
i.e., Example 2:
If x<0, determine if the inequality x/5 > -x/4 is true.
Let's produce the model answer:�
To determine if the inequality x/5 > -x/4 is true for x < 0, we can follow these steps:
The inequality given is x/5 > -x/4.
2. Multiply through by 20 (the least common multiple of 5 and 4) to clear the fractions:
This gives 4x > -5x.
3. Move terms involving x to one side of the inequality:
This gives 4x + 5x > 0 which simplifies to 9x > 0.
4. Solve the resulting inequality:
This gives x > 0.
Now, let's consider the domain we are interested in: x < 0. The inequality x > 0 doesn't have any solutions in the domain x < 0. Therefore, the inequality x/5 > -x/4 is not true for x < 0.
Use the above example to "train" the LLM on how to solve these problems, step by step
Plug (Q, A, Q) into ChatGPT:
https://poe.com/s/hMvHWreMVnKZMvBwsRcK
Got it CORRECT!!!
Because we "trained/taught" the LLM how to solve such problems, step by step.
4B. Example of CoT from F1 Physics�Energy (potential + kinetic)
Original Question (O-Q): A roller coaster car with a mass of 500 kg is at the top of a hill that is 30 meters high. If the car is released, what will be its speed at the bottom of the hill? Assume that all the potential energy at the top of the hill is converted into kinetic energy at the bottom of the hill, and that there is no friction or air resistance.
The teacher produces the following
True (Teacher Generated) Answer:
At the top of the hill, the car has potential energy given by: PE = m * g * h where: m is the mass of the car (500 kg); g is the acceleration due to gravity (approximately 9.81 m/s²); h is the height of the hill (30 m)
At the bottom of the hill, the car has kinetic energy given by: KE = 0.5 * m * v² where v is the speed of the car at the bottom of the hill. Since the potential energy at the top is converted into kinetic energy at the bottom, we can equate the two expressions:
m * g * h = 0.5 * m * v²
We can now solve for v: v² = 2 * g * h which implies that v = sqrt(2 * g * h) = sqrt(2 * 9.81 m/s² * 30 m) =sqrt(588.6 m²/s²) ≈ 24.26 m/s
So, the speed of the roller coaster car at the bottom of the hill will be approximately 24.26 m/s.
4B. CoT Example in Physics
Consider teaching the LLM how to solve a "similar problem":
Example: (CoT-Q): A roller coaster car with a mass of 400 kg is at the top of a hill that is 20 meters high. If the car is released, what will be its speed at the bottom of the hill? Assume that all the potential energy at the top of the hill is converted into kinetic energy at the bottom of the hill, and that there is no friction or air resistance.
Let's produce the model answer (CoT-A):
At the top of the hill, the car has potential energy given by: PE = m * g * h where: m is the mass of the car (400 kg); g is the acceleration due to gravity (approximately 9.81 m/s²); h is the height of the hill (20 m)
At the bottom of the hill, the car has kinetic energy given by: KE = 0.5 * m * v² where v is the speed of the car at the bottom of the hill. Since the potential energy at the top is converted into kinetic energy at the bottom, we can equate the two expressions:
m * g * h = 0.5 * m * v²
We can now solve for v: v² = 2 * g * h which implies that v = sqrt(2 * g * h) = sqrt(2 * 9.81 m/s² * 20 m) =sqrt(392.4 m²/s²) ≈ 19.81 m/s
So, the speed of the roller coaster car at the bottom of the hill will be approximately 19.81 m/s.
Physics Results:
ChatGPT:
https://poe.com/s/xZDXIl5N5CAz3ynPMSRb
Got 34.0 m/s - which is wrong!
ChatGPT:
https://poe.com/s/nVQI4gG66UTIjLbItNqu
Got it correct!! i.e., got 24.27m/s