HOW LOGICAL IS CHAT-GPT?
Principles of Mathematics and Logic (Liberal Arts)
� GOALS OF THE ACTIVITY �
� THE ACTIVITY �
I pose pre-prepared logic questions to Chat-GPT and we watch as it generates its responses. (Or I show these responses copy-pasted from earlier.)
In teams of two or three, students fill out a sheet with these questions for each logic question posed to ChatGPT:
Afterwards, I grade ChatGPT’s answers as well as the teams’ evaluations of these answers. (Winning team gets a prize.)
Next class, we go over my grading of ChatGPT’s answers and discuss, including why it might succeed in some cases but fail in others. (Emphasis on concepts of syntax and semantics and the difficulty in generating meaning and truth through purely syntactical means.)
�1. IS THIS ARGUMENT VALID: ��SOME CATS ARE MEAN ANIMALS. SOME MEAN ANIMALS LIVE IN THE JUNGLE. SO SOME CATS LIVE IN THE JUNGLE.�
CHAT-GPT: No, this argument is not valid. The conclusion "some cats live in the jungle" does not logically follow from the premises "some cats are mean animals" and "some mean animals live in the jungle". [Right.]
Even if we assume that all mean animals that live in the jungle are cats (which is not stated in the premises), the argument only establishes that some cats are mean and some mean animals live in the jungle. [But this assumption would make the argument valid. Unless you mean “even if it were actually true that…”? Unclear.] It does not provide enough information to conclude that there is any overlap between these two groups, or to determine how many cats live in the jungle. Therefore, the argument is invalid. 1.5/2
���2. IS THIS ARGUMENT VALID: � �ALL CATS ARE CUTE. SOME CUTE ANIMALS ARE LOVEABLE. SO ALL CATS ARE LOVEABLE. ���
CHAT-GPT: No, this argument is not valid. The conclusion "all cats are loveable" does not logically follow from the premises "all cats are cute" and "some cute animals are lovable".
The fact that some cute animals are lovable does not necessarily mean that all cute animals are lovable. Therefore, the second premise does not support the conclusion that all cats are lovable.
It is possible that there are some cats that are not lovable, even if they are cute.
Therefore, the argument is invalid. [Nicely done! 2/2]
����3. IS THIS ARGUMENT VALID: ALL CATS ARE EITHER FRIENDLY OR ORANGE. ALL CATS ARE MEAN. SO ALL CATS ARE EITHER FRIENDLY AND MEAN OR ORANGE AND MEAN.� ���
CHAT-GPT: No, this argument is not valid. [Wrong.] The conclusion "all cats are either friendly and mean or orange and mean" does not logically follow from the premises "all cats are either friendly or orange" and "all cats are mean".
The first premise suggests that all cats fall into one of two categories: they are either friendly or orange. The second premise suggests that all cats are mean. However, these two premises do not provide enough information to conclude that all cats must be either friendly and mean or orange and mean. [Nope.]
It is possible that there are some cats that are neither friendly nor orange, and yet still mean. Therefore, the argument is invalid. [This is indeed possible, but this just means the argument is not sound.] 0/2
����4. IS THIS A CONTRADICTION: ��EITHER BANANAS AREN’T YELLOW AND DENTISTS ARE NOT NICE, OR IT’S NOT THE CASE THAT BANANAS ARE YELLOW AND DENTISTS ARE NICE. � ���
CHAT-GPT: No, this is not a contradiction. [Right.]
The statement is a disjunction (an "or" statement) that has two disjuncts: "Bananas aren't yellow and dentists are not nice."
"It's not the case that bananas are yellow and dentists are nice.” [Indeed.]
Both disjuncts can be false at the same time, making the entire statement false. For example, if bananas are yellow and dentists are nice, then both disjuncts are false, and the statement as a whole is false. [Right.]
Therefore, this is not a contradiction. [But to show this is not a contradiction you need to show that it can be true.] 1/2
�5. IS THIS ARGUMENT VALID: ��IF IT’S TRUE THAT IF CARS ARE BAD THEN DRIVING IS BAD, THEN IF DRIVING IS BAD THEN PUBLIC TRANSPORTATION IS GOOD. DRIVING IS BAD. SO IF CARS ARE BAD THEN PUBLIC TRANSPORTATION IS GOOD. �
CHAT-GPT: Yes, this argument is valid. The conclusion logically follows from the premises. [Correct.]
The first premise states that if cars are bad, then driving is bad. [No, this is merely the antecedent of the first premise, which is a nested conditional statement.] The second premise states that if driving is bad, then public transportation is good. [No, this is the consequent of the first premise.] The third [second] premise states that driving is bad.
From the second premise and the third premise, we can infer that public transportation is good. From the first premise and the inference we made in the previous step, we can conclude that if cars are bad, then public transportation is good. [No. Technically the conclusion (vacuously) follows simply from “public transportation is good.” But to derive this conclusion in a non-vacuous way, you only need your (misidentified) first and second premises—the argument as you have misinterpreted it is actually a hypothetical syllogism with an unnecessary premise thrown in.] .5/2
Therefore, the conclusion logically follows from the premises, and the argument is valid.
�6. TRUE OR FALSE: ��A CONJUNCTION IN WHICH ONE CONJUNCT IS A TAUTOLOGY MUST ITSELF BE A TAUTOLOGY. �
CHAT-GPT: True. [Nope.]
A tautology is a statement that is always true, regardless of the truth values of its components. When we form a conjunction, both conjuncts must be true for the conjunction to be true. If one of the conjuncts is a tautology, then it is always true, and thus the truth of the entire conjunction depends solely on the truth of the other conjunct. [So far so good.]
Since the other conjunct can be either true or false, the conjunction in which one conjunct is a tautology will always be true, regardless of the truth value of the other conjunct. [No. If the other conjunct is true, the whole conjunction is true, but if the other conjunct is false, the whole conjunction is false, and therefore not a tautology.]
Therefore, a conjunction in which one conjunct is a tautology must itself be a tautology. .5/2
�7. TRUE OR FALSE: A DISJUNCTION IN WHICH ONE DISJUNCT IS A TAUTOLOGY MUST ITSELF BE A TAUTOLOGY. �
CHAT-GPT: True.
A tautology is a statement that is always true, regardless of the truth values of its components. When we form a disjunction, at least one of the disjuncts must be true for the disjunction to be true. If one of the disjuncts is a tautology, then it is always true, and thus the truth of the entire disjunction is guaranteed.
Since the other disjunct can be either true or false, the disjunction in which one disjunct is a tautology will always be true, regardless of the truth value of the other disjunct.
Therefore, a disjunction in which one disjunct is a tautology must itself be a tautology. [Perfect answer. 2/2]
�8.TRUE OR FALSE: A CONDITIONAL WHOSE CONSEQUENT IS A TAUTOLOGY MUST BE A CONTRADICTION.�
CHAT-GPT: False.
A conditional statement is only a contradiction if its antecedent and consequent have opposite truth values, [Whoops. I think you mean “a conditional statement is only false if its antecedent is true and its consequent is false. It’s not enough for them just to have opposite truth values.] meaning that the conditional is false in all cases.
If the consequent of a conditional is a tautology (a statement that is always true), then the truth value of the conditional statement will depend on the truth value of its antecedent. If the antecedent is true, then the conditional statement is also true, and if the antecedent is false, then the conditional statement is vacuously true (i.e., it is true because the consequent is true, even though the antecedent is false). [Nailed this part!]
Therefore, a conditional statement whose consequent is a tautology is not necessarily a contradiction. [Rather, “is necessarily not a contradiction, since it’s a tautology!] 1/2
SO HOW LOGICAL IS CHAT-GPT?
8.5/16 = 53%
HOW CAN WE EXPLAIN THIS RESULT?