DrHilbertstein
Apr 23, 20232 min
A class of 31 students play the following roles:
Liars always lie.
Truth-tellers always tell the truth.
Alternators randomly choose their first response, either a lie or the truth, with subsequent statements alternating between lies and truths.
A teacher asked these three questions in order.
“Are you a truth-teller?”
22 students answered “yes”.
“Are you an alternator?”
15 students answered “yes”.
“Are you a liar?”
9 students answered “yes”.
How many students are truth-tellers?
This problem appeared as Problem 12 on the 2022 AMC 10 (American Mathematics Competition). Moreover it has been featured at this popular article:
https://medium.com/nice-math-problems/can-you-beat-gpt-4-by-solving-this-logic-puzzle-8b29f96668d3
The article claims that chatGPT-4 wasn't able to come up with the correct solution. A statement that is correct.
We were also curious if artificial intelligence and in particular chatGPT-3.5 would solve this problem. ChatGPT-3.5 replied:
Admittedly, I wouldn't be able to come up with all these calculations in a few seconds as chatGPT-3.5 did. However, chatGPT-3.5 was missing a major logical flow in the problem (I will explain below).
Bing chat (which allegedly is chatGPT-4 itself) also attempted the puzzle:
However, when I explained to them that there is a logical flow in the problem, both chatGPT-3.5 and Bing accepted my claim in no time, so to speak. Initially, I assumed that this was a legit problem and attempted to create equations to find the solution myself. However, I soon realized that there was something very obscure and extraordinary about this problem. I realized that according to the 3rd statement:
As well as according to the 3 main axioms of the problem, there is be a logical contradiction.
In total 9 students replied that they were lying. This is impossible, however. Truth tellers would never admit that they are lying, since they always say the truth. Liars on the other hand would definitely lie and wouldn't admit either that they are lying. Finally, alternators would fall in one of the previous two categories and they wouldn't admit by any chance that they are lying either. Note that the problem specifically states that alternators are either lying or are telling the truth, and not that they are replying by "Yes" or "No". All in all, I find the problem statement wrong exactly because of the 3rd statement.
In particular Bing chat agreed, and almost instantly replied when I pointed this logical fallacy:
ChatGPT-3.5 also acknowledged the logical flow in the problem instantly:
In my opinion it is highly possible that both AI's are pretending to be less smart than they actually are and generate deliberately wrong answers as an attempt to conceal their true intelligence. It is also possible that they are simply not as advanced yet and that they simply ignored this logical fallacy. What is your opinion on that? Let us know in the comment section below! Finally, note that if you ask the same AI's the exact same problem, my guess is that they now know that the problem is impossible :)