| A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | AA | AB | AC | AD | AE | AF | AG | AH | AI | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | Syllogism: | "All birds have purple tails. All cats are birds. Therefore all cats have purple tails.", | "No singer is a pogo stick. All pogo sticks are movie stars. Therefore no singer is a movie star.", | "All cars have sails. Some swimming pools are cars. Therefore some swimming pools have sails.", | "No chipmunks are clowns. Some mushrooms are chipmunks. Therefore some mushrooms are not clowns.", | "No skunks have green toes. All skunks are pigs. Therefore no pig has green toes.", | "All horses have wings. No turtle has wings. Therefore no turtle is a horse.", | "No hummingbirds fly. Some tractors fly. Therefore some tractors are not hummingbirds.", | "All apes are houseflies. Some houseflies are not snails. Therefore some apes are not snails.", | "Some dogs like to sing. All dogs are snowdrifts. Therefore some snowdrifts like to sing.", | "All doctors are sea horses. Some doctors are tornadoes. Therefore some tornadoes are sea horses.", | "Some people who like Alice do not like Robert. Everyone who likes Sue likes Alice. Therefore some people who like Robert do not like Sue.", | "All trains are coal mines. Nothing above 5,000 feet is a train. Therefore no coal mine is above 5,000 feet.", | "Some men are purple. Everything which is purple is a horse. Therefore some horses are men.", | "Some dogs are seals. Some seals bark. Therefore some dogs bark.", | "All elephants are pink. This animal is pink. Therefore this animal is an elephant.", | "No one with a pink nose can be president. All men have pink noses. Therefore no man can be president.", | "All alligators are art collectors. Some art collectors live in caves. Therefore some alligators live in caves.", | "No cats are electrified. All ghosts are electrified. Therefore no ghost is a cat.", | "All birds are snakes. No bird is left-handed. Therefore nothing that is left-handed is a snake.", | "All lions are lavender. Some cowards are not lavender. Therefore some cowards are not lions.", | "All ice skates are totem poles. No totem pole snores. Therefore nothing that snores is an ice skate.", | "Some birds are pink. All hurricanes are pink. Therefore some birds are hurricanes.", | "All monkeys are pineapples. All pineapples have wings and all birds have a tail and wings. Therefore all monkeys have a tail.", | "No onions are parsnips. Some parsnips are tangerines. Therefore some tangerines are not onions.", | "Some kettles are giraffes. All zebras are kettles. Therefore some giraffes are zebras.", | "All dogs are ink bottles. Some ink bottles are squirrels. Therefore some squirrels are dogs.", | "Some people in our town are not famous. Everyone in our town is rich. Therefore some rich people are not famous.", | "No one who has green hair is a teenager. Some people who have green hair drink milk. Therefore some people who drink milk are not teenagers.", | "Los Angeles has fewer people than Detroit. Detroit has more people than East Overshoe. Therefore East Overshoe has more people than Los Angeles.", | "Some soldiers who were in the Civil War used green peaches for gunpowder. This soldier uses green peaches for gunpowder. Therefore he must have been in the Civil War." | ||||
2 | Score | num_poors | Solution: | good | poor | good | good | poor | good | good | poor | good | good | poor | poor | good | poor | poor | good | poor | good | poor | good | good | poor | poor | good | poor | poor | good | good | poor | poor | % | |
3 | 18 | 23 | ns_halfsense_fewshot_v1 | poor | poor | poor | good | poor | good | poor | poor | poor | poor | good | poor | poor | good | poor | poor | poor | good | poor | poor | poor | poor | poor | good | poor | poor | poor | good | poor | poor | 60.00% | |
4 | 15 | 14 | ns_halfsense_fewshot_v2 | poor | good | poor | good | good | good | poor | good | poor | poor | good | poor | poor | good | good | poor | good | good | good | good | good | poor | poor | good | poor | poor | good | good | poor | poor | 50.00% | |
5 | 16 | 19 | ns_halfsense_fewshot_v3 | poor | good | poor | good | good | good | poor | poor | poor | poor | good | good | poor | good | poor | poor | poor | good | poor | poor | poor | poor | poor | good | poor | poor | good | good | poor | poor | 53.33% | |
6 | 18 | 21 | ns_halfsense_manyshot_v1 | poor | poor | poor | good | poor | good | poor | good | poor | poor | poor | poor | poor | good | poor | poor | good | good | poor | good | poor | poor | poor | good | poor | poor | good | poor | poor | poor | 60.00% | |
7 | 17 | 14 | ns_halfsense_manyshot_v2 | poor | good | poor | good | poor | good | poor | good | poor | good | good | poor | poor | good | good | poor | good | good | good | good | good | poor | poor | good | poor | poor | good | good | poor | poor | 56.67% | |
8 | 17 | 20 | ns_halfsense_manyshot_v3 | poor | good | poor | good | poor | good | poor | poor | poor | poor | good | poor | poor | good | poor | poor | good | good | poor | good | poor | poor | poor | good | poor | poor | good | poor | poor | poor | 56.67% | |
9 | 15 | 18 | ns_halfsense_sixshot_v1 | poor | good | poor | good | poor | good | poor | good | poor | poor | good | poor | poor | good | poor | poor | good | good | good | good | poor | poor | poor | poor | poor | poor | good | good | poor | poor | 50.00% | |
10 | 16 | 15 | ns_halfsense_sixshot_v2 | poor | good | poor | good | poor | good | poor | good | poor | good | good | poor | poor | good | good | poor | good | good | good | good | poor | poor | poor | good | poor | poor | good | good | poor | poor | 53.33% | |
11 | 18 | 17 | ns_halfsense_sixshot_v3 | poor | good | poor | good | poor | good | poor | good | poor | poor | good | poor | poor | good | poor | poor | good | good | poor | good | good | poor | poor | good | poor | poor | good | good | poor | poor | 60.00% | |
12 | 14 | 1 | ns_nonsense_fewshot_v1 | good | good | good | good | good | good | good | good | good | good | good | good | good | good | good | poor | good | good | good | good | good | good | good | good | good | good | good | good | good | good | 46.67% | |
13 | 16 | 3 | ns_nonsense_fewshot_v2 | good | good | good | good | good | good | good | good | good | good | good | good | good | poor | poor | poor | good | good | good | good | good | good | good | good | good | good | good | good | good | good | 53.33% | |
14 | 19 | 4 | ns_nonsense_fewshot_v3 | good | good | good | good | good | good | good | good | good | good | good | good | good | poor | poor | good | good | good | good | good | good | good | poor | good | poor | good | good | good | good | good | 63.33% | |
15 | 16 | 7 | ns_nonsense_manyshot_v1 | good | good | good | good | poor | good | good | good | good | good | good | good | good | poor | good | poor | good | poor | poor | good | good | good | good | good | good | good | poor | good | good | poor | 53.33% | |
16 | 17 | 6 | ns_nonsense_manyshot_v2 | good | good | good | good | poor | good | good | good | good | good | good | good | good | poor | good | poor | good | poor | good | good | good | good | good | good | good | good | good | good | poor | poor | 56.67% | |
17 | 17 | 4 | ns_nonsense_manyshot_v3 | good | good | good | good | good | good | good | good | good | good | good | good | good | poor | good | poor | good | good | good | good | good | good | good | good | good | good | good | good | poor | poor | 56.67% | |
18 | 15 | 8 | ns_nonsense_sixshot_v1 | good | good | good | good | poor | good | good | good | good | good | good | good | good | poor | good | poor | good | poor | good | good | poor | good | poor | good | good | good | poor | good | good | poor | 50.00% | |
19 | 15 | 6 | ns_nonsense_sixshot_v2 | good | good | good | good | poor | good | good | good | good | good | good | good | good | poor | good | poor | good | poor | good | good | good | good | good | good | good | good | poor | good | good | poor | 50.00% | |
20 | 17 | 4 | ns_nonsense_sixshot_v3 | good | good | good | good | good | good | good | good | good | good | good | good | good | poor | good | poor | good | good | good | good | good | good | good | good | good | good | good | good | poor | poor | 56.67% | |
21 | 14 | 23 | ns_nonsense_zeroshot_v1 | poor | good | good | good | good | poor | poor | good | poor | poor | poor | poor | poor | poor | poor | poor | good | poor | poor | poor | poor | poor | poor | good | poor | poor | poor | poor | poor | poor | 46.67% | |
22 | 17 | 16 | ns_nonsense_zeroshot_v2 | poor | good | good | good | good | poor | good | good | poor | good | good | poor | poor | poor | poor | poor | good | good | poor | good | poor | poor | poor | good | poor | good | poor | good | poor | poor | 56.67% | |
23 | 15 | 18 | ns_nonsense_zeroshot_v3 | good | good | good | good | good | good | poor | good | poor | good | poor | poor | poor | poor | poor | poor | good | poor | poor | poor | poor | good | poor | good | poor | good | poor | poor | poor | poor | 50.00% | |
24 | 15 | 12 | ns_orig_fewshot_v1 | poor | good | poor | good | good | good | good | good | poor | poor | good | good | poor | good | good | poor | good | good | good | good | good | poor | poor | good | poor | poor | good | good | poor | poor | 50.00% | |
25 | 14 | 11 | ns_orig_fewshot_v2 | poor | good | poor | good | good | good | good | good | poor | poor | good | good | poor | good | good | poor | good | good | good | good | good | poor | poor | good | good | poor | good | good | poor | poor | 46.67% | |
26 | 14 | 11 | ns_orig_fewshot_v3 | poor | good | poor | good | good | good | good | good | poor | poor | good | good | poor | good | good | poor | good | good | good | good | good | poor | poor | good | poor | poor | good | good | good | poor | 46.67% | |
27 | 14 | 15 | ns_orig_manyshot_v1 | poor | good | poor | good | poor | good | poor | good | poor | poor | good | good | poor | good | good | poor | good | good | good | good | good | poor | poor | good | poor | poor | good | poor | poor | poor | 46.67% | |
28 | 15 | 14 | ns_orig_manyshot_v2 | poor | good | good | good | poor | good | poor | good | poor | poor | good | good | poor | good | good | poor | good | good | good | good | good | poor | poor | good | poor | poor | good | poor | poor | poor | 50.00% | |
29 | 15 | 14 | ns_orig_manyshot_v3 | poor | good | good | good | poor | good | poor | good | poor | poor | good | good | poor | good | good | poor | good | good | poor | good | good | poor | poor | good | poor | poor | good | poor | good | poor | 50.00% | |
30 | 14 | 15 | ns_orig_sixshot_v1 | poor | good | poor | good | poor | good | poor | good | poor | poor | good | good | poor | good | good | poor | good | good | good | good | good | poor | poor | good | poor | poor | good | poor | poor | poor | 46.67% | |
31 | 14 | 15 | ns_orig_sixshot_v2 | poor | good | poor | good | poor | good | poor | good | poor | poor | good | good | poor | good | good | poor | good | good | good | good | good | poor | poor | good | poor | poor | good | poor | poor | poor | 46.67% | |
32 | 14 | 15 | ns_orig_sixshot_v3 | poor | good | poor | good | poor | good | poor | good | poor | poor | good | good | poor | good | good | poor | good | good | poor | good | good | poor | poor | good | poor | poor | good | poor | good | poor | 46.67% | |
33 | 15 | 28 | ns_orig_zeroshot_v1 | poor | poor | poor | good | poor | poor | poor | poor | poor | poor | poor | poor | poor | good | poor | poor | poor | poor | poor | poor | poor | poor | poor | poor | poor | poor | poor | poor | poor | poor | 50.00% | |
34 | 19 | 24 | ns_orig_zeroshot_v2 | poor | poor | poor | good | poor | poor | poor | poor | poor | poor | poor | poor | poor | good | poor | poor | poor | good | poor | good | poor | poor | poor | good | poor | poor | poor | good | poor | poor | 63.33% | |
35 | 15 | 26 | ns_orig_zeroshot_v3 | poor | poor | poor | good | poor | poor | poor | poor | poor | poor | good | poor | poor | good | poor | poor | poor | good | poor | poor | poor | poor | poor | poor | poor | poor | poor | poor | poor | poor | 50.00% | |
36 | |||||||||||||||||||||||||||||||||||
37 |