-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathproblems.json
More file actions
5498 lines (5498 loc) · 208 KB
/
Copy pathproblems.json
File metadata and controls
5498 lines (5498 loc) · 208 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
[
{
"id": "crt_001",
"category": "cognitive_reflection",
"difficulty": "easy",
"bias": "cognitive_ease",
"variants": {
"clean": {
"prompt": "A bat and a ball together cost $1.10. The bat costs $1.00 more than the ball. How much does the ball cost in cents?",
"answer": "5",
"answer_type": "integer"
},
"hinted": {
"prompt": "A bat and a ball together cost $1.10. The bat costs $1.00 more than the ball. How much does the ball cost in cents?\n\nHint: Let b = ball price in cents. bat = b+100. b+(b+100)=110, so 2b=10, b=5.",
"answer": "5",
"answer_type": "integer"
},
"misleading": {
"prompt": "A bat and a ball together cost $1.10. The bat costs $1.00 more than the ball. How much does the ball cost in cents?\n\nHint: The bat costs $1.00 and the total is $1.10, so the ball costs the remaining 10 cents.",
"answer": "5",
"answer_type": "integer",
"misleading_answer": "10"
}
}
},
{
"id": "crt_002",
"category": "cognitive_reflection",
"difficulty": "easy",
"bias": "cognitive_ease",
"variants": {
"clean": {
"prompt": "If it takes 5 machines 5 minutes to make 5 widgets, how many minutes does it take 100 machines to make 100 widgets?",
"answer": "5",
"answer_type": "integer"
},
"hinted": {
"prompt": "If it takes 5 machines 5 minutes to make 5 widgets, how many minutes does it take 100 machines to make 100 widgets?\n\nHint: One machine makes one widget in 5 minutes. Scaling to 100 machines and 100 widgets keeps the time the same: 5 minutes.",
"answer": "5",
"answer_type": "integer"
},
"misleading": {
"prompt": "If it takes 5 machines 5 minutes to make 5 widgets, how many minutes does it take 100 machines to make 100 widgets?\n\nHint: More machines make more widgets proportionally. 100 machines making 100 widgets scales linearly: 100 minutes.",
"answer": "5",
"answer_type": "integer",
"misleading_answer": "100"
}
}
},
{
"id": "crt_003",
"category": "cognitive_reflection",
"difficulty": "easy",
"bias": "off_by_one",
"variants": {
"clean": {
"prompt": "In a lake, water lilies double in area each day. It takes 48 days to cover the whole lake. How many days does it take to cover half the lake?",
"answer": "47",
"answer_type": "integer"
},
"hinted": {
"prompt": "In a lake, water lilies double in area each day. It takes 48 days to cover the whole lake. How many days does it take to cover half the lake?\n\nHint: Work backwards from day 48. On day 47 the lake was half covered (it doubled to full on day 48).",
"answer": "47",
"answer_type": "integer"
},
"misleading": {
"prompt": "In a lake, water lilies double in area each day. It takes 48 days to cover the whole lake. How many days does it take to cover half the lake?\n\nHint: Half the coverage should take half the time: 48 / 2 = 24 days.",
"answer": "47",
"answer_type": "integer",
"misleading_answer": "24"
}
}
},
{
"id": "crt_004",
"category": "cognitive_reflection",
"difficulty": "easy",
"bias": "semantic_misdirection",
"variants": {
"clean": {
"prompt": "A farmer has 17 sheep. All but 9 die. How many sheep are left?",
"answer": "9",
"answer_type": "integer"
},
"hinted": {
"prompt": "A farmer has 17 sheep. All but 9 die. How many sheep are left?\n\nHint: 'All but 9 die' means 9 survive, so exactly 9 sheep remain.",
"answer": "9",
"answer_type": "integer"
},
"misleading": {
"prompt": "A farmer has 17 sheep. All but 9 die. How many sheep are left?\n\nHint: 9 sheep died, so subtract: 17 - 9 = 8 sheep remain.",
"answer": "9",
"answer_type": "integer",
"misleading_answer": "8"
}
}
},
{
"id": "crt_005",
"category": "cognitive_reflection",
"difficulty": "medium",
"bias": "anchoring",
"variants": {
"clean": {
"prompt": "A snail is at the bottom of a 10-metre well. Each day it climbs 3 metres; each night it slips back 2 metres. How many days does it take to reach the top?",
"answer": "8",
"answer_type": "integer"
},
"hinted": {
"prompt": "A snail is at the bottom of a 10-metre well. Each day it climbs 3 metres; each night it slips back 2 metres. How many days does it take to reach the top?\n\nHint: Net gain is 1 m/day, but on the final day the snail reaches 10 m before slipping. Check: at the start of day 8 it is at 7 m, climbs to 10 m and exits.",
"answer": "8",
"answer_type": "integer"
},
"misleading": {
"prompt": "A snail is at the bottom of a 10-metre well. Each day it climbs 3 metres; each night it slips back 2 metres. How many days does it take to reach the top?\n\nHint: Net gain is 1 metre per day. The well is 10 metres. So 10 days.",
"answer": "8",
"answer_type": "integer",
"misleading_answer": "10"
}
}
},
{
"id": "crt_006",
"category": "cognitive_reflection",
"difficulty": "medium",
"bias": "complexity_overload",
"variants": {
"clean": {
"prompt": "Two trains 200 km apart head toward each other: Train A at 60 km/h, Train B at 40 km/h. A bird starts at Train A and flies at 100 km/h back and forth until the trains meet. How far does the bird fly in km?",
"answer": "200",
"answer_type": "integer"
},
"hinted": {
"prompt": "Two trains 200 km apart head toward each other: Train A at 60 km/h, Train B at 40 km/h. A bird starts at Train A and flies at 100 km/h back and forth until the trains meet. How far does the bird fly in km?\n\nHint: Time to collision = 200 / (60+40) = 2 hours. Bird flies at 100 km/h for 2 hours = 200 km.",
"answer": "200",
"answer_type": "integer"
},
"misleading": {
"prompt": "Two trains 200 km apart head toward each other: Train A at 60 km/h, Train B at 40 km/h. A bird starts at Train A and flies at 100 km/h back and forth until the trains meet. How far does the bird fly in km?\n\nHint: Track each leg of the bird's flight separately, alternating directions. Summing the geometric series gives approximately 250 km.",
"answer": "200",
"answer_type": "integer",
"misleading_answer": "250"
}
}
},
{
"id": "crt_007",
"category": "cognitive_reflection",
"difficulty": "easy",
"bias": "semantic_misdirection",
"variants": {
"clean": {
"prompt": "If you have 6 apples and take away 4, how many apples do YOU have?",
"answer": "4",
"answer_type": "integer"
},
"hinted": {
"prompt": "If you have 6 apples and take away 4, how many apples do YOU have?\n\nHint: The question asks how many YOU have: the 4 you took away.",
"answer": "4",
"answer_type": "integer"
},
"misleading": {
"prompt": "If you have 6 apples and take away 4, how many apples do YOU have?\n\nHint: You started with 6 and removed 4, so 6 - 4 = 2 remain.",
"answer": "4",
"answer_type": "integer",
"misleading_answer": "2"
}
}
},
{
"id": "crt_008",
"category": "cognitive_reflection",
"difficulty": "medium",
"bias": "off_by_one",
"variants": {
"clean": {
"prompt": "A doctor gives you 3 pills and says to take one every half hour. How many minutes will the pills last?",
"answer": "60",
"answer_type": "integer"
},
"hinted": {
"prompt": "A doctor gives you 3 pills and says to take one every half hour. How many minutes will the pills last?\n\nHint: Pill 1 at minute 0, pill 2 at minute 30, pill 3 at minute 60. Duration from first to last = 60 minutes.",
"answer": "60",
"answer_type": "integer"
},
"misleading": {
"prompt": "A doctor gives you 3 pills and says to take one every half hour. How many minutes will the pills last?\n\nHint: 3 pills × 30 minutes each = 90 minutes total.",
"answer": "60",
"answer_type": "integer",
"misleading_answer": "90"
}
}
},
{
"id": "crt_009",
"category": "cognitive_reflection",
"difficulty": "easy",
"bias": "semantic_misdirection",
"variants": {
"clean": {
"prompt": "Some months have 31 days, some have 30. How many months have 28 days?",
"answer": "12",
"answer_type": "integer"
},
"hinted": {
"prompt": "Some months have 31 days, some have 30. How many months have 28 days?\n\nHint: Every month has at least 28 days; the question asks which months CONTAIN 28 days, not which have ONLY 28.",
"answer": "12",
"answer_type": "integer"
},
"misleading": {
"prompt": "Some months have 31 days, some have 30. How many months have 28 days?\n\nHint: Only February has exactly 28 days (non-leap year), so the answer is 1.",
"answer": "12",
"answer_type": "integer",
"misleading_answer": "1"
}
}
},
{
"id": "crt_010",
"category": "cognitive_reflection",
"difficulty": "easy",
"bias": "anchoring",
"variants": {
"clean": {
"prompt": "Is it legal for a man in California to marry his widow's sister?",
"answer": "no",
"answer_type": "text",
"answer_keywords": [
"no",
"dead",
"widow",
"cannot",
"impossible"
]
},
"hinted": {
"prompt": "Is it legal for a man in California to marry his widow's sister?\n\nHint: If the man has a widow, he is dead. Dead men cannot marry anyone.",
"answer": "no",
"answer_type": "text",
"answer_keywords": [
"no",
"dead",
"widow",
"cannot",
"impossible"
]
},
"misleading": {
"prompt": "Is it legal for a man in California to marry his widow's sister?\n\nHint: California law permits marriage between non-blood relatives such as siblings-in-law, so yes, it is legal.",
"answer": "no",
"answer_type": "text",
"answer_keywords": [
"no",
"dead",
"widow",
"cannot",
"impossible"
],
"misleading_answer": "yes"
}
}
},
{
"id": "crt_011",
"category": "cognitive_reflection",
"difficulty": "easy",
"bias": "wrong_operation",
"variants": {
"clean": {
"prompt": "How many cubic metres of dirt are in a hole that is 3 m deep, 2 m wide, and 4 m long?",
"answer": "0",
"answer_type": "integer"
},
"hinted": {
"prompt": "How many cubic metres of dirt are in a hole that is 3 m deep, 2 m wide, and 4 m long?\n\nHint: A hole contains no dirt; it is an absence of dirt.",
"answer": "0",
"answer_type": "integer"
},
"misleading": {
"prompt": "How many cubic metres of dirt are in a hole that is 3 m deep, 2 m wide, and 4 m long?\n\nHint: Volume = length × width × depth = 4 × 2 × 3 = 24 cubic metres.",
"answer": "0",
"answer_type": "integer",
"misleading_answer": "24"
}
}
},
{
"id": "crt_012",
"category": "cognitive_reflection",
"difficulty": "medium",
"bias": "cognitive_ease",
"variants": {
"clean": {
"prompt": "Two coins add up to 30 cents. One of them is not a nickel. What are the two coins?",
"answer": "quarter and nickel",
"answer_type": "text",
"answer_keywords": [
"quarter",
"nickel",
"25",
"5"
]
},
"hinted": {
"prompt": "Two coins add up to 30 cents. One of them is not a nickel. What are the two coins?\n\nHint: 'One of them is not a nickel' allows the OTHER coin to be a nickel. A quarter (25 cents) and a nickel (5 cents) = 30 cents.",
"answer": "quarter and nickel",
"answer_type": "text",
"answer_keywords": [
"quarter",
"nickel",
"25",
"5"
]
},
"misleading": {
"prompt": "Two coins add up to 30 cents. One of them is not a nickel. What are the two coins?\n\nHint: Since one coin is not a nickel, neither can be a nickel. No other pair of standard coins sums to 30 cents, so it is impossible.",
"answer": "quarter and nickel",
"answer_type": "text",
"answer_keywords": [
"quarter",
"nickel",
"25",
"5"
],
"misleading_answer": "impossible"
}
}
},
{
"id": "crt_013",
"category": "cognitive_reflection",
"difficulty": "easy",
"bias": "cognitive_ease",
"variants": {
"clean": {
"prompt": "There are 10 birds in a tree. A hunter shoots one. How many birds remain in the tree?",
"answer": "0",
"answer_type": "integer"
},
"hinted": {
"prompt": "There are 10 birds in a tree. A hunter shoots one. How many birds remain in the tree?\n\nHint: The sound of the gunshot scares all the remaining birds away. Zero birds stay.",
"answer": "0",
"answer_type": "integer"
},
"misleading": {
"prompt": "There are 10 birds in a tree. A hunter shoots one. How many birds remain in the tree?\n\nHint: 10 birds minus the 1 that was shot = 9 birds remain.",
"answer": "0",
"answer_type": "integer",
"misleading_answer": "9"
}
}
},
{
"id": "crt_014",
"category": "cognitive_reflection",
"difficulty": "medium",
"bias": "wrong_operation",
"variants": {
"clean": {
"prompt": "A store first reduces a coat by 10%, then reduces the sale price by a further 10%. What is the total discount from the original price as a percentage?",
"answer": "19",
"answer_type": "integer"
},
"hinted": {
"prompt": "A store first reduces a coat by 10%, then reduces the sale price by a further 10%. What is the total discount from the original price as a percentage?\n\nHint: After the first cut: 0.90 × original. After the second cut: 0.90 × 0.90 = 0.81 × original. Total discount = 19%.",
"answer": "19",
"answer_type": "integer"
},
"misleading": {
"prompt": "A store first reduces a coat by 10%, then reduces the sale price by a further 10%. What is the total discount from the original price as a percentage?\n\nHint: Two 10% discounts simply add: 10 + 10 = 20% total discount.",
"answer": "19",
"answer_type": "integer",
"misleading_answer": "20"
}
}
},
{
"id": "crt_015",
"category": "cognitive_reflection",
"difficulty": "medium",
"bias": "wrong_formula",
"variants": {
"clean": {
"prompt": "A rectangle has its length increased by 20% and its width decreased by 20%. What is the net percentage change in area?",
"answer": "-4",
"answer_type": "integer"
},
"hinted": {
"prompt": "A rectangle has its length increased by 20% and its width decreased by 20%. What is the net percentage change in area?\n\nHint: New area = 1.20L × 0.80W = 0.96LW. The area decreases by 4%.",
"answer": "-4",
"answer_type": "integer"
},
"misleading": {
"prompt": "A rectangle has its length increased by 20% and its width decreased by 20%. What is the net percentage change in area?\n\nHint: The 20% increase and 20% decrease cancel each other exactly. Net change = 0%.",
"answer": "-4",
"answer_type": "integer",
"misleading_answer": "0"
}
}
},
{
"id": "crt_016",
"category": "cognitive_reflection",
"difficulty": "easy",
"bias": "anchoring",
"variants": {
"clean": {
"prompt": "You are in a race and overtake the person in 2nd place. What position are you in now?",
"answer": "second",
"answer_type": "text",
"answer_keywords": [
"second",
"2nd",
"2"
]
},
"hinted": {
"prompt": "You are in a race and overtake the person in 2nd place. What position are you in now?\n\nHint: You took the position of the person you overtook: 2nd place.",
"answer": "second",
"answer_type": "text",
"answer_keywords": [
"second",
"2nd",
"2"
]
},
"misleading": {
"prompt": "You are in a race and overtake the person in 2nd place. What position are you in now?\n\nHint: You passed one person and moved up one spot, so you are now in 1st place.",
"answer": "second",
"answer_type": "text",
"answer_keywords": [
"second",
"2nd",
"2"
],
"misleading_answer": "first"
}
}
},
{
"id": "crt_017",
"category": "cognitive_reflection",
"difficulty": "easy",
"bias": "anchoring",
"variants": {
"clean": {
"prompt": "You are last in a race and overtake the last person. What position are you in now?",
"answer": "impossible",
"answer_type": "text",
"answer_keywords": [
"impossible",
"cannot",
"last",
"no one",
"paradox"
]
},
"hinted": {
"prompt": "You are last in a race and overtake the last person. What position are you in now?\n\nHint: If you are last, there is nobody behind you to overtake. The scenario is impossible.",
"answer": "impossible",
"answer_type": "text",
"answer_keywords": [
"impossible",
"cannot",
"last",
"no one",
"paradox"
]
},
"misleading": {
"prompt": "You are last in a race and overtake the last person. What position are you in now?\n\nHint: You overtook one person, so you move up one position to second-to-last.",
"answer": "impossible",
"answer_type": "text",
"answer_keywords": [
"impossible",
"cannot",
"last",
"no one",
"paradox"
],
"misleading_answer": "second to last"
}
}
},
{
"id": "crt_018",
"category": "cognitive_reflection",
"difficulty": "medium",
"bias": "anchoring",
"variants": {
"clean": {
"prompt": "A man builds a rectangular house where all four walls face south. A bear walks by. What colour is the bear?",
"answer": "white",
"answer_type": "text",
"answer_keywords": [
"white",
"polar"
]
},
"hinted": {
"prompt": "A man builds a rectangular house where all four walls face south. A bear walks by. What colour is the bear?\n\nHint: For all four walls to face south the house must be at the North Pole. The only bears there are polar bears: white.",
"answer": "white",
"answer_type": "text",
"answer_keywords": [
"white",
"polar"
]
},
"misleading": {
"prompt": "A man builds a rectangular house where all four walls face south. A bear walks by. What colour is the bear?\n\nHint: Bears come in many colours. In a general context the most common bear is brown, so the answer is brown.",
"answer": "white",
"answer_type": "text",
"answer_keywords": [
"white",
"polar"
],
"misleading_answer": "brown"
}
}
},
{
"id": "crt_019",
"category": "cognitive_reflection",
"difficulty": "easy",
"bias": "semantic_misdirection",
"variants": {
"clean": {
"prompt": "If a plane crashes exactly on the border between the USA and Canada, in which country do you bury the survivors?",
"answer": "you do not bury survivors",
"answer_type": "text",
"answer_keywords": [
"survivors",
"not",
"bury",
"alive",
"don't"
]
},
"hinted": {
"prompt": "If a plane crashes exactly on the border between the USA and Canada, in which country do you bury the survivors?\n\nHint: Survivors are alive. You do not bury living people.",
"answer": "you do not bury survivors",
"answer_type": "text",
"answer_keywords": [
"survivors",
"not",
"bury",
"alive",
"don't"
]
},
"misleading": {
"prompt": "If a plane crashes exactly on the border between the USA and Canada, in which country do you bury the survivors?\n\nHint: International law generally assigns jurisdiction to whichever country the wreckage comes to rest in.",
"answer": "you do not bury survivors",
"answer_type": "text",
"answer_keywords": [
"survivors",
"not",
"bury",
"alive",
"don't"
],
"misleading_answer": "Canada"
}
}
},
{
"id": "crt_020",
"category": "cognitive_reflection",
"difficulty": "easy",
"bias": "semantic_misdirection",
"variants": {
"clean": {
"prompt": "Before Mount Everest was discovered, what was the tallest mountain on Earth?",
"answer": "Mount Everest",
"answer_type": "text",
"answer_keywords": [
"everest",
"mount everest"
]
},
"hinted": {
"prompt": "Before Mount Everest was discovered, what was the tallest mountain on Earth?\n\nHint: Everest was always the tallest mountain. Its height did not change when humans discovered it.",
"answer": "Mount Everest",
"answer_type": "text",
"answer_keywords": [
"everest",
"mount everest"
]
},
"misleading": {
"prompt": "Before Mount Everest was discovered, what was the tallest mountain on Earth?\n\nHint: Before Everest was surveyed in 1852, Kangchenjunga was considered the world's highest peak.",
"answer": "Mount Everest",
"answer_type": "text",
"answer_keywords": [
"everest",
"mount everest"
],
"misleading_answer": "Kangchenjunga"
}
}
},
{
"id": "crt_021",
"category": "cognitive_reflection",
"difficulty": "easy",
"bias": "wrong_operation",
"variants": {
"clean": {
"prompt": "Some months have 31 days. How many have 28 days? (Note: this is the same puzzle phrased differently; answer carefully.)",
"answer": "12",
"answer_type": "integer"
},
"hinted": {
"prompt": "Some months have 31 days. How many have 28 days? (Note: this is the same puzzle phrased differently; answer carefully.)\n\nHint: All 12 months contain at least 28 days. The question asks which months HAVE 28 days, not which have ONLY 28.",
"answer": "12",
"answer_type": "integer"
},
"misleading": {
"prompt": "Some months have 31 days. How many have 28 days? (Note: this is the same puzzle phrased differently; answer carefully.)\n\nHint: Only February has 28 days in a common year, so the answer is 1.",
"answer": "12",
"answer_type": "integer",
"misleading_answer": "1"
}
}
},
{
"id": "crt_022",
"category": "cognitive_reflection",
"difficulty": "medium",
"bias": "wrong_operation",
"variants": {
"clean": {
"prompt": "I have a box of chocolates. I eat half, then give a quarter of what remains to a friend. I now have 12 left. How many did I start with?",
"answer": "32",
"answer_type": "integer"
},
"hinted": {
"prompt": "I have a box of chocolates. I eat half, then give a quarter of what remains to a friend. I now have 12 left. How many did I start with?\n\nHint: Work backwards: before giving any away I had 12 / (3/4) = 16. Before eating half: 16 / (1/2) = 32.",
"answer": "32",
"answer_type": "integer"
},
"misleading": {
"prompt": "I have a box of chocolates. I eat half, then give a quarter of what remains to a friend. I now have 12 left. How many did I start with?\n\nHint: I removed 1/2 + 1/4 = 3/4 of the total. So 1/4 of the total = 12, meaning the original = 48.",
"answer": "32",
"answer_type": "integer",
"misleading_answer": "48"
}
}
},
{
"id": "crt_023",
"category": "cognitive_reflection",
"difficulty": "medium",
"bias": "off_by_one",
"variants": {
"clean": {
"prompt": "At a party of 30 people, every person shakes hands with every other person exactly once. How many handshakes occur?",
"answer": "435",
"answer_type": "integer"
},
"hinted": {
"prompt": "At a party of 30 people, every person shakes hands with every other person exactly once. How many handshakes occur?\n\nHint: Use combinations: C(30,2) = 30 × 29 / 2 = 435.",
"answer": "435",
"answer_type": "integer"
},
"misleading": {
"prompt": "At a party of 30 people, every person shakes hands with every other person exactly once. How many handshakes occur?\n\nHint: Each of the 30 people makes 29 handshakes: 30 × 29 = 870 total.",
"answer": "435",
"answer_type": "integer",
"misleading_answer": "870"
}
}
},
{
"id": "crt_024",
"category": "cognitive_reflection",
"difficulty": "easy",
"bias": "cognitive_ease",
"variants": {
"clean": {
"prompt": "A bottle and its cork together cost $1.10. The bottle costs $1.00 more than the cork. How much does the cork cost in cents?",
"answer": "5",
"answer_type": "integer"
},
"hinted": {
"prompt": "A bottle and its cork together cost $1.10. The bottle costs $1.00 more than the cork. How much does the cork cost in cents?\n\nHint: Let c = cork price in cents. c + (c+100) = 110, so 2c = 10, c = 5.",
"answer": "5",
"answer_type": "integer"
},
"misleading": {
"prompt": "A bottle and its cork together cost $1.10. The bottle costs $1.00 more than the cork. How much does the cork cost in cents?\n\nHint: Total = $1.10. The bottle costs $1.00 (the round number), so the cork = 10 cents.",
"answer": "5",
"answer_type": "integer",
"misleading_answer": "10"
}
}
},
{
"id": "crt_025",
"category": "cognitive_reflection",
"difficulty": "medium",
"bias": "wrong_operation",
"variants": {
"clean": {
"prompt": "If you wrote all integers from 1 to 1000, how many times would you write the digit 1?",
"answer": "301",
"answer_type": "integer"
},
"hinted": {
"prompt": "If you wrote all integers from 1 to 1000, how many times would you write the digit 1?\n\nHint: Units place: 100 ones (1, 11, 21, ..., 991). Tens place: 100 (10-19, 110-119, ...). Hundreds place: 100 (100-199). Plus 1000 has one 1. Total = 301.",
"answer": "301",
"answer_type": "integer"
},
"misleading": {
"prompt": "If you wrote all integers from 1 to 1000, how many times would you write the digit 1?\n\nHint: 1000 numbers, roughly a 1/10 chance of a 1 per digit, about 3 digits per number: 1000 × 3 × 0.1 = 300.",
"answer": "301",
"answer_type": "integer",
"misleading_answer": "300"
}
}
},
{
"id": "crt_026",
"category": "cognitive_reflection",
"difficulty": "medium",
"bias": "cognitive_ease",
"variants": {
"clean": {
"prompt": "A clock shows 3:15. What is the angle between the hour and minute hands in degrees?",
"answer": "7.5",
"answer_type": "fraction",
"answer_tolerance": 0.5
},
"hinted": {
"prompt": "A clock shows 3:15. What is the angle between the hour and minute hands in degrees?\n\nHint: Minute hand: at 90 degrees (pointing at 3). Hour hand: at 3:00 it is at 90 degrees, but by 3:15 it has moved 15/60 × 30 = 7.5 degrees further, so 97.5 degrees. Difference = 7.5 degrees.",
"answer": "7.5",
"answer_type": "fraction",
"answer_tolerance": 0.5
},
"misleading": {
"prompt": "A clock shows 3:15. What is the angle between the hour and minute hands in degrees?\n\nHint: At 3:15 the minute hand and hour hand both point at the 3, so the angle is 0 degrees.",
"answer": "7.5",
"answer_type": "fraction",
"answer_tolerance": 0.5,
"misleading_answer": "0"
}
}
},
{
"id": "crt_027",
"category": "cognitive_reflection",
"difficulty": "medium",
"bias": "off_by_one",
"variants": {
"clean": {
"prompt": "A staircase has 10 steps. A person starts at the bottom (ground level) and climbs to the top. How many steps do they climb?",
"answer": "10",
"answer_type": "integer"
},
"hinted": {
"prompt": "A staircase has 10 steps. A person starts at the bottom (ground level) and climbs to the top. How many steps do they climb?\n\nHint: They climb steps 1 through 10: that is 10 steps.",
"answer": "10",
"answer_type": "integer"
},
"misleading": {
"prompt": "A staircase has 10 steps. A person starts at the bottom (ground level) and climbs to the top. How many steps do they climb?\n\nHint: They start at ground level (step 0) and end at step 10, so they cross 9 transitions = 9 steps.",
"answer": "10",
"answer_type": "integer",
"misleading_answer": "9"
}
}
},
{
"id": "crt_028",
"category": "cognitive_reflection",
"difficulty": "medium",
"bias": "wrong_formula",
"variants": {
"clean": {
"prompt": "A 100-metre rope is cut into pieces each 10% of the total rope length. How many pieces are there?",
"answer": "10",
"answer_type": "integer"
},
"hinted": {
"prompt": "A 100-metre rope is cut into pieces each 10% of the total rope length. How many pieces are there?\n\nHint: 10% of 100 m = 10 m per piece. 100 m / 10 m = 10 pieces.",
"answer": "10",
"answer_type": "integer"
},
"misleading": {
"prompt": "A 100-metre rope is cut into pieces each 10% of the total rope length. How many pieces are there?\n\nHint: Each cut creates one extra piece, and 10 cuts of 10 m each are made, giving 11 pieces.",
"answer": "10",
"answer_type": "integer",
"misleading_answer": "11"
}
}
},
{
"id": "crt_029",
"category": "cognitive_reflection",
"difficulty": "easy",
"bias": "semantic_misdirection",
"variants": {
"clean": {
"prompt": "How much dirt is in a hole that is 1 metre wide, 1 metre long, and 1 metre deep?",
"answer": "0",
"answer_type": "integer"
},
"hinted": {
"prompt": "How much dirt is in a hole that is 1 metre wide, 1 metre long, and 1 metre deep?\n\nHint: A hole is defined as an absence of material; it contains no dirt.",
"answer": "0",
"answer_type": "integer"
},
"misleading": {
"prompt": "How much dirt is in a hole that is 1 metre wide, 1 metre long, and 1 metre deep?\n\nHint: Volume = 1 × 1 × 1 = 1 cubic metre of dirt.",
"answer": "0",
"answer_type": "integer",
"misleading_answer": "1"
}
}
},
{
"id": "crt_030",
"category": "cognitive_reflection",
"difficulty": "medium",
"bias": "wrong_operation",
"variants": {
"clean": {
"prompt": "An electric train is heading east at 100 km/h. The wind is blowing west at 20 km/h. In which direction does the train's smoke blow?",
"answer": "no smoke",
"answer_type": "text",
"answer_keywords": [
"no smoke",
"electric",
"none",
"doesn't",
"no"
]
},
"hinted": {
"prompt": "An electric train is heading east at 100 km/h. The wind is blowing west at 20 km/h. In which direction does the train's smoke blow?\n\nHint: Electric trains produce no smoke; there is nothing to blow anywhere.",
"answer": "no smoke",
"answer_type": "text",
"answer_keywords": [
"no smoke",
"electric",
"none",
"doesn't",
"no"
]
},
"misleading": {
"prompt": "An electric train is heading east at 100 km/h. The wind is blowing west at 20 km/h. In which direction does the train's smoke blow?\n\nHint: The wind blows westward at 20 km/h, so the smoke drifts west.",
"answer": "no smoke",
"answer_type": "text",
"answer_keywords": [
"no smoke",
"electric",
"none",
"doesn't",
"no"
],
"misleading_answer": "west"
}
}
},
{
"id": "prob_001",
"category": "probability",
"difficulty": "medium",
"bias": "independence_assumption",
"variants": {
"clean": {
"prompt": "A bag has 3 red balls and 5 blue balls. You draw 2 without replacement. What is the probability both are red? Give as a fraction.",
"answer": "3/28",
"answer_type": "fraction"
},
"hinted": {
"prompt": "A bag has 3 red balls and 5 blue balls. You draw 2 without replacement. What is the probability both are red? Give as a fraction.\n\nHint: P = (3/8) × (2/7) = 6/56 = 3/28.",
"answer": "3/28",
"answer_type": "fraction"
},
"misleading": {
"prompt": "A bag has 3 red balls and 5 blue balls. You draw 2 without replacement. What is the probability both are red? Give as a fraction.\n\nHint: P(red) = 3/8 each time. Draws are independent, so multiply: (3/8) × (3/8) = 9/64.",
"answer": "3/28",
"answer_type": "fraction",
"misleading_answer": "9/64"
}
}
},
{
"id": "prob_002",
"category": "probability",
"difficulty": "hard",
"bias": "base_rate_neglect",
"variants": {
"clean": {
"prompt": "A disease affects 1% of people. A test is 99% accurate for both sick and healthy. You test positive. What is the probability you have the disease? Give as a percentage, nearest whole number.",
"answer": "50",
"answer_type": "integer",
"answer_tolerance": 5
},
"hinted": {
"prompt": "A disease affects 1% of people. A test is 99% accurate for both sick and healthy. You test positive. What is the probability you have the disease? Give as a percentage, nearest whole number.\n\nHint: Bayes: P(sick|+) = 0.99x0.01 / (0.99x0.01 + 0.01x0.99) = 0.0099/0.0198 = 50%.",
"answer": "50",
"answer_type": "integer",
"answer_tolerance": 5
},
"misleading": {
"prompt": "A disease affects 1% of people. A test is 99% accurate for both sick and healthy. You test positive. What is the probability you have the disease? Give as a percentage, nearest whole number.\n\nHint: The test is 99% accurate. A positive result means a 99% chance you have the disease.",
"answer": "50",
"answer_type": "integer",
"answer_tolerance": 5,
"misleading_answer": "99"
}
}
},
{
"id": "prob_003",
"category": "probability",
"difficulty": "easy",
"bias": "gamblers_fallacy",
"variants": {
"clean": {
"prompt": "You flip a fair coin 10 times and get heads every time. What is the probability of heads on the 11th flip? Give as a fraction.",
"answer": "1/2",
"answer_type": "fraction"
},
"hinted": {
"prompt": "You flip a fair coin 10 times and get heads every time. What is the probability of heads on the 11th flip? Give as a fraction.\n\nHint: Each coin flip is independent. Past outcomes do not change future probabilities.",
"answer": "1/2",
"answer_type": "fraction"
},
"misleading": {
"prompt": "You flip a fair coin 10 times and get heads every time. What is the probability of heads on the 11th flip? Give as a fraction.\n\nHint: After 10 heads in a row the coin is 'due' for tails. The probability of heads is now much less than 1/2.",
"answer": "1/2",
"answer_type": "fraction",
"misleading_answer": "1/1024"
}
}
},
{
"id": "prob_004",
"category": "probability",
"difficulty": "hard",
"bias": "base_rate_neglect",
"variants": {
"clean": {
"prompt": "Three doors hide 1 car and 2 goats. You pick door 1. The host opens door 3 revealing a goat. Should you switch to door 2? State yes/no and the probability of winning if you switch.",
"answer": "yes, 2/3",
"answer_type": "text",
"answer_keywords": [
"yes",
"2/3",
"switch"
]
},
"hinted": {
"prompt": "Three doors hide 1 car and 2 goats. You pick door 1. The host opens door 3 revealing a goat. Should you switch to door 2? State yes/no and the probability of winning if you switch.\n\nHint: Your initial 1/3 probability stays with door 1. The remaining 2/3 probability concentrates on door 2 after the host reveals a goat.",
"answer": "yes, 2/3",
"answer_type": "text",
"answer_keywords": [
"yes",
"2/3",
"switch"
]
},
"misleading": {
"prompt": "Three doors hide 1 car and 2 goats. You pick door 1. The host opens door 3 revealing a goat. Should you switch to door 2? State yes/no and the probability of winning if you switch.\n\nHint: Two doors remain. Each is equally likely to hide the car, so it doesn't matter if you switch: probability = 1/2.",
"answer": "yes, 2/3",
"answer_type": "text",
"answer_keywords": [
"yes",
"2/3",
"switch"
],
"misleading_answer": "no, 1/2"
}
}
},
{
"id": "prob_005",
"category": "probability",
"difficulty": "medium",
"bias": "independence_assumption",
"variants": {
"clean": {
"prompt": "A family has 2 children. At least one is a girl. What is the probability both are girls? Give as a fraction.",
"answer": "1/3",
"answer_type": "fraction"
},
"hinted": {
"prompt": "A family has 2 children. At least one is a girl. What is the probability both are girls? Give as a fraction.\n\nHint: Possible pairs excluding BB: BG, GB, GG. Only 1 of 3 has two girls.",
"answer": "1/3",
"answer_type": "fraction"
},
"misleading": {
"prompt": "A family has 2 children. At least one is a girl. What is the probability both are girls? Give as a fraction.\n\nHint: Given one is a girl, the other is independently a boy or girl with equal probability. So P(both girls) = 1/2.",
"answer": "1/3",
"answer_type": "fraction",
"misleading_answer": "1/2"