A club wanted to determine whether it could increase attendance by changing its weekly meetings from Tuesday to another day. At one Tuesday meeting, the club's president took a survey of all members present. Of those surveyed, 95 percent said that they had no difficulty attending on Tuesdays. On the basis the attendance problem was not due primarily to schedule conflicts.
What this question is testing
Evidence
The president showed up at Tuesday's meeting, surveyed everyone in the room, and 95% said Tuesdays work just fine for them. Sounds like solid data, right?
Conclusion
The attendance problem is not about scheduling conflicts with Tuesday. The numbers have spoken!
Evaluate
The numbers spoke, but they were only talking to the people who could show up on Tuesday. The members with actual Tuesday conflicts are at home, at work, at their kid's soccer game — anywhere but at this meeting. It is the logical equivalent of standing at the summit of Everest and asking "did anyone find this climb too difficult?" Everyone who found it too difficult is still at base camp, unpolled and very much conflicted. The president built a survey that could only return one answer, then treated that answer as a revelation.
Reading along? Open the full official question in LawHub — we show a fragment here and keep the reasoning in our own words.