Tuesday, June 4, 2013

Confirmation Bias, Student Grades, and Prayer

This is the season, after grades have been submitted, when student emails complaining about grades come in. A particular pattern emerges. I couldn't resist writing it up to use in my Critical Thinking classes. The other obvious application of the Confirmation Bias point is to the way people often think about prayer.

Confirmation Bias is the mistake of selecting evidence that corroborates a pet hypothesis while ignoring or neglecting evidence that would disprove it. Humans are guilty of committing it in a wide range of circumstances. At the end of each semester, many students, including students in Critical Thinking and Theory of Knowledge where we study confirmation bias extensively, blunder into it. They get a grade for the course that is surprisingly low and send an email to their professor asking to know what happened. As far as they knew, they were doing great in the course. They recall getting an A on an assignment, and going ok on the midterm, and feeling pretty optimistic, so they can’t understand the low grade. Notice the parts highlighted in red here (a real email):

 Student 1: I just checked my grades for the Spring semester and was surprised to have earned an F grade. I completed the major assignments for the course and did well on the midterm (90%) and well on the final (85%). I know I didn't participate in the online forum as much as was required but I'm still confused about the grade. I took the class material seriously and did my best on every assignment assigned.

Professor McCormick (notice the grades highlighted in red here): Here are the grades I have for you. This syllabus gives the details about the grade structure. Check the math and check your returned assignments to make sure it's all right. If there's a clerical error, I'll fix it right away:
Question Sets: 0, 82, 0, 75, 0, 95 (6% each)
First paper: 78
Midterm: 90.5
Second paper: 85
Final: 85
Outside projects: 7/8
Google Group: 0/8 
Attendance and participation: 0/8 

So between the skipped question sets, the Group discussion and attendance, you gave up 34% of the grade. Even if you were making an A on everything else, that would put it down to a D.

 Student 2: I'm emailing you in regards to my final grades. I was hoping you could provide a detailed summary of my grades for the semester so that I can understand how I received a D. I felt as though I did fairly well, particularly improving on the more major assignments, so I would just like to know how I still failed to pass. If you could email me a detailed summary of my grades, I would greatly appreciate it. Thank you.

 Professor McCormick: Yeah, I was disappointed in your grades too. It seemed to me that you are capable of doing much better work, and being more responsible about turning stuff in. Here are the grades I have. Check the math with the grade structure on the syllabus and let me know if there is a clerical error asap: 

Question sets: 76, 0, 82, 0, 85, 56, 80 (6% each)
Evil paper 65
Midterm 68.5 
Second paper: 75
Final exam: 85
Outside Projects: 6/8
Google Groups: 4/8 
Attendance and participation: 6/8

So the skipped question sets took 12% off of your grade. You got a D on the first paper and didn't take the opportunity to rewrite it that I gave the class. You could have brought that up substantially. The Google Group points would have helped too since your overall score came out at 68%.

When we commit confirmation bias, we cherry pick the evidence that suits us. The student actively remembers the good grades, but missed assignments and low scores are forgotten. But clearly, having an accurate and objective grasp of the relevant evidence would serve us well. We don’t want to ignore evidence indicating something negative, disastrous, or dangerous because it doesn’t suit what we want to be true. Imagine if a doctor acquired a skewed view of the evidence concerning a potentially fatal disease this way. Suppose the Secretary of State ignored significant negative indicators in the behavior of an aggressive and hostile foreign country. Suppose a potential employer asked you how you did in your Critical Thinking course in college, and then she checked your transcripts against your distorted report. Suppose you spend thousands of dollars over the years on losing lottery tickets because the occasional wins stick out in your mind so prominently, while the loses are forgotten.  Suppose you spend time praying to God frequently, hundreds or thousands of times in your life, and on the rare occasion when something vaguely resembling what you prayed for came true, you count that as an answered prayer, while ignoring the thousands of misses.  Suppose you sustain your belief in God for years on the basis of this mistake.


Unknown said...
This comment has been removed by a blog administrator.
Ed said...

Hi Matt--
Re "student grades" post, you may be interested in Wikipedia article, "Dunning-Kruger effect."

Matt McCormick said...

Thanks Ed. Dunning-Kruger is a different sort of mistake. I teach that one in the same class. But I thought the grade emails were a particularly vivid illustration of confirmation bias worth sharing.

Andrew Hall said...

Excellent real world examples of how confirmation bias leads people astray.

Starchild646 said...

"I believe that the characteristic feature of primitive man is that he does not act from opinions (contrary to Frazer). I read/ among many similar examples/ of a Rain-King in Africa to whom people pray when the rainy period comes. But surely they do not believe that he can make it rain/ otherwise they would do it in the dry periods of the year in which the land is a "parched and arid desert." For if one assumes that the people formerly instituted this office of Rain-King out of stupidity/ it is nevertheless certainly clear that they had previously experienced that the rains begin in March and then they would have had the Rain-King function for the other part of the year. Or again: toward morning when the sun is about to rise/ rites of daybreak are celebrated by the people/ but not during the night/ when they simply burn lamps."

L. Wittgenstein
Remark's on Frazer's Golden Bough

If there was a practice that people engaged in/ and I constructed a hypothesis as to it's purpose and then I proved to people that it didn't accomplish it's purpose. . . and people continued the practice anyways (say sodomy) would I give up my hypothesis about its purpose or would I cling to it and conclude that the people were simply ignorant?

Unknown said...

Reminds me of my Philosophy Professor who told us that if we were not happy with the grade, we could talked to her about it. But she reminded us that it was a philosophy class and any argument for a grade change had better be valid and sound. She also reminded us that she could change the grade up or down. To my knowledge nobody asked to have their grade changed.

Matt said...


I am totally stealing this for use in my critical thinking class. With a little (possibly ill-placed) luck, I can head off the same problem. :P

Matt H

Unknown said...

Hey Matt,
I'm not sure if Bob H was a student in my class, or whether he's had another prof who used this same approach, but I use it in all my classes. I am certainly happy to clarify why a student received the grade she/he did, but to change it requires an argument. Then I tell them the story of my first year Chemistry course where that professor accepted my exam for grade review, agreed the TA neglected to credit me a point where I noted it was due, but found the TA was overly generous elsewhere. Made me more attentive to my own confirmation bias (though I didn't know at the time it had a name). Also let me know that grades can go up, but they can also down... I don't get many challenges about grades.

B Berry said...

{THE Copied and pasted commentary below is of the sounding call of God's finished Mystery Rev:10:7, contact the sites when the sounding signs come} Go by the blog that Richard Dawkins could not answer the simple question asked atheists and Christians, one question that must have three answers.. Go to the Historian faith and atheism debate symposium blog found at most search engines, also visit the blog wherein God cannot be denied at Adam and Eve in Action blog where in you can find free ebooks on atheism and religion.. Also go to The Historian faith and atheism debate symposium blog free ebooks on atheism and religion as well as proofs.. Amazing! THE Righteous Word researcher at Adam and Eve seed gathering Word research Ministry has discovered all righteousness in Word from God to God's science {And the purpose for viewing the Galaxies} And is now preparing the end of aging and premature death with the combined word of Knowledge given us by the Only God! For evidence conclusive that all Etymologies have been to end the disease contracted first by Eve our Mother in the Garden go to the websites now mentioned below in past Space.com/yahoo news commentary now copied and pasted below... It has Begun ... Our Lord's science has proved His mystery Revelation 10:7 and instead His children of Science deny Him while reaping the benefits of His delivered Knowledge! For evidence that He and He alone brought forth Science go to the Proof in the news and science page, and the Targum page and the Proof of His healing in painting and verse page on the website at google search of Adam and Eve seed gathering Ministry!

Also on the Prophecy and the Signs page are the prewritten news events of this day prewritten four years ago and some more, and are now falling each week, such as the Japan quake and tidal wave, and the DC/Virginia Beach Virginia quake our Lord Jesus the Great scientist/physician brought on the same day as the unveiling of the martin luther king statue { with as we speak more quakes are coming to DC/V.Beach, as well as three meteors to Sudan Africa } also the record disturbances on the Suns surface all prophecy before their taking place so all of Science and Theology will know the Minister of the site is sent John 13:20 and a Righteous man! Many other signs listed there have come and are coming for this sounding to Science of God's Mystery being now finished by Science Revelation 10:7 for other proofs contact the minister for questions or more answers! Also go to the site at Adam and Eve in Action blog at google search, and therein are videos and links and pages of undeniable evidence that our Lord now prepares His earth for His retaking of it Matthew 5:5 and 25:34 Rev:5:10 and 21:3-4.... Respect to all scientists of Adam/God's seed/of Science/Nasa http://adamandeveinaction.blogspot.com/ http://www.adamandeveseedgatheringministry.com/prophecy_and_the_signs

Anonymous said...

Suppose God gives you a mouth to answer your own prayers and you ignore.

Anonymous said...

Suppose God answers prayers before you're born and gives hands yes

Anonymous said...

And when you ask for a sign, ask desperately truthfully, don't ask based on humans (like if you're true, make him move his left arm) because God could say do that and he refuse. . .but if you ask it God-Related, and ask for spiritual things, He'll eventually say Yes, it's His job. . .now, let us not be evil without even meeting Him, or He could do the same eventually. . .

Anonymous said...

Suppose He answers prayers only as a secretary, in due time. Suppose He'd answer it in your bias (never), GOD FORBID! Suppose so many things you have not listed, God grant you accept Him because in the end, each goes to heaven or hell based on a "personal interview" judgment where everything is taken into consideration: were you good, did you accept God. All heard of God, He sent angels, burning bushes, disguised angels, . . .