Great reminder! I also tell my students that sometimes we can get a highly significant finding, e.g., with a probability of occurring less than once in 10,000 times, (p < .001) quite consistently, but we have to ask ourselves the question of whether it is meaningful. As demonstrated in this link, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4456887/ we are talking about a highly reliable difference of about 20 msec for simple reaction times between men (faster) and women (slower). I cannot think of anything we do as human beings where this would be truly meaningful. So we have to be cautious. Also sample sizes can affect significance levels and so on. A BASIC understanding cannot be ignored. I think you did a very nice job of that basic understanding.
Any credit has to go to Emily Oster on this one :-) I rack my brain to come up with ways to play with metaphor to get ideas across and along comes Oster and nails it. I really just wanted to get those ideas across too, and drive people to her site to get more of it :-)
Thanks for the link Annette - so interesting for the 20 msec and also for the fact that the study population was yet another student cohort in the prime of their lives, with probably not much application to the real world anyway.
Great reminder! I also tell my students that sometimes we can get a highly significant finding, e.g., with a probability of occurring less than once in 10,000 times, (p < .001) quite consistently, but we have to ask ourselves the question of whether it is meaningful. As demonstrated in this link, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4456887/ we are talking about a highly reliable difference of about 20 msec for simple reaction times between men (faster) and women (slower). I cannot think of anything we do as human beings where this would be truly meaningful. So we have to be cautious. Also sample sizes can affect significance levels and so on. A BASIC understanding cannot be ignored. I think you did a very nice job of that basic understanding.
Any credit has to go to Emily Oster on this one :-) I rack my brain to come up with ways to play with metaphor to get ideas across and along comes Oster and nails it. I really just wanted to get those ideas across too, and drive people to her site to get more of it :-)
Thanks for the link Annette - so interesting for the 20 msec and also for the fact that the study population was yet another student cohort in the prime of their lives, with probably not much application to the real world anyway.
So glad you’re here.