Alicia PhD

Alicia PhD
Location
New Hampshire, United States
Birthday
September 08
Bio
Alicia has a PhD in Experimental Pathology and, after having worked in a genetics lab for her dissertation, now edits scientific manuscripts full-time from the comfort of the White Mountains. Alicia is also a writer, contributing health commentary and articles on disease and anatomy to many online publishers. She upkeeps a number of blogs devoted to her interests in public health and science.

MY RECENT POSTS

APRIL 5, 2010 5:01PM

The Significance of Significance

Rate: 6 Flag

I've been uber busy lately, but I was just reading this article on the use of statistics in science and wanted to share it.

It's from Science News on March 27: Odds Are, It's Wrong

It's a discussion of how reliant scientists have become on statistics even while not fully understanding how to use statistics, and how statistical significance has been misinterpreted as real world significance. 

 This topic appeals to me because it was something I struggled with as a graduate student. My dissertation work required statistical evaluations of genetic associations. I even had a biostatistician on my dissertation committee because my advisor was dead-set on it being done right. I met with him time and time again, trying to understand what I was testing and why. I had to limit my lessons to only what I needed for my work, fully planning to expand and learn it better afterwards. Sorry to say, I don't even remember the appropriate methods used to compared results from my own dissertation - statistics are just beyond my long-term comprehension skills (I'm also not good at calculus and physics!) 

But this is what science needs - dedicated math people to work alongside bench scientists, ensuring that the results are properly compared and interpreted. There also needs to be that moment of teaching where a future scientist is told "statistical significance is not significant on its own". 

That article is very good and very important, and I just wanted to share. 

Your tags:

TIP:

Enter the amount, and click "Tip" to submit!
Recipient's email address:
Personal message (optional):

Your email address:

Comments

Type your comment below:
First, I like your title -- what is the significance of significance is a great question that too often gets overlooked.

Statistics are so easily misread unintentionally and so easily misused intentionally that they are an important entity that must be understood fully to have any value at all.

Good post.

RATED
I am definitely not a statistician, and some of the discussions of statistics and the bell curve in natural medicine showed us to look at what is actually being found, and why it is or isn't deemed important. There are patients who fall on the bell curve, or within it, and then a whole bunch who don't. Often, they are the ones we get to see. Nothing more frustrating than a doc who tells you nothing is wrong because they can't find anything and they haven't even looked in places they didn't think to look. With statistics, we learn often what is "likely or probable", and forget what that the numbers of "actual and unlikely but still happened" are also attached to real people. Those patients always find themselves both relevant and significant.
I am definitely not a statistician, and some of the discussions of statistics and the bell curve in natural medicine showed us to look at what is actually being found, and why it is or isn't deemed important. There are patients who fall on the bell curve, or within it, and then a whole bunch who don't. Often, they are the ones we get to see. Nothing more frustrating than a doc who tells you nothing is wrong because they can't find anything and they haven't even looked in places they didn't think to look. With statistics, we learn often what is "likely or probable", and forget what that the numbers of "actual and unlikely but still happened" are also attached to real people. Those patients always find themselves both relevant and significant.
43% of all statistics are made up on the spot
DH, 80% of what you said is correct lol

Rick, thanks!

Oryoki, I've had numerous arguments with people along those same lines. "the odds are against it!" "but that doesn't change the fact that it happened!" "but the odds are against it!" "but. it. happened. Something has to happen, and yes, the odds are against any particular thing on its own, but overall, something. happened. and this. was. it."
nice article. yeah statistics is full of subtleties. in my opinion a lot of the subtlety is around "false positive vs false negative" & a course that emphasizes these will cover most of the key issues. I had a serious college sophomore level statistics class.. didnt you? but yeah, even an experienced statistician can get tripped up on how to state results accurately etc.
ps have an article on the use of statistics & gaussian curves in finance in my blog, some here might enjoy it
I agree that doing statistics right is important. (I'm not a statistician, but I do try to keep in mind the difference between statistical significance and real-world significance). I do think that the author of the Science News piece seriously overstates things here, though:

The “scientific method” of testing hypotheses by statistical analysis stands on a flimsy foundation. Statistical tests are supposed to guide scientists in judging whether an experimental result reflects some real effect or is merely a random fluke, but the standard methods mix mutually inconsistent philosophies and offer no meaningful basis for making such decisions.

I don't think this is at all the case. Sure, there are arguments in the statistical literature (e.g. between frequentists and Bayesians), but the author might as well say that because of disagreements between idealists, realists, and nominalists, science is also a "mix of mutually inconsistent philosophies and offer[s] no meaningful basis for making such decisions." I mean, we don't even have a universally agreed upon philosophical justification for induction!
Thanks for the referral. My background is in finance, but statistics kicked my rump. I'm not sure if I'm intrigued because of it's value or because it almost defeated me. At any rate, I've bookmarked the article so I can read it later. Trying to catch up in OS. Thanks again for always offering something of value.
GREAT topic. With type 1 and type 2 errors abounding in published scientific literature, its certainly worth a serious discussion... We all could use some stats refreshers now and then - no matter how far along in our research careers.

thanks!