How come there would be a significance difference between two levels but that difference does not exist when a post hoc is ran?
posted by Unknown @ 6:08 PM
This blog is for the use of students enrolled in either section of PG851. It is intended to be a forum for asking, discussing, clarifying, and helping.
posted by Unknown @ 6:08 PM
4 Comments:
At 6:25 PM,
Jeffy said…
I'm not sure if I understand your question correctly. Please try to be more specific, but I'll try to answer as best I can. If you are refering to the differences between a t-test and an ANOVA post-hoc test, that differnce is the reason you're running the analysis both ways. The ANOVA with a post-hoc is designed to decrease the probability of making a type 1 error, and is therfore more conservative. Running multiple t-test will inflate the p value and be more liberal. You are more likely to get a significant result due to random chance the more analyses you run unless you account for the increased probability of error. The ANOVA with a post-hoc parcels out that error, meaning a significant result can more likely be trusted. The different post-hoc tests complete this function in different ways for specific situations and with varying levels of success. So when a t-test says there is a significant difference and the ANOVA and post-hoc says there isn't, it's probable that the difference can be accounted for by the way the respective test calculates for error.
At 10:05 PM,
Unknown said…
Jeff, when I ran an ANOVA there was a significance difference but once I ran the post hoc, there was no significance. Am I doing it wrong possibly?
At 10:33 PM,
Unknown said…
In my comparison of two groups they were proven to be significant, but once I ran the post hoc, it showed that they weren't. What's up with that.
At 7:16 AM,
Mari Clements said…
If you have an ANOVA with only two groups, there's no need to run a post-hoc. If you have an ANOVA with MORE than 2 groups, you have no way of knowing where the significant pairwise differences lie without doing a post-hoc, so I don't understand your comment, "In my comparison of two groups they were proven to be significant, but once I ran the post hoc, it showed that they weren't."
If instead you are saying that you compared two groups in a t-test, but then in a post-hoc comparison of a 3 group ANOVA, those same two groups did not significantly differ, then please refer back to your notes about inflation of alpha and Jeff's post above.
It is also possible to get a significant overall finding in an ANOVA, but have none of the pairwise comparisons be significant. This depends on how conservative the post-hoc test you chose was.
As a learning exercise (NOT as something you would ever do in real research), re-run the example we did in class, where some comparisons were significant with one post-hoc procedure but not with other post-hoc procedures.
Post a Comment
<< Home