286 out of 294 psychology papers reported a positive result. Great paper from 1959.

- Edit
- Delete
- Tags
- Autopost
286 out of 294 psychology papers reported a positive result. Great paper from 1959.

This would be a good paper for anyone running a journal club wanting interesting old papers that presage modern issues, like publication bias: 286 out of 294 psychology papers reported a positive result.
http://www.jstor.org/pss/2282137

Contemporary stuff on the same issue here:
http://www.badscience.net/category/publication-bias/
Nothing has changed. In fact, four decades later, the same researcher came back to the same question – a lifetime later - and concluded little had changed.

http://www.jstor.org/stable/2684823
Sterling TD, Rosenbaum WL, Weinkam JJ. Publication decisions revisited – the effect of the outcome of statistical tests on the decision to publish and vice-versa. Am Stat 1995;49:108–12.
- 12 January 2012
- Views
- Permalink
- Tweet
- 5 responses
- Like
- Comment
4 months ago PsychFileDrawer (Twitter) responded:

Now we have the technology to fix this. Post brief reports of replication attempts at www.psychfiledrawer.org - easier than publishing a paper. The hard part is changing the psychology of researchers. It seems to be changing- awareness has been growing rapidly in the last couple years.

4 months ago Finn Årup Nielsen liked this post.

4 months ago muscleguy responded:

The other problem is that when a project returns non significant results there are always lots of post hoc reasons why it was never going to work. Not to mention that more than a couple of such results will likely result in the PI losing their job and the data being lost. When the PI has moved on to a teaching only post under publish or perish, who posts the result?

2 months ago SauropodMike (Twitter) responded:

Another bright sign suggesting with might be coming towards the end of this era: PLoS ONE explicitly established itself as a journal where peer-review is intended only as a filter on quality and soundness, not on "importance" or "impact". Other "Open Access Megajournals" have followed its lead, so there is now a smallish but significant ecosystem of journals that will publish replication studies (whether "successful" or not), rather than rejecting them as lacking novelty, instead filling their limited space with more novel studies that in turn are doomed to go unreplicated.
It's ironic that John Bargh's meltdown on being contradicted focussed so much ire on PLoS ONE, where the study that failed to replicate his own was published: "The Doyen et al. article appeared in an online journal, PLoS ONE, which quite obviously does not receive the usual high scientific journal standards of peer-review scrutiny ... the journal promises a "rigorous peer review" for technical soundness but not as to the importance of the finding."
In fact, that very policy regarding peer-review -- that articles are published or not according to whether they are good rather than whether they are subjectively judged as "important" -- is the reason why an "unimportant" study such as a failed replication attempt was published at all, rather than being shunted aside for lack of impact. Which is ironic again, given how much impact this study has in fact had. If more journals adopted this approach, Bargh's 1996 study might not have had to wait sixteen years before being challenged.
(By the way, I should mention that Pete Binfield, Managing Editor at PLoS ONE, set the record straight on Bargh's misstatements in a comment on the article -- see
http://www.psychologytoday.com/blog/the-natural-unconscious/201203/nothing-in-their-heads/comments#comment-216610
and note the forebearing tone!)

2 months ago gfrancis responded:

I would like to note that there is a statistical test that can sometimes reveal the presence of publication bias. Details at
Francis, G. (2012). Too good to be true: Publication bias in two prominent studies from experimental psychology. Psychonomic Bulletin & Review, 19, 151-156. DOI 10.3758/s13423-012-0227-9
Ben Goldacre's Space

I like stats. I'm a doctor. I write about dodgy scientific claims in my spare time.
This isn't my main blog. Find me here:
Blog:
www.badscience.net/
Book:
http://www.amazon.co.uk/Bad-Science-Ben-Goldacre/dp/000728487X/?tag=bs0b-21
Tweet:
www.twitter.com/bengoldacre
TED talk:
http://www.ted.com/talks/ben_goldacre_battling_bad_science.html

bengoldacre - secondary blog

286 out of 294 psychology papers reported a positive result. Great paper from 1959.

Ben Goldacre's Space

About Ben Goldacre

Archive

Follow Me