Spurious significance, junk science

Andrew Gelman links to this nice paper by Nosek, Spies and Motel, about an exciting “result” in psychological research: instead of rushing to publish, they scrupulously rushed to replicate, and the result disappeared. The fairy tale ending is that they got a nice publication from using this experience to tell us what we already know – that “significant” results obtained from small, ad hoc experimental samples are pretty much worthless. Continue reading

Advertisements

“A Vast Graveyard of Undead Theories: Publication Bias and Psychological Science’s Aversion to the Null”

You have to love that title, which comes from a paper by Christopher Ferguson and Moritz Heene, which the excellent Andrew Gelman parses, and passes on to the rest of us. Any field that uses statistics is susceptible to publication bias (i.e., not publishing statistical analyses that find “no effect”). It is notorious in pharmaceutical research, where money talks shouts. I am guessing that the reason psychology gets a particularly bad reputation for publication bias, compared with other social sciences, is that it deals with a lot of small experimental data sets – so you really do have a situation where nearly identical experiments can be run twenty times by different researchers, and the one that gets a significant effect gets published. Statistical work in economics and political science tends to keep re-using a small number of mostly public data sets, so the problems are different.

Dodgy journals go for gold

“Gold Open Access” – where authors pay to have their papers published, and made freely available on line – is one response to the well documented predatory practices of commercial journal publishers (for discussion of the latter problem, mostly as regards economics journals, see Ted Bergstrom’s website.)

On-line publishing, though, has low entry costs – especially if you don’t do any real peer review, editing, or archiving – and the combination of on-line publishing and gold open access has produced new “publishers”, often with unfeasibly large suites of new journals that look very much alike, and torrents of spam-ish “calls for papers” in academic email inboxes. Jeffrey Beall, a librarian at the University of Colorado, Denver, publishes a list of “Potential, possible, or probable predatory scholarly open-access publishersContinue reading

Weak intellectual property and economic catch up – the German case

In Der Spiegel, Frank Thadeusz reviews Eckhard Höffner’s work. The story: 19th century Germany had far better dissemination of new scientific & technical ideas, in part because weak copyright enforcement forced publishers into aggressive pricing & paperback editions. In England publishers thrived but most people couldn’t afford their products. This difference helped Germany catch up.

What Höffner describes in 19th century Germany is a sort of open innovation system – not one without intellectual property protection, but one with weak protection. Continue reading

Open access publishing and not-for-profit publishing

Is the UK government’s new requirement of (slightly delayed) free access to publications based on government-funded research a blow to the extortionate power of commercial academic publishers, or will it just entrench them further? Continue reading