cognitive science
and more
Some details and (my) thoughts on the Diederik Stapel academic fraud case

A few weeks ago it became known that Diederik Stapel, a Professor at the University of Tilburg, and one of Hollands most celebrated social psychologists, committed academic fraud. I didn't write anything about it before, because the details and the extent of the fraud were largely unknown. But after the publication of an interim rapport on Monday, I felt the urge to write a few words about it. After all, this case reflects very badly on psychological research, and perhaps on the whole of science.

So what happened? Basically, Stapel made up data. Lots of it. And some of it got published in prestigious academic journals such as Science. It appears that Stapel acted alone, and played an elaborate act to fool his co-workers. According to the interim rapport, the usual chain of events was as follows:

Together with a co-worker, which could be one of his PhD-students, a post-doc, or a fellow senior academic, Stapel came up with a research question and, together, they designed an experiment. All the details of the study were discussed and decided on, again, together. So far, so good. Next, Stapel went to a school to conduct the actual experiment (he often worked, or pretended to work, with schools). By himself, this time. Because, or so Stapel argued, the schools knew him. They trusted him. And it would be inappropriate and unnecessary for other researchers to accompany him on these visits.

In actuality, we now know, Stapel did not visit any schools at all. Instead, he made up everything, from the schools (!?) to the research-assistants who helped him, and, of course, the experimental data. The make-believe data was handed over to his co-workers (allegedly, Stapel often said "Do you realize that you've got gold in your hands!"), who faithfully wrote manuscripts that subsequently got sent to, and published in academic journals.

So, in a nutshell, this is what happened. But what are we to make of this? Is this case diagnostic of a diseased system? Or is it just an unfortunate incident?

It strikes me that it's frequently emphasized, in the interim rapport as well, that Stapel acted alone. And that he was very clever and careful in forging his data. This implies that the scientific community has been victimized by a cunning, fraudulent genius. That Stapel is the rare exception, the evil mastermind, who managed to exploit an otherwise well-functioning system.

But the reality, I think, is much more painful. In psychology, and I suppose in many other fields of research as well, it's extremely easy to forge data. There are hardly any safeguards against fraud. So what do you expect? This particular case is spectacular, I'll grant you that, but I can't say that I'm too surprised. What I find more troubling is that many of the same researchers who now cry in moral outrage are opposed to regulations that would help to prevent academic fraud in the future.

The most obvious measure that could be taken is to impose the requirement that all experimental data should be made publicly available. All of it, always, with no exceptions, and accessible for anyone. This could be done quite easily, particularly in the social sciences, where the complete data set often consists of only a few spreadsheets. Right now, you can ask a scientist for his or her data, of course. But the fact that you need to ask is already a big hurdle, because asking implies (although it shouldn't!) a lack of confidence in the capabilities of the scientist. That is why data should be available at the (anonymous) click of a button. This wouldn't make it impossible to forge data, of course. But in the case of Diederik Stapel, it would have made it considerably more difficult for him, because he wouldn't have gotten away with only a statistical summary, as he apparently often did.

But, unfortunately, you will be hard pressed to find a psychologist who would support a requirement to make all of his/ her data publicly available. Because everybody has little dark secrets. They are generally little secrets, relatively harmless, and certainly nothing like the massive fraud committed by Stapel. But still, not the type of stuff that you'd like your fellow scientists to know about. For example, psychologists generally exclude some data points from their analysis that are considered 'outliers' (i.e., irrelevant noise). Because there are no fixed criteria for what an outlier is, researchers are free to pick the criteria that suit them best. And they (we) do, of course.

Another reason why researchers like to keep their data to themselves is that they (being human, after all) are afraid of having made mistakes. And if someone else re-analyzes their data, such mistakes could surface and cause considerable embarrassment. An understandable human sentiment, but not one that leads to an atmosphere of openness.

But can you blame scientists for being protective of their data? Perhaps you can, I'll leave that to the ethicists, but it's an inevitable consequence of the academic system. More specifically, it's a consequence of the pressure to publish papers, and the fact that you can only publish positive, straight-forward results. If your data contains some irregularities (and chances are that it does) you better hide them or your paper won't get published! And if your paper doesn't get published, you'll soon find yourself out of a job.

The point is, I think that the whole system of academic publishing is conducive to the type of fraud that we now see in the Stapel case. At least if there are no proper safeguards, which, right now, there aren't. It's important to remember that scientists are not rewarded for conducting sound research. They are rewarded for publishing papers. In order to publish papers, you shouldn't be too critical of your own data. And, in order to prevent other people from becoming too critical, you shouldn't be too open either.

To my mind, a (beginning of a) solution is obvious: Open up the data! Little dark secrets? I'll show you mine, if you show me yours.

Update Nov 4: This case has received quite a bit of attention in the media. A few picks: Science provides a factual summary of the interim rapport; The New York times has an interesting opinion piece, which paints a bleak picture of psychological research,