COGSCIdotNL
Home Blog, news, book reviews Miscellaneous
 
Miscellaneous
The black swan

Let’s consider a biologist with an interest in swan coloration. She goes on an expedition to an area where two groups of swans live, to investigate whether the two groups have different colors. The biologist takes her job very seriously, and first calibrates a photometer against two reference colors: One for the ideal black swan; one for the ideal white swan. She then measures the color (or rather luminance) of ten specimens from each group, obtaining a range of values where 0 is ideal black and 100 is ideal white:

To analyze her results, she runs an independent samples t-test on the measurements, which tells her that p = .0001. This leads her to conclude that the two groups have different colors. Just as she suspected all along:

Our biologist is probably satisfied at this point. But we are not. What exactly has she learned from this t-test and the resulting p-value? Let’s start with the basics: What exactly does p = .0001 mean? Well … it means that if the two groups were really of the same color, the chance of observing a color difference as extreme as she observed, or more extreme, is .0001. This is an odd and counter-intuitive statement. Yet it is the foundation of most research.

 
Running psychological experiments on a Raspberry Pi with OpenSesame

I’m typing this blog on a Raspberry Pi, a £25 / €30 / $40 mini computer that is literally the size of a credit card.

The Pi is an adorable machine (if you’re into that kind of stuff): Just a small printboard with connectors for a monitor, mouse, keyboard, and an ethernet cable. The system boots from an SD card, so there is no hard disk. There is a choice of Linux-based operating systems, the most commonly used being Raspbian, a Debian spin-off that has been optimized for the Pi. This is also what I installed. And in case you’re wondering: I didn’t open the Pi up – It just doesn’t come with a casing!

Because the Pi is extremely cheap, some people have wondered whether it could be used to equip low-budget psychology labs. This is also how I came into the possession of this diminutive cutie: It’s a gift from Clayton, who wondered how well OpenSesame would fare on the Pi. Thanks Clayton!

 
The pros and cons of pre-registration in fundamental research

Based on (critical) responses that I received, and discussions that I had after this post, I have added some footnotes to elaborate on certain aspects. The main criticisms are that pre-registration is not necessarily as rigid as I depict it to be (which may be true), and that questioning statistical guidelines is dangerous (which is certainly true, but also a moralistic fallacy: something can be correct and dangerous to say at the same time). Also, see my (sort of) follow-up post The Black Swan and NeuroSkeptic’s response.

In response to the many recent cases of scientific fraud, a debate has ignited about how science can be made more transparent, and how some of the public trust can be regained. Suggestions include …


An evil scientist.

  • making all research data publicly available, not just the summarized results.
  • making all scientific papers publicly available (i.e. open access).
  • investing more time in replicating results, those of others as well as your own (e.g., the reproducibility project).
  • and pre-registering all studies.

A slightly mysterious, but influential voice in this debate is Neuroskeptic. In a recent post, Neuroskeptic interviews Jona Sassenhagen, a neurolinguist from the University of Marburg, who decided to pre-register his EEG study. So what does it mean to pre-register a study, and why would anyone do this?

 
Poll results: How do psychologists feel about Open Access?

Over the past month I ran a small poll on the OpenSesame website about the attitude of researchers towards open access (OA) in academic publishing.

I received 767 responses, where each question was answered only once per IP address. Questions appeared in random order, and the order of the answers was randomized as well. The usual caveat applies, though: Because I ran the poll on the OpenSesame page, most respondents are presumably early-career experimental psychologists, and not representative of the ‘average’ scientist. Nevertheless, I think it’s interesting to see how this particular segment of the research population feels about OA.

So here we go.

Let’s first take a look at the general attitude towards OA. Clearly, as you can see in the figure below, respondents overwhelmingly perceive OA as a good thing. A minority indicates that they are not too interested in the matter, and no-one expresses an outright dislike of OA.

 
Running psychological experiments on a tablet with OpenSesame

Since this post, the OpenSesame runtime has been ported to Android. This provides an even easier way to run experiments on a tablet device. For more information, see http://osdoc.cogsci.nl/getting-started/android/.

As you can see in the video below, it is possible to run OpenSesame on a tablet! This way you can take your experiments anywhere.

I will post more detailed instructions on the OpenSesame documentation page soon, but for now some basic info should get you started: The operating system that you see in the video is Ubuntu Linux. The tablet is a Nexus 7. Installing Ubuntu onto a Nexus 7 tablet is a straightforward process, and so is restoring the device to it's factory settings. Once you have Ubuntu running on your tablet, you can install OpenSesame directly from the Cogsci.nl PPA.