So is collecting reaction time data online a good idea?
First, there’s the keyboard and display lag. Keyboard lag is the time it takes from participants pressing the response key on their computer to the response being registered by the computer. That can be up to tens of milliseconds depending on their hardware. The display lag is usually even larger: the time it takes from the stimulus being “sent” to the display and when it is actually displayed by the monitor. In the lab, CRT monitors are usually used as they have a much lower lag than LCD screens. But in the real world hardly anyone still owns a CRT monitor.
Then there’s software measurement error. Some web browsers rely on the time of the operating system which is inexact. Web browsers also vary in performance, for instance Internet Explorer seems far less reliable than Chrome or Firefox for reaction time measurement. As a note to developers, the best way we found to minimise browser error is to use the performance library (performance.now function available on Chrome versions 24+, Mozilla Firefox 15+, Internet Explorer 10+, and Opera). And this is only for PCs with keyboards, don’t get me started on reaction time error on touch screen devices, I’ll save this for a future post.
So is it a good idea to take your reaction time experiments online? We would say a firm no if you are looking at measuring absolute reaction times to compare against existing benchmarks. However, if you are looking at differences between groups then given a large enough sample you should be able to detect an effect. If you study a population that would be unlikely to come into the lab during working hours (so pretty much anyone other than psychology students) then it might be a good option. Even if you could test people in the lab, you may get a more diverse sample and get more exposure for your research online, so it may be an idea to do both.
