Led by a keen interest in the measurement of human differences, in 1885 Francis Galton set up the first anthropometric laboratory at the London Health Exhibition. He used devices, many of which he designed, to measure performance on cognitive tasks. An important indication of intellectual performance, he believed, was the subjects’ speed on those task, or reaction time. In the first year of opening, over 9,000 participants paid a small fee to be measured and receive their results!
Since the end of the 19th century, psychologists have relied on reaction time measurement to find out about people’s intelligence, memory, sensory processes, personality, social and group processes, pretty much every area of psychological research. In the last few decades, reaction time measurement has relied on computers, where participants stare at a computer screen and go through sometimes endless exercises of pressing keys when relevant stimuli flash up. More recently, researchers have started to take their experiments online and give people everywhere a chance to take part in their studies using their web browsers. Online studies are not without criticism though as data collection is far less rigorous than in a lab setting.
So is collecting reaction time data online a good idea?
Well, yes and no. We recently looked into the practicalities and limitations of measuring reaction time through a web browser. In the lab, you would probably use the same hardware and software to test all participants. Online, however, the differences in architecture will lead to all sorts of sources of error.
First, there’s the keyboard and display lag. Keyboard lag is the time it takes from participants pressing the response key on their computer to the response being registered by the computer. That can be up to tens of milliseconds depending on their hardware. The display lag is usually even larger: the time it takes from the stimulus being “sent” to the display and when it is actually displayed by the monitor. In the lab, CRT monitors are usually used as they have a much lower lag than LCD screens. But in the real world hardly anyone still owns a CRT monitor.
Then there’s software measurement error. Some web browsers rely on the time of the operating system which is inexact. Web browsers also vary in performance, for instance Internet Explorer seems far less reliable than Chrome or Firefox for reaction time measurement. As a note to developers, the best way we found to minimise browser error is to use the performance library (performance.now function available on Chrome versions 24+, Mozilla Firefox 15+, Internet Explorer 10+, and Opera). And this is only for PCs with keyboards, don’t get me started on reaction time error on touch screen devices, I’ll save this for a future post.
Reaction time measurements in experiments usually range around 300-600 milliseconds. All the sources of error above can add up to over 50 milliseconds, so quite a significant measurement error.
So is it a good idea to take your reaction time experiments online? We would say a firm no if you are looking at measuring absolute reaction times to compare against existing benchmarks. However, if you are looking at differences between groups then given a large enough sample you should be able to detect an effect. If you study a population that would be unlikely to come into the lab during working hours (so pretty much anyone other than psychology students) then it might be a good option. Even if you could test people in the lab, you may get a more diverse sample and get more exposure for your research online, so it may be an idea to do both.
