Designing tests for better performance

There are a number of factors that can interfere with a person’s ability to recall things. For example, asking too many questions about one specific topic results in ‘output interference’, the gradual decrease in correct responses that occurs as a test progresses. Can this handicap be overcome? According to Kenneth Malmberg of the University of South Florida and his colleagues from Syracuse University and from Indiana University, it can at least be diminished.

To test this, the researchers used a master list of 1200 English words, 150 each from eight distinct categories such as countries, professions, animals, etc. 86 undergraduates were each shown 150 target words (75 from one category and 75 from another) along with 150 decoy words from the same two categories. Thus, there were 150 words in each category, half of which had been on the original list. Participants were asked to identify whether each word had been in the original list.

This experiment was run in three different ways. In one case, all 150 words from one category were presented first, followed by the 150 words in the second category (blocked condition). In another case, the 300 words were randomly mixed (random condition). Finally, the categories alternated between subjects after every five words (short-block condition).

All participants showed a decline in recall as the test progressed.  They chose the correct word at least 90% of the time at the start of the exercise but were scoring below 80% by the last few words. However, for those students encountering the blocked condition of word order, accuracy jumped over 10% between the end of one block and the start of the next. Apparently, shifting categories led to a renewed ability to recall words.

Looking at the data, I’m not convinced that planning tests in this manner is the best policy. While it is true that no such jump in accuracy occurred for either the random or short-block conditions, students exposed to those two conditions did better during the first 150 trials. In other words, the blocked condition gave students an advantage at the break point between categories, but was detrimental up to that point. To me, it seems like a bit of a wash. 

If you’re a teacher constructing a test for your students, I wouldn’t worry too much about question order. Design your tests anyway you like.