A New York Times article (http://www.nytimes.com/2011/01/21/science/21memory.html?_r=1&ref=science) based upon a recent Purdue study by Jeffrey D. Karpicke and Janell R. Blunt shows that when students take tests they learn science better than when they use study aids such as concept maps and other learning devices.
Purdue study abstract:
Educators rely heavily on learning activities that encourage elaborative studying, while activities that require students to practice retrieving and reconstructing knowledge are used less frequently. Here, we show that practicing retrieval produces greater gains in meaningful learning than elaborative studying with concept mapping. The advantage of retrieval practice generalized across texts identical to those commonly found in science education. The advantage of retrieval practice was observed with test questions that assessed comprehension and required students to make inferences. The advantage of retrieval practice occurred even when the criterial test involved creating concept maps. Our findings support the theory that retrieval practice enhances learning by retrieval-specific mechanisms rather than by elaborative study processes. Retrieval practice is an effective tool to promote conceptual learning about science.
On the surface this appears to be a significant finding that supports testing as a method for education. However, it is important to remember that there are two well established forms of assessment, formative and summative. Summative results show what a student has learned. Formative assessments show what a student needs to learn. Furthermore these results can be reported as normative or criterion referenced scores. Normative scores rank all the students taking the test in order of scores and criterion referenced tests compare the scores to an established standard for each construct being measured. A study like this must be very careful to avoid confounding criterion referenced and normative reports of the results. Just because a group of students do better than another group (normative results) on an inferential test doesn’t mean they know their science (criterion referenced results).
In an article posted on skepticalist.org(http://skepticalist.com/2011/01/23/research-finds-practicing-retrieval-is-best-tool-for-learning/) “Our view is that learning is not about studying or getting knowledge ‘in memory,’” said Purdue psychology professor Jeffrey Karpicke, the lead investigator for the study that appears today in the journal Science. “Learning is about retrieving. So it is important to make retrieval practice an integral part of the learning process.”
It would make sense then that for tasks related to retrieval, it would be better to learn through taking tests, a kind of formative assessment. Groups such as Kaplan have made small fortunes from this observation. However, is retrieval really the goal? What criterion do we want to reference? Do we want students of science to be exceptional at retrieving information, or do we want them to think like a scientist? It strikes me that this study might say more about being a student than being a scientist or even a student of science. Perhaps it says that practicing tests will make students better at taking tests. Is this the criterion we want to reference?
Science itself makes use of testing to study nature. Observations lead to questions that become studies with tests to answer the questions. Feedback from these tests creates new questions and the process of learning more about nature continues. If a scientist asks questions about molecules, the tests are conducted with instruments that observe molecules. Since testing and feedback form the basis for science it would make sense that testing would increase understanding. But what sorts of things to scientists test? Scientists don’t depend upon retrieval when testing hypotheses, they depend upon instrumentation and results. How does the above study improve student performance on hypothesizing and interpreting results?
As a researcher situated in the School of Education at UC Davis, these kinds of studies bother me. Am I the only one?