ASA Voices » Science Education

Does taking tests help students learn science?

Scot Sutherland — Mon, 24 Jan 2011 07:05:04 +0000

A New York Times article (http://www.nytimes.com/2011/01/21/science/21memory.html?_r=1&ref=science) based upon a recent Purdue study by Jeffrey D. Karpicke and Janell R. Blunt shows that when students take tests they learn science better than when they use study aids such as concept maps and other learning devices.

Purdue study abstract:

Educators rely heavily on learning activities that encourage elaborative studying, while activities that require students to practice retrieving and reconstructing knowledge are used less frequently. Here, we show that practicing retrieval produces greater gains in meaningful learning than elaborative studying with concept mapping. The advantage of retrieval practice generalized across texts identical to those commonly found in science education. The advantage of retrieval practice was observed with test questions that assessed comprehension and required students to make inferences. The advantage of retrieval practice occurred even when the criterial test involved creating concept maps. Our findings support the theory that retrieval practice enhances learning by retrieval-specific mechanisms rather than by elaborative study processes. Retrieval practice is an effective tool to promote conceptual learning about science.

On the surface this appears to be a significant finding that supports testing as a method for education. However, it is important to remember that there are two well established forms of assessment, formative and summative. Summative results show what a student has learned. Formative assessments show what a student needs to learn. Furthermore these results can be reported as normative or criterion referenced scores. Normative scores rank all the students taking the test in order of scores and criterion referenced tests compare the scores to an established standard for each construct being measured. A study like this must be very careful to avoid confounding criterion referenced and normative reports of the results. Just because a group of students do better than another group (normative results) on an inferential test doesn’t mean they know their science (criterion referenced results).

In an article posted on skepticalist.org(http://skepticalist.com/2011/01/23/research-finds-practicing-retrieval-is-best-tool-for-learning/) “Our view is that learning is not about studying or getting knowledge ‘in memory,’” said Purdue psychology professor Jeffrey Karpicke, the lead investigator for the study that appears today in the journal Science. “Learning is about retrieving. So it is important to make retrieval practice an integral part of the learning process.”

It would make sense then that for tasks related to retrieval, it would be better to learn through taking tests, a kind of formative assessment. Groups such as Kaplan have made small fortunes from this observation. However, is retrieval really the goal? What criterion do we want to reference? Do we want students of science to be exceptional at retrieving information, or do we want them to think like a scientist? It strikes me that this study might say more about being a student than being a scientist or even a student of science. Perhaps it says that practicing tests will make students better at taking tests. Is this the criterion we want to reference?

Science itself makes use of testing to study nature. Observations lead to questions that become studies with tests to answer the questions. Feedback from these tests creates new questions and the process of learning more about nature continues. If a scientist asks questions about molecules, the tests are conducted with instruments that observe molecules. Since testing and feedback form the basis for science it would make sense that testing would increase understanding. But what sorts of things to scientists test? Scientists don’t depend upon retrieval when testing hypotheses, they depend upon instrumentation and results. How does the above study improve student performance on hypothesizing and interpreting results?

As a researcher situated in the School of Education at UC Davis, these kinds of studies bother me. Am I the only one?

Faith-Based Hiring?

Randy Isaac — Mon, 20 Dec 2010 20:23:03 +0000

Astronomer Martin Gaskell has sued the University of Kentucky for discrimination based on religious faith when, in 2007, they bypassed him in favor of someone else to head their observatory. According to the NYTimes, the lawsuit alleges that Gaskell was denied the position on the basis of his religious beliefs. Martin and his wife have been long-time members of ASA. Though we do not have access to all relevant documents and evidence in order to make a judgment on the legal merits of this particular case, we at ASA are deeply concerned about the tendency toward, and the perception of, religious discrimination in hiring for scientific positions and in funding of research grants. We support Martin in his effort to clear the record in his case.
Several issues come to mind. First is the influence of faith as a factor in hiring decisions for science positions. In the case of private institutions whose mission involves a commitment to a statement of faith, a hiring decision in their science department might justifiably begin with a faith position. But regarding the credentials for a scientific position in a public institution or any other private laboratory, neither religious faith nor atheistic convictions should be a factor. It is not always easy to distinguish the real reason for a hiring decision and we must strive to eliminate any discrimination on the basis of religion.
Another issue is the potential influence of ideology or background beliefs of scientists on their work in science. ASA member Jitse Van Der Meer wrote an insightful article called Ideology and Science on this topic. He notes that whereas scientists do bring background beliefs to their work, the aim is always to achieve a scientific result independent of that ideology. The consilience of results from diverse background beliefs contributes to the confidence that the scientific work is in the right direction. Accordingly, an evangelical doing science correctly would achieve the same result as a Muslim or an atheist. The criterion of a good scientist should not be their religious belief itself but how those beliefs influence the science. Unfortunately, many evangelicals have been prone to advocate a modification of scientific results in order to meet their ideological preferences. Often they attempt to justify their approach by the perception that mainstream scientists have modified their science to meet atheistic preferences. Neither is correct.
There is an ever present tendency for profiling. We hear a lot about racial or ethnic profiling and here we may have indications of religious profiling. Widely disseminated media reports in the past decade or more have described evangelicals who have denied standard scientific methodology or scientific results based on their interpretation of the Bible. Whether it is the age of the earth or the validity of evolution, their skepticism is identified with evangelicalism. It is hard for the secular community to appreciate the nuances of the spectrum of opinions within evangelicalism. It is easier to profile all evangelicals as advocates of non-mainstream science.
We must stand against religious discrimination of any kind, as appears to have been the case with Gaskell. No decision of hiring for a scientific position or of funding of scientific research grants should be based on religious beliefs but on the qualifications of the individual alone. Being “potentially an evangelical” has no bearing on one’s scientific merit. Similarly, mainstream scientists are not to be portrayed as atheists whose lack of belief in a creator influences their science. The few who do attempt to alter their science to match their beliefs should not taint the vast majority of honest scientists whether they are evangelicals or atheists. Whatever statements Gaskell may have made in the past about issues beyond his field of expertise, it is abundantly clear that his own science is stellar and that he was the best candidate for the job. We urge a fair resolution of his case and a deeper understanding by everyone that being evangelical does not mean their science is distorted.

Smithsonian Human Origins Initiative

Randy Isaac — Thu, 11 Mar 2010 22:07:34 +0000

The Smithsonian National Museum of Natural History has announced its new Human Origins Initiative. Their website was launched yesterday. On Wednesday March 17 they will open the new David H. Koch Hall of Human Origins which features a major new exhibit on what it means to be human. As part of this initiative, the Smithsonian has convened a Broad Social Impact Committee (BSIC), acknowledging the spectrum of responses that might greet the exhibit. I have the privilege of being asked to be part of this committee which represents the wide range of religious and cultural traditions in America. The BSIC website lists the attendees but has not yet posted any of the collective input. The committee had no opportunity to influence the content of the exhibit. Rather, the objective was to help edit the FAQ section both in the exhibit and on the website. We have also been asked to help advise on the training of docents.

On Sunday March 21 from 5pm to 7pm, the committee will hold a press conference to talk about some of the reactions. Our committee met last June in a stimulating session sharing each of our representative groups’ potential reaction to the exhibit. We were able to view some of the back room activities in the museum as well as look at the exhibit construction in progress. We will meet again on March 21 and 22 and be able to view the final product. Those of you who are planning to come to the ASA meeting in Washington DC are encouraged to take the time to visit the exhibit.

To help me provide some input at the March 21-22 meeting, I’d appreciate your browsing the website and submitting your comments here.

Eugenie Scott at Colorado State University

Terry M. Gray — Fri, 12 Feb 2010 16:17:39 +0000

On Monday, January 25, Eugenie Scott of the National Center for Biology Education (NCBE) spoke at CSU (promo blurb). A panel discussion followed on the state of science education with Dr. Scott, some CSU profs, and some local science teachers participating. I also had the opportunity to have breakfast with Dr. Scott the following morning. Here are a few reactions.

Dr. Scott went out of her way to distinguish between religion and science. Surprisingly (to me), she from the outset made room for religion. When asked explicitly about her religious faith later in the Q&A, she distanced herself from the question and claimed that it had no place in this forum. This was a talk about science education and not about her private religious beliefs. It seemed clear to me that the questioner was pushing for a “Darwinian evolution leads to atheism” type position and Dr. Scott refused to go there. The talk was sponsored in part by a new student group at CSU, Leaders in Free Thought (LIFT), who were probably disappointed with all of this. I was glad to see it and it clearly set the tone of the rest of the talk. Given this I could have seen a local ASA chapter (not yet existent) or our CSU Christian Faculty Network help sponsor this talk.

Interestingly, during the panel discussion this thread was continued. The question of religious belief came up again and all of the panelists were quick to say that education about evolution had nothing necessarily to do with religious belief. One panelist was a self-admitted religious person but from a non-Christian perspective. During the panel Q&A a couple of apparent evangelical questioners asked about how the teachers handled or should handle students who were convinced young-earth creationists (YEC) or intelligent design (ID) advocates. Somewhat shocking to me, all agreed that the issue was not whether the students agreed with what they were being taught, but whether they understood the material. Frankly, I was pleased to hear this and was glad that public school educators respected the private opinions of their students and their families. Their goal was to teach science (at least the content side–there was a significant emphasis on the importance of teaching scientific methodology) as what scientists currently think.

There was a certain inconsistency, however, and I attempted to raise this in the Q&A time. I’m not sure I was clear, because the responses were somewhat generic. Both Dr. Scott’s talk and the panel began with a reference to surveys that indicated that most Americans don’t accept evolution and that this indicated a failure of American science education. The inconsistency that I see is that “accepting evolution” doesn’t seem to be the goal of science education if all that is important is that students understand what scientists think. Thus, the framing of the question is critical here. Is the question “Do you accept evolution?” or “Do you understand evolution?” The only answer I got was that in polls the framing of the question is critical.

I’d be curious about what ASAers think about this issue. Is the goal of science education (at least the content side) merely knowing what scientists think or is it accepting it as true (within the limitations that all scientific knowledge has a certain tentativity to it)? At what point in the science education or science practice process does the requirement that scientists accept the conclusions of science kick in? Granting of the Ph.D.? Granting of tenure? Peer review? Obviously, the answers differ depending on the institution and the nature of the research journal.

The title of Dr. Scott’s lecture was “Not Over After Dover” and was a review of the various court rulings forbidding the teaching of creationism and intelligent design in the public school. While in general I am not very sympathetic with demarcationist arguments and I’m not at all convinced that the current views of separation of church and state are the right interpretation of the Constitution, I was nearly in total agreement with the scientific arguments presented (critical of YEC and ID). Her clear definition of evolution, her distinction between common ancestry and various mechanisms of evolution, and her rebuttals of Dembski and Behe were all things that I agreed with and have used in my own presentations. In general, I would call NCBE a friend of ASA.

I had two pieces of advice for Dr. Scott.

1. That she recognize that the search for and recognition of intelligent agents is scientific. After all, her background is physical anthropology. Anthropology, archaeology, the search for extraterrestrial intelligence (SETI), forensics, etc. are all scientific. Social and human sciences all deal with intelligent agents. Admit it, and then explain why ID is different. When I mentioned this, her response was positive. Her explanation had to do with the identity and scientific accessibility of the designer. We recognize ID because we are familiar with designers that design similar things. All the disciplines mentioned follow that pattern. I think this is close to what Randy Isaac is getting at in his review of Meyer’s Signature in the Cell (most recent post in that series). God as designer is unlike any designer we know. Not only is his handiwork designed, he upholds and controls the very substances things are made of, the laws that govern it all, and any evolutionary history that may have led to their being.

2. That she appeal to evangelical supporters of evolution (ASA, Francis Collins, Denis Lamoureux, B.B. Warfield, John Stott, perhaps Tim Keller, etc.). Appeal to Roman Catholic or even mainline Protestant thinkers, doesn’t really help convince an evangelical. The evangelical needs to see fellow evangelicals that they trust in order to move ahead here. Appeal to Catholic or possibly theologically liberal theologians does no good. Even appeal to ASA, Collins, Lamoureaux, and others willing to “give up” a historical Adam and Eve or a historical Fall will not be accepted. While I’m not questioning anyone’s evangelicalism in saying this, it is largely the case that once you have given up a historical Adam and Eve that many (maybe most) evangelicals will suggest that you have moved significantly in the direction toward the theological liberal. The fact that there are examples of thinkers on this issue who don’t do this (Warfield, Stott, Keller, myself, etc.) will enable some to accept evolution. Dr. Scott agreed with my assessment here but pointed out that her motivation was primarily to show that accepting evolution was not inherently atheistic. In other words, it was part of her setting the context that being pro-evolution is not to be anti-religious.