The Patriot Post® · You Can Trust Scientific Research Reports (We Think; We Hope)
A recent story in the Chronicle of Higher Education suggests that Michael LaCour, a UCLA graduate student, has fabricated data for another journal article. Science magazine has retracted the article, due to “the misrepresentation of survey incentives, the false sponsorship statement, and Mr. LaCour’s inability to produce original data.” Unfortunately, this is only the latest in a long string of integrity issues in research publications. The scary reality, though, is not the articles that have been found to be questionable, but the possibility of many other fabricated articles that have not been discovered and retracted.
Meanwhile, John Bohannon intentionally published some weak and questionable findings related to chocolate just to demonstrate how quickly non-refereed journals will snatch up research. He claims, “I fooled millions into thinking chocolate helps weight loss.” Sign me up!
The Office of Research Integrity oversees integrity on behalf of the Secretary of Health and Human Services. They are currently investigating 50 cases of research misconduct.
Granted, the vast majority of published research is carefully reviewed and published with full integrity (we think; we hope). Nevertheless, one should be properly skeptical of the scientific claims. How can we be more informed consumers of research claims?
There is some comfort that in the internet age, it is becoming harder to succeed in research fraud. The ease of communication, the requirements of documentation, and the focus on accountability should strongly communicate that research fraud will ruin careers.
Consumers should realize that not all journals are created equal. The best and most trustworthy journals are peer reviewed and have a low acceptance and publication rate. By comparison, there is reason to be more cautious about fee-charging open access journals. However, the LaCour article noted above was published in a highly regarded peer reviewed journal. Even the best journals sometimes make the wrong decision.
Most importantly, consumers should look for replication. Scientific findings that are reported by only one researcher or one laboratory should be considered to be only working hypotheses. Our confidence level should increase only when findings are replicated by others. The 1989 claim of cold fusion is a case in point. This exciting claim caught the attention of the scientific world, but those who sought to replicate the findings were not able to do so. Indeed, good science requires replication. When multiple researchers in multiple labs using multiple methods reach similar conclusions, we have more confidence in their accuracy. Replication provides the best protection against scientific fraud.
Ultimately, I would suggest, it is always wise and prudent to be skeptical about scientific research. Good science gives us valuable new insights into the world in which we live. Nevertheless, even when properly conducted, science does not yield “proof.” The testing of hypotheses is based on statistics and probabilities. Scientific research always includes some probability that the findings resulted from random noise rather than systematic effects. We can sometimes reduce the probability of reaching false conclusions to 1 percent, or .1 percent, or even .000001 percent, but we can never reduce the probability to zero. Anything more than zero means that we have not (indeed we cannot) achieve statistical “proof.” The words “prove” and “proof” are valuable and appropriate in the field of mathematics. They are not, however, acceptable in research terminology. Good research seeks to provide new evidence; it does not seek to achieve “proof.”
Because there is always a non-zero chance of error, it is always wise to be skeptical of science. Indeed, the skeptic is the individual who asks the “what if” questions that enable the research to move to the next level. Unfortunately, many will look at these cases of research fraud and become research cynics. This, however, is the wrong response. The scientific method is a most valuable tool to help us understand how the world works. Yes, there is some risk that individuals might seek to publish fraudulent reports as a means to advance their career. There is some risk that individuals might seek to publish fraudulent reports as a means to advance their agenda. As a protection, however, good science always calls for replication.
I note that simple replication of previous research was often viewed negatively. The best scientist is always trying to do something new and different, in order to advance the field. In fact, however, simple replication of previous research is more important than ever, as a means of validating the integrity of research. Indeed, this has become a new focus in my discipline of social psychology.
Although there is no way to prove the accuracy of any study (but of course, “prove” is not a scientific word), our pursuit of knowledge and our desire to understand our world require good research efforts. Let us not allow the fraud of some to thwart the ongoing search for knowledge.
Dr. Gary L. Welton is assistant dean for institutional assessment, professor of psychology at Grove City College, and a contributor to The Center for Vision & Values. He is a recipient of a major research grant from the Templeton Foundation to investigate positive youth development.