The determined patient is aware that just because a result is reported does not mean that it is reliable! An excellent article that appeared in the New York Times noted some reporting biases that tend to make reported medical discoveries to be not as promising as initially hoped.
Publication bias. Determined patients need to be aware of publication bias, that is, the tendency of a publication to publish positive results more often than negative results. Journals and news media like to report positive results that showed something helped rather than negative ones that showed no benefit of some treatment. Often, studies that showed negative results aren’t published. In other words, there may be a research study showing a benefit of Drug A, but ten other studies that did not find a benefit are not published. The reality may be that the study showing benefit is not reliable because the results were a consequence of random luck. This is why metanalyses are done: studies where similar studies are analyzed to put the results of any single study in context. Looking at just one study can be misleading.
Outcome reporting. A second bias to be aware of is outcome reporting, that is, making negative results sound positive by changing the question (or hypothesis) after the results are analyzed. For example, the initial study may be to determine whether Drug A is better than Drug B. If Drug A is not found to be better, then the results are reported as, “Drug B is just as effective as Drug A.” This may not be correct since concluding that Drug A and B are equally effective requires different statistical tests and maybe even a different experimental design. Drug companies are always trying to “salvage” something positive to report from clinical trials that show negative results.
Spin. The third bias to be aware of is bias that comes in the form of “spin,” where results that aren't significant are reported as if they were. For example, results may not reach statistical significance but still show a positive trend. The trend is then reported as a more positive result than is warranted. Let's say an overly eager investigator tries a new treatment, say Drug A, on a couple of patients who have some aches and pains—to good results. Then, the investigator claims that Drug A works for aches and pains when the truth is that the patients’ problems might have gone away by themselves and Drug A had no effect. To show that Drug A was effective, the investigator would probably have to study dozens or even hundreds of patients.
Readers should not only read the abstract or summary of the research study but also keep these biases in mind and consider reading the entire study. This is especially true when reading research reports in newspapers and on websites, since the reporters often interview researchers who, understandably, want to play up the positives and downplay the negatives of the results.
Understanding publication bias, outcome reporting, and spin on research can help you, as a determined patient, cut through to the valuable information you need to make choices regarding your health care.