When we’re trying to decide which medicines, remedies, diets, and lifestyle changes might be most helpful or harmful, there are many variables to consider. One of the tools available to help make those big decisions on both a public and a personal level is scientific health research.
The scientific method of assessment looks at the effects of an intervention (be it diet, supplement, drug, or surgery) across a broad population. Ideally, a study includes at least two groups of people (“randomized” for similar health makeups), one of which gets the intervention and another that is controlled to not get the intervention or (better yet, when possible and ethical) to receive a placebo.
The Scientific Method and Natural Medicine
The scientific method has a long history in many cultures including ancient Egypt, Babylon, Greece, India, and the Middle East. It entered mainstream medicine in the nineteenth and early twentieth centuries as a method of developing standard medical recommendations and protocols that exist to this day.
The scientific method has a complicated history in natural and holistic medicine, initially used in the early 1900s via campaigns such as the Flexner Report as a way of shutting down holistic and integrative medical schools. But now we have hundreds of thousands of studies on herbs, dietary supplements, and dietary interventions with largely positive results.
Assessing The Research
How do you find and assess these studies for yourself to help you make decisions and also help cut through the misinformation and bias to see how helpful or harmful your options really are?
Whether you’re trying to decipher the latest COVID-19 debacle or simply figure out which herb is most likely to help you sleep or ease joint discomfort, being able to find, read, and assess studies puts more power into your own hands.
Here are some things to keep in mind as you look at scientific literature and read articles or hear news referring to studies.
-
What kind of study is it? How well designed?
-
Good: Human Clinical Trials
Human clinical trials—especially randomized, placebo-controlled, and triple- or double-blinded—are the most clinically relevant to our reality. Studies with more people are generally better for more accurate and meaningful results (like, 50 to hundreds of people . . . not six to 12).
-
Better: Meta-Analyses / Systematic Reviews
Meta-analyses and systematic reviews are even better because they summarize the highest quality clinical trials for a particular health concern.
-
Meh: Observational Studies
Observational population “epidemiological” studies are cheaper and helpful for making sense of what might be happening, but they don’t prove causation, only correlation.
-
Meh: Poorly-Run Human Studies
Human studies that lack randomization, controls, and blinding are methodologically poor. They make it very easy to skew results or report results that are not as impressive as they seem. Also be wary of trials run on very small groups of people.
-
Meh: Literature Reviews
“Literature reviews” that pull together lots of different types of studies are interesting but often biased and include data from weaker studies that are less valid or clinically relevant.
-
Worst: Lab Studies / Animal Studies
Lab and animal studies are cheaper and easier to run with more dramatic results, but they’re far too preliminary to have clinical relevance.
What happens in a petri dish with single cells doused in herbs (or when something is fed to an animal as 25 percent of its feed) often doesn’t translate to the human body in typical doses.
These studies just point us in new directions to follow up with better quality human trials or to help us understand a possible mechanism of action.
-
-
Who is reporting on or funding this study? Are they biased?
The sad truth is that almost everyone has a bias in their reporting and research, especially extreme political agendas (on either side), pharma and supplement companies that fund or otherwise support research, or other groups that want the intervention to fail.
Whenever an intriguing or controversial scientific study comes out, I like to find the study itself and look at the full text to see what it actually is, actually says, who may have funded it, and how impressive the results were.
-
How significant are the results?
But how impressive are the results . . . really? Reporters can over-inflate (or deflate) results— especially with studies of low scientific meaning such as population, lab, and animal studies. Reporters and scientists may cherry-pick data only from the groups or studies that prove what they want and ignore everything else.
In dietary supplement studies, participants may have seen very little improvement in their actual symptoms, yet the researcher will highlight some vague lab marker that improved. (Do I care if my C-reactive protein inflammatory marker was significantly changed? Not if my pain is the same!)
Sometimes what is “statistically significant” may not actually be impressive or clinically relevant (like losing two pounds after four months of adding one-third of a spice jar of cayenne to breakfast).
Clinical trials, meta-analyses, and systematic reviews are the most likely to provide the details on this; you may need to pull up the full text, but sometimes it’s in the abstract.
You may need to look at the raw numbers to calculate the percentage of improvement yourself. (See this spreadsheet I made that will help you calculate this.)
-
How replicable is the study?
One study is just one study. It means more when additional studies—ideally by different researchers— find similar results. Who else has studied this? What kinds of results are they getting?
If you see different studies with very different results, look more closely to see why. Population type? Duration? Dose? Format, type of extract, or species of herb? Funding bias?
Getting access to free full-text meta-analyses and systematic analyses can be helpful: They often provide charts of the best-designed studies and summaries of what was studied and the results.
Finding the Studies
Digging into the scientific evidence on both holistic and conventional medicine can be a frustrating, daunting, mind-boggling experience. But it can also be enlightening and a way to put the power back into your own hands to help you make personal health decisions and to bridge conventional and holistic communities by speaking a similar language.
Increasingly, scientific study summaries (called abstracts) and often the entire study (full text) are available for free and easily searchable and downloadable on the internet. Many services also collect and summarize the data for easier understanding.
PubMed is a free, searchable database of study abstracts provided by the National Institutes of Health. It’s incredibly useful but can be overwhelming for a newbie. In the September issue of remedies, I’ll provide a step-by-step guide to finding and assessing studies from PubMed and other sources.
