SS 188 – Why You Can’t Believe Many Nutritional Studies

Episode 188 Show Notes

 

Grant and Heavey take on the realm of nutrition studies. Are they all-conclusive? Should you really trust them and follow their recommendations? Now, what if the study turns out to be erroneous? All this and more in this episode!

 

BTW- please link up with Grant and Heavey on Facebook or Instagram. Or email them at [email protected] or [email protected]. Let them know if you like these kinds of discussions.

 

[01:45] Mediterranean Diet, Carb Timing, and Other Research

 

Grant presents an article about how a massive study (that is the backbone for the reasons we think the Mediterranean is so good) is actually crap. Nevertheless, there are very important takeaways about nutritional science that listeners can benefit from. That said, they need to cover some background on how science is conducted because you need that to understand what happened with the study.

 

Last week, the podcast tackled carb timing and how effective it was based on scientific research. The advice Heavey gave in that episode was not based on one study that looked at carb timing. End of story. It was based on a collection of research that’s pooled together to allow scientists to form broad conclusions based on a wide body of evidence. It’s not just based on a single study. But even then, it doesn’t make that information a fact. So they concluded with the episode with the idea that most people aren’t going to benefit from carb timing unless under a very specific set of conditions. This doesn’t mean carb timing won’t work for you. It just means that for the average person, carb timing is not helpful.

 

All of these studies are based on averages and it’s important for people to understand how people are drawing conclusions from it.

 

[06:05] The Challenges of Conducting Nutritional Studies

 

Back to the article, Grant points out that they were directly studying the importance of the study, to prove whether Mediterranean diet is the best. Heavey says the cool thing about the study was that it was a randomized control trial which is very challenging to conduct when it comes to nutritional science because it’s very expensive in most implementations. In the most controlled environment, people are put in metabolic worlds and fed meals all day. So imagine how expensive that can get for a group of people doing that.

 

What they did in this study was educate people on the low-fat Mediterranean diet and gave them free olive oil or free nut deliveries to their homes.

 

In comparison, the way most nutritional studies are actually conducted is by doing observational research. That’s when you see big collections of data covering hundreds of thousands of people. It allows us to form a hypothesis but it doesn’t allow us to do cause and effect analysis.

 

What allows cause and effect is randomized controlled trials. However, even though they’re considered to be the gold standard of science, they come with their own set of issues.

 

[08:53] Lower Risk of Cardiac Diseases

 

When performed perfectly, you’re taking a group of people and randomly selecting who’s going to be assigned to what diet. For instance, you can’t put people of the same household in the same group as the results could account for the genes, and not for the thing they’re studying on.

 

The outcome they found from the study was that people had a one-third lower risk of cardiovascular disease than the controlled group. This is massive!

 

Considering the rate of mortality for just the disease being 33% lower, if we can find an intervention like that, then it’s going to be more powerful than any other pharmaceutical we can come across.

 

[12:05] Too Good to Be True?

 

What happened in the study was that they started poking at it. This study was actually released back in 2013, which was done over the course of five years. So it’s relatively new.

 

When you see this dramatic of a response, it causes people to scratch their head a little bit. Naturally, there were data scientists looking at the researchers’ data trying to figure out what’s going on.

 

They found that the randomization didn’t seem right. So they sent the study to 17 different study centers. One study center consists of many clinics and they found that one particular study center put the randomization in a way that there was some sort of geographical bias into the data. Across the entire study, when two people were in the same household, they ended up assigning them into the same group instead of randomizing them. Certainly, this is more convenient when you share a household. But you can see the challenges where that starts to cluster people together and that reduces the randomization of participants.

 

Moreover, they found that 20% of participants in the entire study had some sort of randomization error inherent in them. This was the first big thing they uncovered in the study. They certainly had some non-standard practices.

 

[15:15] The Impact in the Results

 

Heavey explains that slight 20% is a fifth of the data. The study was published on the New England Journal of Medicine, which is a prestigious journal. This discovery basically led the journal to delete the paper from their records. Ultimately, they re-published a new version of the paper that’s supposedly factored in the randomization errors, which still shows a strong benefit for the Mediterranean diet

 

Now, the authors of the study did not find this issue. And now that we’ve seen something sloppy with their randomization, how do we know they weren’t sloppy with other things?

 

Even in the new version of the paper, there are people starting to find inconsistencies in tables and figures reported in it.

 

[18:15] The Benefits of Meta Analysis

 

With that said, Heavey wants to point out that science is tough especially at large scale. It’s hard to put all of your faith in the outcome of a single study.

 

This is why meta analyses are so powerful because they allow us to collect multiple studies and then treat them like a larger study.

 

The idea is that you’re able to factor these studies out. When they perform these meta analyses, they’re not just allowing every single study in. They have qualification procedures to make sure that the study scored well.

 

They’re trying to replicate some of the studies, which is a very important part of science and they’re not able to replicate a large percentage of the studies that varied from their original intent. Now the gold standard for randomized controlled trials is to declare your intent in advance before beginning your study. So the idea of what’s good science is even evolving today.

 

[20:45] Who Should You Go To: Doctors, Dietitians, or Nutritionists?

 

Moreso, nutrition studies make these studies even harder because the human body is so complicated that we don’t really understand much of the body. This makes it challenging because there are so many variables we might not even be controlling for and don’t even know it.

 

Most people go to doctors, not nutritionists. Heavey thinks “how to eat for you”  is something we can’t get from a doctor. One, they don’t have enough time with you to help you through that. Two, most of them are not well-educated enough on these topics. They simply can’t dive into nutrition. In medical school, they get a very short education on nutrition and that’s what they essentially carry forward with them. Unfortunately, it does a disservice to their patients.

 

However, we’re not seeing a change in the medical industry where practices are adding other people – registered dietitians, nutritionists, or health coaches.  In most states, anybody can call themselves a nutritionist. But a registered dietitian is a licensed title in the 50 states.

 

So if you want to focus on your diet, are you better suited to go to a dietitian? Here’s the thing.  A lot of them still have come up through this traditional medical approach to nutrition, which is very slow moving and doesn’t necessarily keep up with the research.

For instance, the American Heart Association recently just reduced their restriction on dietary cholesterol. It’s old information but they’ve only recently changed it. This sort of traditional industry is the way many dietitians come up and they may still subscribe to that way of thinking. Hence, Heavey can’t really say who can give the best information.

Hopefully, there is a trend in medicine that thinks outside of the disease and more about health and thriving, and as a part of that, incorporating good, solid nutrition principles. However, people are on their own right now to do that.

[25:25] Heavey’s Recommendation

Heavey recommends you find somebody you can trust somehow. Maybe look up publishing articles or podcasts. Hear them out over time until you’re able to build trust. Once you feel like you can trust them, then approach them and see if they could work with you. Another way is to find a referral from someone you already trust and has worked with that professional.

[26:25] Science Doesn’t Just End on Paper

Grants reiterates that this serves as a reminder that even when these studies come out and look so great, that doesn’t mean science is done. Science doesn’t stop when a paper comes out.

Heavey adds that science is rarely done as it’s always evolving. We’re always going back and rechecking things and building a body of evidence. No single study or research or person can make a definitive proclamation. It’s based on consensus.

[27:40] Heavey is Working with Solely Nutrition Clients Too

A considerable portion (about 40%) of Heavey’s clients are nutrition clients. This said, he thinks it’s rare for people to pay for this type of service especially because there’s so much free information out there. Sure you can do your research, but sometimes, it’s really complicated. It’’s good to have somebody that has worked with similar people as yourself and can guide you.

 

Like Heavey has said, all the science is reported back in averages. So even though he finds carb timing doesn’t work for people on average, maybe it’ll work for you. In a way, each client equals one experiment. This is just part of the process for getting things dialed in depending on the client, their goals, and circumstances.

Links:

This Mediterranean Diet Study Was Hugely Impactful – The Science Just Fell Apart