Why Science Is Never Settled (And How to Spot Ego in Experts)

How to Tell Which Health Experts to Trust

No, by nature of what science is, it's never fucking settled. So when someone tells you the science is settled on a supplement or a training split, what they are telling you is that they stopped asking questions.

Before getting into how to spot it, it helps to see how the whole thing is supposed to work.

A researcher notices something and writes down a guess about why it happens, which is called a hypothesis, meaning a specific statement about the world that could turn out to be false. Then he designs a study whose entire purpose is to catch that statement being false.

That part sounds backwards to most people. You do not go looking for evidence that you are right. Your goal is to come up with a hypothesis and then try to prove yourself wrong when you have that hypothesis.

That's the way that we go about studying these things.

The reason this works is that evidence confirming your idea is cheap. If I believe creatine ruins kidneys, I can find a case report, and if that is not enough, I can dig up a forum post from someone whose labs looked off after he started taking it. Confirmation is always available, which is exactly why it proves nothing on its own.

Disconfirmation costs you something, because you have to go hunting for the result that would wreck your position, and if you hunt hard and it still does not show up, now the idea has survived something.

A finding that survives ten honest attempts to kill it is worth more than a hundred studies that never tried.

Which means every conclusion in science is temporary by design. It is the best standing answer until someone runs the experiment that breaks it. That's the greatest thing about science, is the fact that we're always questioning ourselves, we're always asking questions, we're always looking for new information.

So the loudest, most certain voice in the room has the weakest relationship to the method.

A true scientist understands the importance of not having an ego and not settling on science and being okay being wrong. That is a working requirement, because a person who cannot tolerate being wrong will never design the study that might make him wrong, and he will never update when someone else's data contradicts him.

Defending a position and investigating one are different activities, even though on camera they look identical.

So when you're trying to figure out who to believe, the very first thing that you should be looking at is, is this person bringing their ego into the conversation? That's the first thing I would look for.

What that looks like in practice is specific. Watch what happens when someone presents a study that cuts against him. Does he engage with the methodology, the sample size, the population studied? Or does he go after the person who brought it up, wave it away, or pivot to his credentials?

Watch whether he has ever publicly changed his position on anything. Someone who has been producing content for five years and has never updated a single take has stopped reading the literature.

Watch whether he tells you where his confidence runs out. Strong on this, thin evidence on that, unsure here. That kind of mapping is hard to fake and most people do not bother.

Then do the part nobody wants to do, which is check for yourself. Find the actual paper and look at who was studied, how many of them, for how long, and whether the outcome measured was the one you care about. It takes twenty minutes and tells you more about a creator than a year of watching his videos.

Set your ego aside, try to prove yourself wrong, do your own research and evaluate creators based upon how well their viewpoints align with the research that you did for yourself.

The test is not whether someone is right, since nobody bats a thousand and everybody holds at least one belief that will not survive better data. The test is what he does on the day the evidence turns on him.