Are ANY vaccines properly safety tested? A look at the process and the data

Quant genius Mathew Crawford is doing a brilliant series of articles exploring the possibility that vaccine efficacy — for all vaccines — may be zero and that any claimed benefit from shots is actually a wealth effect (said differently, when one controls for wealth any supposed benefit from the vaccine disappears). I would like to add my two cents to this debate.

Toby Rogers

As many of you know, I got a master of public policy degree from UC Berkeley in 2012. What you may not know is that UC Berkeley is usually the top rated quantitative public policy program in the country. So what that means in this case is that it is heavily focused on econometrics.

Econometrics is beautiful. It usually starts with a large data set for a population. Then one uses sophisticated statistical software (STATA, SPSS) to analyze the data. Econometrics involves massive equations that are looking for the effect of a particular variable, while controlling for a wide range of additional variables.

There is a humility to econometrics as well — so-called “dummy variables” are added to account for factors one may have overlooked and error measures are added to almost every equation to reflect that the data is gathered by humans and there will always be some error.

What’s fascinating about econometrics is that if one really builds the model correctly, the largest effect size that one will ever see for a single variable is about 0.3. This means that X intervention explains 30% of the outcome — the rest of the variables explain the rest. We live in a multivariate world.

Even with all of that complexity, there are still those (such as the great economist Steve Keen) who argue that econometrics, with its 15, 50, or even 100 variables is still completely inadequate and that if one really wants to understand how the world works, one must utilize the tools of physics (and supercomputers) to build models with millions of variables and account for things like chaos theory (this approach is called econophysics).

So that was my background when, in 2015, in the midst of a Ph.D. program in political economy, I started researching autism and decided to read a vaccine safety study for the first time. There are about 20 studies that the CDC points to as showing that there is no relationship between vaccines and autism. I assumed that I would not be able to read or understand these studies at all. Given what I knew about the complexity of econometrics, and knowing that the human body, biology, chemistry, and the immune system are even more complex than economics, I assumed that I would be looking at equations involving calculus, that used advanced statistical software to analyze hundreds or perhaps thousands of variables that impact health and disease.

The reality is quite different. Vaccine safety studies tend to be bivariate — they only look at two variables — the vaccine (independent variable) and whether someone suffered an adverse event (dependent variable). It’s actually even worse than that. Vaccine safety studies almost always look at children who have received the full vaccine schedule as compared with children who have received the full schedule +1 more vaccine. And on that basis, they decide whether the 1 additional vaccine is safe. There is no unvaccinated control group. And they do not control for hundreds of other variables that influence health and disease.

When I read a vaccine safety study for the first time, a sickening panic swept over me — “no, no, no, this cannot possibly be. THIS is what the CDC is relying upon!? THIS is what the CDC is using to claim that vaccines are safe!?” Far from being too advanced, these studies are so crude they would fail any Statistics 101 class in any college in America. Tears streamed down my face.