August 23, 2018
he BBC headline was ”Low-carb diets could shorten life, study suggests” (Ref 1). In the US, CNN went with “Low and high carb diets increase risk of early death, study finds” (Ref 2). There were many similar, irresponsible, headlines worldwide that emanated from a study published in August 2018 in The Lancet Public Health journal (Ref 3). The Sydney Morning Herald warned “People on low carb diets die younger, says science” (Ref 4).
Let’s look at the ‘science’…
We need to make a critical point up front: every headline using the words “low carb” was wrong. The first sentence of the paper was “Low carbohydrate diets…” This was also wrong. The full paper used the words “low carbohydrate” 40 times. That was also wrong – 40 times. Low carb diets have not been studied by this paper. Full stop. The average carbohydrate intake of the lowest fifth of people studied was 37%. That’s a high carb diet to anyone who eats a low carb diet. As we will see below, the researchers managed to find just 315 people out of over 15,000 who consumed less than 30% of their diet in the form of carbohydrate. The average carb intake of these 315 people was still over 26%. Not even these people were anywhere near low carb eating. Hence, if you do eat a low carbohydrate diet, don’t worry – this paper has nothing to do with you.
You’re welcome to continue reading to see what else was wrong with this paper.
The study
The study was in three parts, but all were based on population studies and so the usual limitations of these apply. The first part was a study of 15,428 adults aged 45-64 years, in four US communities, who completed a dietary questionnaire at enrolment into the Atherosclerosis Risk in Communities (ARIC) study (between 1987 and 1989) (Ref 5). The primary outcome of interest was all-cause mortality. The second part of the study involved combining the data from the ARIC study with data from seven multinational population studies in a meta-analysis. The final part was an assessment of whether the substitution of animal or plant sources of fat and protein for carbohydrate affected mortality.
The findings
The findings from Part 1, the ARIC study, were that there were 6,283 deaths during the 25 year follow-up. The conclusion from this part of the study was that 50-55% of energy from carbohydrate was associated with the lowest risk of mortality. The average intake of carbohydrate in the ARIC study was 49%. The paper presented a U-shaped curve to indicate that carbohydrate intake below 30% and above 65% was associated with the highest risk of mortality.
In Part 2, when the meta-analysis pooled together the results from ARIC with seven other population studies, the U-shaped association was observed again. Both lower carbohydrate intake (<40%) and higher carbohydrate consumption (>70%) were associated with higher mortality than ‘moderate’ carbohydrate intake.
Part 3 reported that the results varied depending on the source of macronutrients. It was claimed that mortality increased when carbohydrates were exchanged for animal-derived fat or protein and mortality decreased when the substitutions were plant-based.
The headline
The headline that generated so much media attention was a purely statistical calculation in the paper, which resulted in the claim “we estimated that a 50-year-old participant with intake of less than 30% of energy from carbohydrate would have a projected life expectancy of 29·1 years, compared with 33·1 years for a participant who consumed 50–55% of energy from carbohydrate… Similarly, we estimated that a 50-year-old participant with high carbohydrate intake (>65% of energy from carbohydrate) would have a projected life expectancy of 32·0 years, compared with 33·1 years for a participant who consumed 50–55% of energy from carbohydrate.”
I’ll show below how this claim was generated – when we address the biggest issue with the paper.
10 things wrong with this article
There are a number of issues with all epidemiological studies. The two most important generic limitations are:
1) Association does not mean causation; and
2) Relative risk is presented when absolute risk is invariably tiny.
All dietary epidemiological studies also suffer from three further limitations:
3) The Food Frequency Questionnaire.
4) Incomplete adjustment of the data.
5) The healthy person confounder.
This particular study then additionally suffers from the following limitations:
6) Failure to adjust for a serious confounder (alcohol).
7) The small comparator group issue.
8) The selection of the reference group.
9) The meta-analysis.
10) The claims related to food ‘exchanges.’
I’m going to go through each of these issues, with particular reference to this Lancet Public Health paper, to show why the findings of this paper and the headlines can’t be trusted:
1) Association does not mean causation.
Population studies enable us to observe that people who, for example, eat broccoli die older than people who don’t. They cannot conclude that eating broccoli causes you to die older. Equally possible is that people who tend to eat broccoli tend to be generally healthy and therefore tend to die older.
If association is observed, the association should be subjected to The Bradford Hill Criteria to test association (Ref 6). The first test is “strength of the association” – if this isn’t double, or greater, there’s little point looking at any of the other criteria – you aren’t going to be able to claim causation. This brings us to…
2) Relative risk is presented when absolute risk is invariably tiny.
This study didn’t present its claims in the usual way “37% greater chance of dying” – which is a measure of relative risk. It is as if the world has become immune to such headlines. This paper chose a new way of trying to shock the world: “You’ll die 4 years earlier unless you eat the ‘perfect’ intake of carbohydrate.”
The main paper didn’t present the relative risk numbers. The appendix did, however. The following data are extracted from Supplementary Table 1 from the appendix:
The table below shows: the carb ranges subjectively selected by the researchers (we’ll come back to this); the number of people that ended up in each range; and the deaths that occurred in that carb range during the 25 year follow-up. I have then included the Hazard (risk) Ratio (HR) for the most adjusted model (Model 2). Model 2 adjusted for age, race, gender, which test center the person attended, total energy consumption, diabetes, cigarette smoking, physical activity, income level and education. Please note that Model 2 – the most adjusted model – didn’t adjust for BMI. (It didn’t adjust for something else either, as point 6 will cover).
Table 1
| Carb range | <30% | 30-40% | 40-50% | 50-55% | 55-65% | >65% |
| People | 315 | 2,242 | 6,097 | 3,026 | 3,034 | 714 |
| Deaths | 163 | 986 | 2,533 | 1,162 | 1,150 | 289 |
| Adjusted HR | 1.37 (1.16-1.63) | 1.21 (1.11-1.32) | 1.11 (1.03-1.19) | Ref (1.0) | 1.01 (0.93-1.10) | 1.16 (1.02-1.33) |
The Hazard Ratios give us the relative risks. It is claimed that, relative to the group chosen as the reference group (50-55% carbohydrate intake), a carb intake <30% gives a relative risk of 1.37 (37% greater) and a carb intake of >65% gives a relative risk of 1.16 (16% greater). Neither of these “strengths of association” come anywhere close to double and so the possibility of carbohydrate intake being causal in these relative risks can be discounted. We could stop here. However, there are many more issues to note…
The numbers are always reported as relative risk; the absolute numbers are usually not worth getting excited about. In the ARIC study, there were 6,283 deaths among 15,428 adults who were studied for 25 years having been recruited between the ages of 45-64 years. The average age was 54, so an average person would have been 80 at the end of the follow-up period, so survivors did pretty well. The overall death rate in this study was 40%. The annual death rate was 1.63%.
A 37% difference on an annual death rate of 1.63% is the difference between a death rate of 1.38% and 1.88% (Ref 7). That’s the absolute difference and that’s in middle aged people followed for 25 years.
3) The Food Frequency Questionnaire.
Each population study relies for dietary information on a self-reported recall of what one ate some time ago. The paper reported that the ARIC study used the Harvard Food Frequency Questionnaire (FFQ), which was developed by Walter Willett – one of the paper authors – and colleagues. This questionnaire involved asking people how often, on average, over the previous year, they consumed ‘standard portion sizes’ of each of 66 items. Credit to Nina Teicholz for getting hold of the precise questionnaire used in the ARIC study (Ref 8). If you only look at one reference, please let it be this one – there’s nothing like seeing a FFQ first hand. As you can see, there were nine possible responses, ranging from “never” to “six or more times a day.”
Can you remember what you ate last year? How standard were your portions? Did you have 5-6 ‘pats’ of butter a week or did it tip over to 1 a day? What’s a pat anyway? Did your diet then stay the same for 20-25 years?
This questionnaire was administered when the participants were first recruited (Baseline – between 1987 and 1989). It was repeated between 1993 and 1995 at Visit 3. The paper reported that, up to Visit 3, carbohydrate intake from Visit 1 (baseline) was used for the analysis – obviously. After Visit 3, “the cumulative average of carbohydrate intake was calculated on the basis of the mean [average] of baseline and Visit 3 FFQ responses.” It was assumed therefore, that the ARIC participants did not change their diet for approximately 20 years, even if they did change their diet between 1987-89 and 1993-95. That’s a big assumption.
There was a more serious assumption. The paper reported: “We did not update carbohydrate intake exposures of participants that developed heart disease, diabetes, and stroke before Visit 3, to reduce potential confounding from changes in diet that could arise from the diagnosis of these diseases.” This means, if Fred is in the lower carb group at baseline and he develops diabetes and goes to see a dietician who tells him to eat ‘healthy’ whole grains, he will be more likely to die (from diabetes), but this will be a death attributed to the lower carb group. Conversely, however, if Sally is in the moderate carb group at baseline and she develops heart disease and decides to cut her carb intake, she will be more likely to die (from heart disease), but this will be a death attributed to the moderate carb group. Ostensibly, this appears to be a reasonable assumption. In this paper it isn’t, for two reasons:
i) Conventional dietary advice is to consume approximately 55% of one’s diet in the form of carbohydrate. If someone in the ARIC study developed cardiovascular disease or diabetes between the two questionnaires (i.e. between 1987-89 and 1993-95), they would be in the healthcare system. As a result, they would be advised to consume c. 55% of their diet in the form of carbohydrate. This period pre-dated the literature, which has grown in the past decade, showing the benefit of very low carbohydrate diets for obesity, diabetes and chronic conditions (Ref 9) and hence no individual would have been advised to cut their carbohydrate intake following a health diagnosis. People who developed a life threatening chronic condition would, therefore, have wrongly stayed assigned to a lower carbohydrate group rather than being reassigned to a higher carbohydrate group. The converse would almost certainly not have happened.
ii) We’ll come on to the small comparator group issue soon, but it has an impact with this assumption too. You can see in Table 1 above that the <30% carb group is by far the smallest – just 315 people. Because it is so small, it is far more sensitive to small changes. It would take only 68 people to be reallocated from the <30% carb group (if they were diagnosed with a condition and advised to increase their carb intake and then assigned to their new carb group) for this group to have the same (unadjusted) death rate as the ‘optimal’ carb group (down from 51.7% to 38.4%). If, conversely, 68 people from the ‘optimal’ carb group changed their carb intake and were reallocated from that group, the death rate in this group would barely drop a percentage point (from 38.4% to 37%).
The assumption not to revise the carb group for those diagnosed with a serious condition was highly unfavourable to the lowest carb intake group.
We know that there is something seriously wrong with the FFQ data in this paper because of the average calorie intake in the characteristics table. The characteristics table splits the 15,428 people into five equal groups (quintiles) from lower carb intake to higher carb intake. The calorie intake ranges from 1,558 calories per day in the lower carbohydrate quintile to 1,660 calories in the middle carbohydrate intake group. No group apparently consumed more than an average of 1,660 calories a day in this American study in the past 25 years. Really?!
4) Incomplete adjustment of the data.
Table 1 above contains extracted data from the appendix to the paper. This reported that the data were adjusted for age, race, gender, which test center the person attended, total energy consumption, diabetes, cigarette smoking, physical activity, income level and education. BMI was not included in factors adjusted for and thus this appears to have been an adjustment omission. That is an additional flaw of this paper.
Dietary epidemiology studies are rarely adjusted for the whole diet. This paper did not adjust for any other aspect of the diet. Carbohydrate intake was the sole focus of the paper – the type of carbohydrate, or intake of vegetables oils, or processed meat was not adjusted for in the ARIC study.
Carbohydrate can be found in the following diverse items in the FFQ used in this study: dairy; fruit; vegetables (and these vary from broccoli to yams); candy; pie; cakes; cookies; bread (whole wheat or not); cereal; rice; pasta (whole grains or not); chips; nuts and sugar. Carbohydrate from green vegetables, nuts and dairy products is very different to carbohydrate from chocolate, chips and cookies.
None of this was adjusted for. The whole diet was not taken into account (Ref 10).
5) The healthy person confounder.
I wondered what kind of person would be consuming a lower carbohydrate diet in the late 1980s/early 1990s (when the questionnaire was done). The characteristics table in the paper tells us exactly what kind of person was in the lowest carbohydrate group. They were far more likely to be: male; diabetic; current smokers; with a higher BMI; and far less likely to be in the highest exercise category. The ARIC study adjusted for most of these characteristic, but not BMI. Even if they had adjusted for all characteristics, as I often say, you can’t adjust for a whole type of person. An unhealthy person is not differentiated by carbohydrate intake alone once someunhealthy characteristics have been adjusted for.
I joked in a recent review of a classic whole grain epidemiological paper that I expect people who consume whole grains regularly (that’s <5% of Americans) to: not smoke; not drink; be affluent; do yoga; be slim; shop at Whole Foods; eat at restaurants, not takeaways; have children called Olivia and Tarquin and so on. The whole grain consumption is a marker of good health, not the maker of good health (Ref 11).
Visualise for a moment the unhealthy person confounder in this study – the overweight, smoking, diabetic, male couch-potato, and then we can turn to the first error unique to this paper, as opposed to all epidemiological papers…
6) Failure to adjust for a serious confounder (alcohol).
Many thanks to George Henderson (who tweets as @puddleg – well worth a follow), for spotting that there was no mention of alcohol in the main paper at all. This means that alcohol was not accounted for or adjusted for. This is also despite the fact that the Food Frequency Questionnaire used in the ARIC study did include questions about alcohol (beer, wine and liquor to be precise). Maybe alcohol accounted for the missing calories in the total energy intake numbers? The whole alcohol issue is a major error.
There is also a confounder of this error in that, we know from the characteristics table in the main paper that those in the lower carb intake group were more likely to be smokers. There is a positive association between smoking and drinking: smokers are more likely to be drinkers (of coffee and alcoholic drinks). This unexplained omission was highly unfavourable to the lower carb intake group.
7) The small comparator group issue.
We now come on to the single biggest issue in my view – indeed it may be fair to call it a manipulation. It’s what I call the “small comparator group issue.” I have explained this here. It is summarised here again for completeness:
The characteristics table in the main paper split the 15,428 people into equal groups (of 3,085-3,086) from the lowest to the highest carb intake. This is the objective way to review data, because there is no argument that you drew the line in a particular place to bias the finding. The appendix revealed what had been done to produce the U-shaped finding that grabbed the headlines. The numbers were extracted in Table 1 above – repeated here for convenience:
Table 1
| Carb range | <30% | 30-40% | 40-50% | 50-55% | 55-65% | >65% |
| People | 315 | 2,242 | 6,097 | 3,026 | 3,034 | 714 |
| Deaths | 163 | 986 | 2,533 | 1,162 | 1,150 | 289 |
| Adjusted HR | 1.37 (1.16-1.63) | 1.21 (1.11-1.32) | 1.11 (1.03-1.19) | Ref (1.0) | 1.01 (0.93-1.10) | 1.16 (1.02-1.33) |
The groups have been subjectively chosen – not even the carb ranges are even. Most covered a 10% range (e.g. 40-50%), but the range chosen for the ‘optimal’ group (50-55%) was just 5% wide. This placed as many as 6,097 people in one group and as few as 315 in another. The subjective group divisions introduced what I call “the small comparator group issue.”
If 20 children go skiing – 2 of them with autism – and 2 children die in an avalanche – 1 with autism and 1 without – the death rate for the non-autistic children is 1 in 18 (5.5%) and the death rate for the autistic children is 1 in 2 (50%). Can you see how bad (or good?) you can make things look with a small comparator group?
To then get the media headlines about life expectancy, the researchers applied a statistical technique (called Kaplan-Meier estimates) (Ref 12) to try to estimate when people would die and conversely life expectancy. This is purely a statistical exercise – we don’t know when people will die. We just know how many have died so far.
This exercise resulted in the claim “we estimated that a 50-year-old participant with intake of less than 30% of energy from carbohydrate would have a projected life expectancy of 29·1 years, compared with 33·1 years for a participant who consumed 50–55% of energy from carbohydrate… Similarly, we estimated that a 50-year-old participant with high carbohydrate intake (>65% of energy from carbohydrate) would have a projected life expectancy of 32·0 years, compared with 33·1 years for a participant who consumed 50–55% of energy from carbohydrate.”
Do you see how both of these claims have used the small comparator group extremes to make the reference group look better?
Back to the children skiing… If we were to use the data we have so far (50% of autistic children died and 5.5% of non-autistic children died) and to extrapolate this out to predict survival, life expectancy for the autistic children looks catastrophic. This is exactly what has happened with the small groups – <30% carb and >65% carb – in this study.
8) The selection of the reference group.
Food Frequency Questionnaires have so many limitations; there are arguments that they are worthless. They have the fewest limitations when they study the extremes. People are far more likely to accurately recall if they never have something and if they have something six or more times a day. Secondly, dietary studies are at their ‘least worst’ when they compare the lowest intake with the highest intake group for a particular food item, for example eggs, and not when they try to aggregate something found in many different foods (in this case carbohydrate).
This study featured both of these limitations. First the researchers subjectively chose unevenly distributed groups (Table 1) and then they used the narrowest group chosen (50-55%) as the reference group. This chosen reference group was in the middle of the groups – far from the extremes. This is the least robust part of a FFQ – where someone is trying to recall and estimate if they consumed a table spoon of salad dressing four times a week, or was it five? Then, this was confounded further by the study looking at many food items and not just one. What exactly was that small, selectively chosen, reference group representing? High green vegetables, whole grains and nuts or moderate candy, cakes and cookies?
9) The meta-analysis
Part 2 of the paper took the ARIC (US) study above and claimed to have combined it with seven other population studies in a meta-analysis. Three of the seven other studies were from Europe: Lagiou et al – Swedish women; Nilsson et al – Swedish men and women; and Trichopoulou et al – Greek men and women. Two were from the US. These were the Harvard favourites – the female Nurses Health Study and the male Health Professionals Follow-up Study. Both of these studies were authored by Fung et al. The final two studies were the Nippon study from Japan and the multinational PURE study (Ref 13).
The purpose of this part of the paper seems to have been to corroborate the U-shaped curve finding claimed in Part 1. The researchers managed to do this, but not without a couple of magic tricks…
i) On p3, under “Statistical analysis”, the paper reported “… papers were eligible for inclusion if they … adjusted for at least three of the following factors: age, sex, obesity, smoking status, diabetes, hypertension, hypercholesterolaemia, history of cardio-vascular disease, and family history of cardiovascular disease.” That means that any of the included studies need not have adjusted for many vital factors that would determine all-cause mortality, while having nothing whatsoever to do with carbohydrate consumption.
ii) On p3, under “Statistical analysis” again, the researchers described their meta-analysis as an update of a previously published meta-analysis from 2012 (Ref 14). The 2012 meta-analysis, by Noto et al, included Lagiou et al, Nilsson et al, Trichopoulou et al, and the two Fung et al studies (NHS and HPFS). The latter two were combined into one finding, so there were four sets of numbers in total. The ‘updated’ meta-analysis simply added the ARIC study to the Noto et al meta-analysis. Other than this, the exact same Hazard Ratios from the Noto et al paper were used. The narrative even reported “This relationship remained significant if the ARIC study was excluded from the analysis (1·31, 1·07–1·58).” Well it would do, because the meta-analysis without ARIC was an exact repeat of the 2012 Noto et al meta-analysis. This was hardly an update. The next point may help to explain why the researchers did this…
iii) The researchers decided to split their meta-analysis into two parts: Part A contained the three studies from Europe along with the Fung et al combined result and the ARIC study from the US; Part B contained just the Japanese and PURE study. The rationale for this was given as: “Because there was significantly lower consumption of carbohydrate in European and North American regions compared with Asian countries, low-income countries, and multinational cohorts, studies fell into two categories in the meta-analysis: North American and European studies (mean carbohydrate intake approximately 50%) that compared low carbohydrate diets with primarily moderate carbohydrate consumption as the reference, and Asian and multinational studies (mean carbohydrate intake approximately 61%) that compared high carbohydrate consumption with moderate carbohydrate consumption as the reference.”
I interpret the rational as – if we pool them all together we don’t get a significant result, but if we pool Europe and America (which are lower in carb anyway, but lower than average carb is relatively worse) and if we pool Japan with the multi-national study (these two being higher in carb, but higher than average carb is relatively worse), then we get two significant results.
The PURE study covers Asia, Africa, North and South America, the Middle East and Europe. If the researchers insist on isolating Europe and North America, then the PURE data for Europe and North America should be put in that meta-analysis. I’m not sure if the rationale for ‘what’s left’ is higher carbohydrate regions, or poor regions, or Asian regions.
The PURE data tell us that the high carb regions (averaging above 60%) should be China, south Asia and Africa, which I think you’ll find are irredeemably confounded by these regions being poor and the diet being high carbohydrate as a consequence of low affluence. If the researchers want to ignore Africa, South America and the Middle East altogether and isolate Asia as a region, then the Japanese study could be combined with the Asian regions from the PURE study, since PURE helpfully separates the Asian regions (their Fig 2A). For interest, the PURE study found that there was no statistical significance for the highest intake of carbohydrate vs. the lowest in Asian regions (1·09, 0·94–1·26).
My review of the meta-analysis part of the paper suggests that it is a ‘fudge’ to provide support to the ‘sweet spot’ for carbohydrate claim made in Part 1 of the paper (the ARIC study).
10) The claims related to food ‘exchanges.’
The final part of the paper reported that mortality results varied depending on the source of macronutrients. It was claimed that “mortality increased when carbohydrates were exchanged for animal-derived fat or protein and mortality decreased when the substitutions were plant-based.”
This is so disingenuous, it’s difficult to know where to start. When the aforementioned George Henderson and I worked together on the recent UK saturated fat rebuttal (Ref 15), George really hammered home to me how often the ‘exchanging/swapping/replacing’ foods argument is made in epidemiological studies and yet this doesn’t happen. Swapping one food out and another in is the domain of randomised controlled trials (RCTs). Epidemiological studies collect information from Food Frequency Questionnaires. An individual may consume less carbohydrate and more animal products but a) they didn’t swap one for the other and b) individuals are not studied. Epidemiological studies aggregate all the people and make general statements about people in the same group.
This paper aggregated people with lower carbohydrate and higher fat and protein from animal foods. It also aggregated people with lower carbohydrate and higher fat and protein from plant sources (Ref 16). The data for this part of the paper come from the ARIC study again.
The paper helpfully shares some of the observations, which should have been called dietary confounders. (Direct extracts from the paper are in italics and quotation marks; my comment follows):
– “The plant-based low carbohydrate dietary score was associated with higher average intake of vegetables but lower fruit intake.” So, did low fruit (sugar) intake confer any observed benefit?
– “By contrast, the animal-based low carbohydrate dietary score was associated with lower average intake of both fruit and vegetables.” So, did the lack of vegetables confer any observed detriment?
– “Overall, total protein intake was higher in the animal-based diet.” So, did higher protein intake confer any observed detriment?
– The study determined the five foods that differed most significantly between the highest and lowest groups of animal-based and plant-based low carbohydrate dietary score: “The animal-based low carbohydrate diet had more servings per day than did higher carbohydrate diets of beef, pork, and lamb as the main dish; beef, pork, and lamb as a side dish; chicken with the skin on; chicken with the skin off; and cheese.” The more accurate description from the FFQ is “beef, pork or lamb as a sandwich or mixed dish, e.g. stew, casserole, lasagna, etc.” So, now we’re talking bacon sandwiches, microwave lasagna and lamb curries. For chicken intake in the US, I suspect we’re talking KFC (an American institution since 1930).
“The plant-based low carbohydrate diet had more servings per day of nuts, peanut butter, dark or grain breads, chocolate, and white bread than did higher carbohydrate diets.” In contrast, this list doesn’t look like a ready meal or takeaway menu. This part of the paper confirmed the “healthy person confounder”, or, in this case – the unhealthy male, smoking, drinking, overweight, diabetic, takeaway lover!
Close
Even if this study had analysed groups fairly in the quintiles in the characteristics table…
Even if the Food Frequency Questionnaire (FFQ) had been robust and accurately reflective of what people actually ate during the whole 25 year study…
Even if people had been allocated properly to reflect their actual carb consumption following a health diagnosis…
Even if an average calorie intake of 1,560-1,660 could be explained…
Even if the study had adjusted for the whole diet…
Even if ‘carbohydrates’ didn’t mean tens of different things (from kale to cake)…
Even if the paper had managed to overcome the whole ‘healthy person’ confounder…
Even if alcohol had been taken into account and adjusted for…
Even if the researchers hadn’t manipulated the data to benefit from the small group comparator advantage…
Even if the life expectancy had been calculated fairly, without this significant small comparator group manipulation…
Even if the reference group had been set at the most robust spectrum of the quintiles (the extremes, not the middle)…
Even if the subject under examination (an entire macronutrient) were suitable for averaging across already limited FFQs…
Even if the strength of association had been double…
Even if examination of the Bradford Hill criteria had established that causation might be likely…
…the purpose of epidemiological studies is to establish relationships that should then be tested in randomised controlled trials.
Even if all of those ‘even ifs’ had stacked up, the researchers would then merely have had something to test in a randomised controlled trial…
As Professor Noakes once tweeted, if epidemiology has validity, RCTs will back it up. Over to you Professor Willett to test your hypothesis!
References
Ref 1: https://www.bbc.co.uk/news/health-45195474
Ref 3: Seidelmann SB, Claggett B, Cheng S, et al. Dietary carbohydrate intake and mortality: a prospective cohort study and meta-analysis. The Lancet Public Health 2018. https://www.thelancet.com/journals/lanpub/article/PIIS2468-2667(18)30135-X/fulltext
Ref 5: https://jandonline.org/article/S2212-2672(17)31238-8/pdf
Ref 6: http://www.zoeharcombe.com/2016/09/the-bradford-hill-criteria/
Ref 7: I keep a spreadsheet to calculate this in the simplest way possible. The goal in this particular example is to find the two numbers that maintain the average of 1.63%, while one is 37% bigger than the other.
Ref 8: https://regepi.bwh.harvard.edu/health/FFQ/files/80out.pdf
Ref 9: Westman EC, Yancy WS, Mavropoulos JC, Marquart M, McDuffie JR. The effect of a low-carbohydrate, ketogenic diet versus a low-glycemic index diet on glycemic control in type 2 diabetes mellitus. Nutrition & metabolism. 2008.
Feinman RD, Pogozelski WK, Astrup A, et al. Dietary Carbohydrate restriction as the first approach in diabetes management. Critical review and evidence base. Nutrition. 2014.
Hallberg et al. “Effectiveness and Safety of a Novel Care Model for the Management of Type 2 Diabetes at 1 Year: An Open-Label, Non-Randomized, Controlled Study.” Diabetes Therapy. 2018. (https://link.springer.com/article/10.1007%2Fs13300-018-0373-9)
Ref 10: The adjustment generally looked odd. The table below contains the data from the appendix. I added in a line (where it says ZH) for an unadjusted HR. This simply takes the 50-55% group as the reference point of 1.0 and then works out what the death rate in each group would be relative to 1.0/ This gives an HR of 1.35 (with no adjustment) for the lower carb group.
Given that the lower carb consumers had most of the characteristics stacked against them (gender, diabetes, smoking, BMI and lower exercise), I would expect the bottom line of that table – the adjusted risk ratio (HR) – to have reduced the ZH line (my calculation with no adjustment) substantially. You can see that it has barely moved for the lower carb group. That doesn’t make sense
| Carb range | <30% | 30-40% | 40-50% | 50-55% | 55-65% | >65% |
| People | 315 | 2,242 | 6,097 | 3,026 | 3,034 | 714 |
| Deaths | 163 | 986 | 2,533 | 1,162 | 1,150 | 289 |
| Death rate | 51.7% | 44.0% | 41.5% | 38.4% | 37.9% | 40.5% |
| ZH – no adjust[1] | 1.35 | 1.15 | 1.08 | 1.00 | 0.99 | 1.05 |
| Adjusted HR | 1.37 (1.16-1.63) | 1.21 (1.11-1.32) | 1.11 (1.03-1.19) | Ref | 1.01 (0.93-1.10) | 1.16 (1.02-1.33) |
Ref 11: http://www.zoeharcombe.com/2018/08/whole-grains-cvd-cancer-mortality/
Ref 12: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3059453/
Ref 13: Dehghan M, Mente A, Zhang X, et al. Associations of fats and carbohydrate intake with cardiovascular disease and mortality in 18 countries from five continents (PURE): a prospective cohort study. The Lancet 2017 doi: 10.1016/S0140-6736(17)32252-3
Ref 14: Noto H, Goto A, Tsujimoto T, Noda M. Low-carbohydrate diets and all-cause mortality: a systematic review and meta-analysis of observational studies. PLoS One 2013.
Ref 15: http://www.zoeharcombe.com/2018/07/saturated-fat-consultation-sacn-my-response/
Ref 16: The paper described “We created animal-based and plant-based scores by dividing participants into deciles for either animal-derived or plant-derived fat and protein, and carbohydrate intake, expressed as a percentage of energy as previously described. For carbohydrate, participants in the lowest decile received 10 points, whereas participants in the highest decile received 1 point. The order was reversed for animal-derived or plant-derived fat and protein, so that the highest score represented low carbohydrate and high animal-derived or plant-derived fat and protein intake.”