Critique 238: Differences in evaluating confounding in epidemiologic studies when judging the effects of alcohol consumption on the risk of cardiovascular disease – 7 April 2020
Wallach JD, Serghiou S, Chu L, Egilman AC, Vasiliou V, Ross JS, Ioannidis JPA. Evaluation of confounding in epidemiologic studies assessing alcohol consumption on the risk of ischemic heart disease. BMC Medical Research Methodology 2020;20:64 https://doi.org/10.1186/s12874-020-0914-6
Authors’ Abstract
Background: Among different investigators studying the same exposures and outcomes, there may be a lack of consensus about potential confounders that should be considered as matching, adjustment, or stratification variables in observational studies. Concerns have been raised that confounding factors may affect the results obtained for the alcohol-ischemic heart disease relationship, as well as their consistency and reproducibility across different studies. Therefore, we assessed how confounders are defined, operationalized, and discussed across individual studies evaluating the impact of alcohol on ischemic heart disease risk.
Methods: For observational studies included in a recent alcohol-ischemic heart disease meta-analysis, we identified all variables adjusted, matched, or stratified for in the largest reported multivariate model (i.e. potential confounders). We recorded how the variables were measured and grouped them into higher-level confounder domains. Abstracts and Discussion sections were then assessed to determine whether authors considered confounding when interpreting their study findings.
Results: 85 of 87 (97.7%) studies reported multivariate analyses for an alcohol-ischemic heart disease relationship. The most common higher-level confounder domains included were smoking (79, 92.9%), age (74, 87.1%), and BMI, height, and/or weight (57, 67.1%). However, no two models adjusted, matched, or stratified for the same higher level confounder domains. Most (74/87, 85.1%) articles mentioned or alluded to “confounding” in their Abstract or Discussion sections, but only one stated that their main findings were likely to be affected by residual confounding. There were five (5/87, 5.7%) authors that explicitly asked for caution when interpreting results.
Conclusion: There is large variation in the confounders considered across observational studies evaluating the impact of alcohol on ischemic heart disease risk and almost all studies spuriously ignore or eventually dismiss confounding in their conclusions. Given that study results and interpretations may be affected by the mix of potential confounders included within multivariate models, efforts are necessary to standardize approaches for selecting and accounting for confounders in observational studies.
Forum Comments
All epidemiologic studies seek to judge the “true” relations between whatever exposure is being evaluated (e.g., alcohol intake) and relevant outcomes, especially disease and death. Most studies try to include in their multivariable analyses all potential confounders for which they have data. It is appreciated that many studies have incomplete assessments of potential confounders, especially the new genetic factors currently being widely studied. Thus, failure to include all potential factors should be expected in an individual study.
In the present paper, based on a recent meta-analysis of alcohol and disease (including coronary heart disease, CHD) from the Global Burden of Disease, it appears that their assessment of confounding was based primarily on the number of confounders reported for each paper, and some studies considered a huge number of potential confounders. The authors report that the most common confounders adjusted for were (appropriately) smoking, age, BMI, height and weight, physical activity, and education. However, it can be assumed that many of the additional factors that were counted would not be expected to have an effect on the results; for example, included in their list of potential confounders were gastritis, friends, fish/fruit intake, emotional control, parity, life control, contraception, car use, truancy, blood hemoglobin, divorced parents, vitamins, run away from home, ulcer, and siblings. If the present analyses are based only on the total number of confounders used in each study, this could cause problems in interpretation; judging the magnitude of effect of each potential confounder on estimating risk of the outcome would be preferable.
Need to have adequate measures of the “exposure,” including the type of beverage and pattern of drinking: It appears that using this meta-analysis, the authors were limited in their ability to judge the measures used to define the exposure in their analyses. For example, a key factor in all such analyses is the assessment of alcohol exposure: often, the “average” intake over a week/month/year are used in the analyses, but data on the pattern of drinking may be incomplete or unavailable. In too many studies, subjects consuming 7 drinks on two days of the week are combined with those drinking two drinks on 7 days per week, and included within the same category and classified as “moderate drinkers.” The health effects within this category will differ markedly according to the pattern of drinking.
The pattern of drinking as well as the type of beverage consumed are important modifiers of the effect of alcohol on CHD; consumers of wine almost always show more favorable health effects than consumers of other beverages. Even with adjustments for these variables, studies almost always show a J-shaped curve between alcohol and risk. And this is not a new finding, since almost two decades ago Grønbæk et al published findings from Copenhagen studying more than 20,000 persons with respect to mortality and alcohol type. Compared with nondrinkers, light drinkers who avoided wine had a relative risk for death from all causes of 0.90 (95% CI, 0.82 to 0.99) and those who drank wine had a relative risk of 0.66 (CI, 0. 55 to 0.77). Wine drinkers had significantly lower mortality from both CHD and cancer than did non-wine drinkers (P = 0.007 and P = 0.004, respectively).
It is further appreciated that information on other factors related to alcohol intake varies among studies. For example, adjusting for previous alcohol consumption among current “non-drinkers” is usually found to be a very important modifier of effect among populations with a relatively large number of ex-drinkers, but would essentially have no effect if the population has a very small number of such subjects. For example, it would not be expected to be relevant in a study of Asian women, who generally drink little alcohol, if at all. Thus, a particular potential confounder may not be relevant in certain situations. In any case, most papers include a large number of potential confounders in their analyses initially, but exclude many that appear to not be important for their population in their final analyses.
Should all studies adjust for the same confounding factors? We are not quite sure if every study should adjust for the same set of confounders when assessing alcohol consumption in relation to the risk of CHD. Sometimes so called multivariable-adjusted models may over-adjust for alcohol effect on risk of CHD. And we know that alcohol consumption habit often forms in early adulthood, while epidemiologic studies of CHD are conducted among older populations; thus, the alcohol exposure is not an incident, but a prevalent, risk factor. This makes selection of potential confounders more challenging because some confounders may be mediators of the effect of alcohol consumption.
Residual confounding in meta-analyses: Problems related to adjustments for confounding have been found by many authors (e.g., Goenwold et al, Hemkens et al, Muckholm et al, Ioannidis et al). This may be especially important in meta-analyses, where there may be important differences in the populations studied, in the assessment of intake, and in the ascertainment of outcomes (in addition to differences in potential confounders considered).
Our Forum recently reviewed one of the papers from the Ioannidis’ group on vibration of effects in studies assessing alcohol consumption and the risk of breast cancer (Chu et al). That paper concluded that “Greater transparency when it comes to the choice, measurement, and impact of potential confounding variables is necessary. Without these efforts, the associations reported in observational studies of alcohol consumption on ischemic heart disease may need to be interpreted with great caution.” In our Forum critique of this paper (www.alcoholresearchforum.org/critique-236), reviewers appreciated the attempt to explain differences in results of studies to get a more valid estimate from meta-analyses of the true relation of alcohol consumption to the risk of breast cancer. The authors’ analyses presented were done appropriately; however, Forum members considered that their approach to dealing with such differences in meta-analyses may not be especially useful or valid. Many Forum members considered that results from single, large-scale and well-done studies might provide more precise estimates of true effect.
What is the magnitude of effect of specific confounding factors? This paper does not delve into the importance of an attempt to judge the extent to which each omitted factor might affect their results; if the estimated effects would be minimal, the absence of a single potential confounder from an analysis may not be that important. Further, it turns out that the type of beverage consumed may be important, and most studies show wine consumption to usually relate to more favorable results compared to consumption of beer or spirits; considering all types of alcohol together may not be appropriate. Grønbæk mentions some possible confounders related to wine drinkers: better self-reported subjective health, superior intelligence, drinking pattern and a healthier diet. However, regarding diet as confounder he concludes that (1) there is no clear evidence that there is a large protective effect of the healthy diet on morbidity or mortality; (2) that studies suggesting any such protective effect have not controlled their results for intake of alcohol or type of alcohol; and (3) that it appears that even a very strong confounder (odds ratio = 0.3 or 0.1) would have to exhibit a very uneven distribution among wine drinkers and non-wine drinkers to fully explain the findings of the demonstrated relation of wine and CHD.
From their study of alcohol and coronary disease, Poikkolainein et al stated: “Of the 16 comparisons (of potential major confounders) under study, seven showed significant differences between never‐drinkers and light drinkers. Five of the differences favoured never‐drinkers and two showed a disadvantage. The latter were low BMI and low leisure‐time physical activity, both more common among never‐drinkers than among light drinkers. In contrast, smoking, sleep disturbances, trait anxiety, effort–reward imbalance and dependent life events were less common among never‐drinkers than among light drinkers.” The authors found that the differences associated with such factors were unlikely to be large enough to explain the lower risk observed among light drinkers compared to abstainers; they concluded that no single one of the risk factors studied was a likely candidate for an unknown confounder that had a large effect on their results.
Is it necessary for authors to point out specifically in a paper that residual confounding is present? While residual confounding is to be expected in all observational studies, the authors of the present paper seem to be perturbed that some authors do not especially point out that their results “may be affected by residual confounding” or do not explicitly “ask for caution when interpreting results.” These limitations are fundamental for all observational studies, one that knowledgeable readers know well. (Similar to “further research is needed.”) Such statements should be assumed for all observational studies, as no study can address all possible confounders. (Or perhaps the underlying concern of the authors of the present study are particularly interested in ensuring that any documented health benefits of alcohol consumption be discounted, based on an underlying perspective that alcohol can never declared to be “beneficial” for health.)
The bottom line: what do essentially all studies of alcohol and disease indicate? Finally, the authors seem to ignore one key fact: in essentially all of these studies, regardless of variations in the number or type of confounders evaluated, the net results indicate a “J-shaped curve” for the relation of alcohol to CHD. This is not discussed. Thus, the relation of alcohol intake to CHD may be analogous to the relation of randomly collected measurements of blood pressure to the risk of CHD. Even though there is huge variability in the way blood pressure is measured (e.g., a rushed reading by a nurse as soon as someone comes into a medical office, a well-calibrated seated measurement after a period of rest, use of various devices, etc.), the net result is that blood pressure, however measured, turns out to always be an important risk factor for heart disease. Regardless of the many variations in confounder assessment that are apparently of serious concern to the authors of this paper, the consistency of results of epidemiologic studies on the effects of alcohol on CHD risk suggest that differences in potential confounders, or their number included in an analysis, may not have a major impact on results.
References from Forum critique
Chu L, Ioannidis JPA, Egilman AC, Vasiliou V, Ross JS, Wallach JD. Vibration of effects in epidemiologic studies of alcohol consumption and breast cancer risk. Inter J Epidemiol 2020;10:1–11. doi:10.1093/ije/dyz271
GBD 2016 Alcohol Collaborators. Global use and burden for 195 countries and territories, 1990-2016: a systematic analysis for the Global Burden of Disease 2016. Lancet 2018;392:1015-1035.
Groenwold RH, Van Deursen AM, Hoes AW, Hak E. Poor quality of reporting confounding bias in observational intervention studies: a systematic review. Ann Epidemiol 200818:746-751
Gronbaek M. Factors influencing the relation between alcohol and cardiovascular disease. Curr Opin Lipidol 2006;17:17-21.
Grønbæk M, Deis A, Sørensen TIA, et al. Mortality associated with moderate intake of wine, beer, or spirits. BMJ 1995;310:1165–1169.
Hemkens LG, Ewald H, Naudet F, Ladanie A, Shaw JG, Sajeev G5, Ioannidis JPA. Interpretation of epidemiologic studies very often lacked adequate consideration of confounding. J Clin Epidemiol 2018;93:94-102. doi: 10.1016/j.jclinepi.2017.09.013.
Ioannidis JP, Fanelli D, Dunne DD, Goodman SN. Meta-research: Evaluation and Improvement of Research Methods and Practices. PLoS Biol 2015;13:e1002264.
Munkholm K, Faurholt-Jepsen M, Ioannidis JPA, Hemkens LG. Consideration of confounding was suboptimal in the reporting of observational studies in psychiatry: a meta-epidemiological study. J Clin Epidemiol 2020;119:75-84.
Poikolainen K, Vahtera J, Virtanen V, et al. Alcohol and coronary heart disease risk—is there an unknown confounder? Addiction 2005;100:1150-1157.
Forum Summary
The authors of the present paper compared studies included in a recent meta-analysis on alcohol consumption and the risk of coronary heart disease (CHD) for the number of potential confounding variables included in the analysis for each paper. While they found that most studies included adjustments for smoking, age, BMI, height and weight, physical activity, and education, many studies included adjustments for a multitude of other factors. They reach the conclusion that the large variation between studies in adjusting for confounding makes it impossible to accept the finding of a J-shaped curve between alcohol consumption and CHD (despite the consistency of such results).
Forum members agree that evaluating confounders in epidemiologic studies is extremely important. However, standardizing environmental confounders is not possible as there are so many, and so many yet undefined, and these confounders would be expected to vary in their influence among different populations. The authors did not focus on the key factor: the potential impact of each potential confounder. Limited research suggests that any “unknown confounder” would need to be extremely powerful to negate the reported protective effect of light-to-moderate consumption of alcohol, especially of wine, on the risk of CHD.
In addition, the authors were perturbed that many individual studies did not state specifically in their discussion that “residual confounding may be present in our results” or that “results of individual studies must be interpreted with caution.” Forum members assume that most readers of scientific reports realize that there should always be caution in making conclusions from a single study, especially observational studies, without the authors pointing in out in their paper. It may be analogous to stating that “further research is needed,” which should be assumed for any scientific paper.
Members of the Forum acknowledge that confounding makes it a difficult process to judge causality from observational studies, but point out that potential confounders in one study may be insignificant in another. It is not possible to generate a list of potential confounders that would apply to all epidemiologic studies. However, the consistency of the J-shaped curve between alcohol intake and risk of CHD in almost all epidemiologic studies, with support from a multitude of experimental studies, strongly supports the validity of such a relation.
* * * * * *
This critique by the International Scientific Forum on Alcohol Research is based on comments provided by the following members:
Erik Skovenborg, MD, specialized in family medicine, member of the Scandinavian Medical Alcohol Board, Aarhus, Denmark
R. Curtis Ellison, MD, Professor of Medicine, Section of Preventive Medicine & Epidemiology, Boston University School of Medicine, Boston, MA, USA
Andrew L. Waterhouse, PhD, Department of Viticulture and Enology, University of California, Davis, USA
Yuqing Zhang, MD, DSc, Clinical Epidemiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA.
Creina Stockley, PhD, MSc Clinical Pharmacology, MBA; Adjunct Senior Lecturer at the University of Adelaide, Australia
David van Velden, MD, Dept. of Pathology, Stellenbosch University, Stellenbosch, South Africa
Ramon Estruch, MD, PhD, Hospital Clinic, IDIBAPS, Associate Professor of Medicine, University of Barcelona, Spain
Harvey Finkel, MD, Hematology/Oncology, Retired (Formerly, Clinical Professor of Medicine, Boston University Medical Center, Boston, MA, USA)
Giovanni de Gaetano, MD, PhD, Department of Epidemiology and Prevention, IRCCS Istituto Neurologico Mediterraneo NEUROMED, Pozzilli, Italy
Fulvio Mattivi, MSc, Department of Cellular, Computational and Integrative Biology – CIBIO and C3A, University of Trento, Italy
Tedd Goldfinger, DO, FACC, Desert Cardiology of Tucson Heart Center, University of Arizona School of Medicine, Tucson, AZ, USA
Pierre-Louis Teissedre, PhD, Faculty of Oenology–ISVV, University Victor Segalen Bordeaux 2, Bordeaux, France