1、,Variability, validity and Bias,1,Prof. Lu Ciyong Department of Epidemiology and Medical Statistics,Main contents,1. Variability -Random error -Systemic error (Bias) 2.Validity -Internal Validity -External Validity 3.Bias -Selection bias -Information bias -Confounding,2,PART 1Variability,3,Patient P

2、rofile,Fist time: blood pressure: 140/86; total serum cholesterol level: 242mg/dl (nonfasting). Second time: blood pressure: 135/84; total serum cholesterol level: 198mg/dl ( fasting lipid profile ),4,the guidelines of the National Cholesterol Education Program (NCEP),a total serum cholesterol conce

3、ntration: 240 mg/dL : pharmacologic lowering 200239 mg/dL: borderline, dietary intervention 200 mg/dL : normal,5,Variability in measurements can be either systematic or random. Random error: leads to unreliable measurement -due to chance -cannot predict - affect precision Systematic error: leads to

4、biased measurement -constant -predictable -affect validity Term bias should be reserved for systematic error,Variability in Medical Research,6,Example,random error Target A versus target B precision systematic error target D versus target C bias,7,Levels of Varibility,Table 101. Levels of Variabilit

5、y.,8,Variability Within the Individual,Table 102. Potential Sources of Variability in Measurements of Individuals,9,Variations Within Populations,Variability in populations can be considered the cumulative variability of individuals. Physicians use knowledge about variability in populations to defin

6、e what is normal and abnormal. Total serum cholesterol level : 200 mg/dL 50th percentile 240 mg/dL 75th percentile.,10,Variability Related to Measurement,Table 102. Potential Sources of Variability in Measurements,Fist time: total serum cholesterol level: 242mg/dl (nonfasting). Treatment. Second tim

7、e: total serum cholesterol level: 198mg/dl ( fasting lipid profile ) Normal.,11,Variability in Research Studies,12,Sampling Variability,The source population of 20 persons has a 25% prevalence of hypercholesterolemia (elevated cholesterol values are presented in bold). The three random samples of fi

8、ve persons yield prevalence estimates ranging from 0 to 40%.,13,The effect of Sample Size,14,In both studies, the point estimates was same. (9% for diet and 6% for drug. ) In the small study, the 95% confidence intervals are wide, and the difference in risk is not statistically significant. In the l

9、arger study, the 95% confidence intervals are narrower, and the difference in risk is statistically significant.,Example,Point estimates,95% confidence intervals,15,PART 2Validity,16,Definition of Validity,The degree to which a measurement or study reaches a correct conclusion. Types: internal vs ex

10、ternal validity,17,Internal validity,Internal validity is the extent to which the results of an investigation accurately reflect the true situation of the study population. Inferences are correct regarding the participants in the study.,18,Improve internal validity,Restricting the type of subjects a

11、nd the environment in which the study is performed.,19,External Validity,External validity is the extent to which the results of a study are applicable to other populations.(Generalizability of the result) Inferences are correct regarding the population at risk.,20,External validity,Can the study re

12、sults be applied to the eligible population? the source population? other similar populations? Depends on: whether the study population could represent the target population. whether main characteristics are similar between two populations.,21,Example,22,PART 3Bias,23,Definition of bias,Bias is a sy

13、stematic error that occurs in a study and leads to a distorted result. Bias: any trend in the design, conduct, analysis, interpretation, publication or review of data that can lead to conclusions which are systematically different from the truth. (Dictionary of Epidemiology, 3rd ed.),24,Classificati

14、on of bias,Selection bias-differential access to the study population Information bias-inaccuracy in measurement or classification Confounding-mixing of the effect of an extraneous variable,25,The evaluation of bias,Subjective Involves a judgment regarding the likelihood of (1) the presence of bias

15、(2) its direction and potential magnitude Confounding is an exception , it can be quantitated.,26,The influence of bias,Overestimation of a risk ratio!,27,The influence of bias,Underestimation of a risk ratio !,28,1.selection bias,29,Definition of selection bias,Selection bias is a distortion in the

16、 results of a study that arises because of the manner in which the subjects are sampled. Procedures for the selection of subjects depend on: 1. The design of the investigation 2. The setting of the study 3. The disease and exposure of interest.,30,Selection factors could lead to biased results at se

17、veral different steps in the process.,Selection bias,Steps in the selection and maintenance of subjects in a study.,31,Sources of Selection bias(1),Sampling bias -the degree to which the measured subjects are not representative of the population, is due to : Convenience Sample Violated random sampli

18、ng scheme Haphazard Sample,32,Sources of Selection bias(2),Self-selection (volunteerism) bias Healthy (or diseased) people may seek out participation in the study Non-response, refusal bias Response, or lack of it, depends on exposure Loss to follow-up Bias due to differences in completeness of foll

19、ow-up between comparison groups,33,Sources of Selection bias(3),Prevalence-incidence bias Prevalent case is likely to be mild, and survive longer. Referral bias Sicker patients are referred to major health centers. Healthy worker effect The workers have a more favorable outcome simply because they a

20、re healthy enough to be employed.,34,Sources of Selection bias(4),Berksons bias a hospital-based casecontrol study Patients with two medical conditions are more likely to be hospitalized than those with a single disease. might find a link between two diseases or between an exposure and a disease whe

21、n there is no association between them in the general population.,35,Admission rate bias Also known as Berksons bias. This occurs when the combination of exposure and disease under study increases the probability of admission to hospital, leading to a systematically higher exposure rate among hospit

22、al cases than among hospital controls.,Exposure of X factor among case of A disease and B disease,Exposure of X factor among case of A disease and B disease hospitalized for difference cause,Methods to reduce selection bias(1),Rigorous, but feasible random sampling scheme Clear definition of study p

23、opulation Explicit case/control, expose/non-expose definitions Cases and controls from same population,39,Methods to reduce selection bias(2),Selection of cases and controls without knowing exposure status (case-control study) Selection of exposed and non-exposed without knowing disease status (retr

24、ospective cohort study) Strive for high participation rate (incentive),40,2.Information (or misclassification) bias,41,Definition of Information bias,a distortion in the results of a study that arises because of the manner in which data are collected or classified. true-positive and true-negative, f

25、alse-positive and false-negative classifications of subjects with respect to either exposure or disease.,42,Impact of information bias: Misclassification,Measurement error leads to assigning wrong exposure or outcome category. Types: Differential vs Non-differential,43,Non-differential Misclassifica

26、tion,When errors in classification of exposure or disease status do not depend on the other variable, any bias is referred to as nondifferential. Tends to underestimate the association -tends to the null hypothesis or no association,44,Differential Misclassification,When errors in classification of

27、exposure or disease status depend upon the other variable, any bias is referred to as differential. May increase or decrease measure of effect,45,a casecontrol study , MI:myocardial infarction,Example,20%,20%,20%,0%,46,Sources of Information bias(1),Respondent: inability to understand, recall, artic

28、ulate; unwillingness to disclose or social desirability Data collector: unclear or ambiguous questions, lack of a neutral demeanor, insufficiently conscientious, inaccurate transcription, fraud Instrument: inaccurate instrument,47,Sources of Information bias(2),Data managers: inaccurate transcriptio

29、n, mis-reading, miscoding, programming errors Data analysts: variable coding and programming errors Data interpreters: inadequate appreciation of the characteristics of the measure or of the relations being studied,48,Main types of information bias(1),Interviewer bias Systematic difference in solici

30、ting, recording, interpreting information. Reporting bias Exposure may be under-reported because of attitudes, perceptions, or beliefs.,49,Main types of information bias(2),Data collection bias Bias that results from abstracting charts, interviews or surrogate interviews Recall bias results from dif

31、ferential ability of subjects to remember previous activities and exposures. Detection bias The exposure promoted more careful evaluation for the outcome of interest.,50,Techniques to reduce information bias (1),Standardize measurement instruments Administer instruments equally to cases /controls or

32、 exposed/unexposed Use objective parameters if possible, such as biological markers Use multiple sources of information Questionnaires,51,Techniques to reduce information bias (2),Direct measurements Registries Case records Use multiple controls Use re-classified subjects to check for bias Use memor

33、y aids and validate exposures Provide standardized training sessions and protocols,52,Techniques to reduce information bias (3),Try to ensure that question are clearly understood through careful wording and pretesting Blinding: -Blind participants as to study goals and participants classification st

34、atus -Blind researchers as to subjects study status,53,3.Confounding,54,Definition of Confounding,Confounding is a distortion in an association between a study exposure and disease brought about by the influence of extraneous factors.,55,Confounders,extraneous factor that biases the association we a

35、re interested in must satisfy two conditions: (1) related to the risk of disease among unexposed persons (2) are also associated with the exposure of interest (3) but not as a result of being exposed.,56,Exposure,Outcome,Third variable,To be a confounding factor, two conditions must be met:,Be assoc

36、iated with exposure - without being the consequence of exposure,Be associated with outcome - independently of exposure (not an intermediary),Confounding,57,the relationship between total serum cholesterol level and risk of developing myocardial infarction, with confounding by obesity.,Example,58,Eva

37、luation of confounding(1),36 of 60 patients with MI (60%) are found to have high total serum cholesterol levels, and only 24 of 60 controls (40%) are discovered to have elevated serum cholesterol levels.,This would suggest that elevated total serum cholesterol levels are associated with an increased

38、 risk of developing MI.,59,When examined separately in obese and nonobese persons, however.,Evaluation of confounding(2),Thus in the case of both obese and nonobese individuals, elevated total serum cholesterol levels are more common in controls than in patients with myocardial infarction.,60,Evalua

39、tion of confounding(3),the results are confounded by the extraneous variable, obesity,61,Control of confounding(1),Design strategies Randomization -tends to balance the study groups with respect to both known and unknown determinants of the outcome Restriction -investigators purposely limit study subjects Matching -the controls are similar to the cases in the distribution of the confounders.,62,Control of confounding(2),Analys