1、1,Mixed Analysis of Variance Models with SPSS,Robert A.Yaffee, Ph.D. Statistics, Social Science, and Mapping Group Information Technology Services/Academic Computing Services Office location: 75 Third Avenue, Level C-3 Phone: 212-998-3402,2,Outline,Classification of Effects Random Effects Two-Way Ra

2、ndom Layout Solutions and estimates General linear model Fixed Effects Models The one-way layout Mixed Model theory Proper error terms Two-way layout Full-factorial model Contrasts with interaction terms Graphing Interactions,3,Outline-Contd,Repeated Measures ANOVA Advantages of Mixed Models over GL

3、M.,4,Definition of Mixed Modelsby their component effects,Mixed Models contain both fixed and random effects Fixed Effects: factors for which the only levels under consideration are contained in the coding of those effects Random Effects: Factors for which the levels contained in the coding of those

4、 factors are a random sample of the total number of levels in the population for that factor.,5,Examples of Fixed and Random Effects,Fixed effect: Sex where both male and female genders are included in the factor, sex. Agegroup: Minor and Adult are both included in the factor of agegroup Random effe

5、ct: Subject: the sample is a random sample of the target population,6,Classification of effects,There are main effects: Linear Explanatory Factors There are interaction effects: Joint effects over and above the component main effects.,7,8,Classification of Effects-contd,Hierarchical designs have nes

6、ted effects. Nested effects are those with subjects within groups. An example would be patients nested within doctors and doctors nested within hospitals This could be expressed by patients(doctors) doctors(hospitals),9,10,Between and Within-Subject effects,Such effects may sometimes be fixed or ran

7、dom. Their classification depends on the experimental designBetween-subjects effects are those who are in one group or another but not in both. Experimental group is a fixed effect because the manager is considering only those groups in his experiment. One group is the experimental group and the oth

8、er is the control group. Therefore, this grouping factor is a between- subject effect. Within-subject effects are experienced by subjects repeatedly over time. Trial is a random effect when there are several trials in the repeated measures design; all subjects experience all of the trials. Trial is

9、therefore a within-subject effect.Operator may be a fixed or random effect, depending upon whether one is generalizing beyond the sampleIf operator is a random effect, then the machine*operator interaction is a random effect.There are contrasts: These contrast the values of one level with those of o

10、ther levels of the same effect.,11,Between Subject effects,Gender: One is either male or female, but not both. Group: One is either in the control, experimental, or the comparison group but not more than one.,12,Within-Subjects Effects,These are repeated effects. Observation 1, 2, and 3 might be the

11、 pre, post, and follow-up observations on each person. Each person experiences all of these levels or categories. These are found in repeated measures analysis of variance.,13,Repeated Observations are Within-Subjects effects,Trial 1 Trial 2 Trial 3,Group,Group is a between subjects effect, whereas

12、Trial is a within subjects effect.,14,The General Linear Model,The main effects general linear model can be parameterized as,15,A factorial model,If an interaction term were included, the formula would be,The interaction or crossed effect is the joint effect, over and above the individual main effec

13、ts. Therefore, the main effects must be in the model for the interaction to be properly specified.,16,Higher-Order Interactions,If 3-way interactions are in the model, then the main effects and all lower order interactions must be in the model for the 3-way interaction to be properly specified. For

14、example, a 3-way interaction model would be:,17,The General Linear Model,In matrix terminology, the general linear model may be expressed as,18,Assumptions,Of the general linear model,19,General Linear Model Assumptions-contd,1. Residual Normality. 2. Homogeneity of error variance 3. Functional form

15、 of Model: Linearity of Model 4. No Multicollinearity 5. Independence of observations 6. No autocorrelation of errors 7. No influential outliers,We have to test for these to be sure that the model is valid. We will discuss the robustness of the model in face of violations of these assumptions. We wi

16、ll discuss recourses when these assumptions are violated.,20,Explanation of these assumptions,Functional form of Model: Linearity of Model: These models only analyze the linear relationship. Independence of observations Representativeness of sample Residual Normality: So the alpha regions of the sig

17、nificance tests are properly defined. Homogeneity of error variance: So the confidence limits may be easily found. No Multicollinearity: Prevents efficient estimation of the parameters. No autocorrelation of errors: Autocorrelation inflates the R2 ,F and t tests. No influential outliers: They bias t

18、he parameter estimation.,21,Diagnostic tests for these assumptions,Functional form of Model: Linearity of Model: Pair plot Independence of observations: Runs test Representativeness of sample: Inquire about sample design Residual Normality: SK or SW test Homogeneity of error variance Graph of Zresid

19、 * Zpred No Multicollinearity: Corr of X No autocorrelation of errors: ACF No influential outliers: Leverage and Cooks D.,22,Testing for outliers,Frequencies analysis of stdres cksd. Look for standardized residuals greater than 3.5 or less than 3.5 And look for Cooks D.,23,Studentized Residuals,Bels

20、ley et al (1980) recommend the use of studentized Residuals to determine whether there is an outlier.,24,Influence of Outliers,Leverage is measured by the diagonal components of the hat matrix. The hat matrix comes from the formula for the regression of Y.,25,Leverage and the Hat matrix,The hat matr

21、ix transforms Y into the predicted scores. The diagonals of the hat matrix indicate which values will be outliers or not. The diagonals are therefore measures of leverage. Leverage is bounded by two limits: 1/n and 1. The closer the leverage is to unity, the more leverage the value has. The trace of

22、 the hat matrix = the number of variables in the model. When the leverage 2p/n then there is high leverage according to Belsley et al. (1980) cited in Long, J.F. Modern Methods of Data Analysis (p.262). For smaller samples, Vellman and Welsch (1981) suggested that 3p/n is the criterion.,26,Cooks D,A

23、nother measure of influence. This is a popular one. The formula for it is:,Cook and Weisberg(1982) suggested that values of D that exceeded 50% of the F distribution (df = p, n-p) are large.,27,Cooks D in SPSS,Finding the influential outliers Select those observations for which cksd (4*p)/n Belsley

24、suggests 4/(n-p-1) as a cutoff If cksd (4*p)/(n-p-1);,28,What to do with outliers,1. Check coding to spot typos 2. Correct typos 3. If observational outlier is correct, examine the dffits option to see the influence on the fitting statistics. 4. This will show the standardized influence of the obser

25、vation on the fit. If the influence of the outlier is bad, then consider removal or replacement of it with imputation.,29,Decomposition of the Sums of Squares,Mean deviations are computed when means are subtracted from individual scores. This is done for the total, the group mean, and the error term

26、s. Mean deviations are squared and these are called sums of squares Variances are computed by dividing the Sums of Squares by their degrees of freedom. The total Variance = Model Variance + error variance,30,Formula for Decomposition of Sums of Squares,SS total = SS error + SSmodel,31,Variance Decom

27、position,Dividing each of the sums of squares by their respective degrees of freedom yields the variances. Total variance= error variance + model variance.,32,Proportion of Variance Explained,R2 = proportion of variance explained. SStotal = SSmodel + SSerrror Divide all sides by SStotal SSmodel/SSto

28、tal =1 - SSError/SStotal R2=1 - SSError/SStotal,33,The Omnibus F test,The omnibus F test is a test that all of the means of the levels of the main effects and as well as any interactions specified are not significantly different from one another.,Suppose the model is a one way anova on breaking pres

29、sure of bonds of different metals. Suppose there are three metals: nickel, iron, and Copper. H0: Mean(Nickel)= mean (Iron) = mean(Copper) Ha: Mean(Nickel) ne Mean(Iron) or Mean(Nickel) ne Mean(Copper) or Mean(Iron) ne Mean(Copper),34,Testing different Levels of a Factor against one another,Contrast

30、are tests of the mean of one level of a factor against other levels.,35,Contrasts-contd,A contrast statement computes,The estimated V- is the generalized inverse of the coefficient matrix of the mixed model. The L vector is the kb vector. The numerator df is the rank(L) and the denominator df is tak

31、en from the fixed effects table unless otherwise specified.,36,Construction of the F tests in different models,The F test is a ratio of two variances (Mean Squares). It is constructed by dividing the MS of the effect to be tested by a MS of the denominator term. The division should leave only the ef

32、fect to be tested left over as a remainder.,A Fixed Effects model F test for a = MSa/MSerror. A Random Effects model F test for a = MSa/MSab A Mixed Effects model F test for b = MSa/MSab A Mixed Effects model F test for ab = MSab/MSerror,37,Data format,The data format for a GLM is that of wide data.

33、,38,Data Format for Mixed Models is Long,39,Conversion of Wide to Long Data Format,Click on Data in the header bar Then click on Restructure in the pop-down menu,40,A restructure wizard appears,Select restructure selected variables into cases and click on Next,41,A Variables to Cases: Number of Vari

34、able Groups dialog box appears. We select one and click on next.,42,We select the repeated variables and move them to the target variable box,43,After moving the repeated variables into the target variable box, we move the fixed variables into the Fixed variable box, and select a variable for case i

35、din this case, subject.Then we click on Next,44,A create index variables dialog box appears. We leave the number of index variables to be created at one and click on next at the bottom of the box,45,When the following box appears we just type in time and select Next.,46,When the options dialog box a

36、ppears, we select the option for dropping variables not selected.We then click on Finish.,47,We thus obtain our data in long format,48,The Mixed Model,The Mixed Model uses long data format. It includes fixed and random effects. It can be used to model merely fixed or random effects, by zeroing out t

37、he other parameter vector. The F tests for the fixed, random, and mixed models differ. Because the Mixed Model has the parameter vector for both of these and can estimate the error covariance matrix for each, it can provide the correct standard errors for either the fixed or random effects.,49,The M

38、ixed Model,50,Mixed Model Theory-contd,Little et al.(p.139) note that u and e are uncorrelated random variables with 0 means and covariances, G and R, respectively.,V- is a generalized inverse. Because V is usually singular and noninvertible AVA = V- is an augmented matrix that is invertible. It can

39、 later be transformed back to V. The G and R matrices must be positive definite. In the Mixed procedure, the covariance type of the random (generalized) effects defines the structure of G and a repeated covariance type defines structure of R.,51,Mixed Model Assumptions,A linear relationship between

40、dependent and independent variables,52,Random Effects Covariance Structure,This defines the structure of the G matrix, the random effects, in the mixed model. Possible structures permitted by current version of SPSS: Scaled Identity Compound Symmetry AR(1) Huynh-Feldt,53,Structures of Repeated effec

41、ts (R matrix)-contd,54,Structures of Repeated Effects (R matrix),55,Structures of Repeated effects (R matrix) contd,56,R matrix, defines the correlation among repeated random effects,One can specify the nature of the correlation among the repeated random effects.,57,GLM Mixed Model,The General Linea

42、r Model is a special case of the Mixed Model with Z = 0 (which means that Zu disappears from the model) and,58,Mixed Analysis of a Fixed Effects model,SPSS tests these fixed effects just as it does with the GLM Procedure with type III sums of squares. We analyze the breaking pressure of bonds made f

43、rom three metals. We assume that we do not generalize beyond our sample and that our effects are all fixed.,Tests of Fixed Effects is performed with the help of the L matrix by constructing the following F test:,Numerator df = rank(L) Denominator df = RESID (n-rank(X) df = Satherth,59,Estimation: Ne

44、wton Scoring,60,Estimation: Minimization of the objective functions,Using Newton Scoring, the following functions are minimized,61,Significance of Parameters,62,Test one covariance structure against the other with the IC,The rule of thumb is smaller is better -2LL AIC Akaike AICC Hurvich and Tsay BI

45、C Bayesian Info Criterion Bozdogans CAIC,63,Measures of Lack of fit: The information Criteria,-2LL is called the deviance. It is a measure of sum of squared errors. AIC = -2LL + 2p (p=# parms) BIC = Schwartz Bayesian Info criterion = 2LL + plog(n) AICC= Hurvich and Tsays small sample correction on A

46、IC: -2LL + 2p(n/(n-p-1) CAIC = -2LL + p(log(n) + 1),64,Procedures for Fitting the Mixed Model,One can use the LR test or the lesser of the information criteria. The smaller the information criterion, the better the model happens to be. We try to go from a larger to a smaller information criterion wh

47、en we fit the model.,65,LR test,To test whether one model is significantly better than the other. To test random effect for statistical significance To test covariance structure improvement To test both. Distributed as a With df= p2 p1 where pi =# parms in model i,66,Applying the LR test,We obtain t

48、he -2LL from the unrestricted model. We obtain the -2LL from the restricted model. We subtract the latter from the larger former. That is a chi-square with df= the difference in the number of parameters. We can look this up and determine whether or not it is statistically significant.,67,Advantages

49、of the Mixed Model,It can allow random effects to be properly specified and computed, unlike the GLM. It can allow correlation of errors, unlike the GLM. It therefore has more flexibility in modeling the error covariance structure. It can allow the error terms to exhibit nonconstant variability, unl

50、ike the GLM, allowing more flexibility in modeling the dependent variable. It can handle missing data, whereas the repeated measures GLM cannot.,68,Programming A Repeated Measures ANOVA with PROC Mixed,Select the Mixed Linear Option in Analysis,69,Move subject ID into the subjects box and the repeated variable into the re