I am using Mixed Model in SPSS and I have had this question often so I decided to post it....
I start out with one model with Groups (2: Patient, Control) and 9 language area connections (IFGL_IFGR, MFGL_MFGR, etc). In setting up the Linear Mixed Model in SPSS, the subject numbers (there are 54 total) are "Subject" and these 9 language area connections (ROI_Connection) are "Repeated". I am using "Unstructured" covariance structure bc that gives me the best AIC. "Dependent variable" is a Fisher's Z score (for each of the 9 ROI_Connections repeated). And my "Factors" are Group and ROI_Connection. I build the model, run the analysis and get the results. There is a main effect of Group (F(1,52)=4.752, p=0.034) and of ROI_Connection (F(8,52)=13.484, p=0.000), but no interaction of Group x ROI_Connection. The main effect of Group is my main hypothesis for the study, so I am happy and write these numbers in my manuscript.
However, later I decide that I am interested in adding another variable, such as Language Dominance (2 choices: Atypical, Typical). So now everything is the same as above in the model but I have added a "Factor" of Language Dominance and I now have 3 Factors (Group, ROI_Connection, Language Dominance). I build the model, run the analysis and get the results. I see that there is still a main effect of Group (F(1,50)=14.303, p=0.000)) and ROI_Connection (F(8,50)=6.246, p=0.000)), as before, as well as the new main effect of Language Dominance (F(1,50)=7.920, p=0.007)) and a Group x Language Dominance interaction (F(1,50)=6.918, p=0.011)). None of the other 2- or 3-way interactions in the model are significant.
If I decide to include this second analysis/model in the paper, when I report the main effect of Group (the main hypothesis for the study) is it best to report the numbers from the second model or the first model? Obviously the numbers are quite different for the main effect of Group with the addition of the Language Dominance factor, but I originally just planned to look at the first model. So I am not sure if I should report the first model for the Group effect and the second model for the Language Dominance effect or if it is best to have everything in the same model.
Maybe I should check the AIC for each model and decide that way?
This will continue to happen bc I keep thinking of additional things to look at, so I wanted to ask the experts.
Any help is greatly appreciated!