Previous page: Analyzing Cholesterol Dataset – Part 2
So far all of our analyses have asked questions about the manipulation of a single independent variable. The t-test can compare two groups/levels while the ANOVA can ask about the differences between multiple levels. But, what do we do when there is more than one independent variable being manipulated within our experiment? What if we want to know how these factors interact with each other to produce our final result? To demonstrate how you could assess these questions we’ll again go through our
Two-Way Mixed ANOVA
To correctly analyze this dataset we use a two-way mixed model ANOVA. It is “two-way” because there are two independent variables being manipulated. It is a “mixed model” because one independent variable, participation time, is within-subjects and the other independent variable, margarine type, is between-subjects.
To run the two-way mixed NOVA model, apply the following steps:
1. Load the
Cholesterol.csv data at the top of the MagicStat (version 1.1.3) and press
Explore to begin.
2. Select a model to analyze your data.
3. After choosing the Two-Way Mixed ANOVA (Factorial Between and Within Subjects ANOVA) model you will be asked, Is your dataset long or wide format? This data uses one row per participant so we should select
wide and proceeded. At this point the left panel of your screen should show the following.
Select a between subjects variable
Since we are using a mixed ANOVA model we need to tell the program which column of our dataset denotes the levels of our between subjects variable. You can look to the data preview in the right panel and see the
Margarine column labels each participant as having received either margarine type
Margarine as your between subjects variable.
5. Naming variables: After specifying a between subjects variable you are asked to name the within subjects variable as well as the dependent measure. Although, this step is optional we highly recommend taking a moment to give your variables useful and meaningful names. In the coming steps there will be many charts and tables to consider; having useful labels for our independent and dependent variables helps us keep all of the factors and relevant comparisons straight in our heads.
We chose the label
time for my independent variable and
chol for my dependent variable. Avoid using overly long descriptions for these names because long labels will make the resulting charts and tables harder to read. You want to provide the minimum necessary label to remain useful without cluttering the visual space of your figures.
6. Specify levels of within-subjects variable: The final step before we can run our analysis asks us to choose which columns of our data represent the levels of our
time within-subjects variable.
In the left gray box select
Before then click the rightward-facing arrow,
>, to select it as one of the
time levels. Repeat this process for the
When your display looks like the above image click
Analyze to see the results of your two-way mixed ANOVA.
The output of a two-way ANOVA can seem daunting so we’ll go through it piece-by-piece. Luckily, we’ve established a strong foundation of understanding by beginning with paired-samples t-tests and the one-way repeated measures ANOVA. The concepts we built up there will prove very helpful in tackling this more complex analysis.
The first table of results shown is the breakdown of the sources of variance in our data. As was the case with out one-way ANOVA, each of the rows of interest have sum of square (
SS), degrees of freedom (
df), mean square (
F statistic, and
p value columns. Here we will not focus on the full calculations but it is enough to keep in mind that each
F-value is essentially a ratio of explainable over unexplained error variance.
To guide our reading of this table it is best to remember the paramaters of the experiment.
- We manipulated participation
timein our study within subjects
- We manipulated
Margarinetype between subjects
- We want to know whether these two factors interact with each other to effect cholesterol level
time manipulation we are asking whether the level of the independent
time can explain a statistically significant proportion of the variance in our data? Looking to the
time row we see the observed
259.49 and the associated
p value of
0.000. As per convention, this
p is said to be statistically significant because it is
< 0.05. This means we can say yes, the level of our independent variable
time has a statistically significant effect on the mean level of cholesterol.
Margarine manipulation we are asking a similar question. Does the type of margarine used explain a statistically significant proportion of the variance in our data? Looking to the
Margarine row we see an observed
1.45 and the associated
p value of
0.247. As per convention, this
p is not statistically significant because it is
>= 0.05. We fail to reject the null hypothesis that type of margarine does not have a statistically significant effect on mean level of cholesterol.
For consideration of the interaction between these two factors we look to the
Margarine X time row. Here the question is not about the effects of our factors in isolation but instead we are asking whether the level(s) our factors have an effect on each other. This type of effect is most easily understood in the domain of medicine where we commonly hear it invoked.
Imagine a patient with two underlying health conditions, both requiring medication to manage them. Independently, each of these medications would improve the health of this patient. But, if the effects of these medications interact with one another then the addition of the second medication will change the effectivness of the intervention.
This interaction can play out in many different ways. Together they could lead to more improvement than would be expected by adding up their independent effects (super addativity). Their combined effectivness could be less than would be expected from adding the effects together (sub-addativity). It could even be dangerous and detrimental to health by combining these medications (cross-over interaction). The important thing to understand is that an interaction means a particular combination of the levels of our factors can produce their own effects on the result.
When we look to the
Margarine X time row we see an observed
4.78 and the associated
p value of
0.015. Therefore, we reject the null hypothesis that our two independent variables do not interact with one another.
In summary, our table of ANOVA results revealed the following:
- There is a statistically significant effect of
- There is not a statistically significant effect of
- There is statistically significant interaction of
Margarine x time
These statistically significant ANOVA results only tell us that all levels do not produce the same results. To know which groups differ and the direction(s) of those difference we look to our descriptive statistics and pairwise comparisons.
Above we are given the
Mean, standard deviation (
SD), standard error of the mean (
SEM), and number of participants (
N) for each of our 6 experimental conditions.
Mean cholesterol level for participants given margarine
B decreased across the study. They began at
6.78, dropped to
6.13 after 4 weeks, and ended the 8 week intervention at
Mean cholesterol level for participants given margarine
A also decreased across the study. They began at
6.04, dropped to
5.55 after 4 weeks, and ended the 8 week intervention at
Although we have all of the raw group means in our descriptive statistics, it is often very helpful to visualize the results of our experiment using charts. Both of the charts below are representing the same data obtained from our descriptive statistics. The only difference between the charts is the variable chosen to place on the X-axis. Using multiple representations of the same data is informative because some patterns “pop out” at us more readily in one configuration or another.
In the first chart we see type of
Margarine on the X-axis and each level of
time as a separate line. Cholesterol scores are generally lower for the
A than the
B margarine groups. ANOVA
F-table results tell us this between-subjects manipulation of
Margarine is not statistically significant (
p = 0.247).
Our second chart shows
time on the X-axis and type of margarine as separate lines. Here we see the decrease in cholesterol as the study progresses and we also see that this pattern is largely the same for the
B margarine groups. Differences between the
After4Weeks groups are large; differences between the
After8Weeks groups appear small.
With our general understanding of the patterns in our data we can move on to the pairwise post-hoc comparisons. These comparisons will tell us which of our apparent group differences are statistically significant and which are not.
For our purposes, the most important columns are
Group 2tell us which two experimental conditions are being compared.
Rejecttell us whether the group difference being compared is statistically significant (
Reject = True).
- Confirming our ANOVA result, we see no main effect of
Margarinetype on mean cholesterol level (
p = 0.247).
- The next two rows show statistically significant simple effects of
B-type margarine. More specifically, for participants given
B-type margarine there were statistically significant differences between the
After4Weeksgroups as well as between the
p = 0.000for both comparisons). The next row shows no statistically significant difference between
After8Weeksgroups for participants given
B-type margarine (
p = 0.294).
- The last three rows of the
Margarine Post-Hoc Teststable show the simple effects of
timefor participants given the
A-type margarine. This pattern is largely the same as was observed for participants given
B-type margarine. Differences between
After4Weeksas well as differences between
After8Weeksgroups were statistically significant for participants given
A-type margarine (
p = 0.000for both comparisons). Just as was seen with
B-type margarine, differences between
After8Weekswere not statistically significant for participants given
A-type margarine (
p = 0.060).
- The first three rows of the
time Post-Hoc Testsshow statistically significant differences between each level of the
p = 0.000)
p = 0.000)
p = 0.004)
- The last three rows of the table compare groups given margarine
Ato groups given margarine
Bat each level of the
p = 0.475)
p = 0.633)
p = 0.622)
- None of these three comparisons rises to the level of statistical significance.
Full analysis of our
Cholesterol.csv dataset under a two-way mixed ANOVA model shows a statistically significant effect of our
Participation in this study lead to a decrease in mean cholesterol level for all experimental groups. Cholesterol significantly dropped from during the 1st 4 weeks of participation and continued to drop (although less dramatically) given an additional 4 weeks of margarine use.
The type of margarine used by participants did not have a statistically significant effect on the mean cholesterol level. Both were equally effective at decreasing group mean cholesterol levels.
There was a statistically significant interaction observed between
Margarine although none of the post-hoc comparisons shed light upon how this interaction is operating in our study.
It is possible our comparisons of cell means to assess differential effectiveness of margarine types was underpowered due to small sample sizes within each cell (
N = 9). When comparing
After8Weeks groups for either margarine
B we failed to find significant differences. When using a more powerful test which collapsed over type of
Margarine the difference between
After8Weeks was significant.