Previous page: Analyzing Cholesterol Dataset – Part 2

So far all of our analyses have asked questions about the manipulation of a single independent variable. The t-test can compare two groups/levels while the ANOVA can ask about the differences between multiple levels. But, what do we do when there is more than one independent variable being manipulated within our experiment? What if we want to know how these factors interact with each other to produce our final result? To demonstrate how you could assess these questions we’ll again go through our Cholesterol.csv dataset.

Two-Way Mixed ANOVA

To correctly analyze this dataset we use a two-way mixed model ANOVA. It is “two-way” because there are two independent variables being manipulated. It is a “mixed model” because one independent variable, participation time, is within-subjects and the other independent variable, margarine type, is between-subjects.

To run the two-way mixed NOVA model, apply the following steps:

1. Load the Cholesterol.csv data at the top of the MagicStat (version 1.1.3) and press Explore to begin.

2. Select a model to analyze your data.

3. After choosing the Two-Way Mixed ANOVA (Factorial Between and Within Subjects ANOVA) model you will be asked, Is your dataset long or wide format? This data uses one row per participant so we should select wide and proceeded. At this point the left panel of your screen should show the following.

4. Select a between subjects variable

Since we are using a mixed ANOVA model we need to tell the program which column of our dataset denotes the levels of our between subjects variable. You can look to the data preview in the right panel and see the Margarine column labels each participant as having received either margarine type A or B.

Choose Margarine as your between subjects variable.

5. Naming variables: After specifying a between subjects variable you are asked to name the within subjects variable as well as the dependent measure. Although, this step is optional we highly recommend taking a moment to give your variables useful and meaningful names. In the coming steps there will be many charts and tables to consider; having useful labels for our independent and dependent variables helps us keep all of the factors and relevant comparisons straight in our heads.

We chose the label time for my independent variable and chol for my dependent variable. Avoid using overly long descriptions for these names because long labels will make the resulting charts and tables harder to read. You want to provide the minimum necessary label to remain useful without cluttering the visual space of your figures.

6. Specify levels of within-subjects variable: The final step before we can run our analysis asks us to choose which columns of our data represent the levels of our time within-subjects variable.

In the left gray box select Before then click the rightward-facing arrow, >, to select it as one of the time levels. Repeat this process for the After4Weeks and After8Weeks labels.

When your display looks like the above image click Analyze to see the results of your two-way mixed ANOVA.

Results

The output of a two-way ANOVA can seem daunting so we’ll go through it piece-by-piece. Luckily, we’ve established a strong foundation of understanding by beginning with paired-samples t-tests and the one-way repeated measures ANOVA. The concepts we built up there will prove very helpful in tackling this more complex analysis.

The first table of results shown is the breakdown of the sources of variance in our data. As was the case with out one-way ANOVA, each of the rows of interest have sum of square (SS), degrees of freedom (df), mean square (MS), F statistic, and p value columns. Here we will not focus on the full calculations but it is enough to keep in mind that each F-value is essentially a ratio of explainable over unexplained error variance.

To guide our reading of this table it is best to remember the paramaters of the experiment.

• We manipulated participation time in our study within subjects
• We manipulated Margarine type between subjects
• We want to know whether these two factors interact with each other to effect cholesterol level

For our time manipulation we are asking whether the level of the independent time can explain a statistically significant proportion of the variance in our data? Looking to the time row we see the observed F-value of 259.49 and the associated p value of 0.000. As per convention, this p is said to be statistically significant because it is < 0.05. This means we can say yes, the level of our independent variable time has a statistically significant effect on the mean level of cholesterol.

For the Margarine manipulation we are asking a similar question. Does the type of margarine used explain a statistically significant proportion of the variance in our data? Looking to the Margarine row we see an observed F-value of 1.45 and the associated p value of 0.247. As per convention, this p is not statistically significant because it is >= 0.05. We fail to reject the null hypothesis that type of margarine does not have a statistically significant effect on mean level of cholesterol.

Interactions

For consideration of the interaction between these two factors we look to the Margarine X time row. Here the question is not about the effects of our factors in isolation but instead we are asking whether the level(s) our factors have an effect on each other. This type of effect is most easily understood in the domain of medicine where we commonly hear it invoked.

Imagine a patient with two underlying health conditions, both requiring medication to manage them. Independently, each of these medications would improve the health of this patient. But, if the effects of these medications interact with one another then the addition of the second medication will change the effectivness of the intervention.

This interaction can play out in many different ways. Together they could lead to more improvement than would be expected by adding up their independent effects (super addativity). Their combined effectivness could be less than would be expected from adding the effects together (sub-addativity). It could even be dangerous and detrimental to health by combining these medications (cross-over interaction). The important thing to understand is that an interaction means a particular combination of the levels of our factors can produce their own effects on the result.

When we look to the Margarine X time row we see an observed F-value of 4.78 and the associated p value of 0.015. Therefore, we reject the null hypothesis that our two independent variables do not interact with one another.

ANOVA Conclusions

In summary, our table of ANOVA results revealed the following:

1. There is a statistically significant effect of time
2. There is not a statistically significant effect of Margarine
3. There is statistically significant interaction of Margarine x time

These statistically significant ANOVA results only tell us that all levels do not produce the same results. To know which groups differ and the direction(s) of those difference we look to our descriptive statistics and pairwise comparisons.

Descriptive Statistics

Above we are given the Mean, standard deviation (SD), standard error of the mean (SEM), and number of participants (N) for each of our 6 experimental conditions.

Mean cholesterol level for participants given margarine B decreased across the study. They began at 6.78, dropped to 6.13 after 4 weeks, and ended the 8 week intervention at 6.07.

Mean cholesterol level for participants given margarine A also decreased across the study. They began at 6.04, dropped to 5.55 after 4 weeks, and ended the 8 week intervention at 5.49.

Charts

Although we have all of the raw group means in our descriptive statistics, it is often very helpful to visualize the results of our experiment using charts. Both of the charts below are representing the same data obtained from our descriptive statistics. The only difference between the charts is the variable chosen to place on the X-axis. Using multiple representations of the same data is informative because some patterns “pop out” at us more readily in one configuration or another.

In the first chart we see type of Margarine on the X-axis and each level of time as a separate line. Cholesterol scores are generally lower for the A than the B margarine groups. ANOVA F-table results tell us this between-subjects manipulation of Margarine is not statistically significant (p = 0.247).

Our second chart shows time on the X-axis and type of margarine as separate lines. Here we see the decrease in cholesterol as the study progresses and we also see that this pattern is largely the same for the A and B margarine groups. Differences between the Before and After4Weeks groups are large; differences between the After4Weeks and After8Weeks groups appear small.

Pairwise Comparisons

With our general understanding of the patterns in our data we can move on to the pairwise post-hoc comparisons. These comparisons will tell us which of our apparent group differences are statistically significant and which are not.

For our purposes, the most important columns are Group 1, Group 2, Reject, and p value.

• Group 1 and Group 2 tell us which two experimental conditions are being compared.
• p value and Reject tell us whether the group difference being compared is statistically significant (Reject = True).

• Confirming our ANOVA result, we see no main effect of Margarine type on mean cholesterol level (p = 0.247).
• The next two rows show statistically significant simple effects of time with the B-type margarine. More specifically, for participants given B-type margarine there were statistically significant differences between the Before and After4Weeks groups as well as between the Before and After8Weeks groups (p = 0.000 for both comparisons). The next row shows no statistically significant difference between After4Weeks and After8Weeks groups for participants given B-type margarine (p = 0.294).
• The last three rows of the Margarine Post-Hoc Tests table show the simple effects of time for participants given the A-type margarine. This pattern is largely the same as was observed for participants given B-type margarine. Differences between Before and After4Weeks as well as differences between Before and After8Weeks groups were statistically significant for participants given A-type margarine (p = 0.000 for both comparisons). Just as was seen with B-type margarine, differences between After4Weeks and After8Weeks were not statistically significant for participants given A-type margarine (p = 0.060).

• The first three rows of the time Post-Hoc Tests show statistically significant differences between each level of the time variable.
  • Before vs After4Weeks (p = 0.000)
  • Before vs After8Weeks (p = 0.000)
  • After4Weeks vs After8Weeks (p = 0.004)

• The last three rows of the table compare groups given margarine A to groups given margarine B at each level of the time variable.
  • A vs B at Before (p = 0.475)
  • A vs B at After4Weeks (p = 0.633)
  • A vs B at After4Weeks (p = 0.622)
• None of these three comparisons rises to the level of statistical significance.

Conclusions

Full analysis of our Cholesterol.csv dataset under a two-way mixed ANOVA model shows a statistically significant effect of our time intervention.

Participation in this study lead to a decrease in mean cholesterol level for all experimental groups. Cholesterol significantly dropped from during the 1st 4 weeks of participation and continued to drop (although less dramatically) given an additional 4 weeks of margarine use.

The type of margarine used by participants did not have a statistically significant effect on the mean cholesterol level. Both were equally effective at decreasing group mean cholesterol levels.

There was a statistically significant interaction observed between time and Margarine although none of the post-hoc comparisons shed light upon how this interaction is operating in our study.

It is possible our comparisons of cell means to assess differential effectiveness of margarine types was underpowered due to small sample sizes within each cell (N = 9). When comparing After4Weeks to After8Weeks groups for either margarine A or B we failed to find significant differences. When using a more powerful test which collapsed over type of Margarine the difference between After4Weeks and After8Weeks was significant.