*Previous page:* Analyzing Cholesterol Dataset – Part 2

So far all of our analyses have asked questions about the manipulation of a single independent variable. The *t*-test can compare two groups/levels while the ANOVA can ask about the differences between multiple levels. But, what do we do when there is more than one independent variable being manipulated within our experiment? What if we want to know how these factors interact with each other to produce our final result? To demonstrate how you could assess these questions we’ll again go through our `Cholesterol.csv`

dataset.

**Two-Way Mixed ANOVA**

To correctly analyze this dataset we use a two-way mixed model ANOVA. It is “two-way” because there are two independent variables being manipulated. It is a “mixed model” because one independent variable, participation time, is within-subjects and the other independent variable, margarine type, is between-subjects.

To run the two-way mixed NOVA model, apply the following steps:

**1.** Load the `Cholesterol.csv`

data at the top of the MagicStat (version 1.1.3) and press `Explore`

to begin.

**2.** Select a model to analyze your data.

**3.** After choosing the Two-Way Mixed ANOVA (Factorial Between and Within Subjects ANOVA) model you will be asked, Is your dataset long or wide format? This data uses one row per participant so we should select `wide`

and proceeded. At this point the left panel of your screen should show the following.

**4.** `Select a between subjects variable`

Since we are using a mixed ANOVA model we need to tell the program which column of our dataset denotes the levels of our between subjects variable. You can look to the data preview in the right panel and see the `Margarine`

column labels each participant as having received either margarine type `A`

or `B`

.

Choose `Margarine`

as your between subjects variable.

**5.** **Naming variables**: After specifying a between subjects variable you are asked to name the within subjects variable as well as the dependent measure. Although, this step is optional we *highly recommend* taking a moment to give your variables useful and meaningful names. In the coming steps there will be many charts and tables to consider; having useful labels for our independent and dependent variables helps us keep all of the factors and relevant comparisons straight in our heads.

We chose the label `time`

for my independent variable and `chol`

for my dependent variable. Avoid using overly long descriptions for these names because long labels will make the resulting charts and tables harder to read. You want to provide the minimum necessary label to remain useful without cluttering the visual space of your figures.

**6.** **Specify levels of within-subjects variable**: The final step before we can run our analysis asks us to choose which columns of our data represent the levels of our `time`

within-subjects variable.

In the left gray box select `Before`

then click the rightward-facing arrow, `>`

, to select it as one of the `time levels`

. Repeat this process for the `After4Weeks`

and `After8Weeks`

labels.

When your display looks like the above image click `Analyze`

to see the results of your two-way mixed ANOVA.

**Results**

The output of a two-way ANOVA can seem daunting so we’ll go through it piece-by-piece. Luckily, we’ve established a strong foundation of understanding by beginning with paired-samples *t*-tests and the one-way repeated measures ANOVA. The concepts we built up there will prove very helpful in tackling this more complex analysis.

The first table of results shown is the breakdown of the sources of variance in our data. As was the case with out one-way ANOVA, each of the rows of interest have sum of square (`SS`

), degrees of freedom (`df`

), mean square (`MS`

), `F`

statistic, and * p value* columns. Here we will not focus on the full calculations but it is enough to keep in mind that each

`F`

-value is essentially a ratio of explainable over unexplained error variance.To guide our reading of this table it is best to remember the paramaters of the experiment.

- We manipulated participation
`time`

in our study within subjects - We manipulated
`Margarine`

type between subjects - We want to know whether these two factors interact with each other to effect cholesterol level

For our `time`

manipulation we are asking whether the level of the independent `time`

can explain a statistically significant proportion of the variance in our data? Looking to the `time`

row we see the observed `F`

-value of `259.49`

and the associated * p value* of

`0.000`

. As per convention, this *is said to be statistically significant because it is*

`p`

`< 0.05`

. This means we can say yes, the level of our independent variable `time`

has a statistically significant effect on the mean level of cholesterol.For the `Margarine`

manipulation we are asking a similar question. Does the type of margarine used explain a statistically significant proportion of the variance in our data? Looking to the `Margarine`

row we see an observed `F`

-value of `1.45`

and the associated * p value* of

`0.247`

. As per convention, this *is*

`p`

**not**statistically significant because it is

`>= 0.05`

. We fail to reject the null hypothesis that type of margarine does not have a statistically significant effect on mean level of cholesterol.**Interactions**

For consideration of the interaction between these two factors we look to the `Margarine X time`

row. Here the question is not about the effects of our factors in isolation but instead we are asking whether the level(s) our factors have an effect on each other. This type of effect is most easily understood in the domain of medicine where we commonly hear it invoked.

Imagine a patient with two underlying health conditions, both requiring medication to manage them. Independently, each of these medications would improve the health of this patient. But, if the effects of these medications interact with one another then the addition of the second medication will change the effectivness of the intervention.

This interaction can play out in many different ways. Together they could lead to more improvement than would be expected by adding up their independent effects (super addativity). Their combined effectivness could be less than would be expected from adding the effects together (sub-addativity). It could even be dangerous and detrimental to health by combining these medications (cross-over interaction). The important thing to understand is that an interaction means a particular combination of the levels of our factors can produce their own effects on the result.

When we look to the `Margarine X time`

row we see an observed `F`

-value of `4.78`

and the associated * p value* of

`0.015`

. Therefore, we reject the null hypothesis that our two independent variables do not interact with one another.**ANOVA Conclusions**

In summary, our table of ANOVA results revealed the following:

- There is a statistically significant effect of
`time`

- There is not a statistically significant effect of
`Margarine`

- There is statistically significant interaction of
`Margarine x time`

These statistically significant ANOVA results only tell us that all levels do not produce the same results. To know which groups differ and the direction(s) of those difference we look to our descriptive statistics and pairwise comparisons.

**Descriptive Statistics**

Above we are given the `Mean`

, standard deviation (`SD`

), standard error of the mean (`SEM`

), and number of participants (`N`

) for each of our 6 experimental conditions.

Mean cholesterol level for participants given margarine `B`

decreased across the study. They began at `6.78`

, dropped to `6.13`

after 4 weeks, and ended the 8 week intervention at `6.07`

.

Mean cholesterol level for participants given margarine `A`

also decreased across the study. They began at `6.04`

, dropped to `5.55`

after 4 weeks, and ended the 8 week intervention at `5.49`

.

**Charts**

Although we have all of the raw group means in our descriptive statistics, it is often very helpful to visualize the results of our experiment using charts. Both of the charts below are representing the same data obtained from our descriptive statistics. The only difference between the charts is the variable chosen to place on the X-axis. Using multiple representations of the same data is informative because some patterns “pop out” at us more readily in one configuration or another.

In the first chart we see type of `Margarine`

on the X-axis and each level of `time`

as a separate line. Cholesterol scores are generally lower for the `A`

than the `B`

margarine groups. ANOVA `F`

-table results tell us this between-subjects manipulation of `Margarine`

is not statistically significant (* p = 0.247*).

Our second chart shows `time`

on the X-axis and type of margarine as separate lines. Here we see the decrease in cholesterol as the study progresses and we also see that this pattern is largely the same for the `A`

and `B`

margarine groups. Differences between the `Before`

and `After4Weeks`

groups are large; differences between the `After4Weeks`

and `After8Weeks`

groups appear small.

**Pairwise Comparisons**

With our general understanding of the patterns in our data we can move on to the pairwise post-hoc comparisons. These comparisons will tell us which of our apparent group differences are statistically significant and which are not.

For our purposes, the most important columns are `Group 1`

, `Group 2`

, `Reject`

, and * p value*.

`Group 1`

and`Group 2`

tell us which two experimental conditions are being compared.and`p value`

`Reject`

tell us whether the group difference being compared is statistically significant (`Reject = True`

).

- Confirming our ANOVA result, we see
**no main effect of**type on mean cholesterol level (`Margarine`

`p = 0.247`

). - The next two rows show statistically significant simple effects of
`time`

with the`B`

-type margarine. More specifically, for participants given`B`

-type margarine there were statistically significant differences between the`Before`

and`After4Weeks`

groups as well as between the`Before`

and`After8Weeks`

groups (`p = 0.000`

for both comparisons). The next row shows no statistically significant difference between`After4Weeks`

and`After8Weeks`

groups for participants given`B`

-type margarine (`p = 0.294`

). - The last three rows of the
`Margarine Post-Hoc Tests`

table show the simple effects of`time`

for participants given the`A`

-type margarine. This pattern is largely the same as was observed for participants given`B`

-type margarine. Differences between`Before`

and`After4Weeks`

as well as differences between`Before`

and`After8Weeks`

groups were statistically significant for participants given`A`

-type margarine (`p = 0.000`

for both comparisons). Just as was seen with`B`

-type margarine, differences between`After4Weeks`

and`After8Weeks`

were not statistically significant for participants given`A`

-type margarine (`p = 0.060`

).

- The first three rows of the
`time Post-Hoc Tests`

show statistically significant differences between each level of the`time`

variable.`Before`

vs`After4Weeks`

(`p = 0.000`

)`Before`

vs`After8Weeks`

(`p = 0.000`

)`After4Weeks`

vs`After8Weeks`

(`p = 0.004`

)

- The last three rows of the table compare groups given margarine
`A`

to groups given margarine`B`

at each level of the`time`

variable.`A`

vs`B`

at`Before`

(`p = 0.475`

)`A`

vs`B`

at`After4Weeks`

(`p = 0.633`

)`A`

vs`B`

at`After4Weeks`

(`p = 0.622`

)

- None of these three comparisons rises to the level of statistical significance.

**Conclusions**

Full analysis of our `Cholesterol.csv`

dataset under a two-way mixed ANOVA model shows a statistically significant effect of our `time`

intervention.

Participation in this study lead to a decrease in mean cholesterol level for all experimental groups. Cholesterol significantly dropped from during the 1st 4 weeks of participation and continued to drop (although less dramatically) given an additional 4 weeks of margarine use.

The type of margarine used by participants did not have a statistically significant effect on the mean cholesterol level. Both were equally effective at decreasing group mean cholesterol levels.

There was a statistically significant interaction observed between `time`

and `Margarine`

although none of the post-hoc comparisons shed light upon how this interaction is operating in our study.

It is possible our comparisons of cell means to assess differential effectiveness of margarine types was underpowered due to small sample sizes within each cell (`N = 9`

). When comparing `After4Weeks`

to `After8Weeks`

groups for either margarine `A`

or `B`

we failed to find significant differences. When using a more powerful test which collapsed over type of `Margarine`

the difference between `After4Weeks`

and `After8Weeks`

was significant.

## Leave a Reply