Lecture 6: Conditional independence continued

We considered D-separation last week. Now we extend that from single paths so graphs.

D-Separation for Paths in Graphs

One extra rule needed: conditioning on the descendent of a collider has the same effect as conditioning on the collider.

It is a common consequence of the parents in the collidor.

TODO REVIEW slide 5-29 for full definition of d-separation.

The full graphical criterion translates to statements about conditional independence, and thus something about how your probability density factorizes.

This is very nice, because then we do not have to analytically mess around with factorization.

This definition can be extended for sets of variables. See slide 5-30.

The Ancestral Moral Graph

How can we compute d-separation statements? I.e. how do we check it in software?

d-separation uses directional paths, but the amount of possible paths between variables grows worse than exponential.

But there is a more simple way to decompose this problem.

D-separation is not NP-hard, but can be solved in polynomial time and sometimes almost linearly.

The DAG can be separated into undirected graphs (a la max-flow-min-cut )

This undirected graph is called the moral graph.

  1. We basically "marry" parents of children that are not already married and that's why (lol) this is called a "moral" graph.
  2. Then we replace directed edges by undirected edges between the same nodes.

Lauritzen & Spiegelhalter Theorem:

Set Z d-separates a DAG when Z separates X and Y in the ancestral moral graph.

So you can use separation algorithms like max-flow-min-cut. This is a typical graph theory problem, so we can use all default algorithms already used for this problem.

Using D-separator to check models

D-separation implies conditional independence, and conditional independence can be seen from the data.

So checking the data for conditional independences tells us something about the model structure.

In slide 5-33, M1 and M2 are mediators.

Why do we have at least two testable implications (concerning independencies) here? (See 5-33)

If we have two independent variables, we obviously cannot have a direct line between them.

So this means that a fully connected graph has no conditional independencies left. This is the most complex model that we can make, and this model cannot be tested. We end up with a situation where our assumptions are not testable and falsifiable.

This network can fit the data well, but ... and overfitting.

The only thing we can test in a Bayesian network is the absence of an effect.

The less complex, the easier to falsify. So there is a direct relation between complexity

We try to specify a structure with as few edges as possible, but not too little such that we miss dependencies.

So the answer of the question: the amount of testable hypotheses is determined by the edges that are not there.

Spotting errors in models

Now imagine M1 is not actually on the path from X to Y, but U1 is (and M1 is a derived measure).

So U1 is unobserved. Then we have one testable hypotheses less, because we can only test observed variables. Then our true model has one implication less than our faulty model.

How do we find that the second implication does not hold?

Example 1: Testing a Collider Model with Chi-Squared

X -> Z <- Y

The collider model has no edge between X and Y, so our testable hypothesis is X independent of Y.

We can test dependencies using chi-square test.

  • Sum over combinations of x and y
  • For each combination, how often do we observe that combination?
  • Subtract that from the expected occurrences of that combination if x and y were independent.

In R, table makes a contingency table.

Expected frequencies are then the outer product of row sums and column sums, under the assumption of independence P(X,Y)=P(X)P(Y).

The residual is the difference between expectation and value, squared and normalized to the expectation.

Chi-square is the summing of the residuals.

The degrees of freedom is N_x-1 and N_y-1, for a chi-squared distribution.

REVIEW 5-36:

With K-1 degrees of freedom in Chi, we have expected value K-1 (? TODO REVIEW; think I wrote this wrongly)

The p-value we find, does not ...

Normally, we are "happy" with a small p-value. But here we want the p-value to be high; we do not want to reject our model; we do not want to find evidence that our model is wrong.

I.e. we are not doing null hypothesis testing but direct hypothesis testing.

Further worked out chi-squared test.

Example 2: Testing of a Fork Model

X <- Z -> Y has conditional independence X ind. Y | Z.

Now we consider conditional dependence testing. Luckily, this is relatively easy with Chi-Squared.

Approach:

  • Do different independence tests for each value of Z
  • Combine the results

But once we have more levels, we need to look at more p-values, so that's why we prefer to somehow combine the results of all tests.

(This is a research field in its own; "multiple testing"). But for Chi-Square Tests this is relatively easy, because the sum of two chi-squared distributed variables is again chi-distributed. And the degrees of freedom of the combined result, is the sum of the DOF from all separate tests. Now we get a single p-value.

Dealing with test results

In 5-43 we see that one of our assumptions does not hold. But how do we fix the network?

Here we have to use our theoretical knowledge about the modeled domain. There is no easy fix. You can always add extra edges, but is this reasonable? You can add a variable, but is this justified?

Big Data Brings High Detection Power

If we have enough data, any Bayesian Network model can be falsified and in fact probably is. With much data, we expect to find that most of our predictions are wrong and we get extremely small p-values. (All models are wrong; practically nothing is perfectly uncorrelated in reality )

P-values are too dependent on sample size, so we do need to have a look at effect size as well.

P-values conflate information about dependence strength and sample size.

One option for effect size that uses Chi-Squared: RMSEA Root Mean Squared Error Approximation.

RMSEA is nice because it is independent of the sample size.

In practice, RMSEA < 0.05 are not considered to be important. Even if they are highly significant, the effect is too weak.

In 5-47, we have the strongest dependency between marital status and sex. The data is weird; for some reason there is a larger proportion of married man (which makes no sense in a population where dominantly males and females marry).

So we are not only testing our model, we are also "testing" the data. This data sample is not representative.