Lecture 5 Bayesian Networks: Conditional Independende and d-separation

Back to the spirit of the first lecture: How can we use Bayesian Networks?

Objectives:

  • Concept of conditional independence
  • Understand and be able to apply d-separation criterion.
  • Use these concepts to test the structure of the network against categorical data
    • Probably for next week

Model building cycle

  • Hypothesis about the world
  • Formalized as causal diagram
    • Makes predictions about your ...
    • (d-Separation!)
  • Data
  • falsify or corroborate hypothesis; back to top.

Mechanistic modeling vs. statistical modeling:

  • Mechanistic: formulate a theory about mechanical relations (e.g. causal), then test it. Example of a model: F=m*a.
  • This approach is different from just finding patterns.

For assignment: draw diagram that encodes your hypothesis.

This scientific method is obviously not limited to Bayesian Networks. We need to learn how to evaluate the structure of our Bayesian Networks, because BN is a particular type of model.

Motivation

  • Bayesian Networks are models of variable relationships in a particular domain
  • Your network is a picture of your assumptions
  • Incorrect assumptions may lead to incorrect inferences
  • Once a Bayes net is constructed, we can test some of the assumptions it encodes against data.

Model testing never guarantees a correct model. We can only try to refute bad models, but not prove that our model is correct.

Independence

Statistical independence is symmetric.

Repetition of marginal independence and conditional independence.

Full independence testing can be difficult. In practice, we often rely on simpler correlation measures that give an approximately correct indication of independence.

For independent variables X and Y, we still allow X -> Y in our network. This would be an error with no influence.

We allow this because there is nothing in the data that falsifies this statement.

Consistency

REVIEW calibration plots in R guide.

We call a probability density P consistent with G if P factorizes according to G.

If P is consistent with G, then all probabilities predicted by the fitted ntework will match the probabilities estimated directly from the data. In other words, the network will be well calibrated to the data.

So inconsistency leads to bad calibration.

Example 1

X -> Y is a saturated model because it is not falsifiable!

P(x,y) = P(y|x)P(x).

This holds for every probability density, so does not imply any restrictions on P.

Example 2

(X) (Y) does have an implication, because this commits to marginal independence.

Example 3: Fork

Fork: X <- M -> Y

P(x,m,y)=P(m)(x|m)P(y|m)

Therefore,

P(x,y|m) = (P(x,y,m)/P(m)) = P(x|m)P(y|m)

So this model implies conditional independence.

Example 4: Mediation

X -> M -> Y

Same conclusion as example 3! Also commits to the same conditional independence!

We have two different models with two different meanings, but they make the same statistical implications.

They imply the same, but they encode different hypotheses.

Lesson: the same data can be explained by different hypotheses! (infinite many)

Example 5: Collider

X -> M <- Y

Factorization: P(x,m,y)=P(x)P(y)P(m|x,y)

Reshape into P(x,y)=P(x)P(y)

This model also implies that X and Y are independent.

d-separation

I assume the d stands for dependency, but never checked this :-D.

We derived conditional independence for these shapes by factorizing first. This takes time.

But we see that each of these shapes is a graphical criterion for conditional independence!

Logic:

  • If X and Y are "d-separated" by Z , then ( X independent Y given Z ) is guaranteed.
  • If they are not, the may or may not be conditionally independent.

So d-separation is a sufficient but not necessary criterion for conditional independence, i.e. if we have d-separation we know we have conditional independence, but non-d-separated variables may still be conditionally independent.

Like before, only the absence of an arrow commits to conditional independence, not the other way around.

Paths

Path: sequence of variables in which each adjacent pair is connected by an edge.

Notice that this definition does not care about the direction of the edge. Whatever the direction, they are all paths. Normally this is not the case. Normally X -> Y would be a path from X to Y, but not Y to X.

By convention, we only consider paths with nodes only occurring once (no loops).

Note that only the collider has a different statistical implication, namely of marginal independence.

d-Separation for paths

See slide 20 for a formal definition.

Note that a path of length 2 can never be d-separated, i.e. the smallest units of d-separation are triplets.

Interpretation of Collider Case

Collider can cause spurious relations!

If you condition on the wrong thing, two unrelated variables can suddenly appear to be conditionally dependent!

Pay tuition (wealth) -> Go to uni <- Get scholarship (IQ)

Here, wealth and IQ are unrelated. But if you go to this university, you will find a negative spurious correlation between wealth and IQ. This is the consequence of conditioning on the collider; in this case only considering people that went to university.

Johannes Textor and others created: dagitty.net

dagitty is also an R package.

Will there still be separation if we condition on all nodes?