Lecture 8: Latent Variables

  • Derive implies covariance matrices from SEMs
  • Understand the concept of global model fit
  • Understand how latent variables can be incorporated into SEMs

Motivation

Example: reading and writing are learned and measured at the same time.

So reading -> writing does not make sense, but neither does writing -> reading.

So there seems to be a hidden common cause, for which we can use the shorthand notation writing <-> reading. There is some other variable influencing both reading and writing that we have not measured so far. This does not mean reciprocal causation.

We can still test the model structure like we normally would. But what when we want to fit model parameters?

Meaning of latent variables

Latent:

  • Hidden: Something that could be measured in principle, but is not currently in the data for whatever reason (e.g. too costly to measure; or unethical to measure).
  • Conceptual: constructed measures that cannot be measured directly and perhaps only exist in theory
    • Example: intelligence, charisma
    • You cannot directly measure intelligence (as an intrinsic cognitive ability); only indirectly through intelligence tests.

This also means that latent variables can be represented explicitly (but unobserved/unmeasured), or implicitly using a bi-directed arrow.

For intelligence, we would not explicitly add an "intelligence" node in our network.

Representing Latent Variables Nonparametrically

How do we represent latent variables that are not observed?

When linking the variables together, we can combine their prob. tables in a single joint probability table. This is fine if these latent variables are on top of the graph, but problematic in other places.

X -> Y <-> Z.

This would require a full joint table over X, Y, Z, rendering the network useless (since we do not use the structure anymore).

Slide 7-9. Alternative: make assumptions about the distribution over latent variables, and parameterize the a residual covariance. Normally in SEMs we assume the residuals are independent.

Estimating Residual Covariances

Sometimes we can measure the observed covariance between two variables, e.g. reading <-> writing. But that works easy because they have no parents.

If they have parents, the "linear regression" approach which assumes independence (node by node) doesn't work anymore, i.e. local estimation. The covariance is then compound (not just influenced by those two variables).

In principle SEMs only work using continuous variables.

(Linear probability model; applying regression on binary data. A lot of discussion whether this is useful and appropriate. )

Global Estimation of BNs

Instead we then need to do global estimation.

  • exogenous = has no parents
  • endogenous = has parents

Exercises on papers.

Estimating Latent Variables

TREK rule.

Issue: when applying TREK rule we have another issue to solve.

Unidentified covariance matrices

If there are more unknowns (parameters) than data points (covariance matrix entries);

REVIEW Slide 7-22; interesting point. Why can we set variance to 1?

We can think of variance as a type of unit. (E.g. the same measurement in centimers will have higher variance than one in meters).

Because we cannot measure this unidentified variable (such as intelligence), we can assume variance=1. Also assume both paths from unknown latent variable have the same covariance. (WHY?)

We cannot tell in advance if a model is identified. Having not more parameters than entries in the covariance matrix is not sufficient for the model being identified. This is an unsolved problem.

Mixed discrete and continuous networks

We now have:

  • BNs based on probability tables
  • BNs based on Gaussian-linear functions which work with continuous data

We can make continuous data discrete by binning. But we cannot make discrete data continuous easily. (Perhaps that trick from Rguide with ordinal data?)

So we have to make some difficult choices. Often continuous data is easier to work with and offers more flexibility; e.g. in modeling latent variables.

Unbinning

REVIEW THIS WHOLE LAST SECTION OF THE LECTURE.

Can we "unbin" data?

Sometimes we may have some discrete variables that are "binned" but which we suspect may be really continuous underneath. Then it would be nice to be able to treat it as a continuous variable.

E.g. SES Socio-economic status.

So we have "True SES" (unobserved, continuous) -> "measured SES" (observed, discrete).

There is no unique way to go about this. We are trying to find the most likely cutoffs in an underlying Guassian distribution such that it will result in the observed probability distribution in the observed discrete distribution.

R: use cumsum and qnorm

Polyserial Correlations

Binning the "true" continuosu variable leasds to some loss of information, e.g. leading to an underestimated correlation.

We can account for this by explicitly incorporating the true continuous variable and its relation with its binned version.

This is not a typical SEM:

Two types of parameters:

  • Correlations
  • Threshold (new)

Polychoric correlations

Maximum Likelihood Estimate approach.

lavCor is for polychoric correlation matrix (includes thresholds).

We can convert this whole approach basically into a preprocessing step, because any method working with correlation matrices can also work with the polychoric matrix in Lavaan.