Classification after image segmentation

Example: lung scans. We may be interested in:

  • nodular structures
  • low opacities (e.g. emphysema or COPD)
    • air is trapped in pockets of air; lungs are not capable of releasing them anymore
    • darker: more air

If we want to do automatic classification, we first need to extract image features that are indicative of a particular type of problem.

Statistical texture analysis

  • First-order: histogram
  • Second-order: using covariance matrices (or coherence matrices? Didn't hear)
  • Third order: measures based on run-length (?)

We only focus on first-order features. Note that nowadays deep NNs are dominant, where you commonly do not explicitly engineer features. Features are thus not used that much anymore.

Histogram-based

First we normalize a histogram. Then typically six features are extracted as mathematical descriptors of textures:

  1. Mean
  2. Variance
  3. Skewness
  4. Kurtosis
  5. Energy
  6. Entropy

This is "first-order" because it only operates on the pixel level (i.e. on pixel intensity). Problems:

  • does not consider spatial location/relationship of a pixel
  • e.g. does not consider (spatial) correlation between pixels

Of course, spatial information between pixels is very defining for a given texture. Only considering histogram-based features is thus not a very good descriptor of texture.

  1. For example, this can be shown by histogram matching two particular textures such that, by definition, their histograms are very similar. You'll see that the textures are still very different.
  2. Another obvious way to show this is by shuffling all pixels: the histogram will stay the same, but the texture will be different.

Local Binary Pattern

Apply LBP to a pixel with intensity $g_c$ and its P neighbours with intensity $g_p$ within a particular distance R (radius).

$$LBP_{P,R} = \sum_{p=0}^{P-1} s(g_c - g_p)*2^p$$

And $s(x)=1$ if x<=0, 0 otherwise.

So for each neighbour it only considers its value if the intensity of the neighbour is larger than the intensity $g_c$ of the "parent" pixel.

If s(x)=1, you add $2^p$ to the LBP. This is a bit weird, but I guess they do this because then the actual value of the LBP can always be shown to come from a particular patterns over the ordered parent nodes. That is, each possible LBP value corresponds to a particular texture.

Gabor filters

Gabor filters are Gaussian kernels multiplied with sinusoidal waves.

You can change:

  1. The frequency of the sinusoid
  2. Change orientation of the wave
  3. Change scale of Gaussian

Why is this handy?

Well before we could only take into account intensities without taking into account some of the spatial characteristics of textures. It may be that some textures have this wave-like pattern. This way we can pick that up.

This filter is expressed using a real and imaginary component. See slides. This will be implemented in many packages, but in any case you need to understand the parameters. Actually, Wikipedia has a good Python implementation.

Convolution of Gabor filter

By varying on both the filter size and the filter type (i.e. vary on all parameters) we can define a whole bank of filters. By convolving with all these filters, we get a lot of responses for a single input image. For each response, we can compute statistics again. This way, for each input sample we can produce a quite long vector of features.

Notice the difference from before: now we do not just have six histogram-based features directly on the input image, but also many many features based on the results after convolution with all these (Gabor) filters that pick up spatial texture info.

Now we have typical high-dimensional feature that can be the input for classification.

k-NN classifier

One way to classify a new data point is for example to look at its neighbours, i.e. using k-NN.

Rule of thumb: K is $\approx$ the square root of the number of samples in the training set. The actual classification heavily depends on the value of k.

This is a model-free classifier because we 'only' need to store the training samples.

For any algorithm using distances, it is very beneficial to do feature normalization. Otherwise features using a very large scale will dominate others in the distance measure, even if they are in fact pretty useless. This normalization step is done per feature!

Classification Performance Metrics

Again, sensitivity, specificity, accuracy.

Always ask: which measure do we want to maximize? This depends on what we want our system to do.

ROC analysis is very helpful here.

ROC_curve.md