Classification after image segmentation
Example: lung scans. We may be interested in:
- nodular structures
- low opacities (e.g. emphysema or COPD)
- air is trapped in pockets of air; lungs are not capable of releasing them anymore
- darker: more air
If we want to do automatic classification, we first need to extract image features that are indicative of a particular type of problem.
Statistical texture analysis
- First-order: histogram
- Second-order: using covariance matrices (or coherence matrices? Didn't hear)
- Third order: measures based on run-length (?)
We only focus on first-order features. Note that nowadays deep NNs are dominant, where you commonly do not explicitly engineer features. Features are thus not used that much anymore.
Histogram-based
First we normalize a histogram. Then typically six features are extracted as mathematical descriptors of textures:
- Mean
- Variance
- Skewness
- Kurtosis
- Energy
- Entropy
This is "first-order" because it only operates on the pixel level (i.e. on pixel intensity). Problems:
- does not consider spatial location/relationship of a pixel
- e.g. does not consider (spatial) correlation between pixels
Of course, spatial information between pixels is very defining for a given texture. Only considering histogram-based features is thus not a very good descriptor of texture.
- For example, this can be shown by histogram matching two particular textures such that, by definition, their histograms are very similar. You'll see that the textures are still very different.
- Another obvious way to show this is by shuffling all pixels: the histogram will stay the same, but the texture will be different.
Local Binary Pattern
Apply LBP to a pixel with intensity $g_c$ and its P neighbours with intensity $g_p$ within a particular distance R (radius).
$$LBP_{P,R} = \sum_{p=0}^{P-1} s(g_c - g_p)*2^p$$
And $s(x)=1$ if x<=0, 0 otherwise.
So for each neighbour it only considers its value if the intensity of the neighbour is larger than the intensity $g_c$ of the "parent" pixel.
If s(x)=1, you add $2^p$ to the LBP. This is a bit weird, but I guess they do this because then the actual value of the LBP can always be shown to come from a particular patterns over the ordered parent nodes. That is, each possible LBP value corresponds to a particular texture.
Gabor filters
Gabor filters are Gaussian kernels multiplied with sinusoidal waves.
You can change:
- The frequency of the sinusoid
- Change orientation of the wave
- Change scale of Gaussian
Why is this handy?
Well before we could only take into account intensities without taking into account some of the spatial characteristics of textures. It may be that some textures have this wave-like pattern. This way we can pick that up.
This filter is expressed using a real and imaginary component. See slides. This will be implemented in many packages, but in any case you need to understand the parameters. Actually, Wikipedia has a good Python implementation.
Convolution of Gabor filter
By varying on both the filter size and the filter type (i.e. vary on all parameters) we can define a whole bank of filters. By convolving with all these filters, we get a lot of responses for a single input image. For each response, we can compute statistics again. This way, for each input sample we can produce a quite long vector of features.
Notice the difference from before: now we do not just have six histogram-based features directly on the input image, but also many many features based on the results after convolution with all these (Gabor) filters that pick up spatial texture info.
Now we have typical high-dimensional feature that can be the input for classification.
k-NN classifier
One way to classify a new data point is for example to look at its neighbours, i.e. using k-NN.
Rule of thumb: K is $\approx$ the square root of the number of samples in the training set. The actual classification heavily depends on the value of k.
This is a model-free classifier because we 'only' need to store the training samples.
For any algorithm using distances, it is very beneficial to do feature normalization. Otherwise features using a very large scale will dominate others in the distance measure, even if they are in fact pretty useless. This normalization step is done per feature!
Classification Performance Metrics
Again, sensitivity, specificity, accuracy.
Always ask: which measure do we want to maximize? This depends on what we want our system to do.
ROC analysis is very helpful here.