Segmentation in Medical Imaging

Image segmentation is the decomposition of a scene into its components.

Components are ideally particular objects we care about.

E.g. segmenting blood vessels in a retina, segmenting glands, tumors in liver, nuclei segmentation etc.

Thresholding

A baseline method for segmentation is simple thresholding using a step function. In the case of binary segmentation, everything above a particular grey intensity belongs to segment 1, otherwise to segment 0. We can also use multiple thresholds if we want to identify more segments. This is only effective in cases where segments can be clearly distinguished according to grey values.

The threshold is a tunable parameter. How can we choose an appropriate threshold?

We can always use expert knowledge.

But we can also find thresholds automatically by looking at histograms.

If we have a bimodal histogram, intuitively we may separate two classes by segmenting between the modes (i.e. split the two distributions coming from different objects). We can use the Otsu algorithm for this.

Otsu algorithm

First, we need to be able to differentiate foreground from background. For each, we can compute the mean, variance, and a particular weight w that is the pixel contribution #pixels per class / #all pixels.

Similar to clustering, we want to:

  • minimize the intra-class variance
  • maximize the inter-class variance

Algorithm:

  • Set a threshold t
  • Computer per-class variances, weights, and means
  • Compute the intra-class variance and inter-class variance
  • Pick the threshold t that maximizes the inter-class variance

You can do this for all many thresholds, e.g. all possible intensity values.

N.B. just finding "foreground" regions is usually not enough.

Connected components labeling

Once we have found foreground regions, how can be label them? Within the foreground regions, we may further identify structures by finding groups of pixels that are connected to each other.

This is called connected component analysis.

In 2D:

  • 4-connectivity if Euclidean distance is 1 (i.e. the pixel above, left, right and below)
  • 8-connectivity if Euclidean distance <2 (i.e. now also top-right, right-bottom etc. are added)

These definitions lead to different results. You tend to get more segments (i.e. instances) with 4-connectivity.

In 3D:

  • 6-connectivity for Euclidean distance < 1
  • 26-connectivity for Euclidean distance < 2
    • 8+9+9; the +2 is due to the adjacent layers including the middle pixel

When checking connections it may happen a new cell is connected to a pixel that we already visited with label 1 and one with label 2; in that case we pick the lowest label.

We can end up with different unique but equivalent labels for a similar segment. You resolve these with a backward pass where you replace each label with the lowest connected label.

scipy.ndimage.label(img) does this connected components labeling.

Alternative is: skimage.morphology

Region growing algorithm

  1. Initialize several "seed points" that are defined by a certain characteristic, e.g. a certain intensity or texture characteristics.
  2. Given a selected connectivity, check the neighbours of the seed points
  3. If a neighbour fulfills some criteria (e.g. similar intensity, texture etc.) then the neighbour is included in the segmentation
    • You usually use some threshold or double threshold (e.g. both -5 and +5) to define what is "similar"
  4. Repeat 2-3, because now new neighbours are included in the segmentation
  5. Stop when there are no changes anymore

When to use this approach? This is a common approach for airway segmentation. You take trachea points as seed point(s). This works well because the trachea mostly contains air and thus will all have low intensity.

The difficulty is actually setting the correct seed point. (We saw in the first assignment this can easily go wrong)

Evaluation metrics

  • On the pixel level we can define accuracy, sensitivity and specificity
    • Recall is what's called "sensitivity" in medical circles, i.e. the true positive rate TP/TP+FN
    • Specificity is not the same as precision; it is the true negative rate TN / TN + FP
  • On regions of an image:

    • Jaccard measure: intersection of the region union between reference and segmentation (TP/(TP+FP+FN)) ; so ignores true negatives.
    • To reason about this, draw two circles, define one as ref, the other as segmentation
    • Dice coefficient: similar to Jaccard but emphasizes true positives ( 2TP / 2TP + FP + FN)
  • Foreground is always defined as positive class

  • Background is always defined as negative class

Datasets often provide masks for the area to exclude in the evaluation.

Medical imaging segmentation often has a class imbalance, namely when most of the pixels belong to the background. This makes the accuracy measure more or less useless, as predicting the majority class will easily give 90% accuracy.