Hard Negative Mining

  • Main problems is that the randomly selected negatives will be very easy to classify

In some problems you have a situation where you have few positive samples and many "easy" negative samples. One such problem, discussed in ISMI-6b-Object_Detection_Sampling_strategies , is object detection. The example there is detection of mitotic cells in breast tissue, which is an indication of aggressive breast carcinoma.

If you randomly sample patches from an input image as training data for a CNN (ensuring you have the positive samples in, at least), then you'll probably sample many negative samples with a very common background color.

This can lead to very high false positives. Ideally, you want the network to have negative samples that are not mitotic but are suspects, because these are the cases you want to discriminate. E.g. with only easy negatives with a very typical background color, the network may learn that darker cells are mitotic cells; but not pick up that the difference between true positives and false positives may be an irregular shape.

A solution is to look for negative samples that are hard to predict and train your network on those.

You can use a first CNN with the above problem to identify negatives that are hard to classify. The false positives of CNN-1 are apparently hard to distinguish from the true positives. If you select those as negative samples for CNN-2 (still be alongside random negative sampling), then this second network will be better at discriminating the interesting cases.

For each sample, CNN-1 can produce a likelihood of being mitosis. That is, we have a likelihood function over training samples. We can pass this likelihood function for negative samples into numpy.random.choice so that the negative samples that are more likely to be misclassified are picked more often.

This basic approach can be done offline, but also online if the size of the images allows you to do it. Then you'd:

  • Train for an epoch, do inference using a forward pass, get likelihood scores over negative samples
  • The samples used for the next epoch can be informed by the likelihoods from the previous one.

Example: Mark J.J.P. van Grisven et al, Fast Convolutional Neural Network Training Using Selective Data Sampling: Application to Hemorrhage Detection in Color Fundus Images

They showed that convergence was significantly faster using selective sampling (3x as fast).

So why does this work so good again? It's a good technique to consider for object detection because there we often have:

  • Very few positive samples
  • Many easy negative samples