Training samples selection
Especially in object detection problems (but not exclusively) ...
- most of the image area is quite easy to recognize, i.e. most spots are negatives
- whereas there are sparse positive objects
- and difficult negatives are difficult the identify up front
So here sampling strategies can help to make the network do better on difficult examples.
Example using patch classification
Example Ciresan 2013: detect mitotic figures in breast cancer histology using patch classification.
AMIDA13 challenge: Lot's of mitosis in breast is a very strong indicator of invasive breast carcinoma. So we can use DNNs to detect high mitotic activity as an indicator of aggressive cancer.
General approach of Ciresan 2013 (winning solution):
- Cropped patched based on manually labelled mitosis
- Train CNN to classify for each patch: mitosis yes/no.
- Apply something something in sliding window fashion to get maps
- Pick local maximum
What distinguished the winners from the others is their smart sampling strategy.
Sampling strategies
So as input you get an input image which you'll divide up into patches to create training samples. For example, your mini-batch size may be 16 and in the whole input image there are only three cases of high mitosis. How do you now sample the patches i.e. the training data points?
- Uniform?
- (Quasi-)balanced: Ensure mitosis spots are sampled, random otherwise
- Main problems is that the randomly selected negatives will be very easy to classify
- This tends to lead to false positives
- Why? Ideally, you want the network to have negative samples that are not mitoses but are suspects, because these are the cases you want to discriminate
- E.g. with only easy negatives with a very typical background color, the network may learn that darker cells are mitotic cells; but not pick up that the difference between true positives and false positives may be shape
Hard-negative mining
You can use a first CNN with the above problem to identify negatives that are hard to classify. The false positives of CNN-1 are apparently hard to distinguish from the true positives. If you select those as negative samples for CNN-2 (still be alongside random negative sampling), then this second network will be better at discriminating the interesting cases.
For each sample, CNN-1 can produce a likelihood of being mitosis.
That is, we have a likelihood function over training samples.
We can pass this likelihood function for negative samples into numpy.random.choice so that the negative samples that are more likely to be misclassified are picked more often.
This basic approach can be done offline, but also online if the size of the images allows you to do it. Then you'd:
- Train for an epoch, do inference using a forward pass, get likelihood scores over negative samples
- The samples used for the next epoch can be informed by the likelihoods from the previous one.
Example: Mark J.J.P. van Grisven et al, Fast Convolutional Neural Network Training Using Selective Data Sampling: Application to Hemorrhage Detection in Color Fundus Images
They showed that convergence was significantly faster using selective sampling (3x as fast).
So why does this work so good again? It's a good technique to consider for object detection because there we often have:
- Very few positive samples
- Many easy negative samples