Dilated Convolutions
Developed in 2016.
In order to be successful, ConvNets require a large enough receptive field. Pooling is one way of increasing the receptive field of neurons, but dilated convolutions are another strategy.
Idea: keep the same amount of parameters while also increasing the receptive field.
For dilation rate $k$, we introduce $k-1$ spaces within the elements of a filter. Another way to remember this convention is: how many steps do you need to take from one coefficient until the next one?
This means that a filter is treated as if it is in fact larger. If I'm not mistaken, the "filter size" after dilation is f+(f-1)(k-1).
This however avoids the downside of pooling because it does not downsample the input. You instead increase the receptive field by looking at a larger input region.
Pros:
- Exponentially expanding receptive field
- Easy to plug into existing architectures
- You do not have to design new filters
- No pooling, so no subsampling / lower resolution
- Allows dense prediction at full resolution
- I don't really get this. Wouldn't this require extreme padding?
Con:
- The receptive field looks more "square" than normal
- They tend to produce checkerboard artefacts