Beyond the Borders

Note: This section is a bit free-floating (pedagogically speaking) and may serve best as reference for when it gets linked back to.

Sometimes we perform calculations on pixels that requires knowing about their neighbors, and this can cause issues when there are missing neighbors neighbors. For instance, imagine taking the average of the 8-neighbors of a pixel. What should we do if the pixel is on the corner?

Or another case, what do we do when an image is rotated? Clipping off the now-protruding corners is one option, but what if we want to keep the whole image? What do we fill the void space with?

There are a few common strategies for dealing with the void beyond. Let's go back to our first example, where we need to perform calculations using the 8-neighbors of a pixel that is on the corner.

The simplest strategy is to surround the image with however many pixels of black that you need. Sometimes a single black pixels border works, sometimes a thick border is needed - it just depens on the calculation. Since pure black has an intensity of 0, this is often known as 'zero-padding':

While this strategy is simple, it's not particularly useful beyond that. Ideally, you would want to simulate what the environment should look like beyond the edges of the image, and only in very few cases is a border of pure black a good approximation.

The next level of complexity is to extend the border of pixels of the current image out as much as necessary. This is known as 'replicate padding':

Given a more complex image, this would look like a 'smearing' of the border.

The final level of complexity (which is still not too terribly complex) is to reflect the image across the borders of the image (known as 'mirror padding'):

In general, mirror padding tends to work well for images that have details on the edges, while replicate-padding tends to work well for images that are constant at the borders (although I don't particularly see why mirror padding wouldn't also work in that case, too)