Section 8 of 12 · 7 lessons

Computer Vision with Neural Networks

We can write rules to find an edge or match a known pattern in a picture, but deciding which patterns matter becomes difficult as the task grows. Neural networks let us learn useful image features from examples. We will use them to recognise objects, label individual pixels, and find similar pictures.

0 of 7 finished

Spatial Layers

An edge is still an edge when it moves across a picture. We need layers that can apply the same detector in different places and summarise what it finds.

  1. 01ConvolutionA useful image pattern can appear anywhere in a picture. Convolution applies the same small set of weights at different positions, producing a map of where that pattern receives a response.
  2. 02PoolingAfter detecting local patterns, we may not need every response at its original resolution. Pooling summarises nearby values, reducing the amount of data while giving up some information about exact positions.

Convolutional Networks

Knowing which object is in a picture is one task; knowing exactly which pixels belong to it is another. We will build toward both from the image layers we have already covered.

  1. 03Convolutional NetworksRecognising an object usually takes more than finding one edge. A convolutional network combines learned local patterns through layers, using the resulting features to make a prediction about the image.
  2. 04U-NetSometimes we need to know which pixels belong to an object, not just whether the object is present. U-Net learns from images paired with labelled masks and carries spatial detail forward to help reconstruct those boundaries.

Embedding Pictures

Two pictures of the same object can have very different pixel values. To search by similarity, we need a representation that preserves the differences relevant to the task.

  1. 05A Vector for a PictureTwo photographs of the same object may look very different pixel by pixel. A trained network can provide a compact vector of features that gives us a more useful way to compare them.
  2. 06Learning the Metric ItselfWhat makes two pictures similar depends on the task. By training on examples of useful matches, we can learn a representation in which the differences we care about affect the distance.
  3. 07Searching a Collection of PicturesOnce pictures have numerical representations, we can search for nearby ones. Comparing every picture works for a small collection, but a larger collection gives us a reason to consider search indexes and their compromises.