Convolutional Neural Networks and Architectural Analysis

Convolutional Neural Networks and Architectural Analysis

In addition to generating new images, a trained GAN can also provide analytic insight into the training set. This is due to the fact that, through the training, the GAN learns to recognize the low and high-level image features that define the essential qualities of the training set.  These learned features can be viewed by looking at the activation patterns of the neural network layers that comprise the generator and discriminator networks of the GAN.  Figure 1 shows activation patterns from selected neural network layers within the discriminator network of the GAN trained with noise augmentations.  Layer one of the figure shows low-level features like horizontal and vertical edges being learned – represented by the brighter pixels in the activation map.  Layer two of the figure, shows higher level features such as walls, windows, and cabinetry being learned.  In layer three, more complex features such as the larger organizing grid of the house is being learned.  Figure 2 then shows samples of the neural network layers in the generator network of the GAN – which work in the reverse order of the discriminator.  Both networks can provide insight into the deep organizational structure of Le Corbusier’s house plans, but the activation patterns learned by the DNN still require careful study and interpretation. Further, because there are often hundreds of neural layers, it can be difficult to know what each has learned.