Promote convolutional filters in tensorflow

Question

I’m trying to visualize the filters of a neural network and I’m not getting to understand how to display the filters as images. I’m currently using the CIFAR10 base (32x32 color images).

I’m using these libraries

import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten
from tensorflow.keras.layers import Conv2D, MaxPooling2D, BatchNormalization
from tensorflow.keras import layers, models, utils

This is the test network

model = models.Sequential()
model.add(layers.Conv2D(64, (10, 10), activation='relu', input_shape=(X.shape[1:])))
model.add(layers.MaxPooling2D((2, 2)))

model.add(layers.Conv2D(16, (6, 6), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))

model.add( layers.Flatten( ) )

model.add( layers.Dense(128, activation='relu') )
model.add( layers.Dense(len(CATEGORIES) , activation='softmax') )

With this code I return the filters and show their dimensions

k=0
for layer in model.layers:

    if 'conv' in layer.name: 

        filter_, bias_= layer.get_weights()

        f_min, f_max = filter_.min(), filter_.max()
        filter_ = (filter_ - f_min) / (f_max - f_min)

        print('Filtros da {}° camada'.format(k+1))
        print('\n')
        print(filter_.shape)
        print('\n')

    k=k+1

Exit

Filters of 1°layer

(10, 10, 3, 64)

Filters of 3°layer

(6, 6, 64, 16)

The first layer I get, plus the others I don’t understand how their dimensions work

Code to display filters (which only works on the first layer)

k=0
for layer in model.layers:

    if 'conv' in layer.name: 

        fig= plt.figure(figsize=(10,10))

        filter_, bias_= layer.get_weights()

        f_min, f_max = filter_.min(), filter_.max()
        filter_ = (filter_ - f_min) / (f_max - f_min)

        print('Filtros da {}° camada'.format(k+1))
        
        for i in range(filter_.shape[3]):

            p = filter_[:,:,:,i]

            ax = fig.add_subplot(8, 8, i+1)
            ax.imshow(p)

        plt.show()
    k=k+1

Answer 1

In Keras, the filters of a convolutional layer have the following formatting: (kernel_size, kernel_size, channels_in, channels_out). Here I explain a little what each dimension of the tensor represents:

1. kernel_size. Filter size (3x3, 5x5, 7x7, etc),
2. channels_in. Number of channels of convolutional layer input tensor. It is easy to view as follows. Suppose your input is a grayscale image (hence 1 channel), then your tensor will be (kenel_size, kernel_size, 1, channels_out).
3. channels_out. Number of channels of output tensor, or number of filters.

For the first convolutional layer, it’s easy to view: you just need to show each filter as an image (kernel_size x kernel_size). An exeplo of this can be found in this notebook colab, where each filter is viewed as an image using the imshow function of the matplotlib library. For more layers, or with more color channels, the analysis gets a little more complicated. This post here (in English) discusses this in depth.