TensorFlow - Python Deep Learning Neural Network API

Create an artificial neural network with TensorFlow's Keras API

In this episode, we'll demonstrate how to create a simple artificial neural network using a Sequential model from the Keras API integrated within TensorFlow.

In the last episode, we generated some data from an imagined clinical trial, and now we'll build a simple model for which we can train on this data.

Code Setup

First, we need to import all the libraries we'll be making use of.

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Activation, Dense
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.metrics import categorical_crossentropy

We'll use all of these modules, except for the last two, to build our neural network. Note that we'll make use of the last two modules in the next episode when we train the model.

A GPU is not required to follow this course, but if you are using one, you'll need to first follow the GPU setup we covered in a previous episode. We can then check to be sure that TensorFlow is able to identify the GPU using the code below. It's also useful to enable memory growth on the GPU.

physical_devices = tf.config.experimental.list_physical_devices('GPU')
print("Num GPUs Available: ", len(physical_devices))
tf.config.experimental.set_memory_growth(physical_devices[0], True)

> Num GPUs Available:  1

set_memory_growth() attempts to allocate only as much GPU memory as needed at a given time, and continues to allocate more when needed. If this is not enabled, then we may end up running into the error below when we train the model later.

Blas GEMM launch failed

Build a Sequential Model

Let's now create our model. We first create a variable named model and define it as follows.

model = Sequential([
    Dense(units=16, input_shape=(1,), activation='relu'),
    Dense(units=32, activation='relu'),
    Dense(units=2, activation='softmax')
])

model is an instance of a Sequential object. A tf.keras.Sequential model is a linear stack of layers. It accepts a list, and each element in the list should be a layer.

As you can see, we have passed a list of layers to the Sequential constructor. Let's go through each of the layers in this list now.

First hidden layer

Our first layer is a Dense layer. This type of layer is our standard fully-connected or densely-connected neural network layer. The first required parameter that the Dense layer expects is the number of neurons or units the layer has, and we're arbitrarily setting this to 16.

Additionally, the model needs to know the shape of the input data. For this reason, we specify the shape of the input data in the first hidden layer in the model (and only this layer). The parameter called input_shape is how we specify this.

As discussed, we'll be training our network on the data that we generated and processed in the previous episode, and recall, this data is one-dimensional. The input_shape parameter expects a tuple of integers that matches the shape of the input data, so we correspondingly specify (1,) as the input_shape of our one-dimensional data.

You can think of the way we specify the input_shape here as acting as an implicit input layer. The input layer of a neural network is the underlying raw data itself, therefore we don't create an explicit input layer. This first Dense layer that we're working with now is actually the first hidden layer.

Lastly, an optional parameter that we'll set for the Dense layer is the activation function to use after this layer. We'll use the popular choice of relu. Note, if you don't explicitly set an activation function, then Keras will use the linear activation function.

If you need to review the topics of layers, neurons, activation functions, etc., then check out the Deep Learning Fundamentals course where we cover these topics in detail.

Second hidden layer

Our next layer will also be a Dense layer, and this one will have 32 nodes. The choice of how many neurons this node has is also arbitrary, as the idea is to create a simple model, and then test and experiment with it. If we notice that it is insufficient, then at that time, we can troubleshoot the issue and begin experimenting with changing parameters, like number of layers, nodes, etc.

This Dense layer will also use relu as its activation function.

Output layer

Lastly, we specify the output layer. This layer is also a Dense layer, and it will have 2 neurons. This is because we have two possible outputs: either a patient experienced side effects, or the patient did not experience side effects.

This time, the activation function we'll use is softmax, which will give us a probability distribution among the possible outputs.

Note that we can call summary() on our model to get a quick visualization of it.

model.summary()

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense (Dense)                (None, 16)                32        
_________________________________________________________________
dense_1 (Dense)              (None, 32)                544       
_________________________________________________________________
dense_2 (Dense)              (None, 2)                 66        
=================================================================
Total params: 642
Trainable params: 642
Non-trainable params: 0
_________________________________________________________________

Now we've created our very first model using the intuitive tf.keras.Sequential model type. In the next episode we'll train this model on the data we created last time.