How To Create A Neural Network In Python

To create a neural network, you need to decide what you want to learn. Here, I'm going to choose a fairly simple goal: to implement a three-input XOR gate. (It's an exclusive OR gate.) The table shows the function we want to implement as an array. I will use the information in the table below to create a neural network with python code only:

image for post — The Truth Table (a Three-Input XOR Gate)
for the Neural Network

Before I get into building a neural network with Python, I will suggest that you first go through this article to understand what a neural network is and how it works. Now let's get started with this task to build a neural network with Python.

Also, Read – GroupBy Function in Python.

Neural Network with Python:

I'll only be using the Python library called NumPy, which provides a great set of functions to help us organize our neural network and also simplifies the calculations.

Now, let start with the task of building a neural network with python by importing NumPy:

                                    import              numpy              as              np          
                      Code language:            JavaScript            (            javascript            )

Next, we define the eight possibilities of our inputs X1 – X3 and the output Y1 from the table above:

                                    # X = input of our 3 input XOR gate              # set up the inputs of the neural network (right from the table)              X = np.array(([0,0,0],[0,0,1],[0,1,0], \               [0,1,1],[1,0,0],[1,0,1],[1,1,0],[1,1,1]), dtype=float)              # y = our output of our neural network              y = np.array(([1], [0], [0], [0], [0], \               [0], [0], [1]), dtype=float)          
                      Code language:            PHP            (            php            )

We must now choose a value to predict:

                                    # what value we want to predict              xPredicted = np.array(([0,0,1]), dtype=float) X = X/np.amax(X, axis=0)              # maximum of X input array              # maximum of xPredicted (our input data for the prediction)              xPredicted = xPredicted/np.amax(xPredicted, axis=0)          
                      Code language:            PHP            (            php            )

Save our squared loss results in a file to be used by Excel by epoch:

# set up our Loss file for graphing lossFile = open("SumSquaredLossList.csv", "w")

Code language: PHP ( php )

Build the Neural_Network class for our problem. The table above shows the network we are building. You can see that each of the layers is represented by a line in the network:

                                                    class                Neural_Network                (object):                              def                __init__                (self):              #parameters              self.inputLayerSize =              3              # X1,X2,X3              self.outputLayerSize =              1              # Y1              self.hiddenLayerSize =              4              # Size of the hidden layer                      
                      Code language:            Python            (            python            )

Now set all the weights in the network to random values to start:

# build weights of each layer # set to random values # look at the interconnection diagram to make sense of this # 3x4 matrix for input to hidden self.W1 = np.random.randn(self.inputLayerSize, self.hiddenLayerSize) # 4x1 matrix for hidden layer to output self.W2 = np.random.randn(self.hiddenLayerSize, self.outputLayerSize)

Code language: PHP ( php )

The function below implements the feed-forward path through our neural network:

                      def feedForward(self, X):              # feedForward propagation through our network              # dot product of X (input) and first set of 3x4 weights              self.z = np.dot(X,              self.W1)              # the activationSigmoid activation function - neural magic              self.z2 =              self.activationSigmoid(self.z)              # dot product of hidden layer (z2) and second set of 4x1 weights              self.z3 = np.dot(self.z2,              self.W2)              # final activation function - more neural magic              o =              self.activationSigmoid(self.z3)              return              o          
                      Code language:            PHP            (            php            )

And now we need to add the backwardPropagate function which implements the real trial and error learning that our neural network uses:

def backwardPropagate(self, X, y, o): # backward propagate through the network # calculate the error in output self.o_error = y - o # apply derivative of activationSigmoid to error self.o_delta = self.o_error*self.activationSigmoidPrime(o) # z2 error: how much our hidden layer weights contributed to output # error self.z2_error = self.o_delta.dot(self.W2.T) # applying derivative of activationSigmoid to z2 error self.z2_delta = self.z2_error*self.activationSigmoidPrime(self.z2) # adjusting first set (inputLayer --> hiddenLayer) weights self.W1 += X.T.dot(self.z2_delta) # adjusting second set (hiddenLayer --> outputLayer) weights self.W2 += self.z2.T.dot(self.o_delta)

Code language: PHP ( php )

To train the network at a particular time, we will call the backwardPropagate and feedForward functions each time we train the network:

                      def trainNetwork(self, X, y):              # feed forward the loop              o =              self.feedForward(X)              # and then back propagate the values (feedback)              self.backwardPropagate(X, y, o)          
                      Code language:            PHP            (            php            )

The sigmoid activation function and the first derivative of the sigmoid activation function are as follows:

                      def activationSigmoid(self, s):              # activation function              # simple activationSigmoid curve as in the book              return              1/(1+np.exp(-s)) def activationSigmoidPrime(self, s):              # First derivative of activationSigmoid              # calculus time!              return              s * (1              - s)          
                      Code language:            PHP            (            php            )

Then save the epoch values of the loss function to a file for Excel and the neural weights:

                      def saveSumSquaredLossList(self,i,error):     lossFile.write(str(i)+","+str(error.tolist())+'\n') def saveWeights(self):              # save this in order to reproduce our cool network              np.savetxt("weightsLayer1.txt",              self.W1, fmt="%s")     np.savetxt("weightsLayer2.txt",              self.W2, fmt="%s")          
                      Code language:            PHP            (            php            )

Next, we run our neural network to predict the outputs based on the weights currently being trained:

                      def predictOutput(self):              print              ("Predicted XOR output data based on trained weights: ")              print              ("Expected (X1-X3): \n"              + str(xPredicted))              print              ("Output (Y1): \n"              + str(self.feedForward(xPredicted))) myNeuralNetwork = Neural_Network() trainingEpochs =              1000              #trainingEpochs = 100000                      
                      Code language:            PHP            (            php            )

What follows is the main learning loop that crosses all requested eras. Edit the trainingEpochs variable above to vary the number of epochs you want to train your network:

                                    for              i in range(trainingEpochs):              print              ("Epoch # "              + str(i) +              "\n")              print              ("Network Input : \n"              + str(X))              print              ("Expected Output of XOR Gate Neural Network: \n"              + str(y))              print              ("Actual Output from XOR Gate Neural Network: \n"              + \ str(myNeuralNetwork.feedForward(X))              # mean sum squared loss              Loss = np.mean(np.square(y - myNeuralNetwork.feedForward(X)))   myNeuralNetwork.saveSumSquaredLossList(i,Loss)              print              ("Sum Squared Loss: \n"              + str(Loss))              print              ("\n")   myNeuralNetwork.trainNetwork(X, y)          
                      Code language:            PHP            (            php            )

Save your training results for reuse and predict the output of the requested value:

myNeuralNetwork .saveWeights() myNeuralNetwork .predictOutput()

Code language: CSS ( css )

Now after running your python file, you will see the program start to cycle through 1000 training epochs, print the results of each epoch, and then finally show the final input and output.

Epoch # 999 Network Input : [[0. 0. 0.]  [0. 0. 1.]  [0. 1. 0.]  [0. 1. 1.]  [1. 0. 0.]  [1. 0. 1.]  [1. 1. 0.]  [1. 1. 1.]] Expected Output of XOR Gate Neural Network: [[1.]  [0.]  [0.]  [0.]  [0.]  [0.]  [0.]  [1.]] Actual Output from XOR Gate Neural Network: [[0.93419893]  [0.04425737]  [0.01636304]  [0.03906686]  [0.04377351]  [0.01744497]  [0.0391143 ]  [0.93197489]] Sum Squared Loss: 0.0020575319565093496 Predicted XOR output data based on trained weights: Expected (X1-X3): [0. 0. 1.] Output (Y1): [0.04422615]

Also, Read – Lung Segmentation with Machine Learning.

So this is how to build a neural network with Python code only. I hope you liked this article on building a neural network with python. Feel free to ask your valuable questions in the comments section below. You can also follow me on Medium to learn every topic of Machine Learning and Python.