TensorFlow Tutorial 5: Running the Neural Net

By Dante Sblendorio

In the preceding sections of this contest, we’ve gone through all the steps of installing TensorFlow, setting up a Jupyter Notebook and creating a neural net. Now it’s time for the real fun: Running the neural net and viewing our results. Keep reading for this last and final step in our journey.

Now that we have defined all the functions we need, we can construct the neural net. We first initialize all the variables and parameters based on the shape of the training set, and then define the learning rate, number of epochs, and batch size. The learning rate determines how fast the mathematical operators converge on the minimum cost value. The number of epochs is the number of times training data is fed through the net. The batch size is the size of each random subsample. Each parameter has a role in the final test accuracy and how fast the net converges.

from tensorflow.python.framework import ops
import math

def nn(train_x, train_y, test_x, test_y, learning_rate ,num_epochs, batch_size, print_cost = True):
       ops.reset_default_graph()
       tf.set_random_seed(1)
       (n_x, m) = train_x.shape
       n_y = train_y.shape[0]
       
       # Initialize
       costs = []
       X = tf.placeholder(tf.float32, [n_x, None], name="X")
       Y = tf.placeholder(tf.float32, [n_y, None], name="Y")
       parameters = init_parameters(13)
       Z3 = for_prop(X, parameters)
       cost = c(Z3, Y)
       optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)
       init = tf.global_variables_initializer()

       # Forward propagation
       with tf.Session() as sess:
           sess.run(init)

           #training loop
           for epoch in range(num_epochs):
               epoch_cost = 0.
               num_batches = int(m / batch_size)
               batches = rand_batches(train_x, train_y, batch_size, 13)
               for batch in batches:
                   (batch_X, batch_Y) = batch
                   _ , batch_cost = sess.run([optimizer, cost], feed_dict={X: batch_X, Y: batch_Y})
                   epoch_cost += batch_cost / num_batches

               # print the cost at every 50 epochs
               if print_cost == True and epoch % 50 == 0:
                   print ("Epoch %i cost: %f" % (epoch, epoch_cost))
               if print_cost == True and epoch % 5 == 0:
                   costs.append(epoch_cost)

           # Save the parameters
           parameters = sess.run(parameters)
           print("Parameters trained...")

           # Calculate the correct predictions
           correct_prediction = tf.equal(tf.argmax(Z3), tf.argmax(Y))

           # Accuracy of the test set
           accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))

           print("Train Accuracy:", accuracy.eval({X: train_x, Y: train_y}))
           print("Test Accuracy:", accuracy.eval({X: test_x, Y: test_y}))

           return parameters

Now that we have defined the neural net function, we can pick values for each parameter, and run:

learning_rate = 0.001 #change this to change learning rate
num_epochs = 1000     #change this to change number of epochs
batch_size = 15       #change this to change batch size
parameters = nn(train_x, train_y, test_x, test_y, learning_rate ,num_epochs, batch_size)

With the learning rate, number of epochs, and batch size defined as they are, the neural net does a pretty good job with the test set, accurately predicting the Class more than 96% of the time. Try changing the values of each parameter and observe how the accuracies change (it is also a good exercise to do to better understand how the neural net functions). For more on learning rate, epochs, and batch size, there is a great article to read here.

To generate an entry code for Challenge 5, create a new code cell in your Jupyter notebook and paste the following code in it:

member_number = 12345678

one = [member_number, int(member_number/5), int(member_number/25), int(member_number/50), int(member_number/100)]

two = [0.02, 0.05, 0.08, 0.11, 0.14]

a = tf.placeholder(tf.float32, shape=(5))

b = tf.placeholder(tf.float32, shape=(5))

result = tf.tensordot(a, b, 1)

with tf.Session() as sess:

   print(int(result.eval(feed_dict={a: one, b: two})))

Next, replace 12345678 with your CodeProject member number. Run the code, and the number that is printed will be your entry code for the final challenge of the AI contest. Please click here to enter the contest entry code.