CNN for handwritten numbers recognition

Build a Convolutional Neural Network for image classification by using Tensorflow library

Image classification is an important task that may be needed in several fields in our daily life, there are many datasets on which machine learning algorithms are applied.

In this article, I'll show a basic program for a constitutional neural network applied on MNIST dataset for handwritten images using Keras.

MNIST dataset Overview

MNIST dataset is composed of 60000 images for the training and 10000 images for training. 

Each image is black and white and has a size of 28x28 pixels.

This image shows some samples.

                                                                Source: Wikipedia

To load the dataset in a Python program, Kears gives a simle command line to do so.

what is a CNN?

CNN's architecture is very simple. It's in the form of convolution layers stacked to fully connected layers. The convolution layers aim to reduce the conditionality of the input data before feeding it to the neural network. 

This idea was first introduced by Yann Lecun who is an artificial intelligence scientist in Facebook. The special thing about CNN is that they are capable of learning the most characterizing features in an image.

                                                                  Source: Wikipedia

In our case, a simple architecture of CNN is used with  two convolution layers followed by maxpooling layers. At last we have two dense layers.

RESULTS

Keep reading, there are some visuals at the end.

	#Handwritten numbers classification
	import tensorflow as tf
	from tensorflow import keras
	from keras.callbacks import EarlyStopping
	import numpy as np
	import matplotlib.pyplot as plt
	tf.executing_eagerly()
	(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
	#Explore data
	print(y_train[12])
	print(np.shape(x_train))
	print(np.shape(x_test))
	#we have 60000 imae for the training and 10000 for testing
	#print(x_train[12])
	# We should normalize data
	x_train = x_train/255
	x_test = x_test/255
	#reshape the data
	#Plot data
	plt.imshow(x_train[12],cmap =plt.cm.binary)
	plt.show()
	x_train = x_train.reshape(60000,28,28,1)
	x_test = x_test.reshape(10000,28,28,1)
	y_train = y_train.reshape(60000,1)
	y_test = y_test.reshape(10000,1)
	print(np.shape(x_train))
	#Create a model
	model = keras.Sequential([
	keras.layers.Conv2D(4,(3,3),(1,1),padding = "same",input_shape=(28,28,1)),
	keras.layers.MaxPooling2D(pool_size = (2,2),padding = "valid"),
	keras.layers.Conv2D(32,(3,3),(1,1),padding = "same"),
	keras.layers.MaxPooling2D(pool_size = (2,2),padding = "valid"),
	keras.layers.Flatten(),
	keras.layers.Dense(128,activation = "relu"),
	keras.layers.Dense(10,activation = "softmax")])
	model.compile(optimizer = "adam",
	loss = "sparse_categorical_crossentropy",
	metrics = ['accuracy'])
	early_stopping = EarlyStopping(monitor='val_loss', patience=2)
	history = model.fit(x_train, y_train, epochs=3, validation_split=0.2, callbacks=[early_stopping])
	print("\ntest accuracy:",test_acc)
	plt.plot(history.history['accuracy'])
	plt.plot(history.history['val_accuracy'])
	plt.title('model accuracy')
	plt.ylabel('accuracy')
	plt.xlabel('epoch')
	plt.legend(['train', 'test'], loc='upper left')
	plt.show()
	# summarize history for loss
	plt.plot(history.history['loss'])
	plt.plot(history.history['val_loss'])
	plt.title('model loss')
	plt.ylabel('loss')
	plt.xlabel('epoch')
	plt.legend(['train', 'test'], loc='upper left')
	plt.show()

view raw CNN hosted with ❤ by GitHub

LOG:

After 3 epochs of training, we've got this 98% accuracy on the test data. We can achieve better result by parameter tuning.

Train on 48000 samples, validate on 12000 samples
Epoch 1/3
48000/48000 [==============================] - 39s 816us/sample - loss: 0.1872 - accuracy: 0.9445 - val_loss: 0.0820 - val_accuracy: 0.9766
Epoch 2/3
48000/48000 [==============================] - 37s 767us/sample - loss: 0.0635 - accuracy: 0.9800 - val_loss: 0.0740 - val_accuracy: 0.9793
Epoch 3/3
48000/48000 [==============================] - 41s 851us/sample - loss: 0.0418 - accuracy: 0.9864 - val_loss: 0.0523 - val_accuracy: 0.9841

test accuracy: 0.9857

Conclusion

Miraculously, the accuracy is of 98%! And it was achieved with a shallow convolutional neural network. It could be improved by hyper-parameters tuning.

This means that the model is almost always capable of correctly identifying the number in the image.

Connect with me:

LinkedIn

Github