Introduction to Neural Networks

In our previous blog, we discussed some basics of machine learning. Now it is time to discuss further by introducing machine learning.

If you have looked at the examples for each machine learning technique, you might notice that neural networks are used in all the three cases. Obviously, neural networks fall under machine learning. It is so powerful algorithm and very popular among machine learning enthusiasts.

Neural networks are based on biological neural networks. In the brain, when a signal from one of the sensors was received, that signal is directed to so many neurons in the brain. The outputs of those neurons are then connected to so many more other neurons. In these artificial neural networks also there are so many interconnections between each other.

Simple Nerual Network

The interconnections inside an artificial neural network are arranged in this manner. First, the input to the neural network is connected to a set of neurons. The output of that set is then connected to another set of neurons. These sets are called layers. So in this network, there are 3 layers; an input layer, a hidden layer and an output layer. Sometimes this network is called a 2 layer network because some people don’t want to count the input layer as a “layer”. Anyway, it is better to tell that this as a neural network with one hidden layer.

As I told in the previous article, neural networks are trained with supervised, unsupervised and reinforcement learning methods. But all in these cases, a technique called backpropagation is used to update the variables in the neural network.

When the neural network has more hidden layers it is called a deep neural network.


When the interconnection between the neurons and their arrangement change in a nerual network, the performance and the functionality of the network changes. Researches have found so many ways to connect neurons so that it will gives a usefull output. Some of those types are given below.

  1. Feedforward Neural Network (Artificial Neural Network)
    • Single-layer perceptron
    • Multi-layer perceptron
  2. Radial basis function Neural Network
  3. Kohonen Self Organizing Neural Network
  4. Recurrent Neural Network
    • Long short-term memory (LSTM)
    • Gated recurrent unit (GRU)
    • Recurrent multilayer perceptron network (RMPN)
    • Neural Turing machines (NTM)
    • Continuous time recurrent neural network (CTRNN)
  5. Convolutional Neural Network
    • LeNet
    • AlexNet
    • VGG
    • Inception
    • ResNet
    • DenseNet

  1. ↩︎