Deep Learning Frameworks

Deep learning is a part of broader family of machine learning based on artificial neural networks. Neural networks are basically computing system vaguely inspired by biological neural networks in animal brains. These systems are trained and learn to perform specific tasks based on rules. These are performed without having prior knowledge.

Basically they automatically generate identifying characteristics from the process. It uses neurons to transmit data in form of input values and output values through connections. Deep learning attempts to form relationship between a stimuli (one on who the effect is being observed) and associated neural responses. Deep learning uses neural networks to make its own decisions like image recognition, speech recognition and give proper output.

Why is it needed when machine learning is doing its work?

Machine learning is used for parsing data, learn from the data and make decisions from it. We have the data and we train it, test it and make predictions and see if they are right or wrong. But deep learning is something which makes decisions on its own. It is structured algorithm works in layers to create an ‘artificial network’ which can learn and make decisions on its own.
Suppose we have a collection of cats and dogs in form of images. Now, we have to identify which is image of dogs and which is of cats through algorithms.

There are two ways to do it.

1) Machine learning
2) Deep learning networks.

But how does algorithm know which one is which?
By machine learning, it happens with structured data. You simply label the images of dogs and cats in a way in which it will define specific features of both the animals. This is enough for machine learning algorithms to learn, and then it will classify millions of images as it will understand the classification.
By Deep learning, it will go through a different approach. The artificial neural networks uses deep learning algorithm which sends input as image through different layers, with each network hierarchically defining and classifying different features of animals. This in short is how human brain functions, classifies objects, pass queries through various levels.

How it came into existence? What is the need for it?

In 1943, Warren McCulloch and Walter Pitts made a computer based model based on human brain with a combination of mathematics and algorithms to mimic the logic of thought process.
The need of deep learning is for better results when other techniques like machine learning fail to perform. Deep learning out performs with large amount of data as it can learn in a good and effective way as compared to small datasets. Machine learning is good for small datasets.
When there is lack of feature extraction and its understanding, deep learning out performs in these tasks and makes it easy to make choice.

Structure of deep learning:

In deep learning, there are neural networks whose structure is as follows:
1) Neurons: It is a mathematical function designed to virtually show the functioning of biological neuron. It calculates the weighted average of input and pass on the information through non linear function also known as activation function (for example sigmoid)

2) Weights and Connections: One neuron is connected with another having some value or weight in same layer or another. Each connection is the weight value linked to it. The weight actually means the strength between the units. Our aim is to decrease the weight which will automatically reduce the loss.

3) Propagation function: Two propagation functions work in a Neural Network : these functions delivers ‘predicted value’ and also delivers ‘error values’. To know more about functions in deep learning, Click here

Types of deep learning

1) Deep neural networks
2) Deep belief networks
3) Recurrent neural networks
4) Convolutional neural networks

1. Deep neural network:

Deep neural networks are artificial neural networks in which there is an input layer, an output layers and some hidden layers. These hidden layers undergo mathematical calculation to turn the input into the output. These hidden layers calculate mathematical probability for each layer.
For example, a deep neural network is trained to recognize a dog breed will go over dog’s image and will calculate probability that dog in image is of certain breed.

2. Deep belief networks

Deep belief networks is a graphical model , a class of deep neural network which consists of multiple layers of variables that are not directly observed but are inferred from other variables that are observed. These layers are in connection between the layers but not between the units within each layer. This is the image to understand what is deep belief network.

3. Recurrent neural network

Recurrent neural networks are class of artificial neural networks which allows output to be used as input while having hidden states. It is generally used for natural language processing and speech recognition. RNN ‘s are designed to recognize patterns and predict next likely scenario. To know more, Click here

4. Convolutional neural network

This is a most common used neural network used to analyze visual imagery. These are inspired by biological processes in that connectivity pattern of neurons resembles organization of animal cortex. Having known about connection with human beings, convolutional neural networks are good at recognizing human faces, object recognition and many more.

What tools are used to implement deep learning?

As above you have learnt how huge this deep learning is, tools have to be of large functioning power. These tools might come handy to use for overcoming specific challenges easily for human ease. Following are the frameworks:
1) Tensorflow
2) PyTorch
3) Keras
4) Caffe and Caffe2
5) DeepLearning4J

1) Tensorflow

It is an open source end to end platform for machine learning. Tensorflow has comprehensive, flexible ecosystems for tools, libraries. It is made by Google brain team for numerical computation and for large scale machine learning that makes it faster and easier. Tensorflow eases the data acquiring, training models serving predictions and refining future results. It uses python for comfortable front end API for building applications using frameworks in high performance C++.

Tensorflow has its special usage in digit recognition, pattern recognition word embedding, natural language processing and partial differential equation (PDE) based simulations.
The reason it is so popular because it uses multiple languages like Python, C++, and R.
The best thing about Tensorflow is that is has pre-written codes for most of the complex deep learning models you’ll come across like recurrent neural networks, convolutional neural networks.

Tensorflow application can run on most of any local machine, on cloud, iOS and android phones, CPU’s and GPU’s. The Google’s benefit will also come in picture in a way if you have your own Google cloud, you can run Tensorflow on Google’s custom Tensorflow Processing Unit (TPU) silicon for further acceleration.
The new version of Tensorflow 2.0 was released in October 2019, with new added features to make it easier to work.

2) PyTorch

PyTorch is an open source machine learning library for based on Torch library. Torch library is again an open source machine learning library for scientific computer framework and a scripting language written in Lua programming language. It is mainly used for NLP.

PyTorch is developed by Facebook AI research Lab. It is the most used framework for Deep learning. It is flexible to use. It is python package which provides Tensor computations. The meaning of tensor is- it is a multi dimensional array like numpy’s ndarray which can run on GPU as well. To know the difference between Tensorflow and PyTorch, go to our post Difference between Tensorflow and PyTorch

PyTorch has a package named auto grad package by which it creates computation graphs from tensors and automatically computes gradient. This is the power of PyTorch by which it can make work happen with ease.

3) Keras

This is another framework for deep learning written in Python. If you are comfortable using python then it is best for you to start with deep learning. It is an open source library for neural network. It is a high level API developed with the intention of fast, efficient experimentation. So for quick results, Keras is the one which take care of main tasks and will generate the output. Both convolutional neural networks and recurrent neural networks support Keras. Also it works in an excellent way with CPU’s as well as GPU’s.
It is actually made for to make user’s tasks minimal and also makes it easy to understand models.

Features of Keras that makes it so powerful, easy and widely used framework are:
• It iterates the speed of thought.
• It makes machine learning more powerful with its integration.
• It can be deployed anywhere as it can export its model in javascript, Tensorflow lite to run on iOS, Android, and embedded devices and also an API.
• It is a vast ecosystem as it is a central part and is tightly connected to Tensorflow ecosystem.
• It makes user experience better and is one of the most widely used deep learning tools among university students.

4) Caffe

It is another popular framework of deep learning. It is mostly bent towards image processing field. Caffe is originally developed at UC Berkeley, an open source library written in C++ with a python interface.
Caffe supports many types of deep learning projects related to image classification and image segmentation. RCNN and CNN are supported by Caffe.

It has high performance than Tensorflow by 1.2 to 5 times as per internal benchmarking by Facebook. It works well on image but not on sequence and recurrent neural networks.

5) Deeplearning4J

This is a very different type of framework. You all must be wondering what does this J means. This is a framework for Java community. Deeplearning4J is a deep learning library written for Java and Java Virtual Machine developed by machine learning group headquartered in San Francisco. It is computing framework with support of deep learning algorithms.
Deeplearning4J includes an n-dimensional array class that allows scientific computing in Java and Scala which is very similar to NumPy provides to Python.

*Cuda is Compute Unified Device Architecture. It is a parallel computing platform and application programming interface model created by Nvidia. Parallel computing is computing that carries out calculations or execution of processes simultaneously.

Conclusion

In this blog, we learnt what exactly is deep learning, its existence, how it is most preferred than Machine Learning, how it is used and why it is used. Deep learning is one of the most emerging and more used branch of Artificial Intelligence. Its user experience and accuracy level is good which is the main reason of its high demand. We also learnt about deep learning frameworks like keras, tensorflow, caffe, etc.

These all are having their own special features which makes them useful for different fields like user experience, image processing, etc. Thank you for reading till the end. You can also go through our post Tensorflow : Tensor | Shape | Dtype | Graph | Session