How Machine Learning is Different from Deep Learning

Ah! So you have heard about Deep Learning! 

Deep learning is actually a sub-field of Machine Learning. So, if you were to represent Machine Learning and Deep Learning by a simple Venn diagram – it will look like this:

Machine learning and deep learning are on a rage! All of a sudden everyone is talking about them – irrespective of whether they understand the differences or not! Whether you have been actively following data science or not – you would have heard these terms.

 Table of Contents 

1. What are Machine Learning and Deep Learning?
   1. What is Machine Learning?
   2. What is Deep Learning?
2. Comparison of Machine Learning and Deep Learning
1. Data Dependencies
2. Hardware Dependency
3. Problem Solving Approach
4. Feature Engineering
5. Future trends

What are Machine Learning and Deep Learning?

Let us start with the basics – What is Machine Learning and What is Deep Learning. If you already know this, feel free to move to section 2.

1.1 What is Machine Learning?

The widely-quoted definition of Machine learning by Tom Mitchell best explains machine learning in a nutshell. Here’s what it says:

“A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E ”

Did that sound puzzling or confusing? Let’s break this down with simple examples.

Example 1 – Machine Learning – Predicting weights based on height

Let us say you want to create a system that tells expected weight based on the height of a person. There could be several reasons why something like this could be of interest. You can use this to filter out any possible fraud or data capturing errors. The first thing you do is collect data. Let us say this is how your data looks like:

1.2 What is Deep Learning?

The concept of deep learning is not new. It has been around for a couple of years now. But nowadays with all the hype, deep learning is getting more attention. As we did in Machine Learning, we will look at a formal definition of Deep Learning and then break it down with examples.

“Deep learning is a particular kind of machine learning that achieves great power and flexibility by learning to represent the world as nested hierarchy of concepts, with each concept defined in relation to simpler concepts, and more abstract representations computed in terms of less abstract ones.”

Now – that one would be confusing. Let us break it with a simple example.

Example 1 – Shape detection

Let me start with a simple example that explains how things happen at a conceptual level. Let us try and understand how we recognize a square from other shapes.

The first thing our eyes do is check whether there are 4 lines associated with a figure or not (simple concept). If we find 4 lines, we further check, if they are connected, closed, perpendicular and that they are equal as well (nested hierarchy of concept).

So, we took a complex task (identifying a square) and broke it into simple less abstract tasks. Deep Learning essentially does this at a large scale.

Example 2 – Cat vs. Dog

Let’s take an example of an animal recognizer, where our system has to recognize whether the given image is of a cat or a dog.

If we solve this as a typical machine learning problem, we will define features such as if the animal has whiskers or not, if the animal has ears & if yes, then if they are pointed. In short, we will define the facial features and let the system identify which features are more important in classifying a particular animal.

Now, deep learning takes this one step ahead. Deep learning automatically finds out the features which are important for classification, wherein in Machine Learning we had to manually give the features.

2. Comparison of Machine Learning and Deep Learning

Now that you have understood an overview of Machine Learning and Deep Learning, we will take a few important points and compare the two techniques.

2.1 Data dependencies

The most important difference between deep learning and traditional machine learning is its performance as the scale of data increases. When the data is small, deep learning algorithms don’t perform that well. This is because deep learning algorithms need a large amount of data to understand it perfectly. On the other hand, traditional machine learning algorithms with their handcrafted rules prevail in this scenario. The below image summarizes this fact.

2.2 Hardware dependencies

Deep learning algorithms heavily depend on high-end machines, contrary to traditional machine learning algorithms, which can work on low-end machines. This is because the requirements of a deep learning algorithm include GPUs which are an integral part of its working. Deep learning algorithms inherently do a large amount of matrix multiplication operations. These operations can be efficiently optimized using a GPU because GPU is built for this purpose.

2.3 Feature engineering

Feature engineering is a process of putting domain knowledge into the creation of feature extractors to reduce the complexity of the data and make patterns more visible to learning algorithms to work. This process is difficult and expensive in terms of time and expertise.

In Machine learning, most of the applied features need to be identified by an expert and then hand-coded as per the domain and data type.

For example, features can be pixel values, shape, textures, position, and orientation. The performance of most Machine Learning algorithms depends on how accurately the features are identified and extracted.

Deep learning algorithms try to learn high-level features from data. This is a very distinctive part of Deep Learning and a major step ahead of traditional Machine Learning. Therefore, deep learning reduces the task of developing a new feature extractor for every problem. Like, Convolutional NN will try to learn low-level features such as edges and lines in early layers then parts of faces of people, and then a high-level representation of a face.

2.4 Problem Solving approach

When solving a problem using a traditional machine learning algorithm, it is generally recommended to break the problem down into different parts, solve them individually and combine them to get the result. Deep learning in contrast advocates solving the problem end-to-end.

Let’s take an example to understand this.

Suppose you have a task of multiple object detection. The task is to identify what is the object and where is it present in the image.

2.5 Future Trends

The above article would have given you an overview of Machine Learning and Deep Learning and the difference between them. In this section, I’m sharing my views on how Machine Learning and Deep Learning would progress in the future.

• First of all, seeing the increasing trend of using data science and machine learning in the industry, it will become increasingly important for each company that wants to survive to inculcate Machine Learning in their business. Also, each and every individual would be expected to know the basic terminologies.
• Deep learning is surprising us each and every day and will continue to do so in the near future. This is because Deep Learning is proving to be one of the best techniques to be discovered with state-of-the-art performances.
• Research is continuous in Machine Learning and Deep Learning. But unlike in previous years, where research was limited to academia, research in Machine Learning and Deep Learning is exploding in both industry and academia. And with more funds available than ever before, it is more likely to be a keynote in human development overall.

I personally follow these trends closely. I generally get a scoop from Machine Learning/Deep Learning newsletters, which keep me updated with recent happenings. Along with this, I follow research papers and their respective code, which are published every day.

Endnotes

In this article, we had a high-level overview and comparison between deep learning and machine learning techniques. I hope I could motivate you to learn further in machine learning and deep learning.  Here is the link for the Learning path for machine learning.

If you have any questions, feel free to drop them below in the comments section.