The difference between Machine Learning & Deep Learning

What’s the difference between Machine Learning & Deep Learning? And when to choose one or the other?

More classical Machine Learning techniques often require feature engineering. This means that someone uses their knowledge to list relevant features in the data. The upside is that you need less data because you utilize expert knowledge, the downside is the additional work and risk of no creating the right features. These features are then fed to the Machine Learning algorithm for training. The more features you select, the more computational power you need. If you don’t have unlimited power, consider the following:

Use as few features as possible.
Focus on minimum redundancy of features yet maximum relevance in the output (Correlation Based Feature Selection).
Good feature subsets contain features highly correlated with the classification, yet uncorrelated to each other (Correlation Based Feature Selection).

Deep Learning is a more advanced technique within Machine Learning. It needs larger data sets and more computation power, but it offers automatic improvement and a high accuracy level.

In Deep Learning the task of finding relevant features is part of the algorithm and is automated. Deep Learning requires more data but less a priori knowledge and searches for the most optimal features, whereas with feature engineering we can not assume that the features are optimal.

In the example below, a Machine Learning algorithm detects a car from a picture. The algorithm learns from analyzing lots of car images marked as ‘car’ and other images marked as ‘not car’. Next, you can show other pictures to your algorithm and it will tell you if it’s a car or not. The example shows an additional feature extraction step for the feature-based “Machine Learning” that has to be performed by experts and programmers. The Deep Learning algorithm clearly incorporates that into its neural network.

However, the algorithm won’t tell you how it came to its conclusion. You can test the algorithm to see if it consistently gives the right answers and after that you can decide to trust it or not. This process of validation is an important one and should be performed with care.

This is very similar to teaching a human to execute a task: you teach, test and give a diploma that says ‘this person has got this task covered’. You don’t analyze how their brain translates the question into the answer.

There are techniques that attempt to visualize how a Deep Learning algorithm interprets images in its internal workings, as shown below. It’s like trying to “read the mind of the neural network”.