What’s the difference between Machine Learning and Deep (Machine) Learning?
And when to choose one or the other? We will refer to feature based Machine Learning simply as “Machine learning” in this section for brevity.
More classical Machine Learning techniques often require feature engineering. This means that someone uses his/her knowledge to list relevant features in the data. The upside is that you need less data because you utilize expert knowledge, the downside is the additional work and risk of no creating the right features. These features are then fed to the Machine Learning algorithm for training. The more features you select, the more computational power you need. If you don’t have unlimited power, consider the following:
- Use as few features as possible.
- Focus on minimum redundancy of features yet maximum relevance in the output (Correlation Based Feature Selection).
- Good feature subsets contain features highly correlated with the classification, yet uncorrelated to each other (Correlation Based Feature Selection).
Deep Learning, is a more advanced technique within Machine Learning. It needs larger data sets and more computation power, but it offers automatic improvement and a high accuracy level. In Deep Learning the task of finding relevant features is part of the algorithm and is automated. Deep Learning requires more data but less a-priori knowledge and searches for the most optimal features whereas with feature engineering we can not assume that the features are optimal.
In the above example, a Machine Learning algorithm detects a car from a picture. The algorithm learns from analyzing lots of car images marked as ‘car’ and other images marked as ‘not car’. Next, you can show other pictures to your algorithm and it will tell you if it’s a car or not. The example shows an additional feature extraction step for the feature based “Machine Learning” that has to be performed by experts and programmers. The Deep Learning algorithm clearly incorporates that into its neural network.
However, the algorithm won’t tell you how it came to its conclusion. You can test the algorithm to see if it consistently gives the right answers and after that you can decide to trust it or not. This process of validation is an important one and should be performed with care.
This is very similar to teaching a human to execute a task: you teach, test and give a diploma that says ‘this person has got this task covered’. You don’t analyze how his/her brain translates the question into the answer.
There are techniques that attempt to visualize how a Deep Learning algorithm interprets images in its internal workings, as shown below. This is like trying to “read mind of the neural network”.
