What makes Deep Learning different from traditional Machine Learning methods?


Original Source Here

Photo by Moritz Kindler on Unsplash

What makes Deep Learning different from traditional Machine Learning methods?

3 key characteristics that set Deep Neural Networks apart

All opinions expressed are my own. This article is heavily inspired by Prof. Zsolt Kira’s awesome CS7643 lectures.

So you are at a party where you c̶a̶n̶’̶t̶ ̶s̶e̶e̶m̶ ̶t̶o̶ ̶s̶h̶u̶t̶ ̶u̶p̶ ̶ are telling everyone enthusiastically about how AI will change the world and you know that because you are an AI Enthusiast or a Machine Learning Practitioner or whatever fancy-sounding title you use to describe yourself. Then, someone who reads the newspaper every now and then asks you “I don’t really get the difference between Deep Learning and traditional Machine Learning. What is the difference between Deep Learning and (classical) Machine learning?”. You give some handwavy explanation about how neurons in a neural network are based on the human brain blah blah blah…an answer that leaves people unconvinced and leaves you embarrassed. So here’s how to properly answer that question — there are 3 key characteristics of Deep Learning that set it apart:

  1. Hierarchical Compositionality
  2. Distributed Representation
  3. End-to-end Learning

Hierarchical Compositionality

Photo by Terry Vlisidis on Unsplash

Deep Learning model architectures tend to be based on the idea that the world around exhibits hierarchical compositionality. In simple terms, every complex thing in the world is made up of simple building blocks and these simple building blocks are in turn made up of even simpler building blocks. This is analogous to how in Chemistry, a compound is made up of molecules and molecules in turn are made of atoms or in a business context, how the organisation is made up of departments, departments are made of teams and teams are made of employees.

Likewise, in an image, pixels form edges, edges form shapes, shapes form textons (a complex combination of shapes), textons form objects and objects form a complete image. In the case of Natural Language Processing, characters form words, words form phrases, phrases form sentences, sentences form paragraphs. By mirroring this hierarchical nature of the data through their architectures, Deep Learning models are able to learn how the simpler parts form the complex whole by modeling the hierarchical relationships in the data.

Feature Visualization of Convnet trained on ImageNet [Zeiler & Fergus 2013]

In a traditional Machine Learning approach, an expert would get involved to hand-pick these low-level features and then hand-engineer the extraction of these low-level features, which could then be fed to model such as a SVM classifier. Not only is this process cumbersome, choosing and defining the features involves some degree of interpretative decision making , which is prone to bias and loss of information due to oversimplification of the data. Deep Learning, on the other hand, takes the raw data as its input and automatically learns the hierarchical elements and their relationships through the training process.


Trending AI/ML Article Identified & Digested via Granola by Ramsey Elbasheer; a Machine-Driven RSS Bot

%d bloggers like this: