Tensors: Data structure for Deep Learning

Original Source Here

Scalars (0 dimensional tensors)

Any quantity that can be represented by a Real Number is called as Scalar. That means the height of a person, weight of a brick, temperature of the room etc are all scalar quantities. This number can be positive or negative. For instance a temperature reading of -18C is a valid scalar quantity. You can also say that a scalar quantity is just a Point since it has got only magnitude/size and no length, breadth or height and hence it has 0 dimension.

In deep learning, a tensor that contains only 1 number is called as a scalar. If you notice carefully we don’t call it as a number but we say that it’s a tensor of 0 dimension that contains a number of integer or float type. Which means that even though we are dealing with Real numbers here, we usually won’t use it as raw input (integer or float type) but rather convert it into tensor type and use it. Libraries such as numpy help us to do that.

The values of x0, x1 and x2 are integers. The only difference is in the data type. x0 is of an int type whereas x1 and x2 are of type numpy.int32 and numpy.ndarray respectively. Since numpy converted x1 and x2 to tensor we are able to execute functions like ndim (numpy function to get the dimension of a tensor) and shape (I will explain this below). This cannot be done for x0 since it is not a tensor and doesn’t contain dimension and shape properties.

In this case, since x1 and x2 are scalar we got dimension as 0 and shape as none but the point is these variables are tensors.

Vectors (1 dimensional tensors)

A quantity that is described by its magnitude/size and in addition to that needs another parameter called as direction, to describe itself, is called a vector. A simple example would be Force. Force has got a magnitude (the amount of force applied on an object) and a direction in which it is applied. If we could plot this quantity on a horizontal plane, we will start with a point (scalar with a magnitude) and stretch until we have reached another point, we would get a line segment that then describes this quantity. Amount of stretch that we had to draw gives us the length/direction and the horizontal plane is called as its axis. Since we need only one axis to describe it, it’s one dimensional.

Back to deep learning: An array of numbers is called a vector. In other words to describe a vector you need to use an array of scalars. Let’s take can example of color. In computer graphics color is a vector quantity. For instance, a RGB color system consists of 3 scalar quantity Red, Green and Blue. The combined intensity of these 3 components determines the output color.

Matrices (2 dimensional tensors)

A Matrix consists of an array of vectors and has got 2 axes which are usually referred to as rows and columns.

The most simplest example (and commonly used one) is that of a display unit, example: computer monitor, LCD screen, television set etc. The screen is a matrix of fixed number of rows and columns(1920×1080, 1024×768 etc) called as its resolution . If rows and columns could be denoted as x and y respectively, then at each intersection of (x, y) on the screen, we get a Picture Element or Pixel: the basic unit of any image displayed on the screen. For a grayscale image the value of a pixel (or its intensity) will range any where between 0–255.

If we consider a very tiny image of 10×10 pixels, it can be represented as:

3D, 4D and 5D ( higher dimensional tensors)

If we have got an array of 2D tensors, it will form a 3D tensor. Similarly if we have an array of 3D tensor it will form a 4D tensor and an array of 4D tensor will form a 5D tensor. Higher dimension tensors are used for processing highly complex tasks such as Image and Video processing.


Trending AI/ML Article Identified & Digested via Granola by Ramsey Elbasheer; a Machine-Driven RSS Bot

%d bloggers like this: