Tensors, Matrices, Vectors and Python – Part One

What are tensors?

In the AI field it becomes extremely important to learn how to handle and manage data in the form of tensors. Keep in mind that in deep learning, in fact, data are treated exclusively as tensors. But what are they?

A formal definition could be the following: a tensor is a mathematical object defined in a vector space and therefore not dependent on a particular frame of reference. Ok, but what does that mean? We can try to simplify by saying that tensors are the natural generalization of the concept of a vector, that is, an object that has magnitude and direction. Interesting, but what does that mean? I could go on like this ? I will try to give an explanation approaching it exclusively from a computer science point of view.

An array in computer science denotes a complex, static, and homogeneous data structure, and a vector can be defined as a one-dimensional 1D array, that is, a list of numbers.

V = [1 3 5 8]

Vectors can in turn be subdivided into simpler components — the elements referred to by indices. Thus with  one indicates the i-th element.

A matrix, on the other hand, is a two-dimensional 2D array:

In a matrix denotes the element of the i-th row and j-th column.

By doing this, adding a dimension, we could draw a “cubic matrix” etc etc … by analogy we thus arrive at the definition of a tensor.

The tensor is a multidimensional array, therefore both the vector and the matrix are examples of tensors of dimension 1 and 2 respectively.

At this point we can introduce two fundamental properties of tensors, which we will also hear about in the AI field.

1. Rank indicates the number of dimensions of the tensor (for example, a cubic matrix has order 3, a matrix has order 2).
2. Shape represents the form of the tensor and is expressed in the following format n x m x k  in the case of a cubic matrix. (for example a cubic matrix can have shape 1x1x1 or 2x5x7).

Let’s do some examples:

A Tensor with Rank=2 and Shape=2×4 is a matrix with 2 rows and 4 columns

A Tensor with Rank=2 and Shape=4×2 is a matrix of 4 rows and 2 columns.

A tensor of 5 elements in a single dimension is a vector V = [1 , 3 , 5, 8]

How can a vector be represented and used in Python? The easiest way is to use numpy, a very useful mathematical library.

Therefore, referring back to the previous example of vector V, in Python we should write something like this:

import numpy as np

v=np.array([1,3,5,8])

At this point we can use Python to calculate, for example, the shape and rank of v

np.shape(v)

whose result is (4,), i.e. a tensor with a single dimension of 4 elements

np.rank(v)

whose result is 1 dimension.

Let’s do another example; let’s consider a matrix :

In Python we will have:

>>> M = np.array([[1,2,3],[3,4,5]])

>>> np.shape(M)

(2, 3)

>>> np.rank(M)

2

>>> 

In this case then we will have :

shape= 2 x 3

rank = 2

Second part >>