Basic functions of processing tensors in 5 PyTorch

Every beginner of deep learning should know these five basic functions of pytoch.

Being able to construct neural networks in an accurate and effective way is one of the most sought after skills of recruiters in deep learning engineers. PyTorch is a Python library mainly used for deep learning. One of the most basic and important parts of PyTorch is to create tensors, which are numbers, vectors, matrices or any n-dimensional array. In order to reduce the computational speed, explicit loops must be avoided when building neural networks. We can use vectorization to avoid this loop. When constructing neural networks, the ability to calculate matrix operations quickly enough is very important.

"Why not use the NumPy library?"

For deep learning, we need to calculate the derivative of model parameters. PyTorch provides the ability to track derivatives during back propagation, while numpy does not, which is called "Auto Grad" in PyTorch. PyTorch provides built-in support for fast execution using GPU. This is crucial in training models. Since numpy lacks the ability to transfer its calculations to GPU, the time to train the model will eventually become very large.

All deep learning projects using PyTorch start with the creation of tensors. Let's look at some of the functions we must know, which are the backbone of any deep learning project involving building neural networks.

• torch.tensor()
• torch.sum()
• torch.index_select()
• torch.stack()
• torch.mm()

After installing pytoch, you can import it directly in the code:

# Import torch and other required modules
import torch

torch.tensor()

Firstly, we define an auxiliary function, describe (x), which will summarize various properties of tensor x, such as tensor type, tensor dimension and tensor content.

# Helper function
def describe(x):
  print("Type: {}".format(x.type()))
  print("Shape/size: {}".format(x.shape))
  print("Values: \n{}".format(x)

Use torch Tensor creates tensors in PyTorch

PyTorch allows us to create tensors in many different ways using the torch package. One way to create a tensor is to initialize a random tensor by specifying its dimension

describe(torch.Tensor(2, 3))

Create tensors declaratively using Python lists

We can also create tensors using python lists. We just need to pass the list as a parameter to the function, and we have its tensor form.

x = torch.Tensor([[1, 2, 3],[4, 5, 6]])
describe(x)

Creating tensors using NumPy arrays

We can also create PyTorch tensors from the NumPy array. The type of tensor is Double Tensor instead of the default Float Tensor. The data type corresponding to NumPy is float64, as shown below.

import numpy as np
npy = np.random.rand(2, 3)
describe(torch.from_numpy(npy))

What can't we do with tensors? Tensors must be real or complex and should not be strings or characters.

torch.tensor([[1, 2], [3, 4, 5]])


---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-5-28787d136593> in <module>
      1 # Example 3 - breaking (to illustrate when it breaks)
----> 2 torch.tensor([[1, 2], [3, 4, 5]])

ValueError: expected sequence of length 2 at dim 1 (got 3)

torch.tensor() forms the core of any PyTorch project, literally because it is a tensor.

torch.sum()

This function returns the sum of all elements in the input tensor.

describe(torch.sum(x, dim=0,keepdims=True))

If you know NumPy, you may have noticed that for 2D tensors, we represent rows as dimension 0 and columns as dimension 1. torch. The sum () function allows us to calculate the sum of rows and columns.

We also pass True for keepdims to preserve the dimensions in the results. By defining dim = 1, we tell the function to collapse the array by column.

torch.sum(npy,dim=1,keepdims=True)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-17-1617bf9e8a37> in <module>()
      1 # Example 3 - breaking (to illustrate when it breaks)
----> 2 torch.sum(npy,dim=1,keepdims=True)

TypeError: sum() received an invalid combination of arguments - got (numpy.ndarray, keepdims=bool, dim=int), but expected one of:
 * (Tensor input, *, torch.dtype dtype)
      didn't match because some of the keywords were incorrect: keepdims, dim
 * (Tensor input, tuple of ints dim, bool keepdim, *, torch.dtype dtype, Tensor out)
 * (Tensor input, tuple of names dim, bool keepdim, *, torch.dtype dtype, Tensor out)

This function is very useful in calculating indicators and loss functions.

torch.index_select()

This function returns a new tensor that indexes the input tensor along the dimension dim using the entry in the index (LongTensor).

indices = torch.LongTensor([0, 2])
describe(torch.index_select(x, dim=1, index=indices))

We can pass the index as a tensor and define the axis as 1, which returns a new tensor size
rows_of_original_tensor x length_of_indices_tensor.

indices = torch.LongTensor([0, 0])
describe(torch.index_select(x, dim=0, index=indices))

We can pass the index as a tensor and define the axis as 0, which returns a size of columns_of_original_tensor x length_of_indices_tensor's new tensor.

indices = torch.FloatTensor([0, 2])
describe(torch.index_select(x, dim=1, index=indices))

This function is useful in complex indexes such as discontinuous indexes of tensors.

torch.stack()

This connects a series of tensors along the new dimension.

describe(torch.stack([x, x, x],dim = 0))

We can pass the tensor we want to connect as a tensor list with dim 0 to stack it along the line.

describe(torch.stack([x, x, x],dim = 1))

We can pass the tensor we want to join as a tensor list with dim 1 to stack it along the column.

y = torch.tensor([3,3])
describe(torch.stack([x, y, x],dim = 1))

--------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-37-c97227f5da5c> in <module>()
      1 # Example 3 - breaking (to illustrate when it breaks)
      2 y = torch.tensor([3,3])
----> 3 describe(torch.stack([x, y, x],dim = 1))

RuntimeError: stack expects each tensor to be equal size, but got [2, 3] at entry 0 and [2] at entry 1

This function is the same as torch index_ The combination of select () is very useful to flatten the matrix.

torch.mm()

This function performs matrix multiplication of a matrix.

mat1 =torch.randn(3,2)
describe(torch.mm(x, mat1))

By simply passing the matrix as a parameter, we can easily perform matrix multiplication, which will produce a new tensor as the product of the two matrices.

mat1 = np.random.randn(3,2)
mat1 = torch.from_numpy(mat1).to(torch.float32)
describe(torch.mm(x, mat1))

In the above example, we define a NumPy array and convert it to a tensor of type float32. Now we can successfully perform matrix multiplication on tensors. The data types of the two tensors must match in order to operate successfully.

mat1 =torch.randn(2,3)
describe(torch.mm(x, mat1))

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-62-18e7760efd23> in <module>()
      1 # Example 3 - breaking (to illustrate when it breaks)
      2 mat1 =torch.randn(2,3)
----> 3 describe(torch.mm(x, mat1))

RuntimeError: mat1 and mat2 shapes cannot be multiplied (2x3 and 2x3)

In order to perform a successful matrix multiplication, the columns of matrix 1 and the rows of matrix 2 must match. torch.mm() function follows the basic rule of matrix multiplication. Even if the order of the matrix is the same, it will not be automatically multiplied by the transpose of another matrix, and you must define it manually.

In order to calculate the derivative in back propagation, it is necessary to be able to effectively perform matrix multiplication, which is torch Where mm () appears.

summary

This concludes our research on five basic PyTorch functions. From basic tensor creation to advanced and little-known functions with specific use cases, such as torch index_ Select (), PyTorch provides many such functions to make the work of data science enthusiasts easier.

Author: Inshal Khan

Finally, if you are interested in participating in the Kaggle competition, please write to me and invite you to join the Kaggle competition exchange group