### Pytorch learning notes (5) -- dimensional transformation of pytorch Foundation

# Catalog

## view/reshape

The usage of view is basically the same as that of reshape. View is the api of version 0.3 of pytorch0.4. In order to keep the reshaping dimension function name consistent with that of numpy, reshape is introduced

In [1]: import torch In [2]: a = torch.rand(4,1,28,28) #Randomly generate a four-dimensional sensor to represent the gray-scale image #Corresponding to batchsize channel height width In [3]: a.shape Out[3]: torch.Size([4, 1, 28, 28]) In [4]: a.view(4,28*28)#As long as the product is used for dimension reconstruction, no error will be reported Out[4]: tensor([[0.3060, 0.2680, 0.3763, ..., 0.6596, 0.5645, 0.8772], [0.7751, 0.9969, 0.0864, ..., 0.7230, 0.4374, 0.1659], [0.5146, 0.5350, 0.1214, ..., 0.2056, 0.2646, 0.8539], [0.5737, 0.5637, 0.2420, ..., 0.7731, 0.6198, 0.6113]]) #Equivalent to merging channel height width In [5]: a.view(4,28*28).shape Out[5]: torch.Size([4, 784]) In [6]: a.view(4*28,28).shape Out[6]: torch.Size([112, 28]) #Equivalent to merging batchsize channel height In [7]: a.view(4*1,28,28).shape Out[7]: torch.Size([4, 28, 28]) #Equivalent to merging batchsize channel In [8]: b = a.view(4,784) In [9]: b.view(4,28,28,1)

Disadvantages: if you want to restore data after using view, you must know the order of data storage, otherwise the wrong order will cause data pollution, such as the last two lines of code

## Add / delete dimensions - squeeze / unssqueeze

Add dimensions using unsqueeze

In [12]: a.unsqueeze(0).shape Out[12]: torch.Size([1, 4, 1, 28, 28]) #The parameter passed in the unsqueeze is equivalent to the index indicating the subscript of the dimension to be added #unsqueeze(0) means to add a dimension at subscript 0 In [13]: a.shape Out[13]: torch.Size([4, 1, 28, 28]) In [15]: a.unsqueeze(-1).shape Out[15]: torch.Size([4, 1, 28, 28, 1]) #Pass in - 1 for the last subscript pass in - 2 for the last subscript #unsqueeze(-1) is to add a dimension to the last subscript In [16]: a.unsqueeze(4).shape Out[16]: torch.Size([4, 1, 28, 28, 1]) #unsqueeze(4) adds a dimension to the 5th subscript because the subscript starts from 0 In [17]: a.unsqueeze(-4).shape Out[17]: torch.Size([4, 1, 1, 28, 28]) #unsqueeze(-4) adds a dimension to the last four subscript, because there are five dimensions after adding a dimension. The fourth dimension is the second dimension In [18]: a.unsqueeze(-5).shape Out[18]: torch.Size([1, 4, 1, 28, 28]) #unsqueeze(-5) add a dimension to the last five subscripts because there are five dimensions after adding one dimension, and the fifth dimension from the bottom is the first dimension In [19]: a.unsqueeze(5).shape #Because there are five dimensions after adding one dimension, it is wrong to use subscript 5 --------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) <ipython-input-19-b54eab361a50> in <module> ----> 1 a.unsqueeze(5).shape RuntimeError: Dimension out of range (expected to be in range of [-5, 4], but got 5)

In [20]: a = torch.tensor([1.2,2.3]) In [21]: a.unsqueeze(-1).shape Out[21]: torch.Size([2, 1]) In [22]: a.shape Out[22]: torch.Size([2]) In [23]: a.unsqueeze(-1) Out[23]: tensor([[1.2000], [2.3000]]) In [24]: a Out[24]: tensor([1.2000, 2.3000]) In [25]: a.unsqueeze(0) Out[25]: tensor([[1.2000, 2.3000]]) In [26]: a.unsqueeze(0).shape Out[26]: torch.Size([1, 2])

Delete dimensions using squeeze

If the deleted dimension length is not 1, the original data will be returned

In [27]: b = torch.rand(1,32,1,1) In [28]: b.shape Out[28]: torch.Size([1, 32, 1, 1]) In [29]: b.squeeze().shape Out[29]: torch.Size([32]) In [30]: b.squeeze(0).shape Out[30]: torch.Size([32, 1, 1]) #Delete first dimension In [31]: b.squeeze(-1).shape Out[31]: torch.Size([1, 32, 1]) #Delete last dimension In [32]: b.squeeze(1).shape Out[32]: torch.Size([1, 32, 1, 1]) #Delete the second dimension because the second dimension is not 1, so it remains unchanged In [34]: b.squeeze(-4).shape Out[34]: torch.Size([32, 1, 1]) #Delete the last four dimensions, that is, delete the first dimension

## Dimension extension - Expand/repeat

Difference: expand dimension expansion adds data when necessary, but only changes the way of data understanding when not adding data, thus saving memory (recommended operation)

repeat adds data by copying

Expand dimensions with expand

There are two conditions to use Expand:

1. The dimension of the extended sensor must be the same as that of the pre extended sensor

2. The dimension length to be extended can only be 1, otherwise it cannot be extended

In [35]: a = torch.rand(4,32,14,14) In [36]: b = torch.rand(1,32,1,1) #Extend b to the same format as a In [37]: a.shape Out[37]: torch.Size([4, 32, 14, 14]) In [38]: b.shape Out[38]: torch.Size([1, 32, 1, 1]) In [39]: b.expand(4,32,14,14).shape#Dimension extension. The extension dimension length is 1 Out[39]: torch.Size([4, 32, 14, 14]) In [40]: b.expand(-1,32,-1,-1).shape#Use - 1 to indicate that the original dimension length is unchanged Out[40]: torch.Size([1, 32, 1, 1]) In [41]: b.expand(-1,32,-1,-4).shape#No error will be reported, but there is a bug in this way Out[41]: torch.Size([1, 32, 1, -4])

Use repeat to extend dimensions (not recommended)

Unlike expand, the parameter passed in in expand is the length of the dimension after expansion, while the parameter passed in repeat is the number of times to copy

In [42]: b.shape Out[42]: torch.Size([1, 32, 1, 1]) In [43]: b.repeat(4,1,1,1).shape#dim0 copies 4 times other dimensions copy once Out[43]: torch.Size([4, 32, 1, 1]) In [44]: b.repeat(4,32,1,1).shape#dim0 replicates 4 times, dim1 replicates 32 times Out[44]: torch.Size([4, 1024, 1, 1]) In [45]: b.repeat(4,1,32,32).shape#dim0 replicates 4 times, dim2 replicates 32 times, dim3 replicates 32 times Out[45]: torch.Size([4, 32, 32, 32])

## Dimension transpose ---- transpose/t/permute

Use t for dimension transpose

This function can only be used for data of two dimension matrix types to transpose data of more than two dimensions. An error will be reported

Use post to transpose

The reason for using the continuous function in the code is that after the dimension is reshaped, the stored data changes from continuous data to discontinuous data, so it is necessary to convert it into continuous data through this function, which is related to the data storage method in pytorch. Please refer to link

In [15]: a.shape Out[15]: torch.Size([4, 3, 32, 32]) #[b,c,h,w] In [16]: a1 = a.transpose(1,3).view(4,3*32*32).view(4,3,32,32) #Transfer (1,3) exchange dim1 and dim3 into [b, W, h, C] = = > [4,32,32,3] #view(4,3*32*32) becomes [b,w*h*c] #view(4,3,32,32) becomes [b,c,w,h] format consistent with the source data, but the order has been disordered, causing data pollution --------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) <ipython-input-16-262e84a7fdcb> in <module> ----> 1 a1 = a.transpose(1,3).view(4,3*32*32).view(4,3,32,32) RuntimeError: invalid argument 2: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Call .contiguous() before .view(). at c:\a\w\1\s\windows\pytorch\aten\src\th\generic/THTensor.cpp:213 In [17]: a1 = a.transpose(1,3).contiguous().view(4,3*32*32).view(4,3,32,32) #Transfer (1,3) exchange dim1 and dim3 into [b, W, h, C] = = > [4,32,32,3] #view(4,3*32*32) becomes [b,w*h*c] #view(4,3,32,32) becomes [b,c,w,h] format consistent with the source data, but the order has been disordered, causing data pollution In [18]: a2 = a.transpose(1,3).contiguous().view(4,3*32*32).view(4,32,32,3).transpose(1,3) #Transfer (1,3) exchange dim1 and dim3 into [b, W, h, C] = = > [4,32,32,3] #view(4,3*32*32) becomes [b,w*h*c] #view(4,3,32,32) becomes [b,w,h,c] #Transform (1,3) to [b,c,h,w] format in the same order as the source data In [19]: a1.shape,a2.shape Out[19]: (torch.Size([4, 3, 32, 32]), torch.Size([4, 3, 32, 32])) In [20]: torch.all(torch.eq(a,a1)) Out[20]: tensor(0, dtype=torch.uint8) #All data are the same return 1 otherwise return 0 In [21]: torch.all(torch.eq(a,a2)) Out[21]: tensor(1, dtype=torch.uint8)

Transpose with permute

#Use translate and permute respectively to set [b, C, h, w] = = > [b, h, W, C] In [22]: a = torch.rand(4,3,28,28) In [23]: a.transpose(1,3).shape Out[23]: torch.Size([4, 28, 28, 3]) In [24]: b = torch.rand(4,3,28,32) In [25]: b.transpose(1,3).shape Out[25]: torch.Size([4, 32, 28, 3]) #[b,c,h,w]==>[b,w,h,c] In [26]: b.transpose(1,3).transpose(1,2).shape Out[26]: torch.Size([4, 28, 32, 3]) #[b,c,h,w]==>[b,w,h,c]==>[b,h,w,c] In [27]: b.permute(0,2,3,1).shape Out[27]: torch.Size([4, 28, 32, 3]) #[b, C, h, w] = > [b, h, W, C] exchange dimensions directly according to Subscripts