Friends who have seen my previous ResNet18 and ResNet34 may wonder whether the method of building 18 and 34 layers can be directly used in the construction of ResNet above 50 layers. I have also tried. However, the network construction above ResNet50 is not like 18 to 34 layers, which can be completed by simply modifying the number of convolution units. The three networks above ResNet50 are the same, but the number of layers is different, so 34 to 50 layers can be used as a construction watershed.
In addition, I'm a beginner of PyTorch and deep neural network, and I don't know much about the efficient construction of using BasicBlock and BottleNeck, so here is a construction method of simple violent stacking network layer similar to the first two resnets
ResNet50 network architecture
ResNet for all different layers:
Here are the detailed parameters and execution flow chart of ResNet50 network:
Direct code
model.py model part:
import torch
import torch.nn as nn
from torch.nn import functional as F
"""
there ResNet50 The construction of is a form of violence, which is directly accumulated to complete the construction, but it is not adopted BasicBlock and BottleNeck
first DownSample Class for defining shortcut Model function to complete two layer Dotted line between shortcut，be responsible for layer1 The rise of the dotted line is 4 times channel And others layer 2 times of the dotted line channel
Observe each layer Rise at the dotted line of channel Just rise channel The number before and after is different, and stride Different, for kernel_size and padding Both are 1 and 0, respectively. Do not act DownSample Model parameters of network class
parameter in_channel That is, the number of channels before the rise, out_channel That is, the number of channels after the rise, stride That is, every rise channel different stride Step size, for layer1 Ascending channel stride=1，other layer Ascending channel stride=2，Pay attention to different
"""
"""
Be sure to pay attention to:
In this network ResNet50 In class forward Inside the function: layer1_shortcut1.to('cuda:0')；layer2_shortcut1.to('cuda:0')And other statements will be instantiated DownSample
Network model train.py Defined in the training script GPU In the same environment, the following errors will be reported without this sentence:
Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same
"""
class DownSample(nn.Module):
def __init__(self, in_channel, out_channel, stride): # Number of channel s before and after incoming downsampling and stripe step size
super(DownSample, self).__init__() # Inherit parent class
self.down = nn.Sequential( # Define a model container down
nn.Conv2d(in_channel, out_channel, kernel_size=1, stride=stride, padding=0, bias=False), # It is responsible for the only and important convolution of dashed shortcut
nn.BatchNorm2d(out_channel), # Between convolution and ReLU nonlinear activation, add BatchNormalization
nn.ReLU(inplace=True) # At the end of shortcut, add an activation function and set inplace=True to operate in place to save memory
)
def forward(self, x):
out = self.down(x) # The forward propagation function only completes the operation of the down container
return out
"""
first ResNet50 Class, not used BottleNeck Unit complete ResNet50 The building above the floor can be used directly forward Plus the front DownSample Model class functions, specifying ResNet50 Build model with all parameters
"""
class ResNet50(nn.Module):
def __init__(self, classes_num): # ResNet50 only transmits one classification number and writes all the data involved. For specific data, please refer to the following picture
super(ResNet50, self).__init__()
# Before entering layer 1234, preprocessing is carried out, mainly convolution and pooling, from [batch, 3, 224, 224] = > [batch, 64, 56, 56]
self.pre = nn.Sequential(
# Convolution channel is upgraded from 3 channels of original data to 64 channels with 64 convolution cores. The size, step size and padding of convolution cores are fixed
nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False),
nn.BatchNorm2d(64), # The convolution is followed by BatchNormalization
nn.ReLU(inplace=True),
nn.MaxPool2d(kernel_size=3, stride=2, padding=1) # Preprocess the last maximum pool operation, and the data is fixed
)
"""
every last layer The operation of is divided into one-time operation first，And used many times next form, first Responsible for each layer Cubic convolution of the first unit (with dotted line) of, next Responsible for the cubic convolution of the remaining units (direct connection)
"""
# --------------------------------------------------------------
self.layer1_first = nn.Sequential(
nn.Conv2d(64, 64, kernel_size=1, stride=1, padding=0, bias=False), # layer1_ The first convolution keeps the channel unchanged for the first time, which is different from the first convolution of other layers
nn.BatchNorm2d(64),
nn.ReLU(inplace=True),
nn.Conv2d(64, 64, kernel_size=3, stride=1, padding=1, bias=False), # layer1_first second convolution stripe and other layers_ The first string is different
nn.BatchNorm2d(64),
nn.ReLU(inplace=True),
nn.Conv2d(64, 256, kernel_size=1, stride=1, padding=0, bias=False), # layer1_ The first third convolution is the same as other layers, and the channel is increased by 4 times
nn.BatchNorm2d(256) # Note that the ReLU activation function is not added at the end of the last convolution
)
self.layer1_next = nn.Sequential(
nn.Conv2d(256, 64, kernel_size=1, stride=1, padding=0, bias=False), # layer1_ The first convolution of next is responsible for reducing the channel and reducing the amount of training parameters
nn.BatchNorm2d(64),
nn.ReLU(inplace=True),
nn.Conv2d(64, 64, kernel_size=3, stride=1, padding=1, bias=False),
nn.BatchNorm2d(64),
nn.ReLU(inplace=True),
nn.Conv2d(64, 256, kernel_size=1, stride=1, padding=0, bias=False), # layer1_ The last convolution of the next is responsible for adding the channel to the shortcut
nn.BatchNorm2d(256)
)
# -------------------------------------------------------------- # The operation of layer234 is basically the same. Only layer2 is introduced here
self.layer2_first = nn.Sequential(
nn.Conv2d(256, 128, kernel_size=1, stride=1, padding=0, bias=False), # And layer1_ The first convolution is different, and the channel needs to be reduced to 1 / 2
nn.BatchNorm2d(128),
nn.ReLU(inplace=True),
nn.Conv2d(128, 128, kernel_size=3, stride=2, padding=1, bias=False), # Note that stripe = 2 here is the same as layer34, but different from layer1
nn.BatchNorm2d(128),
nn.ReLU(inplace=True),
nn.Conv2d(128, 512, kernel_size=1, stride=1, padding=0, bias=False), # Raise channel again
nn.BatchNorm2d(512)
)
self.layer2_next = nn.Sequential(
nn.Conv2d(512, 128, kernel_size=1, stride=1, padding=0, bias=False), # Responsible for the general operation of the cycle
nn.BatchNorm2d(128),
nn.ReLU(inplace=True),
nn.Conv2d(128, 128, kernel_size=3, stride=1, padding=1, bias=False),
nn.BatchNorm2d(128),
nn.ReLU(inplace=True),
nn.Conv2d(128, 512, kernel_size=1, stride=1, padding=0, bias=False),
nn.BatchNorm2d(512)
)
# --------------------------------------------------------------
self.layer3_first = nn.Sequential(
nn.Conv2d(512, 256, kernel_size=1, stride=1, padding=0, bias=False),
nn.BatchNorm2d(256),
nn.ReLU(inplace=True),
nn.Conv2d(256, 256, kernel_size=3, stride=2, padding=1, bias=False),
nn.BatchNorm2d(256),
nn.ReLU(inplace=True),
nn.Conv2d(256, 1024, kernel_size=1, stride=1, padding=0, bias=False),
nn.BatchNorm2d(1024)
)
self.layer3_next = nn.Sequential(
nn.Conv2d(1024, 256, kernel_size=1, stride=1, padding=0, bias=False),
nn.BatchNorm2d(256),
nn.ReLU(inplace=True),
nn.Conv2d(256, 256, kernel_size=3, stride=1, padding=1, bias=False),
nn.BatchNorm2d(256),
nn.ReLU(inplace=True),
nn.Conv2d(256, 1024, kernel_size=1, stride=1, padding=0, bias=False),
nn.BatchNorm2d(1024)
)
# --------------------------------------------------------------
self.layer4_first = nn.Sequential(
nn.Conv2d(1024, 512, kernel_size=1, stride=1, padding=0, bias=False),
nn.BatchNorm2d(512),
nn.ReLU(inplace=True),
nn.Conv2d(512, 512, kernel_size=3, stride=2, padding=1, bias=False),
nn.BatchNorm2d(512),
nn.ReLU(inplace=True),
nn.Conv2d(512, 2048, kernel_size=1, stride=1, padding=0, bias=False),
nn.BatchNorm2d(2048)
)
self.layer4_next = nn.Sequential(
nn.Conv2d(2048, 512, kernel_size=1, stride=1, padding=0, bias=False),
nn.BatchNorm2d(512),
nn.ReLU(inplace=True),
nn.Conv2d(512, 512, kernel_size=3, stride=1, padding=1, bias=False),
nn.BatchNorm2d(512),
nn.ReLU(inplace=True),
nn.Conv2d(512, 2048, kernel_size=1, stride=1, padding=0, bias=False),
nn.BatchNorm2d(2048)
)
self.avg_pool = nn.AdaptiveAvgPool2d((1, 1)) # After the final adaptive mean pooling, it is [batch, 2048, 1, 1]
# Define the last full connection layer
self.fc = nn.Sequential(
nn.Dropout(p=0.5), # Inactivate neurons with a probability of 0.5
nn.Linear(2048 * 1 * 1, 1024), # First full connection layer
nn.ReLU(inplace=True),
nn.Dropout(p=0.5),
nn.Linear(1024, classes_num) # The second full connection layer outputs class results
)
"""
forward()The forward propagation function is responsible for ResNet Class, and then with the above DownSample Class perfect combination
"""
def forward(self, x):
out = self.pre(x) # Preprocess the input and output out = [batch, 64, 56, 56]
"""
Each floor layer The operation consists of two parts, the first is the convolution unit with dotted line, and the other is the ordinary convolution unit completed by loop shortcut Directly connected convolution unit
"""
layer1_shortcut1 = DownSample(64, 256, 1) # Instantiate a network model layer1 using DownSample_ Shortcut1, the parameter is the ascending channel data at the dotted line. Note that stripe = 1
layer1_shortcut1.to('cuda:0')
layer1_identity1 = layer1_shortcut1(out) # Call layer1_shortcut1 calculates the identity at the dotted line for the convolution unit input out, which is used to add with the convolution unit output later
out = self.layer1_first(out) # Call layer1_first completes the first special convolution unit of layer1
out = F.relu(out + layer1_identity1, inplace=True) # Add the identity to the output of the convolution unit and activate the function through relu
for i in range(2): # Use the loop to complete the convolution unit with the same input and output operations
layer_identity = out # Direct connection identity equals input
out = self.layer1_next(out) # The input passes through the ordinary convolution unit
out = F.relu(out + layer_identity, inplace=True) # The two results are added and then passed through the activation function
# --------------------------------------------------------------Later, layer234 is similar, and only layer2 is introduced here
layer2_shortcut1 = DownSample(256, 512, 2) # Note that the input and output channel s of layer234 are different, and the same is true for stripe = 2
layer2_shortcut1.to('cuda:0')
layer2_identity1 = layer2_shortcut1(out)
out = self.layer2_first(out)
out = F.relu(out + layer2_identity1, inplace=True) # Complete the first convolution unit of layer2
for i in range(3): # Loop the remaining convolution units of layer2
layer_identity = out
out = self.layer2_next(out)
out = F.relu(out + layer_identity, inplace=True)
# --------------------------------------------------------------
layer3_shortcut1 = DownSample(512, 1024, 2)
layer3_shortcut1.to('cuda:0')
layer3_identity1 = layer3_shortcut1(out)
out = self.layer3_first(out)
out = F.relu(out + layer3_identity1, inplace=True)
for i in range(5):
layer_identity = out
out = self.layer3_next(out)
out = F.relu(out + layer_identity, inplace=True)
# --------------------------------------------------------------
layer4_shortcut1 = DownSample(1024, 2048, 2)
layer4_shortcut1.to('cuda:0')
layer4_identity1 = layer4_shortcut1(out)
out = self.layer4_first(out)
out = F.relu(out + layer4_identity1, inplace=True)
for i in range(2):
layer_identity = out
out = self.layer4_next(out)
out = F.relu(out + layer_identity, inplace=True)
# The last full connection layer
out = self.avg_pool(out) # After the final adaptive mean pooling, it is [batch, 2048, 1, 1]
out = out.reshape(out.size(0), -1) # Flatten the convolution input [batch, 2048, 1, 1] to [batch, 2048*1*1]
out = self.fc(out) # After the last full connection unit, the output classification is out
return out
The training of ResNet50 can refer to the training and testing part of my ResNet18:
Use PyTorch to build ResNet18 network and use CIFAR10 data set to train and test
After the violent construction of handwritten ResNet50 network model, I realized that in order to build ResNet and other complex networks, the premise must be clear about the whole process of the model
For example, the in of dimension upgrading operation in each shortcut in ResNet50_ channel,out_ channel,kernel_ The parameter size of size, stripe and padding changes
What are the specific parameters of each convolution unit and how to maximize and simplify the code;
In addition, in building a complex network model, we must build and test step by step. Each step must be understood and justified, and finally the whole network can be built
Another unexpected gain is the discovery of such errors during training:
Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same
The original reason is that the input data type is torch cuda. Floattensor, indicating that the input data is in GPU. The data type of model parameters is torch Floattensor, indicating that the model is still in the CPU
Therefore, add the instantiated DownSample network to and train in the forward() function of ResNet50 Py solves the error under the same GPU of the network model instantiated by ResNet50