日本免费全黄少妇一区二区三区-高清无码一区二区三区四区-欧美中文字幕日韩在线观看-国产福利诱惑在线网站-国产中文字幕一区在线-亚洲欧美精品日韩一区-久久国产精品国产精品国产-国产精久久久久久一区二区三区-欧美亚洲国产精品久久久久

  1. 首頁 > 云知道 > >

Resnet50詳解與實(shí)踐 resnet網(wǎng)絡(luò)結(jié)構(gòu)詳解( 二 )

云知道


注意：如果殘差映射(F(x))的結(jié)果的維度與跳躍連接(x)的維度不同 ， 那咱們是沒有辦法對(duì)它們兩個(gè)進(jìn)行相加操作的 ， 必須對(duì)x進(jìn)行升維操作 ， 讓他倆的維度相同時(shí)才能計(jì)算 。
升維的方法有兩種：
1、用0填充；
2、采用1*1的卷積 。一般都是采用1*1的卷積 。
#以下是代碼：#導(dǎo)入庫mport torchimport torch.nn as nnimport torch.nn.functional as Ffrom torch.autograd import Variable#定義殘差塊（BasicBlock是小殘差塊 ， Bottleneck是大殘差塊）class BasicBlock(nn.Module):#定義blockexpansion = 1def __init__(self, in_channels, channels, stride=1, downsample=None):#輸入通道 ， 輸出通道 ， stride ， 下采樣super(BasicBlock, self).__init__()self.conv1 = conv3x3(in_channels, channels, stride)self.bn1 = nn.BatchNorm2d(channels)self.relu = F.relu(inplace=True)self.conv2 = conv3x3(channels, channels)self.bn2 = nn.BatchNorm2d(channels)self.downsample = downsampleself.stride = stridedef forward(self, x):residual = xout = self.conv1(x)out = self.bn1(out)out = self.relu(out)out = self.conv2(out)out = self.bn2(out)if self.downsample is not None:residual = self.downsample(x)out += residualout = self.relu(out)return out#block輸出class Bottleneck(nn.Module):expansion = 4def __init__(self, in_planes, planes, stride=1):super(Bottleneck, self).__init__()self.conv1 = nn.Conv2d(in_planes, planes, kernel_size=1, bias=False)self.bn1 = nn.BatchNorm2d(planes)self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride, padding=1, bias=False)self.bn2 = nn.BatchNorm2d(planes)self.conv3 = nn.Conv2d(planes, self.expansion*planes, kernel_size=1, bias=False)self.bn3 = nn.BatchNorm2d(self.expansion*planes)self.shortcut = nn.Sequential()if stride != 1 or in_planes != self.expansion*planes:self.shortcut = nn.Sequential(nn.Conv2d(in_planes, self.expansion*planes, kernel_size=1, stride=stride, bias=False),nn.BatchNorm2d(self.expansion*planes))def forward(self, x):out = F.relu(self.bn1(self.conv1(x)))out = F.relu(self.bn2(self.conv2(out)))out = self.bn3(self.conv3(out))out += self.shortcut(x)out = F.relu(out)return out#定義殘差網(wǎng)絡(luò)class ResNet(nn.Module):def __init__(self, block, num_blocks, num_classes=9,embedding_size=256):super(ResNet, self).__init__()self.in_planes = 64self.conv1 = nn.Conv2d(1, 64, kernel_size=3, stride=1, padding=1, bias=False)self.bn1 = nn.BatchNorm2d(64)self.layer1 = self._make_layer(block, 64, num_blocks[0], stride=1)self.layer2 = self._make_layer(block, 128, num_blocks[1], stride=2)self.layer3 = self._make_layer(block, 256, num_blocks[2], stride=2)self.layer4 = self._make_layer(block, 512, num_blocks[3], stride=2)self.avg_pool = nn.AdaptiveAvgPool2d([4, 1])self.fc=nn.Linear(512*4, embedding_size)self.linear = nn.Linear(embedding_size, num_classes)def _make_layer(self, block, planes, num_blocks, stride):strides = [stride] + [1]*(num_blocks-1)layers = []for stride in strides:layers.append(block(self.in_planes, planes, stride))self.in_planes = planes * block.expansionreturn nn.Sequential(*layers)def forward(self, x):x = torch.tensor(x, dtype=torch.float32)out = F.relu(self.bn1(self.conv1(x)))out = self.layer1(out)out = self.layer2(out)out = self.layer3(out)out = self.layer4(out)out =self.avg_pool(out)out = out.view(out.size(0), -1)embedding=self.fc(out)out = self.linear(embedding)return out,embedding#從18層的到101層的 ， 可以根據(jù)自己需要選擇網(wǎng)絡(luò)大小 ， 大的網(wǎng)絡(luò)選用了大的殘差塊 ， #第一個(gè)參數(shù)指明用哪個(gè)殘差塊 ， 第二個(gè)參數(shù)是一個(gè)列表 ， 指明殘差塊的數(shù)量 。def ResNet18():return ResNet(BasicBlock, [2,2,2,2])def ResNet34():return ResNet(BasicBlock, [3,4,6,3])def ResNet50():return ResNet(Bottleneck, [3,4,6,3])def ResNet101():return ResNet(Bottleneck, [3,4,23,3])def ResNet152():return ResNet(Bottleneck, [3,8,36,3])總結(jié)：在使用了ResNet的結(jié)構(gòu)后 ， 可以發(fā)現(xiàn)層數(shù)不斷加深導(dǎo)致的訓(xùn)練集上誤差增大的現(xiàn)象被消除了 ， ResNet網(wǎng)絡(luò)的訓(xùn)練誤差會(huì)隨著層數(shù)增加而逐漸減少 ， 并且在測(cè)試集上的表現(xiàn)也會(huì)變好 。原因在于 ， Resnet學(xué)習(xí)的是殘差函數(shù)F(x) = H(x) – x, 這里如果F(x) = 0, 那么就是上面提到的恒等映射 。事實(shí)上 ， resnet是“shortcut connections”的,在connections是在恒等映射下的特殊情況 ， 學(xué)到的殘差為0時(shí) ， 它沒有引入額外的參數(shù)和計(jì)算復(fù)雜度 ， 且不會(huì)降低精度 。在優(yōu)化目標(biāo)函數(shù)是逼近一個(gè)恒等映射 identity mapping, 而學(xué)習(xí)的殘差不為0時(shí) ，  那么學(xué)習(xí)找到對(duì)恒等映射的擾動(dòng)會(huì)比重新學(xué)習(xí)一個(gè)映射函數(shù)要更容易 。

推薦閱讀