用TensorFlow實現ResNeXt和DenseNet，超簡單！

GIF/1.7M

圖：pixabay

原文來源：GitHub

「機器人圈」編譯：嗯~阿童木呀、多啦A亮

ResNeXt-Tensorflow

使用Cifar10數據集的ResNeXt在Tensorflow上的實現。

如果你想查看原作者的代碼，請參考此鏈接https://github.com/facebookresearch/ResNeXt

要求

? Tensorflow 1.x

? Python 3.x

? tflearn（如果你覺得全局平均池比較容易，你應該安裝tflearn）

架構比較

ResNet

ResNeXt

? 我實施了（b）

? （b）是分割 + 變換（Bottleneck）+ 拼接 + 轉換 + 合併

問題

什麼是「分割」？

def split_layer(self, input_x, stride, layer_name):

with tf.name_scope(layer_name) :

layers_split = list()

for i in range(cardinality) :

splits = self.transform_layer(input_x, stride=stride, scope=layer_name + '_splitN_' + str(i))

layers_split.append(splits)

return Concatenation(layers_split)

? 基數是指你想要分割多少次

什麼是「變換」？

def transform_layer(self, x, stride, scope):

with tf.name_scope(scope) :

x = conv_layer(x, filter=depth, kernel=[1,1], stride=stride, layer_name=scope+'_conv1')

x = Batch_Normalization(x, training=self.training, scope=scope+'_batch1')

x = Relu(x)

x = conv_layer(x, filter=depth, kernel=[3,3], stride=1, layer_name=scope+'_conv2')

x = Batch_Normalization(x, training=self.training, scope=scope+'_batch2')

x = Relu(x)

return x

什麼是「轉換」？

def transition_layer(self, x, out_dim, scope):

with tf.name_scope(scope):

x = conv_layer(x, filter=out_dim, kernel=[1,1], stride=1, layer_name=scope+'_conv1')

x = Batch_Normalization(x, training=self.training, scope=scope+'_batch1')

return x

結果比較（ResNet、DenseNet、ResNet）

相關作品

? DenseNet （https://github.com/taki0112/Densenet-Tensorflow）

參考

分類數據集結果（http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html）

Densenet-Tensorflow

使用Cifar10、MNIST數據集的Densenet（https://arxiv.org/abs/1608.06993）在Tensorflow上的實現。

? 實現本論文的代碼是Densenet.py。

? 有一些細微的區別是，我用的是AdamOptimizer。

如果你想查看原作者的代碼或其他實現，請參考此鏈接（https://github.com/liuzhuang13/DenseNet）

要求

? Tensorflow 1.x

? Python 3.x

? tflearn（如果你對全局平均池（global average pooling）能夠很好地掌握，那你就應該安裝tflearn。

但是，我使用的是tf.layers將它實現的，因此你也不必過分擔心。

問題

defBatch_Normalization(x, training, scope):

witharg_scope([batch_norm],

scope=scope,

updates_collections=None,

decay=0.9,

center=True,

scale=True,

zero_debias_moving_mean=True) :

returntf.cond(training,

lambda :batch_norm(inputs=x, is_training=training, reuse=None),

lambda :batch_norm(inputs=x, is_training=training, reuse=True))

理念

什麼是全局平均池（global average pooling）？

defGlobal_Average_Pooling(x, stride=1) :

width=np.shape(x)[1]

height=np.shape(x)[2]

pool_size= [width, height]

# The stride value does not matter

如果你使用的是tflearn，請參考此鏈接（http://tflearn.org/layers/conv/#global-average-pooling）

defGlobal_Average_Pooling(x):

什麼是密集連接（Dense Connectivity）？

什麼是Densenet架構？

池化減少了特徵映射的大小

特徵映射大小在每個塊內匹配

defDense_net(self, input_x):

x =conv_layer(input_x, filter=2*self.filters, kernel=[7,7], stride=2, layer_name='conv0')

x =Max_Pooling(x, pool_size=[3,3], stride=2)

x =self.dense_block(input_x=x, nb_layers=6, layer_name='dense_1')

x =self.transition_layer(x, scope='trans_1')

x =self.dense_block(input_x=x, nb_layers=12, layer_name='dense_2')

x =self.transition_layer(x, scope='trans_2')

x =self.dense_block(input_x=x, nb_layers=48, layer_name='dense_3')

x =self.transition_layer(x, scope='trans_3')

x =self.dense_block(input_x=x, nb_layers=32, layer_name='dense_final')

x =Batch_Normalization(x, training=self.training, scope='linear_batch')

x =Relu(x)

x =Global_Average_Pooling(x)

x = Linear(x)

return x

什麼是密集塊（Dense Block）？

defdense_block(self, input_x, nb_layers, layer_name):

withtf.name_scope(layer_name):

layers_concat=list()

layers_concat.append(input_x)

x =self.bottleneck_layer(input_x, scope=layer_name+'_bottleN_'+str(0))

layers_concat.append(x)

for i inrange(nb_layers-1):

x =Concatenation(layers_concat)

x =self.bottleneck_layer(x, scope=layer_name+'_bottleN_'+str(i +1))

layers_concat.append(x)

return x

什麼是Bottleneck 層？

defbottleneck_layer(self, x, scope):

withtf.name_scope(scope):

x =Batch_Normalization(x, training=self.training, scope=scope+'_batch1')

x =Relu(x)

x =conv_layer(x, filter=4*self.filters, kernel=[1,1], layer_name=scope+'_conv1')

x =Drop_out(x, rate=dropout_rate, training=self.training)

x =Batch_Normalization(x, training=self.training, scope=scope+'_batch2')

x =Relu(x)

x =conv_layer(x, filter=self.filters, kernel=[3,3], layer_name=scope+'_conv2')

x =Drop_out(x, rate=dropout_rate, training=self.training)

return x

什麼是轉換層（Transition Layer）？

deftransition_layer(self, x, scope):

withtf.name_scope(scope):

x =Batch_Normalization(x, training=self.training, scope=scope+'_batch1')

x =Relu(x)

x =conv_layer(x, filter=self.filters, kernel=[1,1], layer_name=scope+'_conv1')

x =Drop_out(x, rate=dropout_rate, training=self.training)

x =Average_pooling(x, pool_size=[2,2], stride=2)

return x

結構比較（CNN、ResNet、DenseNet）

結果

? （MNIST）最高測試精度為99.2％（該結果不使用dropout）。

? 密集塊層數固定為4個。

for i inrange(self.nb_blocks) :

# original : 6 -> 12 -> 48

x =self.dense_block(input_x=x, nb_layers=4, layer_name='dense_'+str(i))

x =self.transition_layer(x, scope='trans_'+str(i))

CIFAR-10

CIFAR-100

Image Net

喜歡這篇文章嗎？立刻分享出去讓更多人知道吧！