加载后更改 MobileNet 丢失

Question

我正在处理迁移学习问题。当我仅从 Mobilenet 创建新模型时，我设置了一个 dropout。base_model = MobileNet(weights='imagenet', include_top=False, input_shape=(200,200,3), dropout=.15)x = base_model.outputx = GlobalAveragePooling2D()(x)x = Dense(10, activation='softmax')(x)我在训练时使用model_checkpoint_callback. 当我训练时，我会发现过度拟合发生的地方，并调整冻结层的数量和学习率。当我再次保存加载的模型时，我是否也可以调整 dropout？我看到了这个答案，但是 Mobilenet 中没有实际的 dropout 层，所以这个for layer in model.layers:    if hasattr(layer, 'rate'):        print(layer.name)        layer.rate = 0.5什么都不做。

Answer 1

过去，您必须克隆模型才能让新的 dropout 接受。我最近没试过。

# This code allows you to change the dropout

# Load model from .json

model.load_weights(filenameToModelWeights) # Load weights

model.layers[-2].rate = 0.04  # layer[-2] is my dropout layer, rate is dropout attribute

model = keras.models.clone(model) # If I do not clone, the new rate is never used. Weights are re-init now.

model.load_weights(filenameToModelWeights) # Load weights

model.predict(x)

归功于

http://www.gergltd.com/home/2018/03/changing-dropout-on-the-fly-during-training-time-test-time-in-keras/

如果模型一开始就没有 dropout 层，就像 Keras 的预训练移动网络一样，您必须使用方法添加它们。这是您可以做到的一种方法。

用于添加单层

def insert_single_layer_in_keras(model, layer_name, new_layer):

    layers = [l for l in model.layers]

    x = layers[0].output

    for i in range(1, len(layers)):

        x = layers[i](x)

        # add layer afterward

        if layers[i].name == layer_name:

            x = new_layer(x)

    new_model = Model(inputs=layers[0].input, outputs=x)

    return new_model

用于系统地添加层

def insert_layers_in_model(model, layer_common_name, new_layer):

    import re

    layers = [l for l in model.layers]

    x = layers[0].output

    layer_config = new_layer.get_config()

    base_name = layer_config['name']

    layer_class = type(dropout_layer)

    for i in range(1, len(layers)):

        x = layers[i](x)

        match = re.match(".+" + layer_common_name + "+", layers[i].name)

        # add layer afterward

        if match:

            layer_config['name'] = base_name + "_" + str(i)  # no duplicate names, could be done different

            layer_copy = layer_class.from_config(layer_config)

            x = layer_copy(x)

    new_model = Model(inputs=layers[0].input, outputs=x)

    return new_model

像这样跑

import tensorflow as tf

from tensorflow.keras.applications.mobilenet import MobileNet

from tensorflow.keras.layers import Dropout

from tensorflow.keras.models import Model

base_model = MobileNet(weights='imagenet', include_top=False, input_shape=(192, 192, 3), dropout=.15)

dropout_layer = Dropout(0.5)

# add single layer after last dropout

mobile_net_with_dropout = insert_single_layer_in_model(base_model, "conv_pw_13_bn", dropout_layer)

# systematically add layers after any batchnorm layer

mobile_net_with_multi_dropout = insert_layers_in_model(base_model, "bn", dropout_layer)

顺便说一句，您绝对应该进行实验，但您不太可能希望在 batchnorm 之上对像 mobilenet 这样的小型网络进行额外的正则化。