如何提高深度学习的准确性和验证准确性

Question

我正在用自己的数据训练 CNN，我在相同的数据上尝试了 resnet50 和 resnet101 以及我自己的模型，准确度为 63，验证准确度为 0.08。我知道问题出在我的数据上，我想在拆分数据之前尝试对数据进行洗牌，但我的数据分为 26 个不同的类，如何在将数据拆分为训练集和验证集之前对数据进行洗牌。我的数据集超过 36K 图像。(trainX, testX, trainY, testY) = train_test_split(data, labels,    test_size=0.25, stratify=labels, random_state=42)# initialize the training data augmentation objecttrainAug = ImageDataGenerator(    rotation_range=30,    zoom_range=0.15,    width_shift_range=0.2,    height_shift_range=0.2,    shear_range=0.15,    horizontal_flip=True,    fill_mode="nearest")# initialize the validation/testing data augmentation object (which# we'll be adding mean subtraction to)valAug = ImageDataGenerator()# define the ImageNet mean subtraction (in RGB order) and set the# the mean subtraction value for each of the data augmentation# objectsmean = np.array([123.68, 116.779, 103.939], dtype='float32')trainAug.mean = meanvalAug.mean = meanmodel = Sequential()# The first two layers with 32 filters of window size 3x3model.add(Conv2D(32, (5, 5), padding='same', activation='relu', input_shape=(224, 224, 3)))model.add(Conv2D(32, (5, 5), activation='relu'))model.add(MaxPooling2D(pool_size=(2, 2)))model.add(Dropout(0.25))model.add(Conv2D(64, (5, 5), padding='same', activation='relu'))model.add(Conv2D(64, (5, 5), activation='relu'))model.add(MaxPooling2D(pool_size=(2, 2)))model.add(Dropout(0.25))model.add(Conv2D(128, (3, 3), padding='same', activation='relu'))model.add(Conv2D(128, (3, 3), activation='relu'))model.add(MaxPooling2D(pool_size=(2, 2)))model.add(Dropout(0.25))model.add(Flatten())model.add(Dense(512, activation='relu'))model.add(Dropout(0.5))model.add(Dense(labels, activation='softmax'))

Answer 1

您可以使用 ImageDataGenerator 的验证 split 关键字自动拆分训练数据和测试数据。

train_datagen = ImageDataGenerator(rescale=1./255,

    shear_range=0.2,

    zoom_range=0.2,

    horizontal_flip=True,

    validation_split=0.2) # set validation split

train_generator = train_datagen.flow_from_directory(

    train_data_dir,

    target_size=(img_height, img_width),

    batch_size=batch_size,

    class_mode='binary',

    subset='training') # set as training data

validation_generator = train_datagen.flow_from_directory(

    train_data_dir, # same directory as training data

    target_size=(img_height, img_width),

    batch_size=batch_size,

    class_mode='binary',

    subset='validation') # set as validation data

model.fit_generator(

    train_generator,

    steps_per_epoch = train_generator.samples // batch_size,

    validation_data = validation_generator, 

    validation_steps = validation_generator.samples // batch_size,

    epochs = nb_epochs)

当ImageDataGenerator自动打乱您的输入数据时，您使用ImageDataGenerator的数据会被打乱和分割。

在你的情况下，你需要flow而不是flow_from_directory