1 回答

TA贡献1839条经验 获得超15个赞
NaN 通常是由学习率过高或优化过程中类似的不稳定性引起的,从而导致梯度爆炸。这也可以通过设置来防止clipnorm。设置具有适当学习率的优化器:
opt = keras.optimizers.Adam(0.001, clipnorm=1.)
model.compile(loss=["mse", "mse"], loss_weights=[0.9, 0.1], optimizer=opt)
可以在笔记本上进行更好的训练:
Epoch 1/20
363/363 [==============================] - 1s 2ms/step - loss: 1547.7197 - main_output_loss: 967.1940 - aux_output_loss: 6772.4609 - val_loss: 19.9807 - val_main_output_loss: 20.0967 - val_aux_output_loss: 18.9365
Epoch 2/20
363/363 [==============================] - 1s 2ms/step - loss: 13.2916 - main_output_loss: 14.0150 - aux_output_loss: 6.7812 - val_loss: 14.6868 - val_main_output_loss: 14.5820 - val_aux_output_loss: 15.6298
Epoch 3/20
363/363 [==============================] - 1s 2ms/step - loss: 11.0539 - main_output_loss: 11.6683 - aux_output_loss: 5.5244 - val_loss: 10.5564 - val_main_output_loss: 10.2116 - val_aux_output_loss: 13.6594
Epoch 4/20
363/363 [==============================] - 1s 1ms/step - loss: 7.4646 - main_output_loss: 7.7688 - aux_output_loss: 4.7269 - val_loss: 13.2672 - val_main_output_loss: 11.5239 - val_aux_output_loss: 28.9570
Epoch 5/20
363/363 [==============================] - 1s 2ms/step - loss: 5.6873 - main_output_loss: 5.8091 - aux_output_loss: 4.5909 - val_loss: 5.0464 - val_main_output_loss: 4.5089 - val_aux_output_loss: 9.8839
它的表现并不令人惊讶,但您必须从这里优化所有超参数才能将其调整到满意的程度。
您还可以按照您最初的预期使用 SGD 来观察 Clipnorm 的效果:
opt = keras.optimizers.SGD(0.001, clipnorm=1.)
model.compile(loss=["mse", "mse"], loss_weights=[0.9, 0.1], optimizer=opt)
这样训练得当。但是,一旦删除clipnorm,您就会得到NaNs。
添加回答
举报