Keras與Tensorflow在訓練上的差異？

Question

大家好，我刚学神经网路不久。我想到一个regression实验，想测试神经网路能不能学到这个特例。这是一个简单的试验，training data是随机产生的10,000笔资料，每笔资料特征向量维度为10x1，label为特征向量第1个元素值。 from numpy.random import RandomState rdm=RandomState(1) data_size=10000 xdim=10 X=rdm.rand(data_size,xdim) Y = [x1[0] for x1 in X] 我用一个一层的网路来训练，预期输出是Weights=[1,0,0,0,0,0,0,0,0,0]，bias=0。 我分别写了tensorflow跟keras两种版本。奇怪的是keras可以得到正确结果，tensorflow的训练却无法收敛。而且两个版本的loss差异很大。 Tensorflow版本: import tensorflow as tf x=tf.placeholder(tf.float64,shape=(None,xdim)) y=tf.placeholder(tf.float64,shape=(None)) Weights = tf.Variable(tf.random_normal([xdim, 1],dtype=tf.float64)) biases = tf.Variable(0.1,dtype=tf.float64) y_predict = tf.matmul(x, Weights) + biases loss = tf.reduce_mean(tf.square(y_predict - y)) optimizer = tf.train.GradientDescentOptimizer(0.01).minimize(loss) batch_size=100 with tf.Session() as sess: sess.run(tf.global_variables_initializer()) for i in range(10001): start = i * batch_size % data_size end = min(start + batch_size,data_size) sess.run(optimizer,feed_dict={x:X[start:end],y:Y[start:end]}) if i % 1000 == 0: ypred,training_loss= sess.run([y_predict,loss],feed_dict={x:X,y:Y}) print("Epoch %d: loss=%g"%(i,training_loss)) Tensorflow版本的输出: Epoch 0: loss=1.0679 Epoch 1000: loss=0.11685 Epoch 2000: loss=0.0842979 Epoch 3000: loss=0.0827121 Epoch 4000: loss=0.0824983 Epoch 5000: loss=0.0824296 Epoch 6000: loss=0.0824021 Epoch 7000: loss=0.0823903 Epoch 8000: loss=0.0823851 Epoch 9000: loss=0.0823826 Epoch 10000: loss=0.0823814 Keras版本: from keras.models import Sequential from keras.layers import Dense import numpy as np model = Sequential() model.add(Dense(units=1, input_dim=xdim)) model.compile(loss='mse', optimizer='sgd') batch_size=100 for i in range(10001): start = i * batch_size % data_size end = min(start + batch_size,data_size) cost = model.train_on_batch(X[start:end], np.array(Y[start:end])) if i % 1000 == 0: print("Epoch %d: loss=%g"%(i,cost)) Keras版本的输出: Epoch 0: loss=0.261707 Epoch 1000: loss=0.00811771 Epoch 2000: loss=0.000325865 Epoch 3000: loss=2.21623e-05 Epoch 4000: loss=4.63907e-06 Epoch 5000: loss=1.66684e-06 Epoch 6000: loss=6.55329e-07 Epoch 7000: loss=2.61024e-07 Epoch 8000: loss=1.04213e-07 Epoch 9000: loss=4.16416e-08 Epoch 10000: loss=1.66369e-08 我认为两段代码应该是等效的，差别是我不知道keras内部如何设定learning rate。为什么keras可以得到正确结果，tensorflow却不行呢？请问我哪边弄错呢？