自适应参数化ReLU是一种动态激活函数，对所有输入不是“一视同仁”，在2019年5月3日投稿至IEEE Transactions on Industrial Electronics，2020年1月24日录用，2020年2月13日在IEEE官网公布。
本文在调参记录21的基础上，增加卷积核的个数，也就是增加深度神经网络的宽度，继续尝试深度残差网络+自适应参数化ReLU激活函数在Cifar10数据集上的效果。
自适应参数化ReLU激活函数的原理如下：
Keras程序：
#!/usr/bin/env python3# -*- coding: utf-8 -*-"""Created on Tue Apr 14 04:17:45 2020Implemented using TensorFlow 1.0.1 and Keras 2.2.1Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht,Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, IEEE Transactions on Industrial Electronics, 2020, DOI: 10.1109/TIE.2020.2972458Date of Publication: 13 February 2020@author: Minghang Zhao"""from __future__ import print_functionimport kerasimport numpy as npfrom keras.datasets import cifar10from keras.layers import Dense, Conv2D, BatchNormalization, Activation, Minimumfrom keras.layers import AveragePooling2D, Input, GlobalAveragePooling2D, Concatenate, Reshapefrom keras.regularizers import l2from keras import backend as Kfrom keras.models import Modelfrom keras import optimizersfrom keras.preprocessing.image import ImageDataGeneratorfrom keras.callbacks import LearningRateSchedulerK.set_learning_phase(1)# The data, split between train and test sets(x_train, y_train), (x_test, y_test) = cifar10.load_data()x_train = x_train.astype('float32') / 255.x_test = x_test.astype('float32') / 255.x_test = x_test-np.mean(x_train)x_train = x_train-np.mean(x_train)print('x_train shape:', x_train.shape)print(x_train.shape[0], 'train samples')print(x_test.shape[0], 'test samples')# convert class vectors to binary class matricesy_train = keras.utils.to_categorical(y_train, 10)y_test = keras.utils.to_categorical(y_test, 10)# Schedule the learning rate, multiply 0.1 every 150 epochesdef scheduler(epoch): if epoch % 150 == 0 and epoch != 0: lr = K.get_value(model.optimizer.lr) K.set_value(model.optimizer.lr, lr * 0.1) print("lr changed to {}".format(lr * 0.1)) return K.get_value(model.optimizer.lr)# An adaptively parametric rectifier linear unit (APReLU)def aprelu(inputs): # get the number of channels channels = inputs.get_shape().as_list()[-1] # get a zero feature map zeros_input = keras.layers.subtract([inputs, inputs]) # get a feature map with only positive features pos_input = Activation('relu')(inputs) # get a feature map with only negative features neg_input = Minimum()([inputs,zeros_input]) # define a network to obtain the scaling coefficients scales_p = GlobalAveragePooling2D()(pos_input) scales_n = GlobalAveragePooling2D()(neg_input) scales = Concatenate()([scales_n, scales_p]) scales = Dense(channels//16, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales) scales = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(scales) scales = Activation('relu')(scales) scales = Dense(channels, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales) scales = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(scales) scales = Activation('sigmoid')(scales) scales = Reshape((1,1,channels))(scales) # apply a paramtetric relu neg_part = keras.layers.multiply([scales, neg_input]) return keras.layers.add([pos_input, neg_part])# Residual Blockdef residual_block(incoming, nb_blocks, out_channels, downsample=False, downsample_strides=2): residual = incoming in_channels = incoming.get_shape().as_list()[-1] for i in range(nb_blocks): identity = residual if not downsample: downsample_strides = 1 residual = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(residual) residual = Activation('relu')(residual) residual = Conv2D(out_channels, 3, strides=(downsample_strides, downsample_strides), padding='same', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(residual) residual = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(residual) residual = Activation('relu')(residual) residual = Conv2D(out_channels, 3, padding='same', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(residual) residual = aprelu(residual) # Downsampling if downsample_strides > 1: identity = AveragePooling2D(pool_size=(1,1), strides=(2,2))(identity) # Zero_padding to match channels if in_channels != out_channels: zeros_identity = keras.layers.subtract([identity, identity]) identity = keras.layers.concatenate([identity, zeros_identity]) in_channels = out_channels residual = keras.layers.add([residual, identity]) return residual# define and train a modelinputs = Input(shape=(32, 32, 3))net = Conv2D(32, 3, padding='same', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(inputs)net = residual_block(net, 20, 32, downsample=False)net = residual_block(net, 1, 64, downsample=True)net = residual_block(net, 19, 64, downsample=False)net = residual_block(net, 1, 128, downsample=True)net = residual_block(net, 19,128, downsample=False)net = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(net)net = Activation('relu')(net)net = GlobalAveragePooling2D()(net)outputs = Dense(10, activation='softmax', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(net)model = Model(inputs=inputs, outputs=outputs)sgd = optimizers.SGD(lr=0.1, decay=0., momentum=0.9, nesterov=True)model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])# data augmentationdatagen = ImageDataGenerator( # randomly rotate images in the range (deg 0 to 180) rotation_range=30, # Range for random zoom zoom_range = 0.2, # shear angle in counter-clockwise direction in degrees shear_range = 30, # randomly flip images horizontal_flip=True, # randomly shift images horizontally width_shift_range=0.125, # randomly shift images vertically height_shift_range=0.125)reduce_lr = LearningRateScheduler(scheduler)# fit the model on the batches generated by datagen.flow().model.fit_generator(datagen.flow(x_train, y_train, batch_size=100), validation_data=(x_test, y_test), epochs=500, verbose=1, callbacks=[reduce_lr], workers=4)# get resultsK.set_learning_phase(0)DRSN_train_score = model.evaluate(x_train, y_train, batch_size=100, verbose=0)print('Train loss:', DRSN_train_score[0])print('Train accuracy:', DRSN_train_score[1])DRSN_test_score = model.evaluate(x_test, y_test, batch_size=100, verbose=0)print('Test loss:', DRSN_test_score[0])print('Test accuracy:', DRSN_test_score[1])实验结果如下：
Using TensorFlow backend.x_train shape: (50000, 32, 32, 3)50000 train samples10000 test samplesEpoch 1/500177s 354ms/step - loss: 5.4649 - acc: 0.3787 - val_loss: 4.7056 - val_acc: 0.5364Epoch 2/500134s 268ms/step - loss: 4.3931 - acc: 0.5448 - val_loss: 3.8649 - val_acc: 0.6462Epoch 3/500134s 267ms/step - loss: 3.6521 - acc: 0.6280 - val_loss: 3.2183 - val_acc: 0.7019Epoch 4/500134s 267ms/step - loss: 3.0934 - acc: 0.6764 - val_loss: 2.7007 - val_acc: 0.7536Epoch 5/500134s 267ms/step - loss: 2.6582 - acc: 0.7114 - val_loss: 2.3136 - val_acc: 0.7880Epoch 6/500134s 267ms/step - loss: 2.3051 - acc: 0.7361 - val_loss: 2.0292 - val_acc: 0.7951Epoch 7/500134s 267ms/step - loss: 2.0207 - acc: 0.7557 - val_loss: 1.7687 - val_acc: 0.8134Epoch 8/500134s 267ms/step - loss: 1.7859 - acc: 0.7732 - val_loss: 1.5536 - val_acc: 0.8292Epoch 9/500134s 267ms/step - loss: 1.6043 - acc: 0.7845 - val_loss: 1.3916 - val_acc: 0.8385Epoch 10/500134s 267ms/step - loss: 1.4540 - acc: 0.7942 - val_loss: 1.2741 - val_acc: 0.8426Epoch 11/500134s 267ms/step - loss: 1.3272 - acc: 0.8037 - val_loss: 1.1760 - val_acc: 0.8461Epoch 12/500133s 267ms/step - loss: 1.2285 - acc: 0.8110 - val_loss: 1.0927 - val_acc: 0.8514Epoch 13/500133s 267ms/step - loss: 1.1442 - acc: 0.8174 - val_loss: 0.9914 - val_acc: 0.8639Epoch 14/500133s 267ms/step - loss: 1.0733 - acc: 0.8252 - val_loss: 0.9532 - val_acc: 0.8634Epoch 15/500133s 267ms/step - loss: 1.0186 - acc: 0.8297 - val_loss: 0.9055 - val_acc: 0.8638Epoch 16/500133s 267ms/step - loss: 0.9759 - acc: 0.8329 - val_loss: 0.8713 - val_acc: 0.8664Epoch 17/500133s 267ms/step - loss: 0.9408 - acc: 0.8357 - val_loss: 0.8173 - val_acc: 0.8798Epoch 18/500133s 267ms/step - loss: 0.9071 - acc: 0.8412 - val_loss: 0.8102 - val_acc: 0.8730Epoch 19/500133s 267ms/step - loss: 0.8804 - acc: 0.8437 - val_loss: 0.7703 - val_acc: 0.8819Epoch 20/500133s 267ms/step - loss: 0.8613 - acc: 0.8478 - val_loss: 0.7809 - val_acc: 0.8739Epoch 21/500133s 267ms/step - loss: 0.8408 - acc: 0.8498 - val_loss: 0.7378 - val_acc: 0.8870Epoch 22/500133s 267ms/step - loss: 0.8260 - acc: 0.8515 - val_loss: 0.7501 - val_acc: 0.8812Epoch 23/500133s 267ms/step - loss: 0.8093 - acc: 0.8561 - val_loss: 0.7612 - val_acc: 0.8719Epoch 24/500134s 267ms/step - loss: 0.8009 - acc: 0.8579 - val_loss: 0.7348 - val_acc: 0.8814Epoch 25/500133s 267ms/step - loss: 0.7908 - acc: 0.8585 - val_loss: 0.7542 - val_acc: 0.8741Epoch 26/500133s 266ms/step - loss: 0.7787 - acc: 0.8636 - val_loss: 0.7407 - val_acc: 0.8771Epoch 27/500133s 266ms/step - loss: 0.7765 - acc: 0.8622 - val_loss: 0.6996 - val_acc: 0.8934Epoch 28/500133s 266ms/step - loss: 0.7662 - acc: 0.8658 - val_loss: 0.7118 - val_acc: 0.8855Epoch 29/500133s 266ms/step - loss: 0.7623 - acc: 0.8661 - val_loss: 0.7267 - val_acc: 0.8808Epoch 30/500133s 266ms/step - loss: 0.7654 - acc: 0.8652 - val_loss: 0.7112 - val_acc: 0.8846Epoch 31/500133s 266ms/step - loss: 0.7575 - acc: 0.8675 - val_loss: 0.6885 - val_acc: 0.8944Epoch 32/500133s 267ms/step - loss: 0.7513 - acc: 0.8691 - val_loss: 0.6925 - val_acc: 0.8930Epoch 33/500133s 267ms/step - loss: 0.7455 - acc: 0.8724 - val_loss: 0.6935 - val_acc: 0.8910Epoch 34/500133s 267ms/step - loss: 0.7411 - acc: 0.8722 - val_loss: 0.6856 - val_acc: 0.8938Epoch 35/500133s 267ms/step - loss: 0.7418 - acc: 0.8729 - val_loss: 0.7001 - val_acc: 0.8881Epoch 36/500133s 267ms/step - loss: 0.7354 - acc: 0.8739 - val_loss: 0.6869 - val_acc: 0.8895Epoch 37/500133s 266ms/step - loss: 0.7337 - acc: 0.8767 - val_loss: 0.6840 - val_acc: 0.8962Epoch 38/500133s 266ms/step - loss: 0.7360 - acc: 0.8765 - val_loss: 0.6967 - val_acc: 0.8914Epoch 39/500133s 267ms/step - loss: 0.7316 - acc: 0.8780 - val_loss: 0.6687 - val_acc: 0.8993Epoch 40/500133s 267ms/step - loss: 0.7253 - acc: 0.8811 - val_loss: 0.6886 - val_acc: 0.8949Epoch 41/500133s 267ms/step - loss: 0.7240 - acc: 0.8809 - val_loss: 0.7086 - val_acc: 0.8887Epoch 42/500133s 267ms/step - loss: 0.7247 - acc: 0.8802 - val_loss: 0.6879 - val_acc: 0.8944Epoch 43/500133s 267ms/step - loss: 0.7266 - acc: 0.8794 - val_loss: 0.6762 - val_acc: 0.9009Epoch 44/500133s 266ms/step - loss: 0.7206 - acc: 0.8820 - val_loss: 0.7067 - val_acc: 0.8874Epoch 45/500133s 266ms/step - loss: 0.7233 - acc: 0.8823 - val_loss: 0.6840 - val_acc: 0.8944Epoch 46/500133s 266ms/step - loss: 0.7163 - acc: 0.8839 - val_loss: 0.6924 - val_acc: 0.8926Epoch 47/500133s 266ms/step - loss: 0.7189 - acc: 0.8842 - val_loss: 0.6761 - val_acc: 0.8982Epoch 48/500133s 266ms/step - loss: 0.7137 - acc: 0.8841 - val_loss: 0.7079 - val_acc: 0.8931Epoch 49/500133s 266ms/step - loss: 0.7139 - acc: 0.8851 - val_loss: 0.6882 - val_acc: 0.8954Epoch 50/500133s 266ms/step - loss: 0.7129 - acc: 0.8859 - val_loss: 0.6681 - val_acc: 0.9011Epoch 51/500133s 266ms/step - loss: 0.7157 - acc: 0.8838 - val_loss: 0.6726 - val_acc: 0.9000Epoch 52/500133s 266ms/step - loss: 0.7108 - acc: 0.8858 - val_loss: 0.6720 - val_acc: 0.9002Epoch 53/500133s 266ms/step - loss: 0.7137 - acc: 0.8866 - val_loss: 0.6790 - val_acc: 0.8982Epoch 54/500133s 266ms/step - loss: 0.7151 - acc: 0.8859 - val_loss: 0.6823 - val_acc: 0.8998Epoch 55/500133s 266ms/step - loss: 0.7139 - acc: 0.8870 - val_loss: 0.7120 - val_acc: 0.8894Epoch 56/500133s 266ms/step - loss: 0.7093 - acc: 0.8884 - val_loss: 0.6790 - val_acc: 0.9013Epoch 57/500133s 266ms/step - loss: 0.7113 - acc: 0.8880 - val_loss: 0.6772 - val_acc: 0.9038Epoch 58/500133s 266ms/step - loss: 0.7042 - acc: 0.8908 - val_loss: 0.6758 - val_acc: 0.9042Epoch 59/500133s 266ms/step - loss: 0.7107 - acc: 0.8881 - val_loss: 0.6771 - val_acc: 0.9001Epoch 60/500133s 266ms/step - loss: 0.7082 - acc: 0.8878 - val_loss: 0.6848 - val_acc: 0.8998Epoch 61/500133s 266ms/step - loss: 0.7039 - acc: 0.8920 - val_loss: 0.6842 - val_acc: 0.9002Epoch 62/500133s 266ms/step - loss: 0.7049 - acc: 0.8908 - val_loss: 0.6577 - val_acc: 0.9076Epoch 63/500133s 265ms/step - loss: 0.7005 - acc: 0.8914 - val_loss: 0.6904 - val_acc: 0.8962Epoch 64/500133s 266ms/step - loss: 0.7042 - acc: 0.8916 - val_loss: 0.7025 - val_acc: 0.8910Epoch 65/500133s 266ms/step - loss: 0.7037 - acc: 0.8904 - val_loss: 0.6811 - val_acc: 0.9038Epoch 66/500133s 266ms/step - loss: 0.7085 - acc: 0.8908 - val_loss: 0.7166 - val_acc: 0.8915Epoch 67/500133s 265ms/step - loss: 0.6981 - acc: 0.8939 - val_loss: 0.6934 - val_acc: 0.8978Epoch 68/500133s 266ms/step - loss: 0.7087 - acc: 0.8917 - val_loss: 0.6868 - val_acc: 0.9026Epoch 69/500133s 266ms/step - loss: 0.6994 - acc: 0.8932 - val_loss: 0.6792 - val_acc: 0.9016Epoch 70/500133s 266ms/step - loss: 0.7040 - acc: 0.8931 - val_loss: 0.6695 - val_acc: 0.9042Epoch 71/500133s 266ms/step - loss: 0.7022 - acc: 0.8933 - val_loss: 0.6771 - val_acc: 0.9039Epoch 72/500133s 266ms/step - loss: 0.6975 - acc: 0.8954 - val_loss: 0.6789 - val_acc: 0.9043Epoch 73/500133s 266ms/step - loss: 0.6935 - acc: 0.8953 - val_loss: 0.6664 - val_acc: 0.9070Epoch 74/500133s 266ms/step - loss: 0.6956 - acc: 0.8943 - val_loss: 0.6633 - val_acc: 0.9124Epoch 75/500133s 266ms/step - loss: 0.6966 - acc: 0.8934 - val_loss: 0.6719 - val_acc: 0.9057Epoch 76/500133s 266ms/step - loss: 0.7008 - acc: 0.8942 - val_loss: 0.6872 - val_acc: 0.8993Epoch 77/500133s 266ms/step - loss: 0.6923 - acc: 0.8950 - val_loss: 0.6961 - val_acc: 0.9007Epoch 78/500133s 266ms/step - loss: 0.6966 - acc: 0.8951 - val_loss: 0.6771 - val_acc: 0.9010Epoch 79/500133s 266ms/step - loss: 0.6988 - acc: 0.8952 - val_loss: 0.6752 - val_acc: 0.9046Epoch 80/500133s 266ms/step - loss: 0.6946 - acc: 0.8970 - val_loss: 0.6716 - val_acc: 0.9073Epoch 81/500133s 266ms/step - loss: 0.6979 - acc: 0.8950 - val_loss: 0.6785 - val_acc: 0.9049Epoch 82/500133s 266ms/step - loss: 0.6956 - acc: 0.8968 - val_loss: 0.6916 - val_acc: 0.8987Epoch 83/500133s 266ms/step - loss: 0.6946 - acc: 0.8964 - val_loss: 0.6816 - val_acc: 0.9054Epoch 84/500133s 266ms/step - loss: 0.6921 - acc: 0.8972 - val_loss: 0.6834 - val_acc: 0.9044Epoch 85/500133s 265ms/step - loss: 0.6909 - acc: 0.8983 - val_loss: 0.6983 - val_acc: 0.8966Epoch 86/500133s 266ms/step - loss: 0.6991 - acc: 0.8959 - val_loss: 0.6677 - val_acc: 0.9096Epoch 87/500133s 266ms/step - loss: 0.6932 - acc: 0.8996 - val_loss: 0.6768 - val_acc: 0.9078Epoch 88/500133s 266ms/step - loss: 0.6961 - acc: 0.8974 - val_loss: 0.6895 - val_acc: 0.9016Epoch 89/500133s 266ms/step - loss: 0.6919 - acc: 0.9001 - val_loss: 0.6846 - val_acc: 0.9060Epoch 90/500133s 267ms/step - loss: 0.6937 - acc: 0.8986 - val_loss: 0.6677 - val_acc: 0.9106Epoch 91/500134s 268ms/step - loss: 0.6880 - acc: 0.9007 - val_loss: 0.6800 - val_acc: 0.9038Epoch 92/500134s 268ms/step - loss: 0.6910 - acc: 0.8982 - val_loss: 0.6843 - val_acc: 0.9035Epoch 93/500134s 268ms/step - loss: 0.6888 - acc: 0.8995 - val_loss: 0.7000 - val_acc: 0.8988Epoch 94/500134s 268ms/step - loss: 0.6865 - acc: 0.8998 - val_loss: 0.6852 - val_acc: 0.9047Epoch 95/500134s 268ms/step - loss: 0.6970 - acc: 0.8963 - val_loss: 0.7136 - val_acc: 0.8964Epoch 96/500134s 268ms/step - loss: 0.6883 - acc: 0.9005 - val_loss: 0.6620 - val_acc: 0.9128Epoch 97/500134s 268ms/step - loss: 0.6923 - acc: 0.8986 - val_loss: 0.6725 - val_acc: 0.9088Epoch 98/500134s 268ms/step - loss: 0.6889 - acc: 0.9005 - val_loss: 0.6813 - val_acc: 0.9058Epoch 99/500134s 268ms/step - loss: 0.6915 - acc: 0.8992 - val_loss: 0.6781 - val_acc: 0.9048Epoch 100/500134s 268ms/step - loss: 0.6876 - acc: 0.9011 - val_loss: 0.6740 - val_acc: 0.9062Epoch 101/500134s 268ms/step - loss: 0.6886 - acc: 0.9015 - val_loss: 0.6744 - val_acc: 0.9074Epoch 102/500134s 268ms/step - loss: 0.6904 - acc: 0.8995 - val_loss: 0.6853 - val_acc: 0.9028Epoch 103/500134s 268ms/step - loss: 0.6860 - acc: 0.9018 - val_loss: 0.6714 - val_acc: 0.9111Epoch 104/500134s 268ms/step - loss: 0.6921 - acc: 0.8997 - val_loss: 0.6827 - val_acc: 0.9026Epoch 105/500134s 268ms/step - loss: 0.6849 - acc: 0.9013 - val_loss: 0.7103 - val_acc: 0.8968Epoch 106/500134s 268ms/step - loss: 0.6896 - acc: 0.9010 - val_loss: 0.6898 - val_acc: 0.9017Epoch 107/500134s 268ms/step - loss: 0.6874 - acc: 0.9018 - val_loss: 0.6835 - val_acc: 0.9019Epoch 108/500134s 268ms/step - loss: 0.6879 - acc: 0.9017 - val_loss: 0.6864 - val_acc: 0.9057Epoch 109/500134s 268ms/step - loss: 0.6900 - acc: 0.9024 - val_loss: 0.6853 - val_acc: 0.9032Epoch 110/500134s 268ms/step - loss: 0.6811 - acc: 0.9038 - val_loss: 0.6793 - val_acc: 0.9062Epoch 111/500134s 268ms/step - loss: 0.6848 - acc: 0.9011 - val_loss: 0.6824 - val_acc: 0.9069Epoch 112/500134s 268ms/step - loss: 0.6864 - acc: 0.9025 - val_loss: 0.6786 - val_acc: 0.9075Epoch 113/500134s 268ms/step - loss: 0.6831 - acc: 0.9023 - val_loss: 0.6813 - val_acc: 0.9018Epoch 114/500134s 268ms/step - loss: 0.6832 - acc: 0.9033 - val_loss: 0.6756 - val_acc: 0.9078Epoch 115/500134s 268ms/step - loss: 0.6813 - acc: 0.9049 - val_loss: 0.6847 - val_acc: 0.9030Epoch 116/500134s 268ms/step - loss: 0.6899 - acc: 0.8999 - val_loss: 0.6872 - val_acc: 0.9067Epoch 117/500134s 268ms/step - loss: 0.6816 - acc: 0.9038 - val_loss: 0.6873 - val_acc: 0.9084Epoch 118/500134s 268ms/step - loss: 0.6832 - acc: 0.9025 - val_loss: 0.6646 - val_acc: 0.9142Epoch 119/500134s 268ms/step - loss: 0.6754 - acc: 0.9053 - val_loss: 0.6790 - val_acc: 0.9053Epoch 120/500134s 268ms/step - loss: 0.6800 - acc: 0.9050 - val_loss: 0.6888 - val_acc: 0.9062Epoch 121/500134s 268ms/step - loss: 0.6821 - acc: 0.9043 - val_loss: 0.6804 - val_acc: 0.9076Epoch 122/500134s 268ms/step - loss: 0.6821 - acc: 0.9047 - val_loss: 0.6873 - val_acc: 0.9074Epoch 123/500134s 268ms/step - loss: 0.6862 - acc: 0.9017 - val_loss: 0.6817 - val_acc: 0.9061Epoch 124/500134s 267ms/step - loss: 0.6827 - acc: 0.9034 - val_loss: 0.6852 - val_acc: 0.9070Epoch 125/500133s 266ms/step - loss: 0.6801 - acc: 0.9050 - val_loss: 0.6793 - val_acc: 0.9080Epoch 126/500133s 266ms/step - loss: 0.6857 - acc: 0.9038 - val_loss: 0.6788 - val_acc: 0.9059Epoch 127/500133s 266ms/step - loss: 0.6817 - acc: 0.9042 - val_loss: 0.6804 - val_acc: 0.9065Epoch 128/500133s 266ms/step - loss: 0.6851 - acc: 0.9036 - val_loss: 0.7013 - val_acc: 0.9027Epoch 129/500133s 266ms/step - loss: 0.6850 - acc: 0.9024 - val_loss: 0.6965 - val_acc: 0.9042Epoch 130/500133s 266ms/step - loss: 0.6846 - acc: 0.9050 - val_loss: 0.6797 - val_acc: 0.9104Epoch 131/500133s 266ms/step - loss: 0.6814 - acc: 0.9058 - val_loss: 0.6740 - val_acc: 0.9107Epoch 132/500133s 266ms/step - loss: 0.6835 - acc: 0.9044 - val_loss: 0.7089 - val_acc: 0.8962Epoch 133/500133s 266ms/step - loss: 0.6808 - acc: 0.9066 - val_loss: 0.6767 - val_acc: 0.9105Epoch 134/500133s 265ms/step - loss: 0.6847 - acc: 0.9035 - val_loss: 0.6932 - val_acc: 0.9055Epoch 135/500133s 266ms/step - loss: 0.6832 - acc: 0.9058 - val_loss: 0.6916 - val_acc: 0.9058Epoch 136/500133s 265ms/step - loss: 0.6801 - acc: 0.9041 - val_loss: 0.6851 - val_acc: 0.9073Epoch 137/500133s 266ms/step - loss: 0.6809 - acc: 0.9056 - val_loss: 0.6726 - val_acc: 0.9108Epoch 138/500133s 266ms/step - loss: 0.6813 - acc: 0.9053 - val_loss: 0.6590 - val_acc: 0.9143Epoch 139/500133s 266ms/step - loss: 0.6814 - acc: 0.9057 - val_loss: 0.6746 - val_acc: 0.9085Epoch 140/500133s 266ms/step - loss: 0.6804 - acc: 0.9060 - val_loss: 0.6839 - val_acc: 0.9068Epoch 141/500133s 266ms/step - loss: 0.6810 - acc: 0.9065 - val_loss: 0.7121 - val_acc: 0.8978Epoch 142/500133s 266ms/step - loss: 0.6831 - acc: 0.9054 - val_loss: 0.6893 - val_acc: 0.9067Epoch 143/500133s 266ms/step - loss: 0.6785 - acc: 0.9069 - val_loss: 0.6754 - val_acc: 0.9105Epoch 144/500133s 266ms/step - loss: 0.6810 - acc: 0.9049 - val_loss: 0.6889 - val_acc: 0.9064Epoch 145/500133s 266ms/step - loss: 0.6807 - acc: 0.9074 - val_loss: 0.7067 - val_acc: 0.9023Epoch 146/500133s 266ms/step - loss: 0.6845 - acc: 0.9057 - val_loss: 0.6855 - val_acc: 0.9055Epoch 147/500133s 267ms/step - loss: 0.6779 - acc: 0.9055 - val_loss: 0.6928 - val_acc: 0.9040Epoch 148/500134s 269ms/step - loss: 0.6781 - acc: 0.9069 - val_loss: 0.6760 - val_acc: 0.9086Epoch 149/500133s 267ms/step - loss: 0.6834 - acc: 0.9064 - val_loss: 0.6991 - val_acc: 0.9012Epoch 150/500135s 270ms/step - loss: 0.6809 - acc: 0.9071 - val_loss: 0.6887 - val_acc: 0.9069Epoch 151/500lr changed to 0.010000000149011612133s 267ms/step - loss: 0.5790 - acc: 0.9415 - val_loss: 0.5901 - val_acc: 0.9381Epoch 152/500134s 267ms/step - loss: 0.5211 - acc: 0.9595 - val_loss: 0.5735 - val_acc: 0.9413Epoch 153/500133s 267ms/step - loss: 0.4983 - acc: 0.9645 - val_loss: 0.5626 - val_acc: 0.9440Epoch 154/500134s 267ms/step - loss: 0.4793 - acc: 0.9686 - val_loss: 0.5532 - val_acc: 0.9434Epoch 155/500135s 269ms/step - loss: 0.4689 - acc: 0.9709 - val_loss: 0.5510 - val_acc: 0.9434Epoch 156/500133s 267ms/step - loss: 0.4579 - acc: 0.9716 - val_loss: 0.5398 - val_acc: 0.9444Epoch 157/500133s 266ms/step - loss: 0.4461 - acc: 0.9739 - val_loss: 0.5347 - val_acc: 0.9459Epoch 158/500133s 266ms/step - loss: 0.4325 - acc: 0.9769 - val_loss: 0.5237 - val_acc: 0.9461Epoch 159/500133s 266ms/step - loss: 0.4263 - acc: 0.9767 - val_loss: 0.5284 - val_acc: 0.9435Epoch 160/500133s 266ms/step - loss: 0.4159 - acc: 0.9773 - val_loss: 0.5137 - val_acc: 0.9458Epoch 161/500133s 266ms/step - loss: 0.4084 - acc: 0.9782 - val_loss: 0.5121 - val_acc: 0.9457Epoch 162/500133s 266ms/step - loss: 0.4002 - acc: 0.9792 - val_loss: 0.5061 - val_acc: 0.9463Epoch 163/500133s 266ms/step - loss: 0.3892 - acc: 0.9812 - val_loss: 0.5056 - val_acc: 0.9454Epoch 164/500133s 266ms/step - loss: 0.3828 - acc: 0.9816 - val_loss: 0.5098 - val_acc: 0.9438Epoch 165/500133s 266ms/step - loss: 0.3795 - acc: 0.9811 - val_loss: 0.4993 - val_acc: 0.9436Epoch 166/500133s 266ms/step - loss: 0.3708 - acc: 0.9829 - val_loss: 0.4963 - val_acc: 0.9439Epoch 167/500133s 266ms/step - loss: 0.3640 - acc: 0.9835 - val_loss: 0.4935 - val_acc: 0.9428Epoch 168/500133s 266ms/step - loss: 0.3581 - acc: 0.9835 - val_loss: 0.4856 - val_acc: 0.9440Epoch 169/500133s 266ms/step - loss: 0.3534 - acc: 0.9836 - val_loss: 0.4830 - val_acc: 0.9441Epoch 170/500133s 266ms/step - loss: 0.3478 - acc: 0.9841 - val_loss: 0.4819 - val_acc: 0.9452Epoch 171/500133s 266ms/step - loss: 0.3438 - acc: 0.9836 - val_loss: 0.4810 - val_acc: 0.9432Epoch 172/500133s 266ms/step - loss: 0.3365 - acc: 0.9847 - val_loss: 0.4694 - val_acc: 0.9430Epoch 173/500133s 266ms/step - loss: 0.3307 - acc: 0.9859 - val_loss: 0.4657 - val_acc: 0.9454Epoch 174/500133s 266ms/step - loss: 0.3266 - acc: 0.9849 - val_loss: 0.4566 - val_acc: 0.9474Epoch 175/500133s 265ms/step - loss: 0.3199 - acc: 0.9860 - val_loss: 0.4570 - val_acc: 0.9442Epoch 176/500133s 266ms/step - loss: 0.3156 - acc: 0.9863 - val_loss: 0.4640 - val_acc: 0.9426Epoch 177/500133s 266ms/step - loss: 0.3134 - acc: 0.9863 - val_loss: 0.4648 - val_acc: 0.9405Epoch 178/500133s 266ms/step - loss: 0.3089 - acc: 0.9856 - val_loss: 0.4527 - val_acc: 0.9450Epoch 179/500134s 267ms/step - loss: 0.3021 - acc: 0.9871 - val_loss: 0.4506 - val_acc: 0.9429Epoch 180/500133s 266ms/step - loss: 0.2990 - acc: 0.9868 - val_loss: 0.4441 - val_acc: 0.9460Epoch 181/500133s 267ms/step - loss: 0.2960 - acc: 0.9870 - val_loss: 0.4532 - val_acc: 0.9413Epoch 182/500133s 266ms/step - loss: 0.2916 - acc: 0.9874 - val_loss: 0.4430 - val_acc: 0.9435Epoch 183/500133s 266ms/step - loss: 0.2905 - acc: 0.9860 - val_loss: 0.4414 - val_acc: 0.9445Epoch 184/500133s 266ms/step - loss: 0.2851 - acc: 0.9875 - val_loss: 0.4303 - val_acc: 0.9460Epoch 185/500133s 266ms/step - loss: 0.2804 - acc: 0.9873 - val_loss: 0.4317 - val_acc: 0.9441Epoch 186/500133s 266ms/step - loss: 0.2789 - acc: 0.9872 - val_loss: 0.4294 - val_acc: 0.9449Epoch 187/500133s 266ms/step - loss: 0.2748 - acc: 0.9874 - val_loss: 0.4248 - val_acc: 0.9428Epoch 188/500133s 266ms/step - loss: 0.2700 - acc: 0.9882 - val_loss: 0.4242 - val_acc: 0.9461Epoch 189/500133s 266ms/step - loss: 0.2690 - acc: 0.9875 - val_loss: 0.4235 - val_acc: 0.9402Epoch 190/500133s 266ms/step - loss: 0.2669 - acc: 0.9868 - val_loss: 0.4353 - val_acc: 0.9404Epoch 191/500133s 266ms/step - loss: 0.2636 - acc: 0.9874 - val_loss: 0.4182 - val_acc: 0.9432Epoch 192/500133s 266ms/step - loss: 0.2595 - acc: 0.9882 - val_loss: 0.4160 - val_acc: 0.9439Epoch 193/500133s 266ms/step - loss: 0.2579 - acc: 0.9879 - val_loss: 0.4147 - val_acc: 0.9445Epoch 194/500133s 266ms/step - loss: 0.2556 - acc: 0.9876 - val_loss: 0.4161 - val_acc: 0.9428Epoch 195/500133s 266ms/step - loss: 0.2530 - acc: 0.9875 - val_loss: 0.4120 - val_acc: 0.9420Epoch 196/500133s 266ms/step - loss: 0.2525 - acc: 0.9873 - val_loss: 0.4121 - val_acc: 0.9420Epoch 197/500133s 266ms/step - loss: 0.2492 - acc: 0.9877 - val_loss: 0.4260 - val_acc: 0.9390Epoch 198/500133s 266ms/step - loss: 0.2488 - acc: 0.9867 - val_loss: 0.4126 - val_acc: 0.9424Epoch 199/500133s 266ms/step - loss: 0.2458 - acc: 0.9869 - val_loss: 0.3974 - val_acc: 0.9442Epoch 200/500133s 266ms/step - loss: 0.2418 - acc: 0.9881 - val_loss: 0.3993 - val_acc: 0.9431Epoch 201/500133s 266ms/step - loss: 0.2390 - acc: 0.9878 - val_loss: 0.3998 - val_acc: 0.9448Epoch 202/500133s 266ms/step - loss: 0.2387 - acc: 0.9871 - val_loss: 0.3926 - val_acc: 0.9451Epoch 203/500133s 266ms/step - loss: 0.2354 - acc: 0.9879 - val_loss: 0.4007 - val_acc: 0.9415Epoch 204/500133s 266ms/step - loss: 0.2342 - acc: 0.9878 - val_loss: 0.3996 - val_acc: 0.9410Epoch 205/500133s 266ms/step - loss: 0.2376 - acc: 0.9860 - val_loss: 0.3943 - val_acc: 0.9423Epoch 206/500133s 266ms/step - loss: 0.2315 - acc: 0.9869 - val_loss: 0.3860 - val_acc: 0.9441Epoch 207/500133s 266ms/step - loss: 0.2326 - acc: 0.9865 - val_loss: 0.3913 - val_acc: 0.9417Epoch 208/500133s 266ms/step - loss: 0.2282 - acc: 0.9875 - val_loss: 0.3957 - val_acc: 0.9422Epoch 209/500133s 266ms/step - loss: 0.2272 - acc: 0.9870 - val_loss: 0.4006 - val_acc: 0.9397Epoch 210/500133s 266ms/step - loss: 0.2279 - acc: 0.9867 - val_loss: 0.3860 - val_acc: 0.9437Epoch 211/500133s 266ms/step - loss: 0.2283 - acc: 0.9856 - val_loss: 0.3855 - val_acc: 0.9427Epoch 212/500133s 266ms/step - loss: 0.2238 - acc: 0.9870 - val_loss: 0.3897 - val_acc: 0.9409Epoch 213/500133s 266ms/step - loss: 0.2246 - acc: 0.9864 - val_loss: 0.3808 - val_acc: 0.9425Epoch 214/500133s 266ms/step - loss: 0.2247 - acc: 0.9861 - val_loss: 0.4011 - val_acc: 0.9375Epoch 215/500133s 266ms/step - loss: 0.2213 - acc: 0.9865 - val_loss: 0.3887 - val_acc: 0.9399Epoch 216/500133s 266ms/step - loss: 0.2193 - acc: 0.9867 - val_loss: 0.3850 - val_acc: 0.9420Epoch 217/500133s 266ms/step - loss: 0.2193 - acc: 0.9862 - val_loss: 0.3781 - val_acc: 0.9425Epoch 218/500133s 266ms/step - loss: 0.2194 - acc: 0.9861 - val_loss: 0.3863 - val_acc: 0.9399Epoch 219/500133s 266ms/step - loss: 0.2165 - acc: 0.9869 - val_loss: 0.3795 - val_acc: 0.9417Epoch 220/500133s 266ms/step - loss: 0.2149 - acc: 0.9874 - val_loss: 0.3749 - val_acc: 0.9443Epoch 221/500133s 266ms/step - loss: 0.2171 - acc: 0.9854 - val_loss: 0.3776 - val_acc: 0.9424Epoch 222/500133s 266ms/step - loss: 0.2183 - acc: 0.9845 - val_loss: 0.3854 - val_acc: 0.9397Epoch 223/500133s 266ms/step - loss: 0.2163 - acc: 0.9854 - val_loss: 0.3745 - val_acc: 0.9424Epoch 224/500133s 266ms/step - loss: 0.2138 - acc: 0.9861 - val_loss: 0.3695 - val_acc: 0.9425Epoch 225/500133s 266ms/step - loss: 0.2098 - acc: 0.9868 - val_loss: 0.3634 - val_acc: 0.9459Epoch 226/500133s 266ms/step - loss: 0.2120 - acc: 0.9863 - val_loss: 0.3709 - val_acc: 0.9431Epoch 227/500133s 266ms/step - loss: 0.2122 - acc: 0.9858 - val_loss: 0.3758 - val_acc: 0.9395Epoch 228/500133s 266ms/step - loss: 0.2103 - acc: 0.9861 - val_loss: 0.3628 - val_acc: 0.9423Epoch 229/500133s 266ms/step - loss: 0.2105 - acc: 0.9856 - val_loss: 0.3739 - val_acc: 0.9400Epoch 230/500133s 266ms/step - loss: 0.2109 - acc: 0.9854 - val_loss: 0.3757 - val_acc: 0.9399Epoch 231/500133s 266ms/step - loss: 0.2089 - acc: 0.9860 - val_loss: 0.3677 - val_acc: 0.9412Epoch 232/500133s 266ms/step - loss: 0.2062 - acc: 0.9875 - val_loss: 0.3596 - val_acc: 0.9430Epoch 233/500133s 266ms/step - loss: 0.2068 - acc: 0.9859 - val_loss: 0.3635 - val_acc: 0.9409Epoch 234/500133s 266ms/step - loss: 0.2060 - acc: 0.9863 - val_loss: 0.3792 - val_acc: 0.9381Epoch 235/500133s 266ms/step - loss: 0.2061 - acc: 0.9865 - val_loss: 0.3720 - val_acc: 0.9416Epoch 236/500133s 266ms/step - loss: 0.2066 - acc: 0.9853 - val_loss: 0.3862 - val_acc: 0.9353Epoch 237/500133s 266ms/step - loss: 0.2089 - acc: 0.9846 - val_loss: 0.3698 - val_acc: 0.9387Epoch 238/500133s 266ms/step - loss: 0.2065 - acc: 0.9853 - val_loss: 0.3611 - val_acc: 0.9405Epoch 239/500133s 266ms/step - loss: 0.2070 - acc: 0.9853 - val_loss: 0.3688 - val_acc: 0.9386Epoch 240/500133s 266ms/step - loss: 0.2044 - acc: 0.9858 - val_loss: 0.3689 - val_acc: 0.9398Epoch 241/500133s 266ms/step - loss: 0.2055 - acc: 0.9849 - val_loss: 0.3766 - val_acc: 0.9390Epoch 242/500133s 266ms/step - loss: 0.2028 - acc: 0.9861 - val_loss: 0.3592 - val_acc: 0.9414Epoch 243/500133s 266ms/step - loss: 0.2030 - acc: 0.9863 - val_loss: 0.3616 - val_acc: 0.9431Epoch 244/500133s 266ms/step - loss: 0.2024 - acc: 0.9861 - val_loss: 0.3722 - val_acc: 0.9379Epoch 245/500133s 266ms/step - loss: 0.2038 - acc: 0.9855 - val_loss: 0.3620 - val_acc: 0.9407Epoch 246/500133s 266ms/step - loss: 0.2014 - acc: 0.9865 - val_loss: 0.3740 - val_acc: 0.9376Epoch 247/500133s 266ms/step - loss: 0.2012 - acc: 0.9856 - val_loss: 0.3630 - val_acc: 0.9418Epoch 248/500133s 266ms/step - loss: 0.2045 - acc: 0.9845 - val_loss: 0.3644 - val_acc: 0.9401Epoch 249/500133s 266ms/step - loss: 0.2044 - acc: 0.9845 - val_loss: 0.3605 - val_acc: 0.9384Epoch 250/500133s 266ms/step - loss: 0.2066 - acc: 0.9842 - val_loss: 0.3684 - val_acc: 0.9383Epoch 251/500133s 266ms/step - loss: 0.2005 - acc: 0.9861 - val_loss: 0.3683 - val_acc: 0.9377Epoch 252/500133s 266ms/step - loss: 0.2019 - acc: 0.9856 - val_loss: 0.3663 - val_acc: 0.9382Epoch 253/500133s 266ms/step - loss: 0.2026 - acc: 0.9852 - val_loss: 0.3666 - val_acc: 0.9393Epoch 254/500133s 266ms/step - loss: 0.2034 - acc: 0.9841 - val_loss: 0.3627 - val_acc: 0.9409Epoch 255/500133s 266ms/step - loss: 0.1998 - acc: 0.9857 - val_loss: 0.3609 - val_acc: 0.9408Epoch 256/500133s 266ms/step - loss: 0.1987 - acc: 0.9859 - val_loss: 0.3629 - val_acc: 0.9414Epoch 257/500133s 266ms/step - loss: 0.1971 - acc: 0.9871 - val_loss: 0.3768 - val_acc: 0.9375Epoch 258/500133s 266ms/step - loss: 0.1985 - acc: 0.9860 - val_loss: 0.3706 - val_acc: 0.9375Epoch 259/500133s 266ms/step - loss: 0.2006 - acc: 0.9845 - val_loss: 0.3638 - val_acc: 0.9401Epoch 260/500133s 266ms/step - loss: 0.1991 - acc: 0.9851 - val_loss: 0.3629 - val_acc: 0.9392Epoch 261/500133s 265ms/step - loss: 0.1999 - acc: 0.9854 - val_loss: 0.3603 - val_acc: 0.9420Epoch 262/500133s 266ms/step - loss: 0.2021 - acc: 0.9841 - val_loss: 0.3610 - val_acc: 0.9391Epoch 263/500133s 265ms/step - loss: 0.2009 - acc: 0.9850 - val_loss: 0.3454 - val_acc: 0.9413Epoch 264/500133s 266ms/step - loss: 0.1956 - acc: 0.9866 - val_loss: 0.3662 - val_acc: 0.9382Epoch 265/500133s 266ms/step - loss: 0.2038 - acc: 0.9844 - val_loss: 0.3595 - val_acc: 0.9417Epoch 266/500133s 266ms/step - loss: 0.1982 - acc: 0.9854 - val_loss: 0.3578 - val_acc: 0.9396Epoch 267/500133s 266ms/step - loss: 0.1996 - acc: 0.9844 - val_loss: 0.3662 - val_acc: 0.9397Epoch 268/500133s 266ms/step - loss: 0.1978 - acc: 0.9854 - val_loss: 0.3551 - val_acc: 0.9437Epoch 269/500133s 266ms/step - loss: 0.1986 - acc: 0.9855 - val_loss: 0.3636 - val_acc: 0.9412Epoch 270/500133s 266ms/step - loss: 0.1982 - acc: 0.9851 - val_loss: 0.3495 - val_acc: 0.9415Epoch 271/500133s 266ms/step - loss: 0.2001 - acc: 0.9845 - val_loss: 0.3504 - val_acc: 0.9407Epoch 272/500133s 266ms/step - loss: 0.1982 - acc: 0.9850 - val_loss: 0.3496 - val_acc: 0.9432Epoch 273/500133s 266ms/step - loss: 0.1989 - acc: 0.9849 - val_loss: 0.3589 - val_acc: 0.9395Epoch 274/500133s 266ms/step - loss: 0.1954 - acc: 0.9862 - val_loss: 0.3517 - val_acc: 0.9440Epoch 275/500133s 266ms/step - loss: 0.1964 - acc: 0.9860 - val_loss: 0.3710 - val_acc: 0.9385Epoch 276/500133s 266ms/step - loss: 0.1991 - acc: 0.9848 - val_loss: 0.3572 - val_acc: 0.9375Epoch 277/500133s 266ms/step - loss: 0.1986 - acc: 0.9841 - val_loss: 0.3733 - val_acc: 0.9394Epoch 278/500133s 266ms/step - loss: 0.2004 - acc: 0.9846 - val_loss: 0.3516 - val_acc: 0.9408Epoch 279/500133s 266ms/step - loss: 0.1961 - acc: 0.9854 - val_loss: 0.3687 - val_acc: 0.9387Epoch 280/500133s 266ms/step - loss: 0.1952 - acc: 0.9855 - val_loss: 0.3724 - val_acc: 0.9397Epoch 281/500133s 266ms/step - loss: 0.1960 - acc: 0.9850 - val_loss: 0.3706 - val_acc: 0.9391Epoch 282/500133s 266ms/step - loss: 0.1961 - acc: 0.9855 - val_loss: 0.3711 - val_acc: 0.9376Epoch 283/500133s 266ms/step - loss: 0.1975 - acc: 0.9846 - val_loss: 0.3728 - val_acc: 0.9352Epoch 284/500133s 266ms/step - loss: 0.1980 - acc: 0.9847 - val_loss: 0.3666 - val_acc: 0.9361Epoch 285/500133s 266ms/step - loss: 0.1944 - acc: 0.9862 - val_loss: 0.3651 - val_acc: 0.9382Epoch 286/500133s 266ms/step - loss: 0.1932 - acc: 0.9862 - val_loss: 0.3679 - val_acc: 0.9366Epoch 287/500133s 266ms/step - loss: 0.1988 - acc: 0.9844 - val_loss: 0.3522 - val_acc: 0.9431Epoch 288/500133s 266ms/step - loss: 0.1947 - acc: 0.9860 - val_loss: 0.3574 - val_acc: 0.9394Epoch 289/500133s 266ms/step - loss: 0.1964 - acc: 0.9854 - val_loss: 0.3608 - val_acc: 0.9391Epoch 290/500133s 266ms/step - loss: 0.1955 - acc: 0.9853 - val_loss: 0.3663 - val_acc: 0.9373Epoch 291/500133s 266ms/step - loss: 0.1966 - acc: 0.9849 - val_loss: 0.3614 - val_acc: 0.9392Epoch 292/500133s 266ms/step - loss: 0.1951 - acc: 0.9856 - val_loss: 0.3698 - val_acc: 0.9350Epoch 293/500133s 266ms/step - loss: 0.1969 - acc: 0.9847 - val_loss: 0.3528 - val_acc: 0.9442Epoch 294/500133s 266ms/step - loss: 0.1981 - acc: 0.9846 - val_loss: 0.3644 - val_acc: 0.9395Epoch 295/500133s 266ms/step - loss: 0.1946 - acc: 0.9854 - val_loss: 0.3649 - val_acc: 0.9390Epoch 296/500133s 266ms/step - loss: 0.1973 - acc: 0.9840 - val_loss: 0.3725 - val_acc: 0.9378Epoch 297/500133s 266ms/step - loss: 0.1954 - acc: 0.9849 - val_loss: 0.3412 - val_acc: 0.9448Epoch 298/500133s 266ms/step - loss: 0.1889 - acc: 0.9877 - val_loss: 0.3610 - val_acc: 0.9377Epoch 299/500133s 266ms/step - loss: 0.1894 - acc: 0.9872 - val_loss: 0.3702 - val_acc: 0.9367Epoch 300/500133s 266ms/step - loss: 0.1957 - acc: 0.9846 - val_loss: 0.3614 - val_acc: 0.9400Epoch 301/500lr changed to 0.0009999999776482583133s 266ms/step - loss: 0.1796 - acc: 0.9907 - val_loss: 0.3285 - val_acc: 0.9484Epoch 302/500133s 266ms/step - loss: 0.1694 - acc: 0.9946 - val_loss: 0.3244 - val_acc: 0.9480Epoch 303/500133s 266ms/step - loss: 0.1670 - acc: 0.9951 - val_loss: 0.3218 - val_acc: 0.9495Epoch 304/500133s 266ms/step - loss: 0.1638 - acc: 0.9963 - val_loss: 0.3213 - val_acc: 0.9500Epoch 305/500133s 266ms/step - loss: 0.1625 - acc: 0.9963 - val_loss: 0.3204 - val_acc: 0.9513Epoch 306/500133s 266ms/step - loss: 0.1614 - acc: 0.9967 - val_loss: 0.3201 - val_acc: 0.9507Epoch 307/500133s 266ms/step - loss: 0.1616 - acc: 0.9965 - val_loss: 0.3205 - val_acc: 0.9510Epoch 308/500133s 266ms/step - loss: 0.1600 - acc: 0.9968 - val_loss: 0.3193 - val_acc: 0.9518Epoch 309/500133s 266ms/step - loss: 0.1593 - acc: 0.9970 - val_loss: 0.3213 - val_acc: 0.9520Epoch 310/500133s 266ms/step - loss: 0.1574 - acc: 0.9975 - val_loss: 0.3204 - val_acc: 0.9517Epoch 311/500133s 266ms/step - loss: 0.1578 - acc: 0.9975 - val_loss: 0.3205 - val_acc: 0.9511Epoch 312/500133s 266ms/step - loss: 0.1575 - acc: 0.9972 - val_loss: 0.3199 - val_acc: 0.9518Epoch 313/500133s 266ms/step - loss: 0.1565 - acc: 0.9977 - val_loss: 0.3192 - val_acc: 0.9521Epoch 314/500133s 266ms/step - loss: 0.1563 - acc: 0.9977 - val_loss: 0.3186 - val_acc: 0.9528Epoch 315/500133s 266ms/step - loss: 0.1555 - acc: 0.9980 - val_loss: 0.3194 - val_acc: 0.9523Epoch 316/500133s 266ms/step - loss: 0.1551 - acc: 0.9978 - val_loss: 0.3190 - val_acc: 0.9529Epoch 317/500133s 266ms/step - loss: 0.1542 - acc: 0.9979 - val_loss: 0.3178 - val_acc: 0.9527Epoch 318/500133s 266ms/step - loss: 0.1542 - acc: 0.9981 - val_loss: 0.3160 - val_acc: 0.9535Epoch 319/500133s 266ms/step - loss: 0.1540 - acc: 0.9980 - val_loss: 0.3157 - val_acc: 0.9536Epoch 320/500133s 266ms/step - loss: 0.1536 - acc: 0.9980 - val_loss: 0.3171 - val_acc: 0.9528Epoch 321/500133s 266ms/step - loss: 0.1532 - acc: 0.9979 - val_loss: 0.3203 - val_acc: 0.9519Epoch 322/500133s 266ms/step - loss: 0.1522 - acc: 0.9982 - val_loss: 0.3214 - val_acc: 0.9526Epoch 323/500133s 266ms/step - loss: 0.1524 - acc: 0.9982 - val_loss: 0.3227 - val_acc: 0.9529Epoch 324/500133s 266ms/step - loss: 0.1519 - acc: 0.9983 - val_loss: 0.3229 - val_acc: 0.9527Epoch 325/500133s 266ms/step - loss: 0.1519 - acc: 0.9981 - val_loss: 0.3206 - val_acc: 0.9531Epoch 326/500133s 265ms/step - loss: 0.1510 - acc: 0.9983 - val_loss: 0.3202 - val_acc: 0.9528Epoch 327/500133s 266ms/step - loss: 0.1509 - acc: 0.9984 - val_loss: 0.3216 - val_acc: 0.9530Epoch 328/500133s 266ms/step - loss: 0.1513 - acc: 0.9979 - val_loss: 0.3222 - val_acc: 0.9537Epoch 329/500133s 266ms/step - loss: 0.1506 - acc: 0.9984 - val_loss: 0.3213 - val_acc: 0.9530Epoch 330/500133s 266ms/step - loss: 0.1496 - acc: 0.9985 - val_loss: 0.3221 - val_acc: 0.9527Epoch 331/500133s 266ms/step - loss: 0.1498 - acc: 0.9984 - val_loss: 0.3214 - val_acc: 0.9523Epoch 332/500133s 266ms/step - loss: 0.1484 - acc: 0.9989 - val_loss: 0.3201 - val_acc: 0.9520Epoch 333/500133s 266ms/step - loss: 0.1491 - acc: 0.9983 - val_loss: 0.3206 - val_acc: 0.9520Epoch 334/500133s 266ms/step - loss: 0.1491 - acc: 0.9984 - val_loss: 0.3196 - val_acc: 0.9530Epoch 335/500133s 266ms/step - loss: 0.1484 - acc: 0.9983 - val_loss: 0.3203 - val_acc: 0.9531Epoch 336/500133s 266ms/step - loss: 0.1476 - acc: 0.9986 - val_loss: 0.3195 - val_acc: 0.9527Epoch 337/500133s 266ms/step - loss: 0.1473 - acc: 0.9986 - val_loss: 0.3177 - val_acc: 0.9525Epoch 338/500133s 266ms/step - loss: 0.1470 - acc: 0.9987 - val_loss: 0.3186 - val_acc: 0.9537Epoch 339/500133s 266ms/step - loss: 0.1466 - acc: 0.9986 - val_loss: 0.3175 - val_acc: 0.9546Epoch 340/500133s 266ms/step - loss: 0.1465 - acc: 0.9986 - val_loss: 0.3148 - val_acc: 0.9540Epoch 341/500133s 265ms/step - loss: 0.1464 - acc: 0.9985 - val_loss: 0.3157 - val_acc: 0.9545Epoch 342/500133s 266ms/step - loss: 0.1464 - acc: 0.9983 - val_loss: 0.3149 - val_acc: 0.9540Epoch 343/500133s 266ms/step - loss: 0.1454 - acc: 0.9988 - val_loss: 0.3154 - val_acc: 0.9547Epoch 344/500133s 266ms/step - loss: 0.1449 - acc: 0.9990 - val_loss: 0.3146 - val_acc: 0.9536Epoch 345/500133s 266ms/step - loss: 0.1446 - acc: 0.9988 - val_loss: 0.3149 - val_acc: 0.9537Epoch 346/500133s 266ms/step - loss: 0.1439 - acc: 0.9992 - val_loss: 0.3148 - val_acc: 0.9540Epoch 347/500133s 266ms/step - loss: 0.1450 - acc: 0.9985 - val_loss: 0.3170 - val_acc: 0.9535Epoch 348/500133s 266ms/step - loss: 0.1449 - acc: 0.9985 - val_loss: 0.3167 - val_acc: 0.9539Epoch 349/500133s 266ms/step - loss: 0.1441 - acc: 0.9988 - val_loss: 0.3159 - val_acc: 0.9537Epoch 350/500133s 266ms/step - loss: 0.1439 - acc: 0.9986 - val_loss: 0.3160 - val_acc: 0.9540Epoch 351/500133s 266ms/step - loss: 0.1428 - acc: 0.9988 - val_loss: 0.3141 - val_acc: 0.9537Epoch 352/500133s 266ms/step - loss: 0.1431 - acc: 0.9988 - val_loss: 0.3135 - val_acc: 0.9539Epoch 353/500133s 266ms/step - loss: 0.1422 - acc: 0.9992 - val_loss: 0.3148 - val_acc: 0.9540Epoch 354/500133s 266ms/step - loss: 0.1423 - acc: 0.9988 - val_loss: 0.3155 - val_acc: 0.9540Epoch 355/500133s 266ms/step - loss: 0.1417 - acc: 0.9989 - val_loss: 0.3134 - val_acc: 0.9538Epoch 356/500133s 266ms/step - loss: 0.1417 - acc: 0.9990 - val_loss: 0.3155 - val_acc: 0.9544Epoch 357/500133s 266ms/step - loss: 0.1421 - acc: 0.9985 - val_loss: 0.3138 - val_acc: 0.9546Epoch 358/500133s 266ms/step - loss: 0.1411 - acc: 0.9989 - val_loss: 0.3124 - val_acc: 0.9542Epoch 359/500133s 266ms/step - loss: 0.1409 - acc: 0.9991 - val_loss: 0.3129 - val_acc: 0.9543Epoch 360/500133s 266ms/step - loss: 0.1408 - acc: 0.9989 - val_loss: 0.3110 - val_acc: 0.9556Epoch 361/500133s 266ms/step - loss: 0.1411 - acc: 0.9986 - val_loss: 0.3136 - val_acc: 0.9541Epoch 362/500133s 266ms/step - loss: 0.1401 - acc: 0.9990 - val_loss: 0.3136 - val_acc: 0.9550Epoch 363/500133s 266ms/step - loss: 0.1402 - acc: 0.9986 - val_loss: 0.3134 - val_acc: 0.9553Epoch 364/500133s 266ms/step - loss: 0.1401 - acc: 0.9987 - val_loss: 0.3138 - val_acc: 0.9538Epoch 365/500133s 266ms/step - loss: 0.1394 - acc: 0.9990 - val_loss: 0.3139 - val_acc: 0.9546Epoch 366/500133s 266ms/step - loss: 0.1387 - acc: 0.9992 - val_loss: 0.3130 - val_acc: 0.9545Epoch 367/500133s 266ms/step - loss: 0.1390 - acc: 0.9991 - val_loss: 0.3112 - val_acc: 0.9557Epoch 368/500133s 266ms/step - loss: 0.1396 - acc: 0.9985 - val_loss: 0.3138 - val_acc: 0.9553Epoch 369/500133s 266ms/step - loss: 0.1389 - acc: 0.9989 - val_loss: 0.3153 - val_acc: 0.9545Epoch 370/500133s 266ms/step - loss: 0.1384 - acc: 0.9988 - val_loss: 0.3124 - val_acc: 0.9547Epoch 371/500133s 266ms/step - loss: 0.1378 - acc: 0.9989 - val_loss: 0.3106 - val_acc: 0.9544Epoch 372/500133s 266ms/step - loss: 0.1380 - acc: 0.9990 - val_loss: 0.3092 - val_acc: 0.9556Epoch 373/500133s 266ms/step - loss: 0.1377 - acc: 0.9989 - val_loss: 0.3098 - val_acc: 0.9544Epoch 374/500133s 266ms/step - loss: 0.1368 - acc: 0.9993 - val_loss: 0.3084 - val_acc: 0.9556Epoch 375/500133s 266ms/step - loss: 0.1366 - acc: 0.9991 - val_loss: 0.3074 - val_acc: 0.9558Epoch 376/500133s 266ms/step - loss: 0.1370 - acc: 0.9988 - val_loss: 0.3083 - val_acc: 0.9546Epoch 377/500133s 266ms/step - loss: 0.1369 - acc: 0.9989 - val_loss: 0.3079 - val_acc: 0.9547Epoch 378/500133s 266ms/step - loss: 0.1364 - acc: 0.9989 - val_loss: 0.3075 - val_acc: 0.9551Epoch 379/500133s 266ms/step - loss: 0.1360 - acc: 0.9989 - val_loss: 0.3092 - val_acc: 0.9550Epoch 380/500133s 266ms/step - loss: 0.1367 - acc: 0.9986 - val_loss: 0.3111 - val_acc: 0.9537Epoch 381/500133s 266ms/step - loss: 0.1364 - acc: 0.9986 - val_loss: 0.3126 - val_acc: 0.9548Epoch 382/500133s 266ms/step - loss: 0.1352 - acc: 0.9991 - val_loss: 0.3095 - val_acc: 0.9544Epoch 383/500133s 266ms/step - loss: 0.1352 - acc: 0.9989 - val_loss: 0.3108 - val_acc: 0.9529Epoch 384/500133s 266ms/step - loss: 0.1350 - acc: 0.9989 - val_loss: 0.3122 - val_acc: 0.9534Epoch 385/500133s 266ms/step - loss: 0.1347 - acc: 0.9988 - val_loss: 0.3109 - val_acc: 0.9540Epoch 386/500133s 266ms/step - loss: 0.1340 - acc: 0.9991 - val_loss: 0.3085 - val_acc: 0.9549Epoch 387/500133s 266ms/step - loss: 0.1344 - acc: 0.9988 - val_loss: 0.3081 - val_acc: 0.9554Epoch 388/500133s 266ms/step - loss: 0.1341 - acc: 0.9988 - val_loss: 0.3092 - val_acc: 0.9546Epoch 389/500133s 266ms/step - loss: 0.1336 - acc: 0.9991 - val_loss: 0.3085 - val_acc: 0.9555Epoch 390/500133s 266ms/step - loss: 0.1337 - acc: 0.9989 - val_loss: 0.3086 - val_acc: 0.9557Epoch 391/500133s 266ms/step - loss: 0.1337 - acc: 0.9989 - val_loss: 0.3092 - val_acc: 0.9546Epoch 392/500133s 266ms/step - loss: 0.1329 - acc: 0.9990 - val_loss: 0.3072 - val_acc: 0.9546Epoch 393/500133s 266ms/step - loss: 0.1326 - acc: 0.9991 - val_loss: 0.3077 - val_acc: 0.9547Epoch 394/500133s 266ms/step - loss: 0.1330 - acc: 0.9989 - val_loss: 0.3059 - val_acc: 0.9556Epoch 395/500133s 266ms/step - loss: 0.1320 - acc: 0.9991 - val_loss: 0.3042 - val_acc: 0.9552Epoch 396/500133s 266ms/step - loss: 0.1317 - acc: 0.9993 - val_loss: 0.3054 - val_acc: 0.9549Epoch 397/500133s 266ms/step - loss: 0.1313 - acc: 0.9993 - val_loss: 0.3083 - val_acc: 0.9538Epoch 398/500133s 266ms/step - loss: 0.1316 - acc: 0.9990 - val_loss: 0.3087 - val_acc: 0.9550Epoch 399/500133s 266ms/step - loss: 0.1323 - acc: 0.9988 - val_loss: 0.3056 - val_acc: 0.9546Epoch 400/500133s 266ms/step - loss: 0.1315 - acc: 0.9989 - val_loss: 0.3034 - val_acc: 0.9552Epoch 401/500133s 266ms/step - loss: 0.1311 - acc: 0.9991 - val_loss: 0.3034 - val_acc: 0.9553Epoch 402/500133s 266ms/step - loss: 0.1306 - acc: 0.9991 - val_loss: 0.3006 - val_acc: 0.9556Epoch 403/500133s 266ms/step - loss: 0.1302 - acc: 0.9990 - val_loss: 0.3019 - val_acc: 0.9565Epoch 404/500133s 266ms/step - loss: 0.1308 - acc: 0.9989 - val_loss: 0.3007 - val_acc: 0.9557Epoch 405/500133s 266ms/step - loss: 0.1300 - acc: 0.9992 - val_loss: 0.3011 - val_acc: 0.9561Epoch 406/500133s 266ms/step - loss: 0.1294 - acc: 0.9993 - val_loss: 0.3018 - val_acc: 0.9543Epoch 407/500133s 266ms/step - loss: 0.1297 - acc: 0.9990 - val_loss: 0.3013 - val_acc: 0.9548Epoch 408/500133s 266ms/step - loss: 0.1300 - acc: 0.9989 - val_loss: 0.3017 - val_acc: 0.9554Epoch 409/500133s 266ms/step - loss: 0.1292 - acc: 0.9991 - val_loss: 0.2996 - val_acc: 0.9558Epoch 410/500133s 266ms/step - loss: 0.1293 - acc: 0.9990 - val_loss: 0.3017 - val_acc: 0.9554Epoch 411/500133s 266ms/step - loss: 0.1289 - acc: 0.9990 - val_loss: 0.3029 - val_acc: 0.9545Epoch 412/500133s 266ms/step - loss: 0.1290 - acc: 0.9990 - val_loss: 0.2996 - val_acc: 0.9544Epoch 413/500133s 266ms/step - loss: 0.1287 - acc: 0.9988 - val_loss: 0.3004 - val_acc: 0.9537Epoch 414/500133s 266ms/step - loss: 0.1280 - acc: 0.9992 - val_loss: 0.2980 - val_acc: 0.9540Epoch 415/500133s 266ms/step - loss: 0.1281 - acc: 0.9988 - val_loss: 0.3000 - val_acc: 0.9545Epoch 416/500133s 266ms/step - loss: 0.1281 - acc: 0.9990 - val_loss: 0.3015 - val_acc: 0.9546Epoch 417/500133s 266ms/step - loss: 0.1271 - acc: 0.9993 - val_loss: 0.3014 - val_acc: 0.9552Epoch 418/500133s 266ms/step - loss: 0.1273 - acc: 0.9991 - val_loss: 0.2995 - val_acc: 0.9547Epoch 419/500133s 266ms/step - loss: 0.1269 - acc: 0.9992 - val_loss: 0.3008 - val_acc: 0.9549Epoch 420/500133s 266ms/step - loss: 0.1266 - acc: 0.9992 - val_loss: 0.3001 - val_acc: 0.9533Epoch 421/500133s 266ms/step - loss: 0.1269 - acc: 0.9989 - val_loss: 0.3008 - val_acc: 0.9541Epoch 422/500133s 266ms/step - loss: 0.1266 - acc: 0.9991 - val_loss: 0.3006 - val_acc: 0.9542Epoch 423/500133s 266ms/step - loss: 0.1267 - acc: 0.9991 - val_loss: 0.2985 - val_acc: 0.9546Epoch 424/500133s 266ms/step - loss: 0.1258 - acc: 0.9991 - val_loss: 0.2998 - val_acc: 0.9542Epoch 425/500133s 266ms/step - loss: 0.1259 - acc: 0.9990 - val_loss: 0.3020 - val_acc: 0.9536Epoch 426/500133s 266ms/step - loss: 0.1254 - acc: 0.9990 - val_loss: 0.2985 - val_acc: 0.9547Epoch 427/500133s 266ms/step - loss: 0.1260 - acc: 0.9988 - val_loss: 0.3015 - val_acc: 0.9541Epoch 428/500133s 266ms/step - loss: 0.1257 - acc: 0.9989 - val_loss: 0.3011 - val_acc: 0.9542Epoch 429/500133s 266ms/step - loss: 0.1247 - acc: 0.9992 - val_loss: 0.3011 - val_acc: 0.9541Epoch 430/500133s 266ms/step - loss: 0.1249 - acc: 0.9990 - val_loss: 0.3004 - val_acc: 0.9553Epoch 431/500133s 266ms/step - loss: 0.1247 - acc: 0.9991 - val_loss: 0.3010 - val_acc: 0.9542Epoch 432/500133s 265ms/step - loss: 0.1241 - acc: 0.9992 - val_loss: 0.3029 - val_acc: 0.9541Epoch 433/500133s 266ms/step - loss: 0.1244 - acc: 0.9990 - val_loss: 0.3011 - val_acc: 0.9538Epoch 434/500133s 265ms/step - loss: 0.1239 - acc: 0.9991 - val_loss: 0.3019 - val_acc: 0.9545Epoch 435/500133s 266ms/step - loss: 0.1235 - acc: 0.9992 - val_loss: 0.3008 - val_acc: 0.9548Epoch 436/500133s 266ms/step - loss: 0.1235 - acc: 0.9991 - val_loss: 0.2996 - val_acc: 0.9548Epoch 437/500133s 266ms/step - loss: 0.1232 - acc: 0.9991 - val_loss: 0.2999 - val_acc: 0.9545Epoch 438/500133s 266ms/step - loss: 0.1229 - acc: 0.9991 - val_loss: 0.3015 - val_acc: 0.9542Epoch 439/500133s 266ms/step - loss: 0.1225 - acc: 0.9992 - val_loss: 0.3014 - val_acc: 0.9538Epoch 440/500133s 266ms/step - loss: 0.1233 - acc: 0.9991 - val_loss: 0.3017 - val_acc: 0.9553Epoch 441/500133s 266ms/step - loss: 0.1225 - acc: 0.9992 - val_loss: 0.3018 - val_acc: 0.9546Epoch 442/500133s 266ms/step - loss: 0.1223 - acc: 0.9992 - val_loss: 0.3016 - val_acc: 0.9536Epoch 443/500133s 267ms/step - loss: 0.1221 - acc: 0.9992 - val_loss: 0.3026 - val_acc: 0.9546Epoch 444/500133s 266ms/step - loss: 0.1220 - acc: 0.9990 - val_loss: 0.3024 - val_acc: 0.9553Epoch 445/500133s 266ms/step - loss: 0.1220 - acc: 0.9990 - val_loss: 0.3016 - val_acc: 0.9540Epoch 446/500133s 266ms/step - loss: 0.1219 - acc: 0.9990 - val_loss: 0.2991 - val_acc: 0.9542Epoch 447/500133s 266ms/step - loss: 0.1215 - acc: 0.9990 - val_loss: 0.2986 - val_acc: 0.9546Epoch 448/500133s 266ms/step - loss: 0.1214 - acc: 0.9989 - val_loss: 0.3001 - val_acc: 0.9533Epoch 449/500133s 266ms/step - loss: 0.1211 - acc: 0.9991 - val_loss: 0.2969 - val_acc: 0.9560Epoch 450/500133s 266ms/step - loss: 0.1213 - acc: 0.9991 - val_loss: 0.2956 - val_acc: 0.9547Epoch 451/500lr changed to 9.999999310821295e-05133s 266ms/step - loss: 0.1205 - acc: 0.9993 - val_loss: 0.2953 - val_acc: 0.9552Epoch 452/500133s 266ms/step - loss: 0.1205 - acc: 0.9993 - val_loss: 0.2953 - val_acc: 0.9551Epoch 453/500133s 265ms/step - loss: 0.1202 - acc: 0.9995 - val_loss: 0.2951 - val_acc: 0.9552Epoch 454/500133s 266ms/step - loss: 0.1205 - acc: 0.9991 - val_loss: 0.2949 - val_acc: 0.9550Epoch 455/500133s 266ms/step - loss: 0.1204 - acc: 0.9993 - val_loss: 0.2945 - val_acc: 0.9552Epoch 456/500133s 266ms/step - loss: 0.1203 - acc: 0.9993 - val_loss: 0.2947 - val_acc: 0.9546Epoch 457/500133s 266ms/step - loss: 0.1200 - acc: 0.9995 - val_loss: 0.2948 - val_acc: 0.9545Epoch 458/500133s 266ms/step - loss: 0.1204 - acc: 0.9993 - val_loss: 0.2948 - val_acc: 0.9548Epoch 459/500133s 266ms/step - loss: 0.1200 - acc: 0.9993 - val_loss: 0.2946 - val_acc: 0.9545Epoch 460/500133s 266ms/step - loss: 0.1203 - acc: 0.9992 - val_loss: 0.2942 - val_acc: 0.9551Epoch 461/500133s 266ms/step - loss: 0.1199 - acc: 0.9993 - val_loss: 0.2947 - val_acc: 0.9551Epoch 462/500133s 266ms/step - loss: 0.1201 - acc: 0.9993 - val_loss: 0.2946 - val_acc: 0.9547Epoch 463/500133s 266ms/step - loss: 0.1199 - acc: 0.9994 - val_loss: 0.2945 - val_acc: 0.9548Epoch 464/500133s 266ms/step - loss: 0.1200 - acc: 0.9994 - val_loss: 0.2946 - val_acc: 0.9549Epoch 465/500133s 266ms/step - loss: 0.1200 - acc: 0.9993 - val_loss: 0.2945 - val_acc: 0.9547Epoch 466/500133s 266ms/step - loss: 0.1202 - acc: 0.9990 - val_loss: 0.2941 - val_acc: 0.9552Epoch 467/500133s 266ms/step - loss: 0.1198 - acc: 0.9993 - val_loss: 0.2938 - val_acc: 0.9554Epoch 468/500133s 266ms/step - loss: 0.1195 - acc: 0.9994 - val_loss: 0.2938 - val_acc: 0.9552Epoch 469/500133s 266ms/step - loss: 0.1203 - acc: 0.9993 - val_loss: 0.2938 - val_acc: 0.9553Epoch 470/500133s 265ms/step - loss: 0.1199 - acc: 0.9994 - val_loss: 0.2941 - val_acc: 0.9553Epoch 471/500133s 266ms/step - loss: 0.1198 - acc: 0.9994 - val_loss: 0.2938 - val_acc: 0.9553Epoch 472/500133s 266ms/step - loss: 0.1199 - acc: 0.9993 - val_loss: 0.2938 - val_acc: 0.9551Epoch 473/500133s 266ms/step - loss: 0.1201 - acc: 0.9992 - val_loss: 0.2936 - val_acc: 0.9553Epoch 474/500133s 266ms/step - loss: 0.1197 - acc: 0.9993 - val_loss: 0.2937 - val_acc: 0.9549Epoch 475/500133s 265ms/step - loss: 0.1202 - acc: 0.9991 - val_loss: 0.2936 - val_acc: 0.9553Epoch 476/500133s 265ms/step - loss: 0.1201 - acc: 0.9992 - val_loss: 0.2935 - val_acc: 0.9554Epoch 477/500133s 265ms/step - loss: 0.1203 - acc: 0.9991 - val_loss: 0.2935 - val_acc: 0.9551Epoch 478/500133s 265ms/step - loss: 0.1198 - acc: 0.9992 - val_loss: 0.2938 - val_acc: 0.9553Epoch 479/500133s 265ms/step - loss: 0.1199 - acc: 0.9991 - val_loss: 0.2940 - val_acc: 0.9552Epoch 480/500133s 265ms/step - loss: 0.1199 - acc: 0.9992 - val_loss: 0.2938 - val_acc: 0.9553Epoch 481/500133s 266ms/step - loss: 0.1196 - acc: 0.9994 - val_loss: 0.2936 - val_acc: 0.9553Epoch 482/500133s 265ms/step - loss: 0.1196 - acc: 0.9994 - val_loss: 0.2938 - val_acc: 0.9552Epoch 483/500133s 265ms/step - loss: 0.1198 - acc: 0.9993 - val_loss: 0.2939 - val_acc: 0.9550Epoch 484/500133s 265ms/step - loss: 0.1196 - acc: 0.9994 - val_loss: 0.2942 - val_acc: 0.9549Epoch 485/500133s 265ms/step - loss: 0.1200 - acc: 0.9992 - val_loss: 0.2940 - val_acc: 0.9550Epoch 486/500133s 265ms/step - loss: 0.1194 - acc: 0.9994 - val_loss: 0.2941 - val_acc: 0.9552Epoch 487/500133s 265ms/step - loss: 0.1195 - acc: 0.9993 - val_loss: 0.2936 - val_acc: 0.9549Epoch 488/500133s 265ms/step - loss: 0.1196 - acc: 0.9993 - val_loss: 0.2936 - val_acc: 0.9551Epoch 489/500133s 265ms/step - loss: 0.1195 - acc: 0.9992 - val_loss: 0.2937 - val_acc: 0.9547Epoch 490/500133s 266ms/step - loss: 0.1195 - acc: 0.9992 - val_loss: 0.2936 - val_acc: 0.9550Epoch 491/500133s 266ms/step - loss: 0.1192 - acc: 0.9993 - val_loss: 0.2936 - val_acc: 0.9548Epoch 492/500133s 266ms/step - loss: 0.1197 - acc: 0.9993 - val_loss: 0.2933 - val_acc: 0.9551Epoch 493/500133s 266ms/step - loss: 0.1192 - acc: 0.9994 - val_loss: 0.2930 - val_acc: 0.9550Epoch 494/500133s 266ms/step - loss: 0.1195 - acc: 0.9994 - val_loss: 0.2929 - val_acc: 0.9553Epoch 495/500133s 266ms/step - loss: 0.1192 - acc: 0.9994 - val_loss: 0.2929 - val_acc: 0.9551Epoch 496/500133s 266ms/step - loss: 0.1192 - acc: 0.9993 - val_loss: 0.2930 - val_acc: 0.9553Epoch 497/500133s 266ms/step - loss: 0.1191 - acc: 0.9993 - val_loss: 0.2929 - val_acc: 0.9551Epoch 498/500133s 266ms/step - loss: 0.1192 - acc: 0.9994 - val_loss: 0.2928 - val_acc: 0.9552Epoch 499/500133s 266ms/step - loss: 0.1189 - acc: 0.9993 - val_loss: 0.2925 - val_acc: 0.9548Epoch 500/500133s 266ms/step - loss: 0.1197 - acc: 0.9991 - val_loss: 0.2927 - val_acc: 0.9547Train loss: 0.11755225303769111Train accuracy: 0.9996800003051758Test loss: 0.29267876625061034Test accuracy: 0.9547000050544738比调参记录21的95.12%高了一点。怎么样能够突破96%呢？
Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht, Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, IEEE Transactions on Industrial Electronics, 2020, DOI: 10.1109/TIE.2020.2972458
https://ieeexplore.ieee.org/d...