Adaptively Parametric ReLU是一种动态激活函数，对每个输入样本执行不同的操作，在2019年5月3日投稿至IEEE Transactions on Industrial Electronics，2020年1月24日录用，2020年2月13日在IEEE官网公布。
由于调参记录18依然存在过拟合，本文将Adaptively Parametric ReLU激活函数中最后一层的神经元个数减少为1个，继续测试深度残差网络+Adaptively Parametric ReLU激活函数在Cifar10上的效果。
同时，迭代次数从调参记录18中的5000个epoch，减少到了500个epoch，因为5000次实在是太费时间了，差不多要四天才能跑完。
Adaptively Parametric ReLU激活函数的基本原理如下：
Keras程序如下：
#!/usr/bin/env python3# -*- coding: utf-8 -*-"""Created on Tue Apr 14 04:17:45 2020Implemented using TensorFlow 1.0.1 and Keras 2.2.1Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht,Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, IEEE Transactions on Industrial Electronics, 2020, DOI: 10.1109/TIE.2020.2972458,Date of Publication: 13 February 2020@author: Minghang Zhao"""from __future__ import print_functionimport kerasimport numpy as npfrom keras.datasets import cifar10from keras.layers import Dense, Conv2D, BatchNormalization, Activation, Minimumfrom keras.layers import AveragePooling2D, Input, GlobalAveragePooling2D, Concatenate, Reshapefrom keras.regularizers import l2from keras import backend as Kfrom keras.models import Modelfrom keras import optimizersfrom keras.preprocessing.image import ImageDataGeneratorfrom keras.callbacks import LearningRateSchedulerK.set_learning_phase(1)# The data, split between train and test sets(x_train, y_train), (x_test, y_test) = cifar10.load_data()x_train = x_train.astype('float32') / 255.x_test = x_test.astype('float32') / 255.x_test = x_test-np.mean(x_train)x_train = x_train-np.mean(x_train)print('x_train shape:', x_train.shape)print(x_train.shape[0], 'train samples')print(x_test.shape[0], 'test samples')# convert class vectors to binary class matricesy_train = keras.utils.to_categorical(y_train, 10)y_test = keras.utils.to_categorical(y_test, 10)# Schedule the learning rate, multiply 0.1 every 150 epochesdef scheduler(epoch): if epoch % 150 == 0 and epoch != 0: lr = K.get_value(model.optimizer.lr) K.set_value(model.optimizer.lr, lr * 0.1) print("lr changed to {}".format(lr * 0.1)) return K.get_value(model.optimizer.lr)# An adaptively parametric rectifier linear unit (APReLU)def aprelu(inputs): # get the number of channels channels = inputs.get_shape().as_list()[-1] # get a zero feature map zeros_input = keras.layers.subtract([inputs, inputs]) # get a feature map with only positive features pos_input = Activation('relu')(inputs) # get a feature map with only negative features neg_input = Minimum()([inputs,zeros_input]) # define a network to obtain the scaling coefficients scales_p = GlobalAveragePooling2D()(pos_input) scales_n = GlobalAveragePooling2D()(neg_input) scales = Concatenate()([scales_n, scales_p]) scales = Dense(channels//16, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales) scales = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(scales) scales = Activation('relu')(scales) scales = Dense(1, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales) scales = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(scales) scales = Activation('sigmoid')(scales) scales = Reshape((1,1,1))(scales) # apply a paramtetric relu neg_part = keras.layers.multiply([scales, neg_input]) return keras.layers.add([pos_input, neg_part])# Residual Blockdef residual_block(incoming, nb_blocks, out_channels, downsample=False, downsample_strides=2): residual = incoming in_channels = incoming.get_shape().as_list()[-1] for i in range(nb_blocks): identity = residual if not downsample: downsample_strides = 1 residual = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(residual) residual = aprelu(residual) residual = Conv2D(out_channels, 3, strides=(downsample_strides, downsample_strides), padding='same', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(residual) residual = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(residual) residual = aprelu(residual) residual = Conv2D(out_channels, 3, padding='same', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(residual) # Downsampling if downsample_strides > 1: identity = AveragePooling2D(pool_size=(1,1), strides=(2,2))(identity) # Zero_padding to match channels if in_channels != out_channels: zeros_identity = keras.layers.subtract([identity, identity]) identity = keras.layers.concatenate([identity, zeros_identity]) in_channels = out_channels residual = keras.layers.add([residual, identity]) return residual# define and train a modelinputs = Input(shape=(32, 32, 3))net = Conv2D(16, 3, padding='same', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(inputs)net = residual_block(net, 9, 32, downsample=False)net = residual_block(net, 1, 32, downsample=True)net = residual_block(net, 8, 32, downsample=False)net = residual_block(net, 1, 64, downsample=True)net = residual_block(net, 8, 64, downsample=False)net = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(net)net = aprelu(net)net = GlobalAveragePooling2D()(net)outputs = Dense(10, activation='softmax', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(net)model = Model(inputs=inputs, outputs=outputs)sgd = optimizers.SGD(lr=0.1, decay=0., momentum=0.9, nesterov=True)model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])# data augmentationdatagen = ImageDataGenerator( # randomly rotate images in the range (deg 0 to 180) rotation_range=30, # Range for random zoom zoom_range = 0.2, # shear angle in counter-clockwise direction in degrees shear_range = 30, # randomly flip images horizontal_flip=True, # randomly shift images horizontally width_shift_range=0.125, # randomly shift images vertically height_shift_range=0.125)reduce_lr = LearningRateScheduler(scheduler)# fit the model on the batches generated by datagen.flow().model.fit_generator(datagen.flow(x_train, y_train, batch_size=100), validation_data=(x_test, y_test), epochs=500, verbose=1, callbacks=[reduce_lr], workers=4)# get resultsK.set_learning_phase(0)DRSN_train_score = model.evaluate(x_train, y_train, batch_size=100, verbose=0)print('Train loss:', DRSN_train_score[0])print('Train accuracy:', DRSN_train_score[1])DRSN_test_score = model.evaluate(x_test, y_test, batch_size=100, verbose=0)print('Test loss:', DRSN_test_score[0])print('Test accuracy:', DRSN_test_score[1])实验结果如下：
Using TensorFlow backend.x_train shape: (50000, 32, 32, 3)50000 train samples10000 test samplesEpoch 1/500107s 215ms/step - loss: 2.3702 - acc: 0.3922 - val_loss: 1.9601 - val_acc: 0.5235Epoch 2/50077s 154ms/step - loss: 1.9532 - acc: 0.5157 - val_loss: 1.6734 - val_acc: 0.5998Epoch 3/50077s 154ms/step - loss: 1.6989 - acc: 0.5797 - val_loss: 1.4728 - val_acc: 0.6495Epoch 4/50077s 154ms/step - loss: 1.5366 - acc: 0.6184 - val_loss: 1.3253 - val_acc: 0.6888Epoch 5/50077s 154ms/step - loss: 1.4110 - acc: 0.6444 - val_loss: 1.2022 - val_acc: 0.7197Epoch 6/50077s 154ms/step - loss: 1.3059 - acc: 0.6707 - val_loss: 1.1398 - val_acc: 0.7236Epoch 7/50077s 154ms/step - loss: 1.2295 - acc: 0.6873 - val_loss: 1.0509 - val_acc: 0.7515Epoch 8/50077s 154ms/step - loss: 1.1568 - acc: 0.7041 - val_loss: 0.9907 - val_acc: 0.7686Epoch 9/50077s 154ms/step - loss: 1.1016 - acc: 0.7207 - val_loss: 0.9470 - val_acc: 0.7863Epoch 10/50077s 154ms/step - loss: 1.0521 - acc: 0.7346 - val_loss: 0.9005 - val_acc: 0.7911Epoch 11/50077s 154ms/step - loss: 1.0246 - acc: 0.7423 - val_loss: 0.8991 - val_acc: 0.7881Epoch 12/50077s 154ms/step - loss: 0.9941 - acc: 0.7506 - val_loss: 0.8390 - val_acc: 0.8093Epoch 13/50077s 154ms/step - loss: 0.9642 - acc: 0.7602 - val_loss: 0.8239 - val_acc: 0.8147Epoch 14/50077s 154ms/step - loss: 0.9465 - acc: 0.7652 - val_loss: 0.8057 - val_acc: 0.8170Epoch 15/50077s 154ms/step - loss: 0.9296 - acc: 0.7701 - val_loss: 0.8180 - val_acc: 0.8114Epoch 16/50077s 154ms/step - loss: 0.9103 - acc: 0.7767 - val_loss: 0.7975 - val_acc: 0.8207Epoch 17/50077s 154ms/step - loss: 0.9027 - acc: 0.7801 - val_loss: 0.8048 - val_acc: 0.8186Epoch 18/50077s 154ms/step - loss: 0.8904 - acc: 0.7848 - val_loss: 0.7542 - val_acc: 0.8376Epoch 19/50077s 154ms/step - loss: 0.8765 - acc: 0.7889 - val_loss: 0.7633 - val_acc: 0.8313Epoch 20/50077s 154ms/step - loss: 0.8739 - acc: 0.7913 - val_loss: 0.7411 - val_acc: 0.8432Epoch 21/50077s 154ms/step - loss: 0.8587 - acc: 0.7976 - val_loss: 0.7357 - val_acc: 0.8466Epoch 22/50077s 154ms/step - loss: 0.8505 - acc: 0.7982 - val_loss: 0.7369 - val_acc: 0.8437Epoch 23/50077s 154ms/step - loss: 0.8495 - acc: 0.8014 - val_loss: 0.7507 - val_acc: 0.8415Epoch 24/50077s 154ms/step - loss: 0.8382 - acc: 0.8070 - val_loss: 0.7494 - val_acc: 0.8423Epoch 25/50077s 154ms/step - loss: 0.8339 - acc: 0.8097 - val_loss: 0.7374 - val_acc: 0.8441Epoch 26/50077s 154ms/step - loss: 0.8284 - acc: 0.8105 - val_loss: 0.7195 - val_acc: 0.8517Epoch 27/50077s 154ms/step - loss: 0.8244 - acc: 0.8139 - val_loss: 0.7054 - val_acc: 0.8611Epoch 28/50077s 154ms/step - loss: 0.8242 - acc: 0.8143 - val_loss: 0.6997 - val_acc: 0.8614Epoch 29/50077s 154ms/step - loss: 0.8145 - acc: 0.8186 - val_loss: 0.6966 - val_acc: 0.8598Epoch 30/50077s 154ms/step - loss: 0.8092 - acc: 0.8197 - val_loss: 0.7344 - val_acc: 0.8498Epoch 31/50077s 154ms/step - loss: 0.8048 - acc: 0.8219 - val_loss: 0.7232 - val_acc: 0.8574Epoch 32/50077s 154ms/step - loss: 0.8054 - acc: 0.8244 - val_loss: 0.6888 - val_acc: 0.8652Epoch 33/50077s 154ms/step - loss: 0.8000 - acc: 0.8231 - val_loss: 0.7236 - val_acc: 0.8533Epoch 34/50077s 154ms/step - loss: 0.7994 - acc: 0.8258 - val_loss: 0.7096 - val_acc: 0.8584Epoch 35/50077s 154ms/step - loss: 0.7933 - acc: 0.8291 - val_loss: 0.7063 - val_acc: 0.8602Epoch 36/50077s 154ms/step - loss: 0.7955 - acc: 0.8275 - val_loss: 0.7124 - val_acc: 0.8599Epoch 37/50077s 154ms/step - loss: 0.7961 - acc: 0.8280 - val_loss: 0.7020 - val_acc: 0.8650Epoch 38/50077s 154ms/step - loss: 0.7864 - acc: 0.8332 - val_loss: 0.7201 - val_acc: 0.8573Epoch 39/50077s 154ms/step - loss: 0.7949 - acc: 0.8303 - val_loss: 0.7009 - val_acc: 0.8648Epoch 40/50077s 154ms/step - loss: 0.7781 - acc: 0.8349 - val_loss: 0.6954 - val_acc: 0.8636Epoch 41/50077s 154ms/step - loss: 0.7821 - acc: 0.8352 - val_loss: 0.6819 - val_acc: 0.8736Epoch 42/50077s 154ms/step - loss: 0.7805 - acc: 0.8345 - val_loss: 0.7347 - val_acc: 0.8550Epoch 43/50077s 154ms/step - loss: 0.7749 - acc: 0.8384 - val_loss: 0.7029 - val_acc: 0.8642Epoch 44/50077s 154ms/step - loss: 0.7777 - acc: 0.8368 - val_loss: 0.6967 - val_acc: 0.8676Epoch 45/50077s 154ms/step - loss: 0.7725 - acc: 0.8393 - val_loss: 0.6867 - val_acc: 0.8722Epoch 46/50077s 154ms/step - loss: 0.7737 - acc: 0.8408 - val_loss: 0.7075 - val_acc: 0.8644Epoch 47/50077s 154ms/step - loss: 0.7734 - acc: 0.8395 - val_loss: 0.6958 - val_acc: 0.8667Epoch 48/50077s 154ms/step - loss: 0.7750 - acc: 0.8404 - val_loss: 0.6956 - val_acc: 0.8701Epoch 49/50077s 154ms/step - loss: 0.7691 - acc: 0.8417 - val_loss: 0.6977 - val_acc: 0.8677Epoch 50/50077s 154ms/step - loss: 0.7661 - acc: 0.8433 - val_loss: 0.7094 - val_acc: 0.8683Epoch 51/50077s 154ms/step - loss: 0.7638 - acc: 0.8469 - val_loss: 0.6972 - val_acc: 0.8678Epoch 52/50077s 154ms/step - loss: 0.7613 - acc: 0.8455 - val_loss: 0.7113 - val_acc: 0.8676Epoch 53/50077s 154ms/step - loss: 0.7647 - acc: 0.8460 - val_loss: 0.6946 - val_acc: 0.8692Epoch 54/50077s 154ms/step - loss: 0.7572 - acc: 0.8468 - val_loss: 0.7242 - val_acc: 0.8628Epoch 55/50077s 154ms/step - loss: 0.7560 - acc: 0.8504 - val_loss: 0.7084 - val_acc: 0.8671Epoch 56/50077s 154ms/step - loss: 0.7578 - acc: 0.8473 - val_loss: 0.6979 - val_acc: 0.8724Epoch 57/50077s 154ms/step - loss: 0.7635 - acc: 0.8468 - val_loss: 0.6928 - val_acc: 0.8722Epoch 58/50077s 154ms/step - loss: 0.7563 - acc: 0.8489 - val_loss: 0.6907 - val_acc: 0.8736Epoch 59/50077s 154ms/step - loss: 0.7578 - acc: 0.8495 - val_loss: 0.6854 - val_acc: 0.8757Epoch 60/50077s 154ms/step - loss: 0.7565 - acc: 0.8482 - val_loss: 0.6837 - val_acc: 0.8743Epoch 61/50077s 154ms/step - loss: 0.7570 - acc: 0.8499 - val_loss: 0.6821 - val_acc: 0.8742Epoch 62/50077s 154ms/step - loss: 0.7595 - acc: 0.8484 - val_loss: 0.6889 - val_acc: 0.8722Epoch 63/50077s 154ms/step - loss: 0.7536 - acc: 0.8512 - val_loss: 0.6748 - val_acc: 0.8800Epoch 64/50077s 154ms/step - loss: 0.7539 - acc: 0.8514 - val_loss: 0.6508 - val_acc: 0.8901Epoch 65/50077s 154ms/step - loss: 0.7483 - acc: 0.8535 - val_loss: 0.6852 - val_acc: 0.8777Epoch 66/50077s 154ms/step - loss: 0.7496 - acc: 0.8535 - val_loss: 0.6940 - val_acc: 0.8756Epoch 67/50077s 154ms/step - loss: 0.7568 - acc: 0.8505 - val_loss: 0.6830 - val_acc: 0.8805Epoch 68/50077s 154ms/step - loss: 0.7549 - acc: 0.8508 - val_loss: 0.6732 - val_acc: 0.8840Epoch 69/50077s 154ms/step - loss: 0.7479 - acc: 0.8549 - val_loss: 0.6955 - val_acc: 0.8744Epoch 70/50077s 154ms/step - loss: 0.7468 - acc: 0.8551 - val_loss: 0.6964 - val_acc: 0.8746Epoch 71/50077s 154ms/step - loss: 0.7499 - acc: 0.8553 - val_loss: 0.6850 - val_acc: 0.8784Epoch 72/50077s 154ms/step - loss: 0.7462 - acc: 0.8553 - val_loss: 0.6937 - val_acc: 0.8771Epoch 73/50077s 154ms/step - loss: 0.7467 - acc: 0.8559 - val_loss: 0.6876 - val_acc: 0.8761Epoch 74/50077s 154ms/step - loss: 0.7467 - acc: 0.8559 - val_loss: 0.7029 - val_acc: 0.8715Epoch 75/50077s 154ms/step - loss: 0.7435 - acc: 0.8561 - val_loss: 0.7184 - val_acc: 0.8663Epoch 76/50077s 154ms/step - loss: 0.7467 - acc: 0.8558 - val_loss: 0.6751 - val_acc: 0.8808Epoch 77/50077s 154ms/step - loss: 0.7398 - acc: 0.8575 - val_loss: 0.6843 - val_acc: 0.8812Epoch 78/50077s 154ms/step - loss: 0.7463 - acc: 0.8571 - val_loss: 0.6802 - val_acc: 0.8800Epoch 79/50077s 154ms/step - loss: 0.7395 - acc: 0.8568 - val_loss: 0.6877 - val_acc: 0.8769Epoch 80/50077s 154ms/step - loss: 0.7403 - acc: 0.8580 - val_loss: 0.6912 - val_acc: 0.8792Epoch 81/50077s 154ms/step - loss: 0.7429 - acc: 0.8555 - val_loss: 0.6887 - val_acc: 0.8787Epoch 82/50077s 154ms/step - loss: 0.7408 - acc: 0.8572 - val_loss: 0.7134 - val_acc: 0.8709Epoch 83/50077s 154ms/step - loss: 0.7413 - acc: 0.8573 - val_loss: 0.6921 - val_acc: 0.8776Epoch 84/50077s 154ms/step - loss: 0.7393 - acc: 0.8588 - val_loss: 0.6965 - val_acc: 0.8737Epoch 85/50077s 154ms/step - loss: 0.7440 - acc: 0.8568 - val_loss: 0.6806 - val_acc: 0.8803Epoch 86/50077s 154ms/step - loss: 0.7407 - acc: 0.8589 - val_loss: 0.6658 - val_acc: 0.8871Epoch 87/50077s 154ms/step - loss: 0.7366 - acc: 0.8587 - val_loss: 0.6804 - val_acc: 0.8812Epoch 88/50077s 154ms/step - loss: 0.7406 - acc: 0.8582 - val_loss: 0.6686 - val_acc: 0.8869Epoch 89/50077s 154ms/step - loss: 0.7345 - acc: 0.8611 - val_loss: 0.6744 - val_acc: 0.8836Epoch 90/50077s 154ms/step - loss: 0.7318 - acc: 0.8614 - val_loss: 0.6715 - val_acc: 0.8852Epoch 91/50077s 154ms/step - loss: 0.7376 - acc: 0.8600 - val_loss: 0.6939 - val_acc: 0.8737Epoch 92/50077s 154ms/step - loss: 0.7420 - acc: 0.8586 - val_loss: 0.6890 - val_acc: 0.8763Epoch 93/50077s 154ms/step - loss: 0.7315 - acc: 0.8631 - val_loss: 0.6761 - val_acc: 0.8821Epoch 94/50077s 154ms/step - loss: 0.7341 - acc: 0.8610 - val_loss: 0.6902 - val_acc: 0.8801Epoch 95/50077s 154ms/step - loss: 0.7370 - acc: 0.8604 - val_loss: 0.6938 - val_acc: 0.8742Epoch 96/50077s 154ms/step - loss: 0.7345 - acc: 0.8619 - val_loss: 0.6785 - val_acc: 0.8803Epoch 97/50077s 154ms/step - loss: 0.7356 - acc: 0.8598 - val_loss: 0.6974 - val_acc: 0.8753Epoch 98/50077s 154ms/step - loss: 0.7340 - acc: 0.8622 - val_loss: 0.6847 - val_acc: 0.8821Epoch 99/50077s 154ms/step - loss: 0.7321 - acc: 0.8632 - val_loss: 0.6772 - val_acc: 0.8883Epoch 100/50077s 154ms/step - loss: 0.7301 - acc: 0.8650 - val_loss: 0.6659 - val_acc: 0.8881Epoch 101/50077s 154ms/step - loss: 0.7364 - acc: 0.8625 - val_loss: 0.7062 - val_acc: 0.8735Epoch 102/50077s 154ms/step - loss: 0.7360 - acc: 0.8613 - val_loss: 0.6749 - val_acc: 0.8819Epoch 103/50077s 154ms/step - loss: 0.7305 - acc: 0.8628 - val_loss: 0.6853 - val_acc: 0.8840Epoch 104/50077s 154ms/step - loss: 0.7333 - acc: 0.8638 - val_loss: 0.6813 - val_acc: 0.8800Epoch 105/50077s 154ms/step - loss: 0.7308 - acc: 0.8631 - val_loss: 0.6599 - val_acc: 0.8892Epoch 106/50077s 154ms/step - loss: 0.7355 - acc: 0.8643 - val_loss: 0.6833 - val_acc: 0.8816Epoch 107/50077s 154ms/step - loss: 0.7286 - acc: 0.8654 - val_loss: 0.6744 - val_acc: 0.8830Epoch 108/50077s 154ms/step - loss: 0.7278 - acc: 0.8653 - val_loss: 0.6870 - val_acc: 0.8807Epoch 109/50077s 154ms/step - loss: 0.7270 - acc: 0.8652 - val_loss: 0.6901 - val_acc: 0.8821Epoch 110/50077s 154ms/step - loss: 0.7260 - acc: 0.8646 - val_loss: 0.6908 - val_acc: 0.8820Epoch 111/50077s 154ms/step - loss: 0.7290 - acc: 0.8645 - val_loss: 0.6973 - val_acc: 0.8755Epoch 112/50077s 154ms/step - loss: 0.7336 - acc: 0.8615 - val_loss: 0.6845 - val_acc: 0.8812Epoch 113/50077s 154ms/step - loss: 0.7296 - acc: 0.8635 - val_loss: 0.6835 - val_acc: 0.8811Epoch 114/50077s 154ms/step - loss: 0.7310 - acc: 0.8647 - val_loss: 0.6822 - val_acc: 0.8820Epoch 115/50077s 154ms/step - loss: 0.7251 - acc: 0.8660 - val_loss: 0.6822 - val_acc: 0.8803Epoch 116/50077s 154ms/step - loss: 0.7313 - acc: 0.8633 - val_loss: 0.6572 - val_acc: 0.8908Epoch 117/50077s 154ms/step - loss: 0.7289 - acc: 0.8636 - val_loss: 0.6956 - val_acc: 0.8817Epoch 118/50077s 154ms/step - loss: 0.7233 - acc: 0.8670 - val_loss: 0.7052 - val_acc: 0.8738Epoch 119/50077s 154ms/step - loss: 0.7243 - acc: 0.8667 - val_loss: 0.6675 - val_acc: 0.8891Epoch 120/50077s 154ms/step - loss: 0.7269 - acc: 0.8658 - val_loss: 0.6815 - val_acc: 0.8834Epoch 121/50077s 154ms/step - loss: 0.7248 - acc: 0.8656 - val_loss: 0.6670 - val_acc: 0.8878Epoch 122/50077s 154ms/step - loss: 0.7223 - acc: 0.8690 - val_loss: 0.6658 - val_acc: 0.8892Epoch 123/50077s 154ms/step - loss: 0.7248 - acc: 0.8675 - val_loss: 0.6889 - val_acc: 0.8798Epoch 124/50077s 154ms/step - loss: 0.7209 - acc: 0.8675 - val_loss: 0.6703 - val_acc: 0.8857Epoch 125/50077s 154ms/step - loss: 0.7276 - acc: 0.8668 - val_loss: 0.6875 - val_acc: 0.8791Epoch 126/50077s 154ms/step - loss: 0.7251 - acc: 0.8659 - val_loss: 0.6836 - val_acc: 0.8829Epoch 127/50077s 154ms/step - loss: 0.7280 - acc: 0.8668 - val_loss: 0.6832 - val_acc: 0.8836Epoch 128/50077s 154ms/step - loss: 0.7242 - acc: 0.8672 - val_loss: 0.6848 - val_acc: 0.8847Epoch 129/50077s 154ms/step - loss: 0.7267 - acc: 0.8663 - val_loss: 0.6778 - val_acc: 0.8852Epoch 130/50077s 154ms/step - loss: 0.7289 - acc: 0.8648 - val_loss: 0.6786 - val_acc: 0.8837Epoch 131/50077s 154ms/step - loss: 0.7219 - acc: 0.8685 - val_loss: 0.6562 - val_acc: 0.8899Epoch 132/50077s 154ms/step - loss: 0.7186 - acc: 0.8678 - val_loss: 0.6765 - val_acc: 0.8854Epoch 133/50077s 154ms/step - loss: 0.7199 - acc: 0.8688 - val_loss: 0.6697 - val_acc: 0.8887Epoch 134/50077s 154ms/step - loss: 0.7163 - acc: 0.8687 - val_loss: 0.6692 - val_acc: 0.8881Epoch 135/50077s 154ms/step - loss: 0.7208 - acc: 0.8671 - val_loss: 0.6777 - val_acc: 0.8818Epoch 136/50077s 154ms/step - loss: 0.7257 - acc: 0.8666 - val_loss: 0.6726 - val_acc: 0.8896Epoch 137/50077s 154ms/step - loss: 0.7224 - acc: 0.8658 - val_loss: 0.7068 - val_acc: 0.8746Epoch 138/50077s 154ms/step - loss: 0.7202 - acc: 0.8686 - val_loss: 0.6746 - val_acc: 0.8850Epoch 139/50077s 154ms/step - loss: 0.7253 - acc: 0.8672 - val_loss: 0.6856 - val_acc: 0.8843Epoch 140/50077s 154ms/step - loss: 0.7216 - acc: 0.8681 - val_loss: 0.6837 - val_acc: 0.8835Epoch 141/50077s 154ms/step - loss: 0.7251 - acc: 0.8686 - val_loss: 0.6652 - val_acc: 0.8893Epoch 142/50077s 154ms/step - loss: 0.7200 - acc: 0.8697 - val_loss: 0.6572 - val_acc: 0.8915Epoch 143/50077s 154ms/step - loss: 0.7208 - acc: 0.8682 - val_loss: 0.6792 - val_acc: 0.8858Epoch 144/50077s 154ms/step - loss: 0.7231 - acc: 0.8691 - val_loss: 0.6885 - val_acc: 0.8835Epoch 145/50077s 154ms/step - loss: 0.7191 - acc: 0.8704 - val_loss: 0.6828 - val_acc: 0.8862Epoch 146/50077s 153ms/step - loss: 0.7209 - acc: 0.8689 - val_loss: 0.6849 - val_acc: 0.8812Epoch 147/50077s 154ms/step - loss: 0.7243 - acc: 0.8688 - val_loss: 0.6824 - val_acc: 0.8838Epoch 148/50077s 154ms/step - loss: 0.7194 - acc: 0.8700 - val_loss: 0.6714 - val_acc: 0.8889Epoch 149/50077s 154ms/step - loss: 0.7220 - acc: 0.8691 - val_loss: 0.6686 - val_acc: 0.8902Epoch 150/50077s 154ms/step - loss: 0.7181 - acc: 0.8700 - val_loss: 0.6723 - val_acc: 0.8851Epoch 151/500lr changed to 0.01000000014901161277s 154ms/step - loss: 0.6046 - acc: 0.9093 - val_loss: 0.5729 - val_acc: 0.9191Epoch 152/50077s 154ms/step - loss: 0.5434 - acc: 0.9281 - val_loss: 0.5547 - val_acc: 0.9222Epoch 153/50077s 154ms/step - loss: 0.5269 - acc: 0.9317 - val_loss: 0.5470 - val_acc: 0.9232Epoch 154/50077s 154ms/step - loss: 0.5083 - acc: 0.9357 - val_loss: 0.5377 - val_acc: 0.9255Epoch 155/50077s 154ms/step - loss: 0.4961 - acc: 0.9395 - val_loss: 0.5305 - val_acc: 0.9254Epoch 156/50077s 154ms/step - loss: 0.4827 - acc: 0.9411 - val_loss: 0.5238 - val_acc: 0.9269Epoch 157/50077s 154ms/step - loss: 0.4718 - acc: 0.9440 - val_loss: 0.5187 - val_acc: 0.9289Epoch 158/50077s 154ms/step - loss: 0.4637 - acc: 0.9443 - val_loss: 0.5135 - val_acc: 0.9290Epoch 159/50077s 154ms/step - loss: 0.4554 - acc: 0.9453 - val_loss: 0.5119 - val_acc: 0.9291Epoch 160/50077s 154ms/step - loss: 0.4475 - acc: 0.9456 - val_loss: 0.5078 - val_acc: 0.9271Epoch 161/50077s 154ms/step - loss: 0.4393 - acc: 0.9484 - val_loss: 0.4957 - val_acc: 0.9317Epoch 162/50077s 154ms/step - loss: 0.4290 - acc: 0.9491 - val_loss: 0.4937 - val_acc: 0.9283Epoch 163/50077s 154ms/step - loss: 0.4224 - acc: 0.9501 - val_loss: 0.4897 - val_acc: 0.9293Epoch 164/50077s 154ms/step - loss: 0.4194 - acc: 0.9498 - val_loss: 0.4830 - val_acc: 0.9312Epoch 165/50077s 154ms/step - loss: 0.4101 - acc: 0.9529 - val_loss: 0.4823 - val_acc: 0.9309Epoch 166/50077s 154ms/step - loss: 0.4087 - acc: 0.9508 - val_loss: 0.4761 - val_acc: 0.9302Epoch 167/50077s 154ms/step - loss: 0.3993 - acc: 0.9528 - val_loss: 0.4733 - val_acc: 0.9307Epoch 168/50077s 154ms/step - loss: 0.3958 - acc: 0.9528 - val_loss: 0.4612 - val_acc: 0.9310Epoch 169/50077s 154ms/step - loss: 0.3904 - acc: 0.9536 - val_loss: 0.4725 - val_acc: 0.9294Epoch 170/50077s 154ms/step - loss: 0.3820 - acc: 0.9552 - val_loss: 0.4625 - val_acc: 0.9293Epoch 171/50077s 154ms/step - loss: 0.3769 - acc: 0.9553 - val_loss: 0.4596 - val_acc: 0.9292Epoch 172/50077s 154ms/step - loss: 0.3732 - acc: 0.9567 - val_loss: 0.4686 - val_acc: 0.9271Epoch 173/50077s 154ms/step - loss: 0.3692 - acc: 0.9566 - val_loss: 0.4595 - val_acc: 0.9275Epoch 174/50077s 154ms/step - loss: 0.3697 - acc: 0.9547 - val_loss: 0.4510 - val_acc: 0.9305Epoch 175/50077s 154ms/step - loss: 0.3592 - acc: 0.9577 - val_loss: 0.4485 - val_acc: 0.9294Epoch 176/50077s 154ms/step - loss: 0.3553 - acc: 0.9583 - val_loss: 0.4527 - val_acc: 0.9276Epoch 177/50077s 154ms/step - loss: 0.3519 - acc: 0.9588 - val_loss: 0.4501 - val_acc: 0.9269Epoch 178/50077s 154ms/step - loss: 0.3508 - acc: 0.9571 - val_loss: 0.4489 - val_acc: 0.9253Epoch 179/50077s 154ms/step - loss: 0.3461 - acc: 0.9577 - val_loss: 0.4484 - val_acc: 0.9260Epoch 180/50077s 154ms/step - loss: 0.3446 - acc: 0.9583 - val_loss: 0.4392 - val_acc: 0.9274Epoch 181/50077s 154ms/step - loss: 0.3375 - acc: 0.9591 - val_loss: 0.4435 - val_acc: 0.9287Epoch 182/50077s 154ms/step - loss: 0.3375 - acc: 0.9584 - val_loss: 0.4446 - val_acc: 0.9278Epoch 183/50077s 154ms/step - loss: 0.3358 - acc: 0.9586 - val_loss: 0.4434 - val_acc: 0.9268Epoch 184/50077s 154ms/step - loss: 0.3294 - acc: 0.9607 - val_loss: 0.4529 - val_acc: 0.9267Epoch 185/50077s 154ms/step - loss: 0.3352 - acc: 0.9571 - val_loss: 0.4392 - val_acc: 0.9272Epoch 186/50077s 154ms/step - loss: 0.3289 - acc: 0.9587 - val_loss: 0.4367 - val_acc: 0.9276Epoch 187/50077s 154ms/step - loss: 0.3267 - acc: 0.9595 - val_loss: 0.4333 - val_acc: 0.9257Epoch 188/50077s 154ms/step - loss: 0.3191 - acc: 0.9600 - val_loss: 0.4392 - val_acc: 0.9257Epoch 189/50077s 154ms/step - loss: 0.3169 - acc: 0.9608 - val_loss: 0.4366 - val_acc: 0.9261Epoch 190/50077s 154ms/step - loss: 0.3180 - acc: 0.9594 - val_loss: 0.4283 - val_acc: 0.9274Epoch 191/50077s 154ms/step - loss: 0.3128 - acc: 0.9605 - val_loss: 0.4351 - val_acc: 0.9228Epoch 192/50077s 154ms/step - loss: 0.3105 - acc: 0.9610 - val_loss: 0.4294 - val_acc: 0.9255Epoch 193/50077s 154ms/step - loss: 0.3096 - acc: 0.9605 - val_loss: 0.4258 - val_acc: 0.9272Epoch 194/50077s 154ms/step - loss: 0.3074 - acc: 0.9614 - val_loss: 0.4288 - val_acc: 0.9248Epoch 195/50077s 154ms/step - loss: 0.3120 - acc: 0.9587 - val_loss: 0.4296 - val_acc: 0.9237Epoch 196/50077s 154ms/step - loss: 0.3029 - acc: 0.9617 - val_loss: 0.4240 - val_acc: 0.9248Epoch 197/50077s 154ms/step - loss: 0.3020 - acc: 0.9620 - val_loss: 0.4250 - val_acc: 0.9236Epoch 198/50077s 154ms/step - loss: 0.2999 - acc: 0.9615 - val_loss: 0.4228 - val_acc: 0.9248Epoch 199/50077s 154ms/step - loss: 0.3062 - acc: 0.9591 - val_loss: 0.4214 - val_acc: 0.9238Epoch 200/50077s 154ms/step - loss: 0.2965 - acc: 0.9610 - val_loss: 0.4208 - val_acc: 0.9223Epoch 201/50077s 154ms/step - loss: 0.2991 - acc: 0.9598 - val_loss: 0.4235 - val_acc: 0.9234Epoch 202/50077s 154ms/step - loss: 0.2970 - acc: 0.9607 - val_loss: 0.4145 - val_acc: 0.9254Epoch 203/50077s 154ms/step - loss: 0.2957 - acc: 0.9615 - val_loss: 0.4259 - val_acc: 0.9258Epoch 204/50077s 154ms/step - loss: 0.2985 - acc: 0.9593 - val_loss: 0.4215 - val_acc: 0.9255Epoch 205/50077s 154ms/step - loss: 0.2997 - acc: 0.9586 - val_loss: 0.4152 - val_acc: 0.9226Epoch 206/50077s 154ms/step - loss: 0.2937 - acc: 0.9603 - val_loss: 0.4019 - val_acc: 0.9318Epoch 207/50077s 154ms/step - loss: 0.2948 - acc: 0.9596 - val_loss: 0.4118 - val_acc: 0.9257Epoch 208/50077s 154ms/step - loss: 0.2952 - acc: 0.9597 - val_loss: 0.4051 - val_acc: 0.9306Epoch 209/50077s 154ms/step - loss: 0.2870 - acc: 0.9616 - val_loss: 0.4115 - val_acc: 0.9262Epoch 210/50077s 154ms/step - loss: 0.2926 - acc: 0.9596 - val_loss: 0.4055 - val_acc: 0.9272Epoch 211/50077s 154ms/step - loss: 0.2872 - acc: 0.9613 - val_loss: 0.4165 - val_acc: 0.9229Epoch 212/50077s 154ms/step - loss: 0.2909 - acc: 0.9597 - val_loss: 0.4018 - val_acc: 0.9249Epoch 213/50077s 154ms/step - loss: 0.2857 - acc: 0.9614 - val_loss: 0.4119 - val_acc: 0.9219Epoch 214/50077s 154ms/step - loss: 0.2858 - acc: 0.9603 - val_loss: 0.4023 - val_acc: 0.9258Epoch 215/50077s 154ms/step - loss: 0.2858 - acc: 0.9609 - val_loss: 0.4176 - val_acc: 0.9231Epoch 216/50077s 154ms/step - loss: 0.2861 - acc: 0.9601 - val_loss: 0.4137 - val_acc: 0.9246Epoch 217/50077s 154ms/step - loss: 0.2869 - acc: 0.9604 - val_loss: 0.4088 - val_acc: 0.9245Epoch 218/50077s 154ms/step - loss: 0.2828 - acc: 0.9609 - val_loss: 0.4092 - val_acc: 0.9234Epoch 219/50077s 154ms/step - loss: 0.2807 - acc: 0.9616 - val_loss: 0.4026 - val_acc: 0.9278Epoch 220/50077s 154ms/step - loss: 0.2810 - acc: 0.9608 - val_loss: 0.4045 - val_acc: 0.9275Epoch 221/50077s 154ms/step - loss: 0.2804 - acc: 0.9612 - val_loss: 0.4012 - val_acc: 0.9247Epoch 222/50077s 154ms/step - loss: 0.2819 - acc: 0.9588 - val_loss: 0.4046 - val_acc: 0.9219Epoch 223/50077s 154ms/step - loss: 0.2805 - acc: 0.9599 - val_loss: 0.4007 - val_acc: 0.9247Epoch 224/50077s 154ms/step - loss: 0.2785 - acc: 0.9608 - val_loss: 0.4117 - val_acc: 0.9224Epoch 225/50077s 154ms/step - loss: 0.2783 - acc: 0.9610 - val_loss: 0.4073 - val_acc: 0.9204Epoch 226/50077s 154ms/step - loss: 0.2830 - acc: 0.9599 - val_loss: 0.4135 - val_acc: 0.9203Epoch 227/50077s 154ms/step - loss: 0.2798 - acc: 0.9601 - val_loss: 0.3977 - val_acc: 0.9254Epoch 228/50077s 154ms/step - loss: 0.2780 - acc: 0.9602 - val_loss: 0.3916 - val_acc: 0.9254Epoch 229/50077s 154ms/step - loss: 0.2812 - acc: 0.9589 - val_loss: 0.4020 - val_acc: 0.9254Epoch 230/50077s 154ms/step - loss: 0.2786 - acc: 0.9592 - val_loss: 0.3981 - val_acc: 0.9258Epoch 231/50077s 154ms/step - loss: 0.2787 - acc: 0.9603 - val_loss: 0.4021 - val_acc: 0.9221Epoch 232/50077s 154ms/step - loss: 0.2775 - acc: 0.9607 - val_loss: 0.3934 - val_acc: 0.9268Epoch 233/50077s 154ms/step - loss: 0.2787 - acc: 0.9592 - val_loss: 0.3829 - val_acc: 0.9275Epoch 234/50077s 154ms/step - loss: 0.2748 - acc: 0.9609 - val_loss: 0.3967 - val_acc: 0.9274Epoch 235/50077s 154ms/step - loss: 0.2781 - acc: 0.9589 - val_loss: 0.3909 - val_acc: 0.9275Epoch 236/50077s 154ms/step - loss: 0.2758 - acc: 0.9607 - val_loss: 0.3941 - val_acc: 0.9270Epoch 237/50077s 154ms/step - loss: 0.2767 - acc: 0.9600 - val_loss: 0.4121 - val_acc: 0.9195Epoch 238/50077s 154ms/step - loss: 0.2754 - acc: 0.9608 - val_loss: 0.3978 - val_acc: 0.9221Epoch 239/50077s 154ms/step - loss: 0.2722 - acc: 0.9616 - val_loss: 0.4039 - val_acc: 0.9238Epoch 240/50077s 154ms/step - loss: 0.2685 - acc: 0.9618 - val_loss: 0.3889 - val_acc: 0.9277Epoch 241/50077s 154ms/step - loss: 0.2751 - acc: 0.9596 - val_loss: 0.3960 - val_acc: 0.9274Epoch 242/50077s 154ms/step - loss: 0.2686 - acc: 0.9619 - val_loss: 0.3881 - val_acc: 0.9288Epoch 243/50077s 154ms/step - loss: 0.2744 - acc: 0.9600 - val_loss: 0.3929 - val_acc: 0.9235Epoch 244/50077s 154ms/step - loss: 0.2741 - acc: 0.9596 - val_loss: 0.3775 - val_acc: 0.9274Epoch 245/50077s 154ms/step - loss: 0.2694 - acc: 0.9619 - val_loss: 0.4006 - val_acc: 0.9213Epoch 246/50077s 154ms/step - loss: 0.2760 - acc: 0.9586 - val_loss: 0.3956 - val_acc: 0.9236Epoch 247/50077s 154ms/step - loss: 0.2694 - acc: 0.9612 - val_loss: 0.3956 - val_acc: 0.9232Epoch 248/50077s 154ms/step - loss: 0.2714 - acc: 0.9603 - val_loss: 0.3947 - val_acc: 0.9253Epoch 249/50077s 154ms/step - loss: 0.2718 - acc: 0.9608 - val_loss: 0.4027 - val_acc: 0.9232Epoch 250/50077s 154ms/step - loss: 0.2665 - acc: 0.9624 - val_loss: 0.3955 - val_acc: 0.9243Epoch 251/50077s 154ms/step - loss: 0.2666 - acc: 0.9632 - val_loss: 0.4009 - val_acc: 0.9219Epoch 252/50077s 154ms/step - loss: 0.2733 - acc: 0.9592 - val_loss: 0.4097 - val_acc: 0.9204Epoch 253/50077s 154ms/step - loss: 0.2702 - acc: 0.9602 - val_loss: 0.3962 - val_acc: 0.9213Epoch 254/50077s 154ms/step - loss: 0.2718 - acc: 0.9603 - val_loss: 0.3998 - val_acc: 0.9235Epoch 255/50077s 155ms/step - loss: 0.2679 - acc: 0.9613 - val_loss: 0.4113 - val_acc: 0.9217Epoch 256/50077s 155ms/step - loss: 0.2702 - acc: 0.9605 - val_loss: 0.3947 - val_acc: 0.9203Epoch 257/50077s 155ms/step - loss: 0.2728 - acc: 0.9593 - val_loss: 0.4031 - val_acc: 0.9234Epoch 258/50077s 154ms/step - loss: 0.2719 - acc: 0.9593 - val_loss: 0.3979 - val_acc: 0.9250Epoch 259/50077s 154ms/step - loss: 0.2683 - acc: 0.9620 - val_loss: 0.3881 - val_acc: 0.9264Epoch 260/50077s 154ms/step - loss: 0.2730 - acc: 0.9599 - val_loss: 0.3837 - val_acc: 0.9264Epoch 261/50077s 154ms/step - loss: 0.2681 - acc: 0.9614 - val_loss: 0.3945 - val_acc: 0.9251Epoch 262/50077s 154ms/step - loss: 0.2722 - acc: 0.9595 - val_loss: 0.3893 - val_acc: 0.9248Epoch 263/50077s 154ms/step - loss: 0.2695 - acc: 0.9613 - val_loss: 0.3948 - val_acc: 0.9241Epoch 264/50077s 154ms/step - loss: 0.2691 - acc: 0.9616 - val_loss: 0.3995 - val_acc: 0.9251Epoch 265/50077s 154ms/step - loss: 0.2722 - acc: 0.9601 - val_loss: 0.3898 - val_acc: 0.9248Epoch 266/50077s 154ms/step - loss: 0.2673 - acc: 0.9601 - val_loss: 0.3847 - val_acc: 0.9269Epoch 267/50077s 154ms/step - loss: 0.2641 - acc: 0.9629 - val_loss: 0.3892 - val_acc: 0.9258Epoch 268/50077s 154ms/step - loss: 0.2642 - acc: 0.9622 - val_loss: 0.3875 - val_acc: 0.9266Epoch 269/50077s 154ms/step - loss: 0.2709 - acc: 0.9604 - val_loss: 0.3991 - val_acc: 0.9236Epoch 270/50077s 154ms/step - loss: 0.2675 - acc: 0.9607 - val_loss: 0.3841 - val_acc: 0.9275Epoch 271/50077s 154ms/step - loss: 0.2672 - acc: 0.9618 - val_loss: 0.3863 - val_acc: 0.9254Epoch 272/50077s 154ms/step - loss: 0.2651 - acc: 0.9629 - val_loss: 0.3993 - val_acc: 0.9249Epoch 273/50077s 154ms/step - loss: 0.2675 - acc: 0.9618 - val_loss: 0.3959 - val_acc: 0.9230Epoch 274/50077s 154ms/step - loss: 0.2650 - acc: 0.9625 - val_loss: 0.3901 - val_acc: 0.9248Epoch 275/50077s 154ms/step - loss: 0.2685 - acc: 0.9611 - val_loss: 0.3998 - val_acc: 0.9206Epoch 276/50077s 154ms/step - loss: 0.2645 - acc: 0.9630 - val_loss: 0.3983 - val_acc: 0.9244Epoch 277/50077s 154ms/step - loss: 0.2675 - acc: 0.9614 - val_loss: 0.4014 - val_acc: 0.9227Epoch 278/50077s 154ms/step - loss: 0.2648 - acc: 0.9628 - val_loss: 0.3990 - val_acc: 0.9239Epoch 279/50077s 154ms/step - loss: 0.2648 - acc: 0.9624 - val_loss: 0.4027 - val_acc: 0.9215Epoch 280/50077s 154ms/step - loss: 0.2630 - acc: 0.9634 - val_loss: 0.4132 - val_acc: 0.9198Epoch 281/50077s 154ms/step - loss: 0.2673 - acc: 0.9620 - val_loss: 0.4117 - val_acc: 0.9200Epoch 282/50077s 154ms/step - loss: 0.2654 - acc: 0.9617 - val_loss: 0.4133 - val_acc: 0.9187Epoch 283/50077s 154ms/step - loss: 0.2661 - acc: 0.9621 - val_loss: 0.3970 - val_acc: 0.9250Epoch 284/50077s 154ms/step - loss: 0.2653 - acc: 0.9613 - val_loss: 0.3930 - val_acc: 0.9256Epoch 285/50077s 154ms/step - loss: 0.2615 - acc: 0.9632 - val_loss: 0.4035 - val_acc: 0.9252Epoch 286/50077s 154ms/step - loss: 0.2701 - acc: 0.9606 - val_loss: 0.3921 - val_acc: 0.9264Epoch 287/50077s 154ms/step - loss: 0.2663 - acc: 0.9619 - val_loss: 0.3858 - val_acc: 0.9287Epoch 288/50077s 154ms/step - loss: 0.2623 - acc: 0.9632 - val_loss: 0.3950 - val_acc: 0.9235Epoch 289/50077s 154ms/step - loss: 0.2636 - acc: 0.9625 - val_loss: 0.3869 - val_acc: 0.9281Epoch 290/50077s 154ms/step - loss: 0.2625 - acc: 0.9636 - val_loss: 0.3885 - val_acc: 0.9272Epoch 291/50077s 154ms/step - loss: 0.2623 - acc: 0.9630 - val_loss: 0.3900 - val_acc: 0.9252Epoch 292/50077s 154ms/step - loss: 0.2650 - acc: 0.9615 - val_loss: 0.3916 - val_acc: 0.9264Epoch 293/50077s 154ms/step - loss: 0.2627 - acc: 0.9624 - val_loss: 0.3935 - val_acc: 0.9259Epoch 294/50077s 154ms/step - loss: 0.2664 - acc: 0.9621 - val_loss: 0.3898 - val_acc: 0.9264Epoch 295/50077s 154ms/step - loss: 0.2624 - acc: 0.9629 - val_loss: 0.3937 - val_acc: 0.9264Epoch 296/50077s 154ms/step - loss: 0.2606 - acc: 0.9633 - val_loss: 0.3959 - val_acc: 0.9252Epoch 297/50077s 154ms/step - loss: 0.2621 - acc: 0.9626 - val_loss: 0.3978 - val_acc: 0.9245Epoch 298/50077s 153ms/step - loss: 0.2616 - acc: 0.9624 - val_loss: 0.3976 - val_acc: 0.9245Epoch 299/50077s 154ms/step - loss: 0.2610 - acc: 0.9626 - val_loss: 0.3952 - val_acc: 0.9239Epoch 300/50077s 154ms/step - loss: 0.2659 - acc: 0.9620 - val_loss: 0.4040 - val_acc: 0.9214Epoch 301/500lr changed to 0.000999999977648258377s 154ms/step - loss: 0.2382 - acc: 0.9722 - val_loss: 0.3640 - val_acc: 0.9314Epoch 302/50077s 154ms/step - loss: 0.2191 - acc: 0.9797 - val_loss: 0.3560 - val_acc: 0.9333Epoch 303/50077s 154ms/step - loss: 0.2131 - acc: 0.9809 - val_loss: 0.3548 - val_acc: 0.9346Epoch 304/50077s 154ms/step - loss: 0.2059 - acc: 0.9836 - val_loss: 0.3561 - val_acc: 0.9340Epoch 305/50077s 154ms/step - loss: 0.2049 - acc: 0.9843 - val_loss: 0.3526 - val_acc: 0.9366Epoch 306/50077s 154ms/step - loss: 0.2019 - acc: 0.9852 - val_loss: 0.3509 - val_acc: 0.9373Epoch 307/50077s 154ms/step - loss: 0.2028 - acc: 0.9843 - val_loss: 0.3527 - val_acc: 0.9362Epoch 308/50077s 154ms/step - loss: 0.2023 - acc: 0.9846 - val_loss: 0.3534 - val_acc: 0.9363Epoch 309/50077s 154ms/step - loss: 0.1995 - acc: 0.9855 - val_loss: 0.3533 - val_acc: 0.9367Epoch 310/50077s 154ms/step - loss: 0.1957 - acc: 0.9871 - val_loss: 0.3547 - val_acc: 0.9369Epoch 311/50077s 154ms/step - loss: 0.1947 - acc: 0.9877 - val_loss: 0.3532 - val_acc: 0.9380Epoch 312/50077s 154ms/step - loss: 0.1928 - acc: 0.9878 - val_loss: 0.3533 - val_acc: 0.9380Epoch 313/50077s 154ms/step - loss: 0.1920 - acc: 0.9879 - val_loss: 0.3522 - val_acc: 0.9391Epoch 314/50077s 154ms/step - loss: 0.1923 - acc: 0.9871 - val_loss: 0.3523 - val_acc: 0.9385Epoch 315/50077s 154ms/step - loss: 0.1901 - acc: 0.9885 - val_loss: 0.3517 - val_acc: 0.9375Epoch 316/50077s 154ms/step - loss: 0.1903 - acc: 0.9879 - val_loss: 0.3518 - val_acc: 0.9391Epoch 317/50077s 154ms/step - loss: 0.1883 - acc: 0.9885 - val_loss: 0.3539 - val_acc: 0.9384Epoch 318/50077s 154ms/step - loss: 0.1884 - acc: 0.9883 - val_loss: 0.3568 - val_acc: 0.9376Epoch 319/50077s 154ms/step - loss: 0.1888 - acc: 0.9887 - val_loss: 0.3560 - val_acc: 0.9382Epoch 320/50077s 154ms/step - loss: 0.1862 - acc: 0.9893 - val_loss: 0.3573 - val_acc: 0.9371Epoch 321/50077s 154ms/step - loss: 0.1874 - acc: 0.9880 - val_loss: 0.3561 - val_acc: 0.9386Epoch 322/50077s 154ms/step - loss: 0.1855 - acc: 0.9895 - val_loss: 0.3553 - val_acc: 0.9395Epoch 323/50077s 154ms/step - loss: 0.1846 - acc: 0.9897 - val_loss: 0.3543 - val_acc: 0.9396Epoch 324/50077s 154ms/step - loss: 0.1860 - acc: 0.9890 - val_loss: 0.3560 - val_acc: 0.9382Epoch 325/50077s 154ms/step - loss: 0.1834 - acc: 0.9894 - val_loss: 0.3539 - val_acc: 0.9383Epoch 326/50077s 154ms/step - loss: 0.1847 - acc: 0.9894 - val_loss: 0.3555 - val_acc: 0.9375Epoch 327/50077s 154ms/step - loss: 0.1830 - acc: 0.9896 - val_loss: 0.3559 - val_acc: 0.9370Epoch 328/50077s 154ms/step - loss: 0.1839 - acc: 0.9894 - val_loss: 0.3584 - val_acc: 0.9363Epoch 329/50077s 154ms/step - loss: 0.1808 - acc: 0.9902 - val_loss: 0.3571 - val_acc: 0.9381Epoch 330/50077s 154ms/step - loss: 0.1818 - acc: 0.9899 - val_loss: 0.3556 - val_acc: 0.9377Epoch 331/50077s 154ms/step - loss: 0.1800 - acc: 0.9903 - val_loss: 0.3584 - val_acc: 0.9380Epoch 332/50077s 154ms/step - loss: 0.1811 - acc: 0.9898 - val_loss: 0.3571 - val_acc: 0.9399Epoch 333/50077s 154ms/step - loss: 0.1801 - acc: 0.9900 - val_loss: 0.3574 - val_acc: 0.9390Epoch 334/50077s 154ms/step - loss: 0.1802 - acc: 0.9900 - val_loss: 0.3582 - val_acc: 0.9381Epoch 335/50077s 154ms/step - loss: 0.1797 - acc: 0.9902 - val_loss: 0.3629 - val_acc: 0.9369Epoch 336/50077s 154ms/step - loss: 0.1783 - acc: 0.9908 - val_loss: 0.3563 - val_acc: 0.9390Epoch 337/50077s 154ms/step - loss: 0.1787 - acc: 0.9901 - val_loss: 0.3549 - val_acc: 0.9380Epoch 338/50077s 154ms/step - loss: 0.1780 - acc: 0.9907 - val_loss: 0.3594 - val_acc: 0.9368Epoch 339/50077s 154ms/step - loss: 0.1776 - acc: 0.9905 - val_loss: 0.3556 - val_acc: 0.9384Epoch 340/50077s 154ms/step - loss: 0.1763 - acc: 0.9912 - val_loss: 0.3543 - val_acc: 0.9397Epoch 341/50077s 154ms/step - loss: 0.1760 - acc: 0.9911 - val_loss: 0.3552 - val_acc: 0.9380Epoch 342/50077s 154ms/step - loss: 0.1754 - acc: 0.9911 - val_loss: 0.3567 - val_acc: 0.9387Epoch 343/50077s 154ms/step - loss: 0.1762 - acc: 0.9908 - val_loss: 0.3547 - val_acc: 0.9386Epoch 344/50077s 154ms/step - loss: 0.1746 - acc: 0.9913 - val_loss: 0.3569 - val_acc: 0.9377Epoch 345/50077s 154ms/step - loss: 0.1736 - acc: 0.9920 - val_loss: 0.3596 - val_acc: 0.9381Epoch 346/50077s 154ms/step - loss: 0.1730 - acc: 0.9915 - val_loss: 0.3580 - val_acc: 0.9382Epoch 347/50077s 154ms/step - loss: 0.1727 - acc: 0.9910 - val_loss: 0.3569 - val_acc: 0.9386Epoch 348/50077s 154ms/step - loss: 0.1750 - acc: 0.9907 - val_loss: 0.3595 - val_acc: 0.9379Epoch 349/50077s 154ms/step - loss: 0.1736 - acc: 0.9911 - val_loss: 0.3579 - val_acc: 0.9382Epoch 350/50077s 154ms/step - loss: 0.1740 - acc: 0.9910 - val_loss: 0.3560 - val_acc: 0.9394Epoch 351/50077s 154ms/step - loss: 0.1710 - acc: 0.9919 - val_loss: 0.3584 - val_acc: 0.9381Epoch 352/50077s 154ms/step - loss: 0.1724 - acc: 0.9914 - val_loss: 0.3606 - val_acc: 0.9377Epoch 353/50077s 154ms/step - loss: 0.1704 - acc: 0.9922 - val_loss: 0.3589 - val_acc: 0.9369Epoch 354/50077s 154ms/step - loss: 0.1696 - acc: 0.9928 - val_loss: 0.3549 - val_acc: 0.9397Epoch 355/50077s 154ms/step - loss: 0.1710 - acc: 0.9914 - val_loss: 0.3568 - val_acc: 0.9397Epoch 356/50077s 154ms/step - loss: 0.1690 - acc: 0.9919 - val_loss: 0.3574 - val_acc: 0.9390Epoch 357/50077s 154ms/step - loss: 0.1692 - acc: 0.9919 - val_loss: 0.3604 - val_acc: 0.9363Epoch 358/50077s 154ms/step - loss: 0.1689 - acc: 0.9920 - val_loss: 0.3587 - val_acc: 0.9385Epoch 359/50077s 154ms/step - loss: 0.1680 - acc: 0.9922 - val_loss: 0.3629 - val_acc: 0.9363Epoch 360/50077s 154ms/step - loss: 0.1690 - acc: 0.9923 - val_loss: 0.3575 - val_acc: 0.9385Epoch 361/50077s 154ms/step - loss: 0.1683 - acc: 0.9920 - val_loss: 0.3560 - val_acc: 0.9389Epoch 362/50077s 154ms/step - loss: 0.1699 - acc: 0.9913 - val_loss: 0.3572 - val_acc: 0.9366Epoch 363/50077s 154ms/step - loss: 0.1657 - acc: 0.9925 - val_loss: 0.3546 - val_acc: 0.9385Epoch 364/50077s 154ms/step - loss: 0.1675 - acc: 0.9920 - val_loss: 0.3581 - val_acc: 0.9381Epoch 365/50077s 154ms/step - loss: 0.1661 - acc: 0.9926 - val_loss: 0.3595 - val_acc: 0.9390Epoch 366/50077s 154ms/step - loss: 0.1664 - acc: 0.9927 - val_loss: 0.3576 - val_acc: 0.9391Epoch 367/50077s 154ms/step - loss: 0.1659 - acc: 0.9920 - val_loss: 0.3575 - val_acc: 0.9395Epoch 368/50077s 154ms/step - loss: 0.1662 - acc: 0.9920 - val_loss: 0.3577 - val_acc: 0.9383Epoch 369/50077s 154ms/step - loss: 0.1658 - acc: 0.9923 - val_loss: 0.3596 - val_acc: 0.9383Epoch 370/50077s 154ms/step - loss: 0.1634 - acc: 0.9933 - val_loss: 0.3575 - val_acc: 0.9386Epoch 371/50077s 154ms/step - loss: 0.1657 - acc: 0.9919 - val_loss: 0.3587 - val_acc: 0.9372Epoch 372/50077s 154ms/step - loss: 0.1651 - acc: 0.9922 - val_loss: 0.3546 - val_acc: 0.9371Epoch 373/50077s 154ms/step - loss: 0.1644 - acc: 0.9923 - val_loss: 0.3551 - val_acc: 0.9401Epoch 374/50077s 154ms/step - loss: 0.1643 - acc: 0.9922 - val_loss: 0.3585 - val_acc: 0.9374Epoch 375/50077s 154ms/step - loss: 0.1634 - acc: 0.9926 - val_loss: 0.3567 - val_acc: 0.9373Epoch 376/50077s 154ms/step - loss: 0.1637 - acc: 0.9925 - val_loss: 0.3590 - val_acc: 0.9364Epoch 377/50077s 154ms/step - loss: 0.1636 - acc: 0.9922 - val_loss: 0.3615 - val_acc: 0.9365Epoch 378/50077s 154ms/step - loss: 0.1607 - acc: 0.9934 - val_loss: 0.3587 - val_acc: 0.9380Epoch 379/50077s 154ms/step - loss: 0.1614 - acc: 0.9932 - val_loss: 0.3595 - val_acc: 0.9379Epoch 380/50077s 154ms/step - loss: 0.1634 - acc: 0.9919 - val_loss: 0.3587 - val_acc: 0.9384Epoch 381/50077s 154ms/step - loss: 0.1604 - acc: 0.9932 - val_loss: 0.3607 - val_acc: 0.9367Epoch 382/50077s 154ms/step - loss: 0.1603 - acc: 0.9931 - val_loss: 0.3577 - val_acc: 0.9359Epoch 383/50077s 154ms/step - loss: 0.1620 - acc: 0.9922 - val_loss: 0.3560 - val_acc: 0.9391Epoch 384/50077s 154ms/step - loss: 0.1622 - acc: 0.9923 - val_loss: 0.3579 - val_acc: 0.9363Epoch 385/50076s 153ms/step - loss: 0.1603 - acc: 0.9927 - val_loss: 0.3572 - val_acc: 0.9379Epoch 386/50076s 153ms/step - loss: 0.1602 - acc: 0.9928 - val_loss: 0.3571 - val_acc: 0.9377Epoch 387/50076s 153ms/step - loss: 0.1592 - acc: 0.9929 - val_loss: 0.3576 - val_acc: 0.9366Epoch 388/50076s 153ms/step - loss: 0.1603 - acc: 0.9925 - val_loss: 0.3547 - val_acc: 0.9383Epoch 389/50076s 153ms/step - loss: 0.1589 - acc: 0.9934 - val_loss: 0.3555 - val_acc: 0.9383Epoch 390/50077s 153ms/step - loss: 0.1599 - acc: 0.9930 - val_loss: 0.3567 - val_acc: 0.9386Epoch 391/50077s 153ms/step - loss: 0.1581 - acc: 0.9930 - val_loss: 0.3517 - val_acc: 0.9387Epoch 392/50077s 153ms/step - loss: 0.1583 - acc: 0.9933 - val_loss: 0.3577 - val_acc: 0.9391Epoch 393/50077s 154ms/step - loss: 0.1588 - acc: 0.9933 - val_loss: 0.3571 - val_acc: 0.9400Epoch 394/50077s 153ms/step - loss: 0.1575 - acc: 0.9936 - val_loss: 0.3574 - val_acc: 0.9382Epoch 395/50077s 153ms/step - loss: 0.1566 - acc: 0.9937 - val_loss: 0.3566 - val_acc: 0.9375Epoch 396/50077s 153ms/step - loss: 0.1567 - acc: 0.9937 - val_loss: 0.3607 - val_acc: 0.9374Epoch 397/50077s 153ms/step - loss: 0.1569 - acc: 0.9932 - val_loss: 0.3584 - val_acc: 0.9378Epoch 398/50076s 153ms/step - loss: 0.1569 - acc: 0.9930 - val_loss: 0.3587 - val_acc: 0.9380Epoch 399/50076s 153ms/step - loss: 0.1562 - acc: 0.9934 - val_loss: 0.3573 - val_acc: 0.9392Epoch 400/50076s 153ms/step - loss: 0.1562 - acc: 0.9932 - val_loss: 0.3558 - val_acc: 0.9381Epoch 401/50076s 153ms/step - loss: 0.1558 - acc: 0.9933 - val_loss: 0.3549 - val_acc: 0.9367Epoch 402/50077s 153ms/step - loss: 0.1555 - acc: 0.9933 - val_loss: 0.3580 - val_acc: 0.9343Epoch 403/50076s 153ms/step - loss: 0.1547 - acc: 0.9936 - val_loss: 0.3553 - val_acc: 0.9378Epoch 404/50077s 153ms/step - loss: 0.1557 - acc: 0.9932 - val_loss: 0.3526 - val_acc: 0.9384Epoch 405/50076s 153ms/step - loss: 0.1537 - acc: 0.9939 - val_loss: 0.3525 - val_acc: 0.9390Epoch 406/50076s 153ms/step - loss: 0.1538 - acc: 0.9941 - val_loss: 0.3605 - val_acc: 0.9359Epoch 407/50076s 153ms/step - loss: 0.1530 - acc: 0.9942 - val_loss: 0.3579 - val_acc: 0.9371Epoch 408/50077s 153ms/step - loss: 0.1536 - acc: 0.9937 - val_loss: 0.3553 - val_acc: 0.9374Epoch 409/50077s 154ms/step - loss: 0.1532 - acc: 0.9938 - val_loss: 0.3601 - val_acc: 0.9373Epoch 410/50077s 154ms/step - loss: 0.1539 - acc: 0.9933 - val_loss: 0.3582 - val_acc: 0.9374Epoch 411/50077s 154ms/step - loss: 0.1548 - acc: 0.9930 - val_loss: 0.3574 - val_acc: 0.9394Epoch 412/50077s 153ms/step - loss: 0.1514 - acc: 0.9942 - val_loss: 0.3578 - val_acc: 0.9357Epoch 413/50077s 153ms/step - loss: 0.1542 - acc: 0.9931 - val_loss: 0.3551 - val_acc: 0.9371Epoch 414/50077s 153ms/step - loss: 0.1527 - acc: 0.9935 - val_loss: 0.3562 - val_acc: 0.9379Epoch 415/50077s 153ms/step - loss: 0.1529 - acc: 0.9930 - val_loss: 0.3584 - val_acc: 0.9356Epoch 416/50076s 153ms/step - loss: 0.1520 - acc: 0.9936 - val_loss: 0.3609 - val_acc: 0.9349Epoch 417/50077s 153ms/step - loss: 0.1503 - acc: 0.9941 - val_loss: 0.3589 - val_acc: 0.9369Epoch 418/50077s 153ms/step - loss: 0.1512 - acc: 0.9937 - val_loss: 0.3585 - val_acc: 0.9363Epoch 419/50076s 153ms/step - loss: 0.1518 - acc: 0.9931 - val_loss: 0.3581 - val_acc: 0.9379Epoch 420/50076s 153ms/step - loss: 0.1514 - acc: 0.9935 - val_loss: 0.3608 - val_acc: 0.9369Epoch 421/50076s 153ms/step - loss: 0.1502 - acc: 0.9941 - val_loss: 0.3618 - val_acc: 0.9366Epoch 422/50077s 153ms/step - loss: 0.1503 - acc: 0.9935 - val_loss: 0.3566 - val_acc: 0.9362Epoch 423/50077s 153ms/step - loss: 0.1510 - acc: 0.9933 - val_loss: 0.3574 - val_acc: 0.9362Epoch 424/50077s 153ms/step - loss: 0.1506 - acc: 0.9932 - val_loss: 0.3583 - val_acc: 0.9382Epoch 425/50076s 153ms/step - loss: 0.1504 - acc: 0.9935 - val_loss: 0.3590 - val_acc: 0.9373Epoch 426/50076s 153ms/step - loss: 0.1500 - acc: 0.9937 - val_loss: 0.3567 - val_acc: 0.9360Epoch 427/50076s 153ms/step - loss: 0.1487 - acc: 0.9938 - val_loss: 0.3536 - val_acc: 0.9388Epoch 428/50076s 153ms/step - loss: 0.1476 - acc: 0.9943 - val_loss: 0.3533 - val_acc: 0.9391Epoch 429/50077s 153ms/step - loss: 0.1490 - acc: 0.9936 - val_loss: 0.3537 - val_acc: 0.9401Epoch 430/50076s 153ms/step - loss: 0.1488 - acc: 0.9937 - val_loss: 0.3524 - val_acc: 0.9391Epoch 431/50077s 153ms/step - loss: 0.1482 - acc: 0.9937 - val_loss: 0.3483 - val_acc: 0.9399Epoch 432/50076s 153ms/step - loss: 0.1487 - acc: 0.9935 - val_loss: 0.3526 - val_acc: 0.9377Epoch 433/50077s 153ms/step - loss: 0.1481 - acc: 0.9940 - val_loss: 0.3494 - val_acc: 0.9394Epoch 434/50076s 153ms/step - loss: 0.1483 - acc: 0.9936 - val_loss: 0.3542 - val_acc: 0.9378Epoch 435/50076s 153ms/step - loss: 0.1470 - acc: 0.9936 - val_loss: 0.3503 - val_acc: 0.9394Epoch 436/50076s 153ms/step - loss: 0.1456 - acc: 0.9942 - val_loss: 0.3514 - val_acc: 0.9387Epoch 437/50076s 153ms/step - loss: 0.1472 - acc: 0.9935 - val_loss: 0.3498 - val_acc: 0.9392Epoch 438/50077s 153ms/step - loss: 0.1470 - acc: 0.9936 - val_loss: 0.3485 - val_acc: 0.9386Epoch 439/50077s 153ms/step - loss: 0.1469 - acc: 0.9935 - val_loss: 0.3474 - val_acc: 0.9378Epoch 440/50077s 153ms/step - loss: 0.1467 - acc: 0.9932 - val_loss: 0.3487 - val_acc: 0.9381Epoch 441/50076s 153ms/step - loss: 0.1455 - acc: 0.9939 - val_loss: 0.3450 - val_acc: 0.9383Epoch 442/50077s 153ms/step - loss: 0.1453 - acc: 0.9941 - val_loss: 0.3518 - val_acc: 0.9370Epoch 443/50076s 153ms/step - loss: 0.1450 - acc: 0.9939 - val_loss: 0.3510 - val_acc: 0.9360Epoch 444/50076s 153ms/step - loss: 0.1458 - acc: 0.9942 - val_loss: 0.3553 - val_acc: 0.9366Epoch 445/50076s 153ms/step - loss: 0.1447 - acc: 0.9942 - val_loss: 0.3484 - val_acc: 0.9375Epoch 446/50077s 153ms/step - loss: 0.1431 - acc: 0.9945 - val_loss: 0.3522 - val_acc: 0.9386Epoch 447/50076s 153ms/step - loss: 0.1448 - acc: 0.9939 - val_loss: 0.3548 - val_acc: 0.9359Epoch 448/50076s 153ms/step - loss: 0.1434 - acc: 0.9939 - val_loss: 0.3514 - val_acc: 0.9379Epoch 449/50076s 153ms/step - loss: 0.1439 - acc: 0.9939 - val_loss: 0.3488 - val_acc: 0.9389Epoch 450/50076s 153ms/step - loss: 0.1445 - acc: 0.9937 - val_loss: 0.3523 - val_acc: 0.9380Epoch 451/500lr changed to 9.999999310821295e-0576s 153ms/step - loss: 0.1442 - acc: 0.9938 - val_loss: 0.3510 - val_acc: 0.9385Epoch 452/50077s 154ms/step - loss: 0.1436 - acc: 0.9941 - val_loss: 0.3507 - val_acc: 0.9394Epoch 453/50076s 153ms/step - loss: 0.1423 - acc: 0.9948 - val_loss: 0.3503 - val_acc: 0.9387Epoch 454/50077s 153ms/step - loss: 0.1430 - acc: 0.9942 - val_loss: 0.3499 - val_acc: 0.9391Epoch 455/50077s 153ms/step - loss: 0.1430 - acc: 0.9941 - val_loss: 0.3489 - val_acc: 0.9396Epoch 456/50077s 153ms/step - loss: 0.1418 - acc: 0.9947 - val_loss: 0.3487 - val_acc: 0.9393Epoch 457/50077s 153ms/step - loss: 0.1417 - acc: 0.9949 - val_loss: 0.3478 - val_acc: 0.9395Epoch 458/50076s 153ms/step - loss: 0.1423 - acc: 0.9946 - val_loss: 0.3476 - val_acc: 0.9393Epoch 459/50076s 153ms/step - loss: 0.1408 - acc: 0.9949 - val_loss: 0.3474 - val_acc: 0.9398Epoch 460/50077s 153ms/step - loss: 0.1415 - acc: 0.9947 - val_loss: 0.3482 - val_acc: 0.9397Epoch 461/50076s 153ms/step - loss: 0.1406 - acc: 0.9956 - val_loss: 0.3482 - val_acc: 0.9394Epoch 462/50077s 153ms/step - loss: 0.1409 - acc: 0.9951 - val_loss: 0.3475 - val_acc: 0.9396Epoch 463/50076s 153ms/step - loss: 0.1406 - acc: 0.9950 - val_loss: 0.3471 - val_acc: 0.9395Epoch 464/50076s 153ms/step - loss: 0.1406 - acc: 0.9951 - val_loss: 0.3474 - val_acc: 0.9390Epoch 465/50076s 153ms/step - loss: 0.1408 - acc: 0.9950 - val_loss: 0.3477 - val_acc: 0.9396Epoch 466/50077s 153ms/step - loss: 0.1418 - acc: 0.9948 - val_loss: 0.3478 - val_acc: 0.9385Epoch 467/50076s 153ms/step - loss: 0.1412 - acc: 0.9950 - val_loss: 0.3474 - val_acc: 0.9388Epoch 468/50077s 153ms/step - loss: 0.1400 - acc: 0.9954 - val_loss: 0.3476 - val_acc: 0.9387Epoch 469/50076s 153ms/step - loss: 0.1405 - acc: 0.9957 - val_loss: 0.3474 - val_acc: 0.9384Epoch 470/50077s 153ms/step - loss: 0.1399 - acc: 0.9950 - val_loss: 0.3468 - val_acc: 0.9388Epoch 471/50077s 153ms/step - loss: 0.1400 - acc: 0.9957 - val_loss: 0.3468 - val_acc: 0.9390Epoch 472/50076s 153ms/step - loss: 0.1408 - acc: 0.9949 - val_loss: 0.3473 - val_acc: 0.9390Epoch 473/50076s 153ms/step - loss: 0.1410 - acc: 0.9947 - val_loss: 0.3476 - val_acc: 0.9392Epoch 474/50076s 153ms/step - loss: 0.1401 - acc: 0.9954 - val_loss: 0.3477 - val_acc: 0.9396Epoch 475/50077s 153ms/step - loss: 0.1400 - acc: 0.9952 - val_loss: 0.3478 - val_acc: 0.9397Epoch 476/50076s 153ms/step - loss: 0.1389 - acc: 0.9955 - val_loss: 0.3471 - val_acc: 0.9391Epoch 477/50076s 153ms/step - loss: 0.1404 - acc: 0.9950 - val_loss: 0.3474 - val_acc: 0.9392Epoch 478/50077s 153ms/step - loss: 0.1398 - acc: 0.9953 - val_loss: 0.3475 - val_acc: 0.9390Epoch 479/50077s 153ms/step - loss: 0.1400 - acc: 0.9951 - val_loss: 0.3468 - val_acc: 0.9394Epoch 480/50077s 153ms/step - loss: 0.1394 - acc: 0.9954 - val_loss: 0.3471 - val_acc: 0.9392Epoch 481/50076s 153ms/step - loss: 0.1404 - acc: 0.9951 - val_loss: 0.3467 - val_acc: 0.9393Epoch 482/50076s 153ms/step - loss: 0.1395 - acc: 0.9951 - val_loss: 0.3464 - val_acc: 0.9395Epoch 483/50076s 153ms/step - loss: 0.1392 - acc: 0.9954 - val_loss: 0.3466 - val_acc: 0.9396Epoch 484/50076s 153ms/step - loss: 0.1394 - acc: 0.9952 - val_loss: 0.3467 - val_acc: 0.9395Epoch 485/50076s 153ms/step - loss: 0.1397 - acc: 0.9952 - val_loss: 0.3470 - val_acc: 0.9393Epoch 486/50076s 153ms/step - loss: 0.1391 - acc: 0.9953 - val_loss: 0.3474 - val_acc: 0.9393Epoch 487/50077s 153ms/step - loss: 0.1395 - acc: 0.9955 - val_loss: 0.3479 - val_acc: 0.9395Epoch 488/50077s 153ms/step - loss: 0.1388 - acc: 0.9952 - val_loss: 0.3475 - val_acc: 0.9391Epoch 489/50077s 153ms/step - loss: 0.1393 - acc: 0.9954 - val_loss: 0.3477 - val_acc: 0.9390Epoch 490/50077s 153ms/step - loss: 0.1399 - acc: 0.9953 - val_loss: 0.3482 - val_acc: 0.9390Epoch 491/50076s 153ms/step - loss: 0.1397 - acc: 0.9950 - val_loss: 0.3487 - val_acc: 0.9389Epoch 492/50077s 153ms/step - loss: 0.1395 - acc: 0.9952 - val_loss: 0.3487 - val_acc: 0.9388Epoch 493/50077s 153ms/step - loss: 0.1391 - acc: 0.9956 - val_loss: 0.3491 - val_acc: 0.9390Epoch 494/50076s 153ms/step - loss: 0.1396 - acc: 0.9952 - val_loss: 0.3481 - val_acc: 0.9389Epoch 495/50076s 153ms/step - loss: 0.1395 - acc: 0.9952 - val_loss: 0.3479 - val_acc: 0.9387Epoch 496/50076s 153ms/step - loss: 0.1389 - acc: 0.9953 - val_loss: 0.3482 - val_acc: 0.9386Epoch 497/50076s 153ms/step - loss: 0.1385 - acc: 0.9960 - val_loss: 0.3487 - val_acc: 0.9388Epoch 498/50076s 153ms/step - loss: 0.1386 - acc: 0.9957 - val_loss: 0.3483 - val_acc: 0.9388Epoch 499/50076s 153ms/step - loss: 0.1391 - acc: 0.9955 - val_loss: 0.3482 - val_acc: 0.9390Epoch 500/50076s 153ms/step - loss: 0.1397 - acc: 0.9951 - val_loss: 0.3481 - val_acc: 0.9396Train loss: 0.1304543605595827Train accuracy: 0.9980800018310547Test loss: 0.3480722904205322Test accuracy: 0.9396000015735626相较于调参记录18，训练准确率和测试准确率都降了一点。同时，训练准确率比测试准确率大概高了6%，说明依然存在过拟合。
Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht, Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, IEEE Transactions on Industrial Electronics, 2020, DOI: 10.1109/TIE.2020.2972458, Date of Publication: 13 February 2020
https://ieeexplore.ieee.org/d...