Adaptively Parametric ReLU是一种动态ReLU（Dynamic ReLU），于2019年5月3日投稿至IEEE Transactions on Industrial Electronics，于2020年1月24日录用，于2020年2月13日在IEEE官网公布。
本文在调参记录9的基础上，在数据增强部分增加了shear_range = 30，测试Adaptively Parametric ReLU（APReLU）激活函数在Cifar10图像集上的效果。
Keras里ImageDataGenerator的用法见如下网址：
https://fairyonice.github.io/...
Adaptively Parametric ReLU的基本原理见下图：
Keras程序如下：
#!/usr/bin/env python3# -*- coding: utf-8 -*-"""Created on Tue Apr 14 04:17:45 2020Implemented using TensorFlow 1.0.1 and Keras 2.2.1Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht,Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, IEEE Transactions on Industrial Electronics, 2020, DOI: 10.1109/TIE.2020.2972458,Date of Publication: 13 February 2020@author: Minghang Zhao"""from __future__ import print_functionimport kerasimport numpy as npfrom keras.datasets import cifar10from keras.layers import Dense, Conv2D, BatchNormalization, Activation, Minimumfrom keras.layers import AveragePooling2D, Input, GlobalAveragePooling2D, Concatenate, Reshapefrom keras.regularizers import l2from keras import backend as Kfrom keras.models import Modelfrom keras import optimizersfrom keras.preprocessing.image import ImageDataGeneratorfrom keras.callbacks import LearningRateSchedulerK.set_learning_phase(1)# The data, split between train and test sets(x_train, y_train), (x_test, y_test) = cifar10.load_data()x_train = x_train.astype('float32') / 255.x_test = x_test.astype('float32') / 255.x_test = x_test-np.mean(x_train)x_train = x_train-np.mean(x_train)print('x_train shape:', x_train.shape)print(x_train.shape[0], 'train samples')print(x_test.shape[0], 'test samples')# convert class vectors to binary class matricesy_train = keras.utils.to_categorical(y_train, 10)y_test = keras.utils.to_categorical(y_test, 10)# Schedule the learning rate, multiply 0.1 every 300 epochesdef scheduler(epoch): if epoch % 300 == 0 and epoch != 0: lr = K.get_value(model.optimizer.lr) K.set_value(model.optimizer.lr, lr * 0.1) print("lr changed to {}".format(lr * 0.1)) return K.get_value(model.optimizer.lr)# An adaptively parametric rectifier linear unit (APReLU)def aprelu(inputs): # get the number of channels channels = inputs.get_shape().as_list()[-1] # get a zero feature map zeros_input = keras.layers.subtract([inputs, inputs]) # get a feature map with only positive features pos_input = Activation('relu')(inputs) # get a feature map with only negative features neg_input = Minimum()([inputs,zeros_input]) # define a network to obtain the scaling coefficients scales_p = GlobalAveragePooling2D()(pos_input) scales_n = GlobalAveragePooling2D()(neg_input) scales = Concatenate()([scales_n, scales_p]) scales = Dense(channels, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales) scales = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(scales) scales = Activation('relu')(scales) scales = Dense(channels, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales) scales = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(scales) scales = Activation('sigmoid')(scales) scales = Reshape((1,1,channels))(scales) # apply a paramtetric relu neg_part = keras.layers.multiply([scales, neg_input]) return keras.layers.add([pos_input, neg_part])# Residual Blockdef residual_block(incoming, nb_blocks, out_channels, downsample=False, downsample_strides=2): residual = incoming in_channels = incoming.get_shape().as_list()[-1] for i in range(nb_blocks): identity = residual if not downsample: downsample_strides = 1 residual = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(residual) residual = aprelu(residual) residual = Conv2D(out_channels, 3, strides=(downsample_strides, downsample_strides), padding='same', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(residual) residual = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(residual) residual = aprelu(residual) residual = Conv2D(out_channels, 3, padding='same', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(residual) # Downsampling if downsample_strides > 1: identity = AveragePooling2D(pool_size=(1,1), strides=(2,2))(identity) # Zero_padding to match channels if in_channels != out_channels: zeros_identity = keras.layers.subtract([identity, identity]) identity = keras.layers.concatenate([identity, zeros_identity]) in_channels = out_channels residual = keras.layers.add([residual, identity]) return residual# define and train a modelinputs = Input(shape=(32, 32, 3))net = Conv2D(16, 3, padding='same', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(inputs)net = residual_block(net, 9, 16, downsample=False)net = residual_block(net, 1, 32, downsample=True)net = residual_block(net, 8, 32, downsample=False)net = residual_block(net, 1, 64, downsample=True)net = residual_block(net, 8, 64, downsample=False)net = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(net)net = Activation('relu')(net)net = GlobalAveragePooling2D()(net)outputs = Dense(10, activation='softmax', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(net)model = Model(inputs=inputs, outputs=outputs)sgd = optimizers.SGD(lr=0.1, decay=0., momentum=0.9, nesterov=True)model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])# data augmentationdatagen = ImageDataGenerator( # randomly rotate images in the range (deg 0 to 180) rotation_range=30, # shear angle in counter-clockwise direction in degrees shear_range = 30, # randomly flip images horizontal_flip=True, # randomly shift images horizontally width_shift_range=0.125, # randomly shift images vertically height_shift_range=0.125)reduce_lr = LearningRateScheduler(scheduler)# fit the model on the batches generated by datagen.flow().model.fit_generator(datagen.flow(x_train, y_train, batch_size=100), validation_data=(x_test, y_test), epochs=1000, verbose=1, callbacks=[reduce_lr], workers=4)# get resultsK.set_learning_phase(0)DRSN_train_score = model.evaluate(x_train, y_train, batch_size=100, verbose=0)print('Train loss:', DRSN_train_score[0])print('Train accuracy:', DRSN_train_score[1])DRSN_test_score = model.evaluate(x_test, y_test, batch_size=100, verbose=0)print('Test loss:', DRSN_test_score[0])print('Test accuracy:', DRSN_test_score[1])实验结果如下：
x_train shape: (50000, 32, 32, 3)50000 train samples10000 test samplesEpoch 1/1000113s 225ms/step - loss: 3.2549 - acc: 0.4158 - val_loss: 2.7729 - val_acc: 0.5394Epoch 2/100068s 137ms/step - loss: 2.6403 - acc: 0.5484 - val_loss: 2.3416 - val_acc: 0.6117Epoch 3/100069s 138ms/step - loss: 2.2763 - acc: 0.6049 - val_loss: 2.0151 - val_acc: 0.6705Epoch 4/100069s 137ms/step - loss: 2.0062 - acc: 0.6393 - val_loss: 1.8055 - val_acc: 0.6907Epoch 5/100069s 137ms/step - loss: 1.7997 - acc: 0.6673 - val_loss: 1.6339 - val_acc: 0.7058Epoch 6/100069s 138ms/step - loss: 1.6338 - acc: 0.6849 - val_loss: 1.4391 - val_acc: 0.7345Epoch 7/100069s 138ms/step - loss: 1.4911 - acc: 0.7032 - val_loss: 1.3495 - val_acc: 0.7435Epoch 8/100069s 138ms/step - loss: 1.3733 - acc: 0.7196 - val_loss: 1.2311 - val_acc: 0.7668Epoch 9/100068s 137ms/step - loss: 1.2893 - acc: 0.7308 - val_loss: 1.1543 - val_acc: 0.7741Epoch 10/100068s 137ms/step - loss: 1.2164 - acc: 0.7402 - val_loss: 1.0974 - val_acc: 0.7761Epoch 11/100069s 137ms/step - loss: 1.1580 - acc: 0.7470 - val_loss: 1.0477 - val_acc: 0.7835Epoch 12/100069s 137ms/step - loss: 1.1127 - acc: 0.7519 - val_loss: 1.0269 - val_acc: 0.7813Epoch 13/100069s 138ms/step - loss: 1.0713 - acc: 0.7598 - val_loss: 0.9656 - val_acc: 0.7996Epoch 14/100068s 136ms/step - loss: 1.0369 - acc: 0.7664 - val_loss: 0.9576 - val_acc: 0.7929Epoch 15/100068s 135ms/step - loss: 1.0158 - acc: 0.7677 - val_loss: 0.9189 - val_acc: 0.8064Epoch 16/100068s 135ms/step - loss: 0.9948 - acc: 0.7733 - val_loss: 0.9198 - val_acc: 0.8022Epoch 17/100068s 136ms/step - loss: 0.9720 - acc: 0.7775 - val_loss: 0.9267 - val_acc: 0.7954Epoch 18/100068s 135ms/step - loss: 0.9548 - acc: 0.7813 - val_loss: 0.8897 - val_acc: 0.8043Epoch 19/100068s 135ms/step - loss: 0.9446 - acc: 0.7847 - val_loss: 0.8642 - val_acc: 0.8104Epoch 20/100068s 135ms/step - loss: 0.9290 - acc: 0.7873 - val_loss: 0.8666 - val_acc: 0.8119Epoch 21/100068s 135ms/step - loss: 0.9131 - acc: 0.7913 - val_loss: 0.8433 - val_acc: 0.8202Epoch 22/100068s 135ms/step - loss: 0.9099 - acc: 0.7912 - val_loss: 0.8735 - val_acc: 0.8077Epoch 23/100067s 135ms/step - loss: 0.9000 - acc: 0.7956 - val_loss: 0.8418 - val_acc: 0.8150Epoch 24/100068s 135ms/step - loss: 0.8962 - acc: 0.7966 - val_loss: 0.8452 - val_acc: 0.8181Epoch 25/100068s 135ms/step - loss: 0.8874 - acc: 0.7994 - val_loss: 0.8209 - val_acc: 0.8242Epoch 26/100068s 136ms/step - loss: 0.8810 - acc: 0.8021 - val_loss: 0.8378 - val_acc: 0.8202Epoch 27/100068s 135ms/step - loss: 0.8764 - acc: 0.8026 - val_loss: 0.8474 - val_acc: 0.8173Epoch 28/100067s 135ms/step - loss: 0.8706 - acc: 0.8040 - val_loss: 0.8239 - val_acc: 0.8230Epoch 29/100068s 135ms/step - loss: 0.8655 - acc: 0.8075 - val_loss: 0.8163 - val_acc: 0.8244Epoch 30/100068s 135ms/step - loss: 0.8600 - acc: 0.8074 - val_loss: 0.8065 - val_acc: 0.8288Epoch 31/100068s 135ms/step - loss: 0.8544 - acc: 0.8113 - val_loss: 0.8080 - val_acc: 0.8306Epoch 32/100068s 135ms/step - loss: 0.8510 - acc: 0.8121 - val_loss: 0.8152 - val_acc: 0.8304Epoch 33/100068s 135ms/step - loss: 0.8464 - acc: 0.8142 - val_loss: 0.7827 - val_acc: 0.8387Epoch 34/100068s 135ms/step - loss: 0.8429 - acc: 0.8166 - val_loss: 0.7738 - val_acc: 0.8453Epoch 35/100068s 135ms/step - loss: 0.8366 - acc: 0.8160 - val_loss: 0.7855 - val_acc: 0.8388Epoch 36/100068s 135ms/step - loss: 0.8352 - acc: 0.8191 - val_loss: 0.7651 - val_acc: 0.8468Epoch 37/100068s 135ms/step - loss: 0.8292 - acc: 0.8212 - val_loss: 0.7620 - val_acc: 0.8470Epoch 38/100068s 135ms/step - loss: 0.8319 - acc: 0.8208 - val_loss: 0.7890 - val_acc: 0.8376Epoch 39/100068s 136ms/step - loss: 0.8239 - acc: 0.8256 - val_loss: 0.7870 - val_acc: 0.8370Epoch 40/100068s 135ms/step - loss: 0.8266 - acc: 0.8216 - val_loss: 0.7975 - val_acc: 0.8331Epoch 41/100068s 135ms/step - loss: 0.8209 - acc: 0.8239 - val_loss: 0.7982 - val_acc: 0.8334Epoch 42/100068s 135ms/step - loss: 0.8135 - acc: 0.8276 - val_loss: 0.7722 - val_acc: 0.8427Epoch 43/100068s 135ms/step - loss: 0.8115 - acc: 0.8280 - val_loss: 0.7658 - val_acc: 0.8430Epoch 44/100067s 135ms/step - loss: 0.8166 - acc: 0.8259 - val_loss: 0.7388 - val_acc: 0.8559Epoch 45/100067s 135ms/step - loss: 0.8108 - acc: 0.8293 - val_loss: 0.7728 - val_acc: 0.8436Epoch 46/100068s 135ms/step - loss: 0.8046 - acc: 0.8303 - val_loss: 0.7684 - val_acc: 0.8434Epoch 47/100068s 136ms/step - loss: 0.8055 - acc: 0.8322 - val_loss: 0.7478 - val_acc: 0.8511Epoch 48/100068s 135ms/step - loss: 0.8100 - acc: 0.8290 - val_loss: 0.7644 - val_acc: 0.8445Epoch 49/100068s 135ms/step - loss: 0.8027 - acc: 0.8325 - val_loss: 0.7449 - val_acc: 0.8545Epoch 50/100067s 135ms/step - loss: 0.8052 - acc: 0.8299 - val_loss: 0.7941 - val_acc: 0.8377Epoch 51/100068s 135ms/step - loss: 0.7969 - acc: 0.8339 - val_loss: 0.7617 - val_acc: 0.8481Epoch 52/100068s 135ms/step - loss: 0.7989 - acc: 0.8335 - val_loss: 0.7559 - val_acc: 0.8550Epoch 53/100068s 136ms/step - loss: 0.7927 - acc: 0.8353 - val_loss: 0.7482 - val_acc: 0.8536Epoch 54/100068s 135ms/step - loss: 0.7931 - acc: 0.8365 - val_loss: 0.7405 - val_acc: 0.8570Epoch 55/100068s 135ms/step - loss: 0.7933 - acc: 0.8372 - val_loss: 0.7541 - val_acc: 0.8535Epoch 56/100068s 135ms/step - loss: 0.7887 - acc: 0.8389 - val_loss: 0.7805 - val_acc: 0.8436Epoch 57/100068s 135ms/step - loss: 0.7877 - acc: 0.8385 - val_loss: 0.7304 - val_acc: 0.8617Epoch 58/100068s 135ms/step - loss: 0.7836 - acc: 0.8404 - val_loss: 0.7630 - val_acc: 0.8480Epoch 59/100068s 135ms/step - loss: 0.7859 - acc: 0.8394 - val_loss: 0.7369 - val_acc: 0.8568Epoch 60/100068s 135ms/step - loss: 0.7864 - acc: 0.8376 - val_loss: 0.7606 - val_acc: 0.8492Epoch 61/100068s 135ms/step - loss: 0.7827 - acc: 0.8401 - val_loss: 0.7497 - val_acc: 0.8524Epoch 62/100068s 135ms/step - loss: 0.7804 - acc: 0.8427 - val_loss: 0.7526 - val_acc: 0.8559Epoch 63/100068s 135ms/step - loss: 0.7766 - acc: 0.8435 - val_loss: 0.7448 - val_acc: 0.8586Epoch 64/100068s 135ms/step - loss: 0.7792 - acc: 0.8419 - val_loss: 0.7605 - val_acc: 0.8511Epoch 65/100068s 135ms/step - loss: 0.7790 - acc: 0.8435 - val_loss: 0.7330 - val_acc: 0.8551Epoch 66/100068s 135ms/step - loss: 0.7748 - acc: 0.8435 - val_loss: 0.7528 - val_acc: 0.8543Epoch 67/100068s 135ms/step - loss: 0.7733 - acc: 0.8452 - val_loss: 0.7330 - val_acc: 0.8585Epoch 68/100068s 135ms/step - loss: 0.7759 - acc: 0.8438 - val_loss: 0.7497 - val_acc: 0.8520Epoch 69/100068s 135ms/step - loss: 0.7680 - acc: 0.8466 - val_loss: 0.7422 - val_acc: 0.8606Epoch 70/100068s 135ms/step - loss: 0.7662 - acc: 0.8473 - val_loss: 0.7185 - val_acc: 0.8633Epoch 71/100068s 135ms/step - loss: 0.7658 - acc: 0.8467 - val_loss: 0.7170 - val_acc: 0.8657Epoch 72/100068s 135ms/step - loss: 0.7681 - acc: 0.8464 - val_loss: 0.7325 - val_acc: 0.8600Epoch 73/100068s 135ms/step - loss: 0.7658 - acc: 0.8477 - val_loss: 0.7109 - val_acc: 0.8662Epoch 74/100068s 135ms/step - loss: 0.7616 - acc: 0.8499 - val_loss: 0.7028 - val_acc: 0.8733Epoch 75/100068s 135ms/step - loss: 0.7621 - acc: 0.8482 - val_loss: 0.7178 - val_acc: 0.8639Epoch 76/100068s 135ms/step - loss: 0.7606 - acc: 0.8496 - val_loss: 0.7096 - val_acc: 0.8674Epoch 77/100068s 135ms/step - loss: 0.7590 - acc: 0.8500 - val_loss: 0.7340 - val_acc: 0.8598Epoch 78/100068s 135ms/step - loss: 0.7639 - acc: 0.8475 - val_loss: 0.7212 - val_acc: 0.8655Epoch 79/100068s 135ms/step - loss: 0.7613 - acc: 0.8477 - val_loss: 0.7171 - val_acc: 0.8702Epoch 80/100067s 135ms/step - loss: 0.7562 - acc: 0.8518 - val_loss: 0.7336 - val_acc: 0.8594Epoch 81/100068s 136ms/step - loss: 0.7532 - acc: 0.8515 - val_loss: 0.7229 - val_acc: 0.8607Epoch 82/100068s 135ms/step - loss: 0.7511 - acc: 0.8541 - val_loss: 0.7062 - val_acc: 0.8688Epoch 83/100068s 135ms/step - loss: 0.7510 - acc: 0.8530 - val_loss: 0.6977 - val_acc: 0.8746Epoch 84/100068s 135ms/step - loss: 0.7562 - acc: 0.8524 - val_loss: 0.7319 - val_acc: 0.8595Epoch 85/100067s 135ms/step - loss: 0.7527 - acc: 0.8530 - val_loss: 0.7161 - val_acc: 0.8660Epoch 86/100067s 135ms/step - loss: 0.7523 - acc: 0.8524 - val_loss: 0.7244 - val_acc: 0.8654Epoch 87/100067s 135ms/step - loss: 0.7505 - acc: 0.8532 - val_loss: 0.7192 - val_acc: 0.8636Epoch 88/100068s 135ms/step - loss: 0.7528 - acc: 0.8516 - val_loss: 0.7316 - val_acc: 0.8645Epoch 89/100068s 135ms/step - loss: 0.7480 - acc: 0.8557 - val_loss: 0.7289 - val_acc: 0.8638Epoch 90/100068s 135ms/step - loss: 0.7435 - acc: 0.8550 - val_loss: 0.7020 - val_acc: 0.8763Epoch 91/100068s 135ms/step - loss: 0.7466 - acc: 0.8563 - val_loss: 0.6977 - val_acc: 0.8750Epoch 92/100068s 135ms/step - loss: 0.7438 - acc: 0.8561 - val_loss: 0.7171 - val_acc: 0.8643Epoch 93/100067s 135ms/step - loss: 0.7438 - acc: 0.8564 - val_loss: 0.7189 - val_acc: 0.8687Epoch 94/100068s 135ms/step - loss: 0.7442 - acc: 0.8566 - val_loss: 0.7072 - val_acc: 0.8685Epoch 95/100068s 135ms/step - loss: 0.7468 - acc: 0.8569 - val_loss: 0.7547 - val_acc: 0.8560Epoch 96/100068s 135ms/step - loss: 0.7468 - acc: 0.8547 - val_loss: 0.7080 - val_acc: 0.8699Epoch 97/100068s 135ms/step - loss: 0.7455 - acc: 0.8559 - val_loss: 0.7020 - val_acc: 0.8711Epoch 98/100068s 135ms/step - loss: 0.7427 - acc: 0.8544 - val_loss: 0.7352 - val_acc: 0.8610Epoch 99/100068s 136ms/step - loss: 0.7424 - acc: 0.8567 - val_loss: 0.7480 - val_acc: 0.8583Epoch 100/100068s 135ms/step - loss: 0.7397 - acc: 0.8579 - val_loss: 0.7151 - val_acc: 0.8650Epoch 101/100068s 135ms/step - loss: 0.7447 - acc: 0.8568 - val_loss: 0.7235 - val_acc: 0.8659Epoch 102/100068s 135ms/step - loss: 0.7367 - acc: 0.8598 - val_loss: 0.7229 - val_acc: 0.8623Epoch 103/100067s 135ms/step - loss: 0.7371 - acc: 0.8586 - val_loss: 0.6899 - val_acc: 0.8769Epoch 104/100068s 135ms/step - loss: 0.7401 - acc: 0.8567 - val_loss: 0.7273 - val_acc: 0.8616Epoch 105/100068s 135ms/step - loss: 0.7382 - acc: 0.8578 - val_loss: 0.7089 - val_acc: 0.8682Epoch 106/100068s 135ms/step - loss: 0.7386 - acc: 0.8580 - val_loss: 0.7158 - val_acc: 0.8659Epoch 107/100067s 135ms/step - loss: 0.7361 - acc: 0.8584 - val_loss: 0.7147 - val_acc: 0.8701Epoch 108/100067s 135ms/step - loss: 0.7408 - acc: 0.8580 - val_loss: 0.7083 - val_acc: 0.8686Epoch 109/100068s 135ms/step - loss: 0.7362 - acc: 0.8599 - val_loss: 0.7096 - val_acc: 0.8703Epoch 110/100067s 135ms/step - loss: 0.7335 - acc: 0.8600 - val_loss: 0.7148 - val_acc: 0.8683Epoch 111/100067s 135ms/step - loss: 0.7334 - acc: 0.8626 - val_loss: 0.7050 - val_acc: 0.8741Epoch 112/100068s 135ms/step - loss: 0.7360 - acc: 0.8586 - val_loss: 0.7150 - val_acc: 0.8682Epoch 113/100068s 136ms/step - loss: 0.7371 - acc: 0.8583 - val_loss: 0.7447 - val_acc: 0.8583Epoch 114/100068s 135ms/step - loss: 0.7352 - acc: 0.8599 - val_loss: 0.6937 - val_acc: 0.8755Epoch 115/100068s 135ms/step - loss: 0.7314 - acc: 0.8604 - val_loss: 0.7140 - val_acc: 0.8684Epoch 116/100068s 135ms/step - loss: 0.7333 - acc: 0.8607 - val_loss: 0.7305 - val_acc: 0.8686Epoch 117/100068s 135ms/step - loss: 0.7277 - acc: 0.8617 - val_loss: 0.7002 - val_acc: 0.8719Epoch 118/100068s 135ms/step - loss: 0.7356 - acc: 0.8580 - val_loss: 0.6926 - val_acc: 0.8763Epoch 119/100068s 135ms/step - loss: 0.7244 - acc: 0.8642 - val_loss: 0.7079 - val_acc: 0.8669Epoch 120/100068s 136ms/step - loss: 0.7302 - acc: 0.8613 - val_loss: 0.7113 - val_acc: 0.8695Epoch 121/100068s 135ms/step - loss: 0.7340 - acc: 0.8608 - val_loss: 0.7415 - val_acc: 0.8554Epoch 122/100068s 135ms/step - loss: 0.7304 - acc: 0.8608 - val_loss: 0.6978 - val_acc: 0.8760Epoch 123/100068s 135ms/step - loss: 0.7263 - acc: 0.8630 - val_loss: 0.6974 - val_acc: 0.8734Epoch 124/100068s 135ms/step - loss: 0.7261 - acc: 0.8625 - val_loss: 0.7109 - val_acc: 0.8715Epoch 125/100067s 135ms/step - loss: 0.7313 - acc: 0.8623 - val_loss: 0.6946 - val_acc: 0.8745Epoch 126/100067s 135ms/step - loss: 0.7277 - acc: 0.8620 - val_loss: 0.7178 - val_acc: 0.8685Epoch 127/100068s 135ms/step - loss: 0.7231 - acc: 0.8653 - val_loss: 0.6999 - val_acc: 0.8762Epoch 128/100068s 135ms/step - loss: 0.7252 - acc: 0.8635 - val_loss: 0.7009 - val_acc: 0.8718Epoch 129/100068s 135ms/step - loss: 0.7284 - acc: 0.8626 - val_loss: 0.7148 - val_acc: 0.8682Epoch 130/100068s 135ms/step - loss: 0.7236 - acc: 0.8646 - val_loss: 0.6945 - val_acc: 0.8746Epoch 131/100068s 135ms/step - loss: 0.7203 - acc: 0.8653 - val_loss: 0.7002 - val_acc: 0.8705Epoch 132/100068s 135ms/step - loss: 0.7248 - acc: 0.8626 - val_loss: 0.7097 - val_acc: 0.8718Epoch 133/100067s 135ms/step - loss: 0.7190 - acc: 0.8660 - val_loss: 0.6993 - val_acc: 0.8722Epoch 134/100068s 136ms/step - loss: 0.7206 - acc: 0.8645 - val_loss: 0.7042 - val_acc: 0.8763Epoch 135/100068s 135ms/step - loss: 0.7248 - acc: 0.8637 - val_loss: 0.6742 - val_acc: 0.8844Epoch 136/100068s 135ms/step - loss: 0.7181 - acc: 0.8650 - val_loss: 0.6972 - val_acc: 0.8721Epoch 137/100067s 135ms/step - loss: 0.7170 - acc: 0.8667 - val_loss: 0.7270 - val_acc: 0.8642Epoch 138/100068s 135ms/step - loss: 0.7209 - acc: 0.8649 - val_loss: 0.7107 - val_acc: 0.8687Epoch 139/100068s 136ms/step - loss: 0.7195 - acc: 0.8652 - val_loss: 0.6993 - val_acc: 0.8752Epoch 140/100068s 135ms/step - loss: 0.7229 - acc: 0.8647 - val_loss: 0.6949 - val_acc: 0.8800Epoch 141/100067s 135ms/step - loss: 0.7154 - acc: 0.8674 - val_loss: 0.6828 - val_acc: 0.8780Epoch 142/100067s 135ms/step - loss: 0.7146 - acc: 0.8675 - val_loss: 0.6799 - val_acc: 0.8818Epoch 143/100068s 135ms/step - loss: 0.7131 - acc: 0.8679 - val_loss: 0.7237 - val_acc: 0.8655Epoch 144/100068s 135ms/step - loss: 0.7167 - acc: 0.8662 - val_loss: 0.7140 - val_acc: 0.8696Epoch 145/100068s 136ms/step - loss: 0.7131 - acc: 0.8677 - val_loss: 0.7086 - val_acc: 0.8696Epoch 146/100067s 135ms/step - loss: 0.7184 - acc: 0.8665 - val_loss: 0.7058 - val_acc: 0.8729Epoch 147/100068s 135ms/step - loss: 0.7179 - acc: 0.8654 - val_loss: 0.7021 - val_acc: 0.8741Epoch 148/100067s 135ms/step - loss: 0.7176 - acc: 0.8671 - val_loss: 0.6892 - val_acc: 0.8795Epoch 149/100068s 135ms/step - loss: 0.7123 - acc: 0.8685 - val_loss: 0.7027 - val_acc: 0.8700Epoch 150/100068s 136ms/step - loss: 0.7146 - acc: 0.8671 - val_loss: 0.6926 - val_acc: 0.8755Epoch 151/100068s 135ms/step - loss: 0.7122 - acc: 0.8651 - val_loss: 0.7179 - val_acc: 0.8685Epoch 152/100068s 136ms/step - loss: 0.7149 - acc: 0.8675 - val_loss: 0.7136 - val_acc: 0.8690Epoch 153/100068s 135ms/step - loss: 0.7141 - acc: 0.8669 - val_loss: 0.7193 - val_acc: 0.8672Epoch 154/100068s 136ms/step - loss: 0.7084 - acc: 0.8684 - val_loss: 0.6779 - val_acc: 0.8826Epoch 155/100067s 135ms/step - loss: 0.7143 - acc: 0.8671 - val_loss: 0.7092 - val_acc: 0.8685Epoch 156/100068s 136ms/step - loss: 0.7118 - acc: 0.8674 - val_loss: 0.7010 - val_acc: 0.8732Epoch 157/100069s 138ms/step - loss: 0.7126 - acc: 0.8677 - val_loss: 0.6918 - val_acc: 0.8766Epoch 158/100068s 137ms/step - loss: 0.7064 - acc: 0.8701 - val_loss: 0.7253 - val_acc: 0.8636Epoch 159/100068s 137ms/step - loss: 0.7107 - acc: 0.8674 - val_loss: 0.7008 - val_acc: 0.8745Epoch 160/100068s 137ms/step - loss: 0.7097 - acc: 0.8698 - val_loss: 0.6922 - val_acc: 0.8771Epoch 161/100068s 137ms/step - loss: 0.7091 - acc: 0.8675 - val_loss: 0.6786 - val_acc: 0.8813Epoch 162/100069s 138ms/step - loss: 0.7117 - acc: 0.8680 - val_loss: 0.7017 - val_acc: 0.8740Epoch 163/100069s 137ms/step - loss: 0.7110 - acc: 0.8681 - val_loss: 0.6862 - val_acc: 0.8800Epoch 164/100068s 137ms/step - loss: 0.7099 - acc: 0.8693 - val_loss: 0.7053 - val_acc: 0.8709Epoch 165/100069s 138ms/step - loss: 0.7104 - acc: 0.8694 - val_loss: 0.6846 - val_acc: 0.8828Epoch 166/100068s 136ms/step - loss: 0.7078 - acc: 0.8715 - val_loss: 0.6968 - val_acc: 0.8749Epoch 167/100068s 136ms/step - loss: 0.7076 - acc: 0.8719 - val_loss: 0.6872 - val_acc: 0.8782Epoch 168/100068s 136ms/step - loss: 0.7099 - acc: 0.8679 - val_loss: 0.6928 - val_acc: 0.8755Epoch 169/100068s 136ms/step - loss: 0.7101 - acc: 0.8678 - val_loss: 0.6947 - val_acc: 0.8786Epoch 170/100068s 137ms/step - loss: 0.7097 - acc: 0.8717 - val_loss: 0.6886 - val_acc: 0.8789Epoch 171/100069s 137ms/step - loss: 0.7070 - acc: 0.8702 - val_loss: 0.6878 - val_acc: 0.8793Epoch 172/100069s 137ms/step - loss: 0.7117 - acc: 0.8679 - val_loss: 0.6783 - val_acc: 0.8836Epoch 173/100068s 137ms/step - loss: 0.7102 - acc: 0.8687 - val_loss: 0.6709 - val_acc: 0.8865Epoch 174/100069s 137ms/step - loss: 0.7038 - acc: 0.8717 - val_loss: 0.6839 - val_acc: 0.8804Epoch 175/100068s 137ms/step - loss: 0.7062 - acc: 0.8713 - val_loss: 0.6934 - val_acc: 0.8780Epoch 176/100068s 137ms/step - loss: 0.7092 - acc: 0.8684 - val_loss: 0.7045 - val_acc: 0.8737Epoch 177/100068s 137ms/step - loss: 0.7048 - acc: 0.8703 - val_loss: 0.6935 - val_acc: 0.8764Epoch 178/100068s 137ms/step - loss: 0.7056 - acc: 0.8713 - val_loss: 0.6825 - val_acc: 0.8800Epoch 179/100069s 137ms/step - loss: 0.7027 - acc: 0.8722 - val_loss: 0.6860 - val_acc: 0.8812Epoch 180/100067s 135ms/step - loss: 0.7056 - acc: 0.8699 - val_loss: 0.6882 - val_acc: 0.8762Epoch 181/100067s 135ms/step - loss: 0.6974 - acc: 0.8745 - val_loss: 0.7030 - val_acc: 0.8704Epoch 182/100067s 135ms/step - loss: 0.7028 - acc: 0.8714 - val_loss: 0.6754 - val_acc: 0.8860Epoch 183/100067s 135ms/step - loss: 0.7022 - acc: 0.8715 - val_loss: 0.6635 - val_acc: 0.8842Epoch 184/100068s 136ms/step - loss: 0.7034 - acc: 0.8704 - val_loss: 0.6905 - val_acc: 0.8762Epoch 185/100067s 135ms/step - loss: 0.7058 - acc: 0.8709 - val_loss: 0.7066 - val_acc: 0.8740Epoch 186/100067s 135ms/step - loss: 0.7016 - acc: 0.8726 - val_loss: 0.6842 - val_acc: 0.8784Epoch 187/100067s 135ms/step - loss: 0.6999 - acc: 0.8719 - val_loss: 0.7051 - val_acc: 0.8731Epoch 188/100067s 135ms/step - loss: 0.7026 - acc: 0.8710 - val_loss: 0.6811 - val_acc: 0.8811Epoch 189/100068s 135ms/step - loss: 0.7040 - acc: 0.8711 - val_loss: 0.6794 - val_acc: 0.8786Epoch 190/100067s 135ms/step - loss: 0.7004 - acc: 0.8728 - val_loss: 0.6594 - val_acc: 0.8916Epoch 191/100068s 136ms/step - loss: 0.6982 - acc: 0.8747 - val_loss: 0.6616 - val_acc: 0.8850Epoch 192/100068s 135ms/step - loss: 0.7036 - acc: 0.8718 - val_loss: 0.6959 - val_acc: 0.8730Epoch 193/100067s 135ms/step - loss: 0.7017 - acc: 0.8708 - val_loss: 0.6671 - val_acc: 0.8862Epoch 194/100067s 135ms/step - loss: 0.6982 - acc: 0.8738 - val_loss: 0.6885 - val_acc: 0.8790Epoch 195/100068s 136ms/step - loss: 0.6996 - acc: 0.8714 - val_loss: 0.6892 - val_acc: 0.8770Epoch 196/100068s 136ms/step - loss: 0.7026 - acc: 0.8706 - val_loss: 0.6824 - val_acc: 0.8792Epoch 197/100068s 136ms/step - loss: 0.7061 - acc: 0.8695 - val_loss: 0.6893 - val_acc: 0.8793Epoch 198/100068s 135ms/step - loss: 0.7023 - acc: 0.8714 - val_loss: 0.6797 - val_acc: 0.8819Epoch 199/100067s 135ms/step - loss: 0.7021 - acc: 0.8726 - val_loss: 0.6969 - val_acc: 0.8754Epoch 200/100068s 136ms/step - loss: 0.7023 - acc: 0.8711 - val_loss: 0.6922 - val_acc: 0.8758Epoch 201/100068s 135ms/step - loss: 0.7050 - acc: 0.8705 - val_loss: 0.6879 - val_acc: 0.8792Epoch 202/100068s 135ms/step - loss: 0.7012 - acc: 0.8713 - val_loss: 0.6756 - val_acc: 0.8845Epoch 203/100068s 136ms/step - loss: 0.7021 - acc: 0.8726 - val_loss: 0.6542 - val_acc: 0.8904Epoch 204/100068s 136ms/step - loss: 0.6981 - acc: 0.8741 - val_loss: 0.7060 - val_acc: 0.8739Epoch 205/100068s 135ms/step - loss: 0.7008 - acc: 0.8718 - val_loss: 0.6938 - val_acc: 0.8741Epoch 206/100068s 136ms/step - loss: 0.6974 - acc: 0.8725 - val_loss: 0.6786 - val_acc: 0.8833Epoch 207/100067s 135ms/step - loss: 0.6938 - acc: 0.8739 - val_loss: 0.6928 - val_acc: 0.8750Epoch 208/100068s 135ms/step - loss: 0.7075 - acc: 0.8690 - val_loss: 0.6770 - val_acc: 0.8806Epoch 209/100068s 136ms/step - loss: 0.6978 - acc: 0.8723 - val_loss: 0.6913 - val_acc: 0.8812Epoch 210/100067s 135ms/step - loss: 0.6974 - acc: 0.8727 - val_loss: 0.6764 - val_acc: 0.8827Epoch 211/100068s 136ms/step - loss: 0.6998 - acc: 0.8724 - val_loss: 0.7139 - val_acc: 0.8700Epoch 212/100068s 136ms/step - loss: 0.6975 - acc: 0.8740 - val_loss: 0.6851 - val_acc: 0.8805Epoch 213/100068s 136ms/step - loss: 0.7032 - acc: 0.8704 - val_loss: 0.7101 - val_acc: 0.8712Epoch 214/100068s 135ms/step - loss: 0.6979 - acc: 0.8732 - val_loss: 0.7108 - val_acc: 0.8756Epoch 215/100068s 136ms/step - loss: 0.6986 - acc: 0.8749 - val_loss: 0.7092 - val_acc: 0.8701Epoch 216/100068s 135ms/step - loss: 0.6921 - acc: 0.8757 - val_loss: 0.6868 - val_acc: 0.8792Epoch 217/100068s 135ms/step - loss: 0.6930 - acc: 0.8755 - val_loss: 0.7097 - val_acc: 0.8721Epoch 218/100068s 135ms/step - loss: 0.7039 - acc: 0.8713 - val_loss: 0.6901 - val_acc: 0.8789Epoch 219/100068s 136ms/step - loss: 0.6931 - acc: 0.8757 - val_loss: 0.6927 - val_acc: 0.8793Epoch 220/100068s 136ms/step - loss: 0.6991 - acc: 0.8734 - val_loss: 0.6946 - val_acc: 0.8777Epoch 221/100068s 135ms/step - loss: 0.6950 - acc: 0.8739 - val_loss: 0.6740 - val_acc: 0.8819Epoch 222/100068s 136ms/step - loss: 0.6970 - acc: 0.8739 - val_loss: 0.6847 - val_acc: 0.8787Epoch 223/100068s 136ms/step - loss: 0.7006 - acc: 0.8719 - val_loss: 0.7139 - val_acc: 0.8707Epoch 224/100068s 135ms/step - loss: 0.7014 - acc: 0.8720 - val_loss: 0.6901 - val_acc: 0.8768Epoch 225/100068s 136ms/step - loss: 0.6981 - acc: 0.8739 - val_loss: 0.6758 - val_acc: 0.8853Epoch 226/100068s 136ms/step - loss: 0.6921 - acc: 0.8739 - val_loss: 0.6743 - val_acc: 0.8844Epoch 227/100068s 136ms/step - loss: 0.6983 - acc: 0.8734 - val_loss: 0.7031 - val_acc: 0.8736Epoch 228/100068s 136ms/step - loss: 0.6954 - acc: 0.8734 - val_loss: 0.7181 - val_acc: 0.8663Epoch 229/100068s 136ms/step - loss: 0.6893 - acc: 0.8759 - val_loss: 0.6982 - val_acc: 0.8740Epoch 230/100068s 135ms/step - loss: 0.6964 - acc: 0.8736 - val_loss: 0.6927 - val_acc: 0.8748Epoch 231/100068s 136ms/step - loss: 0.6987 - acc: 0.8742 - val_loss: 0.6898 - val_acc: 0.8772Epoch 232/100067s 135ms/step - loss: 0.6980 - acc: 0.8731 - val_loss: 0.6862 - val_acc: 0.8810Epoch 233/100068s 136ms/step - loss: 0.6975 - acc: 0.8749 - val_loss: 0.6987 - val_acc: 0.8783Epoch 234/100067s 135ms/step - loss: 0.6892 - acc: 0.8778 - val_loss: 0.6902 - val_acc: 0.8773Epoch 235/100068s 135ms/step - loss: 0.6925 - acc: 0.8762 - val_loss: 0.6787 - val_acc: 0.8799Epoch 236/100068s 136ms/step - loss: 0.6954 - acc: 0.8735 - val_loss: 0.6910 - val_acc: 0.8797Epoch 237/100068s 136ms/step - loss: 0.6963 - acc: 0.8746 - val_loss: 0.6886 - val_acc: 0.8785Epoch 238/100068s 136ms/step - loss: 0.6950 - acc: 0.8764 - val_loss: 0.7008 - val_acc: 0.8766Epoch 239/100067s 135ms/step - loss: 0.6969 - acc: 0.8749 - val_loss: 0.7100 - val_acc: 0.8736Epoch 240/100068s 136ms/step - loss: 0.6905 - acc: 0.8757 - val_loss: 0.6971 - val_acc: 0.8733Epoch 241/100068s 136ms/step - loss: 0.6912 - acc: 0.8740 - val_loss: 0.6809 - val_acc: 0.8805Epoch 242/100068s 136ms/step - loss: 0.6949 - acc: 0.8729 - val_loss: 0.6903 - val_acc: 0.8760Epoch 243/100069s 138ms/step - loss: 0.6938 - acc: 0.8753 - val_loss: 0.6809 - val_acc: 0.8823Epoch 244/100068s 136ms/step - loss: 0.6912 - acc: 0.8754 - val_loss: 0.6700 - val_acc: 0.8829Epoch 245/100069s 137ms/step - loss: 0.6939 - acc: 0.8766 - val_loss: 0.6691 - val_acc: 0.8847Epoch 246/100069s 138ms/step - loss: 0.6885 - acc: 0.8756 - val_loss: 0.7018 - val_acc: 0.8782Epoch 247/100069s 138ms/step - loss: 0.6916 - acc: 0.8766 - val_loss: 0.6896 - val_acc: 0.8789Epoch 248/100069s 138ms/step - loss: 0.6918 - acc: 0.8751 - val_loss: 0.7025 - val_acc: 0.8735Epoch 249/100069s 138ms/step - loss: 0.6944 - acc: 0.8756 - val_loss: 0.6754 - val_acc: 0.8811Epoch 250/100069s 138ms/step - loss: 0.6845 - acc: 0.8766 - val_loss: 0.6937 - val_acc: 0.8776Epoch 251/100069s 138ms/step - loss: 0.6915 - acc: 0.8753 - val_loss: 0.6944 - val_acc: 0.8773Epoch 252/100069s 138ms/step - loss: 0.6923 - acc: 0.8751 - val_loss: 0.6830 - val_acc: 0.8790Epoch 253/100069s 138ms/step - loss: 0.6889 - acc: 0.8756 - val_loss: 0.7251 - val_acc: 0.8658Epoch 254/100069s 137ms/step - loss: 0.6963 - acc: 0.8741 - val_loss: 0.6919 - val_acc: 0.8777Epoch 255/100069s 137ms/step - loss: 0.6920 - acc: 0.8759 - val_loss: 0.7098 - val_acc: 0.8706Epoch 256/100069s 138ms/step - loss: 0.6896 - acc: 0.8750 - val_loss: 0.6964 - val_acc: 0.8772Epoch 257/100069s 137ms/step - loss: 0.6891 - acc: 0.8757 - val_loss: 0.6604 - val_acc: 0.8871Epoch 258/100069s 137ms/step - loss: 0.6932 - acc: 0.8753 - val_loss: 0.6820 - val_acc: 0.8803Epoch 259/100069s 138ms/step - loss: 0.6863 - acc: 0.8767 - val_loss: 0.7197 - val_acc: 0.8710Epoch 260/100068s 137ms/step - loss: 0.6900 - acc: 0.8762 - val_loss: 0.6588 - val_acc: 0.8907Epoch 261/100068s 137ms/step - loss: 0.6912 - acc: 0.8750 - val_loss: 0.6815 - val_acc: 0.8833Epoch 262/100069s 137ms/step - loss: 0.6893 - acc: 0.8750 - val_loss: 0.6795 - val_acc: 0.8831Epoch 263/100069s 137ms/step - loss: 0.6920 - acc: 0.8755 - val_loss: 0.6830 - val_acc: 0.8822Epoch 264/100068s 137ms/step - loss: 0.6924 - acc: 0.8743 - val_loss: 0.6989 - val_acc: 0.8762Epoch 265/100069s 138ms/step - loss: 0.6905 - acc: 0.8763 - val_loss: 0.6836 - val_acc: 0.8806Epoch 266/100069s 137ms/step - loss: 0.6883 - acc: 0.8759 - val_loss: 0.6731 - val_acc: 0.8855Epoch 267/100069s 138ms/step - loss: 0.6852 - acc: 0.8768 - val_loss: 0.6898 - val_acc: 0.8794Epoch 268/100069s 138ms/step - loss: 0.6901 - acc: 0.8754 - val_loss: 0.6825 - val_acc: 0.8804Epoch 269/100069s 138ms/step - loss: 0.6929 - acc: 0.8742 - val_loss: 0.6916 - val_acc: 0.8745Epoch 270/100069s 138ms/step - loss: 0.6870 - acc: 0.8772 - val_loss: 0.6772 - val_acc: 0.8804Epoch 271/100068s 137ms/step - loss: 0.6891 - acc: 0.8761 - val_loss: 0.6891 - val_acc: 0.8756Epoch 272/100068s 137ms/step - loss: 0.6863 - acc: 0.8768 - val_loss: 0.6851 - val_acc: 0.8813Epoch 273/100068s 137ms/step - loss: 0.6891 - acc: 0.8765 - val_loss: 0.6921 - val_acc: 0.8776Epoch 274/100069s 138ms/step - loss: 0.6853 - acc: 0.8779 - val_loss: 0.6785 - val_acc: 0.8820Epoch 275/100069s 138ms/step - loss: 0.6879 - acc: 0.8767 - val_loss: 0.6994 - val_acc: 0.8728Epoch 276/100069s 138ms/step - loss: 0.6869 - acc: 0.8767 - val_loss: 0.6949 - val_acc: 0.8732Epoch 277/100069s 137ms/step - loss: 0.6796 - acc: 0.8793 - val_loss: 0.6813 - val_acc: 0.8820Epoch 278/100069s 138ms/step - loss: 0.6909 - acc: 0.8746 - val_loss: 0.6745 - val_acc: 0.8841Epoch 279/100069s 138ms/step - loss: 0.6837 - acc: 0.8777 - val_loss: 0.6951 - val_acc: 0.8761Epoch 280/100069s 137ms/step - loss: 0.6882 - acc: 0.8769 - val_loss: 0.6828 - val_acc: 0.8805Epoch 281/100069s 138ms/step - loss: 0.6909 - acc: 0.8767 - val_loss: 0.6801 - val_acc: 0.8836Epoch 282/100069s 137ms/step - loss: 0.6890 - acc: 0.8743 - val_loss: 0.6931 - val_acc: 0.8757Epoch 283/100069s 138ms/step - loss: 0.6871 - acc: 0.8772 - val_loss: 0.6791 - val_acc: 0.8837Epoch 284/100069s 138ms/step - loss: 0.6846 - acc: 0.8796 - val_loss: 0.7228 - val_acc: 0.8674Epoch 285/100069s 138ms/step - loss: 0.6857 - acc: 0.8792 - val_loss: 0.7068 - val_acc: 0.8735Epoch 286/100069s 138ms/step - loss: 0.6891 - acc: 0.8759 - val_loss: 0.7089 - val_acc: 0.8735Epoch 287/100069s 137ms/step - loss: 0.6927 - acc: 0.8770 - val_loss: 0.6755 - val_acc: 0.8823Epoch 288/100069s 137ms/step - loss: 0.6878 - acc: 0.8770 - val_loss: 0.6939 - val_acc: 0.8761Epoch 289/100068s 137ms/step - loss: 0.6858 - acc: 0.8795 - val_loss: 0.6844 - val_acc: 0.8829Epoch 290/100069s 137ms/step - loss: 0.6901 - acc: 0.8774 - val_loss: 0.6603 - val_acc: 0.8877Epoch 291/100069s 138ms/step - loss: 0.6827 - acc: 0.8805 - val_loss: 0.6700 - val_acc: 0.8877Epoch 292/100069s 137ms/step - loss: 0.6875 - acc: 0.8770 - val_loss: 0.6843 - val_acc: 0.8802Epoch 293/100069s 138ms/step - loss: 0.6861 - acc: 0.8795 - val_loss: 0.6889 - val_acc: 0.8812Epoch 294/100068s 137ms/step - loss: 0.6896 - acc: 0.8759 - val_loss: 0.6688 - val_acc: 0.8874Epoch 295/100069s 138ms/step - loss: 0.6792 - acc: 0.8805 - val_loss: 0.6813 - val_acc: 0.8802Epoch 296/100069s 138ms/step - loss: 0.6946 - acc: 0.8733 - val_loss: 0.6697 - val_acc: 0.8858Epoch 297/100069s 138ms/step - loss: 0.6887 - acc: 0.8755 - val_loss: 0.6707 - val_acc: 0.8848Epoch 298/100069s 138ms/step - loss: 0.6875 - acc: 0.8765 - val_loss: 0.7025 - val_acc: 0.8718Epoch 299/100069s 137ms/step - loss: 0.6853 - acc: 0.8789 - val_loss: 0.6842 - val_acc: 0.8805Epoch 300/100069s 138ms/step - loss: 0.6806 - acc: 0.8809 - val_loss: 0.6948 - val_acc: 0.8809Epoch 301/1000lr changed to 0.01000000014901161269s 138ms/step - loss: 0.5763 - acc: 0.9142 - val_loss: 0.5780 - val_acc: 0.9169Epoch 302/100069s 138ms/step - loss: 0.5127 - acc: 0.9355 - val_loss: 0.5618 - val_acc: 0.9209Epoch 303/100068s 137ms/step - loss: 0.4950 - acc: 0.9401 - val_loss: 0.5561 - val_acc: 0.9223Epoch 304/100068s 137ms/step - loss: 0.4744 - acc: 0.9449 - val_loss: 0.5485 - val_acc: 0.9229Epoch 305/100068s 137ms/step - loss: 0.4602 - acc: 0.9489 - val_loss: 0.5469 - val_acc: 0.9206Epoch 306/100069s 137ms/step - loss: 0.4533 - acc: 0.9479 - val_loss: 0.5368 - val_acc: 0.9209Epoch 307/100069s 137ms/step - loss: 0.4463 - acc: 0.9498 - val_loss: 0.5294 - val_acc: 0.9230Epoch 308/100069s 137ms/step - loss: 0.4371 - acc: 0.9508 - val_loss: 0.5304 - val_acc: 0.9228Epoch 309/100069s 137ms/step - loss: 0.4276 - acc: 0.9515 - val_loss: 0.5217 - val_acc: 0.9236Epoch 310/100068s 136ms/step - loss: 0.4185 - acc: 0.9542 - val_loss: 0.5202 - val_acc: 0.9235Epoch 311/100069s 138ms/step - loss: 0.4079 - acc: 0.9563 - val_loss: 0.5213 - val_acc: 0.9224Epoch 312/100069s 137ms/step - loss: 0.4028 - acc: 0.9559 - val_loss: 0.5149 - val_acc: 0.9241Epoch 313/100068s 136ms/step - loss: 0.3940 - acc: 0.9582 - val_loss: 0.5182 - val_acc: 0.9229Epoch 314/100069s 138ms/step - loss: 0.3913 - acc: 0.9584 - val_loss: 0.5063 - val_acc: 0.9222Epoch 315/100069s 138ms/step - loss: 0.3815 - acc: 0.9599 - val_loss: 0.5065 - val_acc: 0.9242Epoch 316/100069s 138ms/step - loss: 0.3779 - acc: 0.9596 - val_loss: 0.5105 - val_acc: 0.9197Epoch 317/100069s 138ms/step - loss: 0.3734 - acc: 0.9607 - val_loss: 0.4951 - val_acc: 0.9242Epoch 318/100069s 138ms/step - loss: 0.3668 - acc: 0.9608 - val_loss: 0.4984 - val_acc: 0.9226Epoch 319/100068s 137ms/step - loss: 0.3600 - acc: 0.9628 - val_loss: 0.5003 - val_acc: 0.9195Epoch 320/100068s 137ms/step - loss: 0.3562 - acc: 0.9622 - val_loss: 0.4927 - val_acc: 0.9206Epoch 321/100069s 138ms/step - loss: 0.3551 - acc: 0.9619 - val_loss: 0.4883 - val_acc: 0.9233Epoch 322/100069s 138ms/step - loss: 0.3467 - acc: 0.9635 - val_loss: 0.4820 - val_acc: 0.9247Epoch 323/100069s 138ms/step - loss: 0.3468 - acc: 0.9621 - val_loss: 0.4795 - val_acc: 0.9225Epoch 324/100068s 136ms/step - loss: 0.3386 - acc: 0.9651 - val_loss: 0.4927 - val_acc: 0.9205Epoch 325/100068s 135ms/step - loss: 0.3368 - acc: 0.9644 - val_loss: 0.4823 - val_acc: 0.9205Epoch 326/100068s 136ms/step - loss: 0.3284 - acc: 0.9667 - val_loss: 0.4691 - val_acc: 0.9236Epoch 327/100069s 138ms/step - loss: 0.3255 - acc: 0.9658 - val_loss: 0.4734 - val_acc: 0.9252Epoch 328/100068s 136ms/step - loss: 0.3255 - acc: 0.9648 - val_loss: 0.4795 - val_acc: 0.9230Epoch 329/100068s 136ms/step - loss: 0.3257 - acc: 0.9638 - val_loss: 0.4681 - val_acc: 0.9223Epoch 330/100068s 136ms/step - loss: 0.3181 - acc: 0.9648 - val_loss: 0.4670 - val_acc: 0.9215Epoch 331/100068s 136ms/step - loss: 0.3138 - acc: 0.9660 - val_loss: 0.4821 - val_acc: 0.9185Epoch 332/100068s 136ms/step - loss: 0.3140 - acc: 0.9648 - val_loss: 0.4727 - val_acc: 0.9202Epoch 333/100069s 137ms/step - loss: 0.3102 - acc: 0.9663 - val_loss: 0.4632 - val_acc: 0.9231Epoch 334/100068s 137ms/step - loss: 0.3085 - acc: 0.9663 - val_loss: 0.4611 - val_acc: 0.9240Epoch 335/100068s 137ms/step - loss: 0.3019 - acc: 0.9679 - val_loss: 0.4614 - val_acc: 0.9238Epoch 336/100069s 138ms/step - loss: 0.3046 - acc: 0.9654 - val_loss: 0.4635 - val_acc: 0.9202Epoch 337/100068s 137ms/step - loss: 0.3015 - acc: 0.9660 - val_loss: 0.4599 - val_acc: 0.9228Epoch 338/100069s 137ms/step - loss: 0.2992 - acc: 0.9662 - val_loss: 0.4577 - val_acc: 0.9207Epoch 339/100069s 138ms/step - loss: 0.2942 - acc: 0.9669 - val_loss: 0.4702 - val_acc: 0.9172Epoch 340/100069s 137ms/step - loss: 0.2924 - acc: 0.9675 - val_loss: 0.4545 - val_acc: 0.9211...Epoch 597/100068s 135ms/step - loss: 0.2366 - acc: 0.9703 - val_loss: 0.4557 - val_acc: 0.9103Epoch 598/100068s 135ms/step - loss: 0.2399 - acc: 0.9697 - val_loss: 0.4449 - val_acc: 0.9117Epoch 599/100067s 135ms/step - loss: 0.2397 - acc: 0.9689 - val_loss: 0.4359 - val_acc: 0.9147Epoch 600/100068s 136ms/step - loss: 0.2341 - acc: 0.9717 - val_loss: 0.4224 - val_acc: 0.9169Epoch 601/1000lr changed to 0.000999999977648258368s 136ms/step - loss: 0.2082 - acc: 0.9813 - val_loss: 0.3916 - val_acc: 0.9268Epoch 602/100068s 136ms/step - loss: 0.1952 - acc: 0.9865 - val_loss: 0.3854 - val_acc: 0.9281Epoch 603/100068s 136ms/step - loss: 0.1878 - acc: 0.9881 - val_loss: 0.3852 - val_acc: 0.9299Epoch 604/100068s 136ms/step - loss: 0.1846 - acc: 0.9899 - val_loss: 0.3842 - val_acc: 0.9298Epoch 605/100068s 135ms/step - loss: 0.1826 - acc: 0.9909 - val_loss: 0.3829 - val_acc: 0.9326Epoch 606/100068s 136ms/step - loss: 0.1808 - acc: 0.9912 - val_loss: 0.3838 - val_acc: 0.9305Epoch 607/100068s 136ms/step - loss: 0.1771 - acc: 0.9927 - val_loss: 0.3851 - val_acc: 0.9303Epoch 608/100068s 136ms/step - loss: 0.1768 - acc: 0.9922 - val_loss: 0.3898 - val_acc: 0.9304Epoch 609/100068s 135ms/step - loss: 0.1758 - acc: 0.9926 - val_loss: 0.3878 - val_acc: 0.9309Epoch 610/100068s 136ms/step - loss: 0.1739 - acc: 0.9931 - val_loss: 0.3887 - val_acc: 0.9294Epoch 611/100068s 136ms/step - loss: 0.1731 - acc: 0.9934 - val_loss: 0.3874 - val_acc: 0.9311Epoch 612/100068s 136ms/step - loss: 0.1725 - acc: 0.9935 - val_loss: 0.3898 - val_acc: 0.9297Epoch 613/100068s 135ms/step - loss: 0.1717 - acc: 0.9937 - val_loss: 0.3900 - val_acc: 0.9298Epoch 614/100068s 136ms/step - loss: 0.1705 - acc: 0.9937 - val_loss: 0.3912 - val_acc: 0.9299Epoch 615/100068s 136ms/step - loss: 0.1709 - acc: 0.9934 - val_loss: 0.3898 - val_acc: 0.9307Epoch 616/100068s 136ms/step - loss: 0.1686 - acc: 0.9948 - val_loss: 0.3905 - val_acc: 0.9311Epoch 617/100068s 136ms/step - loss: 0.1695 - acc: 0.9942 - val_loss: 0.3948 - val_acc: 0.9303Epoch 618/100068s 136ms/step - loss: 0.1688 - acc: 0.9941 - val_loss: 0.3936 - val_acc: 0.9298Epoch 619/100068s 136ms/step - loss: 0.1679 - acc: 0.9945 - val_loss: 0.3950 - val_acc: 0.9290Epoch 620/100068s 136ms/step - loss: 0.1675 - acc: 0.9941 - val_loss: 0.3940 - val_acc: 0.9300Epoch 621/100068s 136ms/step - loss: 0.1651 - acc: 0.9949 - val_loss: 0.3956 - val_acc: 0.9309Epoch 622/100068s 136ms/step - loss: 0.1653 - acc: 0.9951 - val_loss: 0.3950 - val_acc: 0.9306Epoch 623/100068s 136ms/step - loss: 0.1656 - acc: 0.9946 - val_loss: 0.3947 - val_acc: 0.9306Epoch 624/100068s 136ms/step - loss: 0.1644 - acc: 0.9949 - val_loss: 0.3946 - val_acc: 0.9304Epoch 625/100068s 136ms/step - loss: 0.1636 - acc: 0.9951 - val_loss: 0.3944 - val_acc: 0.9296Epoch 626/100068s 136ms/step - loss: 0.1630 - acc: 0.9951 - val_loss: 0.3937 - val_acc: 0.9295Epoch 627/100068s 136ms/step - loss: 0.1630 - acc: 0.9953 - val_loss: 0.3959 - val_acc: 0.9296Epoch 628/100068s 136ms/step - loss: 0.1627 - acc: 0.9954 - val_loss: 0.3939 - val_acc: 0.9289Epoch 629/100068s 136ms/step - loss: 0.1630 - acc: 0.9947 - val_loss: 0.3937 - val_acc: 0.9303Epoch 630/100068s 135ms/step - loss: 0.1614 - acc: 0.9958 - val_loss: 0.3909 - val_acc: 0.9316Epoch 631/100068s 137ms/step - loss: 0.1624 - acc: 0.9950 - val_loss: 0.3922 - val_acc: 0.9310Epoch 632/100068s 135ms/step - loss: 0.1611 - acc: 0.9954 - val_loss: 0.3907 - val_acc: 0.9313Epoch 633/100068s 136ms/step - loss: 0.1599 - acc: 0.9955 - val_loss: 0.3893 - val_acc: 0.9295Epoch 634/100068s 136ms/step - loss: 0.1600 - acc: 0.9954 - val_loss: 0.3886 - val_acc: 0.9308Epoch 635/100068s 136ms/step - loss: 0.1593 - acc: 0.9953 - val_loss: 0.3926 - val_acc: 0.9297Epoch 636/100068s 136ms/step - loss: 0.1594 - acc: 0.9950 - val_loss: 0.3945 - val_acc: 0.9289Epoch 637/100068s 136ms/step - loss: 0.1595 - acc: 0.9955 - val_loss: 0.3937 - val_acc: 0.9306Epoch 638/100068s 136ms/step - loss: 0.1591 - acc: 0.9958 - val_loss: 0.3882 - val_acc: 0.9306Epoch 639/100068s 135ms/step - loss: 0.1586 - acc: 0.9959 - val_loss: 0.3893 - val_acc: 0.9309Epoch 640/100068s 136ms/step - loss: 0.1588 - acc: 0.9956 - val_loss: 0.3935 - val_acc: 0.9300Epoch 641/100068s 135ms/step - loss: 0.1571 - acc: 0.9960 - val_loss: 0.3917 - val_acc: 0.9298Epoch 642/100068s 136ms/step - loss: 0.1576 - acc: 0.9956 - val_loss: 0.3945 - val_acc: 0.9284Epoch 643/100068s 136ms/step - loss: 0.1570 - acc: 0.9961 - val_loss: 0.3899 - val_acc: 0.9309Epoch 644/100068s 136ms/step - loss: 0.1565 - acc: 0.9962 - val_loss: 0.3918 - val_acc: 0.9307Epoch 645/100068s 136ms/step - loss: 0.1563 - acc: 0.9956 - val_loss: 0.3940 - val_acc: 0.9307Epoch 646/100068s 136ms/step - loss: 0.1563 - acc: 0.9956 - val_loss: 0.3895 - val_acc: 0.9322Epoch 647/100068s 136ms/step - loss: 0.1555 - acc: 0.9963 - val_loss: 0.3903 - val_acc: 0.9302Epoch 648/100068s 135ms/step - loss: 0.1556 - acc: 0.9958 - val_loss: 0.3926 - val_acc: 0.9307Epoch 649/100068s 135ms/step - loss: 0.1542 - acc: 0.9962 - val_loss: 0.3904 - val_acc: 0.9308Epoch 650/100068s 136ms/step - loss: 0.1552 - acc: 0.9959 - val_loss: 0.3934 - val_acc: 0.9295Epoch 651/100068s 136ms/step - loss: 0.1548 - acc: 0.9959 - val_loss: 0.3921 - val_acc: 0.9307Epoch 652/100068s 136ms/step - loss: 0.1537 - acc: 0.9964 - val_loss: 0.3973 - val_acc: 0.9293Epoch 653/100068s 136ms/step - loss: 0.1540 - acc: 0.9958 - val_loss: 0.3950 - val_acc: 0.9287Epoch 654/100068s 136ms/step - loss: 0.1523 - acc: 0.9965 - val_loss: 0.3956 - val_acc: 0.9296Epoch 655/100068s 137ms/step - loss: 0.1532 - acc: 0.9964 - val_loss: 0.3991 - val_acc: 0.9292Epoch 656/100068s 136ms/step - loss: 0.1538 - acc: 0.9957 - val_loss: 0.3995 - val_acc: 0.9296Epoch 657/100068s 136ms/step - loss: 0.1520 - acc: 0.9966 - val_loss: 0.3988 - val_acc: 0.9310Epoch 658/100068s 136ms/step - loss: 0.1532 - acc: 0.9959 - val_loss: 0.3961 - val_acc: 0.9307Epoch 659/100068s 136ms/step - loss: 0.1526 - acc: 0.9958 - val_loss: 0.3948 - val_acc: 0.9306Epoch 660/100068s 136ms/step - loss: 0.1512 - acc: 0.9965 - val_loss: 0.3947 - val_acc: 0.9309Epoch 661/100068s 136ms/step - loss: 0.1519 - acc: 0.9962 - val_loss: 0.3959 - val_acc: 0.9315Epoch 662/100068s 136ms/step - loss: 0.1510 - acc: 0.9963 - val_loss: 0.3962 - val_acc: 0.9312Epoch 663/100068s 136ms/step - loss: 0.1517 - acc: 0.9960 - val_loss: 0.3939 - val_acc: 0.9304Epoch 664/100068s 135ms/step - loss: 0.1494 - acc: 0.9964 - val_loss: 0.3928 - val_acc: 0.9309Epoch 665/100068s 135ms/step - loss: 0.1492 - acc: 0.9966 - val_loss: 0.3900 - val_acc: 0.9320Epoch 666/100068s 136ms/step - loss: 0.1493 - acc: 0.9963 - val_loss: 0.3907 - val_acc: 0.9312Epoch 667/100068s 136ms/step - loss: 0.1491 - acc: 0.9967 - val_loss: 0.3930 - val_acc: 0.9309Epoch 668/100068s 136ms/step - loss: 0.1494 - acc: 0.9960 - val_loss: 0.3923 - val_acc: 0.9301Epoch 669/100068s 136ms/step - loss: 0.1485 - acc: 0.9966 - val_loss: 0.3941 - val_acc: 0.9308Epoch 670/100068s 135ms/step - loss: 0.1486 - acc: 0.9963 - val_loss: 0.3927 - val_acc: 0.9314Epoch 671/100068s 135ms/step - loss: 0.1481 - acc: 0.9965 - val_loss: 0.3939 - val_acc: 0.9322Epoch 672/100068s 136ms/step - loss: 0.1474 - acc: 0.9968 - val_loss: 0.3950 - val_acc: 0.9309Epoch 673/100068s 136ms/step - loss: 0.1471 - acc: 0.9967 - val_loss: 0.3931 - val_acc: 0.9322Epoch 674/100068s 136ms/step - loss: 0.1470 - acc: 0.9968 - val_loss: 0.3934 - val_acc: 0.9319Epoch 675/100068s 136ms/step - loss: 0.1469 - acc: 0.9965 - val_loss: 0.3920 - val_acc: 0.9319Epoch 676/100068s 136ms/step - loss: 0.1469 - acc: 0.9967 - val_loss: 0.3923 - val_acc: 0.9309Epoch 677/100068s 136ms/step - loss: 0.1461 - acc: 0.9968 - val_loss: 0.3940 - val_acc: 0.9297Epoch 678/100068s 135ms/step - loss: 0.1462 - acc: 0.9969 - val_loss: 0.3924 - val_acc: 0.9309Epoch 679/100068s 136ms/step - loss: 0.1443 - acc: 0.9971 - val_loss: 0.3930 - val_acc: 0.9317Epoch 680/100068s 135ms/step - loss: 0.1458 - acc: 0.9966 - val_loss: 0.3978 - val_acc: 0.9296Epoch 681/100068s 135ms/step - loss: 0.1453 - acc: 0.9970 - val_loss: 0.3978 - val_acc: 0.9286Epoch 682/100068s 136ms/step - loss: 0.1444 - acc: 0.9972 - val_loss: 0.3968 - val_acc: 0.9285Epoch 683/100068s 136ms/step - loss: 0.1444 - acc: 0.9969 - val_loss: 0.3922 - val_acc: 0.9299Epoch 684/100068s 136ms/step - loss: 0.1447 - acc: 0.9970 - val_loss: 0.3907 - val_acc: 0.9297Epoch 685/100068s 136ms/step - loss: 0.1441 - acc: 0.9966 - val_loss: 0.3925 - val_acc: 0.9285Epoch 686/100068s 135ms/step - loss: 0.1454 - acc: 0.9964 - val_loss: 0.3939 - val_acc: 0.9296Epoch 687/100068s 135ms/step - loss: 0.1433 - acc: 0.9968 - val_loss: 0.3955 - val_acc: 0.9293Epoch 688/100068s 136ms/step - loss: 0.1435 - acc: 0.9969 - val_loss: 0.3958 - val_acc: 0.9295Epoch 689/100068s 136ms/step - loss: 0.1423 - acc: 0.9972 - val_loss: 0.3981 - val_acc: 0.9305Epoch 690/100068s 136ms/step - loss: 0.1438 - acc: 0.9965 - val_loss: 0.3986 - val_acc: 0.9299Epoch 691/100068s 136ms/step - loss: 0.1422 - acc: 0.9972 - val_loss: 0.3956 - val_acc: 0.9302Epoch 692/100068s 136ms/step - loss: 0.1425 - acc: 0.9968 - val_loss: 0.3962 - val_acc: 0.9309Epoch 693/100068s 136ms/step - loss: 0.1420 - acc: 0.9968 - val_loss: 0.3972 - val_acc: 0.9300Epoch 694/100068s 136ms/step - loss: 0.1422 - acc: 0.9967 - val_loss: 0.3947 - val_acc: 0.9301Epoch 695/100068s 136ms/step - loss: 0.1420 - acc: 0.9970 - val_loss: 0.3945 - val_acc: 0.9306Epoch 696/100068s 136ms/step - loss: 0.1412 - acc: 0.9970 - val_loss: 0.3942 - val_acc: 0.9313Epoch 697/100068s 136ms/step - loss: 0.1402 - acc: 0.9972 - val_loss: 0.3950 - val_acc: 0.9309Epoch 698/100068s 136ms/step - loss: 0.1408 - acc: 0.9969 - val_loss: 0.3931 - val_acc: 0.9307Epoch 699/100068s 136ms/step - loss: 0.1409 - acc: 0.9970 - val_loss: 0.3936 - val_acc: 0.9297Epoch 700/100068s 136ms/step - loss: 0.1404 - acc: 0.9970 - val_loss: 0.3930 - val_acc: 0.9289Epoch 701/100068s 136ms/step - loss: 0.1403 - acc: 0.9972 - val_loss: 0.3905 - val_acc: 0.9308Epoch 702/100068s 136ms/step - loss: 0.1387 - acc: 0.9976 - val_loss: 0.3957 - val_acc: 0.9295Epoch 703/100068s 135ms/step - loss: 0.1402 - acc: 0.9967 - val_loss: 0.3950 - val_acc: 0.9294Epoch 704/100068s 136ms/step - loss: 0.1393 - acc: 0.9971 - val_loss: 0.3950 - val_acc: 0.9298Epoch 705/100068s 136ms/step - loss: 0.1386 - acc: 0.9969 - val_loss: 0.3950 - val_acc: 0.9302Epoch 706/100068s 136ms/step - loss: 0.1384 - acc: 0.9973 - val_loss: 0.3936 - val_acc: 0.9303Epoch 707/100068s 135ms/step - loss: 0.1386 - acc: 0.9970 - val_loss: 0.3974 - val_acc: 0.9290Epoch 708/100068s 136ms/step - loss: 0.1392 - acc: 0.9968 - val_loss: 0.3938 - val_acc: 0.9295Epoch 709/100068s 136ms/step - loss: 0.1383 - acc: 0.9970 - val_loss: 0.3931 - val_acc: 0.9288Epoch 710/100068s 135ms/step - loss: 0.1383 - acc: 0.9970 - val_loss: 0.3905 - val_acc: 0.9305Epoch 711/100068s 136ms/step - loss: 0.1381 - acc: 0.9970 - val_loss: 0.3904 - val_acc: 0.9286Epoch 712/100068s 136ms/step - loss: 0.1375 - acc: 0.9971 - val_loss: 0.3923 - val_acc: 0.9302Epoch 713/100068s 136ms/step - loss: 0.1370 - acc: 0.9972 - val_loss: 0.3931 - val_acc: 0.9308Epoch 714/100068s 136ms/step - loss: 0.1364 - acc: 0.9974 - val_loss: 0.3883 - val_acc: 0.9322Epoch 715/100068s 136ms/step - loss: 0.1364 - acc: 0.9974 - val_loss: 0.3894 - val_acc: 0.9306Epoch 716/100068s 135ms/step - loss: 0.1365 - acc: 0.9972 - val_loss: 0.3894 - val_acc: 0.9290Epoch 717/100068s 135ms/step - loss: 0.1358 - acc: 0.9973 - val_loss: 0.3908 - val_acc: 0.9294Epoch 718/100068s 136ms/step - loss: 0.1360 - acc: 0.9971 - val_loss: 0.3899 - val_acc: 0.9297Epoch 719/100068s 135ms/step - loss: 0.1370 - acc: 0.9969 - val_loss: 0.3880 - val_acc: 0.9311Epoch 720/100068s 135ms/step - loss: 0.1348 - acc: 0.9971 - val_loss: 0.3884 - val_acc: 0.9308Epoch 721/100068s 135ms/step - loss: 0.1354 - acc: 0.9973 - val_loss: 0.3946 - val_acc: 0.9299Epoch 722/100068s 136ms/step - loss: 0.1346 - acc: 0.9973 - val_loss: 0.3890 - val_acc: 0.9313Epoch 723/100068s 136ms/step - loss: 0.1355 - acc: 0.9972 - val_loss: 0.3914 - val_acc: 0.9313Epoch 724/100068s 136ms/step - loss: 0.1353 - acc: 0.9970 - val_loss: 0.3956 - val_acc: 0.9308Epoch 725/100068s 136ms/step - loss: 0.1349 - acc: 0.9972 - val_loss: 0.3914 - val_acc: 0.9303Epoch 726/100068s 136ms/step - loss: 0.1338 - acc: 0.9975 - val_loss: 0.3917 - val_acc: 0.9297Epoch 727/100068s 136ms/step - loss: 0.1335 - acc: 0.9977 - val_loss: 0.3877 - val_acc: 0.9318Epoch 728/100068s 135ms/step - loss: 0.1329 - acc: 0.9977 - val_loss: 0.3830 - val_acc: 0.9324Epoch 729/100068s 136ms/step - loss: 0.1332 - acc: 0.9973 - val_loss: 0.3870 - val_acc: 0.9314Epoch 730/100068s 136ms/step - loss: 0.1330 - acc: 0.9976 - val_loss: 0.3870 - val_acc: 0.9321Epoch 731/100068s 136ms/step - loss: 0.1324 - acc: 0.9978 - val_loss: 0.3841 - val_acc: 0.9308Epoch 732/100068s 136ms/step - loss: 0.1329 - acc: 0.9971 - val_loss: 0.3853 - val_acc: 0.9316Epoch 733/100068s 137ms/step - loss: 0.1323 - acc: 0.9975 - val_loss: 0.3868 - val_acc: 0.9310Epoch 734/100068s 136ms/step - loss: 0.1322 - acc: 0.9975 - val_loss: 0.3882 - val_acc: 0.9301Epoch 735/100068s 135ms/step - loss: 0.1314 - acc: 0.9975 - val_loss: 0.3880 - val_acc: 0.9289Epoch 736/100068s 136ms/step - loss: 0.1327 - acc: 0.9971 - val_loss: 0.3891 - val_acc: 0.9295Epoch 737/100068s 135ms/step - loss: 0.1308 - acc: 0.9978 - val_loss: 0.3862 - val_acc: 0.9303Epoch 738/100068s 136ms/step - loss: 0.1314 - acc: 0.9975 - val_loss: 0.3872 - val_acc: 0.9294Epoch 739/100068s 136ms/step - loss: 0.1305 - acc: 0.9979 - val_loss: 0.3864 - val_acc: 0.9309Epoch 740/100068s 136ms/step - loss: 0.1310 - acc: 0.9973 - val_loss: 0.3896 - val_acc: 0.9307Epoch 741/100068s 136ms/step - loss: 0.1311 - acc: 0.9974 - val_loss: 0.3883 - val_acc: 0.9312Epoch 742/100068s 136ms/step - loss: 0.1315 - acc: 0.9968 - val_loss: 0.3888 - val_acc: 0.9304Epoch 743/100068s 136ms/step - loss: 0.1297 - acc: 0.9977 - val_loss: 0.3892 - val_acc: 0.9307Epoch 744/100068s 136ms/step - loss: 0.1298 - acc: 0.9976 - val_loss: 0.3864 - val_acc: 0.9297Epoch 745/100068s 136ms/step - loss: 0.1299 - acc: 0.9974 - val_loss: 0.3883 - val_acc: 0.9306Epoch 746/100068s 135ms/step - loss: 0.1301 - acc: 0.9972 - val_loss: 0.3892 - val_acc: 0.9290Epoch 747/100068s 136ms/step - loss: 0.1290 - acc: 0.9977 - val_loss: 0.3860 - val_acc: 0.9301Epoch 748/100068s 136ms/step - loss: 0.1299 - acc: 0.9972 - val_loss: 0.3846 - val_acc: 0.9308Epoch 749/100068s 136ms/step - loss: 0.1292 - acc: 0.9973 - val_loss: 0.3888 - val_acc: 0.9293Epoch 750/100068s 135ms/step - loss: 0.1297 - acc: 0.9973 - val_loss: 0.3864 - val_acc: 0.9289Epoch 751/100068s 135ms/step - loss: 0.1296 - acc: 0.9969 - val_loss: 0.3886 - val_acc: 0.9305Epoch 752/100068s 136ms/step - loss: 0.1288 - acc: 0.9972 - val_loss: 0.3893 - val_acc: 0.9285Epoch 753/100068s 136ms/step - loss: 0.1281 - acc: 0.9977 - val_loss: 0.3824 - val_acc: 0.9308Epoch 754/100068s 136ms/step - loss: 0.1288 - acc: 0.9973 - val_loss: 0.3817 - val_acc: 0.9300Epoch 755/100068s 136ms/step - loss: 0.1274 - acc: 0.9978 - val_loss: 0.3818 - val_acc: 0.9290Epoch 756/100068s 136ms/step - loss: 0.1282 - acc: 0.9971 - val_loss: 0.3843 - val_acc: 0.9277Epoch 757/100068s 136ms/step - loss: 0.1276 - acc: 0.9975 - val_loss: 0.3822 - val_acc: 0.9285Epoch 758/100068s 135ms/step - loss: 0.1274 - acc: 0.9974 - val_loss: 0.3837 - val_acc: 0.9301Epoch 759/100068s 136ms/step - loss: 0.1274 - acc: 0.9971 - val_loss: 0.3819 - val_acc: 0.9290Epoch 760/100068s 136ms/step - loss: 0.1261 - acc: 0.9977 - val_loss: 0.3803 - val_acc: 0.9308Epoch 761/100068s 136ms/step - loss: 0.1274 - acc: 0.9970 - val_loss: 0.3834 - val_acc: 0.9297Epoch 762/100068s 136ms/step - loss: 0.1264 - acc: 0.9977 - val_loss: 0.3845 - val_acc: 0.9300Epoch 763/100068s 136ms/step - loss: 0.1271 - acc: 0.9969 - val_loss: 0.3827 - val_acc: 0.9296Epoch 764/100068s 136ms/step - loss: 0.1264 - acc: 0.9974 - val_loss: 0.3772 - val_acc: 0.9316Epoch 765/100068s 136ms/step - loss: 0.1255 - acc: 0.9976 - val_loss: 0.3735 - val_acc: 0.9323Epoch 766/100068s 135ms/step - loss: 0.1253 - acc: 0.9977 - val_loss: 0.3743 - val_acc: 0.9325Epoch 767/100068s 136ms/step - loss: 0.1252 - acc: 0.9977 - val_loss: 0.3774 - val_acc: 0.9319Epoch 768/100068s 136ms/step - loss: 0.1251 - acc: 0.9975 - val_loss: 0.3778 - val_acc: 0.9324Epoch 769/100068s 136ms/step - loss: 0.1261 - acc: 0.9971 - val_loss: 0.3811 - val_acc: 0.9310Epoch 770/100068s 136ms/step - loss: 0.1242 - acc: 0.9979 - val_loss: 0.3808 - val_acc: 0.9295Epoch 771/100068s 136ms/step - loss: 0.1249 - acc: 0.9975 - val_loss: 0.3780 - val_acc: 0.9304Epoch 772/100068s 136ms/step - loss: 0.1247 - acc: 0.9974 - val_loss: 0.3779 - val_acc: 0.9312Epoch 773/100068s 136ms/step - loss: 0.1246 - acc: 0.9974 - val_loss: 0.3811 - val_acc: 0.9314Epoch 774/100068s 135ms/step - loss: 0.1244 - acc: 0.9976 - val_loss: 0.3798 - val_acc: 0.9303Epoch 775/100068s 136ms/step - loss: 0.1243 - acc: 0.9974 - val_loss: 0.3804 - val_acc: 0.9307Epoch 776/100068s 135ms/step - loss: 0.1235 - acc: 0.9975 - val_loss: 0.3800 - val_acc: 0.9310Epoch 777/100068s 136ms/step - loss: 0.1240 - acc: 0.9973 - val_loss: 0.3795 - val_acc: 0.9304Epoch 778/100068s 136ms/step - loss: 0.1234 - acc: 0.9975 - val_loss: 0.3760 - val_acc: 0.9320Epoch 779/100068s 136ms/step - loss: 0.1235 - acc: 0.9976 - val_loss: 0.3750 - val_acc: 0.9312Epoch 780/100068s 136ms/step - loss: 0.1226 - acc: 0.9976 - val_loss: 0.3721 - val_acc: 0.9332Epoch 781/100068s 136ms/step - loss: 0.1227 - acc: 0.9976 - val_loss: 0.3753 - val_acc: 0.9322Epoch 782/100068s 135ms/step - loss: 0.1226 - acc: 0.9975 - val_loss: 0.3756 - val_acc: 0.9316Epoch 783/100068s 135ms/step - loss: 0.1228 - acc: 0.9975 - val_loss: 0.3761 - val_acc: 0.9302Epoch 784/100068s 136ms/step - loss: 0.1216 - acc: 0.9978 - val_loss: 0.3711 - val_acc: 0.9329Epoch 785/100068s 136ms/step - loss: 0.1221 - acc: 0.9975 - val_loss: 0.3750 - val_acc: 0.9300Epoch 786/100068s 136ms/step - loss: 0.1213 - acc: 0.9978 - val_loss: 0.3739 - val_acc: 0.9305Epoch 787/100068s 136ms/step - loss: 0.1211 - acc: 0.9978 - val_loss: 0.3744 - val_acc: 0.9315Epoch 788/100068s 136ms/step - loss: 0.1209 - acc: 0.9978 - val_loss: 0.3730 - val_acc: 0.9321Epoch 789/100068s 136ms/step - loss: 0.1219 - acc: 0.9975 - val_loss: 0.3719 - val_acc: 0.9329Epoch 790/100068s 135ms/step - loss: 0.1212 - acc: 0.9975 - val_loss: 0.3753 - val_acc: 0.9318Epoch 791/100068s 136ms/step - loss: 0.1208 - acc: 0.9974 - val_loss: 0.3744 - val_acc: 0.9310Epoch 792/100068s 135ms/step - loss: 0.1222 - acc: 0.9969 - val_loss: 0.3804 - val_acc: 0.9291Epoch 793/100068s 136ms/step - loss: 0.1209 - acc: 0.9977 - val_loss: 0.3806 - val_acc: 0.9295Epoch 794/100068s 136ms/step - loss: 0.1203 - acc: 0.9977 - val_loss: 0.3809 - val_acc: 0.9282Epoch 795/100068s 136ms/step - loss: 0.1203 - acc: 0.9975 - val_loss: 0.3785 - val_acc: 0.9286Epoch 796/100068s 136ms/step - loss: 0.1199 - acc: 0.9978 - val_loss: 0.3783 - val_acc: 0.9274Epoch 797/100068s 136ms/step - loss: 0.1196 - acc: 0.9977 - val_loss: 0.3780 - val_acc: 0.9281Epoch 798/100068s 135ms/step - loss: 0.1195 - acc: 0.9976 - val_loss: 0.3763 - val_acc: 0.9311Epoch 799/100067s 135ms/step - loss: 0.1193 - acc: 0.9977 - val_loss: 0.3820 - val_acc: 0.9303Epoch 800/100068s 136ms/step - loss: 0.1200 - acc: 0.9972 - val_loss: 0.3833 - val_acc: 0.9296Epoch 801/100068s 135ms/step - loss: 0.1185 - acc: 0.9978 - val_loss: 0.3774 - val_acc: 0.9300Epoch 802/100068s 136ms/step - loss: 0.1195 - acc: 0.9974 - val_loss: 0.3775 - val_acc: 0.9305Epoch 803/100068s 136ms/step - loss: 0.1183 - acc: 0.9976 - val_loss: 0.3759 - val_acc: 0.9308Epoch 804/100068s 136ms/step - loss: 0.1182 - acc: 0.9979 - val_loss: 0.3728 - val_acc: 0.9316Epoch 805/100068s 136ms/step - loss: 0.1191 - acc: 0.9974 - val_loss: 0.3771 - val_acc: 0.9311Epoch 806/100068s 136ms/step - loss: 0.1179 - acc: 0.9977 - val_loss: 0.3768 - val_acc: 0.9299Epoch 807/100068s 136ms/step - loss: 0.1185 - acc: 0.9972 - val_loss: 0.3765 - val_acc: 0.9302Epoch 808/100068s 136ms/step - loss: 0.1173 - acc: 0.9978 - val_loss: 0.3794 - val_acc: 0.9291Epoch 809/100068s 136ms/step - loss: 0.1172 - acc: 0.9978 - val_loss: 0.3773 - val_acc: 0.9297Epoch 810/100068s 136ms/step - loss: 0.1181 - acc: 0.9975 - val_loss: 0.3811 - val_acc: 0.9306Epoch 811/100068s 136ms/step - loss: 0.1173 - acc: 0.9975 - val_loss: 0.3753 - val_acc: 0.9302Epoch 812/100068s 136ms/step - loss: 0.1171 - acc: 0.9975 - val_loss: 0.3812 - val_acc: 0.9285Epoch 813/100068s 136ms/step - loss: 0.1171 - acc: 0.9976 - val_loss: 0.3845 - val_acc: 0.9297Epoch 814/100068s 136ms/step - loss: 0.1163 - acc: 0.9978 - val_loss: 0.3829 - val_acc: 0.9295Epoch 815/100068s 136ms/step - loss: 0.1166 - acc: 0.9979 - val_loss: 0.3807 - val_acc: 0.9284Epoch 816/100068s 136ms/step - loss: 0.1165 - acc: 0.9976 - val_loss: 0.3813 - val_acc: 0.9286Epoch 817/100068s 135ms/step - loss: 0.1170 - acc: 0.9972 - val_loss: 0.3840 - val_acc: 0.9283Epoch 818/100068s 136ms/step - loss: 0.1160 - acc: 0.9973 - val_loss: 0.3826 - val_acc: 0.9274Epoch 819/100068s 136ms/step - loss: 0.1157 - acc: 0.9977 - val_loss: 0.3755 - val_acc: 0.9312Epoch 820/100068s 136ms/step - loss: 0.1155 - acc: 0.9978 - val_loss: 0.3794 - val_acc: 0.9291Epoch 821/100068s 136ms/step - loss: 0.1163 - acc: 0.9973 - val_loss: 0.3751 - val_acc: 0.9293Epoch 822/100068s 137ms/step - loss: 0.1154 - acc: 0.9977 - val_loss: 0.3764 - val_acc: 0.9298Epoch 823/100069s 137ms/step - loss: 0.1143 - acc: 0.9980 - val_loss: 0.3754 - val_acc: 0.9293Epoch 824/100069s 138ms/step - loss: 0.1142 - acc: 0.9979 - val_loss: 0.3743 - val_acc: 0.9304Epoch 825/100069s 138ms/step - loss: 0.1153 - acc: 0.9974 - val_loss: 0.3772 - val_acc: 0.9307Epoch 826/100069s 138ms/step - loss: 0.1149 - acc: 0.9976 - val_loss: 0.3718 - val_acc: 0.9312Epoch 827/100068s 137ms/step - loss: 0.1148 - acc: 0.9976 - val_loss: 0.3777 - val_acc: 0.9317Epoch 828/100069s 138ms/step - loss: 0.1147 - acc: 0.9976 - val_loss: 0.3769 - val_acc: 0.9303Epoch 829/100069s 138ms/step - loss: 0.1137 - acc: 0.9978 - val_loss: 0.3748 - val_acc: 0.9309Epoch 830/100069s 138ms/step - loss: 0.1139 - acc: 0.9978 - val_loss: 0.3728 - val_acc: 0.9308Epoch 831/100068s 137ms/step - loss: 0.1138 - acc: 0.9976 - val_loss: 0.3724 - val_acc: 0.9296Epoch 832/100069s 138ms/step - loss: 0.1132 - acc: 0.9978 - val_loss: 0.3807 - val_acc: 0.9288Epoch 833/100068s 137ms/step - loss: 0.1140 - acc: 0.9975 - val_loss: 0.3810 - val_acc: 0.9290Epoch 834/100069s 138ms/step - loss: 0.1135 - acc: 0.9975 - val_loss: 0.3816 - val_acc: 0.9291Epoch 835/100069s 138ms/step - loss: 0.1130 - acc: 0.9979 - val_loss: 0.3830 - val_acc: 0.9284Epoch 836/100069s 138ms/step - loss: 0.1131 - acc: 0.9976 - val_loss: 0.3792 - val_acc: 0.9278Epoch 837/100069s 137ms/step - loss: 0.1126 - acc: 0.9978 - val_loss: 0.3712 - val_acc: 0.9306Epoch 838/100069s 137ms/step - loss: 0.1126 - acc: 0.9979 - val_loss: 0.3771 - val_acc: 0.9293Epoch 839/100069s 138ms/step - loss: 0.1119 - acc: 0.9981 - val_loss: 0.3768 - val_acc: 0.9288Epoch 840/100069s 138ms/step - loss: 0.1120 - acc: 0.9980 - val_loss: 0.3769 - val_acc: 0.9289Epoch 841/100069s 137ms/step - loss: 0.1120 - acc: 0.9977 - val_loss: 0.3774 - val_acc: 0.9285Epoch 842/100068s 136ms/step - loss: 0.1120 - acc: 0.9975 - val_loss: 0.3718 - val_acc: 0.9312Epoch 843/100068s 136ms/step - loss: 0.1115 - acc: 0.9976 - val_loss: 0.3707 - val_acc: 0.9312Epoch 844/100068s 136ms/step - loss: 0.1120 - acc: 0.9978 - val_loss: 0.3777 - val_acc: 0.9285Epoch 845/100069s 137ms/step - loss: 0.1115 - acc: 0.9978 - val_loss: 0.3777 - val_acc: 0.9284Epoch 846/100069s 138ms/step - loss: 0.1115 - acc: 0.9978 - val_loss: 0.3742 - val_acc: 0.9303Epoch 847/100068s 137ms/step - loss: 0.1113 - acc: 0.9974 - val_loss: 0.3749 - val_acc: 0.9300Epoch 848/100069s 138ms/step - loss: 0.1114 - acc: 0.9976 - val_loss: 0.3795 - val_acc: 0.9286Epoch 849/100069s 138ms/step - loss: 0.1115 - acc: 0.9975 - val_loss: 0.3754 - val_acc: 0.9284Epoch 850/100069s 138ms/step - loss: 0.1105 - acc: 0.9978 - val_loss: 0.3705 - val_acc: 0.9305Epoch 851/100069s 138ms/step - loss: 0.1098 - acc: 0.9978 - val_loss: 0.3752 - val_acc: 0.9290Epoch 852/100069s 138ms/step - loss: 0.1118 - acc: 0.9971 - val_loss: 0.3773 - val_acc: 0.9280Epoch 853/100068s 137ms/step - loss: 0.1103 - acc: 0.9978 - val_loss: 0.3732 - val_acc: 0.9303Epoch 854/100069s 137ms/step - loss: 0.1109 - acc: 0.9977 - val_loss: 0.3715 - val_acc: 0.9302Epoch 855/100069s 137ms/step - loss: 0.1096 - acc: 0.9977 - val_loss: 0.3780 - val_acc: 0.9306Epoch 856/100069s 137ms/step - loss: 0.1100 - acc: 0.9977 - val_loss: 0.3764 - val_acc: 0.9290Epoch 857/100069s 137ms/step - loss: 0.1093 - acc: 0.9981 - val_loss: 0.3750 - val_acc: 0.9291Epoch 858/100069s 138ms/step - loss: 0.1088 - acc: 0.9980 - val_loss: 0.3738 - val_acc: 0.9287Epoch 859/100069s 138ms/step - loss: 0.1098 - acc: 0.9975 - val_loss: 0.3711 - val_acc: 0.9291Epoch 860/100069s 138ms/step - loss: 0.1091 - acc: 0.9979 - val_loss: 0.3636 - val_acc: 0.9302Epoch 861/100069s 138ms/step - loss: 0.1094 - acc: 0.9976 - val_loss: 0.3689 - val_acc: 0.9303Epoch 862/100068s 137ms/step - loss: 0.1088 - acc: 0.9978 - val_loss: 0.3687 - val_acc: 0.9306Epoch 863/100069s 137ms/step - loss: 0.1083 - acc: 0.9978 - val_loss: 0.3720 - val_acc: 0.9318Epoch 864/100069s 138ms/step - loss: 0.1080 - acc: 0.9978 - val_loss: 0.3695 - val_acc: 0.9302Epoch 865/100069s 138ms/step - loss: 0.1093 - acc: 0.9973 - val_loss: 0.3733 - val_acc: 0.9297Epoch 866/100069s 138ms/step - loss: 0.1092 - acc: 0.9974 - val_loss: 0.3713 - val_acc: 0.9296Epoch 867/100069s 137ms/step - loss: 0.1082 - acc: 0.9978 - val_loss: 0.3674 - val_acc: 0.9306Epoch 868/100069s 138ms/step - loss: 0.1087 - acc: 0.9974 - val_loss: 0.3684 - val_acc: 0.9296Epoch 869/100069s 138ms/step - loss: 0.1072 - acc: 0.9982 - val_loss: 0.3684 - val_acc: 0.9307Epoch 870/100068s 137ms/step - loss: 0.1080 - acc: 0.9976 - val_loss: 0.3695 - val_acc: 0.9294Epoch 871/100069s 138ms/step - loss: 0.1075 - acc: 0.9977 - val_loss: 0.3655 - val_acc: 0.9306Epoch 872/100069s 138ms/step - loss: 0.1073 - acc: 0.9979 - val_loss: 0.3667 - val_acc: 0.9303Epoch 873/100069s 138ms/step - loss: 0.1079 - acc: 0.9977 - val_loss: 0.3717 - val_acc: 0.9278Epoch 874/100068s 137ms/step - loss: 0.1081 - acc: 0.9973 - val_loss: 0.3722 - val_acc: 0.9292Epoch 875/100069s 138ms/step - loss: 0.1072 - acc: 0.9975 - val_loss: 0.3716 - val_acc: 0.9298Epoch 876/100069s 137ms/step - loss: 0.1070 - acc: 0.9977 - val_loss: 0.3721 - val_acc: 0.9311Epoch 877/100069s 137ms/step - loss: 0.1066 - acc: 0.9978 - val_loss: 0.3722 - val_acc: 0.9289Epoch 878/100069s 137ms/step - loss: 0.1068 - acc: 0.9977 - val_loss: 0.3736 - val_acc: 0.9296Epoch 879/100068s 137ms/step - loss: 0.1065 - acc: 0.9977 - val_loss: 0.3767 - val_acc: 0.9280Epoch 880/100069s 138ms/step - loss: 0.1055 - acc: 0.9979 - val_loss: 0.3741 - val_acc: 0.9285Epoch 881/100068s 137ms/step - loss: 0.1056 - acc: 0.9979 - val_loss: 0.3716 - val_acc: 0.9290Epoch 882/100069s 138ms/step - loss: 0.1061 - acc: 0.9977 - val_loss: 0.3736 - val_acc: 0.9295Epoch 883/100069s 138ms/step - loss: 0.1066 - acc: 0.9976 - val_loss: 0.3745 - val_acc: 0.9307Epoch 884/100069s 137ms/step - loss: 0.1059 - acc: 0.9975 - val_loss: 0.3702 - val_acc: 0.9302Epoch 885/100069s 138ms/step - loss: 0.1051 - acc: 0.9979 - val_loss: 0.3656 - val_acc: 0.9311Epoch 886/100068s 137ms/step - loss: 0.1051 - acc: 0.9978 - val_loss: 0.3677 - val_acc: 0.9305Epoch 887/100068s 137ms/step - loss: 0.1062 - acc: 0.9974 - val_loss: 0.3636 - val_acc: 0.9315Epoch 888/100069s 137ms/step - loss: 0.1052 - acc: 0.9977 - val_loss: 0.3710 - val_acc: 0.9295Epoch 889/100068s 137ms/step - loss: 0.1046 - acc: 0.9979 - val_loss: 0.3642 - val_acc: 0.9318Epoch 890/100069s 138ms/step - loss: 0.1051 - acc: 0.9975 - val_loss: 0.3673 - val_acc: 0.9306Epoch 891/100069s 138ms/step - loss: 0.1045 - acc: 0.9978 - val_loss: 0.3681 - val_acc: 0.9299Epoch 892/100068s 137ms/step - loss: 0.1043 - acc: 0.9979 - val_loss: 0.3659 - val_acc: 0.9320Epoch 893/100069s 137ms/step - loss: 0.1040 - acc: 0.9979 - val_loss: 0.3627 - val_acc: 0.9326Epoch 894/100069s 138ms/step - loss: 0.1041 - acc: 0.9976 - val_loss: 0.3698 - val_acc: 0.9301Epoch 895/100068s 137ms/step - loss: 0.1039 - acc: 0.9978 - val_loss: 0.3659 - val_acc: 0.9321Epoch 896/100069s 137ms/step - loss: 0.1040 - acc: 0.9978 - val_loss: 0.3718 - val_acc: 0.9300Epoch 897/100068s 137ms/step - loss: 0.1039 - acc: 0.9977 - val_loss: 0.3728 - val_acc: 0.9311Epoch 898/100068s 137ms/step - loss: 0.1044 - acc: 0.9973 - val_loss: 0.3743 - val_acc: 0.9313Epoch 899/100069s 137ms/step - loss: 0.1036 - acc: 0.9976 - val_loss: 0.3675 - val_acc: 0.9312Epoch 900/100069s 138ms/step - loss: 0.1030 - acc: 0.9979 - val_loss: 0.3730 - val_acc: 0.9313Epoch 901/1000lr changed to 9.999999310821295e-0569s 138ms/step - loss: 0.1023 - acc: 0.9982 - val_loss: 0.3709 - val_acc: 0.9310Epoch 902/100069s 137ms/step - loss: 0.1025 - acc: 0.9979 - val_loss: 0.3690 - val_acc: 0.9311Epoch 903/100068s 137ms/step - loss: 0.1024 - acc: 0.9980 - val_loss: 0.3679 - val_acc: 0.9311Epoch 904/100069s 137ms/step - loss: 0.1020 - acc: 0.9982 - val_loss: 0.3673 - val_acc: 0.9315Epoch 905/100069s 138ms/step - loss: 0.1027 - acc: 0.9979 - val_loss: 0.3672 - val_acc: 0.9310Epoch 906/100069s 138ms/step - loss: 0.1015 - acc: 0.9984 - val_loss: 0.3678 - val_acc: 0.9304Epoch 907/100069s 138ms/step - loss: 0.1016 - acc: 0.9984 - val_loss: 0.3673 - val_acc: 0.9302Epoch 908/100069s 138ms/step - loss: 0.1031 - acc: 0.9977 - val_loss: 0.3667 - val_acc: 0.9307Epoch 909/100069s 139ms/step - loss: 0.1019 - acc: 0.9983 - val_loss: 0.3672 - val_acc: 0.9317Epoch 910/100069s 137ms/step - loss: 0.1018 - acc: 0.9983 - val_loss: 0.3671 - val_acc: 0.9313Epoch 911/100069s 137ms/step - loss: 0.1018 - acc: 0.9982 - val_loss: 0.3669 - val_acc: 0.9309Epoch 912/100069s 137ms/step - loss: 0.1014 - acc: 0.9986 - val_loss: 0.3677 - val_acc: 0.9303Epoch 913/100068s 137ms/step - loss: 0.1015 - acc: 0.9982 - val_loss: 0.3666 - val_acc: 0.9303Epoch 914/100069s 138ms/step - loss: 0.1015 - acc: 0.9984 - val_loss: 0.3659 - val_acc: 0.9309Epoch 915/100069s 138ms/step - loss: 0.1013 - acc: 0.9983 - val_loss: 0.3651 - val_acc: 0.9318Epoch 916/100069s 138ms/step - loss: 0.1014 - acc: 0.9983 - val_loss: 0.3652 - val_acc: 0.9322Epoch 917/100069s 137ms/step - loss: 0.1010 - acc: 0.9984 - val_loss: 0.3648 - val_acc: 0.9322Epoch 918/100068s 137ms/step - loss: 0.1016 - acc: 0.9981 - val_loss: 0.3644 - val_acc: 0.9324Epoch 919/100069s 138ms/step - loss: 0.1013 - acc: 0.9983 - val_loss: 0.3635 - val_acc: 0.9319Epoch 920/100069s 138ms/step - loss: 0.1008 - acc: 0.9984 - val_loss: 0.3629 - val_acc: 0.9318Epoch 921/100069s 138ms/step - loss: 0.1006 - acc: 0.9986 - val_loss: 0.3627 - val_acc: 0.9319Epoch 922/100069s 137ms/step - loss: 0.1007 - acc: 0.9985 - val_loss: 0.3632 - val_acc: 0.9314Epoch 923/100069s 137ms/step - loss: 0.1004 - acc: 0.9987 - val_loss: 0.3626 - val_acc: 0.9319Epoch 924/100069s 138ms/step - loss: 0.1012 - acc: 0.9985 - val_loss: 0.3629 - val_acc: 0.9319Epoch 925/100069s 138ms/step - loss: 0.1011 - acc: 0.9983 - val_loss: 0.3620 - val_acc: 0.9318Epoch 926/100069s 138ms/step - loss: 0.1005 - acc: 0.9987 - val_loss: 0.3617 - val_acc: 0.9322Epoch 927/100069s 138ms/step - loss: 0.1013 - acc: 0.9983 - val_loss: 0.3618 - val_acc: 0.9330Epoch 928/100069s 138ms/step - loss: 0.1005 - acc: 0.9985 - val_loss: 0.3614 - val_acc: 0.9321Epoch 929/100069s 137ms/step - loss: 0.1006 - acc: 0.9985 - val_loss: 0.3616 - val_acc: 0.9319Epoch 930/100069s 138ms/step - loss: 0.1006 - acc: 0.9985 - val_loss: 0.3613 - val_acc: 0.9321Epoch 931/100069s 138ms/step - loss: 0.1009 - acc: 0.9985 - val_loss: 0.3612 - val_acc: 0.9328Epoch 932/100069s 138ms/step - loss: 0.1004 - acc: 0.9985 - val_loss: 0.3612 - val_acc: 0.9319Epoch 933/100069s 138ms/step - loss: 0.1004 - acc: 0.9987 - val_loss: 0.3618 - val_acc: 0.9314Epoch 934/100069s 137ms/step - loss: 0.1008 - acc: 0.9983 - val_loss: 0.3615 - val_acc: 0.9316Epoch 935/100069s 138ms/step - loss: 0.1011 - acc: 0.9983 - val_loss: 0.3621 - val_acc: 0.9317Epoch 936/100068s 137ms/step - loss: 0.1008 - acc: 0.9985 - val_loss: 0.3617 - val_acc: 0.9320Epoch 937/100069s 138ms/step - loss: 0.1006 - acc: 0.9984 - val_loss: 0.3613 - val_acc: 0.9322Epoch 938/100069s 137ms/step - loss: 0.1008 - acc: 0.9985 - val_loss: 0.3613 - val_acc: 0.9325Epoch 939/100068s 137ms/step - loss: 0.1006 - acc: 0.9984 - val_loss: 0.3614 - val_acc: 0.9326Epoch 940/100069s 137ms/step - loss: 0.1006 - acc: 0.9983 - val_loss: 0.3612 - val_acc: 0.9320Epoch 941/100069s 137ms/step - loss: 0.1005 - acc: 0.9984 - val_loss: 0.3612 - val_acc: 0.9322Epoch 942/100069s 138ms/step - loss: 0.1001 - acc: 0.9986 - val_loss: 0.3615 - val_acc: 0.9318Epoch 943/100068s 137ms/step - loss: 0.0998 - acc: 0.9987 - val_loss: 0.3613 - val_acc: 0.9320Epoch 944/100069s 137ms/step - loss: 0.1006 - acc: 0.9985 - val_loss: 0.3613 - val_acc: 0.9323Epoch 945/100069s 138ms/step - loss: 0.1000 - acc: 0.9985 - val_loss: 0.3608 - val_acc: 0.9319Epoch 946/100069s 138ms/step - loss: 0.1001 - acc: 0.9987 - val_loss: 0.3608 - val_acc: 0.9313Epoch 947/100069s 138ms/step - loss: 0.0998 - acc: 0.9987 - val_loss: 0.3606 - val_acc: 0.9314Epoch 948/100069s 138ms/step - loss: 0.1000 - acc: 0.9986 - val_loss: 0.3609 - val_acc: 0.9311Epoch 949/100069s 138ms/step - loss: 0.0995 - acc: 0.9988 - val_loss: 0.3610 - val_acc: 0.9316Epoch 950/100069s 137ms/step - loss: 0.0999 - acc: 0.9986 - val_loss: 0.3609 - val_acc: 0.9317Epoch 951/100069s 137ms/step - loss: 0.1002 - acc: 0.9986 - val_loss: 0.3612 - val_acc: 0.9314Epoch 952/100068s 137ms/step - loss: 0.0992 - acc: 0.9989 - val_loss: 0.3618 - val_acc: 0.9312Epoch 953/100068s 137ms/step - loss: 0.0996 - acc: 0.9988 - val_loss: 0.3617 - val_acc: 0.9317Epoch 954/100069s 138ms/step - loss: 0.0994 - acc: 0.9987 - val_loss: 0.3617 - val_acc: 0.9323Epoch 955/100069s 138ms/step - loss: 0.1004 - acc: 0.9984 - val_loss: 0.3610 - val_acc: 0.9320Epoch 956/100069s 138ms/step - loss: 0.1000 - acc: 0.9986 - val_loss: 0.3616 - val_acc: 0.9318Epoch 957/100068s 137ms/step - loss: 0.1000 - acc: 0.9985 - val_loss: 0.3617 - val_acc: 0.9319Epoch 958/100069s 138ms/step - loss: 0.0996 - acc: 0.9985 - val_loss: 0.3627 - val_acc: 0.9323Epoch 959/100069s 137ms/step - loss: 0.0995 - acc: 0.9987 - val_loss: 0.3625 - val_acc: 0.9316Epoch 960/100069s 137ms/step - loss: 0.0995 - acc: 0.9987 - val_loss: 0.3634 - val_acc: 0.9317Epoch 961/100069s 138ms/step - loss: 0.0998 - acc: 0.9985 - val_loss: 0.3636 - val_acc: 0.9318Epoch 962/100069s 138ms/step - loss: 0.0997 - acc: 0.9986 - val_loss: 0.3645 - val_acc: 0.9319Epoch 963/100068s 137ms/step - loss: 0.1001 - acc: 0.9984 - val_loss: 0.3637 - val_acc: 0.9316Epoch 964/100069s 138ms/step - loss: 0.0998 - acc: 0.9985 - val_loss: 0.3631 - val_acc: 0.9317Epoch 965/100069s 137ms/step - loss: 0.0995 - acc: 0.9988 - val_loss: 0.3625 - val_acc: 0.9316Epoch 966/100068s 137ms/step - loss: 0.0998 - acc: 0.9986 - val_loss: 0.3622 - val_acc: 0.9324Epoch 967/100068s 137ms/step - loss: 0.1002 - acc: 0.9985 - val_loss: 0.3623 - val_acc: 0.9327Epoch 968/100068s 137ms/step - loss: 0.0993 - acc: 0.9987 - val_loss: 0.3627 - val_acc: 0.9324Epoch 969/100069s 137ms/step - loss: 0.0996 - acc: 0.9985 - val_loss: 0.3624 - val_acc: 0.9327Epoch 970/100069s 138ms/step - loss: 0.0999 - acc: 0.9985 - val_loss: 0.3618 - val_acc: 0.9323Epoch 971/100069s 137ms/step - loss: 0.1001 - acc: 0.9983 - val_loss: 0.3616 - val_acc: 0.9324Epoch 972/100068s 136ms/step - loss: 0.0994 - acc: 0.9986 - val_loss: 0.3620 - val_acc: 0.9320Epoch 973/100068s 136ms/step - loss: 0.0997 - acc: 0.9985 - val_loss: 0.3627 - val_acc: 0.9324Epoch 974/100068s 136ms/step - loss: 0.1000 - acc: 0.9985 - val_loss: 0.3623 - val_acc: 0.9321Epoch 975/100068s 135ms/step - loss: 0.0989 - acc: 0.9990 - val_loss: 0.3619 - val_acc: 0.9321Epoch 976/100068s 136ms/step - loss: 0.0992 - acc: 0.9987 - val_loss: 0.3612 - val_acc: 0.9323Epoch 977/100068s 136ms/step - loss: 0.0996 - acc: 0.9986 - val_loss: 0.3612 - val_acc: 0.9317Epoch 978/100068s 136ms/step - loss: 0.0997 - acc: 0.9986 - val_loss: 0.3610 - val_acc: 0.9326Epoch 979/100068s 136ms/step - loss: 0.0991 - acc: 0.9987 - val_loss: 0.3611 - val_acc: 0.9327Epoch 980/100069s 137ms/step - loss: 0.0988 - acc: 0.9989 - val_loss: 0.3615 - val_acc: 0.9326Epoch 981/100069s 137ms/step - loss: 0.0992 - acc: 0.9987 - val_loss: 0.3619 - val_acc: 0.9324Epoch 982/100068s 137ms/step - loss: 0.0994 - acc: 0.9986 - val_loss: 0.3619 - val_acc: 0.9332Epoch 983/100069s 137ms/step - loss: 0.0995 - acc: 0.9986 - val_loss: 0.3617 - val_acc: 0.9329Epoch 984/100068s 137ms/step - loss: 0.0991 - acc: 0.9987 - val_loss: 0.3622 - val_acc: 0.9328Epoch 985/100068s 137ms/step - loss: 0.0991 - acc: 0.9987 - val_loss: 0.3628 - val_acc: 0.9322Epoch 986/100068s 137ms/step - loss: 0.0993 - acc: 0.9987 - val_loss: 0.3625 - val_acc: 0.9319Epoch 987/100068s 137ms/step - loss: 0.0995 - acc: 0.9986 - val_loss: 0.3629 - val_acc: 0.9317Epoch 988/100069s 137ms/step - loss: 0.0993 - acc: 0.9985 - val_loss: 0.3628 - val_acc: 0.9319Epoch 989/100069s 137ms/step - loss: 0.0997 - acc: 0.9984 - val_loss: 0.3624 - val_acc: 0.9322Epoch 990/100069s 138ms/step - loss: 0.0993 - acc: 0.9986 - val_loss: 0.3622 - val_acc: 0.9323Epoch 991/100068s 137ms/step - loss: 0.0993 - acc: 0.9986 - val_loss: 0.3625 - val_acc: 0.9327Epoch 992/100069s 137ms/step - loss: 0.0993 - acc: 0.9988 - val_loss: 0.3630 - val_acc: 0.9325Epoch 993/100068s 137ms/step - loss: 0.0992 - acc: 0.9984 - val_loss: 0.3634 - val_acc: 0.9320Epoch 994/100069s 138ms/step - loss: 0.0991 - acc: 0.9988 - val_loss: 0.3627 - val_acc: 0.9328Epoch 995/100069s 138ms/step - loss: 0.0989 - acc: 0.9989 - val_loss: 0.3637 - val_acc: 0.9321Epoch 996/100069s 138ms/step - loss: 0.0994 - acc: 0.9986 - val_loss: 0.3623 - val_acc: 0.9319Epoch 997/100069s 138ms/step - loss: 0.0987 - acc: 0.9987 - val_loss: 0.3622 - val_acc: 0.9322Epoch 998/100069s 138ms/step - loss: 0.0989 - acc: 0.9988 - val_loss: 0.3621 - val_acc: 0.9325Epoch 999/100069s 138ms/step - loss: 0.0993 - acc: 0.9984 - val_loss: 0.3615 - val_acc: 0.9326Epoch 1000/100069s 138ms/step - loss: 0.0986 - acc: 0.9988 - val_loss: 0.3614 - val_acc: 0.9323Train loss: 0.09943642792105675Train accuracy: 0.9982600016593933Test loss: 0.3614072059094906Test accuracy: 0.9322999995946885在使用了shear_range = 30的数据增强以后，准确率降了呢。。
Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht, Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, IEEE Transactions on Industrial Electronics, 2020, DOI: 10.1109/TIE.2020.2972458, Date of Publication: 13 February 2020
https://ieeexplore.ieee.org/d...