续上一篇：
【不可忽视的这一种】Dynamic ReLU：自适应参数化ReLU（调参记录6）
自适应参数化ReLU是一种动态ReLU（Dynamic ReLU），于2019年5月3日投稿至IEEE Transactions on Industrial Electronics，于2020年1月24日录用，于2020年2月13日在IEEE官网公布。
本文冒着过拟合的风险，将卷积核的个数增加成32个、64个和128个，继续测试自适应参数化ReLU在Cifar10上的效果。
自适应参数化ReLU的基本原理如下图所示：
Keras代码如下：
#!/usr/bin/env python3# -*- coding: utf-8 -*-"""Created on Tue Apr 14 04:17:45 2020Implemented using TensorFlow 1.0.1 and Keras 2.2.1Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht,Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, IEEE Transactions on Industrial Electronics, 2020, DOI: 10.1109/TIE.2020.2972458,Date of Publication: 13 February 2020@author: Minghang Zhao"""from __future__ import print_functionimport kerasimport numpy as npfrom keras.datasets import cifar10from keras.layers import Dense, Conv2D, BatchNormalization, Activation, Minimumfrom keras.layers import AveragePooling2D, Input, GlobalAveragePooling2D, Concatenate, Reshapefrom keras.regularizers import l2from keras import backend as Kfrom keras.models import Modelfrom keras import optimizersfrom keras.preprocessing.image import ImageDataGeneratorfrom keras.callbacks import LearningRateSchedulerK.set_learning_phase(1)# The data, split between train and test sets(x_train, y_train), (x_test, y_test) = cifar10.load_data()x_train = x_train.astype('float32') / 255.x_test = x_test.astype('float32') / 255.x_test = x_test-np.mean(x_train)x_train = x_train-np.mean(x_train)print('x_train shape:', x_train.shape)print(x_train.shape[0], 'train samples')print(x_test.shape[0], 'test samples')# convert class vectors to binary class matricesy_train = keras.utils.to_categorical(y_train, 10)y_test = keras.utils.to_categorical(y_test, 10)# Schedule the learning rate, multiply 0.1 every 300 epochesdef scheduler(epoch): if epoch % 300 == 0 and epoch != 0: lr = K.get_value(model.optimizer.lr) K.set_value(model.optimizer.lr, lr * 0.1) print("lr changed to {}".format(lr * 0.1)) return K.get_value(model.optimizer.lr)# An adaptively parametric rectifier linear unit (APReLU)def aprelu(inputs): # get the number of channels channels = inputs.get_shape().as_list()[-1] # get a zero feature map zeros_input = keras.layers.subtract([inputs, inputs]) # get a feature map with only positive features pos_input = Activation('relu')(inputs) # get a feature map with only negative features neg_input = Minimum()([inputs,zeros_input]) # define a network to obtain the scaling coefficients scales_p = GlobalAveragePooling2D()(pos_input) scales_n = GlobalAveragePooling2D()(neg_input) scales = Concatenate()([scales_n, scales_p]) scales = Dense(channels, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales) scales = BatchNormalization()(scales) scales = Activation('relu')(scales) scales = Dense(channels, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales) scales = BatchNormalization()(scales) scales = Activation('sigmoid')(scales) scales = Reshape((1,1,channels))(scales) # apply a paramtetric relu neg_part = keras.layers.multiply([scales, neg_input]) return keras.layers.add([pos_input, neg_part])# Residual Blockdef residual_block(incoming, nb_blocks, out_channels, downsample=False, downsample_strides=2): residual = incoming in_channels = incoming.get_shape().as_list()[-1] for i in range(nb_blocks): identity = residual if not downsample: downsample_strides = 1 residual = BatchNormalization()(residual) residual = aprelu(residual) residual = Conv2D(out_channels, 3, strides=(downsample_strides, downsample_strides), padding='same', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(residual) residual = BatchNormalization()(residual) residual = aprelu(residual) residual = Conv2D(out_channels, 3, padding='same', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(residual) # Downsampling if downsample_strides > 1: identity = AveragePooling2D(pool_size=(1,1), strides=(2,2))(identity) # Zero_padding to match channels if in_channels != out_channels: zeros_identity = keras.layers.subtract([identity, identity]) identity = keras.layers.concatenate([identity, zeros_identity]) in_channels = out_channels residual = keras.layers.add([residual, identity]) return residual# define and train a modelinputs = Input(shape=(32, 32, 3))net = Conv2D(32, 3, padding='same', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(inputs)net = residual_block(net, 9, 32, downsample=False)net = residual_block(net, 1, 64, downsample=True)net = residual_block(net, 8, 64, downsample=False)net = residual_block(net, 1, 128, downsample=True)net = residual_block(net, 8, 128, downsample=False)net = BatchNormalization()(net)net = Activation('relu')(net)net = GlobalAveragePooling2D()(net)outputs = Dense(10, activation='softmax', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(net)model = Model(inputs=inputs, outputs=outputs)sgd = optimizers.SGD(lr=0.1, decay=0., momentum=0.9, nesterov=True)model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])# data augmentationdatagen = ImageDataGenerator( # randomly rotate images in the range (deg 0 to 180) rotation_range=30, # randomly flip images horizontal_flip=True, # randomly shift images horizontally width_shift_range=0.125, # randomly shift images vertically height_shift_range=0.125)reduce_lr = LearningRateScheduler(scheduler)# fit the model on the batches generated by datagen.flow().model.fit_generator(datagen.flow(x_train, y_train, batch_size=100), validation_data=(x_test, y_test), epochs=1000, verbose=1, callbacks=[reduce_lr], workers=4)# get resultsK.set_learning_phase(0)DRSN_train_score = model.evaluate(x_train, y_train, batch_size=100, verbose=0)print('Train loss:', DRSN_train_score[0])print('Train accuracy:', DRSN_train_score[1])DRSN_test_score = model.evaluate(x_test, y_test, batch_size=100, verbose=0)print('Test loss:', DRSN_test_score[0])print('Test accuracy:', DRSN_test_score[1])先复制一次spyder窗口里的实验结果：
Epoch 270/100091s 182ms/step - loss: 0.5576 - acc: 0.9245 - val_loss: 0.6619 - val_acc: 0.8960Epoch 271/100091s 182ms/step - loss: 0.5605 - acc: 0.9250 - val_loss: 0.6675 - val_acc: 0.8908Epoch 272/100091s 182ms/step - loss: 0.5578 - acc: 0.9244 - val_loss: 0.6578 - val_acc: 0.8951Epoch 273/100091s 182ms/step - loss: 0.5625 - acc: 0.9232 - val_loss: 0.6663 - val_acc: 0.8907Epoch 274/100091s 182ms/step - loss: 0.5598 - acc: 0.9246 - val_loss: 0.6435 - val_acc: 0.9059Epoch 275/100091s 182ms/step - loss: 0.5567 - acc: 0.9265 - val_loss: 0.6589 - val_acc: 0.8949Epoch 276/100091s 182ms/step - loss: 0.5616 - acc: 0.9235 - val_loss: 0.6439 - val_acc: 0.9002Epoch 277/100091s 182ms/step - loss: 0.5568 - acc: 0.9258 - val_loss: 0.6731 - val_acc: 0.8913 ETA: 16s - loss: 0.5542 - acc: 0.9269Epoch 278/100091s 182ms/step - loss: 0.5582 - acc: 0.9254 - val_loss: 0.6437 - val_acc: 0.8995Epoch 279/100091s 182ms/step - loss: 0.5530 - acc: 0.9270 - val_loss: 0.6416 - val_acc: 0.9002Epoch 280/100091s 182ms/step - loss: 0.5603 - acc: 0.9245 - val_loss: 0.6566 - val_acc: 0.8960Epoch 281/100091s 182ms/step - loss: 0.5613 - acc: 0.9241 - val_loss: 0.6432 - val_acc: 0.9003Epoch 282/100091s 182ms/step - loss: 0.5568 - acc: 0.9250 - val_loss: 0.6573 - val_acc: 0.8950Epoch 283/100091s 182ms/step - loss: 0.5580 - acc: 0.9253 - val_loss: 0.6518 - val_acc: 0.8961 ETA: 10s - loss: 0.5551 - acc: 0.9260Epoch 284/100091s 182ms/step - loss: 0.5495 - acc: 0.9276 - val_loss: 0.6736 - val_acc: 0.8918Epoch 285/100091s 182ms/step - loss: 0.5611 - acc: 0.9238 - val_loss: 0.6538 - val_acc: 0.8962Epoch 286/100091s 182ms/step - loss: 0.5590 - acc: 0.9250 - val_loss: 0.6563 - val_acc: 0.8965Epoch 287/100091s 182ms/step - loss: 0.5581 - acc: 0.9245 - val_loss: 0.6482 - val_acc: 0.9035Epoch 288/100091s 182ms/step - loss: 0.5607 - acc: 0.9233 - val_loss: 0.6516 - val_acc: 0.8984Epoch 289/100091s 182ms/step - loss: 0.5608 - acc: 0.9252 - val_loss: 0.6562 - val_acc: 0.8984Epoch 290/100091s 182ms/step - loss: 0.5599 - acc: 0.9240 - val_loss: 0.6941 - val_acc: 0.8847Epoch 291/100091s 182ms/step - loss: 0.5600 - acc: 0.9244 - val_loss: 0.6695 - val_acc: 0.8902Epoch 292/100091s 182ms/step - loss: 0.5628 - acc: 0.9232 - val_loss: 0.6580 - val_acc: 0.8979Epoch 293/100091s 182ms/step - loss: 0.5602 - acc: 0.9242 - val_loss: 0.6726 - val_acc: 0.8913Epoch 294/100091s 182ms/step - loss: 0.5582 - acc: 0.9249 - val_loss: 0.6917 - val_acc: 0.8901Epoch 295/100091s 182ms/step - loss: 0.5559 - acc: 0.9265 - val_loss: 0.6805 - val_acc: 0.88967/500 [======================>.......] - ETA: 19s - loss: 0.5537 - acc: 0.9275Epoch 296/100091s 182ms/step - loss: 0.5570 - acc: 0.9265 - val_loss: 0.6315 - val_acc: 0.9039Epoch 297/100091s 182ms/step - loss: 0.5572 - acc: 0.9244 - val_loss: 0.6647 - val_acc: 0.8918Epoch 298/100091s 182ms/step - loss: 0.5555 - acc: 0.9259 - val_loss: 0.6540 - val_acc: 0.8960Epoch 299/100091s 182ms/step - loss: 0.5575 - acc: 0.9266 - val_loss: 0.6648 - val_acc: 0.8941Epoch 300/100091s 182ms/step - loss: 0.5517 - acc: 0.9277 - val_loss: 0.6555 - val_acc: 0.8975Epoch 301/1000lr changed to 0.01000000014901161291s 182ms/step - loss: 0.4683 - acc: 0.9572 - val_loss: 0.5677 - val_acc: 0.9248Epoch 302/100091s 182ms/step - loss: 0.4174 - acc: 0.9735 - val_loss: 0.5622 - val_acc: 0.9256Epoch 303/100091s 182ms/step - loss: 0.3968 - acc: 0.9785 - val_loss: 0.5500 - val_acc: 0.9291Epoch 304/100091s 182ms/step - loss: 0.3806 - acc: 0.9814 - val_loss: 0.5520 - val_acc: 0.9283Epoch 305/100091s 181ms/step - loss: 0.3687 - acc: 0.9832 - val_loss: 0.5442 - val_acc: 0.9306Epoch 306/100091s 181ms/step - loss: 0.3555 - acc: 0.9864 - val_loss: 0.5454 - val_acc: 0.9284Epoch 307/100091s 182ms/step - loss: 0.3485 - acc: 0.9863 - val_loss: 0.5409 - val_acc: 0.9286Epoch 308/100091s 181ms/step - loss: 0.3379 - acc: 0.9885 - val_loss: 0.5383 - val_acc: 0.9305Epoch 309/100091s 181ms/step - loss: 0.3272 - acc: 0.9904 - val_loss: 0.5344 - val_acc: 0.9309Epoch 310/100090s 181ms/step - loss: 0.3213 - acc: 0.9900 - val_loss: 0.5333 - val_acc: 0.9298Epoch 311/100090s 181ms/step - loss: 0.3143 - acc: 0.9909 - val_loss: 0.5365 - val_acc: 0.9283Epoch 312/100090s 181ms/step - loss: 0.3092 - acc: 0.9910 - val_loss: 0.5287 - val_acc: 0.9311Epoch 313/100090s 181ms/step - loss: 0.3006 - acc: 0.9919 - val_loss: 0.5324 - val_acc: 0.9283Epoch 314/100090s 181ms/step - loss: 0.2945 - acc: 0.9916 - val_loss: 0.5286 - val_acc: 0.9300Epoch 315/100090s 181ms/step - loss: 0.2886 - acc: 0.9923 - val_loss: 0.5181 - val_acc: 0.9323Epoch 316/100091s 181ms/step - loss: 0.2823 - acc: 0.9932 - val_loss: 0.5212 - val_acc: 0.9286Epoch 317/100090s 181ms/step - loss: 0.2778 - acc: 0.9930 - val_loss: 0.5182 - val_acc: 0.9296Epoch 318/100091s 181ms/step - loss: 0.2720 - acc: 0.9936 - val_loss: 0.5122 - val_acc: 0.9287Epoch 319/100091s 181ms/step - loss: 0.2662 - acc: 0.9940 - val_loss: 0.5083 - val_acc: 0.9277Epoch 320/100091s 181ms/step - loss: 0.2597 - acc: 0.9944 - val_loss: 0.5018 - val_acc: 0.9315Epoch 321/100091s 181ms/step - loss: 0.2560 - acc: 0.9944 - val_loss: 0.5086 - val_acc: 0.9296Epoch 322/100090s 181ms/step - loss: 0.2526 - acc: 0.9939 - val_loss: 0.5059 - val_acc: 0.9274Epoch 323/100090s 181ms/step - loss: 0.2466 - acc: 0.9945 - val_loss: 0.4991 - val_acc: 0.9302Epoch 324/100090s 181ms/step - loss: 0.2431 - acc: 0.9945 - val_loss: 0.5006 - val_acc: 0.9273Epoch 325/100090s 181ms/step - loss: 0.2384 - acc: 0.9947 - val_loss: 0.4914 - val_acc: 0.9296Epoch 326/100091s 181ms/step - loss: 0.2334 - acc: 0.9948 - val_loss: 0.4938 - val_acc: 0.9291Epoch 327/100091s 181ms/step - loss: 0.2301 - acc: 0.9949 - val_loss: 0.4869 - val_acc: 0.9303Epoch 328/100090s 181ms/step - loss: 0.2253 - acc: 0.9952 - val_loss: 0.4850 - val_acc: 0.9293Epoch 329/100091s 181ms/step - loss: 0.2219 - acc: 0.9953 - val_loss: 0.4858 - val_acc: 0.9272Epoch 330/100090s 181ms/step - loss: 0.2170 - acc: 0.9959 - val_loss: 0.4834 - val_acc: 0.9277Epoch 331/100090s 181ms/step - loss: 0.2140 - acc: 0.9953 - val_loss: 0.4814 - val_acc: 0.9276Epoch 332/100090s 181ms/step - loss: 0.2118 - acc: 0.9951 - val_loss: 0.4767 - val_acc: 0.9273Epoch 333/100090s 181ms/step - loss: 0.2077 - acc: 0.9953 - val_loss: 0.4709 - val_acc: 0.9303Epoch 334/100091s 181ms/step - loss: 0.2042 - acc: 0.9952 - val_loss: 0.4808 - val_acc: 0.9257Epoch 335/100090s 181ms/step - loss: 0.2015 - acc: 0.9951 - val_loss: 0.4691 - val_acc: 0.9287Epoch 336/100090s 181ms/step - loss: 0.1988 - acc: 0.9952 - val_loss: 0.4659 - val_acc: 0.9273Epoch 337/100090s 181ms/step - loss: 0.1930 - acc: 0.9961 - val_loss: 0.4667 - val_acc: 0.9293Epoch 338/100090s 181ms/step - loss: 0.1901 - acc: 0.9961 - val_loss: 0.4559 - val_acc: 0.9299Epoch 339/100090s 181ms/step - loss: 0.1872 - acc: 0.9962 - val_loss: 0.4676 - val_acc: 0.9269Epoch 340/100090s 181ms/step - loss: 0.1890 - acc: 0.9940 - val_loss: 0.4556 - val_acc: 0.9291Epoch 341/100090s 181ms/step - loss: 0.1832 - acc: 0.9954 - val_loss: 0.4552 - val_acc: 0.9268Epoch 342/100090s 181ms/step - loss: 0.1798 - acc: 0.9954 - val_loss: 0.4556 - val_acc: 0.9294Epoch 343/100090s 181ms/step - loss: 0.1782 - acc: 0.9950 - val_loss: 0.4498 - val_acc: 0.9255Epoch 344/100091s 181ms/step - loss: 0.1775 - acc: 0.9943 - val_loss: 0.4522 - val_acc: 0.9263Epoch 345/100090s 181ms/step - loss: 0.1747 - acc: 0.9950 - val_loss: 0.4376 - val_acc: 0.9258Epoch 346/100090s 181ms/step - loss: 0.1702 - acc: 0.9955 - val_loss: 0.4464 - val_acc: 0.9263Epoch 347/100090s 181ms/step - loss: 0.1693 - acc: 0.9949 - val_loss: 0.4515 - val_acc: 0.9269Epoch 348/100090s 181ms/step - loss: 0.1654 - acc: 0.9951 - val_loss: 0.4452 - val_acc: 0.9249Epoch 349/100090s 181ms/step - loss: 0.1649 - acc: 0.9948 - val_loss: 0.4461 - val_acc: 0.9249Epoch 350/100090s 181ms/step - loss: 0.1632 - acc: 0.9944 - val_loss: 0.4301 - val_acc: 0.9291Epoch 351/100091s 181ms/step - loss: 0.1616 - acc: 0.9941 - val_loss: 0.4411 - val_acc: 0.9237Epoch 352/100090s 181ms/step - loss: 0.1594 - acc: 0.9948 - val_loss: 0.4301 - val_acc: 0.9308Epoch 353/100090s 181ms/step - loss: 0.1593 - acc: 0.9937 - val_loss: 0.4230 - val_acc: 0.9265Epoch 354/100090s 181ms/step - loss: 0.1565 - acc: 0.9942 - val_loss: 0.4243 - val_acc: 0.9272Epoch 355/100090s 181ms/step - loss: 0.1532 - acc: 0.9946 - val_loss: 0.4290 - val_acc: 0.9258Epoch 356/100090s 181ms/step - loss: 0.1525 - acc: 0.9945 - val_loss: 0.4171 - val_acc: 0.9294Epoch 357/100090s 181ms/step - loss: 0.1505 - acc: 0.9943 - val_loss: 0.4205 - val_acc: 0.9273Epoch 358/100090s 181ms/step - loss: 0.1481 - acc: 0.9945 - val_loss: 0.4295 - val_acc: 0.9227Epoch 359/100090s 181ms/step - loss: 0.1487 - acc: 0.9938 - val_loss: 0.4185 - val_acc: 0.9248Epoch 360/100091s 181ms/step - loss: 0.1452 - acc: 0.9946 - val_loss: 0.4244 - val_acc: 0.9256Epoch 361/100090s 181ms/step - loss: 0.1481 - acc: 0.9925 - val_loss: 0.4267 - val_acc: 0.9220Epoch 362/100091s 181ms/step - loss: 0.1468 - acc: 0.9929 - val_loss: 0.4009 - val_acc: 0.9265Epoch 363/100090s 181ms/step - loss: 0.1433 - acc: 0.9941 - val_loss: 0.4098 - val_acc: 0.9259Epoch 364/100091s 181ms/step - loss: 0.1441 - acc: 0.9928 - val_loss: 0.4189 - val_acc: 0.9234Epoch 365/100091s 181ms/step - loss: 0.1426 - acc: 0.9934 - val_loss: 0.4099 - val_acc: 0.9251Epoch 366/100090s 181ms/step - loss: 0.1383 - acc: 0.9941 - val_loss: 0.4007 - val_acc: 0.9256Epoch 367/100091s 181ms/step - loss: 0.1395 - acc: 0.9933 - val_loss: 0.3938 - val_acc: 0.9269Epoch 368/100090s 181ms/step - loss: 0.1379 - acc: 0.9934 - val_loss: 0.4024 - val_acc: 0.9253Epoch 369/100090s 181ms/step - loss: 0.1359 - acc: 0.9935 - val_loss: 0.4021 - val_acc: 0.9265Epoch 370/100090s 181ms/step - loss: 0.1370 - acc: 0.9928 - val_loss: 0.3925 - val_acc: 0.9270Epoch 371/100090s 181ms/step - loss: 0.1373 - acc: 0.9924 - val_loss: 0.3932 - val_acc: 0.9259Epoch 372/100090s 181ms/step - loss: 0.1349 - acc: 0.9926 - val_loss: 0.4055 - val_acc: 0.9254Epoch 373/100090s 181ms/step - loss: 0.1342 - acc: 0.9927 - val_loss: 0.3934 - val_acc: 0.9289Epoch 374/100090s 181ms/step - loss: 0.1352 - acc: 0.9919 - val_loss: 0.4131 - val_acc: 0.9225Epoch 375/100091s 181ms/step - loss: 0.1351 - acc: 0.9917 - val_loss: 0.3916 - val_acc: 0.9249Epoch 376/100090s 181ms/step - loss: 0.1317 - acc: 0.9929 - val_loss: 0.4016 - val_acc: 0.9237Epoch 377/100090s 181ms/step - loss: 0.1316 - acc: 0.9930 - val_loss: 0.3906 - val_acc: 0.9259Epoch 378/100090s 181ms/step - loss: 0.1307 - acc: 0.9925 - val_loss: 0.3954 - val_acc: 0.9248Epoch 379/100090s 181ms/step - loss: 0.1328 - acc: 0.9914 - val_loss: 0.3997 - val_acc: 0.9221Epoch 380/100090s 181ms/step - loss: 0.1345 - acc: 0.9902 - val_loss: 0.3934 - val_acc: 0.9260Epoch 381/100090s 181ms/step - loss: 0.1319 - acc: 0.9915 - val_loss: 0.3973 - val_acc: 0.9232Epoch 382/100090s 181ms/step - loss: 0.1307 - acc: 0.9920 - val_loss: 0.4105 - val_acc: 0.9220Epoch 383/100090s 181ms/step - loss: 0.1281 - acc: 0.9924 - val_loss: 0.3980 - val_acc: 0.9242Epoch 384/100090s 181ms/step - loss: 0.1305 - acc: 0.9911 - val_loss: 0.4200 - val_acc: 0.9194Epoch 385/100090s 181ms/step - loss: 0.1311 - acc: 0.9910 - val_loss: 0.4101 - val_acc: 0.9184Epoch 386/100091s 181ms/step - loss: 0.1291 - acc: 0.9913 - val_loss: 0.4074 - val_acc: 0.9225Epoch 387/100090s 181ms/step - loss: 0.1316 - acc: 0.9902 - val_loss: 0.4087 - val_acc: 0.9180Epoch 388/100090s 181ms/step - loss: 0.1306 - acc: 0.9906 - val_loss: 0.4021 - val_acc: 0.9192Epoch 389/100090s 181ms/step - loss: 0.1295 - acc: 0.9910 - val_loss: 0.3877 - val_acc: 0.9250Epoch 390/100090s 181ms/step - loss: 0.1285 - acc: 0.9913 - val_loss: 0.3914 - val_acc: 0.9208Epoch 391/100090s 181ms/step - loss: 0.1284 - acc: 0.9911 - val_loss: 0.3887 - val_acc: 0.9221Epoch 392/100090s 181ms/step - loss: 0.1289 - acc: 0.9911 - val_loss: 0.3992 - val_acc: 0.9262Epoch 393/100090s 181ms/step - loss: 0.1265 - acc: 0.9919 - val_loss: 0.4006 - val_acc: 0.9213Epoch 394/100090s 181ms/step - loss: 0.1261 - acc: 0.9911 - val_loss: 0.3943 - val_acc: 0.9238Epoch 395/100090s 181ms/step - loss: 0.1277 - acc: 0.9908 - val_loss: 0.3963 - val_acc: 0.9236Epoch 396/100090s 181ms/step - loss: 0.1286 - acc: 0.9902 - val_loss: 0.4147 - val_acc: 0.9194Epoch 397/100090s 181ms/step - loss: 0.1309 - acc: 0.9894 - val_loss: 0.3996 - val_acc: 0.9192Epoch 398/100091s 181ms/step - loss: 0.1268 - acc: 0.9912 - val_loss: 0.3952 - val_acc: 0.9225Epoch 399/100090s 181ms/step - loss: 0.1255 - acc: 0.9911 - val_loss: 0.4084 - val_acc: 0.9204Epoch 400/100091s 181ms/step - loss: 0.1268 - acc: 0.9902 - val_loss: 0.3954 - val_acc: 0.9209Epoch 401/100090s 181ms/step - loss: 0.1263 - acc: 0.9902 - val_loss: 0.4022 - val_acc: 0.9224Epoch 402/100090s 181ms/step - loss: 0.1270 - acc: 0.9904 - val_loss: 0.3891 - val_acc: 0.9246Epoch 403/100090s 181ms/step - loss: 0.1272 - acc: 0.9899 - val_loss: 0.4038 - val_acc: 0.9202Epoch 404/100091s 181ms/step - loss: 0.1307 - acc: 0.9885 - val_loss: 0.4022 - val_acc: 0.9205Epoch 405/100091s 181ms/step - loss: 0.1298 - acc: 0.9891 - val_loss: 0.3900 - val_acc: 0.9213Epoch 406/100090s 181ms/step - loss: 0.1277 - acc: 0.9897 - val_loss: 0.3946 - val_acc: 0.9209Epoch 407/100090s 181ms/step - loss: 0.1257 - acc: 0.9905 - val_loss: 0.3962 - val_acc: 0.9216Epoch 408/100091s 181ms/step - loss: 0.1262 - acc: 0.9906 - val_loss: 0.4070 - val_acc: 0.9205Epoch 409/100090s 181ms/step - loss: 0.1273 - acc: 0.9899 - val_loss: 0.3869 - val_acc: 0.9249Epoch 410/100090s 181ms/step - loss: 0.1268 - acc: 0.9902 - val_loss: 0.4044 - val_acc: 0.9201Epoch 411/100090s 181ms/step - loss: 0.1264 - acc: 0.9900 - val_loss: 0.4039 - val_acc: 0.9214Epoch 412/100091s 181ms/step - loss: 0.1278 - acc: 0.9896 - val_loss: 0.4072 - val_acc: 0.9187Epoch 413/100090s 181ms/step - loss: 0.1267 - acc: 0.9900 - val_loss: 0.4132 - val_acc: 0.9174Epoch 414/100090s 181ms/step - loss: 0.1294 - acc: 0.9890 - val_loss: 0.3933 - val_acc: 0.9214Epoch 415/100090s 181ms/step - loss: 0.1236 - acc: 0.9911 - val_loss: 0.4097 - val_acc: 0.9205Epoch 416/100090s 181ms/step - loss: 0.1279 - acc: 0.9896 - val_loss: 0.3939 - val_acc: 0.9206Epoch 417/100090s 181ms/step - loss: 0.1243 - acc: 0.9907 - val_loss: 0.4011 - val_acc: 0.9213Epoch 418/100090s 181ms/step - loss: 0.1255 - acc: 0.9904 - val_loss: 0.4279 - val_acc: 0.9141Epoch 419/100091s 181ms/step - loss: 0.1267 - acc: 0.9905 - val_loss: 0.4297 - val_acc: 0.9130Epoch 420/100090s 181ms/step - loss: 0.1245 - acc: 0.9907 - val_loss: 0.4141 - val_acc: 0.9166Epoch 421/100090s 181ms/step - loss: 0.1270 - acc: 0.9897 - val_loss: 0.3903 - val_acc: 0.9203Epoch 422/100090s 181ms/step - loss: 0.1213 - acc: 0.9916 - val_loss: 0.4057 - val_acc: 0.9199Epoch 423/100091s 181ms/step - loss: 0.1213 - acc: 0.9915 - val_loss: 0.3929 - val_acc: 0.9192Epoch 424/100090s 181ms/step - loss: 0.1215 - acc: 0.9916 - val_loss: 0.3834 - val_acc: 0.9251Epoch 425/100090s 181ms/step - loss: 0.1224 - acc: 0.9905 - val_loss: 0.4071 - val_acc: 0.9215Epoch 426/100091s 181ms/step - loss: 0.1280 - acc: 0.9891 - val_loss: 0.4023 - val_acc: 0.9208Epoch 427/100091s 181ms/step - loss: 0.1274 - acc: 0.9893 - val_loss: 0.3839 - val_acc: 0.9223Epoch 428/100090s 181ms/step - loss: 0.1244 - acc: 0.9904 - val_loss: 0.3948 - val_acc: 0.9215Epoch 429/100090s 181ms/step - loss: 0.1247 - acc: 0.9899 - val_loss: 0.4135 - val_acc: 0.9181Epoch 430/100091s 181ms/step - loss: 0.1218 - acc: 0.9915 - val_loss: 0.3810 - val_acc: 0.9256Epoch 431/100090s 181ms/step - loss: 0.1230 - acc: 0.9905 - val_loss: 0.3961 - val_acc: 0.9203Epoch 432/100091s 182ms/step - loss: 0.1262 - acc: 0.9894 - val_loss: 0.3939 - val_acc: 0.9213Epoch 433/100091s 182ms/step - loss: 0.1273 - acc: 0.9889 - val_loss: 0.4070 - val_acc: 0.9139Epoch 434/100091s 182ms/step - loss: 0.1228 - acc: 0.9911 - val_loss: 0.3896 - val_acc: 0.9214Epoch 435/100091s 182ms/step - loss: 0.1252 - acc: 0.9900 - val_loss: 0.3858 - val_acc: 0.9217Epoch 436/100091s 182ms/step - loss: 0.1246 - acc: 0.9905 - val_loss: 0.3926 - val_acc: 0.9214Epoch 437/100091s 182ms/step - loss: 0.1254 - acc: 0.9897 - val_loss: 0.3927 - val_acc: 0.9247Epoch 438/100091s 182ms/step - loss: 0.1238 - acc: 0.9903 - val_loss: 0.4091 - val_acc: 0.9155Epoch 439/100091s 182ms/step - loss: 0.1259 - acc: 0.9895 - val_loss: 0.4237 - val_acc: 0.9116Epoch 440/100091s 182ms/step - loss: 0.1263 - acc: 0.9896 - val_loss: 0.4008 - val_acc: 0.9178Epoch 441/100091s 182ms/step - loss: 0.1268 - acc: 0.9892 - val_loss: 0.4129 - val_acc: 0.9141Epoch 442/100091s 182ms/step - loss: 0.1261 - acc: 0.9902 - val_loss: 0.3831 - val_acc: 0.9238Epoch 443/100091s 182ms/step - loss: 0.1234 - acc: 0.9905 - val_loss: 0.4066 - val_acc: 0.9175Epoch 444/100091s 182ms/step - loss: 0.1258 - acc: 0.9903 - val_loss: 0.4081 - val_acc: 0.9177Epoch 445/100091s 182ms/step - loss: 0.1279 - acc: 0.9889 - val_loss: 0.3980 - val_acc: 0.9208Epoch 446/100091s 181ms/step - loss: 0.1257 - acc: 0.9896 - val_loss: 0.3887 - val_acc: 0.9220Epoch 447/100091s 182ms/step - loss: 0.1240 - acc: 0.9905 - val_loss: 0.4044 - val_acc: 0.9180Epoch 448/100091s 182ms/step - loss: 0.1270 - acc: 0.9895 - val_loss: 0.4061 - val_acc: 0.9189Epoch 449/100091s 182ms/step - loss: 0.1229 - acc: 0.9911 - val_loss: 0.3971 - val_acc: 0.9220Epoch 450/100091s 182ms/step - loss: 0.1217 - acc: 0.9918 - val_loss: 0.4036 - val_acc: 0.9227Epoch 451/100091s 182ms/step - loss: 0.1240 - acc: 0.9906 - val_loss: 0.4011 - val_acc: 0.9216Epoch 452/100091s 182ms/step - loss: 0.1239 - acc: 0.9901 - val_loss: 0.4079 - val_acc: 0.9173Epoch 453/100091s 182ms/step - loss: 0.1224 - acc: 0.9906 - val_loss: 0.3917 - val_acc: 0.9240Epoch 454/100091s 182ms/step - loss: 0.1265 - acc: 0.9891 - val_loss: 0.3877 - val_acc: 0.9235 ETA: 34s - loss: 0.1254 - acc: 0.9893Epoch 455/100091s 182ms/step - loss: 0.1233 - acc: 0.9910 - val_loss: 0.4031 - val_acc: 0.9177Epoch 456/100091s 182ms/step - loss: 0.1239 - acc: 0.9904 - val_loss: 0.4203 - val_acc: 0.9185Epoch 457/100091s 182ms/step - loss: 0.1240 - acc: 0.9905 - val_loss: 0.3918 - val_acc: 0.9247Epoch 458/100091s 182ms/step - loss: 0.1247 - acc: 0.9898 - val_loss: 0.4155 - val_acc: 0.9176Epoch 459/100091s 182ms/step - loss: 0.1239 - acc: 0.9900 - val_loss: 0.3980 - val_acc: 0.9207Epoch 460/100091s 182ms/step - loss: 0.1296 - acc: 0.9881 - val_loss: 0.3954 - val_acc: 0.9190 ETA: 1:14 - loss: 0.1236 - acc: 0.9908 - ETA: 1:09 - loss: 0.1257 - acc: 0.9903Epoch 461/100091s 182ms/step - loss: 0.1232 - acc: 0.9908 - val_loss: 0.4039 - val_acc: 0.9223Epoch 462/100091s 182ms/step - loss: 0.1283 - acc: 0.9888 - val_loss: 0.4285 - val_acc: 0.9136Epoch 463/100091s 182ms/step - loss: 0.1264 - acc: 0.9899 - val_loss: 0.4025 - val_acc: 0.9191Epoch 464/100091s 181ms/step - loss: 0.1236 - acc: 0.9909 - val_loss: 0.3952 - val_acc: 0.9205Epoch 465/100091s 182ms/step - loss: 0.1204 - acc: 0.9921 - val_loss: 0.4008 - val_acc: 0.9207 ETA: 33s - loss: 0.1189 - acc: 0.9928Epoch 466/100090s 181ms/step - loss: 0.1233 - acc: 0.9905 - val_loss: 0.4098 - val_acc: 0.9158Epoch 467/100091s 182ms/step - loss: 0.1207 - acc: 0.9916 - val_loss: 0.4012 - val_acc: 0.9160Epoch 468/100091s 182ms/step - loss: 0.1231 - acc: 0.9910 - val_loss: 0.3880 - val_acc: 0.9248Epoch 469/100091s 182ms/step - loss: 0.1241 - acc: 0.9900 - val_loss: 0.4136 - val_acc: 0.9175Epoch 470/100091s 182ms/step - loss: 0.1255 - acc: 0.9894 - val_loss: 0.4084 - val_acc: 0.9202Epoch 471/100091s 182ms/step - loss: 0.1253 - acc: 0.9902 - val_loss: 0.3892 - val_acc: 0.9225Epoch 472/100091s 182ms/step - loss: 0.1269 - acc: 0.9891 - val_loss: 0.4101 - val_acc: 0.9201Epoch 473/100091s 182ms/step - loss: 0.1226 - acc: 0.9913 - val_loss: 0.4143 - val_acc: 0.9167Epoch 474/100091s 182ms/step - loss: 0.1230 - acc: 0.9911 - val_loss: 0.4019 - val_acc: 0.9184Epoch 475/100091s 182ms/step - loss: 0.1242 - acc: 0.9902 - val_loss: 0.4229 - val_acc: 0.9181Epoch 476/100091s 182ms/step - loss: 0.1251 - acc: 0.9905 - val_loss: 0.3879 - val_acc: 0.9241Epoch 477/100091s 182ms/step - loss: 0.1243 - acc: 0.9899 - val_loss: 0.4191 - val_acc: 0.9172Epoch 478/100091s 182ms/step - loss: 0.1240 - acc: 0.9907 - val_loss: 0.3942 - val_acc: 0.9230Epoch 479/100091s 182ms/step - loss: 0.1230 - acc: 0.9909 - val_loss: 0.3843 - val_acc: 0.9274Epoch 480/100091s 182ms/step - loss: 0.1207 - acc: 0.9918 - val_loss: 0.4098 - val_acc: 0.9196 ETA: 2s - loss: 0.1208 - acc: 0.9918Epoch 481/100091s 182ms/step - loss: 0.1244 - acc: 0.9905 - val_loss: 0.4048 - val_acc: 0.9172Epoch 482/100091s 182ms/step - loss: 0.1250 - acc: 0.9902 - val_loss: 0.4160 - val_acc: 0.9203Epoch 483/100091s 182ms/step - loss: 0.1226 - acc: 0.9908 - val_loss: 0.4054 - val_acc: 0.9196Epoch 484/100091s 182ms/step - loss: 0.1206 - acc: 0.9917 - val_loss: 0.4020 - val_acc: 0.9218Epoch 485/100091s 182ms/step - loss: 0.1277 - acc: 0.9889 - val_loss: 0.3926 - val_acc: 0.9208 ETA: 3s - loss: 0.1276 - acc: 0.9889Epoch 486/100091s 182ms/step - loss: 0.1244 - acc: 0.9901 - val_loss: 0.3976 - val_acc: 0.9179 ETA: 46s - loss: 0.1215 - acc: 0.9913Epoch 487/100091s 182ms/step - loss: 0.1216 - acc: 0.9915 - val_loss: 0.4025 - val_acc: 0.9215Epoch 488/100091s 182ms/step - loss: 0.1231 - acc: 0.9908 - val_loss: 0.4037 - val_acc: 0.9223Epoch 489/100091s 182ms/step - loss: 0.1259 - acc: 0.9901 - val_loss: 0.4109 - val_acc: 0.9187Epoch 490/100091s 182ms/step - loss: 0.1247 - acc: 0.9908 - val_loss: 0.4085 - val_acc: 0.9206Epoch 491/100091s 182ms/step - loss: 0.1214 - acc: 0.9916 - val_loss: 0.4054 - val_acc: 0.9242 ETA: 7s - loss: 0.1211 - acc: 0.9916Epoch 492/100091s 181ms/step - loss: 0.1253 - acc: 0.9898 - val_loss: 0.4153 - val_acc: 0.9198Epoch 493/100091s 181ms/step - loss: 0.1203 - acc: 0.9919 - val_loss: 0.3971 - val_acc: 0.9212Epoch 494/100091s 181ms/step - loss: 0.1243 - acc: 0.9903 - val_loss: 0.4066 - val_acc: 0.9195Epoch 495/100091s 181ms/step - loss: 0.1256 - acc: 0.9904 - val_loss: 0.4061 - val_acc: 0.9199Epoch 496/100091s 181ms/step - loss: 0.1266 - acc: 0.9890 - val_loss: 0.3927 - val_acc: 0.9221Epoch 497/100090s 181ms/step - loss: 0.1256 - acc: 0.9897 - val_loss: 0.4104 - val_acc: 0.9210Epoch 498/100090s 181ms/step - loss: 0.1218 - acc: 0.9909 - val_loss: 0.4074 - val_acc: 0.9176Epoch 499/100090s 181ms/step - loss: 0.1225 - acc: 0.9907 - val_loss: 0.3931 - val_acc: 0.9219Epoch 500/100090s 181ms/step - loss: 0.1238 - acc: 0.9908 - val_loss: 0.3940 - val_acc: 0.9262Epoch 501/100090s 181ms/step - loss: 0.1217 - acc: 0.9912 - val_loss: 0.4017 - val_acc: 0.9238Epoch 502/100091s 181ms/step - loss: 0.1239 - acc: 0.9906 - val_loss: 0.4000 - val_acc: 0.9217Epoch 503/100091s 181ms/step - loss: 0.1219 - acc: 0.9915 - val_loss: 0.4070 - val_acc: 0.9199Epoch 504/100091s 182ms/step - loss: 0.1237 - acc: 0.9907 - val_loss: 0.4045 - val_acc: 0.9205Epoch 505/100091s 182ms/step - loss: 0.1291 - acc: 0.9884 - val_loss: 0.3828 - val_acc: 0.9203Epoch 506/100091s 182ms/step - loss: 0.1250 - acc: 0.9899 - val_loss: 0.4053 - val_acc: 0.9232Epoch 507/100091s 182ms/step - loss: 0.1248 - acc: 0.9907 - val_loss: 0.4098 - val_acc: 0.9204Epoch 508/100091s 182ms/step - loss: 0.1212 - acc: 0.9920 - val_loss: 0.3999 - val_acc: 0.9222Epoch 509/100091s 182ms/step - loss: 0.1223 - acc: 0.9918 - val_loss: 0.4083 - val_acc: 0.9183Epoch 510/100091s 182ms/step - loss: 0.1250 - acc: 0.9900 - val_loss: 0.3959 - val_acc: 0.9209Epoch 511/100091s 182ms/step - loss: 0.1190 - acc: 0.9919 - val_loss: 0.4029 - val_acc: 0.9237Epoch 512/100091s 182ms/step - loss: 0.1191 - acc: 0.9924 - val_loss: 0.4040 - val_acc: 0.9221Epoch 513/100091s 182ms/step - loss: 0.1229 - acc: 0.9906 - val_loss: 0.3949 - val_acc: 0.9251Epoch 514/100091s 182ms/step - loss: 0.1263 - acc: 0.9895 - val_loss: 0.4191 - val_acc: 0.9186Epoch 515/100091s 182ms/step - loss: 0.1240 - acc: 0.9904 - val_loss: 0.3939 - val_acc: 0.9208Epoch 516/100091s 181ms/step - loss: 0.1240 - acc: 0.9906 - val_loss: 0.3991 - val_acc: 0.9181Epoch 517/100091s 181ms/step - loss: 0.1209 - acc: 0.9915 - val_loss: 0.3953 - val_acc: 0.9216Epoch 518/100091s 182ms/step - loss: 0.1215 - acc: 0.9910 - val_loss: 0.4056 - val_acc: 0.9219Epoch 519/100091s 182ms/step - loss: 0.1232 - acc: 0.9905 - val_loss: 0.4092 - val_acc: 0.9187Epoch 520/100091s 182ms/step - loss: 0.1252 - acc: 0.9899 - val_loss: 0.4108 - val_acc: 0.9190Epoch 521/100091s 182ms/step - loss: 0.1215 - acc: 0.9912 - val_loss: 0.4031 - val_acc: 0.9191Epoch 522/100091s 182ms/step - loss: 0.1236 - acc: 0.9903 - val_loss: 0.3995 - val_acc: 0.9201Epoch 523/100091s 182ms/step - loss: 0.1226 - acc: 0.9916 - val_loss: 0.3823 - val_acc: 0.9264Epoch 524/100091s 182ms/step - loss: 0.1229 - acc: 0.9913 - val_loss: 0.3882 - val_acc: 0.9237本来想着早晨过来看看程序跑得怎么样了，却发现不知道为什么spyder自动退出了。
Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht, Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, IEEE Transactions on Industrial Electronics, 2020, DOI: 10.1109/TIE.2020.2972458, Date of Publication: 13 February 2020
https://ieeexplore.ieee.org/d...
————————————————
版权声明：本文为CSDN博主「dangqing1988」的原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。
原文链接：https://blog.csdn.net/dangqin...