Step 7: Load and Run a NN using GPU¶
In this step, you will learn how to use graphics processing units (GPUs) with MXNet. If you use GPUs to train and deploy neural networks, you may be able to train or perform inference quicker than with central processing units (CPUs).
Prerequisites¶
Before you start the steps, make sure you have at least one Nvidia GPU on your machine and make sure that you have CUDA properly installed. GPUs from AMD and Intel are not supported. Additionally, you will need to install the GPU-enabled version of MXNet. You can find information about how to install the GPU version of MXNet for your system here.
You can use the following command to view the number GPUs that are available to MXNet.
[1]:
from mxnet import np, npx, gluon, autograd
from mxnet.gluon import nn
import time
npx.set_np()
npx.num_gpus() #This command provides the number of GPUs MXNet can access
[1]:
1
Allocate data to a GPU¶
MXNet’s ndarray is very similar to NumPy’s. One major difference is that MXNet’s ndarray has a device attribute specifying which device an array is on. By default, arrays are stored on npx.cpu(). To change it to the first GPU, you can use the following code, npx.gpu() or npx.gpu(0) to indicate the first GPU.
[2]:
gpu = npx.gpu() if npx.num_gpus() > 0 else npx.cpu()
x = np.ones((3,4), device=gpu)
x
[04:21:32] /work/mxnet/src/storage/storage.cc:202: Using Pooled (Naive) StorageManager for GPU
[2]:
array([[1., 1., 1., 1.],
[1., 1., 1., 1.],
[1., 1., 1., 1.]], device=gpu(0))
If you’re using a CPU, MXNet allocates data on the main memory and tries to use as many CPU cores as possible. If there are multiple GPUs, MXNet will tell you which GPUs the ndarray is allocated on.
Assuming there is at least two GPUs. You can create another ndarray and assign it to a different GPU. If you only have one GPU, then you will get an error trying to run this code. In the example code here, you will copy x to the second GPU, npx.gpu(1):
[3]:
gpu_1 = npx.gpu(1) if npx.num_gpus() > 1 else npx.cpu()
x.copyto(gpu_1)
[04:21:32] /work/mxnet/src/storage/storage.cc:202: Using Pooled (Naive) StorageManager for CPU
[3]:
array([[1., 1., 1., 1.],
[1., 1., 1., 1.],
[1., 1., 1., 1.]])
MXNet requries that users explicitly move data between devices. But several operators such as print, and asnumpy, will implicitly move data to main memory.
Choosing GPU Ids¶
If you have multiple GPUs on your machine, MXNet can access each of them through 0-indexing with npx. As you saw before, the first GPU was accessed using npx.gpu(0), and the second using npx.gpu(1). This extends to however many GPUs your machine has. So if your machine has eight GPUs, the last GPU is accessed using npx.gpu(7). This allows you to select which GPUs to use for operations and training. You might find it particularly useful when you want to leverage multiple GPUs
while training neural networks.
Run an operation on a GPU¶
To perform an operation on a particular GPU, you only need to guarantee that the input of an operation is already on that GPU. The output is allocated on the same GPU as well. Almost all operators in the np and npx module support running on a GPU.
[4]:
y = np.random.uniform(size=(3,4), device=gpu)
x + y
[4]:
array([[1.7707204, 1.6089329, 1.3154027, 1.416174 ],
[1.2215444, 1.203621 , 1.5886476, 1.3864298],
[1.0913873, 1.1142501, 1.8820239, 1.1949301]], device=gpu(0))
Remember that if the inputs are not on the same GPU, you will get an error.
Run a neural network on a GPU¶
To run a neural network on a GPU, you only need to copy and move the input data and parameters to the GPU. To demonstrate this you can reuse the previously defined LeafNetwork in Training Neural Networks. The following code example shows this.
[5]:
# The convolutional block has a convolution layer, a max pool layer and a batch normalization layer
def conv_block(filters, kernel_size=2, stride=2, batch_norm=True):
conv_block = nn.HybridSequential()
conv_block.add(nn.Conv2D(channels=filters, kernel_size=kernel_size, activation='relu'),
nn.MaxPool2D(pool_size=4, strides=stride))
if batch_norm:
conv_block.add(nn.BatchNorm())
return conv_block
# The dense block consists of a dense layer and a dropout layer
def dense_block(neurons, activation='relu', dropout=0.2):
dense_block = nn.HybridSequential()
dense_block.add(nn.Dense(neurons, activation=activation))
if dropout:
dense_block.add(nn.Dropout(dropout))
return dense_block
# Create neural network blueprint using the blocks
class LeafNetwork(nn.HybridBlock):
def __init__(self):
super(LeafNetwork, self).__init__()
self.conv1 = conv_block(32)
self.conv2 = conv_block(64)
self.conv3 = conv_block(128)
self.flatten = nn.Flatten()
self.dense1 = dense_block(100)
self.dense2 = dense_block(10)
self.dense3 = nn.Dense(2)
def forward(self, batch):
batch = self.conv1(batch)
batch = self.conv2(batch)
batch = self.conv3(batch)
batch = self.flatten(batch)
batch = self.dense1(batch)
batch = self.dense2(batch)
batch = self.dense3(batch)
return batch
Load the saved parameters onto GPU 0 directly as shown below; additionally, you could use net.collect_params().reset_device(gpu) to change the device.
[6]:
net = LeafNetwork()
net.load_parameters('leaf_models.params', device=gpu)
Use the following command to create input data on GPU 0. The forward function will then run on GPU 0.
[7]:
x = np.random.uniform(size=(1, 3, 128, 128), device=gpu)
net(x)
[04:21:32] /work/mxnet/src/operator/cudnn_ops.cc:421: Auto-tuning cuDNN op, set MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable
[04:21:33] /work/mxnet/src/operator/cudnn_ops.cc:421: Auto-tuning cuDNN op, set MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable
[7]:
array([[ 5.52604 , -0.286441]], device=gpu(0))
Training with multiple GPUs¶
Finally, you will see how you can use multiple GPUs to jointly train a neural network through data parallelism. To elaborate on what data parallelism is, assume there are n GPUs, then you can split each data batch into n parts, and use a GPU on each of these parts to run the forward and backward passes on the seperate chunks of the data.
First copy the data definitions with the following commands, and the transform functions from the tutorial Training Neural Networks.
[8]:
# Import transforms as compose a series of transformations to the images
from mxnet.gluon.data.vision import transforms
jitter_param = 0.05
# mean and std for normalizing image value in range (0,1)
mean = [0.485, 0.456, 0.406]
std = [0.229, 0.224, 0.225]
training_transformer = transforms.Compose([
transforms.Resize(size=224, keep_ratio=True),
transforms.CenterCrop(128),
transforms.RandomFlipLeftRight(),
transforms.RandomColorJitter(contrast=jitter_param),
transforms.ToTensor(),
transforms.Normalize(mean, std)
])
validation_transformer = transforms.Compose([
transforms.Resize(size=224, keep_ratio=True),
transforms.CenterCrop(128),
transforms.ToTensor(),
transforms.Normalize(mean, std)
])
# Use ImageFolderDataset to create a Dataset object from directory structure
train_dataset = gluon.data.vision.ImageFolderDataset('./datasets/train')
val_dataset = gluon.data.vision.ImageFolderDataset('./datasets/validation')
test_dataset = gluon.data.vision.ImageFolderDataset('./datasets/test')
# Create data loaders
batch_size = 4
train_loader = gluon.data.DataLoader(train_dataset.transform_first(training_transformer),batch_size=batch_size, shuffle=True, try_nopython=True)
validation_loader = gluon.data.DataLoader(val_dataset.transform_first(validation_transformer), batch_size=batch_size, try_nopython=True)
test_loader = gluon.data.DataLoader(test_dataset.transform_first(validation_transformer), batch_size=batch_size, try_nopython=True)
Define a helper function¶
This is the same test function defined previously in the Step 6.
[9]:
# Function to return the accuracy for the validation and test set
def test(val_data, devices):
acc = gluon.metric.Accuracy()
for batch in val_data:
data, label = batch[0], batch[1]
data_list = gluon.utils.split_and_load(data, devices)
label_list = gluon.utils.split_and_load(label, devices)
outputs = [net(X) for X in data_list]
acc.update(label_list, outputs)
_, accuracy = acc.get()
return accuracy
The training loop is quite similar to that shown earlier. The major differences are highlighted in the following code.
[10]:
# Diff 1: Use two GPUs for training.
available_gpus = [npx.gpu(i) for i in range(npx.num_gpus())]
num_gpus = 2
devices = available_gpus[:num_gpus]
print('Using {} GPUs'.format(len(devices)))
# Diff 2: reinitialize the parameters and place them on multiple GPUs
net.initialize(force_reinit=True, device=devices)
# Loss and trainer are the same as before
loss_fn = gluon.loss.SoftmaxCrossEntropyLoss()
optimizer = 'sgd'
optimizer_params = {'learning_rate': 0.001}
trainer = gluon.Trainer(net.collect_params(), optimizer, optimizer_params)
epochs = 2
accuracy = gluon.metric.Accuracy()
log_interval = 5
for epoch in range(epochs):
train_loss = 0.
tic = time.time()
btic = time.time()
accuracy.reset()
for idx, batch in enumerate(train_loader):
data, label = batch[0], batch[1]
# Diff 3: split batch and load into corresponding devices
data_list = gluon.utils.split_and_load(data, devices)
label_list = gluon.utils.split_and_load(label, devices)
# Diff 4: run forward and backward on each devices.
# MXNet will automatically run them in parallel
with autograd.record():
outputs = [net(X)
for X in data_list]
losses = [loss_fn(output, label)
for output, label in zip(outputs, label_list)]
for l in losses:
l.backward()
trainer.step(batch_size)
# Diff 5: sum losses over all devices. Here, the float
# function will copy data into CPU.
train_loss += sum([float(l.sum()) for l in losses])
accuracy.update(label_list, outputs)
if log_interval and (idx + 1) % log_interval == 0:
_, acc = accuracy.get()
print(f"""Epoch[{epoch + 1}] Batch[{idx + 1}] Speed: {batch_size / (time.time() - btic)} samples/sec \
batch loss = {train_loss} | accuracy = {acc}""")
btic = time.time()
_, acc = accuracy.get()
acc_val = test(validation_loader, devices)
print(f"[Epoch {epoch + 1}] training: accuracy={acc}")
print(f"[Epoch {epoch + 1}] time cost: {time.time() - tic}")
print(f"[Epoch {epoch + 1}] validation: validation accuracy={acc_val}")
Using 1 GPUs
Epoch[1] Batch[5] Speed: 0.7173089899203501 samples/sec batch loss = 14.109387874603271 | accuracy = 0.4
Epoch[1] Batch[10] Speed: 1.2488635911932617 samples/sec batch loss = 27.856971979141235 | accuracy = 0.475
Epoch[1] Batch[15] Speed: 1.2547331219178495 samples/sec batch loss = 41.62733340263367 | accuracy = 0.5166666666666667
Epoch[1] Batch[20] Speed: 1.2574016512018573 samples/sec batch loss = 55.84801149368286 | accuracy = 0.4875
Epoch[1] Batch[25] Speed: 1.249793820569105 samples/sec batch loss = 69.66716718673706 | accuracy = 0.49
Epoch[1] Batch[30] Speed: 1.2504747636094988 samples/sec batch loss = 83.63934445381165 | accuracy = 0.48333333333333334
Epoch[1] Batch[35] Speed: 1.2540013241700803 samples/sec batch loss = 98.12923693656921 | accuracy = 0.4785714285714286
Epoch[1] Batch[40] Speed: 1.2433366882315402 samples/sec batch loss = 111.83499693870544 | accuracy = 0.48125
Epoch[1] Batch[45] Speed: 1.2445212875529668 samples/sec batch loss = 125.18878197669983 | accuracy = 0.5
Epoch[1] Batch[50] Speed: 1.245335507712452 samples/sec batch loss = 139.3703784942627 | accuracy = 0.495
Epoch[1] Batch[55] Speed: 1.2469224778275927 samples/sec batch loss = 153.2837598323822 | accuracy = 0.4954545454545455
Epoch[1] Batch[60] Speed: 1.2478066605108897 samples/sec batch loss = 167.36257028579712 | accuracy = 0.4875
Epoch[1] Batch[65] Speed: 1.2488965939290726 samples/sec batch loss = 181.38084363937378 | accuracy = 0.49230769230769234
Epoch[1] Batch[70] Speed: 1.2480735339955618 samples/sec batch loss = 194.63530731201172 | accuracy = 0.5071428571428571
Epoch[1] Batch[75] Speed: 1.2476920559844125 samples/sec batch loss = 208.3918468952179 | accuracy = 0.5166666666666667
Epoch[1] Batch[80] Speed: 1.2477543202853443 samples/sec batch loss = 221.81531071662903 | accuracy = 0.51875
Epoch[1] Batch[85] Speed: 1.2461767111831574 samples/sec batch loss = 235.0783576965332 | accuracy = 0.5323529411764706
Epoch[1] Batch[90] Speed: 1.251087407430828 samples/sec batch loss = 248.5095088481903 | accuracy = 0.5333333333333333
Epoch[1] Batch[95] Speed: 1.2485950788408764 samples/sec batch loss = 261.928329706192 | accuracy = 0.5447368421052632
Epoch[1] Batch[100] Speed: 1.2461065520652592 samples/sec batch loss = 275.105082988739 | accuracy = 0.5475
Epoch[1] Batch[105] Speed: 1.2466023705424898 samples/sec batch loss = 288.4014744758606 | accuracy = 0.5547619047619048
Epoch[1] Batch[110] Speed: 1.2527445482382205 samples/sec batch loss = 302.4630672931671 | accuracy = 0.5545454545454546
Epoch[1] Batch[115] Speed: 1.249511507393301 samples/sec batch loss = 315.87427735328674 | accuracy = 0.5543478260869565
Epoch[1] Batch[120] Speed: 1.2469830897237266 samples/sec batch loss = 330.0675938129425 | accuracy = 0.55625
Epoch[1] Batch[125] Speed: 1.2485631141384241 samples/sec batch loss = 344.2156593799591 | accuracy = 0.552
Epoch[1] Batch[130] Speed: 1.2492975072453407 samples/sec batch loss = 358.20309352874756 | accuracy = 0.551923076923077
Epoch[1] Batch[135] Speed: 1.2539363730217234 samples/sec batch loss = 371.67721605300903 | accuracy = 0.5555555555555556
Epoch[1] Batch[140] Speed: 1.254016133598948 samples/sec batch loss = 385.8363575935364 | accuracy = 0.5571428571428572
Epoch[1] Batch[145] Speed: 1.2494756805205474 samples/sec batch loss = 399.2117648124695 | accuracy = 0.5586206896551724
Epoch[1] Batch[150] Speed: 1.2451534305105048 samples/sec batch loss = 412.84080719947815 | accuracy = 0.56
Epoch[1] Batch[155] Speed: 1.2439571132825729 samples/sec batch loss = 426.0250413417816 | accuracy = 0.5629032258064516
Epoch[1] Batch[160] Speed: 1.250896556279965 samples/sec batch loss = 440.5059177875519 | accuracy = 0.5546875
Epoch[1] Batch[165] Speed: 1.2510462659691461 samples/sec batch loss = 454.378933429718 | accuracy = 0.55
Epoch[1] Batch[170] Speed: 1.2525483285410557 samples/sec batch loss = 468.0374982357025 | accuracy = 0.55
Epoch[1] Batch[175] Speed: 1.250540848040186 samples/sec batch loss = 481.37460684776306 | accuracy = 0.5514285714285714
Epoch[1] Batch[180] Speed: 1.2443146226482964 samples/sec batch loss = 494.53990387916565 | accuracy = 0.5541666666666667
Epoch[1] Batch[185] Speed: 1.254542657901756 samples/sec batch loss = 507.77793407440186 | accuracy = 0.5594594594594594
Epoch[1] Batch[190] Speed: 1.2550383602020694 samples/sec batch loss = 521.1585078239441 | accuracy = 0.5592105263157895
Epoch[1] Batch[195] Speed: 1.248124601062049 samples/sec batch loss = 533.4047560691833 | accuracy = 0.5666666666666667
Epoch[1] Batch[200] Speed: 1.2500795213800187 samples/sec batch loss = 545.6380257606506 | accuracy = 0.57375
Epoch[1] Batch[205] Speed: 1.2508536554361183 samples/sec batch loss = 558.0965132713318 | accuracy = 0.5780487804878048
Epoch[1] Batch[210] Speed: 1.2517014923235137 samples/sec batch loss = 570.9625518321991 | accuracy = 0.5785714285714286
Epoch[1] Batch[215] Speed: 1.2552960322845683 samples/sec batch loss = 583.5726790428162 | accuracy = 0.5837209302325581
Epoch[1] Batch[220] Speed: 1.2471114692340513 samples/sec batch loss = 597.2438080310822 | accuracy = 0.5852272727272727
Epoch[1] Batch[225] Speed: 1.2522912222957368 samples/sec batch loss = 610.7035535573959 | accuracy = 0.5855555555555556
Epoch[1] Batch[230] Speed: 1.257931683124963 samples/sec batch loss = 623.757507443428 | accuracy = 0.5869565217391305
Epoch[1] Batch[235] Speed: 1.2581769575322561 samples/sec batch loss = 636.8130487203598 | accuracy = 0.5851063829787234
Epoch[1] Batch[240] Speed: 1.2578736802786734 samples/sec batch loss = 649.8954354524612 | accuracy = 0.5854166666666667
Epoch[1] Batch[245] Speed: 1.2543922040718885 samples/sec batch loss = 663.0327650308609 | accuracy = 0.5867346938775511
Epoch[1] Batch[250] Speed: 1.248841094553405 samples/sec batch loss = 676.6164935827255 | accuracy = 0.588
Epoch[1] Batch[255] Speed: 1.2531124621277645 samples/sec batch loss = 690.2022820711136 | accuracy = 0.5852941176470589
Epoch[1] Batch[260] Speed: 1.2475061352092753 samples/sec batch loss = 702.0996137857437 | accuracy = 0.5884615384615385
Epoch[1] Batch[265] Speed: 1.2421387996904651 samples/sec batch loss = 714.8796883821487 | accuracy = 0.5915094339622642
Epoch[1] Batch[270] Speed: 1.244270787869061 samples/sec batch loss = 727.0544770956039 | accuracy = 0.5935185185185186
Epoch[1] Batch[275] Speed: 1.2443309576611774 samples/sec batch loss = 740.4612075090408 | accuracy = 0.5927272727272728
Epoch[1] Batch[280] Speed: 1.2476181078018296 samples/sec batch loss = 753.5070110559464 | accuracy = 0.5919642857142857
Epoch[1] Batch[285] Speed: 1.2465933858212954 samples/sec batch loss = 766.1015297174454 | accuracy = 0.5929824561403508
Epoch[1] Batch[290] Speed: 1.2449173639880178 samples/sec batch loss = 779.6410132646561 | accuracy = 0.593103448275862
Epoch[1] Batch[295] Speed: 1.2430146432604277 samples/sec batch loss = 792.6876677274704 | accuracy = 0.5949152542372881
Epoch[1] Batch[300] Speed: 1.2515361276164 samples/sec batch loss = 804.2858747243881 | accuracy = 0.5975
Epoch[1] Batch[305] Speed: 1.2583303026141537 samples/sec batch loss = 816.6289802789688 | accuracy = 0.5991803278688524
Epoch[1] Batch[310] Speed: 1.2502803723413254 samples/sec batch loss = 828.6005524396896 | accuracy = 0.6024193548387097
Epoch[1] Batch[315] Speed: 1.2525589890425457 samples/sec batch loss = 839.8461601734161 | accuracy = 0.6031746031746031
Epoch[1] Batch[320] Speed: 1.249410452970806 samples/sec batch loss = 852.4019663333893 | accuracy = 0.603125
Epoch[1] Batch[325] Speed: 1.2496238402843836 samples/sec batch loss = 868.4544899463654 | accuracy = 0.6
Epoch[1] Batch[330] Speed: 1.2523530116511297 samples/sec batch loss = 881.5271074771881 | accuracy = 0.6007575757575757
Epoch[1] Batch[335] Speed: 1.2546176168868743 samples/sec batch loss = 893.2346479892731 | accuracy = 0.6037313432835821
Epoch[1] Batch[340] Speed: 1.246955470728363 samples/sec batch loss = 906.9669165611267 | accuracy = 0.6044117647058823
Epoch[1] Batch[345] Speed: 1.2506402208528993 samples/sec batch loss = 919.7020897865295 | accuracy = 0.6057971014492753
Epoch[1] Batch[350] Speed: 1.2492334145122315 samples/sec batch loss = 933.1867297887802 | accuracy = 0.6057142857142858
Epoch[1] Batch[355] Speed: 1.2538524994592892 samples/sec batch loss = 947.3878728151321 | accuracy = 0.6035211267605634
Epoch[1] Batch[360] Speed: 1.2499582595160668 samples/sec batch loss = 960.3634434938431 | accuracy = 0.6041666666666666
Epoch[1] Batch[365] Speed: 1.2516683411814322 samples/sec batch loss = 972.6316422224045 | accuracy = 0.6047945205479452
Epoch[1] Batch[370] Speed: 1.2519167840129217 samples/sec batch loss = 984.2569028139114 | accuracy = 0.606081081081081
Epoch[1] Batch[375] Speed: 1.2528885254228845 samples/sec batch loss = 998.0109266042709 | accuracy = 0.606
Epoch[1] Batch[380] Speed: 1.2489419639171562 samples/sec batch loss = 1011.4422169923782 | accuracy = 0.6065789473684211
Epoch[1] Batch[385] Speed: 1.2500232648709093 samples/sec batch loss = 1025.3609365224838 | accuracy = 0.6051948051948052
Epoch[1] Batch[390] Speed: 1.251817768659265 samples/sec batch loss = 1038.6429842710495 | accuracy = 0.6051282051282051
Epoch[1] Batch[395] Speed: 1.2551697184272432 samples/sec batch loss = 1051.1454488039017 | accuracy = 0.6044303797468354
Epoch[1] Batch[400] Speed: 1.2505904392522167 samples/sec batch loss = 1063.5903500318527 | accuracy = 0.604375
Epoch[1] Batch[405] Speed: 1.2501570220818985 samples/sec batch loss = 1077.7765802145004 | accuracy = 0.6030864197530864
Epoch[1] Batch[410] Speed: 1.256009407709015 samples/sec batch loss = 1090.1866561174393 | accuracy = 0.6036585365853658
Epoch[1] Batch[415] Speed: 1.2470800439360186 samples/sec batch loss = 1102.3599358797073 | accuracy = 0.6042168674698796
Epoch[1] Batch[420] Speed: 1.2568073817848802 samples/sec batch loss = 1114.840255856514 | accuracy = 0.6053571428571428
Epoch[1] Batch[425] Speed: 1.247749958798082 samples/sec batch loss = 1130.673286318779 | accuracy = 0.6023529411764705
Epoch[1] Batch[430] Speed: 1.250030436362818 samples/sec batch loss = 1143.257992386818 | accuracy = 0.6034883720930233
Epoch[1] Batch[435] Speed: 1.2464936363295949 samples/sec batch loss = 1153.495651960373 | accuracy = 0.6063218390804598
Epoch[1] Batch[440] Speed: 1.2447175858443076 samples/sec batch loss = 1165.5605726242065 | accuracy = 0.6073863636363637
Epoch[1] Batch[445] Speed: 1.2548845022973882 samples/sec batch loss = 1177.6937124729156 | accuracy = 0.6067415730337079
Epoch[1] Batch[450] Speed: 1.253411855657703 samples/sec batch loss = 1192.1498646736145 | accuracy = 0.6066666666666667
Epoch[1] Batch[455] Speed: 1.2513164870329272 samples/sec batch loss = 1205.8402743339539 | accuracy = 0.6065934065934065
Epoch[1] Batch[460] Speed: 1.247389916487588 samples/sec batch loss = 1217.7081812620163 | accuracy = 0.6081521739130434
Epoch[1] Batch[465] Speed: 1.2509053233362604 samples/sec batch loss = 1230.8523312807083 | accuracy = 0.6069892473118279
Epoch[1] Batch[470] Speed: 1.2474988071322672 samples/sec batch loss = 1243.3499066829681 | accuracy = 0.6069148936170212
Epoch[1] Batch[475] Speed: 1.249558690331585 samples/sec batch loss = 1256.9940313100815 | accuracy = 0.6068421052631578
Epoch[1] Batch[480] Speed: 1.2498210999148223 samples/sec batch loss = 1269.0932389497757 | accuracy = 0.6078125
Epoch[1] Batch[485] Speed: 1.249798754970041 samples/sec batch loss = 1282.1142412424088 | accuracy = 0.6087628865979381
Epoch[1] Batch[490] Speed: 1.2474131955984435 samples/sec batch loss = 1296.9114240407944 | accuracy = 0.6071428571428571
Epoch[1] Batch[495] Speed: 1.2518477518440956 samples/sec batch loss = 1309.8559930324554 | accuracy = 0.6080808080808081
Epoch[1] Batch[500] Speed: 1.2459789345602494 samples/sec batch loss = 1323.4580008983612 | accuracy = 0.6085
Epoch[1] Batch[505] Speed: 1.255630017740401 samples/sec batch loss = 1336.2770595550537 | accuracy = 0.6084158415841584
Epoch[1] Batch[510] Speed: 1.255058170153684 samples/sec batch loss = 1351.4411013126373 | accuracy = 0.6058823529411764
Epoch[1] Batch[515] Speed: 1.2495108559772763 samples/sec batch loss = 1363.1846477985382 | accuracy = 0.6063106796116505
Epoch[1] Batch[520] Speed: 1.2527150833033232 samples/sec batch loss = 1375.6460919380188 | accuracy = 0.6076923076923076
Epoch[1] Batch[525] Speed: 1.25798063603048 samples/sec batch loss = 1386.192439198494 | accuracy = 0.61
Epoch[1] Batch[530] Speed: 1.248925228701789 samples/sec batch loss = 1397.0663534402847 | accuracy = 0.6113207547169811
Epoch[1] Batch[535] Speed: 1.259530798723834 samples/sec batch loss = 1409.4639719724655 | accuracy = 0.611214953271028
Epoch[1] Batch[540] Speed: 1.256628993998189 samples/sec batch loss = 1423.510990023613 | accuracy = 0.6101851851851852
Epoch[1] Batch[545] Speed: 1.2539642084042697 samples/sec batch loss = 1437.0432263612747 | accuracy = 0.6096330275229358
Epoch[1] Batch[550] Speed: 1.2498961475354633 samples/sec batch loss = 1450.1948345899582 | accuracy = 0.6086363636363636
Epoch[1] Batch[555] Speed: 1.2492934140440306 samples/sec batch loss = 1463.197319149971 | accuracy = 0.6072072072072072
Epoch[1] Batch[560] Speed: 1.2499930896246991 samples/sec batch loss = 1476.8904753923416 | accuracy = 0.6066964285714286
Epoch[1] Batch[565] Speed: 1.2524270547336498 samples/sec batch loss = 1489.2849966287613 | accuracy = 0.6079646017699115
Epoch[1] Batch[570] Speed: 1.2518859567991858 samples/sec batch loss = 1503.6431480646133 | accuracy = 0.6065789473684211
Epoch[1] Batch[575] Speed: 1.2527742951975238 samples/sec batch loss = 1515.261520266533 | accuracy = 0.6078260869565217
Epoch[1] Batch[580] Speed: 1.2509416053332658 samples/sec batch loss = 1526.9548864364624 | accuracy = 0.6090517241379311
Epoch[1] Batch[585] Speed: 1.2541969680625324 samples/sec batch loss = 1537.8622829914093 | accuracy = 0.6106837606837607
Epoch[1] Batch[590] Speed: 1.2524829668018274 samples/sec batch loss = 1548.5477073192596 | accuracy = 0.6122881355932204
Epoch[1] Batch[595] Speed: 1.2558429968697522 samples/sec batch loss = 1561.7891867160797 | accuracy = 0.6109243697478992
Epoch[1] Batch[600] Speed: 1.2572451406118823 samples/sec batch loss = 1575.6118171215057 | accuracy = 0.61125
Epoch[1] Batch[605] Speed: 1.2513090207948667 samples/sec batch loss = 1588.3178189992905 | accuracy = 0.6119834710743801
Epoch[1] Batch[610] Speed: 1.2537173882242802 samples/sec batch loss = 1599.6253324747086 | accuracy = 0.6131147540983607
Epoch[1] Batch[615] Speed: 1.2523310434813115 samples/sec batch loss = 1611.0392497777939 | accuracy = 0.6142276422764228
Epoch[1] Batch[620] Speed: 1.253102728154078 samples/sec batch loss = 1625.3391102552414 | accuracy = 0.6137096774193549
Epoch[1] Batch[625] Speed: 1.2540733125243213 samples/sec batch loss = 1636.2634762525558 | accuracy = 0.616
Epoch[1] Batch[630] Speed: 1.2508428374270442 samples/sec batch loss = 1648.5310891866684 | accuracy = 0.6158730158730159
Epoch[1] Batch[635] Speed: 1.253749523694156 samples/sec batch loss = 1659.243582367897 | accuracy = 0.6173228346456693
Epoch[1] Batch[640] Speed: 1.2542251900675794 samples/sec batch loss = 1672.3194509744644 | accuracy = 0.6171875
Epoch[1] Batch[645] Speed: 1.2545985713897927 samples/sec batch loss = 1683.6487039327621 | accuracy = 0.6186046511627907
Epoch[1] Batch[650] Speed: 1.2556925129521328 samples/sec batch loss = 1696.872985959053 | accuracy = 0.6180769230769231
Epoch[1] Batch[655] Speed: 1.2500393775588894 samples/sec batch loss = 1710.985644698143 | accuracy = 0.616793893129771
Epoch[1] Batch[660] Speed: 1.2501229280412516 samples/sec batch loss = 1724.8238681554794 | accuracy = 0.6162878787878788
Epoch[1] Batch[665] Speed: 1.249774735045506 samples/sec batch loss = 1739.9732121229172 | accuracy = 0.6150375939849624
Epoch[1] Batch[670] Speed: 1.250887136494731 samples/sec batch loss = 1753.7249166965485 | accuracy = 0.6149253731343284
Epoch[1] Batch[675] Speed: 1.2546571170269594 samples/sec batch loss = 1765.994879245758 | accuracy = 0.6159259259259259
Epoch[1] Batch[680] Speed: 1.2519335994966958 samples/sec batch loss = 1776.8267105817795 | accuracy = 0.6165441176470589
Epoch[1] Batch[685] Speed: 1.2471131378782268 samples/sec batch loss = 1789.9577577114105 | accuracy = 0.6167883211678832
Epoch[1] Batch[690] Speed: 1.2509222049290933 samples/sec batch loss = 1799.4462422132492 | accuracy = 0.6192028985507246
Epoch[1] Batch[695] Speed: 1.2527358489227634 samples/sec batch loss = 1810.1979557275772 | accuracy = 0.6201438848920864
Epoch[1] Batch[700] Speed: 1.2511445994904804 samples/sec batch loss = 1823.6723712682724 | accuracy = 0.6196428571428572
Epoch[1] Batch[705] Speed: 1.2504711287024335 samples/sec batch loss = 1836.2821657657623 | accuracy = 0.6205673758865248
Epoch[1] Batch[710] Speed: 1.2511894797924352 samples/sec batch loss = 1848.7483506202698 | accuracy = 0.6214788732394366
Epoch[1] Batch[715] Speed: 1.2487104999072243 samples/sec batch loss = 1862.0921961069107 | accuracy = 0.6202797202797202
Epoch[1] Batch[720] Speed: 1.251740715660875 samples/sec batch loss = 1873.925968170166 | accuracy = 0.6204861111111111
Epoch[1] Batch[725] Speed: 1.254980248359765 samples/sec batch loss = 1887.4272947311401 | accuracy = 0.6206896551724138
Epoch[1] Batch[730] Speed: 1.2538383498303711 samples/sec batch loss = 1897.7921460866928 | accuracy = 0.6222602739726028
Epoch[1] Batch[735] Speed: 1.2528853442828891 samples/sec batch loss = 1911.5076615810394 | accuracy = 0.6221088435374149
Epoch[1] Batch[740] Speed: 1.2564302383524641 samples/sec batch loss = 1924.5269718170166 | accuracy = 0.6222972972972973
Epoch[1] Batch[745] Speed: 1.2524705310773974 samples/sec batch loss = 1935.5363252162933 | accuracy = 0.6228187919463087
Epoch[1] Batch[750] Speed: 1.2544477289644191 samples/sec batch loss = 1947.629676580429 | accuracy = 0.6233333333333333
Epoch[1] Batch[755] Speed: 1.2525294393261557 samples/sec batch loss = 1960.0555703639984 | accuracy = 0.6235099337748344
Epoch[1] Batch[760] Speed: 1.2522337384312432 samples/sec batch loss = 1971.902447938919 | accuracy = 0.6236842105263158
Epoch[1] Batch[765] Speed: 1.2590799191383462 samples/sec batch loss = 1981.1101183891296 | accuracy = 0.6258169934640523
Epoch[1] Batch[770] Speed: 1.2517838641712986 samples/sec batch loss = 1993.2993923425674 | accuracy = 0.6266233766233766
Epoch[1] Batch[775] Speed: 1.249143379517365 samples/sec batch loss = 2005.0542666912079 | accuracy = 0.6274193548387097
Epoch[1] Batch[780] Speed: 1.2507209612379153 samples/sec batch loss = 2017.8336730003357 | accuracy = 0.6278846153846154
Epoch[1] Batch[785] Speed: 1.2525516014827356 samples/sec batch loss = 2031.7250916957855 | accuracy = 0.6270700636942675
[Epoch 1] training: accuracy=0.6272208121827412
[Epoch 1] time cost: 650.7169613838196
[Epoch 1] validation: validation accuracy=0.7366666666666667
Epoch[2] Batch[5] Speed: 1.2340196984579255 samples/sec batch loss = 13.191937804222107 | accuracy = 0.6
Epoch[2] Batch[10] Speed: 1.2310370853126993 samples/sec batch loss = 26.464403986930847 | accuracy = 0.6
Epoch[2] Batch[15] Speed: 1.2326619696754513 samples/sec batch loss = 41.108960032463074 | accuracy = 0.6
Epoch[2] Batch[20] Speed: 1.2320190106082836 samples/sec batch loss = 55.04953134059906 | accuracy = 0.5875
Epoch[2] Batch[25] Speed: 1.235256537546213 samples/sec batch loss = 65.04635787010193 | accuracy = 0.66
Epoch[2] Batch[30] Speed: 1.2322364535608838 samples/sec batch loss = 75.13800632953644 | accuracy = 0.7
Epoch[2] Batch[35] Speed: 1.233094214476106 samples/sec batch loss = 85.9286288022995 | accuracy = 0.7071428571428572
Epoch[2] Batch[40] Speed: 1.2300262424703376 samples/sec batch loss = 96.62003815174103 | accuracy = 0.70625
Epoch[2] Batch[45] Speed: 1.2289717740089405 samples/sec batch loss = 106.36261355876923 | accuracy = 0.7222222222222222
Epoch[2] Batch[50] Speed: 1.2303652312370434 samples/sec batch loss = 117.58973848819733 | accuracy = 0.725
Epoch[2] Batch[55] Speed: 1.2362896637213086 samples/sec batch loss = 131.9102178812027 | accuracy = 0.7136363636363636
Epoch[2] Batch[60] Speed: 1.2333847524966035 samples/sec batch loss = 143.2937877178192 | accuracy = 0.7125
Epoch[2] Batch[65] Speed: 1.236500870669069 samples/sec batch loss = 153.73799800872803 | accuracy = 0.7192307692307692
Epoch[2] Batch[70] Speed: 1.23287393197801 samples/sec batch loss = 165.10248506069183 | accuracy = 0.7142857142857143
Epoch[2] Batch[75] Speed: 1.231276952872063 samples/sec batch loss = 179.1421970129013 | accuracy = 0.71
Epoch[2] Batch[80] Speed: 1.230124636409622 samples/sec batch loss = 189.26601195335388 | accuracy = 0.709375
Epoch[2] Batch[85] Speed: 1.2319504365338176 samples/sec batch loss = 198.50052571296692 | accuracy = 0.7147058823529412
Epoch[2] Batch[90] Speed: 1.2315103148763311 samples/sec batch loss = 210.53342199325562 | accuracy = 0.7111111111111111
Epoch[2] Batch[95] Speed: 1.235229981223303 samples/sec batch loss = 222.21577048301697 | accuracy = 0.7105263157894737
Epoch[2] Batch[100] Speed: 1.232857986979065 samples/sec batch loss = 235.56106781959534 | accuracy = 0.705
Epoch[2] Batch[105] Speed: 1.228153899896607 samples/sec batch loss = 246.29658603668213 | accuracy = 0.7047619047619048
Epoch[2] Batch[110] Speed: 1.2345419159369972 samples/sec batch loss = 256.8897407054901 | accuracy = 0.7045454545454546
Epoch[2] Batch[115] Speed: 1.230914703045376 samples/sec batch loss = 268.92195451259613 | accuracy = 0.7065217391304348
Epoch[2] Batch[120] Speed: 1.2312387304247892 samples/sec batch loss = 281.6716022491455 | accuracy = 0.70625
Epoch[2] Batch[125] Speed: 1.230405474878182 samples/sec batch loss = 290.8304713964462 | accuracy = 0.71
Epoch[2] Batch[130] Speed: 1.2320699485360205 samples/sec batch loss = 301.62862718105316 | accuracy = 0.7115384615384616
Epoch[2] Batch[135] Speed: 1.2331189569992693 samples/sec batch loss = 312.6072155237198 | accuracy = 0.7074074074074074
Epoch[2] Batch[140] Speed: 1.230335817146746 samples/sec batch loss = 323.0715608596802 | accuracy = 0.7107142857142857
Epoch[2] Batch[145] Speed: 1.2294742297399517 samples/sec batch loss = 333.3850485086441 | accuracy = 0.7120689655172414
Epoch[2] Batch[150] Speed: 1.228943416736621 samples/sec batch loss = 344.21285820007324 | accuracy = 0.7133333333333334
Epoch[2] Batch[155] Speed: 1.230386615983145 samples/sec batch loss = 351.91108351945877 | accuracy = 0.7161290322580646
Epoch[2] Batch[160] Speed: 1.2290947606893128 samples/sec batch loss = 363.82177871465683 | accuracy = 0.7140625
Epoch[2] Batch[165] Speed: 1.2334957460494567 samples/sec batch loss = 375.11759382486343 | accuracy = 0.7151515151515152
Epoch[2] Batch[170] Speed: 1.2350278450031669 samples/sec batch loss = 383.88711553812027 | accuracy = 0.7176470588235294
Epoch[2] Batch[175] Speed: 1.2359369322991975 samples/sec batch loss = 397.7323417067528 | accuracy = 0.7157142857142857
Epoch[2] Batch[180] Speed: 1.2326006591320082 samples/sec batch loss = 408.222943007946 | accuracy = 0.7180555555555556
Epoch[2] Batch[185] Speed: 1.2286690039663009 samples/sec batch loss = 419.9541248679161 | accuracy = 0.7162162162162162
Epoch[2] Batch[190] Speed: 1.2348820352113132 samples/sec batch loss = 436.4125308394432 | accuracy = 0.7118421052631579
Epoch[2] Batch[195] Speed: 1.229741250438049 samples/sec batch loss = 445.3180300593376 | accuracy = 0.7153846153846154
Epoch[2] Batch[200] Speed: 1.2276386885677262 samples/sec batch loss = 458.6773495078087 | accuracy = 0.71375
Epoch[2] Batch[205] Speed: 1.2322305708230772 samples/sec batch loss = 474.16221684217453 | accuracy = 0.7060975609756097
Epoch[2] Batch[210] Speed: 1.236188641564287 samples/sec batch loss = 486.45931726694107 | accuracy = 0.705952380952381
Epoch[2] Batch[215] Speed: 1.2269664998651428 samples/sec batch loss = 497.142437517643 | accuracy = 0.7069767441860465
Epoch[2] Batch[220] Speed: 1.227698877596039 samples/sec batch loss = 511.4827534556389 | accuracy = 0.7034090909090909
Epoch[2] Batch[225] Speed: 1.2295424383087572 samples/sec batch loss = 521.8850936293602 | accuracy = 0.7044444444444444
Epoch[2] Batch[230] Speed: 1.2296271464053932 samples/sec batch loss = 534.2236570715904 | accuracy = 0.7043478260869566
Epoch[2] Batch[235] Speed: 1.228783470222361 samples/sec batch loss = 547.6792865395546 | accuracy = 0.7031914893617022
Epoch[2] Batch[240] Speed: 1.226652788615943 samples/sec batch loss = 560.1972559094429 | accuracy = 0.7020833333333333
Epoch[2] Batch[245] Speed: 1.2333460364060655 samples/sec batch loss = 570.8941144943237 | accuracy = 0.7010204081632653
Epoch[2] Batch[250] Speed: 1.2323818202773085 samples/sec batch loss = 581.0161567926407 | accuracy = 0.702
Epoch[2] Batch[255] Speed: 1.2315246882396889 samples/sec batch loss = 590.5841120481491 | accuracy = 0.703921568627451
Epoch[2] Batch[260] Speed: 1.2313567487492096 samples/sec batch loss = 603.2001624107361 | accuracy = 0.7038461538461539
Epoch[2] Batch[265] Speed: 1.2319485368330394 samples/sec batch loss = 614.1992572546005 | accuracy = 0.7037735849056603
Epoch[2] Batch[270] Speed: 1.228155608101228 samples/sec batch loss = 626.8009022474289 | accuracy = 0.7027777777777777
Epoch[2] Batch[275] Speed: 1.2277637446187692 samples/sec batch loss = 637.8115049004555 | accuracy = 0.7027272727272728
Epoch[2] Batch[280] Speed: 1.2313637980328085 samples/sec batch loss = 653.7936922907829 | accuracy = 0.7
Epoch[2] Batch[285] Speed: 1.2298324767914899 samples/sec batch loss = 665.3686112761497 | accuracy = 0.7008771929824561
Epoch[2] Batch[290] Speed: 1.2271632231151068 samples/sec batch loss = 674.8924590945244 | accuracy = 0.7043103448275863
Epoch[2] Batch[295] Speed: 1.2271321668035655 samples/sec batch loss = 685.4287278056145 | accuracy = 0.7025423728813559
Epoch[2] Batch[300] Speed: 1.229344050771688 samples/sec batch loss = 696.8948901295662 | accuracy = 0.7033333333333334
Epoch[2] Batch[305] Speed: 1.2305269436237125 samples/sec batch loss = 709.539255797863 | accuracy = 0.7032786885245902
Epoch[2] Batch[310] Speed: 1.2277334664989021 samples/sec batch loss = 720.9653010964394 | accuracy = 0.7024193548387097
Epoch[2] Batch[315] Speed: 1.2320834301489585 samples/sec batch loss = 735.4295864701271 | accuracy = 0.7
Epoch[2] Batch[320] Speed: 1.2337472783931034 samples/sec batch loss = 747.0269188284874 | accuracy = 0.69921875
Epoch[2] Batch[325] Speed: 1.2303642387141367 samples/sec batch loss = 758.9655331969261 | accuracy = 0.6976923076923077
Epoch[2] Batch[330] Speed: 1.2318759907398658 samples/sec batch loss = 770.8926784396172 | accuracy = 0.696969696969697
Epoch[2] Batch[335] Speed: 1.2302050046415316 samples/sec batch loss = 781.7621143460274 | accuracy = 0.6977611940298507
Epoch[2] Batch[340] Speed: 1.2321256865118086 samples/sec batch loss = 793.5896945595741 | accuracy = 0.6970588235294117
Epoch[2] Batch[345] Speed: 1.232044162380111 samples/sec batch loss = 804.6628307700157 | accuracy = 0.696376811594203
Epoch[2] Batch[350] Speed: 1.2317640223959818 samples/sec batch loss = 817.6521719098091 | accuracy = 0.6957142857142857
Epoch[2] Batch[355] Speed: 1.2342185993178596 samples/sec batch loss = 828.8623519539833 | accuracy = 0.6957746478873239
Epoch[2] Batch[360] Speed: 1.2368537417348167 samples/sec batch loss = 838.1571943163872 | accuracy = 0.6979166666666666
Epoch[2] Batch[365] Speed: 1.229889545215764 samples/sec batch loss = 850.3256699442863 | accuracy = 0.6972602739726027
Epoch[2] Batch[370] Speed: 1.22365342601773 samples/sec batch loss = 862.8044571280479 | accuracy = 0.6959459459459459
Epoch[2] Batch[375] Speed: 1.2250326060241592 samples/sec batch loss = 874.7381972670555 | accuracy = 0.696
Epoch[2] Batch[380] Speed: 1.2292805478547912 samples/sec batch loss = 885.6196042895317 | accuracy = 0.6953947368421053
Epoch[2] Batch[385] Speed: 1.2262928060364084 samples/sec batch loss = 897.9612233042717 | accuracy = 0.6948051948051948
Epoch[2] Batch[390] Speed: 1.2257007963847661 samples/sec batch loss = 910.6947954297066 | accuracy = 0.6935897435897436
Epoch[2] Batch[395] Speed: 1.2271841376441512 samples/sec batch loss = 920.7800042033195 | accuracy = 0.6955696202531646
Epoch[2] Batch[400] Speed: 1.228623565336638 samples/sec batch loss = 932.4678912758827 | accuracy = 0.694375
Epoch[2] Batch[405] Speed: 1.228689159998872 samples/sec batch loss = 944.510540664196 | accuracy = 0.6938271604938272
Epoch[2] Batch[410] Speed: 1.2343303786888913 samples/sec batch loss = 956.4577099680901 | accuracy = 0.6932926829268292
Epoch[2] Batch[415] Speed: 1.2211435847516017 samples/sec batch loss = 968.2406663298607 | accuracy = 0.6933734939759036
Epoch[2] Batch[420] Speed: 1.223634684290708 samples/sec batch loss = 979.0851874947548 | accuracy = 0.694047619047619
Epoch[2] Batch[425] Speed: 1.2231536646748846 samples/sec batch loss = 990.9242896437645 | accuracy = 0.6935294117647058
Epoch[2] Batch[430] Speed: 1.222433289278115 samples/sec batch loss = 1001.2824978232384 | accuracy = 0.6947674418604651
Epoch[2] Batch[435] Speed: 1.2288869761558563 samples/sec batch loss = 1013.8018770813942 | accuracy = 0.6948275862068966
Epoch[2] Batch[440] Speed: 1.2258045001065998 samples/sec batch loss = 1025.6744262576103 | accuracy = 0.6943181818181818
Epoch[2] Batch[445] Speed: 1.2294223350267197 samples/sec batch loss = 1039.92163926363 | accuracy = 0.6932584269662921
Epoch[2] Batch[450] Speed: 1.2305351567149998 samples/sec batch loss = 1052.8086115717888 | accuracy = 0.6933333333333334
Epoch[2] Batch[455] Speed: 1.226263227823574 samples/sec batch loss = 1063.562524497509 | accuracy = 0.6928571428571428
Epoch[2] Batch[460] Speed: 1.2290582942821577 samples/sec batch loss = 1074.8319408297539 | accuracy = 0.6929347826086957
Epoch[2] Batch[465] Speed: 1.2308020064374732 samples/sec batch loss = 1086.642512023449 | accuracy = 0.6924731182795699
Epoch[2] Batch[470] Speed: 1.2269165213879796 samples/sec batch loss = 1096.9285895228386 | accuracy = 0.6925531914893617
Epoch[2] Batch[475] Speed: 1.2326932159469668 samples/sec batch loss = 1112.122159421444 | accuracy = 0.6915789473684211
Epoch[2] Batch[480] Speed: 1.2319340631147153 samples/sec batch loss = 1125.8898395895958 | accuracy = 0.6916666666666667
Epoch[2] Batch[485] Speed: 1.227795192318318 samples/sec batch loss = 1135.0219516158104 | accuracy = 0.6938144329896907
Epoch[2] Batch[490] Speed: 1.2330217146314293 samples/sec batch loss = 1143.4660302996635 | accuracy = 0.6948979591836735
Epoch[2] Batch[495] Speed: 1.2283184487878263 samples/sec batch loss = 1153.8384173512459 | accuracy = 0.6944444444444444
Epoch[2] Batch[500] Speed: 1.231757058962938 samples/sec batch loss = 1166.0878913998604 | accuracy = 0.6945
Epoch[2] Batch[505] Speed: 1.2287922900483754 samples/sec batch loss = 1178.929158270359 | accuracy = 0.694059405940594
Epoch[2] Batch[510] Speed: 1.228906599246756 samples/sec batch loss = 1190.3485252261162 | accuracy = 0.6936274509803921
Epoch[2] Batch[515] Speed: 1.2278663599174895 samples/sec batch loss = 1202.1567012667656 | accuracy = 0.6941747572815534
Epoch[2] Batch[520] Speed: 1.2290040038628482 samples/sec batch loss = 1212.99599558115 | accuracy = 0.6942307692307692
Epoch[2] Batch[525] Speed: 1.2229375431525127 samples/sec batch loss = 1221.1800859570503 | accuracy = 0.6957142857142857
Epoch[2] Batch[530] Speed: 1.2294278306135973 samples/sec batch loss = 1234.8584129214287 | accuracy = 0.6943396226415094
Epoch[2] Batch[535] Speed: 1.2295529811274073 samples/sec batch loss = 1245.7880756258965 | accuracy = 0.6939252336448598
Epoch[2] Batch[540] Speed: 1.2256907672490176 samples/sec batch loss = 1255.1600200533867 | accuracy = 0.6949074074074074
Epoch[2] Batch[545] Speed: 1.2301580091479405 samples/sec batch loss = 1262.4292892813683 | accuracy = 0.6972477064220184
Epoch[2] Batch[550] Speed: 1.2303730812473594 samples/sec batch loss = 1272.4460882544518 | accuracy = 0.6981818181818182
Epoch[2] Batch[555] Speed: 1.2239685514824734 samples/sec batch loss = 1285.0484514832497 | accuracy = 0.6981981981981982
Epoch[2] Batch[560] Speed: 1.2348070529339519 samples/sec batch loss = 1294.4679728150368 | accuracy = 0.7
Epoch[2] Batch[565] Speed: 1.2309475767769544 samples/sec batch loss = 1303.1254943013191 | accuracy = 0.7013274336283186
Epoch[2] Batch[570] Speed: 1.230075482565809 samples/sec batch loss = 1316.4713049530983 | accuracy = 0.7017543859649122
Epoch[2] Batch[575] Speed: 1.227909316072026 samples/sec batch loss = 1324.9961784482002 | accuracy = 0.7030434782608695
Epoch[2] Batch[580] Speed: 1.2274785426314483 samples/sec batch loss = 1336.1549338698387 | accuracy = 0.7030172413793103
Epoch[2] Batch[585] Speed: 1.226800159993097 samples/sec batch loss = 1348.2013884186745 | accuracy = 0.7029914529914529
Epoch[2] Batch[590] Speed: 1.2298689891540981 samples/sec batch loss = 1358.4156247973442 | accuracy = 0.7038135593220339
Epoch[2] Batch[595] Speed: 1.2273475288875058 samples/sec batch loss = 1370.7123524546623 | accuracy = 0.7033613445378152
Epoch[2] Batch[600] Speed: 1.2289090296731893 samples/sec batch loss = 1381.9451199173927 | accuracy = 0.7041666666666667
Epoch[2] Batch[605] Speed: 1.23099020686782 samples/sec batch loss = 1394.8828805088997 | accuracy = 0.702892561983471
Epoch[2] Batch[610] Speed: 1.22789475737333 samples/sec batch loss = 1405.62037473917 | accuracy = 0.7024590163934427
Epoch[2] Batch[615] Speed: 1.2283783447381098 samples/sec batch loss = 1414.0183553099632 | accuracy = 0.7028455284552846
Epoch[2] Batch[620] Speed: 1.2275000965774134 samples/sec batch loss = 1424.5221813321114 | accuracy = 0.7032258064516129
Epoch[2] Batch[625] Speed: 1.2284641518170887 samples/sec batch loss = 1438.8212504982948 | accuracy = 0.7028
Epoch[2] Batch[630] Speed: 1.2223265039260094 samples/sec batch loss = 1452.665615260601 | accuracy = 0.7027777777777777
Epoch[2] Batch[635] Speed: 1.2281521017916135 samples/sec batch loss = 1463.2709195017815 | accuracy = 0.7031496062992126
Epoch[2] Batch[640] Speed: 1.2262747900459958 samples/sec batch loss = 1474.8328509926796 | accuracy = 0.703515625
Epoch[2] Batch[645] Speed: 1.2305037490018833 samples/sec batch loss = 1482.5473827123642 | accuracy = 0.7046511627906977
Epoch[2] Batch[650] Speed: 1.2268855672815617 samples/sec batch loss = 1496.133915424347 | accuracy = 0.7038461538461539
Epoch[2] Batch[655] Speed: 1.228560586655208 samples/sec batch loss = 1508.042652606964 | accuracy = 0.7034351145038168
Epoch[2] Batch[660] Speed: 1.2305761335556562 samples/sec batch loss = 1518.958142399788 | accuracy = 0.703030303030303
Epoch[2] Batch[665] Speed: 1.23144324361659 samples/sec batch loss = 1528.3985501527786 | accuracy = 0.7041353383458646
Epoch[2] Batch[670] Speed: 1.2258400571491157 samples/sec batch loss = 1539.1386450529099 | accuracy = 0.7044776119402985
Epoch[2] Batch[675] Speed: 1.2295986687642204 samples/sec batch loss = 1549.0479488372803 | accuracy = 0.7051851851851851
Epoch[2] Batch[680] Speed: 1.2276081471389826 samples/sec batch loss = 1557.7756605148315 | accuracy = 0.7066176470588236
Epoch[2] Batch[685] Speed: 1.2255831434630442 samples/sec batch loss = 1568.49676913023 | accuracy = 0.7065693430656934
Epoch[2] Batch[690] Speed: 1.2269875871585485 samples/sec batch loss = 1576.7246295809746 | accuracy = 0.7076086956521739
Epoch[2] Batch[695] Speed: 1.2281175791967158 samples/sec batch loss = 1586.2875626683235 | accuracy = 0.7079136690647482
Epoch[2] Batch[700] Speed: 1.2299273232787562 samples/sec batch loss = 1603.1909262537956 | accuracy = 0.7075
Epoch[2] Batch[705] Speed: 1.2270075983242643 samples/sec batch loss = 1615.0498873591423 | accuracy = 0.7074468085106383
Epoch[2] Batch[710] Speed: 1.2273998771518544 samples/sec batch loss = 1626.8721486926079 | accuracy = 0.7073943661971831
Epoch[2] Batch[715] Speed: 1.2253525591257681 samples/sec batch loss = 1638.3502686619759 | accuracy = 0.7076923076923077
Epoch[2] Batch[720] Speed: 1.2282458799898357 samples/sec batch loss = 1647.9793664813042 | accuracy = 0.7086805555555555
Epoch[2] Batch[725] Speed: 1.2295865030854698 samples/sec batch loss = 1657.6263877749443 | accuracy = 0.7089655172413794
Epoch[2] Batch[730] Speed: 1.2310131488591025 samples/sec batch loss = 1668.8062285780907 | accuracy = 0.7095890410958904
Epoch[2] Batch[735] Speed: 1.2307631814565543 samples/sec batch loss = 1676.3949342370033 | accuracy = 0.7105442176870749
Epoch[2] Batch[740] Speed: 1.2271674418682899 samples/sec batch loss = 1688.0925043225288 | accuracy = 0.7097972972972973
Epoch[2] Batch[745] Speed: 1.223377445182976 samples/sec batch loss = 1696.91601729393 | accuracy = 0.710738255033557
Epoch[2] Batch[750] Speed: 1.225483147285371 samples/sec batch loss = 1709.0300940275192 | accuracy = 0.7103333333333334
Epoch[2] Batch[755] Speed: 1.2295042333696065 samples/sec batch loss = 1716.6224944591522 | accuracy = 0.7112582781456953
Epoch[2] Batch[760] Speed: 1.2277638344670445 samples/sec batch loss = 1727.1613991260529 | accuracy = 0.7115131578947368
Epoch[2] Batch[765] Speed: 1.2299555456373832 samples/sec batch loss = 1739.5258980989456 | accuracy = 0.711437908496732
Epoch[2] Batch[770] Speed: 1.2265002521395307 samples/sec batch loss = 1748.1015099287033 | accuracy = 0.7123376623376624
Epoch[2] Batch[775] Speed: 1.225936886655823 samples/sec batch loss = 1758.5067769289017 | accuracy = 0.7129032258064516
Epoch[2] Batch[780] Speed: 1.2259969985127022 samples/sec batch loss = 1768.2216209173203 | accuracy = 0.7141025641025641
Epoch[2] Batch[785] Speed: 1.2287310938641942 samples/sec batch loss = 1779.2798628807068 | accuracy = 0.7136942675159236
[Epoch 2] training: accuracy=0.7131979695431472
[Epoch 2] time cost: 657.6945281028748
[Epoch 2] validation: validation accuracy=0.7822222222222223
Did this page help you?
Yes
No
Thanks for your feedback!