【机器学习】使用python对数据集进行批处理

只输入一张图像数据过程和一次性处理100张图像数据过程中，数组形状变换如下图所示：

这些数组形状可以在代码中输出出来：

def get_data():(x_train, t_train), (x_test, t_test) = load_mnist(normalize=True, flatten=True, one_hot_label=False)return x_test, t_testdef init_network():with open("sample_weight.pkl", 'rb') as f:network = pickle.load(f)return networkx, t = get_data()
network = init_network()
print(x.shape)
print(x[0].shape)
W1, W2, W3 = network['W1'], network['W2'], network['W3']
print(W1.shape)
print(W2.shape)
print(W3.shape)

输出结果：

(10000, 784)
(784,)
(784, 50)
(50, 100)
(100, 10)

基于批处理的代码实现：

batch_size=100

for i in range(0, len(x), batch_size):这句话的意义，使i从0开始每次增加100 。

x_batch = x[i:i+batch_size]可以取出第i个到第i+100个之间的数据。

这样数据就变成了x[0:100]、x[100:200]、…这样的批数据。

p = np.argmax(y_batch, axis=1)，这句话获取y_batch取最大值时的y_batch数组的下标。 axis=1表示从行方向找最大值。也就是说，输入一个图片，输出一个y，找0-9下标里面y最大的那个下标，就是神经网络根据这个图片猜出来的数字。

import sys, os
sys.path.append(os.pardir)  # 为了导入父目录的文件而进行的设定
import numpy as np
import pickle
from dataset.mnist import load_mnist
from common.functions import sigmoid, softmaxdef get_data():(x_train, t_train), (x_test, t_test) = load_mnist(normalize=True, flatten=True, one_hot_label=False)return x_test, t_testdef init_network():with open("sample_weight.pkl", 'rb') as f:network = pickle.load(f)return networkdef predict(network, x):w1, w2, w3 = network['W1'], network['W2'], network['W3']b1, b2, b3 = network['b1'], network['b2'], network['b3']a1 = np.dot(x, w1) + b1z1 = sigmoid(a1)a2 = np.dot(z1, w2) + b2z2 = sigmoid(a2)a3 = np.dot(z2, w3) + b3y = softmax(a3)return yx, t = get_data()
network = init_network()
print(x.shape)
print(x[0].shape)
W1, W2, W3 = network['W1'], network['W2'], network['W3']
print(W1.shape)
print(W2.shape)
print(W3.shape)batch_size = 100 # 批数量
accuracy_cnt = 0for i in range(0, len(x), batch_size):x_batch = x[i:i+batch_size]y_batch = predict(network, x_batch)p = np.argmax(y_batch, axis=1)accuracy_cnt += np.sum(p == t[i:i+batch_size])print("Accuracy:" + str(float(accuracy_cnt) / len(x)))