待整理

How to Check-Point Deep Learning Models in Keras

---

LossWise
Tensorboard 中文社区

---

谷歌发布TensorBoard API，让你自定义机器学习中的可视化

---

查找tensorflow安装的位置

---

pip show tensorflow-gpu

Name: tensorflow-gpu
Version: 1.0.1
Summary: TensorFlow helps the tensors flow
Home-page: http://tensorflow.org/
Author: Google Inc.
Author-email: opensource@google.com
License: Apache 2.0
Location: /home/bids/.local/lib/python2.7/site-packages
Requires: mock, numpy, protobuf, wheel, six

或者

python
import tensorflow
tensorflow.__path__
dir(tensorflow)

终止某个程序的进程

pkill -f "tensorboard"

---

什么是 TensorBoard

---

TensorBoard 是 TensorFlow 上一个非常酷的功能，神经网络很多时候就像是个黑盒子，里面到底是什么样，是什么样的结构，是怎么训练的，可能很难搞清楚。而 TensorBoard 的作用就是可以把复杂的神经网络训练过程给可视化，可以更好地理解，调试并优化程序。
TensorBoard可以将训练过程中的各种绘制数据展示出来，包括标量（scalars），图片（images），音频（Audio）,计算图（graph）,数据分布，直方图（histograms）和嵌入式向量。

在 scalars 下可以看到 accuracy，cross entropy，dropout，layer1 和 layer2 的 bias 和 weights 等的趋势。
在 images 和 audio 下可以看到输入的数据。展示训练过程中记录的图像和音频。
在 graphs 中可以看到模型的结构。
在 histogram 可以看到 activations，gradients 或者 weights 等变量的每一步的分布，越靠前面就是越新的步数的结果。展示训练过程中记录的数据的分布图
distribution 和 histogram 是两种不同的形式，可以看到整体的状况。
在 embedding 中可以看到用 PCA 主成分分析方法将高维数据投影到 3D 空间后的数据的关系。
Event: 展示训练过程中的统计数据（最值，均值等）变化情况

使用TensorBoard展示数据，需要在执行Tensorflow就算图的过程中，将各种类型的数据汇总并记录到日志文件中。然后使用TensorBoard读取这些日志文件，解析数据并生产数据可视化的Web页面，让我们可以在浏览器中观察各种汇总数据。

---

汇总数据的日志－－callbacks回调执行结果

---

log_filepath = '/tmp/keras_log' model.compile(loss='categorical_crossentropy', optimizer=SGD(lr=0.001), metrics=['accuracy'])    
tb_cb = keras.callbacks.TensorBoard(log_dir=log_filepath, write_images=1, histogram_freq=1)  
# 设置log的存储位置，将网络权值以图片格式保持在tensorboard中显示，设置每一个周期计算一次网络的  
#权值，每层输出值的分布直方图  
cbks = [tb_cb]  
history = model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch,   
verbose=1, callbacks=cbks, validation_data=(X_test, Y_test))  

tensorboard　默认的slcar一栏只记录了训练集和验证集上的loss，如何想记录展示其他指标，在model.compile的metric中进行添加，例如：

model.compile(  loss = 'mean_squared_error',  optimizer = 'sgd',  metrics= c('mae', 'acc')  # 可视化mae和acc  )  

而在keras的call back模块中,tensorborad class类实现源码可以看出，keras默认将模型的所有层的所有weights, bias以及每一层输出的distribution, histogram等传送到tensorborad，方便在浏览器中观察网络的运行情况。实现源码如下：

    def set_model(self, model):  self.model = model  self.sess = K.get_session()  if self.histogram_freq and self.merged is None:  for layer in self.model.layers:  for weight in layer.weights:  tf.summary.histogram(weight.name, weight)  if self.write_images:  w_img = tf.squeeze(weight)  shape = w_img.get_shape()  if len(shape) > 1 and shape[0] > shape[1]:  w_img = tf.transpose(w_img)  if len(shape) == 1:  w_img = tf.expand_dims(w_img, 0)  w_img = tf.expand_dims(tf.expand_dims(w_img, 0), -1)  tf.summary.image(weight.name, w_img)  if hasattr(layer, 'output'):  tf.summary.histogram('{}_out'.format(layer.name),  layer.output)  self.merged = tf.summary.merge_all()  

---

可视化结果

---

python /home/bids/.local/lib/python2.7/site-packages/tensorboard/tensorboard.py --logdir='/tmp/keras_log'

Starting TensorBoard 54 at http://bids:6006
(Press CTRL+C to quit)

---

训练误差和测试误差在同一个图中（tensorflow）

---

具体参看Windows下tensorflow的tensorboard的使用

TensorBoard: Visualizing Learning

使用Tensorboard查看训练过程

TensorFlow深度学习笔记 Tensorboard入门
Tensorflow 自带可视化Tensorboard使用方法 附项目代码

学习TensorFlow，TensorBoard可视化网络结构和参数

tensorflow中如何进行可视化和减轻过拟合

TensorFlow-7-TensorBoard Embedding可视化

from __future__ import absolute_import  
from __future__ import division  
from __future__ import print_function  import argparse  
import sys  import tensorflow as tf  from tensorflow.examples.tutorials.mnist import input_data  FLAGS = None  def train():  # Import data  mnist = input_data.read_data_sets(FLAGS.data_dir,  one_hot=True,  fake_data=FLAGS.fake_data)  sess = tf.InteractiveSession()  # Create a multilayer model.  # Input placeholders  with tf.name_scope('input'):  x = tf.placeholder(tf.float32, [None, 784], name='x-input')  y_ = tf.placeholder(tf.float32, [None, 10], name='y-input')  with tf.name_scope('input_reshape'):  image_shaped_input = tf.reshape(x, [-1, 28, 28, 1])  tf.summary.image('input', image_shaped_input, 10)  # We can't initialize these variables to 0 - the network will get stuck.  def weight_variable(shape):  """Create a weight variable with appropriate initialization."""  initial = tf.truncated_normal(shape, stddev=0.1)  return tf.Variable(initial)  def bias_variable(shape):  """Create a bias variable with appropriate initialization."""  initial = tf.constant(0.1, shape=shape)  return tf.Variable(initial)  def variable_summaries(var):  """Attach a lot of summaries to a Tensor (for TensorBoard visualization)."""  with tf.name_scope('summaries'):  mean = tf.reduce_mean(var)  tf.summary.scalar('mean', mean)  with tf.name_scope('stddev'):  stddev = tf.sqrt(tf.reduce_mean(tf.square(var - mean)))  tf.summary.scalar('stddev', stddev)  tf.summary.scalar('max', tf.reduce_max(var))  tf.summary.scalar('min', tf.reduce_min(var))  tf.summary.histogram('histogram', var)  def nn_layer(input_tensor, input_dim, output_dim, layer_name, act=tf.nn.relu):  """Reusable code for making a simple neural net layer.  It does a matrix multiply, bias add, and then uses relu to nonlinearize.  It also sets up name scoping so that the resultant graph is easy to read,  and adds a number of summary ops.  """  # Adding a name scope ensures logical grouping of the layers in the graph.  with tf.name_scope(layer_name):  # This Variable will hold the state of the weights for the layer  with tf.name_scope('weights'):  weights = weight_variable([input_dim, output_dim])  variable_summaries(weights)  with tf.name_scope('biases'):  biases = bias_variable([output_dim])  variable_summaries(biases)  with tf.name_scope('Wx_plus_b'):  preactivate = tf.matmul(input_tensor, weights) + biases  tf.summary.histogram('pre_activations', preactivate)  activations = act(preactivate, name='activation')  tf.summary.histogram('activations', activations)  return activations  hidden1 = nn_layer(x, 784, 500, 'layer1')  with tf.name_scope('dropout'):  keep_prob = tf.placeholder(tf.float32)  tf.summary.scalar('dropout_keep_probability', keep_prob)  dropped = tf.nn.dropout(hidden1, keep_prob)  # Do not apply softmax activation yet, see below.  y = nn_layer(dropped, 500, 10, 'layer2', act=tf.identity)  with tf.name_scope('cross_entropy'):  # The raw formulation of cross-entropy,  #  # tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(tf.softmax(y)),  #                               reduction_indices=[1]))  #  # can be numerically unstable.  #  # So here we use tf.nn.softmax_cross_entropy_with_logits on the  # raw outputs of the nn_layer above, and then average across  # the batch.  diff = tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y)  with tf.name_scope('total'):  cross_entropy = tf.reduce_mean(diff)  tf.summary.scalar('cross_entropy', cross_entropy)  with tf.name_scope('train'):  train_step = tf.train.AdamOptimizer(FLAGS.learning_rate).minimize(  cross_entropy)  with tf.name_scope('accuracy'):  with tf.name_scope('correct_prediction'):  correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))  with tf.name_scope('accuracy'):  accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))  tf.summary.scalar('accuracy', accuracy)  # Merge all the summaries and write them out to /tmp/tensorflow/mnist/logs/mnist_with_summaries (by default)  merged = tf.summary.merge_all()  train_writer = tf.summary.FileWriter(FLAGS.log_dir + '/train', sess.graph)  test_writer = tf.summary.FileWriter(FLAGS.log_dir + '/test')  tf.global_variables_initializer().run()  # Train the model, and also write summaries.  # Every 10th step, measure test-set accuracy, and write test summaries  # All other steps, run train_step on training data, & add training summaries  def feed_dict(train):  """Make a TensorFlow feed_dict: maps data onto Tensor placeholders."""  if train or FLAGS.fake_data:  xs, ys = mnist.train.next_batch(100, fake_data=FLAGS.fake_data)  k = FLAGS.dropout  else:  xs, ys = mnist.test.images, mnist.test.labels  k = 1.0  return {x: xs, y_: ys, keep_prob: k}  for i in range(FLAGS.max_steps):  if i % 10 == 0:  # Record summaries and test-set accuracy  summary, acc = sess.run([merged, accuracy], feed_dict=feed_dict(False))  test_writer.add_summary(summary, i)  print('Accuracy at step %s: %s' % (i, acc))  else:  # Record train set summaries, and train  if i % 100 == 99:  # Record execution stats  run_options = tf.RunOptions(trace_level=tf.RunOptions.FULL_TRACE)  run_metadata = tf.RunMetadata()  summary, _ = sess.run([merged, train_step],  feed_dict=feed_dict(True),  options=run_options,  run_metadata=run_metadata)  train_writer.add_run_metadata(run_metadata, 'step%03d' % i)  train_writer.add_summary(summary, i)  print('Adding run metadata for', i)  else:  # Record a summary  summary, _ = sess.run([merged, train_step], feed_dict=feed_dict(True))  train_writer.add_summary(summary, i)  train_writer.close()  test_writer.close()  def main(_):  if tf.gfile.Exists(FLAGS.log_dir):  tf.gfile.DeleteRecursively(FLAGS.log_dir)  tf.gfile.MakeDirs(FLAGS.log_dir)  train()  if __name__ == '__main__':  parser = argparse.ArgumentParser()  parser.add_argument('--fake_data', nargs='?', const=True, type=bool,  default=False,  help='If true, uses fake data for unit testing.')  parser.add_argument('--max_steps', type=int, default=1000,  help='Number of steps to run trainer.')  parser.add_argument('--learning_rate', type=float, default=0.001,  help='Initial learning rate')  parser.add_argument('--dropout', type=float, default=0.9,  help='Keep probability for training dropout.')  parser.add_argument('--data_dir', type=str, default='/tmp/tensorflow/mnist/input_data',  help='Directory for storing input data')  parser.add_argument('--log_dir', type=str, default='/tmp/tensorflow/mnist/logs/mnist_with_summaries',  help='Summaries log directory')  FLAGS, unparsed = parser.parse_known_args()  tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)

上面是官方给出的例子实现，下面我给出要修改的地方：

    parser.add_argument('--log_dir', type=str, default='/tmp/tensorflow/mnist/logs/mnist_with_summaries',  help='Summaries log directory')  

修改后的代码：

parser.add_argument('--log_dir', type=str, default='C:/tmp/tensorflow/mnist/logs/mnist_with_summaries',help='Summaries log directory')

这里目录取决你放在哪个盘，这里我放在C盘，打开cmd。在终端输入，如下图：

tensorboard --logdir= C:\tmp\tensorflow\mnist\logs\mnist_with_summaries

打开Google Chrome输入localhost:6006，结果如下图

---

Display Deep Learning Model Training History in Keras

---

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
# Fit the model
history = model.fit(X, Y, validation_split=0.33, epochs=150, batch_size=10, verbose=0)
# list all data in history
print(history.history.keys())
# summarize history for accuracy
plt.plot(history.history['acc'])
plt.plot(history.history['val_acc'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.tight_layout()
plt.savefig('accuracyVSepoch.png')
plt.show()
# summarize history for loss
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.tight_layout()
plt.savefig('lossVSepoch.png')
plt.show()

就会得到如下类似的图

---

Keras输出的loss，val这些值如何保存到文本中去

---

hist=model.fit(train_set_x,train_set_y,batch_size=256,shuffle=True,nb_epoch=nb_epoch,validation_split=0.1)
with open('log_sgd_big_32.txt','w') as f:f.write(str(hist.history))

---

同时可视化多个模型运行结果

---

第一个模型的运行结果

log_filepath = '/tmp/keras_log/run_a' model.compile(loss='categorical_crossentropy', optimizer=SGD(lr=0.001), metrics=['accuracy'])    
tb_cb = keras.callbacks.TensorBoard(log_dir=log_filepath, write_images=1, histogram_freq=1)  
# 设置log的存储位置，将网络权值以图片格式保持在tensorboard中显示，设置每一个周期计算一次网络的  
#权值，每层输出值的分布直方图  
batch_size=10,
cbks = [tb_cb],  
history = model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch,   
verbose=1, callbacks=cbks, validation_data=(X_test, Y_test))  

第二个模型的运行结果

log_filepath = '/tmp/keras_log/run_b' model.compile(loss='categorical_crossentropy', optimizer=adam(lr=0.001), metrics=['accuracy'])    
tb_cb = keras.callbacks.TensorBoard(log_dir=log_filepath, write_images=1, histogram_freq=1)  
# 设置log的存储位置，将网络权值以图片格式保持在tensorboard中显示，设置每一个周期计算一次网络的  
#权值，每层输出值的分布直方图
batchsize=8  
cbks = [tb_cb],  
history = model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch,   
verbose=1, callbacks=cbks, validation_data=(X_test, Y_test))  

可视化结果

python /home/bids/.local/lib/python2.7/site-packages/tensorboard/tensorboard.py --logdir='/tmp/keras_log'

Starting TensorBoard 54 at http://bids:6006
(Press CTRL+C to quit)

就会看到如下类似的图像

---

实时可视化

---

jiandanjinxin－tensorflow-mnist-tutorial
没有博士学位如何玩转TensorFlow和深度学习

Visualizing MNIST: An Exploration of Dimensionality Reduction
Visualizing Representations: Deep Learning and Human Beings

---

Hyperparameter Search

---

jiandanjinxin-tf-dev-summit-tensorboard-tutoria

TensorFlow-dev-summit：TF发展历史以及有趣的应用

---

经典示例 1

---

#-*- coding: UTF-8 -*-
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literalsimport gzip
import struct
import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn import preprocessing
from sklearn.metrics import accuracy_score
import tensorflow as tf# MNIST data is stored in binary format, 
# and we transform them into numpy ndarray objects by the following two utility functions
def read_image(file_name):with gzip.open(file_name, 'rb') as f:buf = f.read()index = 0magic, images, rows, columns = struct.unpack_from('>IIII' , buf , index)index += struct.calcsize('>IIII')image_size = '>' + str(images*rows*columns) + 'B'ims = struct.unpack_from(image_size, buf, index)im_array = np.array(ims).reshape(images, rows, columns)return im_arraydef read_label(file_name):with gzip.open(file_name, 'rb') as f:buf = f.read()index = 0magic, labels = struct.unpack_from('>II', buf, index)index += struct.calcsize('>II')label_size = '>' + str(labels) + 'B'labels = struct.unpack_from(label_size, buf, index)label_array = np.array(labels)return label_arrayprint("Start processing MNIST handwritten digits data...")

train_x_data = read_image("MNIST_data/train-images-idx3-ubyte.gz")
print(type(train_x_data),train_x_data.dtype, train_x_data.shape)
print(train_x_data.max(), train_x_data.min()) 
train_y_data = read_label("MNIST_data/train-labels-idx1-ubyte.gz")
print(type(train_y_data),train_y_data.dtype, train_y_data.shape)
print(train_y_data.max(), train_y_data.min())
test_x_data = read_image("MNIST_data/t10k-images-idx3-ubyte.gz")
print(type(test_x_data),test_x_data.dtype, test_x_data.shape)
print(test_x_data.max(), test_x_data.min()) 
test_x_data = test_x_data.reshape(test_x_data.shape[0], -1).astype(np.float32)
print(type(test_x_data),test_x_data.dtype, test_x_data.shape)
print(test_x_data.max(), test_x_data.min()) 
test_y_data = read_label("MNIST_data/t10k-labels-idx1-ubyte.gz")
print(type(test_y_data),test_y_data.dtype, test_y_data.shape)
print(test_y_data.max(), test_y_data.min())

train_x_minmax = train_x_data / 255.0
test_x_minmax = test_x_data / 255.0
# We evaluate the softmax regression model by sklearn first
eval_sklearn = True
if eval_sklearn:print("Start evaluating softmax regression model by sklearn...")reg = LogisticRegression(solver="lbfgs", multi_class="multinomial")reg.fit(train_x_minmax, train_y_data)#np.savetxt('coef_softmax_sklearn.txt', reg.coef_, fmt='%.6f')  # Save coefficients to a text filetest_y_predict = reg.predict(test_x_minmax)print("Accuracy of test set: %f" % accuracy_score(test_y_data, test_y_predict))eval_tensorflow = True
batch_gradient = Falsedef variable_summaries(var):with tf.name_scope('summaries'):mean = tf.reduce_mean(var)tf.summary.scalar('mean', mean)stddev = tf.sqrt(tf.reduce_mean(tf.square(var - mean)))tf.summary.scalar('stddev', stddev)tf.summary.scalar('max', tf.reduce_max(var))tf.summary.scalar('min', tf.reduce_min(var))tf.summary.histogram('histogram', var)

if eval_tensorflow:print("Start evaluating softmax regression model by tensorflow...")# reformat y into one-hot encoding stylelb = preprocessing.LabelBinarizer()lb.fit(train_y_data)train_y_data_trans = lb.transform(train_y_data)test_y_data_trans = lb.transform(test_y_data)x = tf.placeholder(tf.float32, [None, 784])with tf.name_scope('weights'):W = tf.Variable(tf.zeros([784, 10]))variable_summaries(W)with tf.name_scope('biases'):b = tf.Variable(tf.zeros([10]))variable_summaries(b)with tf.name_scope('Wx_plus_b'):V = tf.matmul(x, W) + btf.summary.histogram('pre_activations', V)with tf.name_scope('softmax'):y = tf.nn.softmax(V)tf.summary.histogram('activations', y)y_ = tf.placeholder(tf.float32, [None, 10])with tf.name_scope('cross_entropy'):loss = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1]))tf.summary.scalar('cross_entropy', loss)with tf.name_scope('train'):optimizer = tf.train.GradientDescentOptimizer(0.5)train = optimizer.minimize(loss)with tf.name_scope('evaluate'):with tf.name_scope('correct_prediction'):correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))with tf.name_scope('accuracy'):accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))tf.summary.scalar('accuracy', accuracy)init = tf.global_variables_initializer()sess = tf.Session()sess.run(init)merged = tf.summary.merge_all()train_writer = tf.summary.FileWriter('/tmp/log/train', sess.graph)test_writer = tf.summary.FileWriter('/tmp/log/test')if batch_gradient:for step in range(300):sess.run(train, feed_dict={x: train_x_minmax, y_: train_y_data_trans})if step % 10 == 0:print("Batch Gradient Descent processing step %d" % step)print("Finally we got the estimated results, take such a long time...")else:for step in range(1000):if step % 10 == 0:summary, acc = sess.run([merged, accuracy], feed_dict={x: test_x_minmax, y_: test_y_data_trans})test_writer.add_summary(summary, step)print("Stochastic Gradient Descent processing step %d accuracy=%.2f" % (step, acc))else:sample_index = np.random.choice(train_x_minmax.shape[0], 100)batch_xs = train_x_minmax[sample_index, :]batch_ys = train_y_data_trans[sample_index, :]summary, _ = sess.run([merged, train], feed_dict={x: batch_xs, y_: batch_ys})train_writer.add_summary(summary, step)#np.savetxt('coef_softmax_tf.txt', np.transpose(sess.run(W)), fmt='%.6f')  # Save coefficients to a text fileprint("Accuracy of test set: %f" % sess.run(accuracy, feed_dict={x: test_x_minmax, y_: test_y_data_trans}))

转自：TensorFlow学习笔记（7）：TensorBoard——Tensor与Graph可视化

---

经典示例 2

---

import tensorflow as tf
import numpy as nptf.set_random_seed(1)
np.random.seed(1)# fake data
x = np.linspace(-1, 1, 100)[:, np.newaxis]          # shape (100, 1)
noise = np.random.normal(0, 0.1, size=x.shape)
y = np.power(x, 2) + noise                          # shape (100, 1) + some noisewith tf.variable_scope('Inputs'):tf_x = tf.placeholder(tf.float32, x.shape, name='x')tf_y = tf.placeholder(tf.float32, y.shape, name='y')with tf.variable_scope('Net'):l1 = tf.layers.dense(tf_x, 10, tf.nn.relu, name='hidden_layer')output = tf.layers.dense(l1, 1, name='output_layer')# add to histogram summarytf.summary.histogram('h_out', l1)tf.summary.histogram('pred', output)loss = tf.losses.mean_squared_error(tf_y, output, scope='loss')
train_op = tf.train.GradientDescentOptimizer(learning_rate=0.5).minimize(loss)
tf.summary.scalar('loss', loss)     # add loss to scalar summarysess = tf.Session()
sess.run(tf.global_variables_initializer())writer = tf.summary.FileWriter('./log', sess.graph)     # write to file
merge_op = tf.summary.merge_all()                       # operation to merge all summaryfor step in range(100):# train and net output_, result = sess.run([train_op, merge_op], {tf_x: x, tf_y: y})writer.add_summary(result, step)

Tensorflow-Tutorial/tutorial-contents/305_tensorboard.py

---

逐层可视化

---

卷积神经网络实战（可视化部分）——使用keras识别猫咪
Visualizing parts of Convolutional Neural Networks using Keras and Cats
All code is on Github: https://github.com/erikreppel/visualizing_cnns

这张图长400像素宽320像素，有三个通道（rgb）的颜色。
那么经过一层卷积运算之后会变成什么样子呢？

这是用一个3*3的卷积核和三个滤波器处理的效果（如果我们有超过3个的滤波器，那么我可以画出猫的2d图像。更高维的话就很难处理）
我们可以看到，图中的猫非常的模糊，因为我们使用了一个随机的初始值，而且我们还没有训练网络。他们都在彼此的顶端，即使每层都有细节，我们将无法看到它。但我们可以制作出与眼睛和背景相同颜色的猫的区域。如果我们将内核大小增加到10x10，会发生什么呢？

我们可以看到，由于内核太大，我们失去了一些细节。还要注意，从数学角度来看，卷积核越大，图像的形状会变得越小。
如果我们把它压扁一点，我们可以更好的看到色彩通道会发生什么？

这张看起来好多了！现在我们可以看到我们的过滤器看到的一些事情。看起来红色替换掉了黑色的鼻子和黑色眼睛，蓝色替换掉了猫边界的浅灰色。我们可以开始看到图层如何捕获照片中的一些更重要的细节。

如果我们增加内核大小，我们得到的细节就会越来越明显，当然图像也比其他两个都小。

增加一个激活层

我们通过添加一个relu，去掉了很多不是蓝色的部分。

增加一个池化层

我们添加一个池化层（摆脱激活层最大限度地让图片更加更容易显示）。

正如预期的那样，猫咪变成了斑驳的，而我们可以让它更加斑驳。

现在图片大约成了原来的三分之一。

激活和最大池化

LeNet

如果我们将猫咪的图片放到LeNet模型中做卷积和池化，那么效果会怎么样呢?

ConvNets功能强大，因为它们能够提取图像的核心特征，并使用这些特征来识别包含其中的特征的图像。即使我们的两层CNN，我们也可以开始看到网络正在对猫的晶须，鼻子和眼睛这样的地区给予很多的关注。这些是让CNN将猫与鸟区分开的特征的类型。
CNN是非常强大的，虽然这些可视化并不完美，但我希望他们能够帮助像我这样正在尝试更好地理解ConvNets的人。

---

可视化模型

---

keras.utils.vis_utils模块提供了画出Keras模型的函数（利用graphviz）

该函数将画出模型结构图，并保存成图片：

from keras.utils import plot_model
plot_model(model, to_file='model.png')

plot_model接收两个可选参数：

show_shapes：指定是否显示输出数据的形状，默认为False
show_layer_names:指定是否显示层名称,默认为True

我们也可以直接获取一个pydot.Graph对象，然后按照自己的需要配置它，例如，如果要在ipython中展示图片

from IPython.display import SVG
from keras.utils.vis_utils import model_to_dotSVG(model_to_dot(model).create(prog='dot', format='svg'))

【Tips】依赖 pydot-ng 和 graphviz，若出现错误，用命令行输入pip install pydot-ng & brew install graphviz

---

keras回调函数中的Tensorboard

---

keras.callbacks.TensorBoard(log_dir='./Graph', histogram_freq=0,  write_graph=True, write_images=True)

tbCallBack = keras.callbacks.TensorBoard(log_dir='./Graph', histogram_freq=0, write_graph=True, write_images=True)
...
model.fit(...inputs and parameters..., callbacks=[tbCallBack])

tensorboard --logdir path_to_current_dir/Graph 

或者

from keras.callbacks import TensorBoardtensorboard = TensorBoard(log_dir='./logs', histogram_freq=0,write_graph=True, write_images=False)
# define model
model.fit(X_train, Y_train,batch_size=batch_size,epochs=nb_epoch,validation_data=(X_test, Y_test),shuffle=True,callbacks=[tensorboard])

https://stackoverflow.com/questions/42112260/how-do-i-use-the-tensorboard-callback-of-keras

---

参考文献

---

Display Deep Learning Model Training History in Keras

Edward－Tensorboard

详解TensorBoard如何调参
https://gist.github.com/dandelionmane/4f02ab8f1451e276fea1f165a20336f1#file-mnist-py

TensorFlow深度学习笔记 Tensorboard入门

Keras中文文档－模型可视化
极客学院－TensorBoard:可视化学习

Tensorboard 可视化之图层

Tensorboard 可视化之训练过程

TensorBoard可视化详解

TensorFlow学习笔记（七）：TensorBoard可视化助手

https://www.tensorflow.org/get_started/summaries_and_tensorboard
https://www.tensorflow.org/get_started/summaries_and_tensorboard
https://www.youtube.com/watch?v=eBbEDRsCmv4

How do I use the Tensorboard callback of Keras?
keras tensorboard的使用, 设置GPU使用的内存
tensorboard的一些问题
模型可视化
Keras中文文档

卷积神经网络实战（可视化部分）——使用keras识别猫咪
Visualizing parts of Convolutional Neural Networks using Keras and Cats
All code is on Github: https://github.com/erikreppel/visualizing_cnns