I have found the DeepSense framework as one of the promising deep learning architectures for processing Time-Series sensing data. In this brief and intuitive overview, I’ll present the main ideas of the original paper titled “Deep Sense: A Unified DL Framework for Time-Series Mobile Sensing Data Processing” by Yao et-al. (Can be found at www2017).

我发现DeepSense框架是用于处理时间序列感应数据的有前途的深度学习架构之一。 在这个简短而直观的概述中，我将介绍Yao等人题为“深度感知：用于时间序列移动传感数据处理的统一DL框架”的原始论文的主要思想。 (可以在www2017上找到)。

主要问题 (The Main Problem)

DeepSense addresses the problem of mobile sensing using time-series data from various sensors (accelerometers, gyroscopes, and more). Due to the quality of the sensor, processing any type of sensor involves noisy measurements. It is difficult to find a distribution that describes noise in real-life, so in the end, our measurements are corrupted by ”unknown” noise (nonlinear, correlated over time, etc.). In addition, for tracking applications, a physical system model should be defined. This model is also affected by “noise” in terms of inaccuracy (as we cannot predict/ define the model without error). As we deal with real-time measurement (and physical model), the behavior can be described as “time-series measurements of physical quantities..”.

DeepSense使用来自各种传感器(加速度计，陀螺仪等)的时间序列数据解决了移动感应的问题。 由于传感器的质量，处理任何类型的传感器都会涉及噪声测量。 很难找到描述现实生活中的噪声的分布，因此最后，我们的测量结果被“未知”噪声(非线性，随时间相关等)破坏了。 此外，对于跟踪应用程序，应定义一个物理系统模型。 该模型在准确性方面也受到“噪声”的影响(因为我们无法无误地预测/定义模型)。 当我们处理实时测量(和物理模型)时，该行为可以描述为“物理量的时间序列测量”。

(核心)想法 (The (core) Idea)

The author proposes a unified DL framework that addresses the challenges described in mobile sensing: integration of Convolution Neural Networks (CNN) and Recurrent Neural Network (RNN). CNN is responsible for the computation of the sensing quantities within the time interval where it extracts the local features for all sensors and combines them into global features. RNN is responsible for computation of the sensing quantities across time intervals, where it extracts temporal dependencies. The DeepSense solves both classification and regression of mobile computing tasks in some unified manner.

作者提出了一个统一的DL框架，以解决移动传感中描述的挑战：卷积神经网络(CNN)和递归神经网络(RNN)的集成。 CNN负责计算时间间隔内的感应量， 在该时间间隔内 ，CNN提取所有传感器的局部特征并将其组合为全局特征。 RNN负责跨时间间隔传感量的计算，在那里它提取时间相关。 DeepSense以某种统一的方式解决了移动计算任务的分类和回归问题。

架构 (The Architecture)

The architecture consists of 3 parts: a convolutional layer, recurrent layer, and output layer. Since DeepSense can be used for classification and regression tasks, the output layer should be set according to the specific task.

该体系结构由三部分组成：卷积层，循环层和输出层。 由于DeepSense可用于分类和回归任务，因此应根据特定任务设置输出层。

Convolutional layer: an individual convolutional subnet can be represented by 3 layers (Individual Convolutional Layers 1,2,3). By applying a 2d filter, the net can learn interaction among sensor measurements and local patterns. Next, by Flatten & Concatenation layer, the matrices are flattened into a vector which is the input to the next 3 layers: The Merge Convolutional Layers. A 2d filter is also applied here to learn the interaction between all K input sensors. For each convolutional layer, DeepSense learns 64 filters by using ReLU. Apply batch normalization at each layer to reduce the internal covariate shift. A Flatten & Concatenation layer completes this phase of the convolutional layer.

卷积层：单个卷积子网可以由3层表示( 单个卷积层 1,2,3)。 通过应用二维过滤器，网络可以了解传感器测量值和局部模式之间的相互作用。 接下来，通过Flatten＆Concatenation层，将矩阵展平为一个向量，该向量是接下来3层的输入： 合并卷积层。 这里还应用了2d滤波器，以了解所有K个输入传感器之间的相互作用。 对于每个卷积层，DeepSense通过使用ReLU学习64个过滤器。 在每一层应用批量归一化以减少内部协变量偏移。 Flatten＆Concatenation层完成了卷积层的此阶段。

Recurrent layer: the power of RNN is in the ability to approximate function and understand the important features for time-series. In this architecture, the RNN extension model that used is a stacked Gated Recurrent Unit (GRU). 2 layers of RNN are implemented with dropout to the connection between these Recurrent Layers (1,2), as also recurrent batch normalization in order to reduce the internal covariance shift among data series.

循环层： RNN的功能在于能够逼近函数并了解时间序列的重要特征。 在此体系结构中，使用的RNN扩展模型是堆叠的门控循环单元(GRU)。 RNN的2层实现为与这些递归层(1,2)之间的连接断开，也为递归批归一化，以减少数据序列之间的内部协方差漂移。

Output Layer: Up to this phase, the net has a series of vectors for each time step. Now, we should handle carefully upon the tasks: regression or classification. For regression, a dictionary should be learned. For classification, the composition of the output layer should be done by averaging the features over time. then, feeding the final features into a softmax layer to generate the prediction.

输出层：到此阶段，网络每个时间步都有一系列矢量。 现在，我们应该认真处理以下任务：回归或分类。 为了回归，应该学习字典。 对于分类，输出层的组成应通过对特征随时间进行平均来完成。 然后，将最终特征输入softmax层以生成预测。

摘要和我的观点 (Summary and My Point of View)

DeepSense seems to be very promising for many time-series tasks, Aside the Temporal Convolutional Networks, WaveNet, and others. When I first read the paper, I thought about the Extended Kalman Filter as a nice classical compression — as it deals with the same type of problem. This deep learning architecture might deal with model uncertainty, noisy measurement, and more.

除了时间卷积网络，WaveNet等，DeepSense对于许多时间序列任务似乎都非常有前途。 当我第一次阅读本文时，我认为扩展卡尔曼滤波器是一种很好的经典压缩方法，因为它可以处理相同类型的问题。 这种深度学习架构可能会处理模型不确定性，噪声测量等问题。

Further reading

进一步阅读

Yao, Shuochao, et al. “Deepsense: A unified deep learning framework for time-series mobile sensing data processing.” Proceedings of the 26th International Conference on World Wide Web. 2017.

姚硕超，等。 “ Deepsense：用于时间序列移动感测数据处理的统一深度学习框架。” 第26届国际万维网会议论文集 。 2017。