GreenEyes: An Air Quality Evaluating Model based on WaveNet

Kan Huang, the Hong Kong University of Science and Technology

Kai Zhang, Lehigh University

Ming Liu, the Hong Kong University of Science and Technology

Outlines

• Motivation
• Contributions
• Datasets
• Methodology
• Experiments
• Conclusion

Motivation

• WaveNet’s residual layers can be used as feature extractors;
• Attention and LSTM are good output endpoints for time series fitting;
• Piece-wise linear function is a simple but useful representation of the trends of the target time series.

​

​

​

​

Contributions

• We use WaveNet’s residual layers as a feature extractor block;
• Stack several WaveNet blocks to build the model’s main body;
• Put Attention and LSTM at the endpoint as output layers, and make ablation study.

Datasets

• Four sensors of the same model;
• At the same place;
• The data are slightly different. Can they be augmented to make better fitting?

Methodology

Figure 4: Overview of the residual block and the entire architecture.

Oord, Aaron van den, et al. "Wavenet: A generative model for raw audio." arXiv preprint arXiv:1609.03499 (2016).

WaveNet Layer

1 Layer

Dilated

Conv

Residual

Input

WaveNet Block

Causal

Conv

k Layers

Dilated

Conv

Residual

Input

WaveNet Model

• WaveNet Layer
• WaveNet Block

k Layers

Dilated

Conv

Residual

Input

Dilated

Conv

Residual

Left layers…

dilation rate: 1

dilation rate: 2

dilation rate: 4, 8, 16…

WaveNet_LSTM Model

WaveNet_LSTM Model: Sequential/Cascading of WaveNet Block

Causal Conv

tanh

Dilated Conv

Residual

tanh

Dilated Conv

Residual

tanh

Dilated Conv

Residual

Input sequence

Target y

Bidirectional

LSTM

Attention

Data Processing

PM2.5 IAQI vs Time

Concentration thresholds of IAQI w.r.t. pollutant

categories, USA

Data Processing

Figure: Sensor 0’s 𝑃𝑀2.5 origin and labeled IAQI level.

Figure: Sensor 0’s labeled 𝑃𝑀2.5 IAQI level (solid line) and its polygonal line (dash line).

Experiments

Best metrics during training when applying Temporal

Attention.

• The less the stride, the better the results.
• Above conclusion is suitable for every channel’s data

Experiments - Ablation Study

Table: Test MSE and MAE for model with and w/o Attention. The model w/o Attention can perform better or is equivalent to the model applied with the Attention layer.

Figure: Validation MSE curves when applying official Attention

(𝑠𝑡𝑟𝑖𝑑𝑒 = 10).

Figure: Validation MSE curves when applying Temporal Attention (𝑠𝑡𝑟𝑖𝑑𝑒 = 10).

The model with the Temporal Attention layer can obtain

smaller loss during training

Experiments -

Figure: Window size 7200 vs 3600.

Thank you!

Q & A