SC-NAFSSR: Perceptual-Oriented Stereo Image Super-Resolution Using
Stereo Consistency Guided NAFSSR�Zidian Qiu, Zongyao He, Zhihao Zhan, Zilin Pan, Xingyuan Xian, and Zhi Jin*
School of Intelligent Systems Engineering, Sun Yat-sen University
Conv3x3
NAFBlock
Conv3x3
Pixel
Shuffle
Conv3x3
NAFBlock
Conv3x3
Pixel
Shuffle
SCAM
N×
(a) Overall Framework
C
U-Net discriminator
LN
Conv1x1
DWConv3x3
Simple Gate
CA
Conv1x1
LN
Conv1x1
Simple Gate
Conv1x1
(b) NAFBlock
LN
Linear
Linear
LN
Linear
Linear
(c) SCAM
matrix multiplication elementwise addition concatenate
softmax transpose
C
Framework
Ablation Study
Results
0028_L
Bicubic
NAFSSR-L
RCAN
Ours (w/o GAN)
iPASSR
ESRGAN
HR
Ours (w/ GAN)
0028_R
Bicubic
NAFSSR-L
RCAN
Ours (w/o GAN)
iPASSR
ESRGAN
HR
Ours (w/ GAN)
HR
Ours (w/ GAN)
Ours (w/o GAN)
NAFSSR-L
Contributions
Methods
Code: https://github.com/FVL2020/SC-NAFSSR
Contact: Zhi Jin jinzh26@mail.sysu.edu.cn
Ablation study of LPIPS loss and stereo consistency loss
Ablation study of different strategies
Qualitative results
Quantitative results
Applying GAN = less regular artifacts
Note: The blue arrows indicate supervision during the training process.
Loss | 1 | 2 | 3 | 4 |
MAE | √ | √ | √ | √ |
LPIPS Loss | | √ | √ | √ |
Parallax Supervision Loss | | | √ | |
Stereo Consistency Loss | | | | √ |
LPIPS | 0.3258 | 0.2170 | 0.2167 | 0.2152 |
SCORE | 0.5721 | 0.6770 | 0.6907 | 0.6894 |
Strategy | 1 | 2 | 3 | 4 | 5 |
MAE | √ | | √ | √ | √ |
MSE | | √ | | | |
VGG Perceptual Loss | | | √ | | |
EMA | | | | √ | |
Online dataset | | | | | √ |
LPIPS | 0.2167 | 0.2167 | 0.2159 | 0.2167 | 0.2156 |
SCORE | 0.6907 | 0.6907 | 0.6913 | 0.6930 | 0.6971 |
Method | #Params. | Flickr1024 | |||
PSNR | SSIM | LPIPS | SCORE | ||
Bicubic | / | 21.8796 | 0.6326 | 0.4091 | 0.4253 |
EDSR | 38.6M | 23.3739 | 0.7296 | 0.3338 | 0.5564 |
RCAN | 15.3M | 23.4561 | 0.7307 | 0.3340 | 0.5631 |
SRGAN | 1.51M | 20.8837 | 0.6240 | 0.2729 | 0.5896 |
ESRGAN | 16.70M | 20.2517 | 0.5999 | 0.2871 | 0.5638 |
PASSRnet | 1.42M | 23.2485 | 0.7167 | 0.3347 | 0.5519 |
iPASSR | 1.37M | 23.3730 | 0.7267 | 0.3389 | 0.5589 |
NAFSSR-L | 23.79M | 24.0854 | 0.7565 | 0.3103 | 0.5984 |
Ours (w/o GAN) | 23.79M | 22.6236 | 0.6918 | 0.2106 | 0.6915 |
Ours (w/ GAN) | 23.79M | 22.4388 | 0.6918 | 0.2100 | 0.6983 |
Team Name | Score |
SRC-B | 0.8622 |
SYSU_FVL | 0.8538 |
webbzhou | 0.8496 |
SSSL | 0.8471 |
Giantpandacv | 0.8351 |
DiffX | 0.8303 |
LongClaw | 0.7994 |
BUPT-PRIV | 0.7992 |
McSR | 0.7960 |
LVGroup_HFUT | 0.7958 |