1 of 1

  1. Ablation study of LPIPS loss and stereo consistency loss
  2. Ablation study of different strategies

SC-NAFSSR: Perceptual-Oriented Stereo Image Super-Resolution Using

Stereo Consistency Guided NAFSSRZidian Qiu, Zongyao He, Zhihao Zhan, Zilin Pan, Xingyuan Xian, and Zhi Jin*

School of Intelligent Systems Engineering, Sun Yat-sen University

Conv3x3

NAFBlock

Conv3x3

Pixel

Shuffle

Conv3x3

NAFBlock

Conv3x3

Pixel

Shuffle

SCAM

(a) Overall Framework

 

 

C

 

 

U-Net discriminator

LN

Conv1x1

DWConv3x3

Simple Gate

CA

Conv1x1

LN

Conv1x1

Simple Gate

Conv1x1

 

 

(b) NAFBlock

LN

Linear

 

 

Linear

LN

Linear

 

 

 

 

 

Linear

 

 

 

 

 

 

(c) SCAM

matrix multiplication elementwise addition concatenate

softmax transpose

 

C

Framework

Ablation Study

Results

0028_L

Bicubic

NAFSSR-L

RCAN

Ours (w/o GAN)

iPASSR

ESRGAN

HR

Ours (w/ GAN)

0028_R

Bicubic

NAFSSR-L

RCAN

Ours (w/o GAN)

iPASSR

ESRGAN

HR

Ours (w/ GAN)

HR

Ours (w/ GAN)

Ours (w/o GAN)

NAFSSR-L

Contributions

Methods

Code: https://github.com/FVL2020/SC-NAFSSR

Contact: Zhi Jin jinzh26@mail.sysu.edu.cn

Ablation study of LPIPS loss and stereo consistency loss

Ablation study of different strategies

 

Qualitative results

Quantitative results

Applying GAN = less regular artifacts

  • Based on the NTIRE 2022 Stereo Image Super-Resolution Challenge champion scheme NAFSSR, we combine the LPIPS perceptual loss with the traditional VGG perceptual loss for training.
  • To improve stereo consistency, we perform supervision on each SCAM.
  • Our method without GAN end up ranking second place on the Flickr1024 test set.

Note: The blue arrows indicate supervision during the training process.

Loss

1

2

3

4

MAE

LPIPS Loss

Parallax Supervision Loss

Stereo Consistency Loss

LPIPS

0.3258

0.2170

0.2167

0.2152

SCORE

0.5721

0.6770

0.6907

0.6894

Strategy

1

2

3

4

5

MAE

MSE

VGG Perceptual Loss

EMA

Online dataset

LPIPS

0.2167

0.2167

0.2159

0.2167

0.2156

SCORE

0.6907

0.6907

0.6913

0.6930

0.6971

Method

#Params.

Flickr1024

PSNR

SSIM

LPIPS

SCORE

Bicubic

/

21.8796

0.6326

0.4091

0.4253

EDSR

38.6M

23.3739

0.7296

0.3338

0.5564

RCAN

15.3M

23.4561

0.7307

0.3340

0.5631

SRGAN

1.51M

20.8837

0.6240

0.2729

0.5896

ESRGAN

16.70M

20.2517

0.5999

0.2871

0.5638

PASSRnet

1.42M

23.2485

0.7167

0.3347

0.5519

iPASSR

1.37M

23.3730

0.7267

0.3389

0.5589

NAFSSR-L

23.79M

24.0854

0.7565

0.3103

0.5984

Ours (w/o GAN)

23.79M

22.6236

0.6918

0.2106

0.6915

Ours (w/ GAN)

23.79M

22.4388

0.6918

0.2100

0.6983

Team Name

Score

SRC-B

0.8622

SYSU_FVL

0.8538

webbzhou

0.8496

SSSL

0.8471

Giantpandacv

0.8351

DiffX

0.8303

LongClaw

0.7994

BUPT-PRIV

0.7992

McSR

0.7960

LVGroup_HFUT

0.7958