SUNet: Swin Transformer with UNet for

Image Denoising

Outline

2

• Introduction

​

​

• Proposed Method

​

​

• Experiments

​

​

• Conclusion

Introduction

3

• Introduction

​

​

• Proposed Method

​

​

• Experiments

​

​

• Conclusion

Introduction (1/2)

4

Image denoising is a challenging ill-posed problem which also

has been a long-standing issue. Moreover, denoising is an

important low-level image processing which could improve the performance in the high-level vision tasks.

Denoising

X

Y

Introduction (2/2)

5

• Contribution
• We proposed a denoising architecture based on the image

segmentation Swin-UNet model called SUNet.

​

• We proposed a dual up-sample block which comprises both subpixel and bilinear up-sample methods to prevent checkboard artifacts.

​

• To the best of our knowledge, our model is the first one to incorporate Swin Transformer and UNet in denoising.

​

• We demonstrate the competitive results of our SUNet in two common datasets for image denoising.

Method

6

• Introduction

​

​

• Proposed Method

​

​

• Experiments

​

​

• Conclusion

Method (1/7)

7

• SUNet

Method (2/7)

8

• SUNet

​

• Shallow feature extraction module (3 x 3 convolution)

​

​

• UNet feature extraction module

​

​

• Reconstruction module (3 x 3 convolution)

Method (3/7)

9

• UNet feature extraction module

​

• Swin Transformer Layer (STL) and Swin Transformer Block (STB)

​

​

• Patch Merging down-sampling

​

​

• Dual up-sampling

Method (4/7)

10

• STL and STB

​

• We use Swin Transformer Block as our feature extraction blocks for

image denoising.

​

• Because of the shifted-window design, the Swin Transformer Layer in Block have to the multiple of even.

Method (5/7)

11

• Patch Merging down-sampling

​

• Concatenate the input features of each group of 2 × 2 neighboring

patches, and then use the linear layer to obtain the specified channel

number of output features.

​

• The operation is same as the Pixel Unshuffled with the convolution.

Method (6/7)

12

• Dual up-sampling

​

• Original Swin-UNet uses patch expanding up-sampling method which is equivalent to transpose convolution. However, the transpose

convolution usually happens the block effects which seriously

influence the denoised performance.

​

• We proposed a up-sampling block called

Dual up-sampling, which comprises

Bilinear and PixelShuffle methods to

prevent checkerboard artifacts.

Method (7/7)

13

Experiments

14

• Introduction

​

​

• Proposed Method

​

​

• Experiments

​

​

• Conclusion

Experiments (1/6)

15

Experiments (2/6)

16

• Dataset
• Testing dataset:

​

CBSD68 which has 68 color images with the resolution of 321 x481.

​

​

Kodak24 which consisting of 24 images with the image size of

768 x 512.

Experiments (3/6)

17

• Denoising result

Experiments (4/6)

18

• Denoising visual result (CBSD68)

Experiments (5/6)

19

• Denoising visual result (Kodak24)

Experiments (6/6)

20

• Additional experiments about Dual up-sampling

Conclusion

21

• Introduction

​

​

• Proposed Method

​

​

• Experiments

​

​

• Conclusion

Conclusion (1/1)

22

• Conclusion
• We present the SUNet architecture which is based on the Swin

Transformer and achieve the competitive results on denoising.

​

• we propose the dual up-sample module to avoid the checkerboard

artifacts.

​

• The potential of Swin Transformer still deserves to be expected in the future.

​

• Our future works are going to attempt more complex restoration tasks, such as real-world noise and real-world motion-blur.

23

Thanks for listening