1 of 8

Team Number 18:

Bhanu Kiran Verma (CE21B031)

Navin Sriram (ED21B044)

Anirud N (CE21B014)

Pranav Debbad (ED21B046)

EE5179: Deep Learning for Imaging

Project Presentation

Image restoration from noisy & blurred images by applying UNet-based Generator in cGAN to preserve defects of interest

​

2 of 8

Approach

  • Requirement: �Denoise degraded images while maintaining the integrity of defect patterns given a defect mask
  • Literature Survey:
    • Image Processing Techniques - NLM, BM3D
    • CNN-based Architecture - DnCNN, UNet, DRUNet, Masked AE, MPRNet
    • GAN-based models - Pix2Pix-UNet, Pix2Pix-ResNet, CycleGAN
  • Ensemble Learning:
    • Fusion Encoder: UNet, Median blur, LPF-FFT, Bilateral Filter, TV Chambolle denoising

Team 18 Project Presentation

2

​

    • DeblurGAN + Deep UNet

3 of 8

UNet

  • LR: 1e-4
  • Optimizer: Adam
  • Activation: ReLU
  • Trained 50 epochs

​

​

Team 18 Project Presentation

3

Skip connections

Bottleneck layer

Encoder

Decoder

  • Validation PSNR: 28.87
  • Validation SSIM : 0.87

​

  • Produces discolouration in the output images

​

4 of 8

Fusion Encoder

  • 5 input images - 13 input channels
    • 1 pre-trained UNet (30 epochs)
    • 4 traditional denoising techniques
  • 8 conv layers - output 3 channels�ReLU activation at each hidden layer
  • Adam optimizer, MSE loss
  • Random artifacts are generated by GAN

​

Team 18 Project Presentation

4

5 of 8

UNet-based DeblurGAN Architecture

  • Discriminator: �To distinguish between real & generated images
    • conv layers (F=3): 3 → 64 → 128 → 256 → 128 → 64 → 16→ 1
      • LeakyReLU between layers
      • BatchNorm after conv2, conv3
    • fc1 (529 → 100), fc2 (100 → 1)
    • Sigmoid activation on output (closer to 1 indicates real)
  • UNet-based Generator: �To generate images that fool the discriminator
    • Degraded image fed into UNet
    • UNet output fed into encoder
      • conv layers (F=3) with ReLU (3→ 64 → 128 → 256)
    • Bottleneck - 1 connecting conv layer (256 → 256), ReLU activation
    • Decoder - Transposed conv layers (256 → 128 → 64 filters)
    • Output layer - transposed conv (64 → 3), tanh activation

​

​

​

​

​

Team 18 Project Presentation

5

6 of 8

UNet-based DeblurGAN Training

  • Discriminator Loss Function: Real loss + Fake loss.
    • Real loss: BCE(discriminator(clean_images), [ones])
    • Fake loss: BCE(discriminator(generated_images), [zeros])

Training results in better classification between real, generated images.

  • Generator Loss function: 0.01*𐊣Gan + 100*𐊣1 + 100*𐊣2 + 2000*𐊣3
    • 𐊣gan: BCE(discriminator(generated_images), [ones])
    • 𐊣1: L2 loss from clean image to maintain overall image quality
    • 𐊣2: L1 loss in mask region from noisy image to preserve defect
    • 𐊣3: L2 loss from UNet output (to minimize artifact generation)

Training results in produced images looking closer to real (clean) images.

  • Optimizer: Adam
  • Learning rate: 1e-4
  • Early-stopping of GAN at 5th epoch to remove random discoloration
  • UNet pre-trained for 30 epochs

Team 18 Project Presentation

6

7 of 8

UNet-based DeblurGAN

Observations:

The addition of DeblurGAN:

  • removes discolouration in UNet output
  • slightly deblurs UNet output
  • minimizes artifact generation

​

Team 18 Project Presentation

7

  • Val PSNR: 28.45
  • Val SSIM: 0.93

8 of 8

UNet-based DeblurGAN

Team 18 Project Presentation

8

Test Images Comparison

Overall

  • Test PSNR: 28.5
  • Test SSIM: 0.925

​

Masked Region

  • Test PSNR: 24.80
  • Test SSIM: 0.995

Failure Cases