Aydin Denoising of the Noisy ‘New York’ Test Image

Here we go through the denoising of the noisy ‘New York’ test image. This is an interesting image because it has highly structured and fine regions that pose challenges for disentangling noise from high-frequency image context. It also has different ‘regions’ with a smooth horizon at the top, regular window grids,  very dark regions at the bottom, and also some very bright uniform regions on the roofs. This is probably one of our favorite test images.

Noisy New York test image:

Image: newyork_noisy.tif
Ground Truth:

Source: Original photograph taken by Loic A. Royer

Top left: ground truth image. Top right: Noisy image.
Bottom left: ground truth image crop.  Bottom right: Noisy image crop.

First, as usual, we try the simplest and probably the fastest of all algorithms available in Aydin: Butterworth denoising:

Default parameters for Butterworth denoiser (as of 25th May 2022).

The default parameters are fine for a first trial. We obtain:

Top: ground truth image crop. Bottom left: raw noisy image crop.

Bottom right: image crop denoised with default Butterworth parameters (as of 25th May 2022)

The Butterworth denoiser is successfully calibrated after 1.78 minutes. And the optimal parameters are:

Screenshot of activity window after calibration. Optimal parameters found can be seen.

The important parameters are: the cut-off frequencies along x and y as well as the order (sharpness) of the filter. The cut-off frequencies are normalized to [0,1]. Higher cut-off values means less filtering and thus less denoising needed. Higher orders means that the filter is sharper. The first observation of the ndenoised image reveals that the filtering is anisotropic: more pronounced along x than y. Indeed the two cut-off values are not the same, the cut-off along x (freq_cutoff_1) is lower than along y (freq_cutoff_0). This is probably caused by some of the pronounced horizontal lines in the image which cause the calibration to over-filter horizontally.  Since we know that our image is fundamentally isotropic, we can enforce that during denoising by choosing the mode: ‘isotropic’:

Screenshot of the denoise tab with Butterworth selected and mode parameter set to ‘isotropic’.

Since there is only one cut-off frequency to determine, the optimization runs faster at 1.6 minutes:

And the result is noticeably better:

Top: ground truth image crop. Bottom left: raw noisy image crop.
Bottom right: image crop denoised with isotropic Butterworth (as of 25th May 2022)

Can we do better? Of course. For example if you go to the menu item preferences, you can turn on advanced features which also reveals some more advanced algorithms. Among these you can find the  ‘fixed dictionary denoising’ approach:

Left: Switch to advanced mode. Right: Select from the longer list of algorithms.  

Applying that algorithm with its default parameters (recommended) gives:

Top: ground truth image crop. Bottom left: raw noisy image crop.
Bottom right: image crop denoised with fixed dictionary denoising (dct).

We see that the very regular parts of the image such as the regular window and balcony grids become very clean and sharp. However, in the areas of the image devoid of structure, the noise is ‘interpreted’ as structure and amplified. See the water, and some of the roofs. This is a case of hallucination and is a serious problem when using this method for very noisy images.

Next, we try the ‘learned dictionary denoising’ algorithm:

Top left: ground truth image crop. Top right: raw noisy image crop.

 Bottom left: image crop denoised with fixed dictionary denoising (dct).
Bottom right:  image crop denoised with learned dictionary.

Learning the dictionary of image patches improves the outcome: more detail and perhaps a bit less artefacts too, but the uniform regions in the image (e.g. water) still have lots of artefacts.

Can we do better? Yes, the algorithm to try, which is a bit slow and should be reserved when your images are small, you have a powerful CPU, and a good measure of patience, iks ‘spectral denoising’. This approach searches for sparse representations of image patches in the DCT ( or other) bases. Using the default parameters, we get:

Top left: ground truth image crop. Top right: raw noisy image crop.

 Bottom left: image crop denoised with learned dictionary denoising.
Bottom right:  image crop denoised with ‘spectral’ denoising.

Definitely better, we see less noise on the uniform regions of the image, and even cleaner denoising in detail-rich parts of the image such as windows. We could try all and each algorithm implemented in Aydin, but in the interest of brevity, lets do one last ‘sub-optimal’ approach before we move to more potent approaches: Total Variation (TV) denoising as implemented in scikit-image. Of course, as all Aydin algorithms we include auto-tuning. The result, using the default parameters in Aydin is:

Top left: ground truth image crop. Top right: raw noisy image crop.

 Bottom left:  image crop denoised with ‘spectral’ denoising.

. Bottom right: image crop denoised with Aydin’s tuned tv-denoiser.

As expected if you know a bit about this type of denoiser, the result is very artefactual: the denoised image is a piecewise constant approximation of the noisy image. Time to move to a potent approach:  Aydin’s Noise2Self-FGR-cb

Top left: ground truth image crop. Top right: raw noisy image crop.

 Bottom left:  image crop denoised with ‘spectral’ denoising.

. Bottom right: image crop denoised with Aydin’s Noise2Self-FGR-cb with default parameters.

The result is clearly much better: less artefacts in the uniform regions, better contrast and sharpness, and details are better preserved. For example, check the pyramidal roof at the right, there is a faint periodic pattern visible in the ground-truth that is almost visible in the denoised image. Previous attempts at denosing this roof (see above) failed by rendering a very artifactual and garbled texture.

What about CNN (Convolutional Neural Network) based denoisers? Frankly we rarely use these denoisers anymore, they train extremely slowly and the results are often very disappointing compared to Noise2Self-FGR-cb. See:

Top left: ground truth image crop. Top right: raw noisy image crop.

 Bottom left: image crop denoised with Aydin’s Noise2Self-FGR-cb with default parameters.
Bottom right: image crop denoised with a Noise2Self-CNN-Jinet denoiser.

Last updated 25th May 2022.