c-GANalyzer project �

• XAI objective – Explain c-GANalyzer project using XAI methods.
• Quick recap of the project to be explained:

​

c-GANalyzer project

• In subsequent iterations, the discriminator receives mini batches of real and generated examples from the generator trained in the previous iterations. The ideal termination condition for this process is when both the generated examples are high-quality and the discriminator is unable to distinguish between “real” and “generated” examples

​

Generator’s performance

Explaining the Discriminator in c-GAN using XAI

• Purpose of the Discriminator:
• Distinguishes real vs. fake images conditioned on class labels.
• Learns to identify whether the image matches the label and resembles training data.

​

• Why Explain It?
• Understand what visual features influence its decisions.
• Detect if the model is overfitting or focusing on irrelevant parts.

​

• Approach:
• Use XAI tools like Grad-CAM, Saliency Maps, and Lime to visualize decision-making.

New approach – xAI-GAN

• Paper: xAI-GAN: Enhancing Generative Adversarial Networks via Explainable AI Systems by Vineel Nagisetty, Laura Graves, Joseph Scott and Vijay Ganesh at University of Waterloo
• Introduction of xAI-GAN, a new kind of Generative Adversarial Network (GAN) that incorporates Explainable AI (xAI) systems to improve training quality and data efficiency.
• What is the problem with the classical GANs?
• GANs require a lot of data for successful training. Humans can learn to replicate digits with a few examples whereas a standard GAN might need thousands. Collecting this high-quality data can be expensive and time-consuming.
• The feedback from the discriminator is a single number (loss). The loss tells the generator whether the discriminator fooled or not, but it doesn't explain why. The generator uses this single value to adjust its parameters through a process called gradient descent, aiming to produce examples that better fool the discriminator -> xAI-GAN

​

​

xAI-GAN

• Idea: Is to provide richer corrective feedback from the discriminator to the generator during training. This “richer” feedback comes from an xAI system.
• Instead of just telling the generator if a generated example was detected as fake (via the loss value), the xAI system explains the reason as to why the discriminator made its classification.

​

• Example:
• Instead of: "This image is fake. Loss: 1.2”
• xAI-GAN says: "This image is fake because of feature A, B and C. Here's how much each pixel contributed to the decision."

​

​

​

xAI-GAN

The xAI-GAN uses an xAI system to modify the standard gradient descent process used to train the generator.

​

​

​

GAN: How it works?

• TODO explain briefly how GAN itself works

xAI-GAN: How it works?

The xAI-GAN uses an xAI system to modify the standard gradient descent process used to train the generator.

• Discriminator evaluates the generated image and the loss is calculated, then the generated image, the discriminator and the discriminator's output are fed into and xAI system.
• The xAI system produces an explanation for the discriminator's decision:

​

​

​

​

​

​

• This explanation matrix guides the training of the generator. In the modified gradient descent, this matrix is used like a mask. It is combined with the standard gradient information such as:
• Standard Gradient: ∇G(z)
• Modified Gradient: ∇G(z) ∘ M (element-wise product with respect to the loss) - xAI-guided gradient descent - only the important parts decided by xAI system are emphasized during learning.

xAI-GAN: How it works?

Explenation methods – Saliency Maps

• Recap: Saliency Map (Simonyan, Vedaldi, and Zisserman 2013) compute the importance of each feature in a given input to the resulting classification by a DNN model. In order to compute a saliency, the loss of the prediction is computed with respect to target label y and used to perform backpropagation to the calculate the gradient ∇x. This is then normalized to produce the saliency map.
• Computes the gradient of the discriminator’s output with respect to each input pixel. Shows which pixels had the biggest impact on the discriminator’s decision.
• How it's applied in xAI-GAN:
• Feed the generated image G(z) to the discriminator D
• Compute the gradient of the discriminator's output with respect to the input image:
• Normalize the saliency values to [0, 1] to get the explanation matrix 𝑀
• Apply this matrix to the generator’s output gradient as a mask.

​

​

Explenation methods – LIME

• Recap: Lime (Ribeiro, Singh, and Guestrin 2016), short for Local Interpretable Mode Agnostic, is used to explain the predictions of a ML classifier by learning an interpretable local model. Given a DNN model M and input x, Lime creates a set of new N inputs x₁ , ..., x_N by slightly perturbing x. It then queries M on these new inputs to generate labels y₁, ..., y_N . The new inputs and labels are used to train a simple regression model which is expected to approximate M well in the local vicinity of x. The weights of this local model are used to determine the feature importance of x.
• How it's applied in xAI-GAN:
• Generate image x=G(z)
• Create perturbed images {x₁, x₂, ..., x_n} by masking patches (like blurring or blacking out regions).
• Get discriminator predictions for each perturbed image: D(x₁), D(x₂), ..., D(x_n)
• Train a simple linear model to approximate the discriminator's output locally.
• Extract weights from the local model → build explanation matrix 𝑀

​

Results in terms of FID metric

• Evaluation Metric: Frechet Inception Distance (FID) – lower means more realistic images
• How Does It Work?
• Both sets of images are passed through a pretrained Inception network (usually Inception-v3, a neural network trained on ImageNet).
• Instead of comparing raw pixels, FID compares the feature vectors from an internal layer of the network — these capture high-level visual characteristics.
• It models these feature vectors as multivariate Gaussian distributions and calculates the Fréchet distance between the two:

​

​

Results in terms of FID metric

Results & take-aways

• Improved Image Quality: xAI-GANs, particularly the version using the LIME system achived better image quality as measured by the FID score. Improvements of up to 23.18% in FID score were observed compared to standard GANs.
• It is believed that xAI-GANs show an improvement over standard GANs in terms of FID score even when using scarce data.
• While xAI-GAN requires more training time compared to standard GANs due to the overhead of the xAI system, we believe this is still an advantageous trade-off.
• Improved Data Efficiency: xAI-GANs demonstrated better performance even when trained on significantly less data than standard GANs. For example, xAI-GAN trained on only 20% of the CIFAR10 dataset still outperformed a standard GAN trained on 100% of the data in terms of FID score

​