1 of 5

Given a query image “I” for which a classifier model predicts class “c”, a counterfactual visual explanation identifies how “I” could change such that the system would output a different specified class c′ .

​

Interpretability using Diffusion

Rushikesh Zawar, Zhili Feng, Yutong He, Zico Kolter�rzawar@andrew.cmu.edu

Motivation

Visual Counterfacts

Guiding conditions

​

Conclusion & Future Work

With increasing use and integration of deep learning models in almost every aspect, a deeper understanding of their working becomes quite important. Diffusion models with their capabilities to generate images, can also be used to understand or even improve other models using Counterfactual Explanations (CFE). CFEs might assist in comprehending the rationale behind a classifier model's decisions and gaining insights into the determinations made by black-box neural network models. This could help evaluate a model especially for applications where false positives have high costs and thus evaluate their dependability.

References

Fig 1: Finding a region in a distractor image [1]

  • Visual Counterfacts need to be understandable or explainable, whereas adversarial examples mostly have noise indistinguishable for the human eye.
  • Counterfactual explanations seek to reveal the learned correlations related to the model’s decisions.

Visual Counterfact

Adversarial example

≠

  • Generating base image and CFE gives more control on the instance and class.
  • Since both images are generated from same model, the problem of difference in frequency domain is avoided.
  • Different models can be used to extend the variations and the method can be used with any field specific diffusion model too and is not restricted to any particular diffusion method

​

​

Fig 4: Examples and their desired counterfactuals

​

Fig 2: CFE generation

  1. Perceptual Similarity: To keep the generated image close to original image.
  2. Classification change:: To guide the image to be classified as the desired class.
  3. Adversarial Robustness: To avoid invisible changes that can change the classification.
  • Changing description at different steps & onwards can have varying effects.
  • Totally replacing a description i.e. a hard switch can have extreme effects. Instead a soft combination can help transition smoothly and generate more aligned image.
  1. Goyal, Y., Wu, Z., Ernst, J., Batra, D., Parikh, D., & Lee, S. (2019, May). Counterfactual visual explanations. In International Conference on Machine Learning (pp. 2376-2384). PMLR.
  2. Augustin, M., Boreiko, V., Croce, F., & Hein, M. (2022). Diffusion visual counterfactual explanations. Advances in Neural Information Processing Systems, 35, 364-377.
  3. Farid, K., Schrodi, S., Argus, M., & Brox, T. (2023). Latent Diffusion Counterfactual Explanations. arXiv preprint arXiv:2310.06668.
  4. Jeanneret, G., Simon, L., & Jurie, F. (2022). Diffusion models for counterfactual explanations. In Proceedings of the Asian Conference on Computer Vision (pp. 858-876)
  5. Wu, Q., Liu, Y., Zhao, H., Kale, A., Bui, T., Yu, T., ... & Chang, S. (2023). Uncovering the disentanglement capability in text-to-image diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition.
  6. Zawar, R., Dewan, S., Saxena, P., Chang, Y., Luo, A., & Bisk, Y. (2024). DiffusionPID: Interpreting Diffusion via Partial Information Decomposition. arXiv preprint arXiv:2406.05191.

​

Positive Samples

Non ideal Samples

Cheetah

Leopard

Cheetah

Tiger

Labrador

Golden Retriever

Loggerhead turtle

Box Turtle

Loggerhead turtle

Terrapin turtle

Cheetah

Cheetah

No change in classification

Changed to unintended class

  • Improving the efficiency as it requires multiple iterations of the reverse process to create one sample, making it very slow.
  • Building a method that need not rely on a robust model during the creation of CFEs and still avoid generating adversarial examples.
  • Use it for downstream applications like causal relation or mutual interaction detection.

Eq.1 : Modified description

Fig.3 : Magnified visual of feature change

Cheetah

Leopard

Fig.6 : Partial Information Decomposition[6]

Fig 5: Samples with their unintended counterfactuals

​

Diffusion for Counterfactual Explanation (CFE)

​

2 of 5

Given a query image “I” for which a classifier model predicts class “c”, a counterfactual visual explanation identifies how “I” could change such that the system would output a different specified class c′ .

​

Interpretability using Diffusion

Rushikesh Zawar, Zhili Feng, Yutong He, Zico Kolter�rzawar@andrew.cmu.edu

Motivation

Visual Counterfacts

Diffusion for Counterfactual Explanations (CFE)

​

Conclusion & Future Work

With increasing use and integration of deep learning models in almost every aspect, a deeper understanding of their working becomes quite important. Diffusion models with their capabilities to generate images, can also be used to understand or even improve other models using Counterfactual Explanations (CE). CEs might assist in comprehending the rationale behind a classifier model's decisions and gaining insights into the determinations made by black-box neural network models. This could help evaluate a model especially for applications where false positives have high costs and thus evaluate their dependability.

References

Fig 1: Finding a region in a distractor image [1]

​

  • Visual Counterfacts need to be understandable or explainable, whereas adversarial examples mostly have noise indistinguishable for the human eye.

​

  • Counterfactual explanations seek to reveal the learned correlations related to the model’s decisions.

Visual Counterfact

Adversarial Example

≠

  • Generating base image and CFE gives more control on the instance and class.
  • Since both images are generated from same model, the problem of difference in frequency domain is avoided.
  • Different models can be used to extend the variations and the method can be used with any field specific diffusion model too and is not restricted to any particular diffusion method

​

​

Fig 5: Generated Examples and their Counterfactuals

​

Fig 2: CFE generation

  • Perceptual Loss: To keep the generated image close to original image
  • Classifier Loss: To guide the change towards the desired image
  • Adversarial Robust loss: To avoid invisible changes that change the classification

Chancing description at different steps onwards can have varying effects. Further totally replacing a description i.e. a hard switch can have adverse effects. So instead a soft combination can help transition smoothly and generate more aligned image.

  • Ho, J., Jain, A., & Abbeel, P. (2020). Denoising diffusion probabilistic models. Advances in neural information processing systems, 33, 6840-6851
  • Rombach, Robin, et al. "High-resolution image synthesis with latent diffusion models." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.
  • Augustin, M., Boreiko, V., Croce, F., & Hein, M. (2022). Diffusion visual counterfactual explanations. Advances in Neural Information Processing Systems, 35, 364-377.
  • Farid, K., Schrodi, S., Argus, M., & Brox, T. (2023). Latent Diffusion Counterfactual Explanations. arXiv preprint arXiv:2310.06668.
  • Jeanneret, G., Simon, L., & Jurie, F. (2022). Diffusion models for counterfactual explanations. In Proceedings of the Asian Conference on Computer Vision (pp. 858-876)
  • Wu, Q., Liu, Y., Zhao, H., Kale, A., Bui, T., Yu, T., ... & Chang, S. (2023). Uncovering the disentanglement capability in text-to-image diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Positive Samples

Non ideal Samples

Cheetah

Leopard

Cheetah

Tiger

Labrador

Golden Retriever

Loggerhead turtle

Box Turtle

Loggerhead turtle

Terrapin turtle

Cheetah

Cheetah

No change in classification

Changed to unintended class

Improving the efficiency as it requires multiple iterations of the reverse process to create one sample, making it very slow.

​

Building a method that need not rely on a robust model during the creation of CEs and still avoid generating adversarial examples.��Use it for downstream applications like casual relation or mutual interaction detection.

Eq.1 : Modified description

3 of 5

Given a query image “I” for which a classifier model predicts class “c”, a counterfactual visual explanation identifies how “I” could change such that the system would output a different specified class c′ .

​

Interpretability using Diffusion

Rushikesh Zawar, Zhili Feng, Yutong He, Zico Kolter�rzawar@andrew.cmu.edu

Motivation

Visual Counterfacts

Diffusion for Counterfactual Explanations (CE)

​

Conclusion & Future Work

With increasing use and integration of deep learning models in almost every aspect, a deeper understanding of their working becomes quite important. Diffusion models with their capabilities to generate images, can also be used to understand or even improve other models using Counterfactual Explanations (CE). CEs might assist in comprehending the rationale behind a classifier model's decisions and gaining insights into the determinations made by black-box neural network models. This could help evaluate a model especially for applications where false positives have high costs and thus evaluate their dependability.

References

Fig 2: Finding a region in a distractor image [1]

Fig 4: CFE generation

​

  • Visual Counterfacts need to be understandable or explainable, whereas adversarial examples mostly have noise indistinguishable for the human eye.

​

  • Counterfactual explanations seek to reveal the learned correlations related to the model’s decisions.

Visual Counterfact

Adversarial Example

≠

Improving the efficiency as it requires multiple iterations of the reverse process to create one sample, making it very slow.

​

Building a method that need not rely on a robust model during the creation of CEs and still avoid generating adversarial examples.��Use it for downstream applications like casual relation or mutual interaction detection.

  • Generating base image and CFE gives more control on the instance and class.
  • Since both images are generated from same model, the problem of difference in frequency domain is avoided.
  • Different models can be used to extend the variations and the method can be used with any field specific diffusion model too and is not restricted to any particular diffusion method

​

​

Fig 5: Generated Examples and their Counterfactuals

​

Fig 3: CNN as a feature extractor, with replaced features as input to classifier [1]

  • Perceptual Loss
  • Classifier Loss
  • CLIP loss

METHOD

Loss

  • Ho, J., Jain, A., & Abbeel, P. (2020). Denoising diffusion probabilistic models. Advances in neural information processing systems, 33, 6840-6851
  • Rombach, Robin, et al. "High-resolution image synthesis with latent diffusion models." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.
  • Augustin, M., Boreiko, V., Croce, F., & Hein, M. (2022). Diffusion visual counterfactual explanations. Advances in Neural Information Processing Systems, 35, 364-377.
  • Farid, K., Schrodi, S., Argus, M., & Brox, T. (2023). Latent Diffusion Counterfactual Explanations. arXiv preprint arXiv:2310.06668.
  • Jeanneret, G., Simon, L., & Jurie, F. (2022). Diffusion models for counterfactual explanations. In Proceedings of the Asian Conference on Computer Vision (pp. 858-876)
  • Wu, Q., Liu, Y., Zhao, H., Kale, A., Bui, T., Yu, T., ... & Chang, S. (2023). Uncovering the disentanglement capability in text-to-image diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Positive Samples

Negative Samples

Cheetah

Leopard

Cheetah

Tiger

Labrador

Golden Retriever

Loggerhead turtle

Box Turtle

Garden spider

Black widow spider

Cheetah

Cheetah

No change in classification

No change in classification

4 of 5

Diffusion for Counterfactual Explanations (CE)

Given a query image “I” for which a classifier model predicts class “c”, a counterfactual visual explanation identifies how “I” could change such that the system would output a different specified class c′ .

​

Interpretability using Diffusion

Rushikesh Zawar, Zhili Feng, Yutong He, Zico Kolter�rzawar@andrew.cmu.edu

Motivation

Visual Counterfacts

Diffusion

Conclusion & Future Work

With increasing use and integration of deep learning models in almost every aspect, a deeper understanding of their working becomes quite important. Diffusion models with their capabilities to generate images, can also be used to understand or even improve other models using Counterfactual Explanations (CE). CEs might assist in comprehending the rationale behind a classifier model's decisions and gaining insights into the determinations made by black-box neural network models. This could help evaluate a model especially for applications where false positives have high costs and thus evaluate their dependability.

References

  • Goyal, Y., Wu, Z., Ernst, J., Batra, D., Parikh, D., & Lee, S. (2019, May). Counterfactual visual explanations. In International Conference on Machine Learning (pp. 2376-2384). PMLR.
  • Ho, J., Jain, A., & Abbeel, P. (2020). Denoising diffusion probabilistic models. Advances in neural information processing systems, 33, 6840-6851
  • Rombach, Robin, et al. "High-resolution image synthesis with latent diffusion models." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.
  • Augustin, M., Boreiko, V., Croce, F., & Hein, M. (2022). Diffusion visual counterfactual explanations. Advances in Neural Information Processing Systems, 35, 364-377.
  • Farid, K., Schrodi, S., Argus, M., & Brox, T. (2023). Latent Diffusion Counterfactual Explanations. arXiv preprint arXiv:2310.06668.
  • Jeanneret, G., Simon, L., & Jurie, F. (2022). Diffusion models for counterfactual explanations. In Proceedings of the Asian Conference on Computer Vision (pp. 858-876).

Fig 1: Ex. of Counter Factual Explanation [4]

Fig 2: Finding a region in a distractor image [1]

Fig 4: Working of Diffusion Model

The input image is converted into a latent code, and a random noise vector is generated. The forward diffusion process gradually corrupts the input image/latent code, while the reverse diffusion process gradually refines the corrupted image back to the original. The diffusion steps and decoder network are trained to minimize a reconstruction loss function which ensures that the output image is similar to the input image.

​

  • Visual Counterfacts need to be understandable or explainable, whereas adversarial examples mostly have noise indistinguishable for the human eye.

​

  • Counterfactual explanations seek to reveal the learned correlations related to the model’s decisions.

Some works like [1] approach this problem by, selecting a ‘distractor’ image I′ that the system predicts as class c′ and identifies spatial regions in I and I′ such that replacing the corresponding region in I with with its counterpart in I′ would push the system towards classifying I as c′.

​

​

Visual Counterfact

Adversarial Example

≠

Improving the efficiency as it requires multiple iterations of the reverse process to create one sample, making it very slow.

​

Building a method that need not rely on a robust model during the creation of CEs and still avoid generating adversarial examples.

  • Due to diffusion models’ probabilistic nature, they produce a diverse set of images.
  • Stochasticity is ideal for CEs as multiple explanations may better describe a classifier’s error modes.
  • To avoid generating adversarial examples instead, often such methods use adversarially robust classifier models.

Fig 5: Diffusion models for counterfactual explanations[6]

Fig 3: CNN as a feature extractor, with replaced features as input to classifier [1]

5 of 5

Diffusion for Counterfactual Explanations (CE)

Given a query image “I” for which a classifier model predicts class “c”, a counterfactual visual explanation identifies how “I” could change such that the system would output a different specified class c′ .

​

Interpretability using Diffusion

Rushikesh Zawar, Zhili Feng, Yutong He, Zico Kolter�rzawar@andrew.cmu.edu

Motivation

Visual Counterfacts

Diffusion

Conclusion & Future Work

With increasing use and integration of deep learning models in almost every aspect, a deeper understanding of their working becomes quite important. Diffusion models with their capabilities to generate images, can also be used to understand or even improve other models using Counterfactual Explanations (CE). CEs might assist in comprehending the rationale behind a classifier model's decisions and gaining insights into the determinations made by black-box neural network models. This could help evaluate a model especially for applications where false positives have high costs and thus evaluate their dependability.

References

  • Goyal, Y., Wu, Z., Ernst, J., Batra, D., Parikh, D., & Lee, S. (2019, May). Counterfactual visual explanations. In International Conference on Machine Learning (pp. 2376-2384). PMLR.
  • Ho, J., Jain, A., & Abbeel, P. (2020). Denoising diffusion probabilistic models. Advances in neural information processing systems, 33, 6840-6851
  • Rombach, Robin, et al. "High-resolution image synthesis with latent diffusion models." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.
  • Augustin, M., Boreiko, V., Croce, F., & Hein, M. (2022). Diffusion visual counterfactual explanations. Advances in Neural Information Processing Systems, 35, 364-377.
  • Farid, K., Schrodi, S., Argus, M., & Brox, T. (2023). Latent Diffusion Counterfactual Explanations. arXiv preprint arXiv:2310.06668.
  • Jeanneret, G., Simon, L., & Jurie, F. (2022). Diffusion models for counterfactual explanations. In Proceedings of the Asian Conference on Computer Vision (pp. 858-876).

Fig 1: Ex. of Counter Factual Explanation [4]

Fig 2: Finding a region in a distractor image [1]

Fig 4: Working of Diffusion Model

The input image is converted into a latent code, and a random noise vector is generated. The forward diffusion process gradually corrupts the input image/latent code, while the reverse diffusion process gradually refines the corrupted image back to the original. The diffusion steps and decoder network are trained to minimize a reconstruction loss function which ensures that the output image is similar to the input image.

​

  • Visual Counterfacts need to be understandable or explainable, whereas adversarial examples mostly have noise indistinguishable for the human eye.

​

  • Counterfactual explanations seek to reveal the learned correlations related to the model’s decisions.

Some works like [1] approach this problem by, selecting a ‘distractor’ image I′ that the system predicts as class c′ and identifies spatial regions in I and I′ such that replacing the corresponding region in I with with its counterpart in I′ would push the system towards classifying I as c′.

​

​

Visual Counterfact

Adversarial Example

≠

Improving the efficiency as it requires multiple iterations of the reverse process to create one sample, making it very slow.

​

Building a method that need not rely on a robust model during the creation of CEs and still avoid generating adversarial examples.

  • Due to diffusion models’ probabilistic nature, they produce a diverse set of images.
  • Stochasticity is ideal for CEs as multiple explanations may better describe a classifier’s error modes.
  • To avoid generating adversarial examples instead, often such methods use adversarially robust classifier models.

Fig 5: Diffusion models for counterfactual explanations[6]

Fig 3: CNN as a feature extractor, with replaced features as input to classifier [1]