PCEvE: Part Contribution Evaluation Based Model Explanation�for Human Figure Drawing Assessment and Beyond

​

^{{ jong980812, jinwoochoi }@khu.ac.kr}

Jongseo Lee

Jinwoo choi^†

What is the (HFD) Human Figure Drawing ?

What is the HFD assesment ?

The House-Tree-Person test is not valid for the prediction of mental health: An empirical study using deep neural networks, Yijing Lin et al

Zalsman, Gil, et al. "Human figure drawings in the evaluation of severe adolescent suicidal behavior." Journal of the American Academy of Child & Adolescent Psychiatry 39.8 (2000): 1024-1031

Rakhmanov, Ochilbek, Nwojo Nnanna Agwu, and Steve Adeshina. "Experimentation on hand drawn sketches by children to classify Draw-a-Person test images in psychology." The Thirty-Third International Flairs Conference. 2020.

​

ASD-TD Screening

Deep learning can be applied to HFD assessments

ASD-TD Screening

Attribution methods require human interpretation of semantic information

“Woman”

Attribution methods require human interpretation of semantic information

Which part

is important?

Our PCEvE can identify which parts are more/most important

Which part

is important?

PCEvE

Which part

is important?

Hair

S-PSV can quantify the contribution of each part of the input image

Sample-Level PCEvE

Part Sampling

Part Detection

Model

Shapley Value

(a)

S-PSV can quantify the contribution of each part of the input image

Sample-Level PCEvE

Part Sampling

Part Detection

Model

Shapley Value

(a)

We can obtain Group-Level contribution

(a)

Task: SCAT-Drawing Class: [Male, Female] Parts: [Hair, Eye, Nose, Mouth, Ear, Hand, Foot]

Hair Eye Hand Foot

Class-Level PCEvE

Sample-Level PCEvE

We can obtain Group-Level contribution

(a)

Task: SCAT-Drawing Class: [Male, Female] Parts: [Hair, Eye, Nose, Mouth, Ear, Hand, Foot]

Class-Level PCEvE

Hair Eye Hand Foot

Task-Level PCEvE

ASD/TD: Sample-level part-based model explanation

ASD/TD:Group-Level part-based model explanation

SCAT:Task-Level part-based model explanation

"The gender of the person who drew the drawing"

"The gender of the person in the drawing"

Extension to Photo-realistic Fine-grained Visual Categorization: Stanford Cars

Summary

• Enhanced Explainability: Provides clearer and more intuitive part contribution histograms compared to traditional pixel-level explanations, improving the interpretability of model decisions.

​

• Multi-Level Analysis: Offers insights at the sample, class, and task levels, providing a comprehensive understanding of model behavior.

​

• Extended Applicability: Demonstrates versatility beyond HFD assessment by successfully applying PCEvE to the photo-realistic Stanford Cars dataset.

​

• Use of Shapley Value: Employs the Shapley Value, a fair and model-agnostic metric, to accurately assess the contribution of each part to model decisions.

• Complexity and Computational Cost: Calculating the Shapley Value involves evaluating all possible part combinations, which can be computationally expensive.

​

• Dependency on Data Quality: The framework relies heavily on the quality and accuracy of part annotations in the datasets.

​

• User Interpretation Required: Despite offering more intuitive explanations, some users may still require additional interpretation to fully understand the results.

​

• Part Detection Errors: Errors in the automatic part detection process can impact the accuracy of the final explanations.

​

Q&A