FOLD-R++ Data Analysis

Alex Kempen and Egan Johnson

FOLD-R++

• Fold-R++ is a machine learning model
• Generates binary predicate function to classify data
• The below predicate function shows that a printer is `good` if it is:
• not an ink printer
• Costs $95.99 or less

label(X,'good') :- not type(X,'ink'), not ab1(X).

ab1(X) :- cost(X,N0), N0>95.99.

Accuracy: 0.5 p value: 0 r value: 0.0 F1 Score: 0

​

Procedure - Analysis

• We wrote various methods to improve interactions with data and speed up analysis process
• Methods include:
• Identification of label column
• Automatic column classification based on data

def classify_columns(attrs, data):

numerical = []

for attr in (attrs):

i = attrs.index(attr)

if (is_column_numerical(data[:, i])):

numerical.append(attr)

return numerical

​

def is_column_numerical(column):

count = 0

for val in column:

if val == '?':

pass

try:

float(val)

count += 1

except ValueError:

pass

# true if nums are more than 50% of column

return count > column.size * 0.5

Methods to classify columns based on data

Dataset 1 (Graduation)

• Predicts graduation of students at a University in Indonesia based on their GPA over their first four semesters
• Data set is unbalanced (Only 8% of students don’t graduate)
• Total data points: 1687
• Best ratio: 0.5

label(X,'yes') :- sem4(X,N3), N3>3.35, not ab1(X).

ab1(X) :- sem2(X,N1), N1>3.25, sem2(X,N1), N1=<3.28, sem3(X,N2), N2>3.05, sem3(X,N2), N2=<3.16.

label(X,'yes') :- sem4(X,N3), N3=<3.35, not ab2(X), not ab6(X).

ab2(X) :- sem1(X,N0), N0=<2.91, sem2(X,N1), N1=<2.5, sem3(X,N2), N2=<2.95, sem4(X,N3), N3>3.12.

ab3(X) :- sem4(X,N3), N3>3.12.

ab4(X) :- sem2(X,N1), N1=<1.97.

ab5(X) :- sem1(X,N0), N0>2.21, sem1(X,N0), N0=<2.52, sem4(X,N3), N3=<2.73, not ab4(X).

ab6(X) :- sem1(X,N0), N0>1.75, sem1(X,N0), N0=<2.91, sem2(X,N1), N1=<3.08, sem3(X,N2), N2>1.4, sem3(X,N2), N2=<2.64, not ab3(X), not ab5(X).

Accuracy: 0.932 p value: 0.9428 r value: 0.9874 F1 Score: 0.9646

Dataset 1 - Analysis

• Accuracy value of 0.932 is good considering inherent imbalance of data set
• Algorithm uses all four semesters, but expectedly relies on later semesters more heavily
• Ratio did not have much effect overall

label(X,'yes') :- sem4(X,N3), N3>3.35, not ab1(X).

ab1(X) :- sem2(X,N1), N1>3.25, sem2(X,N1), N1=<3.28, sem3(X,N2), N2>3.05, sem3(X,N2), N2=<3.16.

label(X,'yes') :- sem4(X,N3), N3=<3.35, not ab2(X), not ab6(X).

ab2(X) :- sem1(X,N0), N0=<2.91, sem2(X,N1), N1=<2.5, sem3(X,N2), N2=<2.95, sem4(X,N3), N3>3.12.

ab3(X) :- sem4(X,N3), N3>3.12.

ab4(X) :- sem2(X,N1), N1=<1.97.

ab5(X) :- sem1(X,N0), N0>2.21, sem1(X,N0), N0=<2.52, sem4(X,N3), N3=<2.73, not ab4(X).

ab6(X) :- sem1(X,N0), N0>1.75, sem1(X,N0), N0=<2.91, sem2(X,N1), N1=<3.08, sem3(X,N2), N2>1.4, sem3(X,N2), N2=<2.64, not ab3(X), not ab5(X).

Accuracy: 0.932 p value: 0.9428 r value: 0.9874 F1 Score: 0.9646

Dataset 2 (Smoker)

label(X,'smoker') :- gene35715(X,N1), N1>-2.052.

label(X,'smoker') :- gene35715(X,N1), N1=<-2.052, not ab1(X).

label(X,'smoker') :- gene35715(X,'na').

ab1(X) :- gene2337(X,N0), N0>1.17, gene2337(X,N0), N0=<1.339.

Accuracy: 0.8341 p value: 0.8341

r value: 1.0 F1 Score: 0.9096

​

• Predicts smoking status from cancer status and genetic information
• Total data points: 1023
• Best ratio: 0.45

Dataset 2 - Analysis

• Rules are able to draw conclusions about combinations of gene activations
• Rules do not mention cancer status
• Smoking behavior determined by genes exclusively- against our expectations
• Possible indication that the algorithm cannot reject bias

label(X,'smoker') :- gene35715(X,N1), N1>-2.052.

label(X,'smoker') :- gene35715(X,N1), N1=<-2.052, not ab1(X).

label(X,'smoker') :- gene35715(X,'na').

ab1(X) :- gene2337(X,N0), N0>1.17, gene2337(X,N0), N0=<1.339.

Accuracy: 0.8341 p value: 0.8341

r value: 1.0 F1 Score: 0.9096

​

Dataset 3 (Water Potability)

• This dataset describes whether water is potable
• Potability depends only on whether each value falls below a given WHO threshold
• The exception to this is pH, which is allowed to range from 6.5 to 8.5
• Total data points: 3276
• Best ratio: 0.8

label(X,'0') :- hardness(X,N1), N1>98.368, not ab16(X), not ab26(X), not ab27(X), not ab28(X).

ab1(X) :- hardness(X,N1), N1=<154.266.

ab2(X) :- trihalomethanes(X,N7), N7=<53.671.

ab3(X) :- hardness(X,N1), N1>236.099, not ab2(X).

ab4(X) :- ph(X,N0), N0=<7.313.

ab5(X) :- ph(X,N0), N0=<7.904, turbidity(X,N8), N8=<3.137, not ab4(X).

ab6(X) :- ph(X,N0), N0>7.312, not ab1(X), not ab3(X), not ab5(X).

ab7(X) :- solids(X,N2), N2=<26912.228, not ab6(X).

ab8(X) :- hardness(X,N1), N1=<174.351.

ab9(X) :- ph(X,N0), N0>7.263, not ab8(X).

ab10(X) :- hardness(X,N1), N1=<187.873.

ab11(X) :- hardness(X,N1), N1=<196.65, not ab10(X).

ab12(X) :- hardness(X,N1), N1>173.382, solids(X,N2), N2>27172.894, sulfate(X,N4), N4>258.444, sulfate(X,N4), N4=<279.767, not ab9(X), not ab11(X).

(16 rules omitted)

Accuracy: 0.654 p value: 0.6464

r value: 0.9704 F1 Score: 0.7759

​

Dataset 3 Analysis

label(X,'0') :- hardness(X,N1), N1>98.368, not ab16(X), not ab26(X), not ab27(X), not ab28(X).

ab1(X) :- hardness(X,N1), N1=<154.266.

ab2(X) :- trihalomethanes(X,N7), N7=<53.671.

ab3(X) :- hardness(X,N1), N1>236.099, not ab2(X).

ab4(X) :- ph(X,N0), N0=<7.313.

ab5(X) :- ph(X,N0), N0=<7.904, turbidity(X,N8), N8=<3.137, not ab4(X).

ab6(X) :- ph(X,N0), N0>7.312, not ab1(X), not ab3(X), not ab5(X).

ab7(X) :- solids(X,N2), N2=<26912.228, not ab6(X).

ab8(X) :- hardness(X,N1), N1=<174.351.

ab9(X) :- ph(X,N0), N0>7.263, not ab8(X).

ab10(X) :- hardness(X,N1), N1=<187.873.

ab11(X) :- hardness(X,N1), N1=<196.65, not ab10(X).

ab12(X) :- hardness(X,N1), N1>173.382, solids(X,N2), N2>27172.894, sulfate(X,N4), N4>258.444, sulfate(X,N4), N4=<279.767, not ab9(X), not ab11(X).

(16 rules omitted)

Accuracy: 0.654 p value: 0.6464

r value: 0.9704 F1 Score: 0.7759

​

• Overall accuracy of 0.654 was less than expected
• Algorithm was unable to reverse engineer WHO criteria
• Tendency to tie conditions together in complex ways, which hurt effectiveness
• For this data set, a column by column approach would likely be more effective

Conclusion

• Pros:
• Explainability makes interpretation straightforward
• Low data pretreatment requirements
• Allows for tools to be built for rapid analysis of many datasets
• Cons:
• Can struggle with certain types of data sets
• Sparse or lopsided sets
• Independent columns
• Non-linear trends

References

FOLD R++:

https://github.com/hwd404/FOLD-R-PP

Dataset 1 (Graduation):

https://www.kaggle.com/oddyvirgantara/on-time-graduation-classification

Dataset 2 (Smoking):

https://www.kaggle.com/devzohaib/smoker-condition

Dataset 3 (Water Potability):

https://www.kaggle.com/adityakadiwal/water-potability