1 of 13

Finding Missing Values and Outliers using WEKA

Data Preprocessing and Quality Analysis

dr. Jamolbek Mattiev

2 of 13

Why Data Quality Matters

  • Missing values and outliers reduce model accuracy
  • They may bias machine learning results
  • Data preprocessing is a critical ML step

3 of 13

Understanding Missing Values

  • Missing values occur when data is not recorded
  • Represented as '?' or empty cells in WEKA
  • Common causes: human error, sensor failure, data corruption

4 of 13

Finding Missing Values in WEKA

  • Open WEKA Explorer → Preprocess tab
  • Load dataset (ARFF or CSV)
  • Check attribute statistics panel
  • Missing values are shown for each attribute

5 of 13

Visualizing Missing Values

  • Use attribute selection panel
  • Attributes with missing values show '?' count
  • Histogram view highlights gaps in data

6 of 13

Handling Missing Values

  • Filters → Unsupervised → Attribute → ReplaceMissingValues
  • Numeric: replaced by mean
  • Nominal: replaced by mode

7 of 13

What Are Outliers?

  • Outliers are data points significantly different from others
  • They may indicate errors or rare but important cases
  • Outliers affect mean and variance

8 of 13

Detecting Outliers in WEKA

  • Use Preprocess → Visualize
  • Scatter plots help identify extreme values
  • Box-like distributions reveal anomalies

9 of 13

Outlier Detection Using Filters

  • Filters → Unsupervised → Attribute → InterquartileRange
  • Identifies extreme and outlier instances
  • Creates new attributes for detection

10 of 13

Using Clustering for Outlier Detection

  • Cluster tab → SimpleKMeans
  • Small or distant clusters may indicate outliers
  • Useful for unlabeled data

11 of 13

Exercise: Practical Steps

  • Load dataset in WEKA
  • Identify attributes with missing values
  • Apply ReplaceMissingValues filter
  • Visualize attributes to find outliers
  • Apply InterquartileRange filter

12 of 13

Best Practices

  • Always inspect data before modeling
  • Do not remove outliers blindly
  • Domain knowledge is important

13 of 13

Conclusion

  • WEKA provides simple tools for data cleaning
  • Missing values and outliers can be detected visually and statistically
  • Proper preprocessing improves ML performance