1 of 10

1

The ECI website has this data.

2 of 10

2

The ECI website has this data.

3 of 10

3

… and, most of the data is in PDFs

4 of 10

4

The PDF files have a reasonably clear structure

5 of 10

5

… that translates into text that can be parsed

6 of 10

6

… which, with some effort, can be converted into a structured format

… and at this point, we need to start checking for errors.

7 of 10

7

At this point, we start checking what’s gone wrong

Each row here is one constituency.

The number of candidates that have contested in each constituency in every year is shown as a table.

You can see that some patterns emerge here.

8 of 10

8

Not every spelling error is easily identifiable by the first letter

Parties are mis-spelt

MADMK

MAMAK

MDMK

Party names change

AIADMK

ADMK

ADK

Parties restructure

INC(I)

INC

Constituency names mis-spelt

BHADRACHALAM

BHADRACHELAM

BHADRAHCALAM

9 of 10

9

Fortunately, large scale data itself can provide a solution

10 of 10

10

… with modern tools that support machine learning