1 of 20

Current Chilean constitution �was created under a dictatorial government

2016 Chilean constitutional process

gathered more than 8,000 small assemblies

Participants produced more than 200,000 �political arguments about a new constitution

2 of 20

200K+ Crowdsourced �Political Arguments for a �New Chilean Constitution

Constanza Fierro Jorge Pérez Mauricio Quezada

Department of Computer Science, Universidad de Chile�Center for Semantic Web Research (CSWR)

​

Claudio Fuentes

Center for Argumentation and Reasoning Studies (CARS)�Universidad Diego Portales

3 of 20

This paper/talk is (mainly) about Steps 2 & 3

Step 2)

Manual processing �of the data: cleaning, normalizing and tagging

Step 4)

Analyze the data and draw conclusions about the people’s opinions

Step 3)

Automatize the manual process to possibly include new opinions

Step 1)

Gather data �from participants

Openly publishing the data

4 of 20

Outline

Self-convened meetings and generated data

​

Manual and automatic tagging of constitutional concepts

​

Manual and automatic tagging of political arguments

5 of 20

Self-convened Local Meetings (SLMs)

10-30 people discussing Constitutional Concepts

  • Half-day group discussion, guided by a form
  • 4 topics: Rights, Values, Duties, Institutions
  • Groups selected 7 constitutional concepts per topic
  • Included justification/argument for every concept�

6 of 20

Self-convened Local Meetings (SLMs)

10-30 people discussing Constitutional Concepts

Constitutional Concept

Argument

Right to a fair wage

​

The worst of all inequalities is the salary of the politicians, congressmen, and Ministers compared with the minimum wage of the Chilean workers.

Equality before the law

There should exist equality before the law for all people without privileges or benefits for business people politicians and their relatives.

...

...

​

​

What should be the fundamental RIGHTS �contained in the Constitution?

7 of 20

Self-convened Local Meetings (SLMs)

10-30 people discussing Constitutional Concepts

​

  • The form suggested ~30 different concepts per topic
  • Groups can freely include new (open) concepts
  • All data should be uploaded to a website�

8 of 20

200,000+ arguments/justifications/thoughts

about Constitutional Concepts

4

Topics

114

Proposed concepts

11,682

Open concepts

205,357

Arguments

4,653,518

Words

9 of 20

Manual processing of the corpus and�automatization of this process

10 of 20

First manual task: classification �of open constitutional concepts

  • 11,000+ concepts openly included by the participants
  • 20,000+ arguments for open concepts
  • Task: classify open (concept,argument) pairs as one �of the concepts proposed by the government�

Constitutional Concept

Argument

Equality of rights for men and women �

Since we are equal there should not exist any discrimination against women ensuring an egalitarian salary.

Gender equity

11 of 20

Can we automatically perform this task?

First manual task: classification �of open constitutional concepts

  • Performed by United Nations Development Program
  • 18 annotators + 4 managers
  • 10,000+ open (concept,argument) pairs �were successfully classified
  • 87% of annotator agreement�

12 of 20

Formalizing the task for automatic classification

We took advantage of the 200K�closed concepts

Right to education

Healthcare

Gender equity

Right to a fair wage

Equality before the law

(… 44 concepts …)

?

Right to a fair wage

The worst of all inequalities is the salary �of the politicians, congressmen, and Ministers compared with the minimum wage of the Chilean workers.

13 of 20

Techniques used: standard ML classifiers +�neural networks architectures for NLP

Logistic Regression, SVM, Random Forests�using several standard features: n-grams, tf-idf, PoS, ...�

Neural-network based classifiers�- FastText word embeddings (word2vec + subword n-grams)�- FastText classifier (embeddings + softmax layer)�- Deep Averaging Networks

�

New: Arora et. al Discourse Vectors, LSTM/RNN (not here)

14 of 20

Task A: Predict the concept �given an argument (for a closed concept)�

Accuracy (test, best model)�

Topic / #C

@1

@5

Values / 37

68%

91%

Rights / 44

71%

92%

Duties / 12

77%

96%

Institutions / 21

70%

92%

15 of 20

We reused the models from task A

Task B: Predict a (closed) concept �given an argument and open concept�

Accuracy over open (concept,argument) pairs�

Topic / #C

@1

@5

Values / 37

63%

91%

Rights / 44

73%

93%

Duties / 12

79%

96%

Institutions / 21

60%

87%

16 of 20

Second manual task: �normalization and classification of arguments

We divided them into:

  • Policies �“Gay marriage should be legal in Chile”�
  • Facts �“Abortion is murder”�
  • Values �“Preservation of nature is most important than any business project”

17 of 20

Second manual task: �normalization and classification of arguments

  • Performed by CARS
  • 100+ annotators
  • 200,000+ arguments annotated (4 months of work!)
  • 85% estimated accuracy�

73%

Policies

20%

Facts

7%

Values

18 of 20

Task C: Predict the argumentation mode�

Macro metrics (best model)�

Precision

Recall

F1

70%

62%

65%

19 of 20

Future work

  • Improve the automatic classification�- better embeddings (Chilean specific?)�- more powerful models�
  • Use our models to detect constitutional concepts proposals and arguments in open text�
  • Detect similar and (more importantly!) dissimilar justifications for the same constitutional concept

20 of 20

200K+ Crowdsourced �Political Arguments for a �New Chilean Constitution

Constanza Fierro Jorge Pérez Mauricio Quezada

Department of Computer Science, Universidad de Chile�Center for Semantic Web Research (CSWR)

​

Claudio Fuentes

Center for Argumentation and Reasoning Studies (CARS)�Universidad Diego Portales