Forum for Information Retrieval Evaluation

Lakshmi S. Gopal

Research Scholar

Aswathy A

Research Scholar

Krishnendu K

Project Assistant

Dr. Hemalatha T

Assistant Professor

Amrita Center for Wireless Networks & Applications (AWNA),

Amrita VIshwa Vidyapeetham,

Amritapuri Campus

Amrita Center for Wireless Networks & Applications (AWNA),

Amrita VIshwa Vidyapeetham,

Amritapuri Campus

Amrita Center for Wireless Networks & Applications (AWNA),

Amrita VIshwa Vidyapeetham,

Amritapuri Campus

Amrita Center for Wireless Networks & Applications (AWNA),

Amrita VIshwa Vidyapeetham,

Amritapuri Campus

Forum for Information Retrieval Evaluation

Our Center

A leading multi-disciplinary research center at Amrita Vishwa Vidyapeetham, our scientists, academicians and students innovate leading edge technologies in the field of wireless, sensors, computing, and networking to solve grand challenges facing humanity today in the areas of disaster monitoring & preparedness, climate change, energy, water, and health.

Forum for Information Retrieval Evaluation

Our Disaster

Management Related Projects

Amrita Wireless Sensor Network for Landslide Detection System for Himalayas

Landslide Multi-Hazard Risk Assessment, Preparedness and Early Warning in South Asia: Integrating Meteorology, Landscape and Society (LANDSLIP)

Flood Evac (Vulnerability of Transportation Structures, Warning and Evacuation in Case of Major Inland Flooding)

Micronet – Mobile Infrastructure for Coastal Region Offshore Communications & Networks

Forum for Information Retrieval Evaluation

Content

Introduction

Task Details

Methodology

Exploratory Data Analysis

Data Preprocessing

Model Creation for Multi Label Classification

Results and Evaluation

Conclusion

Forum for Information Retrieval Evaluation

Introduction

• Vaccination is crucial for public health, saving lives and reducing disease burden.�
• Despite its effectiveness, vaccine hesitancy persists, fueled by easy access to both credible and misleading information in the digital age.

​

• Crowdsourced data platforms provide valuable insights into vaccine-related experiences and concerns.

​

• Machine learning is a vital tool in public health, capable of analyzing large datasets and revealing patterns in vaccine-related sentiments expressed on social media.

Forum for Information Retrieval Evaluation

• The research paper aims to contribute to understanding vaccine hesitancy by analyzing crowdsourced vaccine data using machine learning.

​

• The study seeks a deeper understanding of concerns expressed through crowdsourcing to inform targeted public health interventions and communication strategies.

​

• Leveraging machine learning algorithms, the research aims to shed light on intricate dynamics of vaccine hesitancy for more effective vaccination campaigns and improved public health outcomes.

Introduction

Forum for Information Retrieval Evaluation

Forum for Information Retrieval Evaluation

• Unnecessary: Suggests immunizations aren't necessary or better alternatives exist.
• Mandatory: Opposes making vaccinations obligatory.
• Pharma: Big Pharma prioritizes profit over other considerations.
• Conspiracy: Hints at a broader conspiracy beyond Big Pharma's profit motive.
• Political: Raises concerns that governments and politicians use vaccines to further their agendas.
• Country: Criticizes a vaccination based on its country of origin.
• Rushed: Questions the adequacy of vaccine testing and the reliability of available data.
• Ingredients: Concerns about vaccine contents or the utilized technology.
• Side-effect: Worry about vaccine side effects, including potential deaths.
• Ineffective: Concern that immunizations are ineffective and useless in certain cases.
• Religious: Opposes vaccinations for religious reasons.
• None: No explicit justification is provided.

Forum for Information Retrieval Evaluation

Methodology

• The proposed methodology aims to perform multi label classification on the given dataset. �
• Following steps were employed:

• Exploratory Data Analysis (EDA)�
• Data Preprocessing�
• Model Creation for Multi Label Classification

Forum for Information Retrieval Evaluation

Exploratory Data Analysis

Forum for Information Retrieval Evaluation

• We observed that ‘side-effect’ & ‘religious’ labels was the most frequent & less frequent respectively �
• Around 80% of tweets were found to have a single label�
• Balancing the given data may be necessary to ensure optimal performance�
• Frequent occurrences of terms like 'vaccine,' 'covid,' and 'Pfizer' are observed in the word clouds.�
• Keywords similar to 'side-effect,' such as 'death,' 'adverse reaction,' 'blood clot,' etc., are frequently found in the 'side-effect' word cloud.�
• Keywords similar to the 'religious' label, such as 'religion,' 'faith,' 'psalm,' etc., were found to be less frequent in the 'religious' word cloud.

Forum for Information Retrieval Evaluation

Data Preprocessing

• Basic data cleaning methods were employed�
• Uses Python libraries - NLTK, regular expressions, Spacy

• Word Tokenizations
• Lowercase conversions
• URLs often found along with a tweet (image, video urls)
• Stopwords (the, a, is etc) are removed except for ‘not’ and ‘no’ to maintain the context
• Special characters, smileys

Forum for Information Retrieval Evaluation

Model Creation

• Multi label classification was experimented using classifier chains & multi output classifier
• Classifier chains offer a powerful approach to address interdependencies in multi-label classification tasks

• Each classifier predicts a label based on input features�
• Predictions from previous classifiers are incorporated as additional features for subsequent classifiers

• Train the first classifier (Ingredients) on feature Tweet to predict Label Ingredient.
• Train the second classifier (Side-effect) on features Tweet and Ingredients to predict Label side-effect.
• The key mathematical concept is the iterative updating of features and labels as each classifier is trained and used in the chain, capturing dependencies between labels in the multi-label classification task.

Forum for Information Retrieval Evaluation

• Multi output classifier

• The model used is typically an extension of traditional machine learning models, such as decision trees, random forests, support vector machines, or neural networks.

​

• The model is trained on the dataset using a loss function that considers all the output labels simultaneously.

​

• The loss function is a measure of the difference between predicted and true values for all outputs.

​

• During training, the model adjusts its parameters to minimize this loss across all outputs.

Forum for Information Retrieval Evaluation

• Classification algorithms were wrapped in the classifier chain & multi output classifier
• Classifier chain with Logistic Regression

• In multi-label classification, logistic regression is extended by using a classifier chain to handle multiple labels.

​

• Each classifier in the chain is a logistic regression model.

​

• For label i, logistic regression predicts the probability P(Yi=1∣X,Y1,Y2,...,Yi−1)

​

• Add the predicted probabilities as features for subsequent classifiers.

Forum for Information Retrieval Evaluation

• Classifier chain with Support Vector Machines

• SVM aims to find the hyperplane that maximally separates data points of different classes.

​

• The hyperplane is the decision boundary, and the margin is the distance between the hyperplane and the nearest data points (support vectors).

​

• SVM can efficiently handle non-linear relationships in the data using the kernel trick.

​

• In a classifier chain, each classifier in the chain is an SVM model

Forum for Information Retrieval Evaluation

• Multi output classifier with support vector machines

• Train an SVM classifier on features X and the binary label Yi

​

• Predict the binary label Yi based on X and the predicted labels from previous classifiers

​

• Update Yi to be the true label values for the next iteration

​

• Add the trained SVM model to the multi-output classifier.

• For each label i:

• SVM's ability to handle non-linear relationships is leveraged within the multi-output framework

Forum for Information Retrieval Evaluation

Forum for Information Retrieval Evaluation

Learning curves of Logistic Regression wrapped in Classifier Chain.

(a) shows the learning curve of the model before fine tuning.

(b) shows the learning curve of the model and after fine tuning.

Forum for Information Retrieval Evaluation

Learning curves of Support Vector Machines wrapped in Classifier Chain.

(a) shows the learning curve of the model before fine tuning.

(b) shows the learning curve of the model after fine tuning.

Forum for Information Retrieval Evaluation

Hyper parameter tuning was not performed for the multi output classifier

Forum for Information Retrieval Evaluation

• We utilized social media data related to Covid vaccine to explore public concerns about vaccinations.
• Tested multiple models with the available data.
• Utilized a Classifier Chain model incorporating Support Vector Machines (SVM) and Logistic Regression.
• Employed a Multi-Output Classifier wrapping Support Vector Machines.
• Achieved the highest performance with the Multi-Output Classifier with 64% f1 score

​

Forum for Information Retrieval Evaluation

• Enhancing the dataset quality.
• Exploring additional preprocessing methods.
• Considering data augmentation strategies.

Opportunities for Improvement:

​

• Acknowledged room for performance enhancement.
• Suggested potential improvement avenues:

​

Forum for Information Retrieval Evaluation

Lakshmi S. Gopal

Amrita VIshwa Vidyapeetham,

lakshmisgopal@am.amrita.edu