DATA ANALYSIS COVID-19

Ryan Gading Abdullah

​

​

Business Understanding

Data Wrangling

Data Exploratory

Visualization Analysis (Data Visualization)

Topic

Business Understanding

Business Understanding

​

The COVID-19 pandemic, caused by the SARS-CoV-2 virus, has had unprecedented global impacts on health, economy, and daily life. Despite extensive data collection efforts, Analyzing this vast amount of data to derive meaningful insights remains a significant challenge. The objective of this project is to analyze worldwide COVID-19 data to identify patterns, trends, and correlations that can inform public health strategies and policy decisions. Key aspects of the analysis will include:

​

1. Epidemiologic Trends: Investigating the spread of COVID-19 in the Asian Continent region, examining factors such as new cases and total cases, mortality
2. Demographic Trends: Comparing countries in the Asian continent based on COVID-19 Data

Data Wrangling

Data Wrangling

​

Data Wrangling is a transforms data to make it compatible with the end system, as complex and intricate datasets can hinder data analysis and business processes. (Naeem, 2024) In other definition define is the process of transforming raw data into easily understandable formats and organizing sets into a single structure for further processing. (Corbo, 2024)

​

Data Wrangling is the researcher's step to understand the data and check the data for irrelevant information, data duplication, or data type mismatches.

​

In this project, there will a few step starting from import library, gathering data, assessing data, check data, and cleaning data

import pandas as pd

import matplotlib.pyplot as plt

import seaborn as sns

from IPython.display import display

import plotly.express as px

From those codes:

1. We will import pandas for data manipulation and analysis,
2. matplotlib for creating static visualizations,
3. seaborn for enhancing visualizations with additional functionality,
4. IPython.display and import display for displaying dataframes and other outputs in a Jupyter notebook.
5. import plotly.express as px for creating interactive visualizations.

Import Library

from google.colab import drive

drive.mount('/content/drive’)

# CSV file path in Google Drive

file_path = '/content/drive/My Drive/Colab Notebooks/MIKTI/Project/Project 1/owid-covid-data.csv'

​

# Load dataset using Pandas

df = pd.read_csv(file_path)

​

# Display the top 5 rows of the dataset

df.head()

This step is to connect Google Colab with Google Drive to access the dataset in Google Drive.

​

After that we access the dataset and load to Google Colab. After that we check the data whether it has been successfully loaded

Gathering Data

# Checking data type

df.info()

​

# Displaying summary of descriptive statistics

display(df.describe())

​

# Menampilkan tipe data untuk setiap kolom

display(df.dtypes)

​

# display the entire dataset

display(df)

​

# Displaying the first 10 rows of the data

display(df.head(10))

​

# Displaying the last 10 rows of the data

display(df.tail(10))

​

# Displaying the locations in the dataset in sorted order

display(sorted(set(df['location'])))

Assessing Data

The code in page before explain:

1. In this step we will understanding the information of column data, Non-Null count, data type, etc.
2. This step is used to see a descriptive statistical summary of the data
3. This step is used to display the data types of the data being worked on
4. In this step, we will see all the datasets in the form of rows and columns
5. In this step, we will look at the top 10 of the dataset
6. In this step, we will look at the bottom 10 of the dataset
7. In this step, we will see a view of the sorted data locations which is the country in the dataset.

Assessing Data

According to Data Information, the problem we will solve is to compare COVID-19 spread each countries in Asia Continent using visualization Line Chart

# Count rows of dataset

jumlah_data = len(df)

print("Jumlah data:", jumlah_data)

​

# Counting the number of rows and columns

print("Number of rows and columns:", df.shape)

​

# Counting the number of duplicate entries

# Counting the number of null values in each column

print("Jumlah duplikasi: ", df.duplicated().sum())

print("\n")

​

print("Data Null:")

for key, data in df.isnull().sum().items():

print(f"{key}: {data}")

Check Data

The code in page before explain:

1. In this step, we will count the number of rows of data
2. In this step, we will count the number of rows and columns of data
3. The first part of the code calculates and prints the number of duplicate entries in the DataFrame df. After that, the second part of the code calculates and prints the number of null values in each column of the DataFrame df.

Check Data

After checking the data, the next step is to clean the data

# Dropping specific columns from the data

data = df.drop(['new_tests_per_thousand', 'new_tests_smoothed','iso_code', 'excess_mortality_cumulative_absolute', 'excess_mortality_cumulative', 'excess_mortality', 'excess_mortality_cumulative_per_million'], axis=1)

​

# display the entire dataset

display(data)

​

# Fixing the data type of the 'date' column

data['date'] = pd.to_datetime(data['date'])

Cleaning Data

# List of valid country names

valid_countries = [

…

]

​

# Displaying countries that are not in the valid list

invalid_countries = set(data['location']) - set(valid_countries)

print(f"Invalid countries: {invalid_countries}")

​

# Removing data with invalid countries

data = data[~data['location'].isin(invalid_countries)]

​

# showing the entire of dataset

display(data)

​

# Checking data type and column of dataset

data.info()

Cleaning Data

The code in page before explain:

1. in this step, we will remove the columns of data that are not needed in the dataset.
2. After that, we will display the dataset to check whether or not the unused data columns have been successfully removed.
3. Next, we will delete the country (location column) data rows that do not match the country name
4. And then, we will display the dataset to check whether or not the invalid country information in the location column was successfully removed.
5. At last, we will check the data type and column of dataset

Cleaning Data

Data Exploratory

Data Exploratory

​

Data Exploratory is the first step in data analysis involving the use of data visualization tools and statistical techniques to uncover data set characteristics and initial patterns. (Robinson et al., 2024) is also am essential step in any research analysis (Komorowski et al., 2016) and a fundamental stage in data mining of high-dimensional datasets. (Neme & Nido, 2013)

​

In Data Exploration, researchers will find out information and apply methods to find the results they want to get in accordance with the objectives that have been carried out in the data understanding section.

​

In this project, the exploration results will show the total_cases_max, new_cases_max, and total_deaths_max of covid-19 spread in the Asian continent.

# checking data description

data.describe()

​

# calculates and displays descriptive statistics for all columns in the dataset.

data.describe(include="all")

​

# Filter the data for the continent Asia

data_asia = data[data['continent'] == 'Asia']

​

# Displaying the entire dataset for Asia

display(data_asia)

​

# Checking the data types

data_asia.info()

​

# Displaying the data description

data_asia.describe()

​

# Displaying the complete data description

data_asia.describe(include="all")

Exploratory Data

# Grouping the data and calculating the required aggregations

df_agg = data_asia.groupby(by="location").agg({

"total_cases": ["max", "min", "mean", "std"],

"new_cases": ["max", "min", "mean", "std"],

"total_deaths": ["max", "min", "mean", "std"]

})

​

# Flattening the multi-level index columns

df_agg.columns = ['_'.join(col).strip() for col in df_agg.columns.values]

df_agg.reset_index(inplace=True)

​

# Adding the 'continent' column to the aggregation result

df_continent = data.groupby(by="location")['continent'].first().reset_index()

​

# Merging the aggregation result with the 'continent' dataframe

df_cov = df_continent.merge(df_agg, on="location")

​

# Displaying the merged dataframe

display(df_cov)

Exploratory Data

# Grouping the data by 'location' and summing the numeric columns

summary = df_cov.groupby('location').agg({

'total_cases_max': 'sum',

'total_cases_min': 'sum',

'total_cases_mean': 'sum',

'total_cases_std': 'sum',

'new_cases_max': 'sum',

'new_cases_min': 'sum',

'new_cases_mean': 'sum',

'new_cases_std': 'sum',

'total_deaths_max': 'sum',

'total_deaths_min': 'sum',

'total_deaths_mean': 'sum',

'total_deaths_std': 'sum'

}).reset_index()

​

# Displaying the summarized data

display(summary)

Exploratory Data

# Sum only specific columns, such as total_cases and new_cases:

summary_specific = df_cov.groupby('location').agg({

'total_cases_max': 'sum',

'new_cases_max': 'sum',

'total_deaths_max': 'sum'

}).reset_index()

​

# Displaying the summarized data for specific columns

display(summary_specific)

Exploratory Data

The code in page before explain:

1. In this step, we will calculates and displays descriptive statistics in the dataset

​

• Next, we will calculates and displays descriptive statistics for all columns in the dataset, including both numeric and categorical data.

​

• After that, we will Filtering the data to include only the records for the continent Asia. Displaying the entire dataset filtered for Asia. Checking the data types of the filtered dataset. Displaying the descriptive statistics of the filtered dataset. Displaying the complete descriptive statistics, including both numeric and categorical data, of the filtered dataset.

Exploratory Data

4. In this step, we will Grouping the data by location (country) in Asia and calculating maximum, minimum, mean, and standard deviation for total cases, new cases, and total deaths. Flattening the multi-level index columns for easier readability. Adding the 'continent' column to the aggregation result by getting the continent information for each country. Merging the aggregation result with the dataframe containing continent information. Displaying the merged dataframe containing aggregated information for each country in Asia along with their respective continent.

​

• Next step, we will Grouping the data by the 'location' (country) column and summing the values of the numeric columns. Resetting the index to make 'location' a regular column instead of an index. Displaying the summarized data, which now shows the sum of each numeric column for each country.

Exploratory Data

6. At last, we will Grouping the data by 'location' (country) and summing specific columns, namely 'total_cases_max', 'new_cases_max', and 'total_deaths_max’. Resetting the index to make 'location' a regular column instead of an index. Displaying the summarized data, which now shows the sum of specific columns for each country.

Exploratory Data

Visualisasi Analysis (Data Visualization)

Visualization Analysis (Data Visualization)

​

Visualization Analysis (Data Visualization) is translating information into a visual context. (Hashemi-Pour et al., 2024) other definition define Interpreting information by applying method to put data into visible form. (Hinterberger, 2009)

​

visualization is done with the aim of making it easier for researchers to read information and understand a problem.

​

In this project the visualization will shown bu using Line Chart and Line Chart Interactive

summary_long = summary_specific.melt(id_vars=['location'], value_vars=['total_cases_max', 'new_cases_max', 'total_deaths_max'],

                                     var_name='Metric', value_name='Count')�# Creating an interactive visualization using Plotly

fig = px.line(summary_long, x='location', y='Count', color='Metric',

              title='COVID-19 Cases and Deaths Summary by Location',

              labels={'Count': 'Count (Log Scale)', 'location': 'Location', 'Metric': 'Metric'})�# Adding markers for each country

fig.update_traces(mode='markers+lines', marker=dict(size=5))�# Setting the y-scale to logarithmic

fig.update_yaxes(type='log', title_text='Count (Log Scale)')�# Adjusting the layout of the plot

fig.update_layout(

    xaxis=dict(title_text='Location'),  # x-axis title

    legend=dict(title='Metric', orientation='h', yanchor='bottom', y=1.02, xanchor='right', x=1)  # Legend title

)�# Displaying the plot

fig.show()

The following code uses the Matplotlib library to create a line plot depicting the total cases, new cases, and total deaths of COVID-19 in each country in the Asian continent.

Based on the Line Chart above, it can be concluded that the highest total max deaths are dominated by India, then the highest total max cases and new max cases are dominated by China even though the visualization was not very supported.

plt.figure(figsize=(12, 6))�# Plot for total_cases_max

plt.plot(summary_specific['location'], summary_specific['total_cases_max'], label='Total Cases Max', marker='o')�# Plot for new_cases_max

plt.plot(summary_specific['location'], summary_specific['new_cases_max'], label='New Cases Max', marker='o')�# Plot for total_deaths_max

plt.plot(summary_specific['location'], summary_specific['total_deaths_max'], label='Total Deaths Max', marker='o')�# Adding title and axis labels

plt.title('Summary of COVID-19 Cases and Deaths by Location')

plt.xlabel('Location')

plt.ylabel('Count')�# Adding legend

plt.legend()�# Adding grid

plt.grid(True)�# Rotating x-axis labels if needed (e.g., if labels are long)

plt.xticks(rotation=90)�# Displaying the plot

plt.tight_layout()

plt.show()

The following code uses the Matplotlib library to create an interactive line plot depicting the total cases, new cases, and total deaths of COVID-19 in each country in the Asian continent.

Based on the Interactive Line Chart above, it can be concluded that the highest total maximum deaths are dominated by India, then the highest total maximum cases and maximum new cases are dominated by China. This visualization is used to help Line Charts that do not support visualization.

Export to File

# Path to save the modified CSV file

file_name = input("Enter the file name to save (without the .csv extension): ")

​

# Path to save the modified CSV file

output_file_path = f'/content/drive/My Drive/Colab Notebooks/MIKTI/Project/Project 1/{file_name}.csv’

​

# Save the data into a file with the .csv extension

df_cov.to_csv(output_file_path)

​

print(f"File has been saved to: {output_file_path}")

The code above explain:

1. In this step will input the name of the file will used to save the dataset
2. In this step, we make the path to save the modified data
3. In this step, we will save the data into a file with the .csv extension
4. In this step, it will play a role that the data is successfully saved in the specified path.

Export to File

Data Visualization (Tableau)

Based on the Line Chart above, it can be concluded that the highest total max deaths are dominated by India, then the highest total max cases and new max cases are dominated by China.

Click the document below to open Google Collaboratory

Hashemi-Pour, C., Brush, K., & Burns, E. (2024, April). data visualization. TechTarget. https://www.techtarget.com/searchbusinessanalytics/definition/data-visualization

​

Hinterberger, H. (2009). Data Visualization. In: LIU, L., ÖZSU, M.T. (eds) Encyclopedia of Database Systems. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-39940-9_1370

​

Neme, A., Nido, A. (2013). Exploratory Data Analysis through the Inspection of the Probability Density Function of the Number of Neighbors. In: Tucker, A., Höppner, F., Siebes, A., Swift, S. (eds) Advances in Intelligent Data Analysis XII. IDA 2013. Lecture Notes in Computer Science, vol 8207. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41398-8_27

​

Robinson, S., Hanna, K. T., & Biscobing, J. (2024, March). data exploration. TechTarget. https://www.techtarget.com/searchbusinessanalytics/definition/data-exploration

References

Naeem, T. (2024, March 21). Data Wrangling: Definition, Importance, and Benefits. Astera. https://www.astera.com/type/blog/data-wrangling/

​

Corbo, A. (2024, February 8). What Is Data Wrangling? Built In. https://builtin.com/data-science/data-wrangling

References

Ryan Gading Abdullah

​

​

Thank You