1 of 10

FINAL PROJECT

Breakdown of Movie Genre Based on Viewer Input

Verity Pierson, Amy Ou, and Sultan Mahmud Rahat

2 of 10

Introduction

The film industry is a dynamic landscape and with effective management and analysis informed decisions can be made.
This project focuses on the development of a production ready data pipeline
This pipeline will integrate, clean and transform the data to produce a practical output.
Aim to demonstrate the ability to handle real-world challenges.

Team 6 will be sharing insights into our project, which delves into the dynamic landscape of the film industry. Effective management and analysis of data within this industry is pivotal for making informed decisions and navigating the ever-evolving preferences of audiences.

Our project centers around the development of a production-ready data pipeline, a critical tool in the modern film ecosystem. This pipeline is designed to seamlessly integrate, clean, and transform diverse datasets, culminating in a practical output that holds immense value for stakeholders across the industry.

Our primary objective is not just technical; we aim to demonstrate the tangible ability of our data pipeline to handle real-world challenges. In an industry as dynamic and multifaceted as film, the ability to process and interpret data efficiently becomes a key determinant of success. Today, we’ll explore how our project rises to this challenge and contributes to the continual enhancement of decision-making processes within the film domain.

3 of 10

Method

Data Sources

IMDb Developer Website
Kaggle.com

Initial format

Tab-separated values (TSV) datasets in gzipped files

Conversion

Files were exported to Google Drive and converted from TSV to comma-separated values (CSV) using a Python script

Purpose of conversion

To enhance the compatibility with various data analysis tools and libraries.

The comprehensive nature of our dataset, primarily derived from two key sources - the IMDb Developer Website and kaggle.com. IMDb’s developer website offered a wealth of information, providing us with seven datasets in a tab-separated values (TSV) format, packaged conveniently in gzipped files. This source is foundational for our film industry data analysis. Additionally, Kaggle.com contributed two datasets that augment our understanding of the industry, specifically focusing on movie details and streaming services. This dual-source approach ensures a robust and diverse dataset, laying the groundwork for a nuanced exploration of the film industry landscape.

The initial format of the dataset posed a challenge due to their tab-separated values (TSV) structure within gzipped files. to overcome this, we implemented a meticulous conversion process. The datasets were first exported to Google Drive for efficient storage and sharing. Subsequently, a Python script was employed to convert these datasets from their original TSV format to comma-separated values (CSV). This conversion serves as a critical step in our data preparation, enhancing the compatibility of our dataset with various data analysis tools and libraries. By transitioning to CSV, we ensure that our dataset is accessible and adaptable, allowing for a smoother analytical workflow.

The core objective behind the conversion process was to enhance the compatibility of our dataset with a diverse array of data analysis tools and libraries. This shift from TSV to CSV format not only streamlines out dataset for ease of use but also broadens its applicability. The versatility gained through this conversion allows us to seamlessly integrate our dataset into different analytical environments. It acts as a bridge connecting the initial format to a more accessible, adaptable structure, setting the stage for a comprehensive and insightful analysis of the film industry data. This emphasis on compatibility and adaptability ensures that our dataset remains a valuable resource for diverse analytical methodologies and applications.

4 of 10

Final Schema Diagram

5 of 10

Results

6 of 10

Results

7 of 10

Results

8 of 10

Conclusion

This project demonstrates the complexities of film data and its ability to capture real-world challenges

Helps users make informed decisions

Patterns in the data will uncover audience behavior and content dynamics

knowledge empowers users to make informed decisions and personalized choices

I am excited to share the essence of our work on film data. Our project has successfully tackled complex information to help us make better decisions in the world of cinema. We've learned a lot that can help those who make films and those who watch them through streaming services. This work is helping us improve how we manage content and the overall viewing experience.
The information we've gathered is not just random data; it's the key to understanding what film professionals and avid movie-watchers alike truly need. We've looked into the data to find out what patterns and preferences exist. This helps us to get to know our audience better and what they like to watch. This knowledge is very useful—it helps the film industry to make smart choices and allows viewers to find movies that they will enjoy more.
In short, our project is just the beginning. It leads us to a future where using data smartly can change the way decisions are made. The things we've found out set the stage for a world where movie choices are more thoughtful, content is crafted just right, and viewers enjoy an experience that's not only better but quite extraordinary. By bringing together data and cinema, we're on the path to keep getting better, marking an important point in the story of using data to achieve greatness in the film industry.

9 of 10

Next Steps

While our database has been developed within Azure, there is still some work to be done in order to work out any issues.
Azure is just one option to deploy the database. Another option would be to use Amazon AWS.
Testing the data pipeline
Identify and implement robust security measures
Use the following to connect to the Azure database:

host=imdb2.mysql.database.azure.com, port=3306, user=Team6Project, password=USDpassword507, db=imdb

-we have deployed our dataset into azure cloud. We can also use AWS to deploy our pipeline.Our goal is to create a service that constantly updates the list of movies so that you can always find something great to watch. We already have a vast collection of movie information from IMDB that updates regularly. But to make our service even better, we plan to directly connect with streaming platforms. We’ll use something called an API, which is like a secret handshake between our system and the streaming services to get fresh movie data. This will help us mix it with our current IMDB data and give you, the user, a better and more informed experience Also, we're committed to keeping your data safe. We plan to use special digital locks (specific IP address permissions) and high-tech security measures (like encrypted features from Azure) to protect our data pipeline. This is like having an unbreakable safe where all the valuable movie insights are stored.In simple terms, we're building a smart system that helps you choose movies from various platforms easily and safely.

10 of 10

References

Bhatia, R. (2021). Movies on Netflix, Prime Video, hulu and disney+. Retrieved from https://www.kaggle.com/datasets/ruchi798/movies-on-netflix-prime-video-hulu-and-disney

IMDb.com. (2024). IMDb Non-Commercial Datasets. Retrieved from https://developer.imdb.com/non-commercial-datasets/

Grijalva, D. (2021). Movie Industry. Retrieved from https://www.kaggle.com/datasets/danielgrijalvas/movies