Machine Learning

Movie Profitability Classification

Open Repository on Github

Overview

This project builds a machine learning model to predict whether a movie is successful based on financial and popularity features. It covers a complete data science workflow including data preprocessing, feature engineering, handling imbalanced data, model training, evaluation, and hyperparameter tuning.

This project demonstrates a complete machine learning pipeline with a focus on model evaluation beyond accuracy, including ROC-AUC and confusion matrix analysis.

Objectives

  • Analyze factors influencing movie success
  • Perform data cleaning and feature engineering
  • Build and compare multiple classification models
  • Improve model performance with hyperparameter tuning

Tech Stack

Dataset

The project uses two datasets:

movies_metadata.csv

Contains detailed information about movies.

Main columns:

  • id → Unique movie identifier
  • title → Movie title
  • budget → Production budget
  • revenue → Total revenue
  • popularity → Popularity score
  • runtime → Duration of the movie (minutes)
  • vote_average → Average rating score
  • vote_count → Number of votes
  • release_date → Release date
  • genres → Movie genres (JSON format)
  • original_language → Original language

ratings_small.csv

Contains user ratings for movies.

Main columns:

  • userId → Unique user identifier
  • movieId → Movie identifier (linked to movies dataset)
  • rating → User rating (scale 0–5)
  • timestamp → Time of rating submission

Workflow

Data Preprocessing
Data Preprocessing
  • Handle missing values to maintain data consistency.
  • Convert data types into appropriate formats for analysis.
  • Create new features including Profit and ROI (Return on Investment).
  • Apply log transformations to improve data distribution and reduce skewness.
h
Exploratory Data Analysis (EDA)
Exploratory Data Analysis (EDA)
  • Analyze profit distribution to understand overall performance trends.
  • Detect outliers and extreme values in the dataset.
  • Explore the relationship between runtime and profit.
  • Identify key patterns and insights before model development.
h
Feature Engineering
Feature Engineering
  • Select key features including Budget, Popularity, Vote Average, and ROI.
  • Create log_profit to reduce skewness in profit values.
  • Create log_budget to normalize budget scale.
  • Create log_popularity to stabilize variance.
  • Improve model performance by reducing the impact of outliers.
h
Handling Imbalanced Data
Handling Imbalanced Data
  • Perform feature scaling before balancing.
  • Apply SMOTE (Synthetic Minority Oversampling Technique).
  • Balance minority and majority target classes.
  • Reduce bias and improve model learning.
h
Model Training
Model Training
  • Train K-Nearest Neighbors (KNN) model.
  • Train Decision Tree model.
  • Train Random Forest model.
  • Compare model performance and reliability.
h
Evaluation Metrics
Evaluation Metrics
  • Measure Accuracy, Precision, Recall, and F1-score.
  • Analyze Confusion Matrix.
  • Evaluate ROC-AUC Score.
  • Visualize performance with ROC Curve.
h
Hyperparameter Tuning
Hyperparameter Tuning
  • Use GridSearchCV for model optimization.
  • Test multiple Random Forest parameter combinations.
  • Select the best-performing model configuration.
  • Improve prediction accuracy and consistency.
h

Model Performance

Model Accuracy ROC-AUC
KNN 0.62 0.53
Decision Tree ★ 0.77 0.52
Random Forest 0.77 0.54
Random Forest (Tuned) 0.77 0.52

Confusion Matrix (Final Model)

[ 1  5]
[ 4 29]
Interpretation:
  • Model performs well in predicting successful movies (True Positive)
  • Struggles to correctly identify unsuccessful movies
  • High False Positives → model tends to classify movies as successful

ROC Curve Analysis

  • ROC-AUC scores are relatively low (~0.52–0.54)
  • Indicates model performance is only slightly better than random guessing
  • Suggests that Features are not strongly discriminative and Dataset is highly imbalanced

Key Insights

  • High budget does not guarantee success
  • Popularity is a strong predictor of movie performance
  • ROI provides better signal than raw financial metrics
  • Accuracy alone can be misleading → ROC-AUC gives deeper insight

Future Improvements

  • Improve feature selection
  • Try advanced models (XGBoost, LightGBM)
  • Optimize classification threshold
  • Use cross-validation for all models
  • Deploy model as API or dashboard