





































ARESTY RUTGERS UNDERGRADUATE RESEARCH JOURNAL, VOLUME I, ISSUE V 

 

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. 
 

CLASSIFICATION OF FALL 

OUT BOY ERAS 
 

SHIFRA L. ISAACS, JOSEPH YUDELSON, DR. ENDRE BOROS 

 

✵  ABSTRACT 
This paper explored the use of machine 

learning techniques to differentiate between two 

different musical eras of the same rock band, includ-

ing the technique of Logistic Regression. 

Logistic regression (LR) is a widely-used sta-

tistical modeling method for binary classification in 

supervised machine learning. It is often used to pre-

dict whether a given event belongs to one of two 

categories. The process helps data scientists under-

stand which variables are good predictors of class 

membership. Applications of logistic regression in-

clude loan classification in the financial industry and 

predicting susceptibility to disease in the medical 

field. 

In this particular project, a dataset was con-

structed using data from Spotify and Genius consist-

ing of songs and lyrics written by the band Fall Out 

Boy. A logistic regression model was developed 

from scratch to classify the songs and lyrics into one 

of two eras of the band: before their 2009 hiatus and 

afterward. The study aimed to determine if a com-

puter could differentiate between the two eras. The 

model was also tested against other binary classifi-

cation algorithms, including Random Forest and 

Support Vector Machines. 

 

2  DATA EXTRACTION 
Audio and lyric features from Fall Out Boy’s 

(FOB) music were extracted utilizing the Spotipy 

and LyricsGenius APIs. Detailed information about 

the songs used for the project can be found in Ap-

pendix B of the supplemental document. The ex-

tracted data was subsequently cleaned and com-

bined to form a cohesive dataset, which was then 

explored and visualized. Before modeling, the da-

taset was split into a training set of 98 records and 

test set of 43 records, for a total of 141 records. 

The data extraction stage presented several 

challenges, such as the need to clean up sub-head-

ings within the lyrics using complex regular expres-

sions. Additionally, the Spotipy API featured inter-

esting edge cases, such as a post-hiatus Elton John 

cover that was incorrectly labeled as a 1973 release. 

Additionally, one Spotify audio feature 

proved difficult to handle due to its format. For ex-

ample, the API described song durations in millisec-

onds, which were subsequently converted to 

minutes. Care was taken to leave comments explain-

ing that a value of 3.05, for example, should be read 

as 3.05 minutes rather than 3 minutes and 5 sec-

onds. 

Descriptions for all the Spotify features can 

be found in Appendix A of the supplemental docu-

ment. 

 

2  DATA ANALYSIS 
Prior to the modeling process, it is common 

practice to engage in exploratory data analysis 

(EDA) and dive deeper into the features that will be 

modeled. This section will discuss the analyses per-

formed on the Fall Out Boy songs dataset. 

For the quantitative audio features, histo-

grams were created to provide a graphical repre-

sentation of the data distributions. In addition, scat-

terplot matrices were used to visualize any correla-

tions between the different features. 

Figure 1 shows the distributions of the va-

lence and tempo features and indicates the possi-

bility of a small positive correlation between them. 

This is feasible considering that happier, high-va-

lence songs, broadly tend to have faster tempos. 

To analyze the categorical audio features, 

the distribution of categories was examined across 

the pre-hiatus and post-hiatus classes. 

As for the lyrical features, a hypothesis was 

formed that pre-hiatus songs generally contain 

more unique words than post-hiatus songs. A pre-

liminary analysis of the relevant histograms sug-

gests that this may be the case. 



ARESTY RUTGERS UNDERGRADUATE RESEARCH JOURNAL, VOLUME I, ISSUE V 

 

2 
 

 
FIGURE 1: Scatterplot matrix of valence and tempo audio 

features. Valence is the mood of a song, with high values 

indicating positive mood. 

 

 
FIGURE 2: Distribution of unique_words feature across pre-

hiatus and post-hiatus songs 

 

 
FIGURE 3: Frequency Plot for Key Feature Across Classes 

 

However, a two-sample z-test of these da-

tasets yielded a p-value of approximately 0.46. A z-

test is a statistical hypothesis test used to compare 

the means of two independent samples and deter-

mine if they are significantly different. 

The p-value of 0.46 suggests that there is 

not enough evidence to reject the null hypothesis, 

which states that the two samples have no signifi-

cant difference. This indicates no evidence of a sub-

stantial difference in the mean unique lyrics be-

tween pre-hiatus songs and post-hiatus songs. 

 

3 FEATURE ENGINEERING 
In machine learning modeling, it is common 

practice to scale all the numerical features within a 

fixed range while preserving their relative distance. 

This technique speeds up the process and improves 

model accuracy because algorithms are designed 

to work with a fixed value set.  

The majority of audio features obtained 

from the Spotipy API were already scaled between 

0 and 1; therefore, MinMaxScaler was used to scale 

other variables to the same range. Categorical fea-

tures and the target variable were then one-hot en-

coded. 

Backward feature selection was employed 

to determine the optimal feature set based on F1 

scores, which measure a model’s ability to predict 

true positives accurately and identify true positives 

correctly. The algorithm resulted in the selection of 

8 features: danceability, instrumentalness, to-

tal_words, mode, key_6, key_9, key_10, and key_11. 

The mode feature, which indicates whether 

a song's key is major or minor, proved to be a strong 

predictor despite its inability to capture the musical-

ity of most songs. The key numbers above refer to 

specific musical notes; for example, key_6 refers to 

the musical key of F. Meanwhile, the instrumental-

ness feature represents the likelihood from 0-1 that 

the track is instrumental and contains no vocals. 

After a heatmap correlation matrix con-

firmed that there were no significant linear correla-

tions among the selected features, the modeling 

step was initiated. 

 



ARESTY RUTGERS UNDERGRADUATE RESEARCH JOURNAL, VOLUME I, ISSUE V 

 

 

 
FIGURE 4: Heatmap Showing Minimal Correlation in Feature Set 

 

4 LOGISTIC REGRESSION FROM 

SCRATCH 
A Logistic Regression (LR) algorithm was 

constructed from scratch in Python, along with class 

methods to facilitate model evaluation and visuali-

zation. The class was well-documented in section VI 

of the supplemental document. 

Before describing the process in detail, this 

section will introduce the general modeling pro-

cess: 

• Model selection: Design modeling task based 

on the item to be modeled or predicted and se-

lect the model best suited to that task 

• Data pre-processing: Prepare data for model-

ing and split data into training and testing sets 

• Typically 80% of the data is used for 

training and 20% for testing 

• Sometimes a third validation set is used, 

but this dataset is far too small for that 

• Model building 

• Model training: Run the model iteratively on the 

training set and update the model parameters 

on each epoch to improve the model results 

• Model testing: Run the model on the test data 

on each epoch 

• Model evaluation: Run metrics such as accuracy 

and F1 Score to determine quality of model per-

formance 

After creating the basic class properties and 

methods for the model, the focus was on the algo-

rithm itself. A stable sigmoid function was chosen in-

stead of a typical sigmoid function; the regular sig-

moid can sometimes result in an impossible denom-

inator of zero, whereas the changes in the stable sig-

moid prevent zero-division errors. 

The most complex aspect of the algorithm 

was the implementation of the fit method, which in-

volved several steps: 

• collecting properties from the input data. 

• constructing a linear model based on the input 

data. 

• computing binary cross-entropy loss as a meas-

ure of model error. 

• performing gradient descent during training. 

• storing loss values at each training and testing 

stage. 

The final steps of the algorithm involved 

converting the fit scores to probabilities using the 

stable sigmoid function, and then converting those 

probabilities into class predictions for either the 

pre-hiatus or post-hiatus era. 

To evaluate the model, a method was 

added to calculate a confusion matrix and conven-

tional performance metrics, including accuracy, pre-

cision, recall, and the F1 score. 

A model tracking method was also included 

to graph training and test error. 



ARESTY RUTGERS UNDERGRADUATE RESEARCH JOURNAL, VOLUME I, ISSUE V 

 

2 
 

 
FIGURE 5: Graph of Training and Test Error 

 

Figure 5 shows that the training error de-

clined gradually over the training period. Mean-

while, the testing error remained at a plateau near 

0.78 throughout training. This is quite unusual; nor-

mally the testing error should decline as well. This 

anomaly was not investigated further because the 

model performed decently well despite it. 

However, the lack of change is likely due to 

the small size of the test sample, which was only 43 

records. It’s also possible that the model was slightly 

overfitting, or fitting too well, to the training data 

such that the training process didn’t significantly im-

pact the testing performance. 

As a final step, a naive model was included 

for comparison purposes that classified FOB songs 

randomly. 

 

5 COMPARATIVE MODELING RESULTS 
To contextualize the performance of the 

custom-built logistic regression model, several 

other binary classification algorithms were run using 

the SciKit Learn Python library. 

As an additional experiment, logistic regres-

sion was implemented as a single-layer neural net-

work (NN) using a linear layer that was optimized 

with stochastic gradient descent. These complex 

mathematical processes were implemented using 

just two concise lines of code in PyTorch. 

The model hyperparameters were tuned by 

implementing a grid search cross-validation algo-

rithm. The model correctness was then measured by 

counting the numbers of true positives (TP), false 

positives (FP), true negatives (TN), and false nega-

tives (FN) found by the model. Those counts are ex-

panded into specific metrics which are used to eval-

uate model performance: 

 
FIGURE 6: Comparative Model Metrics (SOURCE) 

 

The resulting metrics from all models were 

as follows: 

FIGURE 7: Comparative model metrics. SVC refers to Sup-

port Vector Machines, GaussianNB refers to Gaussian Na-

ive Bayes, and KNN refers to K-Nearest Neighbors 

  

Model 
Accu-

racy 

F1 

Score 

Preci-

sion 
Recall 

SVC 62.79% 52.94% 52.94% 52.94% 

LinearSVC 67.44% 36.36% 23.53% 80.00% 

Gaussi-

anNB 
69.77% 60.61% 58.82% 62.50% 

KNN 65.12% 44.44% 35.29% 60.00% 

NN 25.58% 10.53% 100.00% 11.76% 

Random 

Forest 
79.07% 70.97% 64.71% 78.57% 

LR 

(Scratch) 
41.86% 28.81% 40.48% 100.00% 

LR (SciKit) 76.74% 64.29% 52.94% 81.82% 

https://www.researchgate.net/post/What_is_the_best_metric_precision_recall_f1_and_accuracy_to_evaluate_the_machine_learning_model_for_imbalanced_data


ARESTY RUTGERS UNDERGRADUATE RESEARCH JOURNAL, VOLUME I, ISSUE V 

 

3 
 

As expected, the Random Forest Classifier 

performed well on all metrics, as it is an ensemble 

method that utilizes multiple learners. Both imple-

mentations of the logistic regression algorithm per-

formed better than Random Forest in terms of recall, 

which initially suggests impressive performance. 

However, the low accuracy of the custom-built lo-

gistic regression model renders the high recall es-

sentially meaningless. 

It is worth noting that the optimal logistic re-

gression algorithm using the SciKit Learn library em-

ployed L1 regularization, which is typically used to 

prevent overfitting. Although the conventional lo-

gistic regression model was not overfitting and did 

not appear to require L1 regularization, its inclusion 

improved all of the recorded metrics. 

 

6 CONCLUSION AND NEXT STEPS 
This project involved several key steps: 

• Aggregating data from multiple APIs. 

• Exploring and analyzing the sourced data for 

feature engineering. 

• Building models to evaluate their performance 

on a binary classification task. 

• Answering the question: Can a machine differ-

entiate between the old and new music of Fall 

Out Boy (FOB)? 

The details of all of these steps can be found 

in the supplemental document along with relevant 

code blocks and information sources. 

The results of this project indicate that lo-

gistic regression performed surprisingly well on a 

small dataset of around 140 rows. Random Forest 

also performed well, as the ensemble learning com-

pensated for the smaller size of the data. 

Overall, this project shows that rudimentary 

AI can still simulate the human experience to some 

degree. It’s quite a human thing to notice differ-

ences in musical styles, or to differentiate an artist’s 

old sound from their new one, and this simple AI can 

still accomplish that. It’s noteworthy that complex 

generative models like Chat GPT and DALL E 2 spe-

cialize in tasks such as writing and visual art. We 

once thought machines would struggle with these 

creative tasks, but in the current generative AI land-

scape, it seems that tasks thought to be most human 

are arguably the most conducive to AI. 

A variety of methods are available to ex-

plore these AI tasks. An alternative approach for this 

project would have been to analyze FOB songs as 

complex time series, using deep learning to extract 

relevant features, and fine-tuning a complex neural 

network as precisely as possible. However, the goal 

of this project was to gain experience with data 

wrangling and popular machine learning algo-

rithms, as opposed to advanced time series model-

ing and deep learning. 

The next steps for this project would include 

further model tuning and optimization for the algo-

rithms discussed in the comparative modeling 

stage. Next steps could also include the alternative 

approach of rigorous time series analysis. This ap-

proach would be helpful because researchers could 

derive their own numerical audio features using 

time series patterns instead of using Spotify’s lim-

ited features. This process would require special-

ized audio analysis and time series tools which 

could be found in the StatsModels, Tensorflow-IO, 

and Librosa Python packages∎ 

 

7 ACKNOWLEDGMENTS 
I would like to express my appreciation to 

the following people who contributed a great deal 

to this project: 

Professor Endre Boros, for supervising this 

project, advocating on my behalf on a number of oc-

casions, and going above and beyond to give me 

the best possible college experience. 

FALL OUT BOY, for their wonderful music and 

for appreciating this project. 

JOEY YUDELSON, for providing strategic recom-

mendations, deep Python expertise, and impecca-

ble technical editing. 

NICK SINGH, my mentor, for encouraging me 

to work on data science portfolio projects and 

teaching me how to best leverage my results. 

SIDDHANT KOCHREKAR, for providing advice 

concerning the feature engineering process, model 

tuning, and out-of-the-box ideas to improve this 

project. 

https://falloutboy.com/
https://github.com/JYudelson1
https://www.linkedin.com/in/nick-singh-tech/
https://github.com/siddhantkochrekar


ARESTY RUTGERS UNDERGRADUATE RESEARCH JOURNAL, VOLUME I, ISSUE V 

 

4 
 

VASTAVA, for inspiring this project with her 

CONTENT and her CODE. 

ZACK OVITS, for helping me build the project 

directory, and for providing consistent advice on 

best practices throughout the duration of this pro-

ject. 

 

8 REFERENCES 
[1] Brownlee, J. (2020, September 14). Hyperparameter 

Optimization With Random Search and Grid Search. 

Machine Learning Mastery. Retrieved November 11, 

1111, from  

MACHINELEARNINGMASTERY.COM/HYPERPARAMETER-OPTIMI-

ZATION-WITH-RANDOM-SEARCH-AND-GRID-SEARCH/ 

[2] [Coding Lane]. (2021, February 4). Logistic Regres-

sion in Python from Scratch | Simply Explained 

[Video]. YouTube. 

YOUTUBE.COM/WATCH?V=NZNP05AYBM8 

[3] Dola, P. (2020, September 9). Exploratory Analysis of 

Spotify Tracks: How has music changed over the past 

100 years? RPubs. Retrieved January 1, 1111, from 

RPUBS.COM/PETERDOLA/SPOTIFYTRACKS 

[4] [Elbert]. (2021, January 5). How to automate with Py-

thon, Spotipy, and LyricsGenius [Video]. YouTube. 

YOUTUBE.COM/WATCH?V=CU8YH2RHN6A 

[5] FOB Song Data. (n.d.). Spotify. 

OPEN.SPOTIFY.COM/PLAYLIST/0UBKNC9CTDVXLGBF5JICVQ

?SI=D1A259083136490A&ND=1 

[6] Ho1yShif. (n.d.). GitHub - Ho1yShif/FOB_LR_Public: 

Independent Study Project: Classification of Fall Out 

Boy eras. GitHub. 

GITHUB.COM/HO1YSHIF/FOB_LR_PUBLIC 

[7] Klosterman, S. (2019). Data Science Projects with Py-

thon. Packt. 

[8] Loeber P. (2019, September 15). Logistic Regression 

in Python - Machine Learning From Scratch 03 - Py-

thon Tutorial [Video]. YouTube. 

YOUTUBE.COM/WATCH?V=JDU3AZH3WKG 

[9] Loeber P.  (2019, December 30). PyTorch Tutorial 08 

- Logistic Regression [Video]. YouTube. 

YOUTUBE.COM/WATCH?V=OGPQXIKR4AO 

[10] Parveez, S., & Iriondo, R. (2020, December 28). Gra-

dient Descent for Machine Learning (ML) 101 with 

Python Tutorial. Towards AI. Retrieved November 

11, 1111, from  

PUB.TOWARDSAI.NET/GRADIENT-DESCENT-ALGORITHM-FOR-

MACHINE-LEARNING-PYTHON-TUTORIAL-ML-9DED189EC556 

[11] Pedregosa et al. (2011). Scikit-learn: Machine Learn-

ing in Python 

[12] Scikit-learn developers (2021). 1.1.11. Logistic re-

gression. Scikit-learn.  

SCIKIT-LEARN.ORG/STABLE/MODULES/GENER-

ATED/SKLEARN.LINEAR_MODEL.LOGISTICREGRESSION.HTML 

[13] Scikit-learn developers (2021). 1.1.2. Random-

ForestClassifier. Scikit-learn.  

SCIKIT-LEARN.ORG/STABLE/MODULES/GENER-

ATED/SKLEARN.ENSEMBLE.RANDOMFORESTCLASSIFIER.HTML 

[14] Scikit-learn developers (2021). 1.4. Support Vector 

Machines. Scikit-learn.  

SCIKIT-LEARN.ORG/STABLE/MODULES/SVM.HTML 

[15] Scikit-learn developers (2021). 1.9. Naive Bayes. 

Scikit-learn. 

SCIKIT-LEARN.ORG/STABLE/MODULES/NAIVE_BAYES.HTML 

[16] Statistics Solutions (2021). Assumptions of Logistic 

Regression. 

STATISTICSSOLUTIONS.COM/FREE-RESOURCES/DIRECTORY-OF-

STATISTICAL-ANALYSES/ASSUMPTIONS-OF-LOGISTIC-REGRES-

SION/ 

[17] [StatQuest With Josh Starmer]. (2018, June 4). Lo-

gistic Regression Details Pt1: Coefficients [Video]. 

YouTube.  

YOUTUBE.COM/WATCH?V=VN5CNN2-HWE 

[18] [StatQuest With Josh Starmer]. (2018, June 11). Lo-

gistic Regression Details Pt 2: Maximum Likelihood 

[Video]. YouTube.  

YOUTUBE.COM/WATCH?V=BFKANL1ASG0 

[19] [StatQuest With Josh Starmer]. (2018, June 18). Lo-

gistic Regression Details Pt 3: R-squared and p-value 

[Video]. YouTube. 

YOUTUBE.COM/WATCH?V=XXFYRO8QUXA 

[20] [StatQuest With Josh Starmer]. (2018, May 7). Odds 

and Log(Odds), Clearly Explained!!! [Video]. 

YouTube.  

YOUTUBE.COM/WATCH?V=ARFXDSKQF1Y 

[21] [StatQuest With Josh Starmer]. (2018, March 5). 

StatQuest: Logistic Regression [Video]. YouTube.  

YOUTUBE.COM/WATCH?V=YIYKR4SGZI8 

[22] [Vastava]. (2020, December 4). I trained an AI to tell 

me CORPSE's music genre | data science [Video]. 

YouTube.  

YOUTUBE.COM/WATCH?V=VTU6JLA70VY 

[23] Vastava, S. (2021, May 25). Spotify Genre Classifier 

[Repository]. GitHub.  

GITHUB.COM/VASTAVA/DATA-SCIENCE-PROJECTS/BLOB/MAS-

TER/SPOTIFY-GENRE-CLASSIFIER/GET-SPOTIFY-DATA.IPYNB 

[24] Z. (2020, October 13). The 6 Assumptions of Logistic 
Regression (With Examples). Statology. Retrieved 
November 11, 1111, from  
STATOLOGY.ORG/ASSUMPTIONS-OF-LOGISTIC-REGRESSION/ 

  

https://www.youtube.com/c/vastava
https://www.youtube.com/watch?v=VTU6Jla70VY&ab_channel=vastava
https://github.com/vastava/data-science-projects/blob/master/spotify-genre-classifier/get-spotify-data.ipynb
https://www.boatbomber.com/
https://machinelearningmastery.com/hyperparameter-optimization-with-random-search-and-grid-search/
https://machinelearningmastery.com/hyperparameter-optimization-with-random-search-and-grid-search/
https://www.youtube.com/watch?v=nzNp05AyBM8
https://rpubs.com/PeterDola/SpotifyTracks
https://www.youtube.com/watch?v=cU8YH2rhN6A
https://open.spotify.com/playlist/0ubKnC9ctDVxlGbF5jICvq?si=d1a259083136490a&nd=1
https://open.spotify.com/playlist/0ubKnC9ctDVxlGbF5jICvq?si=d1a259083136490a&nd=1
https://www.youtube.com/watch?v=JDU3AzH3WKg
https://www.youtube.com/watch?v=OGpQxIkR4ao
https://pub.towardsai.net/gradient-descent-algorithm-for-machine-learning-python-tutorial-ml-9ded189ec556
https://pub.towardsai.net/gradient-descent-algorithm-for-machine-learning-python-tutorial-ml-9ded189ec556
https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html
https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html
https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html
https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html
https://scikit-learn.org/stable/modules/naive_bayes.html
https://www.statisticssolutions.com/free-resources/directory-of-statistical-analyses/assumptions-of-logistic-regression/
https://www.statisticssolutions.com/free-resources/directory-of-statistical-analyses/assumptions-of-logistic-regression/
https://www.statisticssolutions.com/free-resources/directory-of-statistical-analyses/assumptions-of-logistic-regression/
https://www.youtube.com/watch?v=vN5cNN2-HWE
https://www.youtube.com/watch?v=BfKanl1aSG0
https://www.youtube.com/watch?v=xxFYro8QuXA
https://www.youtube.com/watch?v=ARfXDSkQf1Y
https://www.youtube.com/watch?v=yIYKR4sgzI8
https://www.youtube.com/watch?v=VTU6Jla70VY
https://github.com/vastava/data-science-projects/blob/master/spotify-genre-classifier/get-spotify-data.ipynb
https://github.com/vastava/data-science-projects/blob/master/spotify-genre-classifier/get-spotify-data.ipynb
https://www.statology.org/assumptions-of-logistic-regression/


ARESTY RUTGERS UNDERGRADUATE RESEARCH JOURNAL, VOLUME I, ISSUE V 

 

 

 
 

Shifra Isaacs is a data scientist, data analyst, and technical writer with industry 

experience at Annalect, JPMorgan Chase, CrashCourse, DataLemur, and 

more. Shifra's passion for data extends beyond her professional work. She has 

contributed to the field through publications, projects, and educational con-

tent that makes data and programming more accessible. 

 

With a diverse skill set including Python, SQL, BI tools, documentation, and 

product management, Shifra demonstrates versatility and adaptability in ad-

dressing complex challenges. Her track record of delivering impactful solu-

tions, coupled with her strong analytical prowess and business acumen, estab-

lishes Shifra as a valuable asset in the field of data science. 
 


