共计 9170 个字符，预计需要花费 23 分钟才能阅读完成。
The University of Melbourne
School of Computing and Information Systems
COMP30027 Machine Learning, 2023 Semester 1
Project 1: Music genre classification with na?¨ve Bayes
Due: 7 pm, 7 April 2021
Submission: Source code (in Python) and written responses
Groups: You may choose to form a group of 1 or 2.
Groups of 2 will respond to more questions, and commensurately produce more
implementation.
Marks: The project will be marked out of 16 points (individual project) or 24 points
(group project). In either case, this project will contribute 20% of your total
mark.
Overview
A visualisation (mel spectrogram) of a
music clip from the GZTAN dataset [3].
State-of-the-art AI research is focused on developing com-
puter systems that can recognize and understand text, im-
ages, and audio in the ways that humans do. A classic
problem in audio AI is the problem of music genre clas-
sification, which is useful for applications like music rec-
ommendation systems. Given a piece of music, how do
we interpret what“type”of music it is (e.g., pop, classical,
hip-hop, or jazz)? This task is challenging for computers
because the artists, styles, and features of music within a
genre can be quite varied, and songs from different genres
may share some features.
In this project, you will implement a supervised na?¨ve
Bayes learner to classify the genre of a music clip from high-level acoustic features. You will train,
test, and evaluate your classifier on a provided dataset, and then you will have a choice of either
extending this basic model in various ways, or using it to answer some conceptual questions about
na?¨ve Bayes.
Data
The data for this assignment is drawn from the GTZAN music genre dataset [1], a dataset for music
genre classification. It consists of 1000 30-second mp3 audio clips from 10 different classes (100
samples per class). The classes are blues, classical, country, disco, hip hop, jazz, metal, pop, reggae,
and rock. For this assignment, we’ll use a processed version of the dataset from Kaggle [2], which
provides 57 high-level acoustic features [3] extracted from the music clips. You do not need the
original audio files for this assignment, though if you are interested, you can download them through
Kaggle.
Separate training and test datasets are provided. Please use the provided train/test splits for this
assignment, unless a question asks you to create your own splits. Each row in the dataset is a music
clip with the class label given in the label column.
Naive Bayes classifier [4 marks]
There are some suggestions for implementing your learner in the“Na?¨ve Bayes”and“Discrete &
Continuous Data”lectures, but ultimately, the specifics of your implementation are up to you. Your
implementation must be able to perform the following functions:
preprocess() the data by reading it from a file and converting it into a useful format for
training and testing
train() by calculating prior probabilities and likelihoods from the training data and using
these to build a naive Bayes model
predict() classes for new items in a test dataset
evaluate() the prediction performance by comparing your model’s class outputs to ground
truth labels
Your implementation should be able to handle numeric attributes and it should assume that nu-
meric attributes are Gaussian-distributed. Your model will not be expected to handle nominal at-
tributes.
Your implementation should actually compute the priors, likelihoods, and posterior probabilities
for the na?¨ve Bayes model. You may use built-in functions to read data and compute Gaussian prob-
abilities. However, you must implement the na?¨ve Bayes algorithm yourself and not simply call an
existing implementation such as GaussianNB from scikit-learn.
Task 1. Pop vs. classical music classification [8 marks]
Use the pop vs classical train.csv dataset to train your na?¨ve Bayes model and then eval-
uate it on the pop vs classical test.csv dataset. Answer questions 1-2 below in a short
write-up (no more than 250 words total).
- Compute and report the accuracy, precision, and recall of your model (treat“classical”as the
“positive”class). [3 marks] - For each of the features X below, plot the probability density functions P (X|Class = pop)
and P (X|Class = classical). If you had to classify pop vs. classical music using just one of
these three features, which feature would you use and why? Refer to your plots to support your
answer. [5 marks]
spectral centroid mean
harmony mean
tempo
Task 2. 10-way music genre classification [4 marks (individual) or 12 marks (group of
2)]
Use the gztan train.csv dataset to train your na?¨ve Bayes model and then evaluate it on the
gztan test.csv dataset. If you are working in a group of 1, answer 1 of the questions below
for 4 marks. If you are working in a group of 2, answer 3 of the questions below for 12 marks. Each
question response should be no more than 250 words and include figures and/or tables as appropriate.
When reporting the performance of a model, you should include a breakdown of the performance over
categories in addition to reporting the overall accuracy. - Compare the performance of the full model to a 0R baseline and a one-attribute baseline. The
one-attribute baseline should be the best possible na?¨ve Bayes model which uses only a prior
and a single attribute. In your write-up, explain how you implemented the 0R and one-attribute
baselines. [4 marks] - Train and test your model with a range of training set sizes by setting up your own train/test
splits. With each split, use cross-fold validation so you can report the performance on the entire
dataset (1000 items). You may use built-in functions to set up cross-validation splits. In your
write-up, evaluate how model performance changes with training set size. [4 marks] - Implement a kernel density estimate (KDE) na?¨ve Bayes model and compare its performance
to your Gaussian na?¨ve Bayes model. You may use built-in functions and automatic (“rule
of thumb”) bandwidth selectors to compute the KDE probabilities, but you should implement
the na?¨ve Bayes logic yourself. You should give the parameters of the KDE implementation
(namely, what bandwidth(s) you used and how they were chosen) in your write-up. [4 marks] - Modify your na?¨ve Bayes model to handle missing attributes in the test data. Recall from lecture
that you can handle missing attributes at test by skipping the missing attributes and computing
the posterior probability from the non-missing attributes. Randomly delete some attributes from
the provided test set to test how robust your model is to missing data. In your write-up, evaluate
how your model’s performance changes as the amount of missing data increases. [4 marks]
Implementation tips
In the training phase of your algorithm, you will need to set up data structures to hold the prior
probabilities for each class, and the likelihoods P (xi|cj) for each attribute xi in each class cj . Recall
that you will need two parameters (mean and standard deviation) to define the Gaussian distribution
for each attribute × class. A 2D array may be a convenient data structure to store these parameters.
You are allowed to use built-in functions to compute the Gaussian pdf, but these functions tend to
be slow, so you may prefer to write your own function to do this. The probability density function for
a Gaussian with mean of μ and standard deviation of σ is:
(1)
Multiplying many probabilities in the range (0, 1] can result in very low values and lead to under-
flow (numbers smaller than the computer can represent). When implementing a na?¨ve Bayes model, it
is strongly recommended to take the log() of each probability and sum them instead of multiplying.
E.g., instead of computing:
log(P (xi|cj)) (3)
Submission
Submission will be made via the LMS. Please submit your code and written report separately:
Your code submission should use the provided .ipynb notebook template. Your submission
must include comments or a README section that explain how to run your code so we can
reproduce your results.
Your written report should be uploaded separately as a .pdf, using the Turnitin submission link.
If you are working in a group, please include both group members’student id numbers on the
written report and in your code file (in the README file or a group.txt file).
Late submission
The submission mechanism will stay open for one week after the submission deadline. Late submis-
sions will be penalised at 10% per 24-hour period after the original deadline. Submissions will be
closed 7 days (168 hours) after the published assignment deadline, and no further submissions will be
accepted after this point.
Assessment - of the marks available for this assignment will be based on the implementation of the na?¨ve Bayes
classifier, specifically the four Python functions specified above. Any other functions you’ve im-
plemented will not be directly assessed, unless they are required to make these four functions work
correctly.
The questions should be answered in a written .pdf report, and will be marked as indicated above.
We will be looking for evidence that you have an implementation that allows you to explore the
problem, but also that you have thought deeply about the data and the behaviour of the relevant
classifier(s).
Because the number of questions depends on the group size, individual projects can receive a
total of 16 marks and group projects can receive a total of 24 marks. In both cases, the project will
contribute 20% of the final mark in this subject. In group projects, both members of the group will
receive the same mark.
Updates to the assignment specifications
If any changes or clarifications are made to the project specification, these will be posted on the LMS.
Academic misconduct
You are welcome — indeed encouraged — to collaborate with your peers in terms of the conceptual-
isation and framing of the problem. For example, we encourage you to discuss what the assignment
specification is asking you to do, or what you would need to implement to be able to respond to a
question.
However, sharing materials beyond your group — for example, plagiarising code or colluding in
writing responses to questions — will be considered cheating. We will invoke University’s Academic
Misconduct policy (http://academichonesty.unimelb.edu.au/policy.html) where
inappropriate levels of plagiarism or collusion are deemed to have taken place.
References
[1] George Tzanetakis and P Cook. Gtzan genre collection. Web resource, 2001. http://marsyas.
info/downloads/datasets.html
[2] Andrada Olteanu, James Wiltshire, Lauren O’Hare and Minyu Lei. GTZAN Dataset – Music
Genre Classification. Web resource, 2020. https://www.kaggle.com/datasets/andradaolteanu/
gtzan-dataset-music-genre-classification
[3] George Tzanetakis and Perry Cook. Musical genre classification of audio signals. IEEE
Transactions on Speech and Audio Processing, 10(5):293–302, 2002.