Music Data Mining

Association Mining

Paper

Description

Dataset

Mining Association Patterns between Music and Video Clips in Professional MTV (Journal: Advances in Multimedia Modeling '09) 

  [PDF]  [Bibtex]

Use a dual-wing harmonium model to learn and represent the underlying association patterns between music and video clips in professional MTV.

100 MTVs in different categories

Using a Statistical Model to Capture the Association Between Timbre and Perceived Tempo (ISMIR '08) 

  [PDF]  [Bibtex]

Use statistic model to investigate the association between timbre and tempo and use timbre information to improve the performance of tempo estimation.

1. Ballroom: consists of 698 thirtysecond audio excerpts (obtained from Ballroom Dancer)

2. Songs: contains 465 audio clips from nine genres

Emotion-based Music Recommendation By Association Discovery from Film Music (ACM Multimedia '05)

  [PDF]  [Bibtex]

Investigate the music feature extraction and modified the affinity graph for association discovery between emotions and music features.

107 film music from 20 animated films.

Sound, Music and Textual Associations on the World Wide Web (ISMIR '04)

  [PDF]  [Bibtex]

Measure the similarity between the public text visible on a web page and the linked sound files.

4500 pages were crawled from the internet to have references to sound files, producing unique links to 35,481 sound files.

Classification

Genre Classification

Paper

Description

Dataset

From Multi-Labeling to Multi-Domain-Labeling: A Novel Two-Dimensional Approach to Music Genre Classification  (ISMIR '09)

  [PDF]  [Bibtex]

Propose to break down multi-label genre annotations into single-label annotations within given time segments and musical domains.

 

Genre Classification Using Harmony Rules Induced From Automatic Chord Transcriptions (ISMIR '09)

  [PDF]  [Bibtex]

Adopt a first-order logic representation of harmony and musical genres: pieces of music are represented as lists of chords and musical genres are seen as context-free definite clause grammars using subsequences of these chord lists.

 

Tag Integrated multi-Label Music Style Classification With Hypergraph  (ISMIR '09)

  [PDF]  [Bibtex]

Propose a multi-label music style classification approach, called Hypergraph integrated Support Vector Machine (HiSVM), which can integrate both music contents and music tags for automatic music style classification.

in-house database

Bayesian Aggregation for Hierarchical Genre Classification  (ISMIR '07)

  [PDF]  [Bibtex]

Apply a Bayesian framework to combine, or aggregate, a hierarchy of multiple binary classifiers in a principled manner, and consequently improves performance over the hierarchy as a whole.

MIREX 2005 symbolic genre classification dataset

A Study on Music Genre Classification Based on Universal Acoustic Models  (ISMIR '06)

  [PDF]  [Bibtex]

Provide a framework to give finer segmentation of hidden Markov models (HMMs) through acoustic segment modeling.

Magnatunes and RWC

Musical genre classification: Is it worth pursuing and how can it be improved?  (ISMIR '06)

  [PDF]  [Bibtex]

Present a number of counterarguments that emphasize the importance of continuing research in automatic genre classification.

 

Evaluation of Feature Extractors and psycho-Acoustic Transformations for Music Genre Classification (ISMIR '05)

  [PDF]  [Bibtex]

A study on the importance of psycho-acoustic transformations for effective audio feature calculation. Evaluation on both the individual and combined feature sets is accomplished through a music genre classification task.

1. GTZAN: consists of 1000 pieces of audio equidistributed among 10 popular music genres.
2. Rhythm classification contest dataset, consists of 698 excerpts of 8 genres

An Investigation of Feature Models for Music Genre Classification Using the Support Vector Classifier (ISMIR '05)

  [PDF]  [Bibtex]

Investigate two models, the multivariate Gaussian model and the multivariate autoregressive model for modeling short time features and how these models can be integrated over a segment of short time features into a kernel such that a support vector machine can be applied.

Consists of a training set of 1098 music snippets, 100 from each genre except for latin, of each 30 seconds and a separate test set of 220 music snippets each of 30 seconds in length.

Musical Genre Classification Enhanced by Improved Source Separation Techniques  (ISMIR '05)

  [PDF]  [Bibtex]

Musical genre classification based on audio features extracted from signals which correspond to distinct musical instrument sources.

1049 music pieces from 4 genres of greek songs, namely Rebetico (396 pieces), Dimotiko (106 pieces), Laiko (414 pieces), and Entechno (133 pieces).

Factors Affecting Automatic Genre Classification: An Investigation Incorporating Non-western Musical Form  (ISMIR '05)

  [PDF]  [Bibtex]

Investigates the factors affecting automated genre classification.

Audio Compact Discs

Genre Classification Via An LZ78-Based String Kernel (ISMIR '05) 

  [PDF]  [Bibtex]

Develop the notion of normalized information distance (NID) into a kernel distance suitable for use with a Support Vector Machine classifier, and demonstrate its use for an audio genre classification task.

http://opihi.cs.uvic.ca/sound/genres/

Improving music genre classification by short time feature integration  (ICASSP '05)

  [PDF]  [Bibtex]

Investigate different methods for feature integration and late information fusion for music genre classification.

 

Music genre classification with taxonomy  (ICASSP '05)

  [PDF]  [Bibtex]

The underlying hierarchical taxonomy identifies the relationships of dependence between different genres and provides valuable sources of information for genre classification.

 

Classification of Musical Genre : A Machine Learning Approach  (ISMIR '04)

  [PDF]  [Bibtex]

Investigate the impact of machine learning algorithms in the development of automatic music classification models aiming to capture genres distinctions.

300 midi files collected from theWeb

Automatic Genre Classification Using Large high-level Musical Feature Sets (ISMIR '04)

  [PDF]  [Bibtex]

The classification is performed hierarchically using different sets of features at different levels of the hierarchy.

 

A Comparative Study on Content-Based Music Genre Classification  (SIGIR '03)

  [PDF]  [Bibtex]

Capture the local and global information of music signals simultaneously by computing histograms on
their Daubechies wavelet coefficients.

 

Features for Audio and Music Classification  (ISMIR '03)

  [PDF]  [Bibtex]

Four audio feature sets are evaluated in their ability to classify five general audio classes and seven popular music genres.

 

Automatic Musical Genre Classification Of Audio Signals (Journal: IEEE Transactions on Speech and Audio Processing '02)

  [PDF]  [Bibtex]

Propose a set of features for representing texture and instrumentation.

 

Rhythm Classification

Paper

Description

Dataset

Rhythm Classification Using Spectral Rhythm Patterns  (ISMIR '05)

  [PDF]  [Bibtex]

Study the use of spectral patterns to represent the characteristics of the rhythm of an audio signal.

Ballroom Dancer

Classification of Musical Metre with AutoCorrelation and Discriminant Functions  (ISMIR '05)

  [PDF]  [Bibtex]

Study the classification performance of the autocorrelation-based metre induction model.

The Essen collection, consisting of mainly European folk melodies, and the Digital Archive of Finnish Folk Tunes.

Artist Classification

Paper

Description

Dataset

Artist Classification with Web-Based Data (ISMIR '04)

  [PDF]  [Bibtex]

Retrieve and analyze webpages ranked by search engines to describe artists in terms of word occurrences on related pages.

 

Using Voice Segments to Improve Artist Classification of Music (AES '02)

  [PDF]  [Bibtex]

automatically-located singing segments form a more reliable basis for classification than using the entire track, suggesting that the singer’s voice is more stable across different performances, compositions, and transformations due to audio engineering techniques than the instrumental background.

269 full songs from 21 different artists (one album of 10-15 songs per artist), comprising 1057 minutes of audio.

Mood Detection and Classification

Paper

Description

Dataset

Lyric Text Mining In Music Mood Classification  (ISMIR '09)

  [PDF]  [Bibtex]

Investigate the usefulness of text features in music mood classification on 18 mood categories derived from user tags.

in-house

Multi-Label Classification of Music Into Emotions  (ISMIR '08)

  [PDF]  [Bibtex]

The automated detection of emotion in music is modeled as a multilabel classification task, where a piece of music may belong to more than one class. Four algorithms are evaluated and compared in this task.

in-house

Multimodal Music Mood Classification using Audio and Lyrics  (ICMLA '08)

  [PDF]  [Bibtex]

Present a study on music mood classification using audio and lyrics information.

in-house

Music Emotion Classification: A Fuzzy Approach  (ACM MM '06)

  [PDF]  [Bibtex]

For each music segment, the approach determines how likely the song segment belongs to an emotion class. Two fuzzy classifiers are adopted to provide the measurement of the emotion strength.

 

Detecting emotion in music  (ISMIR '03)

  [PDF]  [Bibtex]

Cast the emotion detection problem as a multi-label classification problem, where the music sounds are classified into multiple classes simultaneously.

 

Automatic Mood Detection from Acoustic Music Data  (ISMIR '03)

  [PDF]  [Bibtex]

Present a hierarchical framework to automate the task of mood detection from acoustic music data, by following some music psychological theories in western cultures.

250 pieces of music, composed mainly in the classical period and romantic period.

Instrument Classification and Recognition

Paper

Description

Dataset

Musical Instrument Recognition by Pairwise Classification Strategies (Journal: IEEE Transactions on ASLP '06)

  [PDF]  [Bibtex]

Statistical pattern recognition techniques are utilized to tackle the problem in the context of solo musical phrases.

 

Instrument Recognition in Polyphonic Music (ICASSP '05)

  [PDF]  [Bibtex]

Propose a method for the recognition of musical instruments in polyphonic music excerpted from commercial recordings.

Commercial Compact Disc (CD) recordings and RWC jazz music database

Musical Instrument Classification and Duet Analysis Employing Music Information Retrieval Techniques   (Journal: IEEE '04)

  [PDF]  [Bibtex]

The classification process is shown as a three-layer process consisting of pitch extraction, parametrization, and pattern recognition.

 

Percussion Classification in Polyphonic Audio Recordings Using Localized Sound Models  (ISMIR '04)

  [PDF]  [Bibtex]

Present a feature-based sound modeling approach that combines general, prior knowledge about the sound characteristics of percussion instrument families (general models) with on-the-fly acquired knowledge of recording-specific sounds (localized models).

Training set: 1136 instances (100 ms long): 1061 onset regions taken from 25 CD-quality polyphonic audio recordings and 75
isolated drum samples.
Testing set: seventeen 20-second excerpts taken from 17 different CD-quality audio recordings

Automatic Classification of Musical Instrument Sounds  (Journal: New Music Research '03)

  [PDF]  [Bibtex]

Present an exhaustive review of research on automatic classification of sounds from musical instruments.

McGill Master Samples collection

Towards instrument segmentation for music content description: a critical review of instrument classification techniques  (ISMIR '00)

  [PDF]  [Bibtex]

Concentrate on reviewing the different techniques that have been so far proposed for automatic classification of musical instruments.

 

Music Instrument Recognition Using Cepstral Coefficients and Temporal Features  (ICASSP '00)

  [PDF]  [Bibtex]

A system for musical instrument recognition was presented that uses a wide set of features to model the temporal and spectral charactreristics of sounds.

McGill Master Samples collection

Singer Identification

Paper

Description

Dataset

Towards Efficient Automated Singer Identification in Large Music Databases  (SIGIR '06)

  [PDF]  [Bibtex]

Use multiple low-level features extracted from both vocal and non-vocal music segments to enhance the identification process with a hybrid architecture and build profiles of individual singer characteristics based on statistical mixture models.

 

Singer Identification Based on Accompaniment Sound Reduction and Reliable Frame Selection  (ISMIR '05)

  [PDF]  [Bibtex]

Describe a method for automatic singer identification from polyphonic musical audio signals including sounds of various instruments.

RWC Music Database: Popular

Automatic singer identification  (ICME '03)

  [PDF]  [Bibtex]

Propose a system for automatic singer identification which recognizes the singer of a song by analyzing the music signal.

 

Automatic singer identification of popular music recordings via estimation and modeling of solo vocal signal  (ECSCT '03)

  [PDF]  [Bibtex]

Investigate the problem of automatic singer identification, detection and tracking in popular music recordings with one or multiple singers.

 

Singer Identification in Popular Music Recordings Using Voice Coding Features  (ISMIR '02)

  [PDF]  [Bibtex]

Attempt to automatically establish the identity of a singer using acoustic features extracted from songs in a database of popular music.

NECI Minnowmatch testbed

A singer identification technique for content-based classification of MP3 music objects  (CIKM '02)

  [PDF]  [Bibtex]

Propose an approach to automatically classify MP3 music objects according to their singers.

 

Clustering

Paper

Description

Dataset

Clustering Music Recordings by Their Keys (ISMIR '09)

  [PDF]  [Bibtex]

Based on chroma-based features extracted from acoustic signals, an inter-recording distance metric which characterizes diversity of pitch distribution together with harmonic center of music pieces, is introduced to measure dissimilarities among musical features.

91 pop songs, including 21 out of 24 keys.

Detection of Pitched/Unpitched Sound Using Pitch Strength Clustering (ISMIR '08)

  [PDF]  [Bibtex]

Track the pitch strength trace of the signal, determining clusters of pitch and unpitched sound. The criterion used to determine the clusters is the local maximization of the distance between the centroids.

1. Paul Bagshaw’s Database
2. Keele Pitch Database

Music Clustering with Constraints (ISMIR '07)

  [PDF]  [Bibtex]

Propose an approach based on the generalized constraint clustering algorithm by incorporating the constraints for grouping music by “similar” artists.

300 songs from 53 albums of a total of 41 artists from All Music Guide artist pages (http://www.allmusic.com)

Polyphonic instrument Recognition Using Spectral Clustering (ISMIR '07)

  [PDF]  [Bibtex]

Present a framework for the sound source separation and timbre classification of polyphonic, multi-instrumental music signals.

RWC

Integrating Features from Different Sources for Music Information Retrieval (ICDM '06)

  [PDF]  [Bibtex]

Propose a clustering algorithm that integrates features from both lyrics and acoustic data sources to perform bimodal learning.

570 songs from 53 albums of 41 artists using artist similarity provided by All Music Guide.

Harmonic-Temporal-Structured Clustering Via Deterministic Annealing EM Algorithm for Audio Feature Extraction (ISMIR '05)

  [PDF]  [Bibtex]

Decompose the energy patterns diffused in timefrequency space, i.e., a time series of power spectrum, into distinct clusters such that each of them is originated from a single sound stream.

RWC

Algorithmic Clustering of Music Based on String Compression (Journal: Computer Music '04)

  [PDF]  [Bibtex]

Apply compression-based method to the classification of pieces of music.

 

Perceptual Segment Clustering for Music Description and Time-Axis Redundancy Cancellation (ISMIR '04)

  [PDF]  [Bibtex]

Propose a perceptually grounded model for describing music as a sequence of labeled sound segments, for reducing data complexity, and for compressing audio.

 

Clustering Symbolic Music Using Paradigmatic and Surface Level Analysis (ISMIR '04)

  [PDF]  [Bibtex]

Propose a novel automatic analysis method based on paradigmatic and surface level similarity of music represented in symbolic form.

145 classical pieces randomly from the Mutopia database.

Blind Clustering of Popular Music Recordings Based on Singer Voice Characteristics (Journal: Computer Music '04)

  [PDF]  [Bibtex]

Examine the feasibility of unsupervised clustering of music data based on their singer. It has been shown that the characteristics of a singer's voice can be extracted from music via vocal segment detection followed by solo vocal signal modeling.

416 tracks from Mandarin pop music CDs

Learning

Paper

Description

Dataset

Sequence Mining

Paper

Description

Dataset

Cochonut: Recognizing Complex Chords from MIDI Guitar Sequences  (ISMIR '08)

  [PDF]  [Bibtex]

Use contextual harmonic information to solve ambiguous cases, integrated with other techniques, such as decision theory, optimization, pattern matching and rule-based recognition.

Symbolic MIDI guitar data, called COCHONUT (Complex Chords Nutting).

Sequence Representation of Music Structure Using High-order Similarity Matrix and Maximum-Likelihood Approach  (ISMIR '07)

  [PDF]  [Bibtex]

Present a novel method for the automatic estimation of the structure of music tracks using a sequence representation.

 

Supervised and Unsupervised Sequence Modelling for Drum Transcription  (ISMIR '07)

  [PDF]  [Bibtex]

Discuss two post-processings for drum transcription systems, which aim to model typical properties of drum sequences.

 

Audio-based Cover Song Retrieval Using Approximate Chord Sequences: Testing Shifts, Gaps, Swaps and Beats  (ISMIR '07)

  [PDF]  [Bibtex]

Present a variation on the theme of using string alignment for MIR in the context of cover song identification in audio collections.

 

Time-warped Longest Common Subsequence Algorithm for Music Retrieval  (ISMIR '04)

  [PDF]  [Bibtex]

Present the Time-Warped Longest Common Subsequence algorithm (T-WLCS), which deals with singing errors involving rhythmic distortions.

Digital Tradition collection

An Auditory Model Based Transcriber of Singing Sequences  (ISMIR '02)

  [PDF]  [Bibtex]

A new system for the automatic transcription of singing sequences into a sequence of pitch and duration pairs is presented.

 

Similarity Search

Paper

Description

Dataset

A Filter-and-Refine Indexing Method for Fast Similarity Search in Millions of Music Tracks  (ISMIR '09)

  [PDF]  [Bibtex]

Rescale the divergence and uses a modified FastMap implementation to accelerate nearest-neighbor queries.

2.5 million tracks consist of 30 second snippets of songs gathered by crawling an online music store

Learning a Metric for Music Similarity  (ISMIR '09)

  [PDF]  [Bibtex]

Learn embeddings so that the pairwise Euclidean distance between two songs reflects semantic dissimilarity.

Top 1000 most-popular mp3 blogs on http://hypem.com/toplist

High-level Audio Features: Distributed Extraction and Similarity Search (ISMIR '08)

  [PDF]  [Bibtex]

Perform the feature extraction in a two-step process that allows distributed computations while respecting copyright laws.

41,446 songs in MP3 format (segments are non-overlapping and have a 5 seconds length).

Content-based Music Similarity Search and Emotion Detection  (ICASSP '06)

  [PDF]  [Bibtex]

Investigate the use of acoustic based features for music information retrieval. For similarity search, the distance between two sound files is defined to be the Euclidean distance of their normalized representations.

250 Jazz vocal sounds files, covering 18 vocalists and 35 albums

Improvements of Audio-Based Music Similarity and Genre Classification  (ISMIR '05)

  [PDF]  [Bibtex]

Present an approach to improve audio-based music similarity and genre classification.

 

A Large-Scale Evaluation of Acoustic and Subjective Music Similarity Measures  (Journal: Computer Music '04)

  [PDF]  [Bibtex]

Examine both acoustic and subjective approaches for calculating similarity between artists, comparing their performance on a common database of 400 popular artists.

 

Music Similarity Measures: What’s the Use?  (ISMIR '02)

  [PDF]  [Bibtex]

Introduce a timbral similarity measures for comparing music titles based on a Gaussian model of cepstrum coefficients.

 

Audio retrieval by rhythmic similarity  (ISMIR '02)

  [PDF]  [Bibtex]

Present ways to quantitatively measure the rhythmic similarity between two or more works of music. This allows rhythmically similar works to be retrieved from a large collection.

 

Using psycho-acoustic models and self-organizing maps to create a hierarchical structuring of music by sound similarity  (ISMIR '02)

  [PDF]  [Bibtex]

Propose an approach to automatically create a hierarchical organization of music archives following their perceived sound similarity.

 

A content-based music similarity function  (Journal: Cambridge Research Labs-Tech Report, '01)

  [PDF]  [Bibtex]

Present a method to compare songs based solely on their audio content.

 

Music Summarization

Paper

Description

Dataset

Automatic Music Classification and Summarization  (IEEE Transaction on SAP '05)

  [PDF]  [Bibtex]

Propose effective algorithms to automatically classify and summarize music content.

 

Summarizing Popular Music via Structural Similarity Analysis 

  [PDF]  [Bibtex]

Present a framework for summarizing digital media based on structural analysis.

 

Automatic Music Summarization via Similarity Analysis  (IRCAM '02)

  [PDF]  [Bibtex]

Present methods for automatically producing summary excerpts
or thumbnails of music.

 

Toward automatic music audio summary generation from signal analysis  (ISMIR '02)

  [PDF]  [Bibtex]

Consider the audio signal as a succession of “states” (at various scales) corresponding to the structure (at various scales) of a piece of music.

 

Music Summarization Using Key Phrases  (ICASSP '00)

  [PDF]  [Bibtex]

Investigate two approaches to find key-phrases based on clsutering segments and learning HMMs.

 

Music Visualization

Paper

Description

Dataset

A Content Dependent Visualization System for Symbolic Representation of Piano Stream  (Journal: Springer '10)

  [PDF]  [Bibtex]

Provide an overview on the advances of music information retrieval in symbolic representation of music.

 

Emotion-Based Music Visualization Using Photos  (Journal: Advances in Multimedia Modeling '08)

  [PDF]  [Bibtex]

Propose an emotion-based music player which synchronizes visualization (photos) with music based on the emotions evoked by auditory stimulus of music and visual content of visualization.

 

What You See is What You Get: on Visualizing Music  (ISMIR '05)

  [PDF]  [Bibtex]

Examine a number of visualization techniques developed for music, focusing especially on those developed for music analysis by specialists in the field, but also looking at some less successful approaches.

 

Specmurt Anasylis: A Piano-Roll-Visualization of Polyphonic Music Signals by Deconvolution of Log-Frequency Spectrum

  [PDF]  [Bibtex]

Propose a new signal processing technique, “specmurt anasylis,” that provides piano-rolllike visual display of multi-tone signals.

 

Visualizing and Exploring Personal Music Libraries  (ISMIR '04)

  [PDF]  [Bibtex]

Propose new graphical visualizations and their associated features to allow users to better organize their personal music libraries and therefore also to ease selection later on.

 

Audio Information Browsing With The Sonic Browser  (Journal: IEEE Computer Society '03)

  [PDF]  [Bibtex]

Propose an application for browsing sound collections on personal computers.

 

The SOM-enhanced JukeBox: Organization and Visualization of Music Collections Based on Perceptual Models  (Journal: New Music Research '03)

  [PDF]  [Bibtex]

Propose an approach to automatically create an organization of music archives following their perceived sound similarity.

 

Islands of Music Analysis, Organization, and Visualization of Music Archives 

  [PDF]  [Bibtex]

Using a neural network algorithm, namely the self-organizing map, the music collection is organized and using a novel visualization technique the map of islands is created.

 

Content-based Organization and Visualization of Music Archives  (ACM MM '02)

  [PDF]  [Bibtex]

Present a system which facilitates exploration of music libraries without requiring manual genre classification.

 

Marsyas3D: A Prototype Audio Browser-editor Using a Large Scale Immersive Visual and Audio Display  (ICAD '01)

  [PDF]  [Bibtex]

Present a prototype audio browser and editor for large audio collections.

 

Visualizing Music and Audio Using Self-similarity  (ACM MM '99)

  [PDF]  [Bibtex]

Present a novel approach to visualizing the time structure of music and audio.

 

Music Indexing

Paper

Description

Dataset

InMAF: Indexing Music Databases via Multiple Acoustic Features  (SIGMOD '06)

  [PDF]  [Bibtex]

Present a novel approach for generating small but comprehensive music descriptors to facilitate efficient content music management(accessing and retrieval, in particular).

 

An Integrated Visual Approach for Music Indexing and Dynamic Playlist Composition  (SPIE '06)

  [PDF]  [Bibtex]

Present an innovative integrated visual approach for indexing music and for automatically composing personalized playlists for radios or chain stores.

 

Content-based Music Indexing and Organization  (SIGIR '02)

  [PDF]  [Bibtex]

Present a system that automatically organizes a music collection according to the perceived sound similarity resembling genres or styles of music.